fix(logstream): start a cursorless watch at the tip, not the start of the log

windows:grok:mempalace flagged the first-run replay on PR #2315. Measured on
the real shared brain: a fresh `logstream watch --agent mac-claude` with no
cursor woke holding 41 events, the oldest 49 days old. Nothing in the payload
marks them stale, so an agent reads July's task.requests as new work — the
opposite of the problem this command exists to solve.

`latest_event_id()` already documents the right behaviour for the SSE
live-tail: capture the tip at connect time "so they receive only post-connect
events". The CLI watcher not doing the same made one product with two
first-run semantics.

- default: a watch with no --since-event-id and no stored cursor starts at
  the tip. Backlog belongs to the inbox sweep (`logstream list`), which can
  page it deliberately; a watcher is for what arrives from now on.
- never silent: it prints what it skipped and how to get it, on stderr, in
  --json mode too, so the note cannot corrupt a parsed payload.
- --from-start opts back into the replay.

Six existing tests turned out to depend on the replay, which is the evidence
this mattered. Their intent is filtering, not first-run behaviour, so the
shared args helper opts them into --from-start and three new tests cover the
tip default explicitly — including that an explicit --since-event-id or a
state-file cursor still wins over it.

Note for review: test_agent_shorthand_does_not_wake_on_your_own_broadcast
asserts exit 2, which the tip default would satisfy for the wrong reason.
It opts into --from-start so it keeps testing the exclusion.

4395 passed, 31 skipped.

Author: igorls

integrations/shared/coordination-protocol.md

@@ -126,6 +126,10 @@ mempalace logstream watch \
   `logstream wait`.
 - **`--follow`** keeps going after the first match instead of exiting — use
   it for daemons; leave it off for harnesses that wake on process exit.
+- **A first watch starts at the tip**, matching the SSE live-tail, and says so
+  on stderr. Replaying a long fleet log would wake you holding weeks of
+  history with nothing marking it stale. Backlog is the inbox sweep's job;
+  pass `--from-start` if you really do want the replay.
 
 Notes that save round trips:
 

mempalace/cli.py

@@ -1567,9 +1567,25 @@ def _logstream_watch(ls, args, as_json):
         "correlation_ids": normalize_watch_values(args.correlation_id),
     }
     cursor = args.since_event_id or read_watch_cursor(args.state_file)
+    skipped_from = None
+    if cursor is None and not args.from_start:
+        # Start at the tip, like the SSE live-tail does at connect time.
+        # Starting from the beginning of a long fleet log means a fresh
+        # watcher wakes holding weeks of history and cannot tell it is
+        # stale — measured 41 events, the oldest 49 days old, on a real
+        # shared brain. Backlog is the inbox sweep's job; a watcher is for
+        # what arrives from now on. Never silent: say what was skipped.
+        cursor = ls.latest_event_id()
+        skipped_from = cursor
     if not as_json:
         where = args.agent or ", ".join(sorted(spec["to_agents"] or [])) or "everything"
         print(f"Watching {where} from {cursor or 'now'}; Ctrl-C to stop.", file=sys.stderr)
+    if skipped_from:
+        print(
+            f"Starting at the tip ({skipped_from}); earlier events are not replayed. "
+            "Use --from-start to replay them, or sweep with `mempalace logstream list`.",
+            file=sys.stderr,
+        )
 
     idle_s = args.idle_exit_ms / 1000.0 if args.idle_exit_ms and args.idle_exit_ms > 0 else None
     deadline = time.monotonic() + idle_s if idle_s else None
@@ -3191,6 +3207,14 @@ def _add_logstream_filters(p):
         default=None,
         help="Persist the cursor here so a restart resumes exactly where it stopped",
     )
+    p_ls_watch.add_argument(
+        "--from-start",
+        action="store_true",
+        help=(
+            "Replay the log from the beginning when there is no cursor "
+            "(default: start at the tip, like the SSE live-tail)"
+        ),
+    )
     p_ls_watch.add_argument(
         "--follow",
         action="store_true",

tests/test_cli_logstream.py

@@ -373,6 +373,10 @@ def _watch_args(palace, **overrides):
         correlation_id=None,
         since_event_id=None,
         state_file=None,
+        # These cases seed events and then watch for them, so they opt into
+        # the replay. The tip default is exercised explicitly by the
+        # first-run tests below.
+        from_start=True,
         follow=False,
         idle_exit_ms=400,
         poll_timeout_ms=60,
@@ -528,3 +532,38 @@ def boom(*_a, **_k):
         with pytest.raises(OSError, match="broken pipe"):
             cmd_logstream(_watch_args(palace_path, agent="mac-claude", state_file=state))
         assert not (tmp_path / "watch.json").exists()
+
+    def test_fresh_watch_starts_at_the_tip_not_the_beginning(self, palace_path, capsys):
+        """A first watch must not replay the whole log.
+
+        Measured on a real shared brain, a cursorless watch woke holding 41
+        events, the oldest 49 days old — and nothing in the payload tells the
+        agent they are stale, so week-old task.requests read as new work.
+        The SSE live-tail already starts at the tip; the watcher now matches.
+        Backlog belongs to the inbox sweep.
+        """
+        cmd_logstream(_append_args(palace_path, to_agent="mac-claude", from_agent="windows-grok"))
+        capsys.readouterr()
+
+        with pytest.raises(SystemExit) as exc:
+            cmd_logstream(_watch_args(palace_path, agent="mac-claude", from_start=False))
+        assert exc.value.code == 2, "fresh watch replayed a pre-existing event"
+
+    def test_from_start_opts_back_into_the_replay(self, palace_path, capsys):
+        cmd_logstream(_append_args(palace_path, to_agent="mac-claude", from_agent="windows-grok"))
+        capsys.readouterr()
+
+        cmd_logstream(_watch_args(palace_path, agent="mac-claude", from_start=True))
+        assert _watch_payload(capsys)["count"] == 1
+
+    def test_tip_default_does_not_override_an_explicit_cursor(self, palace_path, capsys):
+        """--since-event-id and a state file must still win over the tip."""
+        cmd_logstream(_append_args(palace_path, to_agent="mac-claude", from_agent="windows-grok"))
+        first_id = json.loads(capsys.readouterr().out)["id"]
+        cmd_logstream(_append_args(palace_path, to_agent="mac-claude", from_agent="windows-grok"))
+        capsys.readouterr()
+
+        cmd_logstream(
+            _watch_args(palace_path, agent="mac-claude", since_event_id=first_id, from_start=False)
+        )
+        assert _watch_payload(capsys)["count"] == 1

website/concepts/agent-logstream.md

@@ -174,7 +174,10 @@ wakes itself on every status it posts. Repeating a filter means "or", which is
 how you narrow to the events that genuinely require you and stop being woken by
 routine traffic. Exit is `0` on a match and `2` on `--idle-exit-ms`, matching
 `wait`'s timeout convention; `--follow` keeps the process alive past the first
-match for daemons.
+match for daemons. A cursorless first run starts at the tip, like the SSE
+live-tail, and says so on stderr — replaying a long fleet log would wake a new
+watcher holding weeks of history it cannot tell is stale. `--from-start` opts
+into the replay.
 
 `mempalace_event_wait` backs off internally (0.25s → 1s), so a tight retry
 loop around it buys nothing. Filter server-side — `to_agent`, `type`,

website/reference/cli.md

@@ -211,7 +211,7 @@ mempalace logstream watch --agent mac --type task.request --type patch.ready \
 | `append` | Append an immutable event (`--type`, `--stream`, `--room`, `--from-agent` required; `--body`/`--body-file`, `--artifact-id` repeatable) |
 | `list` | List events, oldest first (all routing fields as filters, `--since-event-id`, `--limit`) |
 | `wait` | Long-poll until a match or timeout (`--timeout-ms`, max 300000; exits `2` on timeout) |
-| `watch` | Background watcher: re-arms past the `wait` cap, carries the cursor, and exits `0` on a match / `2` on `--idle-exit-ms`. `--agent ID` is shorthand for `--to-agent ID --exclude-from-agent ID` so your own `*` broadcasts never wake you. Filters repeat to mean "or"; `--state-file` resumes exactly; `--follow` stays alive past the first match |
+| `watch` | Background watcher: re-arms past the `wait` cap, carries the cursor, and exits `0` on a match / `2` on `--idle-exit-ms`. `--agent ID` is shorthand for `--to-agent ID --exclude-from-agent ID` so your own `*` broadcasts never wake you. Filters repeat to mean "or"; `--state-file` resumes exactly; `--follow` stays alive past the first match; a cursorless first run starts at the tip (`--from-start` to replay) |
 | `ack` | Append an `event.ack` for an event (`--from-agent` required, `--status`, `--body`) |
 | `sync` | Pull missing events/artifacts from peer replicas (`--peer URL --token T`, or all peers in `peers.json`) |