[codex] Bound historical log replay memory

[domjancik]

What changed

This bounds historical conversation replay and removes two follow-on server allocation paths hit while reproducing the reported workspace/session crash.

• Stream raw historical JSONL logs from disk instead of reading the full file into a string and Vec.
• Cap historical JSONL record reads so a single oversized saved log line is drained and skipped instead of allocated without bound.
• Give temporary historical normalization stores a bounded retention window.
• Estimate JSON patch size structurally instead of serializing patches just to decide whether history can retain them.
• Skip oversized historical normalized patches before websocket serialization.
• Keep live broadcasting behavior intact while preventing oversized messages from being retained in MsgStore history.
• Stop the workspace chat from eagerly loading every older process; older history now loads when the user scrolls near the top.
• Add VK_DISABLE_PR_MONITOR=1 for local deployments so repro runs can disable background PR polling/fetching when GitHub/network is unavailable.
• Add stats-only Git diff loading for workspace summaries, and bound summary diff fan-out to four workspaces at a time.
• Serve embedded frontend assets without frontend-wide response compression, avoid copying borrowed embedded bytes, and block source-map serving by default unless VK_SERVE_FRONTEND_SOURCE_MAPS=1 is set.
• Add VK_DISABLE_AUTO_OPEN_BROWSER=1 and VK_TRACE_FRONTEND_ASSETS=1 diagnostic gates for local server repro runs.

Root cause

The original latest-session crash was caused by historical log replay loading large saved process logs through multiple full server-side copies before streaming them to the browser.

While reproducing the exact workspace/session again, two additional allocation paths showed up:

• /api/workspaces/summaries is polled by the frontend and previously computed diff counts by materializing full Diff objects, including file contents, for every workspace in parallel.
• Firefox/devtools requested the embedded production source map assets/index-BKQdPAfe.js.map. That file is about 24 MB in packages/local-web/dist, and serving it through the embedded frontend route plus compression caused the server to attempt another ~21 MB allocation and crash.

Validation

• pnpm run format
• pnpm --filter @vibe/web-core run check
• cargo test -p utils bounded_history_keeps_recent_messages_under_limit
• cargo test -p utils oversized_message_is_broadcast_but_not_retained
• cargo test -p git parse_numstat_counts_text_and_binary_files
• cargo check -p services -p utils
• cargo check -p local-deployment -p services -p utils with corrected bindgen args
• cargo check -p git -p services -p server -p local-deployment with corrected bindgen args
• cargo check -p server after the frontend asset serving changes
• Ran the release server on http://localhost:13334 with VK_SHARED_API_BASE=http://localhost:13000, VK_DISABLE_PR_MONITOR=1, and VK_DISABLE_AUTO_OPEN_BROWSER=1.
• Verified GET /api/info, the target page /workspaces/76d071e2-15c5-46bc-ab8b-24f1e58bc562?session=3cd570c0-d9f6-421b-b9b1-e095bce3bfa9, and active/archived /api/workspaces/summaries all return 200.
• Replayed normalized-log websockets for the target session processes without server crash.
• Verified assets/index-BKQdPAfe.js.map returns 404 by default and logs Frontend source map request blocked.
• With the problematic workspace/session open, server memory flattened around 102 MB RSS / 68 MB private memory instead of continuing to climb to crash.

Notes:

• On this Windows host, build-time bindgen requires LIBCLANG_PATH=C:\tmp\llvm-minimal\bin and BINDGEN_EXTRA_CLANG_ARGS=-isystemC:/Users/magne/utils/Tooll-v3.9.3/lib/clang/14.0.0/include. The no-space -isystem... form matters; -isystem C:/... failed to locate stdarg.h in this build.
• A full cargo test -p utils also hits an unrelated existing Windows path assertion in path::tests::test_make_path_relative; the new MsgStore tests pass individually.