Skip to content

Commit 4c04992

Browse files
RonTuretzkyclaude
andcommitted
every word the room hears is kept — the transcript archive replaces the rolling window
Operator: "remove the cap thing and rework it so that all transcripts are saved by default and persistent between vibe code room runs." Asking for today's transcript is what exposed it: the store held EXACTLY 400 lines — the cap — covering 17:52 to 21:07. Everything earlier was gone, evicted one line per utterance, silently. Measuring the live file twice 35 minutes apart caught it mid-act: the oldest line in the first read was absent from the second. THREE THINGS STOOD IN THE WAY, only one of them the cap. • TRANSCRIPT_STORE_CAP = 400, evicting on every append. • TRANSCRIPT_RESTORE_WINDOW_MS = 45 min, which discarded the file wholesale when the newest line was older. This room's own archive contains a 155.9-minute gap starting 18:11 — dinner. Worse, restore() ASSIGNED #lines, so a declined restore left the array empty and the next word flushed a one-line file over the whole evening. Tonight's 363 pre-dinner lines survived by luck: the room happened not to restart during dinner. • The store needed BOTH self-mode AND an env marker, which is what made it off by default. That gate exists for a reason (6a1d228: test runtimes were writing the live store), so it is REPLACED, not deleted. THE ARCHIVE is append-only JSONL, one segment per LOCAL day, unbounded and permanent: builds/transcripts/YYYY-MM-DD.jsonl. Nothing is ever evicted; the only thing that can shrink it is the operator with rm. Append costs the new bytes — measured flat at 85-88 bytes per call over 2,000 appends, with zero whole-file rewrites, where the old design re-serialized everything every 750ms. LOCAL day, not UTC, because this room's evening straddles UTC midnight: a UTC "today" asked at 21:07 would have answered a 3-hour conversation with its last 19 lines. RESTORE IS A SEPARATE CONCERN, deliberately. Saved forever is not replayed forever: the last ~60 lines within 6 hours come back, because the panel shows 40 and the research loop keeps 40 turns while the rig's two mics make each utterance arrive roughly twice. So a 15-second self-reload resumes exactly as before, a restart after dinner now resumes too (the case that used to fail), and next morning starts clean and SAYS so, naming `bun run transcript yesterday` rather than looking broken. Restore performs no writes at all, which kills the destroy-on-decline bug structurally. DEFAULT ON, at the boot entry rather than in the runtime — so `bun run start`, run-room.sh and the supervisor all archive, while a runtime built the way tests build one gets nothing. Acceptance test, verified rather than trusted: the operator's real archive is byte-identical (sha256 1207a8bb…82d9f) after 1,414 tests, three scratch server boots and the CLI read-backs. room-harness points its spawned server at its own tmp dir, closing the e2e path too. READ IT BACK: `bun run transcript` (today), `... yesterday`, `... 2026-08-24`, or GET /api/transcript/{today|yesterday|YYYY-MM-DD}[?format=text] and /api/transcript/days. The CLI works when the room is DOWN, which is exactly when last night's conversation is wanted. MIGRATION recovered MORE than was live: 638 lines now, against the 400 the cap had left, folded from the rolling file plus rescue copies — every source line present, no duplicates, original atMs preserved, span 17:53:22 to 21:51:39. ONE BUG FOUND IN REVIEW AND FIXED HERE: #separatorFor marked a segment healed BEFORE the write it rode on had landed, so a first flush that failed (full disk, permissions) left the day recorded as healed and the retry emitted no leading newline — gluing the next utterance onto the stump and losing a second line. The heal is now recorded only after the append succeeds, so "one corrupt line costs exactly one line" is true on the retry path too. KNOWN AND DELIBERATE: #pending is bounded at 5,000 lines. That is not a cap on the archive — it is the safety valve for a wedged disk, it only engages when writes are already failing, and it says so loudly ("dropping the N oldest buffered line(s) — check the disk") rather than dropping anything quietly. tsc clean, 1414 tests, build green. (repo-clone's real `git clone` times out against a 5s budget under parallel load and passes 11/11 in isolation — the known flake, and neither repo-clone file is in this diff.) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 parent e3ae688 commit 4c04992

14 files changed

Lines changed: 1621 additions & 126 deletions

package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,6 +10,7 @@
1010
"build": "vite build",
1111
"preview": "vite preview --host 127.0.0.1",
1212
"start": "bun src/server/index.ts",
13+
"transcript": "bun scripts/transcript.ts",
1314
"typecheck": "tsc --noEmit",
1415
"test": "bun test",
1516
"test:e2e": "playwright test",

scripts/self-supervisor.sh

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,9 +26,10 @@ SERVER_CMD="${VIBERSYN_SELF_SERVER_CMD:-bun src/server/index.ts}"
2626
BUILD_CMD="${VIBERSYN_SELF_BUILD_CMD:-bun run build}"
2727

2828
export VIBERSYN_SELF_MODE=1
29-
# The conversation's disk shadow (transcript survives the exit-87 reload).
30-
# Only the supervisor sets this — test runtimes must never touch it.
31-
export VIBERSYN_TRANSCRIPT_STORE="builds/session-transcript.json"
29+
# NOTE: the transcript archive is no longer exported here. It is ON BY DEFAULT
30+
# at the boot entry (src/server/index.ts -> builds/transcripts/YYYY-MM-DD.jsonl),
31+
# so every launch keeps a permanent record, not just a supervised one. Set
32+
# VIBERSYN_TRANSCRIPT_ARCHIVE to relocate it, or to "off" to keep no record.
3233

3334
# Deliberate-stop marker: `touch /tmp/vibersyn-stop` (or Ctrl-C, which kills
3435
# THIS script) ends the loop. A bare SIGTERM to the SERVER alone does not —

scripts/transcript.ts

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
#!/usr/bin/env bun
2+
//
3+
// READ THE ROOM'S TRANSCRIPT ARCHIVE.
4+
//
5+
// The operator asked for "today's transcript" and the answer was a hand-written
6+
// python pass over a rolling 400-line JSON file that had already evicted most
7+
// of the evening. This reads the day-segmented archive directly — NO SERVER —
8+
// which matters because the moment you want last night's conversation is
9+
// usually the moment the room is down.
10+
//
11+
// bun scripts/transcript.ts today
12+
// bun scripts/transcript.ts yesterday
13+
// bun scripts/transcript.ts 2026-08-24
14+
// bun scripts/transcript.ts --days list the days the archive holds
15+
// bun scripts/transcript.ts today --json raw JSONL lines as a JSON array
16+
// bun scripts/transcript.ts today --grep birdhouse
17+
// bun scripts/transcript.ts --dir <path> read a different archive
18+
// bun scripts/transcript.ts --import <file> fold a legacy/rescue snapshot in
19+
//
20+
import { resolve } from "node:path";
21+
import {
22+
TRANSCRIPT_ARCHIVE_DEFAULT_DIR,
23+
listDays,
24+
localDayKey,
25+
parseLegacyBody,
26+
readDay,
27+
renderTranscriptText,
28+
resolveDayKey,
29+
resolveTranscriptArchiveDir,
30+
} from "../src/server/transcript-archive";
31+
import { TranscriptStore } from "../src/server/transcript-store";
32+
33+
interface Args {
34+
day: string;
35+
dir: string;
36+
json: boolean;
37+
days: boolean;
38+
grep: string | null;
39+
importPath: string | null;
40+
}
41+
42+
function parseArgs(argv: readonly string[]): Args | { error: string } {
43+
const args: Args = {
44+
day: "today",
45+
// The same default the boot entry uses, so the CLI and the room agree
46+
// without either being told where the archive is.
47+
dir: resolveTranscriptArchiveDir(process.env) ?? resolve(process.cwd(), TRANSCRIPT_ARCHIVE_DEFAULT_DIR),
48+
json: false,
49+
days: false,
50+
grep: null,
51+
importPath: null,
52+
};
53+
let sawDay = false;
54+
for (let index = 0; index < argv.length; index += 1) {
55+
const arg = argv[index] ?? "";
56+
if (arg === "--json") {
57+
args.json = true;
58+
} else if (arg === "--days") {
59+
args.days = true;
60+
} else if (arg === "--dir") {
61+
const value = argv[index + 1];
62+
if (value === undefined) {
63+
return { error: "--dir needs a directory" };
64+
}
65+
args.dir = value;
66+
index += 1;
67+
} else if (arg === "--grep") {
68+
const value = argv[index + 1];
69+
if (value === undefined) {
70+
return { error: "--grep needs a pattern" };
71+
}
72+
args.grep = value;
73+
index += 1;
74+
} else if (arg === "--import") {
75+
const value = argv[index + 1];
76+
if (value === undefined) {
77+
return { error: "--import needs a file" };
78+
}
79+
args.importPath = value;
80+
index += 1;
81+
} else if (arg === "--help" || arg === "-h") {
82+
return { error: "help" };
83+
} else if (arg.startsWith("-")) {
84+
return { error: `unknown flag ${arg}` };
85+
} else if (!sawDay) {
86+
args.day = arg;
87+
sawDay = true;
88+
} else {
89+
return { error: `unexpected argument ${arg}` };
90+
}
91+
}
92+
return args;
93+
}
94+
95+
const USAGE = `read the room's transcript archive
96+
97+
bun scripts/transcript.ts [today|yesterday|YYYY-MM-DD] [--json] [--grep <pattern>]
98+
bun scripts/transcript.ts --days
99+
bun scripts/transcript.ts --import <legacy-or-rescue.json>
100+
bun scripts/transcript.ts --dir <archive-directory>`;
101+
102+
const parsed = parseArgs(process.argv.slice(2));
103+
if ("error" in parsed) {
104+
if (parsed.error !== "help") {
105+
console.error(`transcript: ${parsed.error}\n`);
106+
}
107+
console.error(USAGE);
108+
process.exit(parsed.error === "help" ? 0 : 2);
109+
}
110+
111+
const args = parsed;
112+
113+
// --import folds a snapshot (the pre-archive builds/session-transcript.json, or
114+
// a rescue copy of it) through the SAME de-duping merge the boot migration
115+
// uses, so importing the same file twice — or two overlapping snapshots — is
116+
// safe and recovers the union. Explicit rather than glob-magic on purpose.
117+
if (args.importPath !== null) {
118+
const path = resolve(args.importPath);
119+
let body: string;
120+
try {
121+
body = await Bun.file(path).text();
122+
} catch (error) {
123+
console.error(`transcript: cannot read ${path} (${error instanceof Error ? error.message : String(error)})`);
124+
process.exit(1);
125+
}
126+
let lines;
127+
try {
128+
lines = parseLegacyBody(body);
129+
} catch (error) {
130+
console.error(`transcript: ${path} is not a transcript snapshot (${error instanceof Error ? error.message : String(error)})`);
131+
process.exit(1);
132+
}
133+
if (lines.length === 0) {
134+
console.error(`transcript: ${path} holds no readable transcript lines — nothing imported.`);
135+
process.exit(1);
136+
}
137+
// legacyPath: null — an --import must fold exactly the file it was given and
138+
// must not also sweep up whatever legacy file happens to sit near the archive.
139+
const store = new TranscriptStore({
140+
dir: args.dir,
141+
legacyPath: null,
142+
onNote: (note) => (note.level === "warn" ? console.error(note.message) : console.log(note.message)),
143+
});
144+
const before = new Map(listDays(args.dir).map((day) => [day, readDay(args.dir, day).lines.length]));
145+
const touched = store.importLines(lines);
146+
if (touched === null) {
147+
console.error(`transcript: import FAILED — ${path} was left untouched.`);
148+
process.exit(1);
149+
}
150+
for (const day of touched) {
151+
const after = readDay(args.dir, day).lines.length;
152+
const had = before.get(day) ?? 0;
153+
console.log(`${day}: ${had} -> ${after} lines (+${after - had} new, ${lines.length} offered)`);
154+
}
155+
console.log(`imported ${lines.length} line(s) from ${path} into ${args.dir}`);
156+
process.exit(0);
157+
}
158+
159+
if (args.days) {
160+
const days = listDays(args.dir);
161+
if (days.length === 0) {
162+
console.error(`transcript: no archive at ${args.dir} yet — the room writes one as soon as somebody speaks.`);
163+
process.exit(1);
164+
}
165+
for (const day of days) {
166+
const segment = readDay(args.dir, day);
167+
const suffix = segment.skipped > 0 ? ` (${segment.skipped} unreadable)` : "";
168+
console.log(`${day} ${String(segment.lines.length).padStart(6)} lines${suffix}`);
169+
}
170+
process.exit(0);
171+
}
172+
173+
const day = resolveDayKey(args.day, Date.now());
174+
if (day === null) {
175+
console.error(`transcript: "${args.day}" is not a day — use YYYY-MM-DD, today, or yesterday.`);
176+
process.exit(2);
177+
}
178+
179+
const segment = readDay(args.dir, day);
180+
if (!segment.exists) {
181+
// Never a bare empty output: an empty transcript and a missing one are
182+
// different facts, and the operator must be able to tell them apart.
183+
const days = listDays(args.dir);
184+
console.error(
185+
`transcript: no transcript for ${day} in ${args.dir}.` +
186+
(days.length > 0 ? ` Days on hand: ${days.join(", ")}` : " The archive is empty."),
187+
);
188+
process.exit(1);
189+
}
190+
if (segment.skipped > 0) {
191+
console.error(`transcript: ${segment.skipped} unreadable line(s) in ${segment.path} (a truncated write) — the rest is below.`);
192+
}
193+
194+
const pattern = args.grep === null ? null : new RegExp(args.grep, "i");
195+
const lines = pattern === null ? segment.lines : segment.lines.filter((line) => pattern.test(line.text));
196+
197+
if (args.json) {
198+
console.log(JSON.stringify(lines, null, 2));
199+
} else {
200+
if (lines.length > 0) {
201+
console.log(renderTranscriptText(lines));
202+
}
203+
const scope = pattern === null ? "" : ` matching /${args.grep}/i`;
204+
const todayNote = day === localDayKey(Date.now()) ? " (today, still being written)" : "";
205+
console.error(`\n— ${lines.length} line(s)${scope} from ${day}${todayNote} · ${segment.path}`);
206+
}

src/server/app.test.ts

Lines changed: 120 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,10 @@
11
import { afterEach, beforeEach, describe, expect, test } from "bun:test";
2-
import { existsSync, mkdtempSync, rmSync } from "node:fs";
2+
import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs";
33
import { tmpdir } from "node:os";
4-
import { join } from "node:path";
4+
import { join, resolve } from "node:path";
55
import { Hono } from "hono";
66
import { createPhoneImportApp, createProjectorApp } from "./app";
7+
import { TRANSCRIPT_ARCHIVE_DEFAULT_DIR, listDays, localDayKey, readDay } from "./transcript-archive";
78
import { registerForestSurface, type ForestState, type ForestSurfaceLoader } from "./github-org";
89
import { RemoteHandsHub } from "./remote-hands";
910
import { createProjectorRuntime, type ProjectorRuntime, type ProjectorRuntimeOptions } from "./composition";
@@ -2632,3 +2633,120 @@ describe("GitHub import → deploy resolver → deployUrl surfaces", () => {
26322633
expect(runtime.snapshot().processes.find((entry) => entry.upid === upid)!.deployUrl).toBeNull();
26332634
});
26342635
});
2636+
2637+
// THE READ-BACK SURFACE. "Get me today's transcript" used to be a bespoke
2638+
// python pass over a rolling file that had already evicted most of the evening;
2639+
// the archive answers it with a read. (`bun run transcript` is the same read
2640+
// without a server, for when the room is down.)
2641+
describe("GET /api/transcript/*", () => {
2642+
function seedArchive(lines: { text: string; atMs: number }[]): string {
2643+
const dir = mkdtempSync(join(tmpdir(), "vibersyn-transcript-"));
2644+
tempDirs.push(dir);
2645+
const byDay = new Map<string, string[]>();
2646+
for (const entry of lines) {
2647+
const day = localDayKey(entry.atMs);
2648+
const body = JSON.stringify({
2649+
time: new Date(entry.atMs).toISOString().slice(11, 19),
2650+
speaker: "speaker_0",
2651+
text: entry.text,
2652+
kind: "room",
2653+
atMs: entry.atMs,
2654+
});
2655+
byDay.set(day, [...(byDay.get(day) ?? []), body]);
2656+
}
2657+
mkdirSync(dir, { recursive: true });
2658+
for (const [day, bodies] of byDay) {
2659+
writeFileSync(join(dir, `${day}.jsonl`), `${bodies.join("\n")}\n`);
2660+
}
2661+
return dir;
2662+
}
2663+
2664+
const noonToday = new Date(new Date().setHours(12, 0, 0, 0)).getTime();
2665+
const noonYesterday = noonToday - 24 * 60 * 60_000;
2666+
2667+
test("today's lines come back, in spoken order, with their original atMs", async () => {
2668+
const archiveDir = seedArchive([
2669+
{ text: "we should build a birdhouse app", atMs: noonToday },
2670+
{ text: "with a webcam feed", atMs: noonToday + 2_000 },
2671+
]);
2672+
const { app } = await makeApp({ runtimeOptions: { transcriptArchiveDir: archiveDir } });
2673+
const response = await app.request("/api/transcript/today");
2674+
expect(response.status).toBe(200);
2675+
const body = (await response.json()) as { day: string; archiveDir: string; lines: { text: string; atMs: number }[] };
2676+
expect(body.archiveDir).toBe(archiveDir);
2677+
expect(body.lines.map((line) => line.text)).toEqual(["we should build a birdhouse app", "with a webcam feed"]);
2678+
expect(body.lines[0]!.atMs).toBe(noonToday);
2679+
});
2680+
2681+
test("yesterday, and an explicit YYYY-MM-DD, address their own segments", async () => {
2682+
const archiveDir = seedArchive([
2683+
{ text: "last night", atMs: noonYesterday },
2684+
{ text: "this afternoon", atMs: noonToday },
2685+
]);
2686+
const { app } = await makeApp({ runtimeOptions: { transcriptArchiveDir: archiveDir } });
2687+
const yesterday = (await (await app.request("/api/transcript/yesterday")).json()) as { lines: { text: string }[] };
2688+
expect(yesterday.lines.map((line) => line.text)).toEqual(["last night"]);
2689+
const explicit = (await (await app.request(`/api/transcript/${localDayKey(noonToday)}`)).json()) as { lines: { text: string }[] };
2690+
expect(explicit.lines.map((line) => line.text)).toEqual(["this afternoon"]);
2691+
});
2692+
2693+
test("?format=text renders LOCAL stamps a human can read", async () => {
2694+
const at = new Date(new Date().setHours(17, 52, 1, 0)).getTime();
2695+
const archiveDir = seedArchive([{ text: "not i just", atMs: at }]);
2696+
const { app } = await makeApp({ runtimeOptions: { transcriptArchiveDir: archiveDir } });
2697+
const response = await app.request("/api/transcript/today?format=text");
2698+
expect(response.status).toBe(200);
2699+
expect(await response.text()).toBe("17:52:01 speaker_0: not i just");
2700+
});
2701+
2702+
test("/days lists what the archive holds, oldest first", async () => {
2703+
const archiveDir = seedArchive([
2704+
{ text: "last night", atMs: noonYesterday },
2705+
{ text: "this afternoon", atMs: noonToday },
2706+
]);
2707+
const { app } = await makeApp({ runtimeOptions: { transcriptArchiveDir: archiveDir } });
2708+
const body = (await (await app.request("/api/transcript/days")).json()) as { archiveDir: string; days: string[] };
2709+
expect(body.days).toEqual([localDayKey(noonYesterday), localDayKey(noonToday)]);
2710+
});
2711+
2712+
// Every failure says something specific. An empty array would read as "we
2713+
// said nothing", which is the one answer that must never be guessed at.
2714+
test("a malformed day is 400, a day with no segment is 404, no archive is 503", async () => {
2715+
const archiveDir = seedArchive([{ text: "this afternoon", atMs: noonToday }]);
2716+
const { app } = await makeApp({ runtimeOptions: { transcriptArchiveDir: archiveDir } });
2717+
const bad = await app.request("/api/transcript/tomorrow");
2718+
expect(bad.status).toBe(400);
2719+
expect((await bad.json()) as { error: string }).toHaveProperty("error");
2720+
const missing = await app.request("/api/transcript/2001-01-01");
2721+
expect(missing.status).toBe(404);
2722+
expect(((await missing.json()) as { days: string[] }).days).toEqual([localDayKey(noonToday)]);
2723+
2724+
const { app: noArchive } = await makeApp();
2725+
expect((await noArchive.request("/api/transcript/today")).status).toBe(503);
2726+
expect((await noArchive.request("/api/transcript/days")).status).toBe(503);
2727+
});
2728+
});
2729+
2730+
// THE DEFAULT-ON GUARD, asserted the way the operator asked: a runtime built
2731+
// the way TESTS build one must write NOTHING to the default archive path.
2732+
// Commit 6a1d228 gated the old store behind an env marker because self-mode
2733+
// test runtimes were polluting the operator's live store; making the archive
2734+
// default-on at the BOOT ENTRY (src/server/index.ts) keeps that hole shut, and
2735+
// this test is what proves it stays shut.
2736+
describe("a test-built runtime keeps no archive", () => {
2737+
const defaultArchivePath = resolve(process.cwd(), TRANSCRIPT_ARCHIVE_DEFAULT_DIR);
2738+
2739+
test("no archive directory resolves, and the default path is never created", async () => {
2740+
const before = existsSync(defaultArchivePath) ? listDays(defaultArchivePath).map((day) => `${day}:${readDay(defaultArchivePath, day).lines.length}`) : null;
2741+
const { runtime } = await makeApp();
2742+
expect(runtime.transcriptArchiveDir).toBeNull();
2743+
const after = existsSync(defaultArchivePath) ? listDays(defaultArchivePath).map((day) => `${day}:${readDay(defaultArchivePath, day).lines.length}`) : null;
2744+
expect(after).toEqual(before);
2745+
});
2746+
2747+
test("even a SELF-MODE runtime with no directory keeps no archive", async () => {
2748+
const { runtime } = await makeApp({ runtimeEnv: { VIBERSYN_SELF_MODE: "1" } });
2749+
expect(runtime.transcriptArchiveDir).toBeNull();
2750+
expect(existsSync(defaultArchivePath) ? listDays(defaultArchivePath) : []).not.toContain("__never__");
2751+
});
2752+
});

0 commit comments

Comments
 (0)