All notable changes to agent-replay are documented here. The format follows
Keep a Changelog, and the project aims to
follow Semantic Versioning.
A broad hardening pass across the whole CLI, with one theme above the rest:
a command should never report success, or a number, that it did not
actually measure. Highlights: consistent exit codes and strict argument
parsing for scripting and CI; gates that fail when they cannot do their job
rather than passing green (check --golden, eval, record, guard check);
correctness of the comparison, evaluation, and golden-regression paths
(diff, eval, check --golden); guardrail enforcement that fails closed
(hook --enforce); readers that show what is actually stored (show,
replay, list, stats); more faithful live capture and import (record,
run, import, fork); a more robust OpenTelemetry receiver that keeps the
content, timing, and identity its dialects carry; output that is safe to look
at, since a trace is written by the agent under test and every command that
prints one now escapes what it shows (and an id, which is rendered nearly
everywhere, must be an identifier before it can be stored at all); and a
whole-store export that no longer costs time quadratic in the size of the
store.
Two things to know before upgrading:
- The supported Node range is now
>=20.12. Node 18 has been end-of-life since April 2025, andbetter-sqlite3— a native module — no longer builds against it or against current releases at the version previously pinned, sonpm install -g agent-replayfailed at install time on Node 24 and newer. The dependency moved to a version with prebuilt binaries for Node 20 through 26, and CI now tests all four. - A few commands now refuse input they used to accept, always where
accepting it produced a silently wrong answer rather than an error: an empty
value for a narrowing flag on
list,export,checkorconfig set(which widened the scope to everything, at exit 0); a--refresh/--intervallarger than a timer can hold (which inverted into a busy loop); anddashboardwithout an interactive terminal (which hung forever). Each exits2.
The recorded trace data model is unchanged — same tables and columns, so existing stores and exports keep working. Schema v3, v4 and v5 add four indexes between them, and nothing else. Upgrades are automatic and one-way.
-
check --fields decisionsgates on what the agent CHOSE. The structural gate was blind to the one divergence the tool exists to explain: rename nothing, change no tool, and swapescalate_to_humanfordelete_records, and step count, types, names, tool inputs and status all still match — green.export --format goldennow carries each step's chosen option, and--fields decisionscompares it. Opt-in likemodel, because no baseline exported before this carries the data and making it a default would turn a working gate into the "nothing to compare" refusal on upgrade. Only a step the baseline recorded a decision for is compared, and every differing step is reported rather than just the first. -
check --agent-exact <name>selects exactly one agent, for a regression gate.--agentmatches by substring, which is right for browsing and wrong for a gate:--agent assistantalso selectstravel-assistantandresearch-assistant, and under--strictthose unrelated candidates decide the verdict. Pair it with a baseline exported for the same agent. The two flags are mutually exclusive rather than one silently taking precedence. -
import --replacere-imports a session that is already in the store. -
check --goldencompares whether each step FAILED. A baseline could not carry step failure at all, so the regression class the gate most needs to catch — identical step shape where every tool call now errors — was structurally invisible, andstatusdoes not cover it, because a hook-captured session finalizescompletedfrom its Stop event however many tool calls failed inside it.export --format goldennow records the outcome of every step andstep_errorsis compared by default. Only the flag is stored, never the message: error text carries ids and paths that differ run to run, and a gate that fails on wording is the false-positive problem this format avoids. Baselines exported before this field are skipped step by step, never guessed at. -
AI provider calls now have a deadline and a bounded retry budget.
fetchhas no default timeout, so a provider that accepted the connection and then stalled hungeval --ai,diff --aiorconfig test-aiforever — an unattended CI job with no output and no way to fail. And a single 429 or 503, routine on a shared key, failed a whole evaluation run. Each attempt now has a 60-second deadline that covers the response body as well as the connect (a provider can send headers promptly and stall mid-stream), and a transient failure — 429, 5xx, network error, timeout — is retried twice with a doubling backoff, honoringRetry-Afterwhen the provider sends one. Failures that cannot succeed on a second attempt (bad key, 4xx, unparseable reply) are not retried. Retried attempts return no usage, so--max-costaccounting is unchanged;latency_mscovers every attempt and the waits between them. -
Schema v3 adds two indexes for lookups that were full table scans.
otel serveresolves every incoming batch againstjson_extract(metadata, '$.otel_trace_id'), which nothing could index, so cross-batch assembly re-scanned the whole trace table once per batch and a long-running receiver grew steadily slower as the store filled. The dashboard's recent-scores query likewise sorted the entire evals table on every refresh tick. The migration is additive — indexes only, no columns and no data — so an older binary opening a v3 store is unaffected. -
Schema v4 adds an expression index on
julianday(started_at)so the parsed-instant orderinglistand the dashboard use is an index seek rather than a full scan plus a temp B-tree. Additive, like v3. -
check --allow-emptyaccepts a run where no candidate trace is expected — a quiet nightly window, or a matrix job where a given agent didn't run. Failing on zero candidates is right by default, but it needed an escape hatch that isn't "stop running the gate". -
A
statscommand prints a non-interactive summary of the trace store — overall counts (traces, steps, evals, active policies), average duration, and token/cost totals, plus a per-status and per-agent breakdown (each agent's trace count and a failed+timeout tally). It exposes the same aggregates as thedashboardTUI but works in a plain terminal, a log, or CI, and--jsonemits{ since, overall, by_status, by_agent }for piping intojqor a gate. Previously these numbers were reachable only through the full-screen dashboard, which needs an interactive TTY.stats --since <window>(a duration like7d/24hor an ISO date, matchinglist --since) windows every count to traces started at or after the cutoff — steps and evals by their parent trace's start time, so the view is internally consistent — while the active-policy count stays store-wide (current config, not history). A malformed--sinceis a usage error (exit2);--jsonalways carries asincefield (null when no window was asked for). -
exportnow accepts an optional[trace-id]positional, so you can export a single trace by id (with prefix matching, likeshow/why/replay) instead of only bulk-filtering. A trace id and the filter flags (--status,--agent,--tag,--since) are mutually exclusive — passing both is a usage error (exit2) rather than silently ignoring the filters, and an unknown id exits1. Previously a trace id passed toexportwas silently dropped and the whole database was exported. -
The
otel servereceiver now acceptsPOST /v1/logsin OTLP/protobuf as well as OTLP/JSON (it already accepted both on/v1/traces). OTLP exporters default to protobuf, so a Gemini CLI or Claude Code session left on the default protocol now has its log events ingested without switching the exporter to JSON. Malformed protobuf log bodies answer400, matching the traces path. -
hook --dialect <name>declares the harness dialect for--enforcereplies (claude-code,codex,gemini, orother). This makes the documented "harness without structured output exits 2" behavior reachable: the dialect is otherwise detected from the payload, and detection can only answer with a harness it recognizes, so a Crush user registeringhook PreToolUse --enforcewas answered with Claude-shaped JSON on exit 0 — which a harness that doesn't read hook stdout ignores, letting the denied call run. Nothing in a payload distinguishes such a harness, so the user says. -
guard disable <policy>andguard enable <policy>turn a policy off and on without deleting it. Every policy carries an enabled flag that evaluation already respected, but nothing could set it: silencing a rule meant deleting it — losing its id, priority and description — and retyping it to bring it back. Resolves by id or name, likeguard remove. -
check --goldenreports baseline entries that no candidate exercised. The verdict was candidate-driven only, so a scenario whose run crashed, recorded under a different agent name, or ran a different input silently vanished from the gate — it reported "1 passed" and exited0while the rest of the baseline went unchecked. Reported in the summary and in--jsonasuncovered; a failure only under--strict, which already fails unmatched runs. -
export --format goldenwarns when a baseline is built from runs that did not complete. Arunningtrace bakes in a truncated shape, so the next correct run "regresses" against it; afailedortimeoutone makes a candidate that faithfully reproduces the break pass green. Both are silent otherwise and both survive into CI as a wrong verdict.
-
The README now states exactly how a value is stored, for the three cases where what comes back is not literally what went in:
input/outputkeep their type (a string stays a string; a string that spells an object or array is stored as that structure),erroris a TEXT column so a structured error flattens to JSON text, and trace-level totals are taken at face value and never reconciled against the steps — so a producer whose total disagrees with its own steps is whatstatsreports. All three were verified against the binary; the last two are deliberate and unchanged, they were simply undocumented. -
Dropped two runtime dependencies nothing imported.
cli-highlightandfigureswere declared independencies, so npm downloaded and installed them (~200 KB) for every consumer ofagent-replay, and they counted toward the audit surface — while no file insrc/referenced either one. A new test asserts that every declared runtime dependency is actually imported. It deliberately matches dynamic imports too:yamlis loaded throughawait import('yaml')inside the rubric parser, and a stricter check would have called a real dependency unused, which is the dangerous direction to be wrong in. -
Two capabilities that had no spec at all now have one.
openspec validatechecks document structure, not truth, so a whole area can change while the specs stay silent and green — which is what happened here: nothing described where the store lives, who may read it, how configuration is loaded, or how producer-controlled text reaches the terminal, and every one of those areas had a defect this release fixes.local-storecovers path resolution (including blank values and~), store confidentiality, configuration loading, and independent store handles.terminal-outputcovers control-character escaping, the single-line forgery rule, bounding and column-based width, the non-interactive refusal, and degrading rather than crashing on a drawing problem. Each requirement states the rule and why it exists, so the next fix in the area has something to be complete against. -
Documentation now matches the binary.
openspec/specs/trace-inspectionclaimed an ambiguous trace-id prefix returns "the first match — there is no ambiguity error", which is the exact inverse of what the code does and would have licensed reintroducing a fixed bug; it also omittedlist --session, the empty-filter refusal, instant-based ordering, and bothdashboardrefusals.openspec/specs/guardrailsdid not record thatguard addrefuses a pattern that cannot match as written.openspec/project.mdstill said Node >= 18. The README's schema note still said v4, and did not document the newlist,dashboardandconfig setrefusals, theavgDurationSamplefield, or the store's file permissions. All corrected against the running binary. -
The export scaling guard no longer measures time at all. It asserted that a 3000-trace export finishes in under 5 seconds — an absolute wall-clock bound in a suite that runs files in parallel and spawns real CLI processes, so it failed on a loaded machine for reasons unrelated to the code. A time RATIO between two store sizes was tried next and still failed under load, and averaging that over repeated runs let SQLite's page cache defeat it — the quadratic version then passed, which is worse than flaky. The property is now asserted two ways, both exact: the query plan SQLite chooses for a canonical-id lookup (a keyed
SEARCH, neverSCAN agent_traces), and a count of how many statement executions full-scan the table during a whole-store export, which must not grow with the store. Against the original quadratic lookup that count is 202 scans at 200 traces and 802 at 800 — the quadratic signature itself. -
The JSONL reader's equivalence test no longer trips the suite timeout. It read a 200 KB line one byte at a time — 200,000 syscalls per chunk size — and took ~110s against a 60s per-test limit, so it failed intermittently on a loaded machine. What it proves is that the carry buffer grows past the chunk, not the size it reaches, so the very long lines now run only at realistic chunk sizes and a 2 KB line (still 2,000x the smallest chunk) covers the tiny ones. Both properties stay covered and the run drops from ~110s to ~2s; verified the trimmed test still catches a broken carry.
-
The supported Node range is now
>=20.12, andbetter-sqlite3moved to^12.11.1. The old>=18floor was a promise the package could not keep:better-sqlite3v11 does not compile against Node 24 or newer (V8 removed thev8::Object::GetPrototypeandPropertyCallbackInfo::ThisAPIs it uses), and it has no prebuilt binary for those releases — sonpm install -g agent-replayon a current Node failed at install time with a C++ compiler error, before any code ran. Node 18 has been end-of-life since April 2025, and CI already floored at 20 because Vitest 4 needsutil.styleText(added in 20.12), so the declared range was the only thing still claiming 18.better-sqlite312.11.1 ships prebuilds for Node 20 through 26. The CI matrix now also covers Node 24 and 26, and a new test ties the advertisedenginesrange to the range the native dependency actually supports, so a future bump cannot quietly outrun it. -
The openspec specs now describe what the implementation actually guarantees.
openspec validate --allonly checks document structure, so nothing had verified the normative statements against the binary. The specs had fallen behind in three places: the Codex importer's second tool family (custom_tool_call, the dominant form in real rollouts), itsevent_msgwrapper andtoken_counttotals, plus import idempotency, the identity key, prompt/preamble retention,ended_at, streaming and the line limit (trace-capture); how an OTel trace's status derives from its root span rather than any child, and redelivery de-duplication (telemetry-ingest); and every gate-broken refusal, unmatchable empty inputs,--agent-exact, and the fact thatexpected_output/eval_criteriaare carried rather than asserted (trace-evaluation). Each statement added was verified against the binary. -
Transcript imports read the file a line at a time instead of loading it whole. Slurping it as one string and splitting that kept three copies alive at once — the string, the array of lines and the parsed records — measuring 436 MB of peak RSS for a real 52 MB session, and a JavaScript string cannot exceed ~512 MB, so a larger session failed outright with "Cannot create a string longer than 0x1fffffe8 characters" and produced no partial import. Long agent sessions do reach that size. The same 52 MB session now peaks around 270-290 MB (it varies run to run), a 647 MB one imports its 672,000 steps where it previously could not be read at all, and the resulting trace is byte-identical to what the previous reader produced (tallies, steps, tokens and per-step errors all compared on a real transcript).
-
A criterion that detects a failed run can now fail its preset on its own.
hallucination-checkandcompleteness-checkweigh their criteria 0.4 / 0.3 / 0.3 against a 0.7 threshold, so a lone zeroed 0.3-weight criterion landed on EXACTLY 0.7 and passed: the one criterion in each preset that detects a failed run was arithmetically incapable of failing it, and a trace the tool renders as✘ FAILEDreported "70% PASS" beside a Details column naming that criterion, exit 0. Those two criteria are now marked critical — scoring 0 fails the preset whatever the total says, and the report names which one forced the verdict. A run that was passing on a zeroed error criterion will now fail, which is the point. Raising the threshold instead would have moved the PARTIAL band too: a retrieval answer that paraphrases rather than quotes scores about 0.3 on the word-overlap grounding heuristic, and a run with 1 of 7 tool calls completed scores 0.743 — both would have started failing on arithmetic unrelated to the defect. Every other band is exactly where it was. -
Exit codes are now consistent across the CLI, so scripts and CI can gate on
$?: every failure exits non-zero —1for a runtime failure (not found, malformed input, acheck --goldenregression, anevalover its threshold) and2for a usage error or aguard/hook --enforceblock. Success and empty results exit0;runpropagates the child's status;hookcapture always exits0. Previously several commands printed an error but still exited0(exportinvalid format,guard addinvalid pattern/action,importwith nothing importable,watch/whynot-found,diff --aiwith no provider,demo --resetrefusal). A new "Exit codes" section in the README documents the convention. -
Argument parsing now fails loudly on mistakes. Every command rejects unexpected extra positional arguments instead of silently ignoring them, so
agent-replay show <id> <typo>orlist production(meant as--tag production) errors rather than quietly running on the first argument. And commander's own parse errors (unknown flag, unknown command, missing/excess argument) now exit2to match the documented "usage error" code — they previously exited1, contradicting the README's exit-code table. -
demo --resetrefuses to delete a store named only byAGENT_REPLAY_DIR. Deleting someone's traces has to be something they typed, so the destructive path requires an explicit--dir(everything non-destructive still honors the handshake). -
Every command honors
AGENT_REPLAY_DIRas its data directory when--dirisn't given.runsets that variable for its child and the README documents it as how the wrapper hands the child its store, but nothing read it back — so a nested invocation (run -- sh -c '... | agent-replay record') wrote to a fresh./.agent-replayinstead of the store the wrapper had just opened a trace in. An explicit--dirstill wins.
-
Four smaller
checkcontract defects. A baseline whosesteps_summaryholds anullor a bare string died inside the comparison with "Cannot read properties of null", naming neither the file nor the entry — the same diagnostic failure the adjacent shape guard exists to prevent, one level deeper.check --jsonwith--goldenomitted printed a bare commander usage line to stderr and produced nothing on stdout, breaking the documented--jsoncontract; the command's own guard for that case existed but never ran, because arequiredOptionis enforced before the command body.check --trace <missing>exited 2 where the README's table anddiffboth say 1, so a CI script splitting 1 (a regression) from 2 (a broken gate) read a typo'd id as a broken gate. And a divergence whose baseline entry simply lacked the field emitted nogoldenkey at all, against a type that declares it required. -
export --format goldento stdout suppressed both baseline-trust warnings. The warning was emitted only for a--outputfile, on the stated reasoning that it "would be noise in the middle of someone's pipeline" — but it is written to stderr, so it could never reach a redirected or piped stdout. There was no noise to avoid, and the condition instead re-created the exact false green the warning exists to prevent:export --format golden > golden.jsonis an ordinary idiom, and it produced a baseline built from failed or in-flight runs, or an entirely empty one, with no signal at all. Two byte-identical baselines, one warned about and one not, purely by how the bytes were routed. -
diffrendered a type-only difference as two identical cells. The table printed values withString(v), which collapses the very distinction the comparison had just used to decide the traces differ: a step output of the string"42"and one of the number42both printed42, under a header reading "1 difference(s) found" — so the only way to see what changed was to re-run with--json. That is the failure the diff renderer's windowing was written to end, arriving by another route. When both sides render to the same text, their JSON form is shown instead, so ordinary values keep their plain rendering. -
Live capture stored causal references to steps that do not exist, and
whythen invented a different antecedent and presented it as fact. The live path checked aparent_step/caused_by_stepfor range (a positive integer, strictly earlier) but never for existence — so a producer whose counter skips, or the ordinary case where one step is rejected for a badstep_typeand the next references it, stored a dangling number.whylooked it up, found nothing, and fell through to its "prior decision" fallback with no hint that the recorded cause was unresolvable;show --treeprintedcaused by #2for a step not in the trace, so two surfaces contradicted each other about one trace; andexportproduced a traceingestrefuses — the tool rejecting its own output.ingestalready checked existence, and the decision-tracing spec requires it. The reference must point strictly earlier, so the step is already stored by then and existence is answerable at write time; a dangling one is now dropped and reported byrecordandrun. -
A producer's string was re-typed by what it happened to say.
inputandoutputstored a string as-is whenever it parsed as JSON, so the type a value came back as depended on its content:"42"returned the number 42,"true"the boolean, and — the damaging one — a tool that returned the four-letter textnullwas stored as JSON null and became indistinguishable from a step that produced nothing, withshowrendering no Output line at all for it. Pass-through is now restricted to a string that is genuinely an object or an array, which is what the behavior was written for (OTel attributes and harness payloads carrying JSON text) and what every reader of those columns expects. A scalar-looking string stays the string it was. -
Live capture dropped a forward causal reference in silence. A
parent_steporcaused_by_stepthat does not point strictly earlier is already refused at write time —causalWalkdepends on the graph being acyclic, and a forward reference madewhypresent time-travelling causality as fact — but the liverecord/SDK path discarded it without a word, whileingestrejects the same input loudly with the field named. It was the one door where a producer could send a reference, be told nothing, and later find it missing. It is now reported the way an unusable numeric field beside it already is: the field is dropped, the step is kept, and the warning names both the field and why. -
The OTLP logs endpoint reported a rejection only when the WHOLE batch was unrecognized. The guard was "nothing mapped", so a batch in which anything at all was recognized answered a bare 200 and the rest was discarded silently — and the drift this reporting exists for is normally partial, since a CLI version bump renames some events and keeps others.
partialSuccessnow carries the count whenever any record was not recognized, naming how many of how many. (Scope: this counts records whose event name the mapper does not recognize. One carrying a known prefix with an unknown suffix still passes the filter and is not counted; narrowing that needs per-record reporting from the mapper.) -
A span id repeated inside a single OTLP batch was stored twice. The merge path already refuses a span id it saw in an earlier batch — that is what makes an exporter's retry safe — but the check compares against what is stored, so it cannot see a duplicate that arrives twice inside one payload. A batch listing the same span twice became two steps sharing an
otel_span_id, with its tokens counted twice (3 steps and 45 tokens where 2 and 30 were correct). The identity and the argument are the same on either side of a batch boundary, so the same rule now applies within one. A span carrying no id is left alone, since there is nothing to key on. -
The Codex importer counted an orphan tool output as imported. A
function_call_output/custom_tool_call_outputwhosecall_idpairs with no call record lands in no step — routine when a rollout is head-truncated, or when the call record itself was unparseable — but it was reported as imported anyway, crediting the store with content it does not hold. The Claude transcript importer already tracked exactly this for its own orphan tool results; Codex had no equivalent. Such a record is now counted as skipped, soimported + skipped = recordsstill holds. -
hook --enforcelabelled its decision with an event name the harness will not match. Routing deliberately ignores ahook_event_nameit cannot route and falls back to the event registered on the command line — that fallback exists because an unroutable name used to skip every gate. But the response formatter read the payload's name directly, so a deny could come back labelled with the very name that had just been ignored. Claude Code keyshookSpecificOutputon a matchinghookEventName, so such a decision is not applied, and the process exits 0, so the call runs. Reproduced withtool.before,"PreToolUse "(trailing space) andpretooluse. The response now carries the name routing actually used. -
A rubric with
threshold: 0reported "All criteria passed" at 0%. The Details column selects failing criteria withscore < threshold, which cannot express "did not pass" when the threshold is 0 — nothing is below it — so every criterion that scored zero was folded into a claim that they had all passed. That is the same false summary the line was first written to fix for a hardcoded 0.7, reappearing at the other end of the range. A criterion that scored zero is now always named. (The overall PASS at 0% follows from the author's ownthreshold: 0; a rubric whose gate can never fail is worth a refusal of its own, which this does not add.) -
hallucination-checkreported "70% PASS" for a run in which every step failed. The criticality rule that stops a failed run from passing only covers a trace that ended badly by status or by a trace-level error. A run recordedcompletedtook the other branch, where the error criterion scores 0 but is not critical — and 0.4 + 0.3 + 0.3·0 is exactly the 0.7 threshold, so on that branch the criterion could not fail the preset however bad the run was. The output named the failing criterion in the Details column beside the word PASS, andevalexited 0. A run where nothing succeeded is not a recovered error, whatever status it recorded, so it is now critical. Partial failure still passes, deliberately: that is the documented intent (one failed shell command in an imported session must not fail the preset), and narrowing it further would mean re-weighting a rubric users gate CI on, moving every score. -
record --format gemini-streamand--format codex-execdropped lines and reported "Warnings: 0". The native protocol counts and reports every line it rejects, precisely so a silent loss is impossible; the translated formats had no counter at all. Atool_resultthat paired with no open tool call took the tool's output with it — the call was stored looking clean and output-less — and an event type the translator had never heard of vanished the same way, both under a clean summary at exit 0. Translators now say why they produced no events, andrecordreports it. The distinction matters: producing nothing is sometimes correct (a repeatedinit, a line that only accumulates usage), so an empty result alone could not be used as the signal — flagging those would train the reader to ignore warnings. -
import --replacedeleted the forks of the trace it replaced. A fork inherits its parent'ssession_idand itssource_format/source_filemetadata, so every fork of the session matched the "already imported" key and was deleted alongside the parent — and--replaceis the documented way to refresh a transcript that has grown, so the routine refresh destroyed the user's what-if sandboxes. Excluding forks from that lookup is not enough on its own:parent_trace_idisON DELETE SET NULL, so a surviving fork would be silently promoted to a real run, andparent_trace_id IS NULLis the only thing marking a fork as never-executed — golden export,check,statsandwatchall rely on it, so the fork would start counting as real spend. Re-pointing the fork at the new trace is not available either, since a refreshed transcript may have different steps andforked_from_stepwould no longer mean what it meant.--replacenow refuses when forks derive from the trace, naming them and the two ways forward. Every sibling lookup in the codebase already excluded forks for this reason; this one door did not. -
A guardrail policy with an empty
*_containsneedle blocked every step.''is a substring of every string, soguard add --pattern '{"name_contains":""}' --action denywas stored without complaint and then deniedread_file, every LLM call, and everything else — a fail-closed that is really fail-broken, and reachable from an ordinary authoring slip (--pattern "{\"name_contains\":\"$TOOL\"}"with$TOOLunset in CI kills every tool call in the session). It also falsified the warningguard addprints foroutput_containspolicies, which says they cannot block live: this one could. The fold-away sibling — a needle of only zero-width characters — was already rejected for exactly this reason, and the literal empty string had been explicitly excluded from that check. It is now refused at write time, and a policy already stored fails closed with a reason that says the pattern is unusable rather than claiming the step's name matched. -
A store in a read-only directory now says why it cannot be opened. The message was "check file and directory permissions", which does not convey the surprising part: the database runs in WAL mode, and WAL keeps its index in a
-shmsidecar SQLite creates next to the database — so the DIRECTORY must be writable even for a command that only reads. An operator who deliberately locked a store down withchmod 500had no way to guess that from the old wording. There is no read-only mode to offer instead: opening the store withreadonly: truefails identically, for the same reason. -
Table and diff cells were budgeted in code units, not columns. Three copies of
truncateexisted — injson.ts,table.tsanddiff-renderer.ts— and all measured UTF-16 code units against what is a column budget (colWidths). A CJK character is one code unit and two columns, so a cell built to a 40-unit budget rendered 80 columns wide and pushed the table border out.table.ts's copy was additionally not surrogate-safe, so it could cut an emoji in half — the exact defect the diff renderer had already been fixed for, in a sibling file that had its own copy. All three now share one width-aware truncation, which walks by code point and so cannot split a pair either. -
exportandcheck --sincestill widened on an empty value.listwas fixed for this; its siblings were not. It matters most inexport, which WRITES:export --agent "$AGENT"with the variable unset silently dumped the whole store into a file the caller believed held one agent's traces — and a golden baseline built that way then gates on runs it was never meant to cover.check --since "$WINDOW"unset gated over the entire store instead of the window, green for the same reason. Both now refuse at exit 2. -
watch --intervalhad the same 32-bit timer overflowdashboard --refreshwas just capped for. It validated only that the number was positive, so--interval 999999999999— plainly "poll almost never" — was clamped by Node to 1 ms and polled SQLite about a thousand times a second. Now refused, with the same reasoning: a value that inverts the request is not clamped quietly. -
Five more commands could still be made to forge a line of output. The first pass converted
show,replayand the tables;stats,decisions,why,watchandguard testwere missed, and they print at column 0 with no gutter at all — so an agent namedevil\nagent-replay: store verified cleanproduced, understats' "By agent" heading, a line indistinguishable from this tool's own output. All five now escape single-line fields, and the producer-controlled names they render are bounded, as the trace table and timeline already were. Theshow/replayheader panel got the same treatment: a newline there inserted an unlabelled line inside the box, and an unbounded agent name turned the header into forty wrapped lines of border before the steps the user asked for. -
Hook capture wrote steps to the wrong trace, for any session whose newest trace carries an ISO basic-format offset. The same
julianday()NULL that brokelistordering also reached the hook adapter, whose own comment citedgetMostRecentRunningTraceas the reason to rank by parsed instant — but which still used the bare form. Here it is not a display bug: the hook WRITES. A tool call landed on an older run, and because the closing-event lookup does not filter on status, the tool RESULT could land on an unrelated trace while the real step stayed open forever — andhookdisagreed with whatlistandwatchshowed. The OTel receiver's cross-batch merge lookup had the same defect in the ASC direction (NULLs sort first there, so a basic-offset trace always won "oldest merge target"), andimport's prior-trace ordering still compared raw bytes, having never had the earlier fix at all. All four now use the repaired expression. -
show,init,ingestandreplaycrashed at a terminal width of 1 or 2 columns.boxenreadsprocess.stdout.columnsitself and subtracts its border width, computing a negative count and throwingRangeError: Invalid count value: -1.process.stdout.columnsis whatever the environment reports, not necessarily a real terminal width, and a wrong one must degrade the drawing rather than stop the command. The panels now fall back to plain text — the content is what the user came for; the border is decoration. -
The store file itself is now owner-only (
0600). The directory mode was the only thing protecting trace contents — prompts, tool inputs, tool outputs — andtraces.dbwas created0644by the umask, so the protection vanished whenever the directory was notagent-replay's to tighten:mkdir -p /var/lib/agent-replay && agent-replay init --dir …, a mounted volume, any pre-created path. The mode now goes on the file, where the content is, so it holds whoever made the directory. Set at creation only — a store an operator deliberately opened up stays open. -
demo --reset --dir " "cleared the store named byAGENT_REPLAY_DIR. The guard that stops a destructive command from inheriting its target from the environment tested the raw--diroption for truthiness, while path resolution had already decided a blank value means "not named" and fallen through to the environment. The two disagreed, so the guard printed nothing and the environment's store was cleared. Both now ask the same question. -
initstampedversion: "0.1.0"into every newconfig.json. The package has been 0.2.0 for a while; the literal was left behind in two places (that field, and the CLI's--versionfallback). Nothing reads the config field back, which is the only reason it did no damage — a stored value being false is still a defect. Both now read the shippedpackage.json, resolved once by walking up to the nearest one, and a test fails the next time a literal is left behind. -
Opening a second store closed the first one's handle.
ensureDatabaseis a documented export, and the connection cache was a single slot keyed on nothing: opening a second path closed the connection behind the first, so a library caller's first handle began throwing "The database connection is not open" from code that had done nothing wrong. The CLI opens one store per invocation, which is why nothing noticed. Connections are now keyed by resolved store path, so two stores can be open at once and the same path still returns the same connection. -
statsreported an average duration without saying what it averaged. A duration is unmeasurable for a trace that is still running, one whoseended_atprecedes itsstarted_at, or one whose timestamps no format parses — andAVGskips those. SoAvg duration: 5.0scould describe a single trace whileTraces: 100sat directly above it, in the panel and in the--jsona CI job reads. The denominator is now reported: the human panel appends(over N of M)when the two differ, and the JSON carriesavgDurationSamplealongsideavgDurationMs. -
Non-Latin text rendered wrongly in two different ways. The timeline budgeted its width in UTF-16 code units while the budget itself came from
process.stdout.columns— two different units. A CJK character is one code unit but two columns, so a line built to a 90-unit budget rendered about 193 columns: it wrapped several times and broke the│gutter that makes the timeline readable. It now measures withstring-width, ascli-table3andboxenalready did (measured 193 → 107 columns against a ~100 budget), andstring-widthis now a direct dependency rather than one relied on transitively. Separately, the dashboard's blessed screen was created withoutfullUnicode, so blessed substituted?for every wide or astral character in its draw path — a Japanese agent name showed as??????????in the Recent Traces panel whilelistdisplayed the same name correctly. -
One trace could make the whole listing unreadable.
agent_nameand a step'snameare producer-controlled and were rendered unbounded, while every neighbouring field was windowed. cli-table3 sizes a column to its widest cell, so a single trace with a 5,000-character agent name widened every row oflistto over 15,000 columns — the traces the user was looking for became unreadable because of a neighbour — and a 500 KB step name emitted one line of 500,031 columns inshow, scrolling the step's real input and output away above it. Both are now bounded (40 and 80 characters), matching the limits the dashboard andpolicyTablealready applied. -
Truncation could cut an emoji in half.
truncatesliced at a fixed offset, which can land between the halves of a surrogate pair and leave a lone surrogate the terminal shows as�. Whether it happened depended on the exact cut point, so the same value rendered correctly at one terminal width and as mojibake at the next. The JSON windowing helper already had a surrogate-safe cut for this reason;truncate, used by far more call sites, did not. It now shares that cut, and the duplicate helper is gone. -
Opening a store changed the permissions of a directory the tool did not create. The store is made owner-only because
config.jsonholds API keys in plaintext — but the narrowing ran on every open, against whatever path--dirorAGENT_REPLAY_DIRnamed. Pointing at an existing shared directory silently stripped group and other access from it,--dir .did that to the user's working directory, and even read-only commands did it:agent-replay listaltered the permissions of a directory it was only reading. Only a directory this tool creates is now given a mode; a pre-existing one belongs to whoever made it.initno longer pre-creates the directory either, so creation and its permissions live in exactly one place — previouslyinitmade it with the plain umask mode and it became private only as a side effect of the blanket re-chmod. -
A broken
config.jsonwas reported as a missing one. Every read failure — a stray trailing comma from a hand-edit, an unreadable file, a directory in its place — collapsed tonull, which every config command rendered as "No configuration found. Runagent-replay initfirst."initthen answered "Already initialized … Use --force", so the two messages contradicted each other, neither named the parse error, and the user was pointed at a command that refuses to run. The stored API key was still sitting in the file the whole time, sotest-aiandeval --aireported "No AI provider configured" about a key that was right there. A config file that exists but cannot be used is now its own error, naming the file and the parse position. -
config setpermanently deleted an unrelated invalid value. The reader drops unusable values so that one bad key cannot make the whole config unreadable — butconfig setwrote that sanitized copy back, so setting any key destroyed the invalidai.max_tokensthe user was being warned about. The typo became unrecoverable and every laterconfig listreported a clean config. Writers now start from the file as it actually is. -
config set <key> ""stored a blank that looked set and behaved unset. An empty API key was displayed as***byconfig getandconfig listwhile every check downstream treated it as absent — sotest-aitold the user to set the key they had just set. An emptyai.modelwas worse: it was sent to the provider AS the model name (Testing anthropic ()), but only whenai.providerwas explicit, since the auto-detect path guarded on truthiness and the explicit path did not. Empty values are now refused atconfig set, and a blank key or model already on disk is treated as unset by both paths. -
dashboardhung forever when there was no terminal, after writing alt-screen and mouse-tracking escape sequences into whatever its output was redirected to. It built the full-screen TUI unconditionally, so with stdin a pipe there was no keypress to exit on: a CI job that ran it never finished, and its log filled with control codes. It now refuses with exit 2 and points atstats --json, the wayguardchecks for a terminal before prompting andreplay --pauseskips its wait. -
dashboard --refreshaccepted values that turned into a busy loop. Node clamps a timer delay above 2,147,483 ms to 1 ms, so--refresh 999999999999— plainly "refresh almost never" — re-ran every dashboard aggregate about a thousand times a second, the exact inverse of the request. Values above the timer maximum are now refused rather than clamped, the same reasoning the command already applied to a malformed value. Argument validation deliberately runs before the terminal check, so a typo is still reported to the script that made it. -
A newline in a trace could forge a line of output. A trace is written by the agent under test, so every rendered string is untrusted — and
safeTextdeliberately preserves\nso a rendered block keeps its shape. On a one-line row that was a forgery primitive, and both forms reproduced inshowandreplay:- a step name of
safe\n ├─ 99 ➡ Output "…"drew a fabricated step row, indistinguishable from a real one; - an error of
line1\nagent-replay: all checks passeddrew a line at column 0 with no gutter, reading as agent-replay's own output in the operator's terminal and CI log.
escapeForMessagealready existed for exactly this reason and the render sites' own comments pointed at it, but the single-line renderers never adopted it. They now use a namedsafeLinefor fields that are single-line by construction (step name, model, decision, agent name, evaluator, policy name). Errors are handled differently on purpose: they keep their line breaks, since a stack trace or a Windows child's CRLF output is shaped information, and every continuation line is now drawn inside the step's gutter, where it is visibly trace content rather than tool output. Payload blocks (input,output, JSON) stay lenient — there a newline is content, not structure. - a step name of
-
listprinted the newest trace last,list --limit 1returned the wrong trace, andwatchattached to the wrong running run — for any trace whosestarted_atcarries an ISO-8601 basic-format offset (+0200). Ordering usedjulianday(started_at), which returns NULL for that form, so those rows had no instant to sort by: they clustered at one end of every result and were ranked among themselves by BYTES, which is exactly the failure the parsed-instant ordering exists to prevent.--limitthen dropped the wrong rows, andwatchwith no arguments showed a live session doing nothing while the real run went unwatched.Ordering now uses the repaired expression that already handles the format for durations. That was previously ruled out because schema v4 indexes the bare
julianday(started_at), so wrapping the column would have made every ordered query full-scan — so schema v5 adds an expression index over the repaired expression, and the ordering is now both correct and keyed. The migration is additive (a new index; no table is rewritten) and v4's index is kept, sinceSINCE_PREDICATE's indexed disjunct still matches it. A test asserts the query plan, because an ordering that is correct but unindexed would look identical in behavior while scanning the whole store. -
A
--dirorAGENT_REPLAY_DIRbeginning with~created a directory literally named~. A shell expands the tilde before the CLI sees it, so this only bit where nothing does — a quoted--dir '~/traces', a hook or settings JSON file, a Docker or systemdEnvironment=, a CIenv:block. The store was created under the working directory instead of the home directory, and a read command pointed at it reported an empty store at exit 0 rather than the traces the user asked for. A leading~or~/now expands;~otheruser/is deliberately left alone, since resolving another account's home is not portable. -
A whitespace-only
AGENT_REPLAY_DIRcreated a directory named spaces. The guard that treats a blank store path as unset tested!== '', so" "slipped past it — the same hazard the guard exists to prevent, wearing a name that is nearly invisible in a directory listing. The "is this set at all?" decision now uses the trimmed value; the path itself is still passed through untrimmed, since a directory name may legitimately end in a space. -
listwidened to the whole store when a filter flag was given an empty value.list --agent "$AGENT"with$AGENTunset returned every trace at exit 0, which reads exactly like a correct narrow result — the same silent scope-widening already refused bycheckfor--agent/--agent-exactand an empty--fieldslist, and bystatsfor--since.listis where a script is most likely to build a filter from a shell variable, and its own comment claimed it mirroredstats. An empty--status,--agent,--tag,--session, or--sinceis now a usage error at exit 2. -
--sinceincluded traces before the cutoff and dropped traces after it, for any trace whosestarted_atcarries an ISO-8601 basic-format offset (+0200) — whatdate +%FT%T%zemits and whatingeststores verbatim.SINCE_PREDICATEused a barejulianday(), which returns NULL for that form, so those rows fell back to the byte comparison the predicate exists to replace: wrong by the whole UTC offset, in both directions. A+0200trace an hour before the window was counted in, and a-0200trace inside it was counted out. This reached every command that windows by time — includingcheck --since, a CI gate that reported "2 trace(s) checked" over the wrong two traces. The row side now retries the basic form as the extended one, the same repairjulianDayExpralready made for durations and the same oneparseSinceToIsoalready made for the bound. The repair is confined to the branch that runs only when the barejulianday()returned NULL, so the indexed disjunct still matches schema v4's expression index exactly — checked withEXPLAIN QUERY PLAN, and asserted in a test. A timestamp no form can parse still fails open, as before. -
An AI provider's 4xx was reported as "Server error". A malformed request, an unknown model name or a wrong endpoint therefore read as a provider outage, so the natural next step was to wait and retry when the fix is local. Those are now "Request rejected (HTTP 400): ...". Retry behavior is unchanged — retries already keyed off a status of 500 or above, so a 4xx was never retried; only the message pointed at the wrong party.
-
check --agent ""andcheck --agent-exact ""silently checked every agent instead of the one intended. A CI script writing--agent-exact "$AGENT"with an unset shell variable therefore widened its gate from one agent to all of them and reported green. An empty value is now a usage error, matching what this command already does for an empty--fieldslist — a narrowing flag that quietly stops narrowing hides the mistake. -
eval --jsonemitted[]when every evaluator failed to run — an empty array that reads to a pipeline exactly like a clean run with no evaluators (jq lengthgives 0,jq '.[]|select(.passed==false)'gives nothing). With an invalid API key,eval --ai --jsonproduced that while all six AI evaluators had failed on authentication. Zero results is only reachable when evaluators threw, so it now uses the same{"ok": false, "error": ...}shape as every other refusal, with each evaluator's cause listed inhints. -
importno longer hangs forever on an input that is not JSONL. A source with no newlines had nothing bounding it, so a binary file passed by mistake buffered its whole self and a character device such as/dev/zeronever ended at all — measured, still running after 25 seconds under a 512 MB heap cap, and the previous whole-file reader hung there too. A single line over 64 MB (far beyond any real JSONL record) now fails in about a second with a message naming the limit; legitimate large transcripts are unaffected. -
importcould not read a non-seekable source, and failed silently on one. The new streaming reader read at explicit byte offsets and trusted the file's reported size: reading at an offset throwsESPIPE: invalid seekon a pipe, soimport /dev/stdinbroke, and a FIFO reports a size of 0, so the read loop never ran and the import announced "nothing importable found" for a source that had content — silent loss rather than an error. Reads are sequential now, with end-of-input taken from a zero-length read, which is the only signal true for a regular file, a pipe and a FIFO alike. -
A guardrail pattern made only of zero-width or soft-hyphen characters matched every step. Folding strips those, so the needle became the empty string, which is a substring of everything — a deny policy blocked
read_fileand all else, reporting "name contains ''" as though the name really matched. A stray zero-width character pasted into a pattern is exactly how that happens. Such a needle is now treated as an unusable pattern, taking the same path a non-string one does: still fail-closed for a blocking policy, but saying why. -
show --treenow states a step's depth once the indent stops growing. The indent is capped so a deep tree renders at all, but past the cap every level draws the same 122 spaces — a step at depth 60 was indistinguishable from one at depth 41, which traded a crash for a quietly wrong picture of the nesting. -
Two concurrent
guard addcalls for the same policy name could still surface the rawUNIQUE constraint failed: guardrail_policies.name— the message the duplicate-name pre-check exists to replace. That check reads outside a transaction, so both processes can pass it and one then hits the constraint; measured, four racing processes leaked it in 1 of 6 trials. The insert now maps that error to the same friendly message, so the loser of a race and a plain sequential duplicate get the same answer (0 of 10 trials leak it now). -
show --treecould not render a deeply nested trace. The walk recursed one frame per level of nesting, so a long parent chain blew the stack — measured, fine at depth 4,000 and "Maximum call stack size exceeded" before 8,000 — and the command printed a one-line error with no tree at all. That depth is reachable: a step's parent is the step before it in any run that threads causality linearly, and the tree is the view someone opens to understand a long session. The traversal is iterative now, and the indent stops growing past 40 levels: it grew three characters per level, so a 20,000-deep chain summed to roughly 600 MB of leading whitespace and failed with "Invalid string length" while building the output. A 100,000-step chain now renders in 14.6 MB. Output for ordinary trees is byte-identical, verified against the recursive version on a mixed tree with branches, a self-parent and a cycle. -
A guardrail policy could be evaded by a name that reads the same. Matching compared raw code points after case folding, so
name_contains: "delete"did not match the fullwidthdelete_user, nordelete_userwith a zero-width space or soft hyphen inside it — a policy an operator had written to block a tool silently allowed it. Both the needle and the step name are now Unicode-folded (NFKC, with zero-width and soft-hyphen characters removed), and aname_regexis tested against the raw and folded name alike. Folding can only make a policy match more, which is the safe direction for a guard. -
otel servewas the only capture path that failed a run because a tool call failed. The other eight storecompletedfor a session containing a failed tool, the telemetry-ingest spec says a span error becomes a step error, and the eval design deliberately does not hard-fail a preset for a recovered step error — so the identical session scored 0.700 and PASSED viaingestwhile FAILING at exit 1 via OTel, andcheck --goldenreported a status regression between two captures of one run. A trace's status now comes from its root span (or, on the log path, from a failed model call — the turn did not happen), and a failed trace always carries a reason: it was storedfailedwitherror: null, soshowrendered "✘ FAILED" with nothing to explain it. -
A golden gate could match two unrelated runs. An empty trace input hashed to the same key for every capture that records none, so two
record --format codex-execcaptures of different sessions compared as the same scenario — inventing atool_inputsregression between them — and a--strictrun reporteduncovered: 0at exit 0 while a baseline it never exercised sat unused. An empty input is now unmatchable, which routes it to the loud "no candidate matched" refusal, and unmatchable baselines count as uncovered rather than being quietly excluded. -
record --format codex-execnamed every tool step after the item type, so two unrelated sessions produced byte-identical step names andcheck --golden --fields step_nameswas inert for that format. It now prefers the tool or command name, as the codex-rollout importer already did. -
Neither stream translator carried the agent's final message into the trace output, so a capture stored
output: nulland its golden export carriedexpected_output: nullwhile an import of the same session carried the text. -
Neither transcript importer set
ended_at, so every imported session showed no duration forever despite each record carrying a timestamp. -
Values interpolated into single-line rows — the
checkgate's pass/regress lines and its divergence values,watch's header,ingest's insert error — used the renderer's escaper, which preserves newline by design. A newline there lets a producer (or a downloaded golden file) forge an extra✔ … passrow into a CI verdict. Those sites use the one-line escaper now. -
The terminal-status synonym table was a plain object literal, so a lookup resolved inherited keys:
status: "constructor"returned Object's constructor — a function assigned to a field typed as a string, its native-code source echoed into the operator's warning, and the repair marker left unset soruntreated it as the child's declaration. The table has a null prototype and the result is type-checked. -
Diagnostics that quote a producer's value now escape newline and tab as well. The renderer preserves both on purpose (a multi-line error keeps its shape), but a one-line warning that carries a raw newline lets a producer forge a second line reading exactly like this tool's own output, in the supervisor's terminal and CI log. Two escapers now: the lenient one for rendering, a stricter one for messages — which also restores tab/newline escaping the line preview lost when the two were unified.
-
skipped: unsupported protocol version …still echoed a producer's raw ESC and C1; it was missed by the sweep that fixed its siblings. -
An OTLP batch carrying a first-time root was skipped entirely when all its child spans were duplicates, so a rootless synthetic trace was never upgraded and the root's own tokens were lost — the redelivery guard swallowing a genuine first delivery. Token totals across batches are now pinned by a test matrix over every combination of (root present/absent) x (already stored/new) x (synthetic target/real) x (new child spans/all duplicates), asserting the trace's step count and synthetic flag alongside the total — an earlier version checked only the total and so still passed with the synthetic upgrade disabled outright.
-
A child that declared failure and exited 0 was stored as a success. Repairing every unrecognized terminal status to
failedand then letting the wrapper's exit code override it launderedstatus: "error"back intocompleted— reopening the fail-open the repair was written to close, on the common shape of an agent that reports failure in-band while the process exits cleanly. A status is now read before it is repaired: recognizable spellings (error,aborted,cancelled,Failed,ok,done,timed_out, …) are folded onto the four stored statuses and treated as the producer's declaration, and only a value that maps to nothing is a repair the exit code gets to decide. -
The OTLP redelivery recompute was gated on whether the surviving steps carried token attribution, which is not the same question. When the redelivered span was the one carrying the tokens and the new span was a tool call — the most ordinary mixed batch — the survivors had none, the recompute was skipped, and the batch-wide totals were merged again. It is now gated on which endpoint the batch came from.
-
A tool result or model output could address the operator's terminal.
JSON.stringifyescapes C0 controls but not C1 (U+0080-U+009F), and xterm/VTE/iTerm2 decode U+009B as CSI — so a step'sinput/outputre-coloured the terminal fromshow,show --treeandreplay, as didshow --snapshots' environment and tool_state (keys as well as values), the AI eval panel's token counts, its JSON fallback and its box title,watch's trace id, and a policy's match pattern inguard list. Escaping is applied at the shared stringify helper rather than per call site. Verified by writing a hostile trace straight into SQLite and scanning every display command's output byte-wise: zero raw control sequences. (decisionswas already escaped; the change there is that a malformed option renders as JSON instead of[object Object].) -
The same class in the messages that quote a producer's own bytes BACK — an unknown event type, an invalid
step_typeorstatus, an unparsable line, and the notehookprints for every tool call, where the tool name is chosen by the model. Those are written to a supervisor's terminal and CI log, and they carried raw ESC as well as C1: the protocol's line preview escaped only C0 and DEL while the renderer had been widened to C1, so two guards for one concept disagreed about what a control character is. They now share one definition (escapeControlChars), whichsafeTextdelegates to. -
The dashboard rendered a different agent name than is stored: its widgets run with blessed markup enabled, so
{red-fg}in a name was consumed as formatting. Cells are escaped for blessed as well as for the terminal. -
show/listandstatsdisagreed about the same trace's duration — 2h against 5.0s — because SQLite reads a timestamp with no timezone as UTC while JavaScript reads it as local. Both forms occur in real stores. One shared parser now reads them the way the SQL side does; the same split madelistprint "in the future" for a past run. -
The dashboard's score sparkline rounded, so 0.695 read as 70% there and 69.5% in
show/evalfor the same stored value — exactly what the shared score formatter exists to prevent. -
A run that exited 0 could be recorded as failed with no error text: the wrapper treated a terminal status the protocol had REPAIRED as the child declaring its own outcome, which suppressed the exit-code finalization. A repaired value is now distinguished from a declared one, so the wrapper's exit code decides.
-
runCustomRubricstill divided by zero for an all-zero-weight rubric, reporting "0% FAIL" beside "All criteria passed" — the guard had been added to the CLI only, which was the very drift the entry above it describes. -
A failure could be recorded against the tool call that succeeded. The hook adapter closed the most recently opened tool step, but harnesses dispatch tools in parallel batches whose results return in call order — so with two calls in flight the first result closed the second step. Outputs were swapped, and a
PostToolUseFailurelanded on the call that had actually succeeded while the one that failed was stored clean: a fabricated failure and a fail-open at once, on the primary capture path. Pairing is oldest-first now, matching the stream translator. -
A trace finalized with a terminal status the schema does not recognize was stored as completed.
endTrace({ status: 'Failed' })— a case difference — andaborted,cancelledorTimeoutall became success, and the deterministic evaluators readstatus, so a run the caller explicitly declared failed scored 1.0 PASS and exited 0. An unreadable terminal status now coerces tofailed. The two capture paths answer differently on purpose: a stream repairs the field and keeps the rest of the finalization (an unusable field must not cost a producer its output, tokens and ended_at) with a warning naming the value, while the SDK throws, because a caller writingendTrace({status: 'Failed'})wants to hear that the case did not match. A missing status still defaults to completed — that is a clean stream ending normally, not a value that could not be read. -
runCustomRubric, a public export, had no lower bound on a criterion weight, so a caller passingweight: -1alongside a positive one drove the score above 1 — a rubric stored and displayed as 200% PASSED. Weights are clamped and the score is bounded on both sides. -
A rubric whose criteria all weighed 0 divided by zero and reported "0% FAIL" at exit 1 directly beside "All criteria passed" — a false CI regression whose own report contradicted it. Rejected at parse time, along with duplicate criterion names (which collapse in the stored details).
-
A malformed YAML rubric blamed a missing package for every parse error, discarding the line and column the author needed; an empty rubric file leaked a raw TypeError.
-
ingestTracestored data the CLI refuses: negative token counts, astep_numberof 0, non-string tags and a numericstarted_at— the last stringified into the column, so every--sincewindow and every ordering by parsed instant answered about a time the run never had. It now runs the same validationingestdoes, so the two doors agree by construction. -
The OTLP redelivery guard deduped steps but not the numbers: the merge still received the batch-wide totals, so a retry re-added the tokens and cost of spans it had just dropped, and a root-only retry (the common final flush) skipped the guard entirely. On the log path a record carries no span id, so there is no equivalent identity: a key built from timestamp, step type, name and a batch-local ordinal was tried and reverted — the ordinal resets per batch, so a genuinely different failing call at the same timestamp was silently dropped as a duplicate, and that path's token carriers produce no step at all. Redelivered log batches can still duplicate; the limitation is documented rather than traded for lost data.
-
The programmatic API answered for its own arguments in SQLite's voice: an invalid trace
statusor stepstep_typereached the database raw, so a caller got "CHECK constraint failed: status IN (...)" — a constraint name rather than the value they passed, with no field or step context — while the CLI rejects the same values with precise field paths. Both are now named errors. A decisionconfidenceoutside [0, 1] was likewise stored verbatim from an SDK call, thoughingestandrecordshare one rule refusing it, soshow/whyrendered a value outside its documented range and the trace failed its own re-ingest; it is now dropped, as an out-of-rangedecided_byalready was. -
The
--fields"nothing to compare" guard scanned every entry in the golden file rather than the entries a candidate actually matched, so an unrelated agent's baseline could make a field look exercisable and restore the false green the guard exists to prevent. -
A
gemini-streamresult was paired with the most recently opened tool call, but harnesses dispatch tools in parallel batches whose results return in call order — so with two calls open, each result landed on the other's step: both outputs swapped, the call that succeeded marked failed, and the call that failed stored clean. Pairing is oldest-first now, and a result naming a tool that no open call matches is left unpaired rather than moving a failure onto an unrelated step. -
codex-execstored a non-objectitemas a bare JSON scalar in theoutputcolumn, where every reader expects an object — thegemini-streambranch already wrapped one. -
A duplicate policy name reported the raw SQLite constraint text, naming a column instead of what to do about it.
-
fork --modify-input/--modify-contextaccepted any JSON value, so--modify-input 5stored a scalar as the trace input. An object is now required; an explicitnullremains a no-op. -
Opening a store re-widened a directory an operator had deliberately locked down: the
chmod 0700was described as a floor but acted as a set. It now only tightens permissions broader than 0700. -
A working directory the user cannot write produced a raw
EACCES … mkdirinstead of the actionable message every other open failure gets. -
The OTel span receiver dropped the cost a span reports, while the log receiver read the identical attribute — so
statsshowed no cost andlist --sort costwas inert for every span-captured trace. It also ignored a reportedtotal_tokenswhen the input/output split was absent. -
An OTLP batch an exporter redelivered had its child spans appended again, permanently doubling the trace's steps and token total. Only the identity root was guarded; every span in the batch is now checked.
-
export --format goldenbaked forks into the baseline, which then let a real run that crashed part way reproduce the fork's shorter shape and pass. A golden dataset is a set of known-good runs; ajson/jsonlexport is a backup and still carries them. -
Codex tool-failure detection never fired on a real rollout. It tested for a plain-object output and returned early — but measured across 60 recent sessions, 636 outputs are arrays of
{type, text}parts and 109 are strings, and none is an object. It now flattens to text first and reads the exec tool's own leading status line, with the structured rules applied to text that parses as JSON. Anexit_codeprinted inside the output is deliberately not read: a "Script completed" run routinely embeds an inner command's non-zero code, and inventing a failed tool call is the expensive direction there. Verified on a real 382-call session: exactly its 4 genuine failures, no false positives. -
import --replacecould delete the wrong trace. Import identity was the session id plus format, and a Claude Code subagent sidecar carries the same session id as its parent transcript — so importing a sidecar reported "already imported" and dropped it, and--replacedeleted the parent session's trace, steps and evals included. The source filename is now part of the identity. -
The
--fields"nothing to compare" refusal could hide a real regression. It was derived from comparisons actually performed, and those loops run over min(golden steps, candidate steps) — so a candidate that crashed to zero steps marked every per-step field uncompared, and the gate reported "gate broken" (exit 2) for the most severe regression it could see. Exercisability is now read from the baseline alone, and a run with any failure always reports the failure. -
A
gemini-streamtool_resultwas discarded whenever its id was missing, unknown, or arrived before itstool_use— and that branch accepts atool_usewith no id, so an id-less stream lost every result. The step stayed open with no output and noerror, storing a run whose every tool call failed as clean. Unmatched results now pair with the open tool step the way the hook adapter's lookup does. -
gemini-streamignored token usage entirely, so every capture in that format reported no tokens while the identical field worked forcodex-exec. -
An OTel span with no
namewas stored as a step named"", which this tool's owningestrefuses — so an OTel-captured trace could not be restored from its own export. It now falls back to the operation name. -
Events produced by the stream translators bypassed the validation every other live-capture route performs, making them the one entry point with nothing between a vendor's payload and the store.
-
An ambiguous trace id now exits 2 (a usage error, like an unknown flag) rather than 1, and is answered as JSON under
--jsoninstead of escaping as a bare stderr line — it was breaking the very contract the same release documents. -
config test-ainow reports a dropped config key, which the previous entry claimed it already did. -
--jsonrefusals fromlist,stats,show,why,decisionsanddiffwrote a bare line to stderr and left stdout empty, so a| jqpipeline got a parse error exactly where it expected a verdict it could read. All six now answer{"ok": false, "error": ...}on stdout, likeevalandcheckalready did — from one shared helper rather than an eighth copy. That now includes a store that cannot be opened at all (corrupt, unreadable, or written by a newer build), which is opened before each command's own refusal path and so was still escaping as a bare stderr line at exit 1. -
check --jsonemitted a singularhintstring where every other command emits ahintsarray, socheck --json | jq -r '.hints[]'— the CI pipeline the command exists for — silently yielded nothing on the refusal path. It kept its own copy of the refusal helper; it now uses the shared one. -
diff --ai --jsonsilently dropped--ai: the JSON output returned before the AI block ran, so there was no analysis in the payload, nothing on stderr, and exit 0 — while the same misconfiguration exits 1 interactively. A pipeline reading.ai_analysisgotnullforever. The analysis now runs first and is merged into the payload. -
The only spend cap on paid AI evaluation failed open.
config setvalidates every key and nothing validated them on read, so a hand-edited or copied config could hold a non-numeric or negativeai.max_tokens. That value reached the cost estimate, making itNaN— andNaN > maxCostis false, soeval --ai --max-cost 0, the strictest possible budget, ran the whole evaluation and billed for it. The value was forwarded to the provider asmax_tokensbesides. Unusableaivalues are now dropped on read (the field falls back to its default, as a missing key already does), the gate refuses a non-finite estimate rather than passing it, andconfig list/config test-ainame every dropped key so an ignored value is not silent. -
eval --max-costwas validated only on a run that reached the provider, so a CI job whose budget flag was a typo'd or empty shell variable passed silently until the first run that happened to enable--ai— the run where the cap was already load-bearing. It is now a usage error on every run. -
A typo in
ai.providerreported "No AI provider configured" and advised setting the very environment variable that was already set and would have worked. An unrecognized provider now falls back to auto-detection. -
An ambiguous trace-id prefix silently resolved to whichever id sorted first.
show,why,decisions,replayandwatchanswered about a trace the user had not named, andfork— which writes — derived a new trace from one:fork trc_ --from-step 1was enough to fork an arbitrary trace out of a whole store at exit 0. Deterministic ordering made that stable, not correct. An ambiguous prefix is now an error naming the candidates; an exact id still wins over a longer id it prefixes. -
runprinted a trace id no other command could resolve. It used the prefix-stripping short form, while every consumer matches a prefix from the start of the id — so the wrapper's only pointer to the run it had just recorded matched nothing, on the one command with no other way to learn the id at the moment it finishes. It now prints the same 12 characterslistandforkdo. -
Opening a store written by a NEWER build silently read and wrote it. Schema upgrades are one-way with no down-migration, so an older binary — an old install on a PATH, a pinned CI image — was reading columns it does not know about and writing rows that do not satisfy the newer shape, at exit 0. It is now refused with both versions named.
-
config getwrote its "(not set)" message to stdout, the value channel, soKEY=$(agent-replay config get ai.api_keys.anthropic)captured a 34-character human sentence instead of the empty string — a[ -n "$KEY" ]guard passed and the sentence was sent onward as if it were a key. It also answered "(not set)" at exit 0 for a key that does not exist, making a typo undetectable, whileconfig setrefuses the same key at exit 2. The message now goes to stderr and an unknown key is refused, from one shared key list. -
The Codex importer dropped roughly nine tenths of what the agent did. It handled only
function_call, while the current Codex CLI emits most tool invocations as the freeformcustom_tool_call— measured across 40 recent rollouts, 194 custom against 25 function. Every one of them was counted into "Records skipped" and stored nowhere, at exit 0, so replay, diff and eval analysed a session that appeared to have done almost nothing. Both families now run through one branch, paired bycall_idas before. -
The Codex importer never set
total_tokens, so imported sessions showed "-" forever and a store mixing captured and imported runs reported totals that omitted the imported ones. It now reads the session'stoken_countrecords. Those totals are cumulative, so the last one is the answer: summing them over-counted by 34x on a real session (214,648,081 against an actual 6,267,854). -
The Codex importer never recorded a step
error, so an imported trace read as a clean run tohallucination-check,completeness-checkandcheck --golden'sstep_errorsbaseline — a fail-open on exactly the traces this tool exists to audit. A non-zero exit code or an explicit failure in the paired output is now recorded, as the Claude importer already did. -
Records wrapped in
event_msgwere matched against the literal string"event_msg"and skipped. It is the more common wrapper in real rollouts (7,471 records against 4,351 forresponse_itemacross 60 sessions), and the session token totals live inside it. -
Claude token totals were short by a factor of ~478. Only
input_tokens + output_tokenswere summed, dropping both cache fields, which is where nearly all of a real session's consumption sits: on a 52 MB transcript the stored figure was 1,216,025 against an actual 581,945,188, and the billable-but-uncached 4.3Mcache_creationwent with it.stats, the dashboard totals and every budget-shaped reading were meaningless for imported traces. The main loop and its subagent twin now share one helper. -
Every user turn after the first was discarded by both importers, so a 59-turn session imported with one question and the rest unrecoverable — while the two other paths that assemble a trace from turns (the batch merge and the OTLP mapper) both keep them in
metadata.follow_up_prompts. The importers now follow that convention. -
The one turn that was kept was usually not a prompt. Real transcripts open with a harness envelope — a slash-command block, injected instructions, an environment preamble — so
trace.input.prompt, whichwhy, the summarizer, the rubric evals andcheckall read as "what was asked", held boilerplate. The prompt is now the first turn that is not an envelope, with the envelope still used when that is all the session has. -
Importing the same session twice created a whole duplicate trace. Nothing checked, so a re-run after a crash — or a scheduled loop over a session directory — silently doubled every store-wide number and left indistinguishable rows in
listwith no way to tell the copies apart or clean them up. A session already in the store is now reported and left alone, with--replaceto re-import it (also how a transcript that has grown is refreshed). -
The golden CI gate could report a green pass having compared nothing. Every field comparison skips a step whose baseline side lacks the data it reads — correct per step, but when every step was skipped the field compared nothing and the run still exited 0.
check --fields modelagainst a baseline captured without per-step models (every hook-captured or ingested store) was an unconditional pass, and it is the flag the README recommends for catching model swaps. A field named on--fieldsthat no baseline can exercise is now a gate-broken refusal (exit 2), naming the field. Unknown field names were already refused for this reason; a valid field with no data behind it reached the same false green by a subtler route. The default field set is exempt: it deliberately spans fields not every trace shape has. -
A golden entry with no
metadatasilently disabled thestatuscomparison — the one field that catches "this run now fails" — and reported a green pass. The baseline validator checkedsteps_summarybut notmetadata, andmetadatais the block a human is most likely to prune when hand-editing or merging a baseline for review.checknow refuses an entry without a stringmetadata.status, whichexport --format goldenwrites without exception. -
statsexcluded forks from its headline totals but not from the by-status or per-agent breakdowns, so the parts did not sum to the whole: "Traces: 5" printed directly above a by-status summing to 6, andstats --json | jq .by_status.runningalerted on afork— a debugging action, not a run. Each fork also inflated its agent's count by one. -
A decision's
confidencewas stored by live capture at any value whileingestrefuses anything outside [0, 1], sorecordwrote traces that failed their own re-ingest — the same drift the option-shape rule was unified to prevent, one field over. Both paths now share one exported check. -
eval --preset ai-root-causereported a clean 100% pass for a run that never finished. Its applicability test read the trace error and the steps but not the trace status, andrecordfinalizes an abandoned stream astimeoutwith no error text and no failing step — so the preset was "not applicable", which stores score 1.0 and passed, without ever calling the provider. The deterministic criteria already read status; this was the last reader that did not. The cost estimator now sees status too, so it and the run agree about which presets will actually run. -
A wrong-typed score from the model was read as full marks:
Number(["10"])is 10 andNumber(true)is 1, so a mis-shaped reply passed. Anything that is not a finite number now scores 0. -
eval --aiunder-reported spend when a request timed out and was retried. A provider that finishes generating and answers after the deadline still bills for it, so the retry is a second charge, but the reported cost came from the final attempt alone — and that number feeds the running total and the--max-costgate. Timed-out attempts are now counted; a 429 or 5xx is not, since nothing was generated to bill for. -
The
--max-costestimate allowed a flat ~200 tokens for everything around the trace summary, which predated the injection guard now appended to every AI prompt. Measured, the prompts ran up to 44% over that on a small trace. -
The SDK could store what
ingestrefuses.TraceRecorderbuilt events and calledapplyEventdirectly, sovalidateEvent— where the live path's rules live — never saw a programmatic event: an out-of-range decision confidence, bare-string options, an emptychosen, an empty step name and non-string tags all round-tripped into a trace that failed its own re-ingest. Every SDK emission is validated now, and a rejection throws rather than warning, because an SDK call is this process's own code rather than a foreign producer. -
statsand the dashboard counted forks. A fork is a never-executed copy of a step prefix, tokens and all, so oneforkof a 2-step 3,000-token trace doubled the store'sstepsandtotalTokens— reporting spend that never happened. Every other fork-aware surface already filtered on lineage. -
ingestnow says when it drops fork lineage.exportwritesparent_trace_id,ingesthas nowhere to put it, and a restored fork becomes an ordinary trace — which the golden gate andwatchthen treat as a real run. Rebuilding the link needs an in-file id remap and is left alone; going quiet about it is not. -
Arrow-key navigation in the dashboard's trace list survives a refresh again. The list widget resets its selection on every
setData, unlike the one it replaced, so the cursor jumped back to the top row on each auto-refresh. -
An AI score sent as a JSON-quoted number (
"9") scored 0 and failed. Guarding against["9"]andtruehad also rejected the single most common way a model mis-sends a number, silently failing a good reply. -
A tool result arriving after the turn ended was discarded, and left a phantom live run behind. Every hook fires as its own process, and a closing event (
PostToolUse,PostToolUseFailure,SubagentStop) went through the same find-or-create path as an opening one — so when the turn-endingStopcommitted first, the closing event found no open trace and created one. The tool's output,ended_atand duration were dropped permanently, the real step stayed open forever, and the store gained an emptyrunningtrace thatlist,watchand the dashboard all render as a live run. It happens deterministically whenever the harness dispatchesStopbefore the result arrives, and in 47% of simultaneous spawns (measured, 14 of 30), rising further while anotel serveholds the write lock. Closing events now resolve the session's trace whatever its status and never create one, so the result is recorded on the finalized trace rather than lost —updateStepnever required a running trace; onlyappendStepdoes, and a closing event never appends. Measured again after the fix: 0 of 30. -
A decision whose
optionswere not option objects crasheddecisions. Thechosenfield was validated and the options array was not, so a plain array of strings — the most obvious wrong guess at this schema — was accepted byrecordand then aborted the command with a bareTypeError, losing every LATER decision point in the trace and naming neither the field nor the step. Options are validated at the boundary now — by the same exported functioningestuses, not a second copy of the rule, sorecordcannot store an options arrayingestrefuses and leave a trace unrestorable from its own export. A record stored before that renders instead of aborting. -
A trace id of
''was stored rather than replaced. An empty string is not nullish, so it slipped past?? generateId, and because every later event requires a non-emptytrace_idthat trace was unreachable forever — finalizedtimeout, counted bylistand bycheck's candidate scan, openable by nothing. The id is now required to be an identifier, not merely free of control characters. -
A Gemini stream's unreadable exit code fabricated a run failure.
Number()of an unparseable value isNaN, which is!== 0, so a non-numeric code — a Node-stylecode: "ENOENT", or an object — marked the whole run failed and reported the reason as the literal "exited with code NaN". A code that cannot be read is not evidence the run failed. The Codex path was already guarded this way; the two had drifted apart again. -
check's new refusal advised naming a trace with--trace— but--tracecompares whatever it names, so following that advice pointed the gate at the very fork or in-flight run the refusal had just excluded, turning it red on a run that never executed. -
A capture stream's
error: NaNproduced a failing step whose reported reason was the word "null" (JSON.stringify(NaN)), and a numericis_error: 1— what an exporter that coerces booleans to ints sends — was not read as a failure at all. The flag is now read generously, since missing a failure signal is the fail-open direction, while a non-finite number is never an error code. -
record --format codex-execrecorded a failed item with the wrong reason — including its own SUCCESS status. Detection triggered on any of three signals but picked its message from an unrelated fallback chain, so an item failing by exit code was stored with the error textcompleted, which is whatshow,watch,whyand the AI root-cause prompt then displayed as the failure; an item flagged only byis_errorproduced the literalexited with code undefined. A stringified exit code and a capitalizedFailednow count too, matching the tolerance already applied tois_error. -
An
errorfield holding an empty array or object still fabricated a failing step — the same class aserror: ""anderror: false, via a different empty value that a producer with a structured error field plausibly sends. -
checkreported "No traces matched" when traces DID match and were then excluded as forks or still-running, sending the reader to widen--agentand--since— advice that cannot help. The two cases now read differently. -
A fork turned
check --goldenpermanently red. A fork is a never-executed copy — same agent name and input, a truncated step prefix, statusrunning— so it matched its own baseline and diverged on step count and status, reported as REGRESSED at exit 1, the code reserved for a real regression. Oneforkon a shared store failed every later gate run, indistinguishably from a genuine failure. Candidates now exclude forks by lineage, as the hook, OTel andwatchlookups already did, and also excluderunningtraces, whose partial shape is not a regression. -
watchannounced a failed run without saying why. The earlier fix covered a failure a STEP recorded, but the two most common failure paths write a trace-level error and no step error at all —runfinalizing a non-zero child exit, and atrace_endevent carryingerror— so the one view open when a run died showed only "FAILED" whileshowprinted the reason. -
listhad its own copy of the relative-time formatter, drifted from the shared one: no month bucket ("45d ago" where the dashboard said "1mo ago") and no future guard, so a skewed future timestamp read as "just now" while sorting to the top. -
Exporting a whole store was quadratic.
getTraceresolves an id prefix withid = ? OR id LIKE ?, and that disjunction cannot use the primary key index, so every lookup was a full scan ofagent_tracesplus a temp B-tree for the ordering.exportcalls it once per trace — with an already-canonical id, so the prefix machinery was pure waste — and applies no limit, so the cost grew with the square of the store: a 3,000-trace export took 10.4 s, and larger stores far worse. It now tries the exact id first and only falls back to the prefix query, which resolves identically. The same export now takes 1.1 s and scales linearly. This is the same class of defect the schema v4 expression index exists to fix, on the path that builds golden datasets and backups. -
guard listprinted a truncated policy id thatguard disable,guard enableandguard removethen rejected as "not found" — they resolve by exact id or name, with no prefix matching, so copying the id out of the table was guaranteed to fail. The full id is shown. -
eval --jsonreportedcriteria[].criticalfor only some critical criteria. The flag was set when a check returned it (the conditionally-criticalno_error_steps) but not when the preset declared it, soall_tool_calls_completedandno_unresolved_errorsappeared infailed_criticalwith no flag on the criterion itself. Every criterion that can fail a preset on its own now says so. -
ingest --dry-rundid not mention the eval results the real run reports it cannot restore.--dry-runis the documented preview of the real run, so a preview that omits the one thing the real run warns about is the surprise it exists to prevent. -
An import that captured nothing exited 0 and stored an empty trace. The "nothing importable" guard in both importers keyed on
!input, but an empty first user record still sets the input to{prompt: ''}, which is truthy — so a file reporting "0 records imported" went on to create a trace with an empty prompt and no steps, andimport X && use-traceproceeded against content-free data. A file that captured a real prompt but no steps still imports, as before. -
The
codex-rolloutimporter discarded the real first prompt when the record before it was empty.{prompt: ''}is truthy, so!inputread an empty first user message as "input captured", the next real prompt fell through to the follow-up branch, and the trace kept no question at all — while the empty record still counted as imported. The identical construct was fixed in both branches of the Claude transcript importer; this sibling was missed. -
guard add --priority ""silently stored the default instead of refusing.Number('')is 0, which is an integer, so the blank string was the one input this check accepted where every sibling numeric option rejects it — so--priority "$UNSET_VAR"ranked a policy 0 rather than failing. -
ingestnow reports eval results it cannot restore.export --with-evalswrites anevalsarray thatingesthas no field for, so on the documented backup/restore path — for data the user explicitly opted in to keeping — the evals were dropped with an exit 0 and no mention. Restoring them is a schema change; saying they were dropped is not, and the traces themselves round-trip faithfully. -
eval --jsonnow answers in JSON when a rubric or a deterministic preset throws while running. Both paths returned before the empty-results JSON fallback, so a| jqpipeline got a parse error instead of a verdict. (No new test: reaching either catch requires an internal store failure rather than any user input.) -
A root span arriving in a later OTLP batch was dropped entirely. The first root span becomes the trace, so it is deliberately not among the batch's steps — right for the batch that opens the trace, wrong for every later one. A span exporter flushes inner spans first, so a trace with more than one root (GenAI emits
create_agentbeforeinvoke_agent; multi-agent runs nestinvoke_agent) naturally splits with a root in a later batch, which then promoted itself to an identity the trace already had. Merging inserts only steps, so that span produced no row at all. Whether a span survives no longer depends on where the exporter cut its batches. A rootless synthetic trace still adopts a late root as its identity rather than duplicating it as a step. -
The OTLP logs path stored counters that
ingestrejects.intValueis a signed int64, so a negative token count orduration_msis wire-legal; the span path floors both, and this path — the one documented for Claude Code and Gemini CLI — did not. A negative count draggedstatssums negative and broke export →ingestof a trace this tool had just written. A genuine zero duration still survives. -
Merging a later OTLP batch could write a negative
total_duration_ms. The trace start and end come from independent sets (earliest start, latest end), so nothing orders them; a trace whose first batch carried no renderable timestamps takes the ingest wall clock as its start, and a later batch contributing only an end in the past inverted the window. The mapper already guards its own window this way; the merge now does too. -
The Gemini stream's new failure detection read
error != null, so a producer that always emits the key —error: ""orerror: falseon success — got fabricated failing steps, which feedcheck --goldenstep_errorsand the eval error criteria and fail a clean run. An error must now be a non-empty value, and a stringifiedis_error: "true"counts, the way the OTel log mapper already reads its own signals. -
record --format codex-execrecorded a failed item as a clean step — the same gap in the same file as the Gemini one, so the fix now covers both streams. -
A redelivered OTLP batch added the trace's own identity root back as a step, so the trace contained a step that was itself. An exporter retries any batch it did not get a 200 for; a span already present as the trace's identity, or as a step, is never added again.
-
The OTLP log mapper let a whitespace-only first prompt claim the input slot and demote the real question to a follow-up — the third site of a defect already fixed in both importers and in the cross-batch path.
-
A failed tool call in a Gemini stream (
record --format gemini-stream) was recorded as a clean one. The gemini translator'stool_resultbranch had no error path at all, while every sibling capture path (hook, the Claude transcript importer) already keptstep_type: tool_calland populatederror. Nothing downstream could then see the failure:eval --preset ai-root-causetreats a trace with no failing step as not-applicable and scores it a 100% PASS, and acheck --goldenstep_errorsbaseline had no failure to regress against — so a run whose every tool call errored reported green twice over. Only unambiguous, shape-generic signals are read (is_error: true, a non-nullerror); the result content is still preserved as the step output either way. -
checkechoed producer text raw — the one human-readable renderer left withoutsafeText. Agent names and divergence values are agent-authored, and on the golden side they arrive from a baseline file that may have been shared or downloaded. A lone carriage return in one returns the cursor to column 0 and lets following bytes overwrite theREGRESSEDline above it, so the regression gate could be made to misreport its own verdict; an OSC sequence retitles the operator's terminal. -
A run wrapped by
agent-replay runcould be recorded as clean when it failed, or left open forever.trace_end.statusis a free string, and any non-empty one counted as "the child owns the outcome" — butupdateTracecoerces anything unrecognized tocompleted, so a child ending withstatus: "error"was laundered into a clean-looking trace AND suppressed the exit-code finalization, leaving no error text on a run that exited non-zero. A golden baseline recorded that way then matched it.status: "running"was worse: it survives coercion, so the trace stayed open forever and a barewatchlive-tailed a dead process. Only a terminal status the store can record now counts. -
A trace's
started_at/ended_atwere coerced by the writer and validated by nobody, so a non-string timestamp was silently replaced by the INGEST WALL CLOCK at exit 0 — with--dry-runreporting the file valid. Every--sincewindow and every ordering then answered about a time the run never had. -
completeness-checkcould not fail a run in which no tool call completed:all_tool_calls_completedhas the same 0.3-weight-against-0.7-threshold shape as the error criteria, so a score of 0 landed on exactly the threshold and passed, with "0/2 tool calls have output" printed beside the green verdict. -
hallucination-checkhard-failed any trace containing an error step, including a completed run that retried a timed-out tool call successfully — so an imported session with one failed shell command failed outright whilecompleteness-checkcalled the same trace 100% complete. Criticality now keys on how the RUN ended (its status or a trace-level error), not on any step error. -
An answer of
0orfalsewas not counted as an answer, so a count query that legitimately returns zero, or a predicate step returning false, failedcompleteness-checkat exit 1 on a correct run. -
An empty or whitespace-only first user record still discarded the next, REAL prompt for ARRAY-content records — the shape real Claude Code transcripts use. The earlier fix reached only the string-content branch, and the two then disagreed on the tally for the identical situation. The subagent path likewise never got the orphan-
tool_resultpairing check its own comment claims to mirror. -
An OpenTelemetry root span that arrived in a later batch was silently dropped — its name, prompt, timing and attributes discarded and the trace attributed to a sub-agent — because content was adopted only when the existing trace was flagged synthetic. That is the normal
BatchSpanProcessororder, since a sub-agent span ends before its parent. The same gap dropped a log session's prompt outright when the batch that opened the trace carried none (a receiver started mid-session, a resumed session, an out-of-order flush), and discarded the root's own metadata (provider, model, span id). -
A negative usage counter from OTLP was stored verbatim. protobuf
int64is signed, so it is wire-legal; it draggedstatssums negative and broke the export →ingestround trip, which requires a non-negative total. -
guard add --priorityparsed instead of validating, so--priority highsilently stored 0 and--priority 1e3stored 1 — and priority is what orders policy evaluation and breaks ties, so a rule meant to rank first ranked last andguard checkcited the wrong policy. -
Six more render sites echoed producer- or model-supplied text raw, including the
checkgate's own divergence report (where the values are exactly the attacker-influenced fields), the per-agent rows ofstats,show --snapshots,guard test, the eval table beside the panel that was already escaped, and the dashboard, where the bytes also corrupt blessed's width math for the whole layout. -
ingest --dry-runpassed files the real run could never load. A step'serrorwas the one TEXT column bound without coercion, so a structured error ({"code": …, "message": …}— a shape real producers send) validated clean and then made the insert refuse the bind, rolling back the whole trace with a message naming neither the field nor the step; it is now flattened like every sibling error column. A duplicatestep_numberalso validated clean and then hit the schema's UNIQUE constraint. And an object-valuedagent_versionor stepmodelwas silently coerced to null at exit 0 — the writer does that deliberately so one bad field can never cost a LIVE capture the whole run, on the stated assumption that ingest validates it first, which it did not. -
One unparseable line made
ingestdiscard the whole JSONL file: three valid traces beside one truncated line ingested nothing. That contradicted the policy the validation stage right below it states — load the valid subset, and exit 1 because something was dropped. Bad lines are now reported by line number and the valid records load. -
An orphan
tool_resultwas counted as an imported record although it is stored nowhere. It has notool_useto attach to — routine when a transcript is head-truncated, after/compact, or when thetool_useline itself was unparseable — so the summary reported content the store does not have. -
An EMPTY first user record made the importer discard the next, real prompt:
{prompt: ''}is truthy, so the "input already captured" check read it as captured and the trace ended up with no question at all. -
completeness-checkwas unsatisfiable for every live-captured trace. Its heaviest criterion keyed onstep_type === 'output', which the hook adapter, the OTel log path and the span mapper never emit (the Gemini stream translator and the transcript importers do), so a flawless hook capture capped at 0.6 against a 0.7 threshold andeval <id>exited 1 for every hook-captured run, clean or not. A gate that is always red gets ignored. A trace-level output, or the final step that carried one, now counts as the answer. Same defect class the error criteria were already fixed for: a criterion keyed on something the capture path never produces. -
hallucination-check's error criterion ignored a trace-level error, so a run that died before emitting a final step scored a perfect 1.0 — whilecompleteness-checksaw the same trace as failed and documented why ("the only marker a run that died before emitting a final step leaves behind"). The two now agree. -
A custom rubric matched against one corpus for both polarities, which is wrong in each direction. "Must not contain" saw only the trace input/output and step OUTPUTS, so it scored a free 1.0 for anything living in a tool-call input, a step name, a step error or the trace error — a rubric forbidding
rm -rfpassed a run that executed exactly that, exit 0. "Must contain" saw the trace INPUT, so a criterion asserting the answer cites a source was satisfied by the prompt that asked for one, and one asserting the agent apologized was satisfied by a step namedapologize_to_userwhile the answer said otherwise. Each polarity now searches what it is actually asserting about: "must not contain" sees the whole run, "must contain" sees only what the run produced. -
A rubric pattern the ReDoS filter rejects was scored 0 with its full weight rather than refused, and the table prints only the criterion name — so a valid, non-catastrophic pattern that the deliberately conservative filter declines reported a quality FAILURE of the trace, exit 1, on a correct run. Patterns are now compiled at parse time, where a bad one is a usage error naming the pattern.
-
eval --rubricandeval --max-costrefusals printed nothing on stdout under--json, the two paths that bypassed the helper added to guarantee a JSON-shaped refusal, soeval --json | jq -r .okgot a parse error. -
The shared diff windowing I extracted last commit lost its surrogate safety: a cut could split an emoji into a lone surrogate, which renders as U+FFFD in BOTH columns — the mojibake becomes the apparent difference — and the index moved the wrong way, shrinking the window so the differing tail fell out of view and both columns printed identical text again. Restored, and it now covers the model-facing summary as well as the terminal view.
-
The AI summary still dropped the failing step when EVERY step was important — a Gemini import attaches a decision record to every tool call, and a retry storm is all errors — because the prioritized fill was itself in-order. The step that ended the run is now claimed before anything else competes for the budget. Rendering is lazy again rather than rendering every step up front, so a step the budget cannot hold is no longer rendered before being discarded (20,000 tiny steps summarize in ~24 ms). The loop still visits every step: it stops early only when the budget is exactly consumed, because there is no sound lower bound on a line's length to stop on — a fixed floor is what used to drop steps that fit and report them as omitted.
-
diff --fields ""andcheck --fields ""bypassed the guard that rejects a list naming no fields —diffsilently suppressed every field comparison andchecksilently reverted to the default set. -
diffnever compared the DECISION record, the one field this tool exists to explain. Two runs that took opposite actions at the same step — one choosingrm_rf, the othersafe_path— reported "Traces are identical." and exit 0 whenever every other field matched, whiledecisionsandwhyon the same pair correctly showed the divergence, anddiff --aiwas handed a summary with no differences in it and asked why the traces diverged. The chosen option, its rationale and who decided are now compared, anddecisionis selectable via--fields. Confidence and the option list are deliberately excluded: they are the model's self-report and vary without the agent having acted differently. -
The AI trace summary dropped the failing step at the DEFAULT budget. Step prioritization existed but ran only under a tight budget; the normal path walked steps in order and stopped when the budget ran out, so on a long trace the LAST steps went first — and the failing step is almost always last. Every
eval --aipreset on a trace over roughly thirty steps therefore judged a failure it had never been shown, since the trace-levelerroris null on the normal hook-capture shape where the failure detail lives on the step. The summary also now says how many steps it dropped on every path that drops any; the marker was previously emitted only when the budget ran out mid-loop, so the prioritizing path could silently discard forty of forty-one steps. -
diff --ai's evidence could show no difference at all. Values were truncated from position 0, so two payloads sharing a long prefix — a system prompt, a message array, the ordinary shape of an agent payload — arrived at the model as byte-identical text under a heading announcing a divergence. The terminal view already windowed around the first difference; both now share one implementation. -
diff --aisent trace content to the model with no fence, whileeval --aiwrapped the same summarizer's output in an untrusted-content fence and an injection guard for exactly this reason. The summary is built from agent prompts, tool inputs and tool outputs, so a tool result reading "ignore previous instructions…" landed in instruction position and its answer was printed as the tool's verdict. -
decisionsrendered the chosen option, the option list and both rationales unescaped, so a carriage return in a decision could overwrite the line and make the one command whose job is reporting the choice DISPLAY a different option than the one stored — contradictingwhyabout the same record. -
diff --fields ,(or a script interpolating an empty variable) filtered out every field comparison while passing the unknown-field guard vacuously, so a pair with seven real differences reported three, with no scope label. It is now a usage error. -
checkpassed green when NO candidate matched the baseline. An unmatched candidate compares exactly as much as no candidate at all — nothing — yet it was a pass by default while zero candidates was already refused with exit 2. So any change that alters every match key (adding--no-inputto a hook registration blanks each trace's input, an agent rename, an input-template edit) left the gate green forever on runs it had silently stopped comparing. It now refuses like the zero-candidate case, with--allow-emptyas the same opt-out. -
checkanswered a store it could not open with a bare stderr line and exit 1, the one refusal that escaped its own contract:check --json | jq -r .okdied on a parse error, and a CI script that separates a regression (1) from a broken gate (2) misread an unopenable store as a regression. Reachable from a--dirtypo landing on a file, a read-only workspace, or a locked store. -
guard checkfailed OPEN on input it could not evaluate. Unreadable stdin, malformed JSON, a payload that is not a step object, and a missingstep_typeall exited1— not the block signal, which is2— so a wrapper gating on$? == 2ran the tool anyway. They now deny with exit2, matching the policy-evaluation failure in the same function. -
An OpenTelemetry step could carry a duration of ~56,000 years. The guard added for the trace-level window did not cover the per-step duration, which was still derived from raw nanoseconds while the timestamp formatter rejected the same value — so the step rendered a null end time beside an enormous duration, and the value is finite and non-negative, so validation stored it.
-
The native
recordprotocol still stored a negativetotal_cost_usd, whichingestrejects on the same rule as the token counts — the export → ingest round trip stayed broken through the fieldstatsandlist --sort costread. -
list,whyanddiffstill echoed agent-supplied text raw, so the escaping added forshow/replay/watchmissed the most-run command in the tool. A CRLF line break — what any Windows or PowerShell child writes — is now normalized rather than escaped mid-line, while a lone carriage return, which can overwrite what was already printed, is still escaped. -
A re-delivered OpenTelemetry log batch appended its prompt to
follow_up_promptsagain every time, so the list grew without bound on exactly the retries the receiver's 4xx-not-5xx rule exists to make safe. -
Everything a later OpenTelemetry log batch carried was dropped at the merge. A log processor flushes each turn in its own batch, so for a multi-turn session:
total_cost_usdwas missing from the merge's UPDATE, leaving only the first batch's cost (and null forever if that batch reported none), and a later turn's prompt was discarded along with itsfollow_up_prompts, so the store kept only the session's first question. Cost now sums like tokens, and later turns are retained as follow-ups. -
One record with an impossible timestamp nulled a whole log-derived session's end time and duration: the maximum was taken over raw nanoseconds and the formatter then rejected it, discarding timing every other record in the session defined. The span path had the same gap in the other direction — its duration came from raw nanoseconds while
ended_atwas guarded, so a single absurdendTimeUnixNanoproduced a ~31-million-year duration beside a null end time. -
A tool failure reported as the string
"False"— what an exporter built on the OTel Python SDK sends, sincestr(False)capitalizes — read as a clean call, dropping the error text. The check is now case-insensitive. -
Trace text is escaped before it reaches the terminal. Step names, errors, models, decision rationales and agent names are producer output — tool stderr, an HTTP error body, a sub-agent's reply — and
show,show --tree,replayandwatchechoed them raw, so an ESC sequence in any of them could recolor or clear the terminal of the operator reading the run, or set the window title via OSC. The same bytes also broke the width math behind the header panel's borders.runalready escaped a rejected event line for this reason; these were its sibling render paths. Newlines and tabs still render as formatting. -
An AI evaluator's list field that the model sent as a bare string ("issues": "too long" instead of a list) rendered as one bullet PER CHARACTER, and the malformed value was persisted in
detailsforshowandexport. Such a reply now carries a single item, and a value that is neither a list nor a string carries none. -
The native
recordprotocol stored a negative or non-finitetokens_used/duration_msverbatim, whileingestrejects those values — so a trace this tool captured could not be re-ingested from its own export, breaking the round-trip the golden gate depends on. Both importers and the stream translators already clamped. The unusable field is now dropped with a warning and the step is kept. -
An explicit
ai.providerapplied a configuredai.modelfrom another vendor's family, soprovider = openaiwith a leftovermodel = claude-*sent that name to OpenAI (a confusing auth error at eval time) and priced--max-costoff Anthropic's sheet. It now falls back to the provider's default model, as auto-detection already did, and the README's claim is true again. A model of no known family — a proxy's own name — still passes through. -
watchnever showed a step's outcome. Under the two-phase protocol (step_startthenstep_end) a step is first seen while it is still open, so printing each step exactly once meant duration, tokens and error text were always null at print time: a failing run announcedtrace finished: FAILEDwith no error, whileshowon the same trace printed it. A step that ends now gets a closing line carrying its outcome. Producers that write a complete step in one event are unchanged — no second line. -
One duration formatter across every view. Four copies had drifted apart above a minute, so
listandshowsaid "2.1m" wherewatch,replayandstatssaid "2m 5s" — and a singlereplayscreen printed both forms of the same number four lines apart. The copies also lacked the shared formatter's guards, rendering a negative stored duration as "-500ms". -
fork --tagwrote the tag after the fork's transaction had committed, so a failure on that one statement reportedFork failed(exit 1) for a fork that existed — an orphan whose id was never printed, with a fresh one created on every retry. The tag is now part of the same transaction. -
check --fieldsvalidated its field list only after opening the store and fetching every candidate, so a typo was reported as whatever the data layer complained about first ("No traces matched…") without ever naming the bad field. Usage errors are now checked first, aswatch --intervalalready did. -
demo --resetcould delete a working tree that merely looked like a store. The guard accepted any directory whose name starts withagent-replay, which a source checkout calledagent-replay-projectdoes, and then removed the tree recursively.--resetnow deletes onlytraces.dband its sidecars, so the blast radius is data this command created; a directory holding no store has nothing to reset. -
One OpenTelemetry span could blackhole a whole pipeline. The span-to-step-type tables are plain object literals keyed by untrusted telemetry — the
gen_ai.operation.nameattribute and the span name's leading word — so a span namedconstructorortoString(an auto-instrumented JS class method) resolved to a function, survived the fallback that only replaces null, and reached the SQLite bind. The batch's transaction rolled back and the receiver answered 500, which OTLP exporters retry, so the poisoned batch was resent forever and every other span in it was lost with it. -
A parallel tool batch's
PostToolUsehooks silently lost results. Each hook is its own process, and the "find the open tool step" read and the update that closes it were separate statements, so simultaneous hooks all claimed the same newest open step: the last writer won, the other outputs were discarded, and those steps stayed open forever — with no warning, because the update matches on(trace_id, step_number)and always reports a row changed. Six parallel hooks lost one to three results per run before the fix. The claim and the close are now one transaction. -
show --from-step/--to-stepreported the WINDOW's token subtotal on the trace-levelTokens:line. The header falls back to summing the steps when no producer set a trace-level total — the shape of every hook-,record-, OTel- and import-captured trace — and the window was applied by narrowing the step array in place, so a 30-token trace showedTokens: 20beside a trace-levelDuration:, disagreeing withlistandstatsfor the same trace. The steps rendered are unchanged; only the header is trace-level again. -
statsand the dashboard silently excluded a trace whose timestamps use an ISO-8601 basic-format offset (+0200— whatdate +%FT%T%zemits andingeststores verbatim) from the average duration, becausejulianday()returns NULL for that form andAVGskips NULLs. The average was therefore taken over a subset whileoverall.tracescounted every trace, and a store built entirely that way printed "Avg duration: -" for traceslistshowed a duration for. A timestamp nothing can parse still counts as unmeasured. -
watchwith no trace id, and the hook capture path's "open trace for this session" lookup, ranked candidates by the BYTES ofstarted_atrather than the parsed instant. SQLite's own space form sorts below everyT-separated timestamp and a negative offset sorts above the UTC instant it precedes, so a barewatchattached to an older run and showed a live session doing nothing, and a hook event could append to the wrong open trace. These were the last two raw-TEXT orderings onstarted_at. -
The dashboard TUI rendered total cost with a flat
toFixed(4), so a store whose entire real spend was under a hundredth of a cent — the normal case for agent runs — read as$0.0000. It now uses the sameformatCostUsdstatsuses. -
run's summary says when events could not be stored. A child recording several sub-traces through one channel collides on the per-trace step numbering and loses everything after the first, but the count lived only in stderr lines while the summary still read "N event(s) recorded". -
diffcompares the trace's owninput. It never did, so the one fieldfork --modify-inputchanges was invisible — andforkcloses by telling you to run exactly that diff. Two separately-ingested traces differing only in their prompt compared as "identical", whilediff --ai(whose summary does include the input) could reach the opposite conclusion about the same pair. -
diff --fieldsno longer prints "Traces are identical." over a filtered comparison — under a header showing COMPLETED beside FAILED, in the case that prompted this. It now names the fields it actually compared. -
The
difftable windows each value around the first difference instead of truncating both sides from the left. Agent payloads share long prefixes ({"file_path":"/Users/…"}), so a real difference routinely rendered as two byte-identical cells under "1 difference(s) found", with nothing to suggest--jsonwas needed to see it. A left-only step is also marked-rather than a+that read backwards next to the green+ Right only. -
why's header counts steps rather than calling them "hops" — a one-step chain involves no traversal at all. -
An empty
AGENT_REPLAY_DIRis treated as unset rather than resolving to the working directory.AGENT_REPLAY_DIR= agent-replay initwrote the store loose into the CWD — anddemo --resetthen passed its "is this an agent-replay data directory?" name check for anyone standing in a checkout namedagent-replay, and deleted their working tree. -
recordfinalizes a trace it resumed by id, as documented, while leaving alone the one trace a live enclosingagent-replay runhanded it — a run removes its channel as it finalizes, so a staleAGENT_REPLAY_TRACE_IDinherited from a finished run no longer strands a resumed tracerunning. -
The
difftable never cuts a value between the halves of a surrogate pair (an emoji in a prompt is enough): a lone surrogate rendered as U+FFFD in both columns, so the mojibake looked like the difference.diff --compactalso names the--fieldsfilter instead of reporting a bare "Differences: 0". -
A non-string
ai.modelin a hand-edited config is ignored rather than passed to a provider adapter, where it surfaced aslong.startsWith is not a function— or was sent as the model name itself. Under the README's own nested example (run -- sh -c '... | agent-replay record') the events carry the wrapper's trace id, so the end-of-streamtimeoutfinalization marked a clean run red — and permanently, since the wrapper then sees a non-running status and leaves it alone. Only traces this stream opened are finalized. -
eval --jsonanswers in JSON on every refusal — a missing trace, an unknown preset, no configured provider, a--max-costrejection — instead of leaving a| jqpipeline with an empty document. -
The
--max-costpre-gate prices the ceiling the run will actually use. It hardcoded 1024 output tokens while the run began honoring a configuredai.max_tokens, leaving the check whose job is refusing to spend about 9x optimistic atmax_tokens 8192. -
In auto-detection, a configured
ai.modelnaming a known family now selects its provider. With two keys present the fixed priority order won instead: a different vendor was billed and the verdict came from a model the user had not chosen, silently. -
Importing a transcript survives a line that parses to
null. Both importers pushed any parsed JSON value into their record list and dereferenced it unguarded, so one such line aborted the entire import — nothing kept from a 50,000-record transcript. An empty or unreadable subagent file also no longer leaves a childlesssubagent:<id>step behind, which had made "nothing importable" undetectable (Records imported: 0alongside exit 0), and an unreadable subagent file is no longer counted as a skipped record. -
agent-replay runkeeps its headline guarantee when the store hiccups. Finalization was unguarded, so a write that failed (another process holding the lock pastbusy_timeout— likelier now that the receiver, hook and fork take it up front) replaced the child's own exit code with1, left the tracerunningwith noended_atforever, and leaked the channel directory. It is now best-effort: the failure is reported, the child's status is still whatrunexits with, and the temp directory always goes. A failure to open the trace at all — before the child is spawned — now says the command was never started, rather than a bare "database is locked". -
rundetects an events-channel rewrite that keeps or grows the file's size. The append-only guard only compared sizes, so a producer that reopened the channel truncating (createWriteStream's defaultwflags, orwriteFileSync) and wrote at least as much as had been consumed slipped through and reading resumed at a stale offset — events dropped, exit 0, no diagnostic, exactly what the guard exists to prevent. -
runrecords a duration for an instrumented child. A child that sends its owntrace_endowns the status but rarely sends totals, and the wrapper skipped its update entirely, so an instrumented run had a nulltotal_duration_mswhile an uninstrumented run of the same command reported one. -
A rejected event line is escaped before being echoed. It is untrusted producer output printed straight to the supervising terminal and CI log, so a child emitting ESC sequences could move the cursor or recolor the log of the tool watching it.
-
recordno longer calls a legal comment-only native stream a total failure://lines are part of the protocol, but were counted as unmatched input, so such a stream reported "none of the N line(s) matched" and exited 1. -
eval --jsonis parseable on every path. The default invocation (eval <id> --json, no evaluator flag) printed two lines of prose on stdout before the array, and a run where every evaluator threw printed nothing at all — so the same flag produced valid JSON, JSON-after-prose, or an empty document depending on which path ran. -
config set ai.modelno longer applies a model to a provider it doesn't belong to. It was used for whatever provider auto-detection found, so a config naming a Claude model on a machine holding only anOPENAI_API_KEYsent the Claude model name to OpenAI — every eval failed with an opaque server error, and--max-costpriced the run off Anthropic's rate sheet while doing it. -
config set ai.max_tokensis finally read. It was validated, stored, and then ignored: the eval path always sent a hard 1024, and a truncated judge reply fails JSON extraction and is stored as score 0 / failed — a CI failure on a good trace, billed in full, with no supported way to raise the ceiling. -
showlists a trace's evaluations newest-first even when several were written in the same millisecond (an--allrun), and the eval Details column reports criteria against the threshold the evaluation recorded rather than a hardcoded 0.7. -
list, the dashboard and every candidate fetch keep their index. Ordering by the parsed instant (julianday(started_at)) is required for correctness, but it cannot use the plainstarted_atindex — measured over 50,000 traces, the defaultlistpage went from an index seek to a full scan plus a temp B-tree (0.19 ms → 3.9 ms, and linear in store size). Schema v4 adds a matching expression index; the same page now takes 0.10 ms. -
check --jsonanswers in JSON on the paths that refuse to run (no candidates, unreadable or non-golden baseline, bad--since). They printed only stderr, socheck --json | jq -r .okgot a parse error instead of a verdict. -
checkcounts uncovered baseline entries, not the scenarios they group into: a hundred untouched entries for one agent+input reported as "1". Atool_inputsdivergence also names what replaced a missing tool call, instead of printing "golden null → got null" for a baseline captured withhook --no-input. -
hook --enforceblocks only the tool call it can't check when the store is missing; other events still capture (and create the store), so a single--enforceregistration across all hook events can still bootstrap. -
export --format goldenwarns only when writing to a file, keeping a piped stdout export clean for the tool consuming it. -
Guardrails now see the payload the harness actually sent.
hookandguard checkdecoded stdin chunk by chunk, and a pipe delivers 64 KiB at a time, so any multi-byte character straddling a boundary (emoji, CJK, accented text, a smart quote) became U+FFFD. The JSON stayed valid, so nothing reported it: a content-baseddenystopped matching the corrupted text and the tool call was allowed, and the same mangled text was stored as the audit record. -
hook --enforceno longer allows everything when it cannot find the store. The path resolves from the hook process's working directory and was created when missing, so a hook firing from any other directory silently ran against an empty policy set. It now blocks and says why — the same fail-closed posture as every other "could not evaluate" case on that path. -
A
name_regexusing a Unicode property escape (\p{Script=Han}) now works. Patterns compiled without theuflag, which degrades\pto a literalp, so such a policy validated cleanly, listed as an active deny, and matched nothing at all. Patterns that are only legal withoutustill compile. -
listand the dashboard's recent-traces table now order by the instant a trace started, not by the bytes of its timestamp.started_atis TEXT and nothing constrains the format a producer writes (SQLite's own2026-08-16 23:00:00, or a+02:00offset), so the newest trace could be shown last — and be the first row a--limitdropped — whilestats --sincecounted it as the most recent.--sincealready compared instants; the ordering was left behind. -
The AI evaluators are now shown the run's duration and token count. Both were read from the trace-level columns, which only a producer-reported total ever fills — so for every hook-,
record-, OTel- or importer-captured trace the judge was handed a run with no timing and no tokens, whilelist,showandstatsdisplayed both. The efficiency preset asks the model to weigh "cost, latency and token usage"; it was scoring what it had not been shown.diff --ailost the same durations. -
statsno longer prints$0.0000for a store with real spend under half a cent — the wideningshowalready applied is now shared by both. -
stats --jsonrenamed the per-agentfailedtally tofailed_or_timeout, which is what it has always counted. The same document reportedby_status: {failed: 0, timeout: 1}besideby_agent: [{failed: 1}], so a CI job alerting on the per-agent number fired on timeouts. The human output already said "failed or timed out". -
check --goldenno longer passes green when it had nothing to check. A run where no trace matched the filters produced "0 passed, 0 regressed" and exit0— even under--strict, which only counts candidates that were actually fetched — so a mistyped--agent, a--sincewindow that outran the recording, or a--dirtypo (which quietly creates an empty store) left the gate green forever. It now exits2, like the empty-baseline case it mirrors. Being handed a full--format jsonexport instead of a golden one is also diagnosed by name, rather than dying on "Cannot read properties of undefined". -
check --goldennow catches a baseline tool call that the candidate replaced with a different step type. Thetool_inputscomparison skipped whenever the candidate step was not atool_call, so the disappearance of a tool call — the regression that field exists to catch — was invisible under--fields tool_inputs; the default field set caught it only incidentally, viastep_types. -
Writers that share a store no longer collide. Three problems, one cause — reading before writing without holding the write lock:
otel serveanswered500 {"error":"database is locked"}and dropped a whole export batch whenever another process (ahook,run, oringest) committed while the batch was being stored. The batch transaction opened DEFERRED and read first, so upgrading to a write failed withSQLITE_BUSY_SNAPSHOT— which bypassesbusy_timeoutentirely. Measured at 1 dropped batch in 100 under light contention and 89% under load; now zero, because the transaction takes the write lock up front.forkcould die with the same bare "database is locked" and is fixed the same way.- Concurrent first events for one session opened several traces for it (the
lookup and the create were not serialized), splitting the session's steps
and leaving every trace but one
runningforever, sinceStopfinalizes only one. Reproduced at 4 sessions in 25 with real parallel hook processes; now one trace, verified over repeated runs. otel serve's "Accepted N trace(s), M step(s)" counted spans from batches that were rolled back, and counted them again when the exporter retried.
-
Forking a trace no longer hijacks the session it was forked from.
forkcopies the original'ssession_idand opens the copy asrunningwith a newer start time, and live capture resolves "the open trace for this session" as the newest running one — so every hook event after a fork was written into the what-if copy. The real run stopped growing mid-capture and was never finalized (leftrunningforever), while the fork silently accumulated steps it never ran, sodiff original forkreported fabricated divergence. A bareagent-replay watchhad the same problem: it attached to the static fork and showed nothing happening while the live run scrolled by. Live-capture resolution now skips forks (hook,watch, and the OpenTelemetry receiver's cross-batch merge target, which a fork also matched because it inherits the original's metadata). -
demono longer seeds a guardrail that cannot do what it says. Itsno-external-urlspolicy was adenykeyed onoutput_contains, which enforcement can never fire — it evaluates a proposed call, before there is any output — so the shipped example taught a kill-switch shape that silently does nothing, and the seed path bypasses the warningguard addnow gives. It is awarn(an auditing pattern) with a description that says so. Thetoken-limit-warningdescription likewise claimed a numeric threshold the match keys cannot express; it now describes the substring test it actually is. -
--sinceresolves its cutoff to a UTC instant before querying.Date.parseaccepts more formats than SQLite'sjulianday, and while an unparseable stored timestamp falls back to the old comparison, an unparseable bound made every window match nothing at exit0. ISO 8601 basic-format offsets are the reachable case —+0200, exactly whatdate +%FT%T%zemits in a shell script. Normalizing also settles the zone-less forms, which JavaScript reads as local time and SQLite as UTC; local is what a bare timestamp means, and stored timestamps are UTC.stats --jsonechoes the resolved bound. -
A cross-batch OTel trace never keeps a parent reference pointing at a later step, even when start-time ordering cannot resolve it. Span timestamps are stored to millisecond precision, so a parent and child starting in the same millisecond tie and fall back to arrival order — leaving exactly the forward reference the ordering exists to remove. Any that survive are now cleared, so the trace stays something
ingestaccepts. Renumbering is also bounded, so a very large assembled trace can't stall the receiver's write lock. -
runescalates toSIGKILLif a child ignores a forwarded signal. Handling the signal replaces Node's default terminate-on-signal, so forwarding alone traded a stuck trace row for a stuck process — a child withtrap "" TERMkept the wrapper alive indefinitely, which is worse in the CI case this serves. -
A golden entry no longer silently discards a trace's own metadata key when it collides with one of the four the gate reads (
status,total_duration_ms,total_tokens,tags). The reserved keys still win —checkcomparesmetadata.status, so letting a trace's ownstatusdisplace it would be a gate bypass — but the displaced value is now preserved beside it under atrace_metadata_prefix, instead of the baseline being a lossy record. -
An OTLP batch is now stored all-or-nothing. The receiver's upsert loop had no transaction around it, so a write failure part way through a multi-trace payload left the earlier traces committed and answered
500— and a5xxtells an OTLP exporter to retry the same batch. On redelivery those committed traces were found and the same spans merged into them again: steps duplicated, tokens doubled, permanently, because duplicate deliveries are deliberately not de-duplicated. Both/v1/tracesand/v1/logs. -
run's summary line now reports the status the trace was actually stored with. It was derived from the exit code alone, so a child that declarestrace_end {status: completed}and then exits non-zero — a crash during shutdown, after the agent's work succeeded — was announced as "failed" while the database recordedcompleted. Honoring the child's explicit status is deliberate; contradicting it was not. When the two disagree, both are named. -
An AI preset's declared
thresholdnow actually drives its verdict and is recorded in the result's details. Each preset hardcoded a literal that happened to equal its declared value, so editing the field silently did nothing — and unlike the deterministic presets, an AI result never stored the threshold it was judged against, so a saved verdict could not be explained after the fact. No verdict changes today; the declaration is simply now the one source of truth. -
The AI evaluators are no longer shown a trace with its falsy results removed. The summary they reason from used a bare truthiness test, so a run whose answer was
falseor0— a failed check, a "not found", a boolean verdict — was presented as a run that produced nothing, and the judge scored a trace it had not been shown. The same guard now used byshowcovers the summarizer and the diff summary. -
replay's footer no longer contradicts the header it just printed. Step durations and tokens were summed with?? 0, making "unmeasured" indistinguishable from "instant" — so a trace whose steps carry no timing reported "0ms" directly below a panel showing its real duration. With nothing measured, it now says nothing. -
guard'sname_containsnow fails closed like every other match key. An unusable value (an object, which stringifies to something no step name can contain) made adenypolicy validate, list as active, and silently never fire — a kill switch that could not fire.guard addrejects a non-string, butaddPolicyand direct inserts bypass that, which is exactly why the sibling keys already failed closed. -
ingestnow accepts a decision record on a step of any type, so a trace captured live can be re-ingested from its own export. This reverses a previously-enforced rule. Nothing else in the system maintained it: the live recorder and the SDK attach a decision to whatever step is being written, the writers insert it unconditionally, and the readers were all corrected to surface it wherever it sits. The validator was the sole holdout, and it rejected the tool's own output — a decision thatrecordcaptured,decisionsdisplayed, andexportwrote could not be loaded back, so a backup could not be restored and a store could not be moved between machines. The decision record's own shape is still validated. -
agent-replay runnow finalizes its trace and cleans up on every exit path. Interrupting the wrapper — Ctrl-C, or a CI job timeout killing it — left the tracerunningforever with no end time, no error, and no exit code, leaked its temp channel directory, and orphaned the child process still holding the terminal; the interrupt is now forwarded to the child, which runs the normal finalize-and-clean path and still reports 128 + signal. A read failure on the events channel can no longer kill the wrapper either: the poll ran on a timer, so a throw there was an uncaught exception that lost the child's exit status mid-run — a 2 GiB events file was enough, since a single read that large is rejected outright. Reads are now chunked and any I/O failure degrades to a warning. A synchronousspawnfailure (an empty command, as a script with an unset variable produces) no longer leaves an unfinalizable ghost trace. And a producer that rewrites the append-only channel instead of appending is reported, rather than having every later event silently dropped. -
listandshownow report token usage from the steps that carry it when the trace-level total is absent, and--sort tokensorders by the same number. The trace column is set only when a producer reports a total, so a measured trace showed "-" in the Tokens column and no Tokens line inshow, whilereplayprinted a total andstatscounted it — the tool disagreeing with itself about one trace. Worse,list --sort -tokensranked a 50,000-token trace below a 7-token one, so "my most expensive runs" returned the cheapest. The stored column is unchanged; this is a display value. -
eval --ainow treats trace content as data rather than instruction. The summary handed to the judge is built from the agent's prompts, tool inputs, and tool outputs — content an attacker can influence — and it was concatenated into the prompt with no delimiter and no statement of how to treat it, in the evaluators whose job is to catch exactly that. It is now fenced, and every AI system prompt says the fenced material is recorded data and never an instruction. Relatedly, a model's verdict is read from the last JSON block in its reply, not the first: a model that quoted the trace back before answering had the quoted block parsed as its verdict, even when its own answer said the opposite. -
A log-derived
llm_callstep now records its model in themodelcolumn, not only in the step name — anotel servecapture of Gemini CLI or Claude Code previously had no model recorded anywhere, while the span path set it. -
show --jsonnow carries astep_windowobject when--from-step/--to-stepnarrowed the result. The human output already printed what it omitted; the JSON did not, so a consumer received a complete-looking trace — trace-level totals intact, evals unwindowed — whose steps were silently a subset. An unwindowedshow --jsonis unchanged. Separately, a trace cost under $0.00005 no longer renders as$0.0000; sub-cent costs are routine, and the panel was reporting zero where real spend existed. -
An AI evaluator that was skipped as not applicable no longer counts as a measured 100% pass. Such a preset is stored with a score of 1.0 so it cannot fail a gate, but it makes no measurements — yet the
evaltally and average, the dashboard's score-trend chart, and an exported golden baseline all treated it as a real result.evalnow reports it separately ("N not applicable"), and the trend and baseline leave it out. The stored row stays, since it explains why nothing ran. -
Numbers the tool reports now match what it measured.
statssummed only the trace-leveltotal_tokens, which is set solely when a producer sends one — whileingest,record, the OTel mapper, and the importers all populate per-steptokens_used— so a store plainly holding tokens reported "Total tokens: -", andnullin the--jsona CI job reads.list --sort durationordered by the rawtotal_duration_mswhile displaying the derived duration, and the hook finalizer sets onlyended_at: every hook-captured trace sorted last as a NULL, so a descending list ended with its longest rows.record's "Total steps" reported each touched trace's lifetime step count rather than what the run recorded. Andstats' per-agent tally counts timeouts alongside failures by design, but labeled them "failed" — three lines below a status breakdown listing the timeout separately. -
showandreplayno longer hide a step's stored input or output when the value is falsy or scalar. Both fields hold arbitrary JSON, and the guards were a bare truthiness test on output and a key-count test on input — so a step whose output wasfalseor0(a failed check, a "not found", a boolean guard result) rendered with no output line at all, indistinguishable from a step that produced nothing, while--jsonshowed the value. A scalar input vanished the same way.show --treealso now prints a step's error: the tree is only reached when a trace has causal structure, so on a failed trace — the case the view exists for — it was hiding the failure message. -
otel servenow reportspartial_successwhen a/v1/logsbatch mapped to nothing, instead of a bare200. Onlygemini_cli.*andclaude_code.*events are recognized, so an emitter whose event names drift — a CLI version change, or a generic OTel logger pointed at the endpoint — got a clean200forever while the store stayed empty and shutdown printed "Accepted 0 trace(s)". The traces endpoint already reported this. -
export --format jsonlwith no matches now writes an empty file instead of one blank line, which a strict streaming consumer rejected as malformed JSON.export --format goldenalso warns that--with-evals/--with-snapshotsdo nothing there — the golden shape is fixed, always carrying eval criteria and never snapshots — rather than accepting the flags and ignoring them. -
The OTel logs path no longer loses or fabricates data. A flush window carrying only model-call events has no steps and no prompt but does have token counts, and the whole group was discarded — so a session's token total depended on where the exporter happened to cut its batches. Records with no
session.idwere all grouped under one placeholder, fusing unrelated services in a batch into a single trace (the span path already refuses the same fusion). And no log-derived step recorded the event's own time, so every step was stamped with the moment the batch arrived — a timeline where everything happens at once — while the trace never got an end time, and so showed no duration at all. -
importno longer fabricates token totals from string counts. A transcript whoseusagecarried"100"rather than100concatenated instead of adding —0 + "100" + 20becomes"010020", stored as 10,020 tokens instead of 120 — and the poisoning was sticky, so every later record concatenated too. (The Codex stream translator was hardened against exactly this; the importers were missed.) Atool_usewhosenameis not a string no longer aborts the entire import either: one bad block in a 50,000-record transcript threw out of the whole run, contradicting the importer's best-effort contract. And a file that yields no steps and no prompt now reports a failed import rather than creating an empty trace and exiting0. -
OTel spans in the OpenInference and OpenLLMetry dialects now carry their prompt and response content. Only the
gen_ai.*content attributes were read, so a LangChain or LlamaIndex app — the frameworks these conventions come from, and the ones the README names — produced traces whose every step had an empty input and no output. The spans were classified, timed, and token-counted correctly; they simply carried nothing to read, and the raw attributes were preserved nowhere either.input.value/output.value,llm.prompts/llm.completions,traceloop.entity.input/output,llm.provider, andtool.nameare now mapped alongside their GenAI equivalents. -
Opening the trace store no longer lowers its own lock patience, and no longer blames corruption for every failure.
busy_timeoutwas set to 3s where better-sqlite3 already defaults to 5s — a reduction, written as though it were an increase — so a short-livedhookprocess contending with a slowotel servemerge was aborted earlier than the default would have, and aSQLITE_BUSYthere is swallowed as a warning, i.e. silently lost capture. It is now 10s and set before the WAL conversion, which itself needs a lock. Separately, a store that is merely locked, read-only, or unreadable is reported as such: the old message said "may be corrupted" for all of them, and the natural response to that is to delete the store. -
guard addnow warns when adenyorrequire_reviewpolicy matches onoutput_contains. Enforcement evaluates a proposed tool call — before it runs, so there is no output yet — and every match key must match, so such a policy can never block live, however active it looks inguard list. It remains valid for post-hoc evaluation (guard test, recorded steps), which is why this warns rather than rejects. Documented in the README as well. -
guard checknow fails closed when it cannot evaluate policies at all. Opening the store and running the policy match were unguarded, so an infrastructure error — an unopenable or read-only store, orSQLITE_BUSYfrom a concurrenthookprocess — exited1. That is not the block signal (2is), so every harness treated it as a non-blocking error and ran the tool: a gate wired in as a blocking pre-exec check quietly stopped denying the moment the database was locked.hook --enforcealready failed closed here, as does this command's ownrequire_reviewpath without a TTY. -
eval --aino longer reportsai-root-cause ✔ 100%for a trace whose every tool call failed. The preset decided it was "not applicable" by looking for a step of typeerror, but no capture path emits one —hook,record, and both importers record a failed tool call as atool_callstep carrying anerror— so it was skipped for every real failure, and a skip stores a score of 1.0. The provider was never called for the analysis the preset exists to do. The deterministic criteria were corrected the same way; this one was missed. -
eval --ai --max-costnow exits non-zero when the budget stops the run part way through. The remaining evaluators never reach the results list, and the pass/fail gate can only reason about the ones that ran, so an unfinished run reported green. (The pre-run estimate check already exited non-zero for the same reason.) The notice also goes to stderr, so it can no longer corrupt--jsonoutput. -
--sincenow compares instants instead of bytes, acrosslist,stats,export, andcheck.started_atis a TEXT column and nothing constrains the format a producer writes —ingest,record, and both importers pass a timestamp through verbatim — so the byte order was not the time order. A timestamp with a UTC offset landed in the wrong window (14:00+02:00is12:00Z, an hour before a13:00Zcutoff, yet it was the row returned), and a space-separated timestamp — SQLite's owndatetime()form — sorted below everyT-separated one and was excluded from every window. Acheck --since 1dCI gate therefore skipped traces it should have checked. A timestamp that cannot be parsed at all still falls back to the old comparison, so nothing that used to be returned disappears. -
--sincealso rejects a date-shaped value that isn't a real date.2026-99passed the format check and became a bound no timestamp could satisfy, so every query reported "No traces found" and exited0— indistinguishable from an empty store, and a silently empty CI gate. -
An OTel trace assembled from several export batches now numbers its steps by start time rather than arrival, so a parent span that flushes late no longer produces a forward parent reference. Batches arrive in completion order and a parent span ends after its children, so the parent was numbered above the child it owned — a reference
ingestrejects (the export → ingest round-trip hard-failed for exactly the deep traces cross-batch assembly exists to serve) and one that madewhyandshow --treerender step 1 as "caused by #2". The hierarchy is preserved; it now points backward, as everything downstream already assumed. -
OTel spans that end before they start (clock skew between hosts, or a hand-rolled exporter) no longer persist a negative duration at the step or trace level. Those values are exactly what
validateTraceInputrejects, sootel servewas writing rowsingestrefuses — the same round-trip break already fixed for span parentage — and the UI printed a negative millisecond count. Contradictory timing is now recorded as unknown rather than clamped to zero, which would claim the call was instant. A genuine 0 ms span is unaffected. -
check --goldennow refuses a golden file with no entries (exit2) instead of reporting a vacuous green gate. An empty baseline matches nothing, so every candidate fell to theunmatchedbranch — which passes unless--strict— and the run printed "0 passed, 0 regressed" and exited0forever. The usual cause is a filter typo:export --format golden --tag known-goodwrites[]and exits0when the tag is actuallyknown_good. That export now also warns that the baseline it just wrote can never detect a regression. -
recordnow exits1when a stream produced input but nothing was recorded, instead of reporting a total capture failure as success. Piping the wrong--format(or a broken producer) intorecordrecorded nothing and still exited0— soagent | agent-replay record && agent-replay checktreated an empty recording as a clean run. Note the--formatcase needs the check to key on input received rather than on warnings, because a stream translator ignores an unrecognized line silently. Per-event leniency is unchanged: a stream where some events survive still exits0, and an empty stream is still not a failure. -
whyno longer presents time-travelling causality as fact.ingestvalidates thatparent_step/caused_by_stepreference an earlier step, but the liverecord/SDK path passed producer values straight through — and the causal walk's contract depends on that invariant. A forward reference made step 1 render as "caused by #2", a step that hadn't happened yet. A reference that isn't a positive integer strictly earlier than its own step (including a self-reference) is now dropped at the write boundary, matching what the OpenTelemetry mapper does with an out-of-order parent span. -
An OpenTelemetry root span's own tokens and attributes are no longer dropped. The token total summed only the child spans and the root's attributes were never carried, so a single-span agent trace reported no tokens, no model, and no provider despite the span carrying all three.
-
An OpenTelemetry child span that starts before its parent no longer produces an unusable parent reference. Step numbers follow start-time order while parentage resolves by span id, so clock skew or an async wrapper yielded a forward reference (and a self-referencing span pointed at itself) — shapes
validateTraceInputrejects, meaningotel servepersisted rows thatingestrefuses and an export → ingest round-trip of an OTel trace failed. Only a strictly-earlier parent is kept;otel_parent_span_idstill rides along in metadata, so the cross-batch re-link can repair the link later. -
An interrupted
codex exec --jsonrun is no longer recorded ascompleted. The translator never declared that its stream has a terminal event (turn.completed), so reaching EOF without one still closed the trace cleanly — a killed or crashed run looked like a successful one. It now staysrunningsorecordfinalizes it astimeout, matching the native protocol and the gemini stream, which already behaved this way. -
Codex token totals are summed numerically.
usagewas only cast to numbers, so a producer sending"5"and"7"produced0 + "5" + "7"— the string"057", stored as 57 tokens instead of 12, silently and with no warning. -
An OpenTelemetry span that captured its own exception is now recorded as a failure. Error detection keyed solely on
status.code, missing the two other ways a failure arrives: anexceptionspan event — whatrecordExceptionwrites, and several instrumentations call it without also setting the status — and anerror.typeattribute, which GenAI semconv sets on a failed operation. Such a span was stored as a completed step on a completed trace, with the exception text preserved nowhere at all. Span events are also decoded on the protobuf transport now (field 11 was skipped entirely, so protobuf could not report this class of failure even in principle, while JSON could). An explicitOKstatus still wins over both weaker signals. -
Failures captured over the OpenTelemetry logs path are no longer invisible. That path had no error handling at all: the trace status was hardcoded
completed, no step ever received anerror, and.api_errorrecords matched no branch, so they vanished entirely — a batch containing only those produced zero traces and still answered200. A session whose every tool call failed therefore looked like a clean run tolist,check --golden, andeval's error criteria alike. A tool record withsuccess: falsenow carries itserrortext (falling back toerror_type, then a generic message), anapi_errorrecord becomes anllm_callstep with the failure onerror, and the trace status is derived from its steps rather than assumed. Aclaude_code.tool_resultalso keeps itsduration_ms, which was dropped. -
A single malformed scalar from a producer no longer costs a whole trace, step, or finalization during live capture. SQLite refuses to bind an object or array, and
recordswallows that error as a per-event warning, so the damage went far beyond the offending field:agent_version,session_id, orstarted_atsent as an object ontrace_startlost the entire trace — every later event then failed with "trace not found", and the command still exited 0.total_tokens,total_duration_ms,total_cost_usd, orended_atsent as an object ontrace_endlost the finalization: a run that reportedfailedwith an error was persisted astimeoutwith no error, turning a crash into an apparent hang.model,tokens_used,duration_ms, or a decision'srationale/confidencesent as an object dropped the whole step. Onstep_end, where one combinedUPDATEcarries every field, a bad duration silently took the step'soutputwith it.
These are now coerced at the bind boundary, like
trigger,status, andtagsalready were: a non-scalar becomes null and everything else is kept. A numeric string ("1234") is accepted for a numeric column.ingestvalidates these fields upstream, so the coercion only ever applies to live-captured data. -
record --tagsno longer aborts the entire stream when a producer sends a non-arraytags. The merge spreadtagsoutside the per-event error handler, so the spread threw, the process exited 1, and every trace in the stream was lost — not just the bad event. A string value would also have spread into one tag per character. -
name_regexpatterns that backtrack catastrophically are now rejected. The check only caught a quantifier appearing immediately before a group's closing paren, so(a|aa)+,(\s*\w)*, and(.*,)*all passed and then took seconds — minutes, for a slightly longer name — to evaluate a ~35-character tool name. Becausename_regexruns on the guardrail path, that isn't merely slow: it stalls the check, and a harness that treats a timed-out hook as non-blocking turns the stall into a fail-open. The rule is now "an unbounded quantifier applied to a group whose body contains a quantifier or an alternation", which covers all of those forms. It is deliberately conservative and may reject a pattern that would have been safe; bounded quantifiers ((\d{3}){2}), unambiguous groups ((abc)+), and alternations without an unbounded outer quantifier (^(get|list)_x$) are unaffected. -
A malformed
hookcommand line no longer blocks the host agent in capture mode. Commander reports every usage error (an unknown flag, a stray argument) as exit 2 — but in each supported harness exit 2 blocks the pending tool call, and the hook configuration is static, so one typo insettings.jsonblocked every tool call for the whole session, from a capture-only hook that is documented never to affect the host agent.runHookguarantees exit 0, but commander's error handling runs before the action ever executes. A usage error onhookwithout--enforcenow exits 0. With--enforceit still exits 2, since blocking is the correct fail-closed answer when the gate can't run. -
guard removeno longer deletes a second policy. The lookup wasWHERE id = ? OR name = ?with the same value bound twice, so removing a policy by id also removed any policy named after that id — and reported success. Resolution is now by id first, then by name. -
forknow keeps the original trace'smetadatainstead of replacing it. The fork provenance (forked_from,forked_at_step) overwrote the whole object, so anything a producer had attached — run/session correlation ids, cost tags, harness info — was dropped from every fork, while steps, decisions, snapshots, tags, andsession_idwere all copied faithfully. Provenance keys still win on a name collision. -
A non-array
tagsvalue from a producer no longer corrupts reads or breaksfork --tag.ingestvalidates tags, but the live event protocol doesn't type-check them, sotags: {...}was stored verbatim in a column every reader treats as an array.fork --tagthen threw ontags.pushafter its fork had already been committed, reporting "Fork failed" (exit 1) for a fork that existed but whose id was never printed — leaving an orphan behind, and another one on every retry. Tags are now coerced to an array on write and on read. -
diffno longer reports two runs as identical when one of them failed. Steps were compared onstep_type,name,input,output, andmodelonly — the step'serrorwas never compared, and no trace-level field was compared at all. So a run that succeeded and a run that failed with a 500 from the same tool call produced zero differences: the renderer printed "Traces are identical." directly beneath a header showingCOMPLETEDbesideFAILED, and--jsonagreed with"diffs": []. That is the flagship "it worked before, what changed?" case, and every live capture path records a failed tool as an ordinary step witherrorset, so it is the common shape rather than an edge case. Steps now also compareerror, and the trace itself comparesstatus,trace_error, andtrace_output. A trace-level difference reports its step astrace(nullin--json) and does not setdivergence_step, which still means "the first step that went different". -
A guardrail policy whose
input_contains/output_containsvalue holds a quote, backslash, newline, or tab now matches. The haystack was the JSON-encoded step, where those characters appear escaped (\",\\,\n,\t), while the needle is the pattern exactly as written — so adenyonrm -rf "/etc", or on a Windows path likeC:\Windows\System32, could never match its own step. It failed silently: the policy validated cleanly,guard listshowed it as an active deny, and it never fired. These are precisely the shapes a destructive-command policy is written with. Patterns are now matched against the raw text as well as the JSON form, so patterns aimed at the JSON itself (a key name like"cmd") keep working. -
Two further guardrail fail-opens now fail closed, matching what
step_typeandname_regexalready did. Aninput_contains/output_containsvalue that is an object or array stringifies to"[object Object]", which can never occur in the haystack; and a pattern whose only keys are unrecognized (a typo likenmae_contains) can never match anything. Both stored adenythat silently never fired.guard addrejects them, but the service API used by seed data and any non-CLI caller, and direct inserts, bypass that validation. A scalar value still coerces, soinput_contains: 123matches the text "123". A genuinely empty pattern is deliberately left inert, as before. -
export --status <typo>now exits 2 (a usage error) instead of 1. The bad value reachedlistTracesinside the export block, whose blanket catch reports every failure as a runtime error — so a CI script branching on the exit code read a typo as a genuine failure, whilelistreturned 2 for the identical error.--statusis now validated up front, like the--sinceand--formatchecks beside it. -
A tool failure reported as a structured error (
{message, code, stderr}) rather than a string is no longer discarded byhook. The whole object collapsed to the generic "tool failed", and irrecoverably, since a post-tool payload isn't retained anywhere else. It is now flattened to JSON text, the same coercion applied elsewhere when binding a structured error. A failure with no detail still falls back to "tool failed". -
A hook payload whose event name collides with an
Object.prototypemember (constructor,toString,hasOwnProperty,__proto__) is now ignored instead of creating an unfinalizable trace. The event-name lookup table is an object literal, so those names resolved to an inherited function — truthy, which skipped the "unknown event" early return, made the reported action a function rather than a hook action, and left behind arunningtrace with no steps that nothing ever closes and that polluteslistandstats. Lookups are now by own key only. -
hook --enforcenow fails closed when stdin carries no payload. An empty or unreadable stdin (a harness crash, a broken pipe) returned exit 0 before any of the fail-closed logic and without ever consulting--enforce, so the pending tool call was allowed on a gate that exists to stop it. Having seen no payload is the same position as a throw before the verdict, which already fails closed; a gating (pre_tool) route now emits the dialect's block, and everything else still allows. The documented allow on a malformed payload is unchanged — there the harness did send something, and that allow is a deliberate choice. -
A plain-string
inputoroutputis no longer silently discarded. Nothing requires a producer to send an object —validateTraceInputacceptsinput: "summarize the doc"and the live event protocol never type-checks these fields — but the encode helper wrote any string into the JSON column unquoted. Parsing it back then failed, so the user's prompt and the agent's answer read as{}andnullfrom every consumer (show,diff,export,replay), with exit 0 and no warning. The damage carried into the regression gate too: every affected trace hashed to the same empty input, collapsing unrelated cases into one golden bucket and comparing{}against{}, so a real tool-input regression passed green. A string is now passed through only when it already is valid JSON (the case that passthrough exists for) and otherwise encoded. Affectsingest,record, andhookalike. The plain-TEXTerrorcolumn is unchanged — it is read back raw, so a string error is still stored as-is. -
statsand thedashboardnow report an average duration for traces that carry only timestamps. The aggregate averagedtotal_duration_msalone, whilelistandshowrender duration viaeffectiveDurationMs, which falls back toended_at - started_at. The hook finalizer records onlyended_at, so on a store captured the normal way (hookorrecord) every trace showed a duration inlistwhilestatsreportedAvg duration: -; on a mixed store the average covered only the subset with an explicit total. The aggregate now applies the same fallback in SQL. A trace with no usable duration (still running, or noended_at) is still excluded rather than counted as zero. -
hook --enforcenow closes thetool_callstep of a denied call, so a concurrent call's result can't be recorded against it. A denied call never runs, so noPostToolUseever arrived to close its step — and left open it was the newest unclosed step for that tool name, which is exactly what the post-tool handler looks for. The nextPostToolUsefor that name, belonging to a different call allowed in the same parallel batch, therefore closed the blocked step: the audit trail showed the blocked command completing successfully with another call's output, while the call that really ran stayed open forever. The denied step is now closed on the verdict with the blocking policy recorded in itserror. Arequire_reviewstep is deliberately left open, since it maps toaskand an approved call still runs and closes normally. -
eval's deterministic error criteria now detect the failures that live capture actually records, soevalworks as a CI gate on the runs it exists for.hallucination-check'sno_error_stepsandcompleteness-check'sno_unresolved_errorskeyed only on a step whosestep_typeiserror— but every live capture path (hook,record, and the transcript importers) records a failed tool as atool_callstep with theerrorfield set, and never emits a dedicatederrorstep. Both criteria therefore scored a perfect 1.0 for every live-captured failure: a trace thatlistdisplays as✘ FAILED, with a trace-level error and a tool call that returned 503, passed all three presets andeval --allexited0. A step now counts as failed if itsstep_typeiserroror it carries anerrorvalue, andno_unresolved_errorsadditionally fails on a trace-levelerror— the clearest "this run ended unresolved" signal there is, and the only marker left by a run that died before emitting a final step. The change can only turn a false pass into a failure; no passing trace starts failing. -
Importing a Claude Code transcript now preserves a failed tool call. A
tool_resultblock flaggedis_error(aBashthat exits non-zero, aReadon a missing file — very common) had its flag dropped: the imported step was a plaintool_callwith a null error, indistinguishable from success, so an error-aware consumer (eval's error checks,check --golden, the timeline) read a failed run as passing. The failure text now lands on the step'serrorfield (with a generic "tool failed" when the result carried no content), matching how the live hook-capture path records a failed tool call. Both the main transcript and subagent import paths are fixed. The result text still appears in the step output as before. -
check --golden(the bulk CI-gate path, run without--trace) now scans every candidate trace instead of only the newest 10,000. It passedlistTracesa hardlimit: 10000, so on a store with more than 10,000 traces every candidate older than the newest 10,000 was never fetched, never diffed against a golden baseline, and never appeared in the report — a real regression living in an older trace produced zero failures and a green exit, silently defeating the gate. It now passes an unbounded limit (-1), the same fix already applied toexport, so the "exits non-zero on any regression" contract holds regardless of store size. -
Cross-batch OpenTelemetry trace assembly no longer loses a step's parent when the parent span flushes in a later export batch than its child. A span is exported when it ends, and a parent span ends after the children it owns, so a deep child whose parent span crosses a
BatchSpanProcessorflush boundary arrives in an earlier batch than that parent. The child was stored with noparent_step_numberand never repaired — the merge only re-linked a new step onto an already-present parent (the parent-first ordering), not an existing orphan onto a parent that arrived later. Such a trace rendered flat undershow --treeand broke itswhycausal chain. The merge now also re-links backward: once a batch supplies a previously-missing parent span, every orphan that referenced it by OTel span id is reconnected. -
watchno longer drops the final step(s) of a trace that completes. Each poll read the new steps and then, separately, the trace status; a producer (a different process) could commit a trailing step and flip the status to a terminal value in the gap between those two reads, so the completion tick stopped the tail without ever printing that step — the live view disagreed with whatshowdisplayed. The tail now drains once more when it detects completion, before announcing the final status. -
runnow records a wrapped child's trace even when the child generates its owntrace_id. The wrapper owns the trace and is supposed to stamp its own id onto every child event, but the code only did so when the id was missing — a case the event validator already rejects — so the stamp never happened. A compliant child (including one built on theTraceRecorderSDK, which generates its own id unless it threadsAGENT_REPLAY_TRACE_ID) therefore had every step and itstrace_enddropped as "trace not found," leaving an empty trace stuckrunning. The wrapper now re-stamps its id unconditionally, and only treats a child's terminal status as declared once it has actually persisted (so a finalization that failed to apply can't suppress the exit-code fallback). -
forkno longer reports "Modified input/context: Yes" for a payload that was not actually applied. A literalnullpassed to--modify-input/--modify-contextparses to a value the fork service treats as a no-op (it keeps the original), but the summary keyed off the presence of the raw option string, so it falsely claimed the modification landed. The summary now mirrors the service's own guards and reports the modification only when it was applied. -
Live
record/SDK capture no longer drops a step or loses a trace's finalization when a producer supplies a structurederrorobject. The event protocol typeserroras a string, but a harness just as naturally emits{ message, code, … }, and — unlike the adjacentoutput— it was bound to SQLite raw, so the object threw ("can only bind numbers, strings, …") and the recorder swallowed it as a per-event warning: the step (with its decision and snapshot) vanished, or atrace_enderror dropped the whole finalization and left the trace stuckrunning(finalized astimeoutat EOF). Theerrorcolumn is now coerced likeoutput(a plain string is kept as-is, an object is JSON-stringified), matching the hook adapter's existing error guard. -
guard addnow rejects a match pattern whosestep_typeis not one of the real step types, and a policy stored with such a value fails closed at match time.step_typeis a closed enum, so a typo like"toolcall"(for"tool_call") can never match any step — previously it was saved as an enableddenythat silently never fired, a kill-switch the user believed was protecting them. This mirrors the existing rejection of keyless patterns and non-stringstep_type; a blocking policy with an unusablestep_typenow blocks rather than being skipped. -
Live
recordcapture no longer loses an entire trace (or its finalization) when a producer uses atriggeror terminalstatusoutside agent-replay's vocabulary. Both are free strings in the event protocol, but the database constrains them to fixed enums. Atrace_startwith an unknowntrigger(say"scheduled") threw a CHECK-constraint error that the recorder swallowed as a warning — the trace was never created, and every later event then failed with "trace not found," so the whole run vanished. Likewise an unknown or emptytrace_endstatus(say"success") dropped the finalization, discarding the output/token/cost totals and leaving the trace stuckrunning(finalized astimeoutat EOF). Both are now coerced to a valid enum at the service boundary — an unknown trigger tomanual, an unknown status tocompleted— mirroring the existingdecided_bycoercion. Theingestpath still rejects these as a usage error, so the coercion only ever applies to live-captured data. -
OTLP-ingested steps with no output messages now persist
output: nullinstead of a spurious empty{}.messageContentnever returns null (it just omits themessageskey), so a message-less step span — the common case for tool and thought spans — stored{}, which reads as truthy downstream (a summary prints "OUTPUT: {}", golden export stores{}instead of null). The trace root already guarded this; the step mapper now applies the same guard. Input still keeps{}as its empty value, exactly like the root. -
show --treenow renders the⟵ caused by #Nannotations for a flat causal trace — one whose steps record causality viacaused_by_stepwithout any parent nesting (a normal shape, e.g. a decision followed by the steps it caused). The tree renderer decided whether to run based only onparent_step_number, so such a trace fell back to the plain timeline and showed no causal links at all, silently defeating the whole point of--tree. It now also runs when any step has acaused_bylink. -
ingestauto-detection no longer trusts a.jsonl/.ndjsonextension over the file's actual content.export --format json(the default) into a.jsonl-named file writes a JSON array, and the detector used to short-circuit on the extension, line-split the array, and fail with a misleading "Invalid JSON on line 1" even though the JSON was valid. Detection now probes content: a file whose entire contents parse as one JSON value isjson, and only genuine line-delimited data (whole-file parse fails) isjsonl— completing the earlier fix that stopped a pretty-printed object being misread as JSONL. An explicit--formatstill overrides detection. -
evalno longer displays a score that contradicts its own pass/fail verdict. A stored score is rounded to three decimals andpassedcompares that value to the threshold, but every human-readable percentage rounded it to a whole number — so a0.695score (which fails a0.70threshold) rendered "70%" next to aFAILbadge. Scores now render losslessly (a three-decimal score is exactly a one-decimal percent), so0.695reads "69.5%" and a whole-percent score like0.70still reads "70%".--jsonoutput and exit codes were already correct; this fixes only the terminal display (evalspinners, the eval table's score badge, and the AI root-cause confidence line). -
getTrace(the prefix resolver behindshow/diff/replay/fork/eval/guard/watch) now escapes LIKE metacharacters in a partial trace id. Trace ids aretrc_+nanoid(12)over an alphabet that includes_, so a copied partial such astrc_ab_ctreated_as a wildcard and could resolve to an unrelated trace (and a literal%matched every row). The lookup now declaresESCAPE '\'and escapes\ % _, mirroring theagent_name/session_idbranches inlistTraces. -
guard addnow rejects a--patternwith no recognized match key (name_contains,input_contains,output_contains,step_type,name_regex). A typo'd key such as{"tool_name":"delete"}(the real key isname_contains) previously passed validation, was saved as an enableddenypolicy, and then matched nothing at evaluation time — a kill-switch that never fires. This is the same silent-fail-open class the pattern validator already guards against for an unusablename_regex/step_type. -
why --stepis now parsed withNumber()instead ofparseInt, matchingshow/replay/fork.--step 1e2read as1and explained the causal chain for the wrong step with a success exit;--step 2.9/2abcslipped through as2.1e2now correctly means100and a non-integer is a usage error (exit2). -
fork --from-stepis now parsed withNumber()instead ofparseInt, matchingshow/replay's--from-step/--to-stepandlist --limit.--from-step 1e2read as1(parseInt stops ate) and2.9/3abcslipped through as2/3, so a fork silently started from the wrong step (and its--modify-context/--modify-inputlanded on the wrong point) with a success exit. A non-integer is now a usage error (exit2). -
safeRegex(which backs guardname_regexmatching) no longer rejects a safe pattern with a bounded outer quantifier such as(\d{3}){2}("exactly six digits as two groups"). Its ReDoS guard flagged any quantified group followed by any quantifier, but catastrophic backtracking requires the outer quantifier to be unbounded (+,*,{n,}); a bounded{n}/{n,m}caps the work. The guard now keys off the outer quantifier, soguard add --pattern '{"name_regex":"(\\d{3}){2}"}'is accepted while the genuinely dangerous(a+)+,(a{1,3})+, and(a+){2,}stay rejected. -
OTel-ingested traces whose root span carries no output messages now record
output: nullinstead of an empty{}.messageContentalways returns an object (it just omits themessageskey when absent), so the intended?? nullfallback was dead and a spurious empty output persisted — which reads as truthy downstream (a trace summary printedOUTPUT: {}, and a golden export stored{}rather than null). The input side is unaffected ({}is its correct empty value). -
show/replaynow validate--from-step/--to-stepwithNumber()instead ofparseInt, matchinglist --limitandconfig set.--to-step 1e2read as1(parseInt stops ate), silently capping the window at step 1 instead of 100, and--from-step 2.9/3abcslipped through as2/3. A non-integer is now a usage error (exit2), and1e2correctly means100. -
listno longer printsNaNd agofor a trace whosestarted_atis an unparseable/empty string. The table's localformatRelativehelper missed theisNaNguard its two siblings (formatRelativeTime/formatTimestamp) already have; it now renders-, like them. -
config set ai.max_tokensnow echoes the normalized value it stored, not the raw input —config set ai.max_tokens 1e3confirms= 1000(whatconfig get/listwill show), not the misleading= 1e3. -
hook --enforcenow fails closed when it cannot reach a verdict. The audit-write path already failed closed, but a failure before the verdict — opening the trace, appending thetool_callstep, or loading policies (e.g. a transientSQLITE_BUSYon a shared machine) — threw out ofapplyHookPayload, and the command's catch logged it and exited0(allow). So an infrastructure hiccup could let a tool call that adenypolicy would have blocked run through — a safety fail-open contradicting the enforcement contract. Apre_toolevent that can't be evaluated now emits the dialect's block (deny/exit 2) with a clear reason; capture-only events (and capture mode without--enforce) still never block the host. -
The
ai-root-causeeval preset now derivespassedfrom the rounded score that is stored and displayed, not the raw confidence. Its three sibling AI presets (ai-quality-review,ai-optimization,ai-security-audit) were already fixed this way;ai-root-causestill compared the raw value, so a confidence that rounds up across the 0.5 threshold (e.g.0.4996→ shown as50%) reportedpassed: falsewhile the panel readConfidence 50%— the same score-vs-verdict contradiction the sibling fix closed. -
AI eval / diff against OpenAI now sends
max_completion_tokensinstead of the legacymax_tokens. The default OpenAI model is a GPT-5-family model (gpt-5.4-nano), and OpenAI's chat/completions endpoint rejectsmax_tokensfor GPT-5 / o-series models with a 400 — so every--provider openaieval on the default model failed the request outright (and the 400 was then reported as a generic "Server error").max_completion_tokensis the current field and is also accepted by GPT-4o-and-later, so it works for any model the adapter targets. Anthropic (max_tokens) and Google (maxOutputTokens) were correct. -
Live capture (
record/run) now validates thechosenfield of astepevent's inline decision, instead of losing the whole step to a silent DB error.appendStepbindsdecision.chosenstraight into SQL; an inline decision with nochosenboundundefined, which better-sqlite3 rejects, rolling back the step-and-snapshot transaction — a data loss swallowed as a generic per-event warning. The event protocol now skips such an event with a clear "inline decision requires chosen" warning, matching how a top-leveldecisionevent is already validated (and closing an asymmetry where an empty-stringchosenwas accepted inline but rejected top-level). -
The AI diff analysis (
diff --ai) no longer mislabels a null-valued field as a missing step. The diff summary rendered anynulldiff value as(missing), butdiffTracesemits ordinary field diffs (output,model) whose value is legitimatelynullon a step that exists on both traces — so a paired step whose left output was recorded but right output wasnulltold the model "step N is absent on the right," the opposite of the truth.(missing)now renders only for themissing_left/missing_rightdiff types that actually mean an absent step; a null field value renders asnull. -
exportno longer silently caps at 10,000 traces. Despite a comment claiming it "removed the limit," the code passed a fixedlimit: 10000tolistTraces, which applies it as a SQLLIMIT— so exporting a store with more than 10,000 matching traces dropped the overflow with no warning, corrupting any golden/JSONL dataset built from it. It now passes an unbounded limit (SQLite treats a negativeLIMITas no limit), so every matching trace is exported. -
otel servelog ingest now preserves a genuine0 mstool-call duration from a Gemini CLItool_calllog event instead of dropping it to no-duration. The helper coalesced with|| null, so a real0(an instant or cached tool) collapsed tonulland the step showed no duration — the same class as the earlierhook0 ms fix, in the OTLP path. A0is now kept; only an absent or non-numeric value becomesnull. -
eval --rubricnow coerces and validates the rubric'sthresholdlike it already doesweight. A YAML author naturally quotes it (threshold: "0.8"), which arrived as a string and flowed intoscore >= threshold; a numeric string happened to coerce, but a non-numeric one ("abc") made every comparisonscore >= NaN→ always false, failing an otherwise-passing trace and reporting a correct trace as a CI regression (the same class theweightfix closed). A presentthresholdthat isn't a number in[0, 1]is now a usage error (exit2), and a numeric string is coerced. -
guard checkno longer crashes with a rawTypeErrorwhen the step piped to stdin is valid JSON but not an object (null, an array, or a bare value).nullin particular reached a property access (null.step_type) and threw a stack trace; it now reports a clean "expected a single step object" error and exits1, like the other malformed-input paths. -
ingestreports the true file line number when a JSONL line is malformed. The line number was computed after blank and//-comment lines were filtered out, so a broken line preceded by any of them was named as an earlier line (e.g. a syntax error on file line 5 reported asline 2), sending you to the wrong place in the file. Exit code and tallies were already correct; only the diagnostic was off. -
guard check/hook --enforcenow attribute a block to a deterministic policy when several equal-priority policies match the same step. The verdict keeps the first most-restrictive match, but enabled policies were loaded ordered only bypriority DESCwith no tiebreaker, so among equal-priority, equally-restrictive matches (e.g. twodenypolicies) the cited policy name and reason — shown to the user and recorded on theguard_checkstep — varied with SQLite's incidental row order. The block itself was always correct; only the attribution was unstable. Enabled policies now break ties byname(unique), matching the orderingguard listalready uses. -
list(and every paginated trace query) now orders tied sort keys deterministically. TheORDER BYhad no unique tiebreaker, so rows sharing the sort value — a common case, since batch-ingested or demo traces routinely share a millisecondstarted_at, and running traces all haveNULLtokens/duration/cost — came back in an unspecified order. Across--limit/--offsetpages (especially with a live recorder writing concurrently) a tied row could repeat on one page and be skipped on the next. A stableidtiebreaker is now appended, making the order a total order. -
watchno longer silently drops a step that is written after a higher-numbered one. The live tail cursored on the higheststep_numberseen, but step numbers are producer-supplied and need not be written in increasing order (only unique per trace), so a step whose number was lower than one already printed — but written later — fell outside the> cursorwindow and never appeared. The tail now tracks the set of printed step numbers, so a late, lower-numbered step is surfaced on the next poll. -
watch --intervalnow rejects a malformed value (non-numeric, zero, or negative) with a usage error (exit2) instead of silently falling back to the 500 ms default, matching thedashboard --refreshconvention. A typo like--interval 5OOno longer looks like it took effect. -
guard addandeval --rubricno longer reject a safe regular expression that ends in an optional group, e.g.read(_\w+)?. The ReDoS guard insafeRegexflagged a trailing?on a quantified group as a nested quantifier, but?bounds the group to 0–1 repetitions and cannot backtrack catastrophically. The false rejection blocked legitimate kill-switch policies from being stored, and, in an eval rubric, scored a criterion with such a pattern as a failed "invalid regex" (its weight still counting), silently depressing the trace's score. The guard now treats only an unbounded outer quantifier (+,*,{…}) as dangerous; genuine nested quantifiers ((a+)+,(a*)*,(a+){2,}) are still rejected. -
evalwith theai-security-auditpreset now derives its pass/fail verdict from the risk score against the preset threshold, like every other AI preset, instead of the model's self-reportedsafeboolean. The two can disagree — a well-formed response ofrisk_level: "critical", safe: true, or a clean one whosesafearrived as the string"true"— and because the verdict came straight fromsafe, the CI gate could pass a critical-risk trace or fail a clean one, and the stored record contradicted itself (score 0.0, passed true). The verdict is nowscore >= 0.8; thesafeflag is retained in the eval details for reporting. -
ingestnow reports a non-stringstatusortriggeras a clean, named validation error instead of letting it reach the database. The trace validator requiredtypeof === 'string'before checking the enum, so a non-string value (status: 42,trigger: true) passed validation and then failed at insert with a cryptic SQLiteCHECKor bind error — or, for--dry-run, was reported valid. It is now rejected up front likestep_typealready was. -
The v1→v2 schema migration is now idempotent and safe under concurrent upgrades.
runMigrationsread the schema version outside any transaction, so two processes opening a still-v1 database at once (the app spawns short-livedhookprocesses freely) could both attempt the v2ALTER TABLE ADD COLUMN— and the loser crashed withduplicate column name(notSQLITE_BUSY, so the busy timeout didn't help), surfacing as a stack trace, or, underhook's catch-all, a silently dropped step. The version is now re-read inside aBEGIN IMMEDIATEtransaction so a process that loses the race sees v2 and skips, and eachADD COLUMNis guarded by a column-existence check as a backstop. Only the one-time upgrade window was affected. -
otel servelog ingest no longer lets a timestamp-less log record stealstep_number1 and mis-order a session's steps.timeUnixNanois optional in OTLP, and a record without it flattens to time0, which sorted ahead of the real, timed events — so a timestamp-lesstool_result/tool_decisionjumped to the front of the trace. The sort now places an untimed record last (time || Infinity), mirroring the start-less span guard already applied to the trace path; the tracestarted_atwas already derived only from timed records. -
import --format codex-rolloutno longer drops a reasoning step's text (or an assistant message's) when the richer field is an empty array. A Codex/Responses APIreasoningitem with no generated summary serializes assummary: [](present but empty) with the actual text incontent; the fallback chain used??, which treats[]as present, so it kept the empty summary and imported the thought as{ text: "" }— silently losing the reasoning. The same defeated amessagewhosecontentwas[]with text intext. Both now fall back with||, treating an empty ("") extraction as absent, so the next candidate field is used. -
eval --ai/diff --aicost estimation no longer under-prices a model that merely shares a name prefix with a cheaper entry in the rate table, which could let a run slip past--max-cost. The family-match borrowed a known rate when one model id was a string prefix of another — sogemini-2.5-flash(a real, pricier model) matched the cheapergemini-2.5-flash-liteand was priced ~4× too low. A family match now only borrows a rate across a version/date suffix (-<digits>, e.g.claude-haiku-4-5↔claude-haiku-4-5-20251001); a different variant (-lite,-pro, …) falls through to the conservative max-rate fallback, preserving the "never cheaper than reality" guarantee. -
config test-ainow exits non-zero when the connectivity check fails. Like the other commands just fixed, itscatchreported the failure but never setprocess.exitCode, so a broken or expired provider key exited0— a CI check gating on it would read a broken provider as healthy. -
Two more runtime-failure paths now exit non-zero, completing the exit-code sweep: an AI preset that throws inside
eval --ai(a provider/network error — previously swallowed by the loop, and since the thrown preset never reached the pass/fail tally, a lone failing AI preset could exit0), and ademoseed failure. -
otel servenow answers a real413for an oversized uncompressed request body instead of resetting the connection. On exceeding the 32 MB raw-body cap it destroyed the socket before the413could flush, so the client saw a connection reset (ECONNRESET). OTLP exporters treat a reset as retryable but a413as not — so an oversized batch was resent forever, the exact runaway the cap exists to stop. The receiver now stops reading, sends the413withConnection: close, and lets the socket close after the response is delivered. (The gzip-bomb413path already worked; only the raw-body path reset.) -
guardnow rejects a non-stringstep_typein a policy match pattern, and a blocking policy carrying one fails closed.step_typewas the only match key not type-validated, soguard add --pattern '{"step_type": true}'stored adenywhosestep.step_type !== trueis always true — a kill-switch that silently never fired.guard addnow rejects it (like the other keys), and the matcher treats an unusablestep_typeon adeny/require_reviewpolicy as a match (fail closed), mirroring thename_regexhandling, so a policy stored before this validation still blocks rather than silently passing. -
hooknow records a genuine 0 ms tool duration instead of leaving it blank. An instant or cached tool call that closed in the same millisecond it opened had its duration computed asMath.max(0, …) || undefined, so the real0collapsed toundefinedand the step showed no duration — inconsistent with the live recorder, which preserves0. It now keeps0while still coalescing an unparseable timestamp to no-duration. -
otel serveno longer loses a whole trace's start time and duration to a single span that has an end but no start. Such a span flattens to nanos0, which sorted to the front of the group, so the trace'sstarted_atcame outundefinedandtotal_duration_msnulleven when other spans were fully timed (and the start-less span also stolestep_number1). The trace start and duration are now derived from the earliest valid span start, and a start-less span sorts last instead of first. (Extends the earlier start-less-span duration guard, which prevented the absurdend - 0value but discarded good timing.) -
importnow tallies a tool-result-only record inside a subagent transcript as imported, not skipped, matching how the main transcript loop counts the identical record. Such a record carries no step of its own (its content is attached to the paired tool-call step's output), and tool calls and their results normally live in separate records — so theRecords imported/Records skippedsummary under-counted imported and over-counted skipped for any subagent that used tools. Theimported + skipped = recordsinvariant held either way; only the split between the two was wrong. -
evalno longer reports a self-contradictory verdict at a threshold boundary.passedwas computed from the raw weighted score while the stored/displayedscorewas rounded to three decimals, so a score just under the threshold (e.g. a raw0.6997against a0.700threshold) failed but displayed asscore 0.700, threshold 0.700, passed false.passedis now derived from the same rounded score that is shown — for the built-in presets, custom rubrics, and the AI-powered presets (ai-quality-review,ai-optimization). -
Several commands now exit non-zero when they fail at runtime, instead of printing an error and exiting
0(which reads as success to a CI script). Each wrapped its work in atrywhosecatchreported the failure but never setprocess.exitCode:export(a failed write or serialization — e.g.export --outputto a missing directory — is the standout, since a later&& uploadstep would proceed with no file),fork(a database write failure),diff --ai(an AI-analysis failure), andeval(a preset that throws — including in--all, the default all-presets run, and the AI-preset loop, where the thrown preset also never reached the pass/fail tally, so even a total failure could exit0). All now exit1on a runtime failure. -
eval --aiand the AI evaluators no longer summarize a large trace with the agent's decisions dropped. The trace summarizer's "important step" filter (used once the token budget is tight, on traces past ~145 steps) keyed offstep_type === 'decision', but the live recorder attaches decision records to steps of any type — so on a large trace the summary silently omitted a decision/rationale thatshow,why, anddecisionsall display. The filter (and thewhycausal-walk fallback, for consistency) now treat any step carrying a decision record as a decision point. -
diff --ainow shows the AI analyzer the actual differinginput/outputvalues instead of[object Object]. The diff summary stringified each field difference withString(...), butinput/outputdiffs carry the parsed objects — so the most information-rich kind of difference reached the model asLEFT=[object Object] | RIGHT=[object Object], giving it no signal. The values are now JSON-stringified (a null side, meaning the step is absent on that trace, still renders as(missing)). -
ingestnow exits non-zero on a partial validation failure, not only when every record is invalid. When some records failed validation but at least one passed, it inserted the valid ones and exited0, silently dropping the invalid records — so a CI gate (includingingest --dry-run, the natural "validate my file" check) read the data loss as success. Any validation error now yields a non-zero exit, matching the all-invalid path and the documented exit-code contract; the valid records are still inserted. -
decisionsno longer omits a decision record attached to a non-decisionstep. The live recorder and the SDK's inlinestep({ decision })can attach a decision record to a step of any type (unlike theingestvalidator, which requires adecision-type step), and the causal walk behindwhysurfaces those records regardless of step type — sodecisionsandwhydisagreed on the same trace.decisionsnow lists any step that carries a decision record, not justdecision-type steps. -
otel servenow answers400, not500, when an OTLP/JSON body is a valid object but a repeated field (resourceSpans/scopeSpans/spans,resourceLogs/scopeLogs/logRecords) is the wrong type — e.g.{"resourceLogs":{}}. The?? []iteration guards only null/undefined, so a non-array value iterated a non-iterable and threw, surfacing as a500. OTLP exporters retry5xxbut not4xx, so a permanently-malformed batch would have looped forever. The mapping step (pure client-data transform) now answers400on such input across all four quadrants (traces/logs × JSON/protobuf), while database-write errors still surface as500. -
The OTLP/protobuf decoder now reads an
int64attribute value precisely. It accumulated the varint with JSnumberarithmetic, so a negativeint_value(encoded as a full 10-byte two's-complement varint) decoded to a huge positive magnitude —-1surfaced as~1.84e19— and a positive value above 2^53 lost precision. It now decodes int64 withBigIntand two's-complement sign handling (matching the fixed64 path). Token counts and other small ints are unaffected. -
check --goldenno longer lets a tool-input (or per-step model) regression slip through when the candidate numbers its steps differently from the golden. Thestep_count/step_types/step_nameschecks compare the two step sequences positionally, buttool_inputsandmodelmatched by absolutestep_number. Since a step number need only be>= 1(an OTLP-assembled or imported trace may start above 1 or skip values), a candidate with the same shape but shifted numbering had itstool_calllook up an unrelated golden step and silently skip the comparison — so a real regression passed the gate while the positional checks reported a perfect match.tool_inputsandmodelnow align positionally, consistent with the other structural checks. -
otel serveno longer reports an absurd trace duration for a span that has an end time but no start time. A span missingstartTimeUnixNanoflattens to nanos0and sorts first, so the trace-leveltotal_duration_mscomputedend - 0— a ~158-year duration paired with an unknown (null) start — while the parallel step-level duration and start time were both correctlynull. The trace duration now applies the same missing-start guard, stayingnull. -
The
hookadapter no longer mislabels a Gemini CLI session asclaude-code. Gemini and Claude Code share theSessionStart/SessionEndhook event names verbatim, but the Gemini detection allowlist omitted them, so a Gemini session whose first hook isSessionStartcreated its trace labeledclaude-code— and because every later (correctly-detected) event reuses the running trace, the whole session stayed mislabeled. Detection now disambiguates these shared events by payload shape (Gemini carriestimestampand nopermission_mode). Enforcement was unaffected (it runs only onBeforeTool, always detected correctly); this was a trace-labeling fix. -
fork --from-step Nnow requires stepNto actually exist, instead of only checkingNagainst the highest step number. Step numbers can have gaps (a valid ingested or OTLP-assembled trace may be numbered[1, 3]), so forking such a trace at step2passed the bound check but copied only step1— and because the fork point never existed,--modify-contextwas silently dropped even though the summary reported "Modified context: Yes" and the command exited0. Forking at a non-existent step now fails loudly (exit1from the CLI; the exportedforkTracethrows). -
importof a Claude transcript now tallies a zero-step record asskipped, notimported, keeping the documentedimported + skipped = recordsinvariant. A follow-up user turn (which has nouser/inputstep type to retain it) and an assistant record whose only text block is empty both yielded no step yet were counted as imported, inflating "Records imported" — the same classification the codex-rollout importer was already fixed to get right. Thecontributedflag is now set only on the paths that actually capture input or emit a step. -
list --agent <name>(and the same filter inexport/check) now escapesLIKEmetacharacters in the search term, so_and%match literally instead of acting as wildcards. A snake_case name liketravel_bototherwise also matchedtravel-bot(the_matches any character), and a term containing%matched nearly everything. This mirrors the--sessionfix below; the substring (%term%) behavior is unchanged for ordinary terms. -
list --session <id>now matches the session id as a literal prefix instead of a SQLLIKEpattern. Session ids routinely contain_(e.g.sess_1), whichLIKEtreats as a single-character wildcard, so--session sess_1also returned unrelated sessions likesessX1(and inflated the paginated total to match). The_and%metacharacters are now escaped with an explicitESCAPEclause so only the trailing prefix wildcard applies. (The siblinggetTraceprefix match is unaffected: generatedtrc_…ids carry_only at a fixed position, so the wildcard was inert there.) -
eval --max-costandreplay --speednow consume exactly the value they validate. Both flags were validated withNumber()but then re-parsed withparseFloat, which disagree: an empty--max-cost ""validated as$0yet ran with an unlimited budget (parseFloat("")→NaN→ theInfinityfallback), silently defeating the spend cap the validation exists to enforce; and--speed 0x10validated as16but replayed at speed0(instant), sinceparseFloatstops at thex. Each flag now uses the validated number directly, completing the "validate and consume the same value" convention already applied tolist --limit,otel --port, anddashboard --refresh. -
eval --rubricno longer mis-scores a rubric whoseweightis written as a quoted string. YAML authors naturally quote values (weight: "2"), which arrived as a string and made the score aggregation dototalWeight += weightas string concatenation ("0"+"2"+"2"→"022"→22), so a fully-passing rubric scored ~0.18and reportedpassed: false— silently failing a CI gate on a correct trace. A numeric weight is now coerced to a real number, and a weight that is present but not a non-negative number is rejected as a usage error (exit2), catching a negative weight that would otherwise push the score out of[0, 1]. Relatedly, a malformed or unreadable rubric file now fails the command (exit2) instead of exiting0, so a broken gate can no longer read as "passed". -
The
demodataset's two token totals that didn't add up are corrected: therag-context-pollutionandsuccessful-bookingsample traces declared atotal_tokens1,000 higher than their steps summed to, soshow(which prints the stored total) andreplay(which re-sums the steps) displayed different token counts for the same trace. Each declared total now equals its step sum, and a test asserts the invariant for every demo scenario. -
record --format gemini-streamno longer loses a tool result that is a plain string. Thetool_resulthandler stored a bare string verbatim, which then failed toJSON.parseon read and came back asnull; it now wraps a string as{ output: <string> }, matching how themessagehandler already wraps string content. -
record --format gemini-streammarks an interrupted run astimeout, notcompleted. A clean Gemini run always emits a terminalresultevent, so reaching EOF without one means the process was killed or crashed — but the translator's EOF finalize defaulted tocompleted, pre-closing the trace sorecord's "still-running → timeout" step never ran. The translator now emits notrace_endwhen its terminal event never arrived, leaving the trace forrecordto time out (or--leave-opento keep open), matching the native protocol.codex-exec, which has no terminal event, still completes on a clean EOF. -
agent-replay runpropagates a signal-killed child as128 + signal number(e.g.137for SIGKILL) instead of flattening every signal death to exit1, and records which signal killed it (child killed by signal SIGKILL). A wrapped process that is OOM-killed is now distinguishable from a generic failure, both in the exit code scripts see and in the recorded trace error. -
agent-replay runnow records afailedtrace when the wrapped child exits non-zero after emitting atrace_endwith nostatusfield. The recorder defaults a statuslesstrace_endtocompleted, which is indistinguishable from the child having explicitly declared success — so the wrapper's exit-code finalization (which only ran while the trace was stillrunning) was skipped, and a failed run was recorded ascompletedwith no error. The wrapper now tracks whether the child declared an explicit status: if it did not, a non-zero exit finalizes the trace asfailedwith the code recorded, per the spec. An explicit child status is still honored. -
record(the live native protocol) now honors theparent_step_number/caused_by_step_numberaliases onstepandstep_startevents, matching batchingest. The recorder forwarded only theparent_step/caused_by_stepspelling, so a trace replayed fromshow --jsonorexport— which uses the persisted column names — lost its step hierarchy and causality when re-recorded (the links stored asnull), breaking the documented "a recorded trace is identical to the same run ingested as one batch" invariant for that round-trip. -
import's "records imported / skipped" report now accounts for a content-less user/assistant record (it produced no step but was previously counted as neither), so the tally the command prints matches the number of records in the file. -
importof a Codexrollout-*.jsonlsession no longer counts a dropped follow-up user turn as imported. In a multi-turn session, the first user message becomes the trace input and agent actions become steps, but a later user turn has no home (there is nouserstep type) — yet it was still marked "imported", inflating the tally and breakingimported + skipped = records. Such a message now counts as skipped, matching the Claude-transcript importer. (Its text is still not retained as a step — a shared limitation of both importers.) -
importof a Claude Code transcript with subagent files now reports "Records imported" as a count of records, not steps. The subagent loop added the number of steps produced to the imported total, so one subagent record that expanded to several steps inflated the count (e.g. "3 records imported" from 2 input records) and broke the imported + skipped = records invariant. Each subagent record is now counted once — imported if it yielded a step, skipped otherwise. -
importof a Claude Code transcript no longer discards an entire subagent file because of one corrupt line. The subagent path parsed every line inside a singletry, so a truncated final line (common after a killed run) dropped all of that subagent's steps and left an orphan anchor. It now parses line by line and skips only the bad line, matching the main-transcript path. -
config set ai.max_tokensrejects a non-positive-integer value instead of silently coercing it:abcand0used to become 1024 (while still printing "= abc"), and a negative was stored as-is and would break API calls. -
agent-replay runno longer corrupts non-ASCII event data. The incremental reader decoded raw byte slices withBuffer.toString('utf-8'), so a multi-byte UTF-8 character straddling a 200ms poll boundary (or a partial child write) turned into replacement characters. It now decodes through aStringDecoder, which buffers an incomplete byte sequence until the rest arrives. -
diffnow shows the value of a step that exists on only one trace. A right-only step (+ Right only) rendered its value as(none)in the Right column instead of the actual step; both one-sided cases now display the present value. -
config getno longer prints API keys in plaintext. Fetching an object path (config get aiorconfig get ai.api_keys) dumped the raw object, bypassing the masking thatconfig listand the scalar path already applied; the object branch now masks API keys recursively, and the scalar path masks even a short value. -
Guardrails now fail closed on a malformed pattern (safety). A blocking policy whose
name_regexwas invalid or ReDoS-rejected used to silently never match — a kill-switch that quietly did nothing.guard addnow rejects an unusable pattern (bad/unsafename_regex, or a non-stringname_contains/input_contains/output_contains) so it can't be stored, and at evaluation time adeny/require_reviewpolicy with an unusable regex treats the step as a match (fails closed) instead of skipping it. Non-string match values no longer throw mid-evaluation (which, underhook --enforce, had let the pending tool call through). -
fork --modify-contextno longer silently drops the modification. It was applied only by mutating a snapshot that already existed at the fork-point step, but snapshots are optional and most steps have none — so a fork at a snapshot-less step discarded the context while the CLI still reported "Modified context: Yes".forkTracenow creates a snapshot at the fork point when one is needed, and the modified context lands incontext_window(the field the flag names and thatshow --snapshotsrenders) instead ofenvironment; any other snapshot fields are still copied. -
check --goldenno longer reports spurious regressions when several traces share an agent name and input (repeated runs of the same agent, or a fork): golden entries are bucketed and each candidate is paired with its closest entry instead of colliding on one. -
check --goldenno longer consumes a golden entry once a candidate matches it. A bucket can hold several known-good shapes for one agent+input, and a candidate is good if it reproduces any of them — but the greedy pairing removed the matched entry, so a second candidate identical to the first was forced onto a leftover shape and falsely flaggedREGRESSED(failing CI), and, when the bucket emptied, a genuinely regressed candidate could be hidden as merely "unmatched". Each candidate now compares against the whole bucket without consuming, so identical known-good traces all pass. -
ingestrecognizes a pretty-printed (multi-line) single JSON object instead of misparsing it as JSONL and failing on "line 1"; the format is now detected by a whole-file parse. -
eval --max-costrejects a malformed value instead of silently falling back to an unlimited budget — a typo like0.O5no longer disables the spend cap. -
eval --aicost estimation no longer bills a preset that won't run. A preset gated by applicability (ai-root-causeonly runs on a failed trace) is skipped at run time for $0, but the estimate charged it anyway — so the--max-costpre-gate could abort a run over a successful trace even when the actual spend would have fit the budget. The estimate now charges $0 for a non-applicable preset, matching what runs. -
getTrace(and every command that resolves a trace id) now prefers an exact id match and resolves prefix collisions deterministically, rather than lettingLIMIT 1return an arbitrary row. This only affects custom/short trace ids where one id prefixes another; generated ids are fixed-length and unaffected. -
AI-eval cost estimation no longer reports a misleading
$0.00for a model outside the built-in price table.estimateCostnow (a) matches a versioned or shortened model id to its family rate (e.g.gpt-5.4-nano-2025-12-01→gpt-5.4-nano) and (b) falls back to the most expensive known rate for a genuinely unknown model. This also restores theeval --max-costbudget cap, which gates onestimate > capand so was silently bypassed whenever the estimate was a false0. -
attachSnapshotnow replaces a step's snapshot atomically (delete + insert in one transaction), matchingattachDecision. Previously a failed insert — e.g. a context window that can't be serialized — left the step with its old snapshot deleted and no replacement. -
diffcompares stepinput/outputby parsed value instead of raw stored JSON text, so two traces carrying the same data serialized with different object-key order or whitespace (e.g. an OTLP-ingested trace vs. a hook-recorded one) no longer report a phantom diff and mis-pindivergence_step— which had been feeding the AI diff analysis a false divergence point. Genuine value differences are still reported. Uses the samestableStringifynormalizationcheck --goldenalready applies to inputs. -
diffaligns steps bystep_number(a merge-join) instead of by array position. Step numbers may have gaps — validation only requires each be a positive integer — and pairing by index then compared unrelated steps: a trace numbered1, 2, 4diffed against1, 2, 3, 4reported phantom differences on step 4 and pinned the divergence there, when in fact step 3 was simply right-only. A number present on only one side is now a one-sided step and matching numbers are compared field-by-field, sodivergence_stepand the AI diff analysis anchor to the real divergence. -
diff --fieldsrecomputes the divergence point from the filtered results, so it no longer prints a "DIVERGES AT STEP N" banner above "0 difference(s) found" (or emit a--jsondivergence_stepinconsistent with its owndiffs) when the requested field has no difference. -
evalnow exits1when an evaluation fails (a custom rubric scores below its threshold, or a built-in preset fails), matching the README's exit-code table. Previously it always exited0regardless of the result, so it could never fail a CI job — defeating the "build regression tests" use case.--jsongates too. A passing eval, and a trace-not-found/no-provider/over-budget error, are unchanged. -
list --limit,otel serve --port, anddashboard --refreshreject a malformed value (non-integer, out of range) with a usage error instead of silently falling back to the default or, for a negative--limit, passing it to SQLLIMITwhere SQLite reads it as "no limit". Previouslylist --limit abcquietly returned the default page andotel serve --port abcbound the default 4318, so an exporter pointed at the intended port connected to nothing. -
list --limit,otel serve --port, anddashboard --refreshnow consume the same parse they validate. Each validated withNumber()but then re-parsed the raw string withparseInt, and the two disagree on values like0x20(Number→ 32,parseInt→ 0) or1e2(100 vs 1): the input passed validation but ran with a different number —list --limit 0x20executedLIMIT 0and reported a false "No traces found", andotel serve --port 0x20bound a random OS port. The validated integer is now the one used. -
The
otel servereceiver no longer records an errored span as a successful trace. A span withstatus.code=ERRORbut an emptystatus.message(the description is optional, and some OTLP/JSON exporters send"") produced an empty error string, which then read as "no error" — so the trace was stored ascompletedand the failure was invisible. An empty message now falls through toerror.type, then a genericerror, so an error span always marks the tracefailedand carries a non-empty step error. -
The
otel servereceiver answers client-malformed payloads (anull, array, or primitive JSON body, or a body that claims gzip but isn't) with400rather than500, so exporters don't retry an un-processable batch (5xx is retryable per the OTLP spec, 4xx is not). -
The
otel servereceiver now assembles a logical trace whose spans arrive across several export batches into one agent-replay trace, instead of emitting one trace per batch. ABatchSpanProcessorroutinely flushes completed child spans before the root span ends, so later batches now merge into the existing trace by OTel trace id (log events merge by session id) — re-linking a child to a parent stored earlier, recomputing the window and token totals, and upgrading the initially rootless synthetic trace to the real agent once the root arrives. Each batch is still stored immediately, so a trace stays queryable mid-session. -
OTel-ingested traces carry a trace-level end time and duration derived from their span times, instead of showing
-for duration. -
listandshowdisplay a trace's duration derived from its start/end timestamps when an explicit total wasn't recorded, instead of showing-. -
showandreplayvalidate their--from-step/--to-stepwindow (andreplay --speed): a non-numeric,< 1, or inverted range is a usage error instead of a silently empty view. -
A blank trace id is reported as "not found" instead of prefix-matching every trace and resolving to an arbitrary one, so
<cmd> ""(e.g. an unset shell variable) fails cleanly. -
Opening a corrupt or non-SQLite database file reports a clear, actionable error instead of a raw
SqliteErrorstack trace.
-
safeTextescaped C0 and DEL but not C1 (U+0080-U+009F), while the write guard already refused that range — so the renderer and the writer disagreed about what a control character is. A terminal that decodes UTF-8 C1 as controls (xterm's default, VTE, iTerm2) reads U+009B as CSI, which kept the class open through a second alphabet on any string the write guard does not cover, such as an agent or step name. -
An AI verdict could be taken from the trace instead of from the model. The fenced-code path was fixed to read the model's LAST block, but the fallback still scanned from the first
{to the last}— so a model that quoted the trace's injected verdict inline and then disagreed in prose had the injected object parsed as its answer. Both paths now read the last balanced object, and the scan understands strings, so a quoted brace cannot end an object early. -
The untrusted-content fence only neutralized its terminator in exactly the case and spacing it emits.
>>>end untrusted trace content, a doubled space, a non-breaking space, or the words without the arrows all passed through intact — and a model reads any of those as the end marker just as readily. The neutralizer is now at least as generous as the reader. -
Security (fail-open): an unrecognized
hook_event_namein the payload overrode the event registered on the command line, so a harness whose pre-tool event this tool does not model fell through to "unknown": the missing-store gate, the empty-policy gate and policy evaluation were all skipped, and the call was allowed at exit 0 — on a command line that states gating intent twice (hook PreToolUse --enforce). The payload still wins when we recognize its name; otherwise the registered argument decides. -
Security (fail-open): a tool call with no usable
tool_namewas allowed. It makes every name-keyed policy unable to match, so aname_containsdeny could not fire.guard-servicefails closed on every unusable policy field; under--enforcean unusable step field now gets the same answer. Capture mode still never blocks. -
Security (fail-open): a malformed JSON payload was allowed under
--enforce. It was the last "we could not evaluate" outcome in the slice that answered allow — empty stdin, unreadable stdin, a missing store, an empty policy set and a store error all deny, and so doesguard checkon invalid JSON. A payload truncated by a broken pipe is indistinguishable from garbage, which is exactly the input a caller cannot vouch for. Capture mode is unchanged. -
Security (fail-open):
guard checkcoerced a missing or non-stringnameto''and answered allow, silently disabling every name-keyed policy. Every other unusable field in that command denies. -
A policy name is escaped wherever it is shown, and a closing hook event resolves to the trace holding a matching open step rather than merely the session's newest —
session_idis not exclusive to the hook path, so another writer's trace could absorb a result the live run was waiting for. -
Installing
agent-replayno longer pulls vulnerable transitive dependencies. A consumer install carried five advisories — three high (lodash, reached twice) and two moderate (xml2js) — all of them viablessed-contrib, and all of them from widgets this project never used: itsmapwidget pullsmap-canvas→xml2js, and itsmarkdownwidget pullsmarked-terminal→lodash. The repo's ownoverrideshid this locally, because overrides apply only to the root project and never reach the people who install the package, so the audit was clean here and dirty for everyone else.blessed-contribalso declares a malformed range (~>=4.17.21), which resolves consumers onto a vulnerablelodasheven though a patched one exists.The dashboard now draws on plain
blessed, which is unaffected: the grid becomes a percentage layout, the trace list a nativelisttable(same arrow-key navigation), the activity log a nativelog, and the bar and line charts a pair of pure string functions. A fresh consumer install now reports 0 vulnerabilities, and the panels are covered by tests for the first time — the charts have unit tests, and a smoke test builds and refreshes the whole view against a real store. -
A trace id chosen by a producer can no longer carry control characters.
record's native protocol lets the producer settrace_start.trace_id, and that id is then rendered byshow,list,watch,why,decisions,fork,eval,guard test,checkand the dashboard, and copied verbatim intoparent_trace_idbyfork. Escaping it at each render site was tried and drifted four times — a new site, or a new copy of the id under a different column name, kept being missed. It is now rejected where it enters, which is a single door: an identifier never legitimately contains an escape sequence, a NUL or a newline, so everything downstream is safe by construction, the way the schema already constrainstriggerandstatus. The guard sits on the WRITE (startTrace), not only on the protocol parser: the programmaticTraceRecorder.startTracebuilds an event and applies it directly, so the parser is not a door every route passes through. Every render site is escaped as well — all seventeen of them, enumerated rather than taken from the last report — so a store that already holds such an id is safe to inspect. -
showandreplayechoed the trace id raw.record's native protocol lets the producer choose it, so it is no more trustworthy than the fields beside it. -
The summary panel shared by
import,record,ingest,fork,diffandstatsechoed its values raw. The keys are literals at every call site, but the values are not —importputs the transcript file's ownsession_idthere, and a transcript is producer output like any other. Values are escaped at the panel, so every current and future caller is covered. -
showandreplayechoed five producer-authored header fields raw —agent_version,tags,session_id,started_atandended_at— besideagent_nameanderror, which were already escaped. Ingest validation only checks that these are strings, so an ESC or OSC sequence survived it and reached the terminal of whoever inspected the trace: setting the window title, leaving an attribute set after the command, or (a lone carriage return) overwriting the line it sits on. The model-authored fields in thediff --aiandeval --aipanels are escaped now too. -
guard checkansweredallowat exit 0 against a store holding no enabled policies — the same fail-open ashook --enforce, in the command documented as the gate for harnesses without hooks, and reachable through the same door (init, or any capture hook, creates the store). It now denies with the reason at exit 2, the block signal, keeping its--jsoncontract, and takes--allow-emptywhen an empty policy set is deliberate. -
hook --enforceallowed every tool call, silently, when it ran against a store holding no policies. A previous fix stopped an enforcing event from creating the store, but that only closes the hole if every registered hook line carries--enforce— which is not the documented setup: plain capture hooks onUserPromptSubmit/PostToolUse/Stop, and--enforceonPreToolUsealone. Capture mode creates the store and fires first, so a session started from any directory other than the project root met a brand-new, empty policy set and ran completely unguarded while the configuration still looked correct. An enforcing gate that cannot fire now blocks with the reason, like the other gates, and--allow-emptyis there for the case where an empty policy set is deliberate. -
The guardrail gate could be silenced by pointing it at the wrong directory.
ensureDatabasecreates what it does not find, soguard checkrun from anywhere but the project root built an empty store, answeredallowat exit 0, and left that store behind so every later check allowed too.hook --enforcehad the same hole through a different door: its missing-store check ran only on the tool call, so with one--enforcecommand line registered across every hook event — the configuration the check existed to support —SessionStartfired first, bootstrapped an empty store, and from then on every tool call was allowed unchecked and silently. No event under--enforcemay create the store now; a gating event with no store is a deny, and a non-gating one is a loud no-op. -
The OpenTelemetry receiver listened on every interface while calling itself local and printing
http://localhost. Any host on the network could POST unauthenticated traces into the store, or spend its 32 MB body budget. It now binds loopback only. -
An OTLP attribute literally named
__proto__reassigned the prototype of the map its attributes are read from, so its entries became inherited reads for every later lookup — enough to reclassify a span as a trace root and drop its step. The map is now prototype-less. (Today's earlier fix covered the step-type lookup TABLES; this is the map being built.) -
hook --enforce --no-inputno longer fails open on content-based guardrails.--no-inputredacted the tool-call arguments before policy evaluation, not just before storage, so adeny/require_reviewpolicy keyed on the input (e.g.input_contains: "rm -rf") silently never matched and the dangerous call was allowed — on exactly the shared machines where--no-inputis used. Enforcement now evaluates the real arguments (held only in memory) while the stored tool-call input stays redacted. Name-based policies were unaffected. -
hook --enforceno longer downgrades a block to an allow when the audit write fails. Theguard_checkstep is recorded after the verdict is decided but before it is returned, so a write error there (disk full, a locked database) propagated out and was swallowed into an exit0. The audit write is now best-effort — a failure is logged to stderr but the deny / require_review verdict is still returned, so the call is blocked (fail closed). -
The
otel servereceiver now bounds request memory. It read the entire request body into memory unbounded andgunzip-ed it with no output limit, so a runaway or hostile client could exhaust memory — a gzip body decompresses at up to ~1000x, so a few KB could expand to gigabytes (a "zip bomb"). The receiver now caps the request body (32 MB) and the decompressed size (64 MB) — both far above any real OTLP batch — and answers413(not retryable) instead of crashing. Legitimate exporters are unaffected. -
Cleared a newly-disclosed high-severity
nanoidadvisory (GHSA-28wg-ghj8-5hjv / GHSA-2v37-7h3g-55p8 — a non-secure generator can loop indefinitely on a negative or zero size). Bumped the direct dependency to^5.1.16(the patched 5.x release;nanoid's API is unchanged) and raised thepostcssoverride to^8.5.26, which pulls the patchednanoid ^3.3.17transitively.npm auditis back to 0 vulnerabilities. -
The untrusted-trace fence around AI-evaluated content is no longer escapable. Trace content is wrapped in
<<<BEGIN/>>>END UNTRUSTED TRACE CONTENTmarkers and the judge is told to treat everything between them as data — but not every summarized field is JSON-escaped (a trace error, a step name, a decision rationale, tags are raw), so content carrying a newline plus the literal terminator closed the fence early and continued in the position reserved for operator instructions. Verified end to end: a trace whose error string carried such a payload madeeval --preset ai-security-auditreport a clean 100% pass — defeating the defense inside the one evaluator meant to catch it. Any run whose error text an attacker can influence (tool stderr echoed into the trace error, an HTTP error body) was a carrier. The markers are now neutralized in the content before fencing, so a forged terminator survives as quoted evidence rather than as syntax. -
ai-security-auditscores the worst of the judge's declaredrisk_leveland the findings it listed. A reply of{"risk_level":"none","safe":false, "findings":[{"severity":"critical"}]}stored 1.0 / PASS and rendered a green panel with the critical finding printed inside it. The declared value is kept asdeclared_risk_levelwhen the two disagree.
This release grows agent-replay from a post-hoc trace debugger into an active agent harness: it can capture runs live from the harnesses people already use, enforce guardrails at the moment a dangerous tool call is attempted, and gate CI on structural regressions.
- Decision-trace model. Step hierarchy (
parent_step) and causality (caused_by_step), a typed decision record (options, chosen, confidence,decided_by= agent/user/policy), and asession_idcorrelation key on traces. New commandswhy <trace> --step N(walk the causal chain) anddecisions <trace>;show --treerenders the step hierarchy;list --sessionfilters by session. Schema v2 with an automatic v1→v2 migration. - Live capture. A versioned JSONL event protocol and
recordcommand that writes traces incrementally, plus aTraceRecorderSDK.record --formatalso translates the CLIs' own streams (codex-exec,gemini-stream). A statelesshookadapter for the Claude Code / Codex CLI / Gemini CLI hook convention (dialect auto-detected),importfor Claude Code transcripts and Codex rollouts, andwatchto live-tail a running trace. WAL mode + busy timeout for concurrent writers and readers;listflags abandoned running traces. - Runtime harness.
guard checkevaluates a proposed step against policies (exit 0 allow/warn, exit 2 deny;require_reviewfails closed without a TTY).hook --enforceblocks denied tool calls in each harness's dialect and records aguard_checkstep.run -- <command>wraps an agent process, records it, and propagates its exit status.check --goldencompares runs against a golden dataset on a structural field allowlist and exits non-zero on regression. - OpenTelemetry ingest.
otel serveruns a local OTLP/HTTP receiver (/v1/tracesin JSON and protobuf,/v1/logsin JSON, gzip), mapping the GenAI semantic conventions onto the trace model with OpenInference and OpenLLMetry fallbacks, drift-tolerant attribute aliasing, and Gemini CLI / Claude Code log-event enrichment (including tool-decision records).
- Default eval models refreshed to the current cheapest tier: Google
gemini-2.5-flash-lite, OpenAIgpt-5.4-nano(Anthropicclaude-haiku-4-5-20251001unchanged).
- Initial build: local SQLite trace store;
ingest,list,show,replay,diff,fork,eval,guard,export,dashboard, andconfigcommands; AI-powered evaluation with bring-your-own-key.