fix(test): keep the microcompact scratch when a case FAILS, not when it passes - #346
fix(test): keep the microcompact scratch when a case FAILS, not when it passes#346codeslake wants to merge 9 commits into
Conversation
a120155 to
2a99715
Compare
2a99715 to
3ba84fc
Compare
Fourteen cases each made their own `mc-` dir. Thirteen removed it on the last line of the test body, so an assertion that threw skipped the removal; the fourteenth (case 11) never removed it at all, so that one leaked on the green path too. None of the fourteen was inside a try/finally — the file's three `finally` blocks belong to withEnv, silenceStderr and silenceStderrAsync, and restore env and stderr rather than remove anything. Reproduced two ways: a clean run of the parent leaves 1 dir behind (case 11), and one injected assertion failure leaves 2. After this change both are 0. Registering the dir at creation and collecting in after() puts the cleanup in one place instead of fourteen, and covers the paths that were missing rather than the thirteen that already worked. The eager `rm` calls stay: they free disk during a long run, and after() is idempotent with force. after() does not cover SIGINT, SIGKILL, or a throw at import time — measured. The first matters: Ctrl-C on a long suite still leaks. The last is vacuous here, since every mcTemp() call is inside a test body. test/proxy-wrapper.test.mjs already carries this shape; this is that pattern, not a new one. Co-Authored-By: Claude <noreply@anthropic.com>
3ba84fc to
a48b69a
Compare
…e cleanup The registrar makes `after()` the single owner of every scratch dir, so the 13 per-case `rm` calls are dead: measured, the file runs in 84 ms and the 14 dirs it mints hold 66 KB together, so "frees disk during a long run" does not hold. Two cleanup disciplines in one file is what produced the leak. Nothing enforced the registrar. A new case pasting back the raw `mkdtemp(join(tmpdir(), ...))` line strands its dir on any throw and the suite stays green, which is how the 14th case leaked here unnoticed. Add the source-level guard the sibling CA and wrapper suites already carry, with the premise assertion so it cannot pass on an empty population. Measured, isolated scratch root, 14 cases seeded to throw: base green 1 stranded dir, failing 14 branch green 0, failing 0 Co-Authored-By: Claude <noreply@anthropic.com>
The guard excluded the registrar by the text `const d = await mkdtemp`, which is a variable name, not a discriminator: a case writing `const d = await mkdtemp(join(tmpdir(), "mc-"))` was excluded as if it were the registrar. Measured with exactly that line in a case body: 36/36 pass and one dir stranded, the leak the guard exists to report. Hoist the prefix to a const so the registrar carries no string literal. The pattern already requires one, so there is nothing to except and the hole closes with the filter. Same line seeded again now fails the guard. Co-Authored-By: Claude <noreply@anthropic.com>
…backtick mints Two measured defects in the source-level guard, both from copying only half of what the sibling guards do: - A comment naming the anti-pattern reds the suite. Seeding a "never do: mkdtemp(join(tmpdir(), ...))" line into a case body failed the guard AT THAT COMMENT'S LINE. Documenting the wrong way broke the check that enforces the right way. Both sibling guards already skip comment lines; this one had kept only the self-escaping half. - A backtick-quoted prefix escaped it entirely: 37/37 green while stranding a directory, which is the exact failure the guard exists to prevent. That shape is not hypothetical, it is already in this test suite as a mkdtempSync with an interpolated tag. The quote class now covers all three JS string delimiters, and the comment filter matches the siblings. Residual ceiling stated in place: the scan is one line at a time, and a const prefix is the very thing that excludes the registrar, so that one shape cannot be closed here. Mutation-checked, four cases: raw mint caught, renamed variable caught, backtick mint caught, comment no longer flagged. File 36/36 with zero directories left under a private TMPDIR. Co-Authored-By: Claude <noreply@anthropic.com>
…onestly The comment filter covered one of JavaScript's three comment syntaxes. Measured, both other styles still turned the suite red on prose that merely NAMES the anti-pattern: /* never do: mkdtemp(join(tmpdir(), "mc-")) */ -> flagged * never do: mkdtemp(join(tmpdir(), "mc-")) -> flagged Same defect class the filter exists to close, half-closed. No file under test/ uses block or JSDoc style today, so nothing was failing; this closes the class rather than the one instance that had been measured. The stated ceiling was not wrong, it was incomplete: it named two residual shapes and there are at least five. A qualified os.tmpdir(), an extra path segment, and an interpolated tag containing a quoted string all pass too. The general sentence is both shorter and true, so the enumeration is gone. Also note at the declaration that SCRATCH_PREFIX is load-bearing. Inlining the literal makes the guard flag its own registrar, and nothing at that line said so. Guard re-verified after the change: raw, renamed-variable, backtick and single-quote mints all still caught; both comment styles now pass. File 36/36 with zero directories left under a private TMPDIR. Co-Authored-By: Claude <noreply@anthropic.com>
…ints
The previous commit widened the guard's comment filter from "//" to also
skip "/*" and "*". That was wrong, and measured wrong in two shapes that
went from CAUGHT to silently green:
/* setup
*/ const _h = await mkdtemp(join(tmpdir(), "mc-")); -> not flagged
const obj = {
*g() { const d = mkdtempSync(join(tmpdir(), "mc-")); return d; },
}; -> not flagged
The asymmetry is the whole point and the previous commit missed it:
nothing can follow "//" on a line, so skipping it can never hide code. A
block comment closes mid-line, and "*" also opens a generator method, so
skipping those lines hides real code sitting on them.
That traded a LOUD failure mode for a SILENT one. Prose that named the
anti-pattern turned the suite red: obvious, one line to fix, and nothing
leaked. A skipped mint leaks a directory with the suite green, which is
the exact failure this guard exists to catch.
The class it closed has zero instances anyway. No file under test/ uses
block or JSDoc comment style. Both sibling guards use the "//" filter,
so this also restores the cross-file consistency the branch argues for.
The ceiling sentence was false while that filter was in, since those
lines carry the exact call form the regex matches and escaped via the
line filter rather than the spelling. It is true again, and the reason
for "//" only is now stated so the next reader does not re-widen it.
Kept from the previous commit: the load-bearing note on SCRATCH_PREFIX
and the general ceiling sentence, both verified.
Re-verified: raw, renamed-variable, backtick, single-quote, block-close
and generator-method mints all caught. File 36/36, zero directories left
under a private TMPDIR.
Co-Authored-By: Claude <noreply@anthropic.com>
There was a problem hiding this comment.
Review: PR #346 microcompact scratch lifecycle
Codex review: cross-LLM review, round 1.
Date: 2026-08-26
Reviewed: test/proxy-microcompact-stability.test.mjs at 7c62f0e918623bb9d93f3bb38c9cd0025174b6ff; PR #347 diff for interaction check
Round: 1
Label applied: changes-requested
Fork PR artifact note: the PR head is codeslake/claude-code-cache-fix, so per repo policy I did not push a committed review artifact to the contributor branch or to the base repo. This formal PR review is the artifact for this round.
What Is Correct
- [Measured] CI is green on the reviewed head:
test (18),test (20),test (22), GitGuardian, and Snyk all reported success for7c62f0e918623bb9d93f3bb38c9cd0025174b6ff. - [Measured] The unchanged focused test file passes on my local runtime:
node --version->v24.11.1;node --test test/proxy-microcompact-stability.test.mjs->36/36passing. After that success run,find /tmp -maxdepth 1 -type d -name 'mc-*'returned no entries, so the success-path cleanup works for the exercised file. - [Read] The new scratch names are minted by
mkdtemp(join(tmpdir(), SCRATCH_PREFIX))with constant prefixmc-attest/proxy-microcompact-stability.test.mjs:20-23; the path does not embed the session id or other request metadata. - [Read] PR #347 is a separate launcher-test isolation change: it adds
test/file-tmpdir.mjs, setsprocess.env.TMPDIRper importing test file, and imports it in launcher/proxy tests. PR #346 does not setTMPDIR, does not add cleanup hooks outsidetest/proxy-microcompact-stability.test.mjs, and PR #347 does not touch the microcompact test file. The interaction shape is indirect only: if both changes later touched the same test process,tmpdir()would follow PR #347'sTMPDIR; today they do not overlap in files or hooks.
Blockers
-
[Measured] The failure-path scratch is still deleted, so the PR does not provide the post-mortem artifact it is meant to preserve.
Code read:
test/proxy-microcompact-stability.test.mjs:26-30registers a file-levelafter()hook that removes every registered scratch directory.node:teststill runs that hook after a failed test.Measurement: in an exported copy of the PR tree, I injected a review-only probe:
test("forced failure preserves registered scratch probe", async () => { const dir = await mcTemp(); console.error(`FORCED_SCRATCH=${dir}`); await writeFile(join(dir, "marker.txt"), "probe"); assert.fail("forced failure for review probe"); });
Running
node --test test/proxy-microcompact-stability.test.mjsfailed as intended and printedFORCED_SCRATCH=/tmp/mc-KSM045, but the post-run checks reported:STATUS=1 FORCED=/tmp/mc-KSM045 MARKER=missing DIR=missingThat is the opposite of the requested lifecycle: success scratch is cleaned, but genuine failure scratch must survive.
-
[Read] The scratch location is still the process temp directory, not a controlled post-mortem location.
The helper still calls
tmpdir()directly attest/proxy-microcompact-stability.test.mjs:22. With noTMPDIRoverride, that is/tmpon the reviewed Linux environment; the forced-failure probe printed/tmp/mc-KSM045. The random suffix avoids collisions and the path name itself does not expose secrets, but the directory is not moved off the shared temp root and the PR does not document a default or precedence for any governing environment variable.
What Needs Attention
- [Read] The guard test at
test/proxy-microcompact-stability.test.mjs:782-799catches only one textual spelling of rawmkdtemp(join(tmpdir(), "...")). It is useful as a narrow tripwire, but it should not be treated as proving that all future scratch creation is registered.
Bloat / Non-Functional
- Production LOC: 0. Test-only diff: +54/-28 in one file. New files: 0. New exports: 0. New env vars: 0. New on-disk paths: no repo/product paths; test scratch remains under
os.tmpdir(). - Bloat finding: None. The size is proportionate to a test cleanup lifecycle change.
Recommendations
- Gate cleanup on the test file's final result, or use an explicit success-only cleanup path that still runs after assertions complete but skips removal when any test in the file failed.
- If the intended destination is not
os.tmpdir(), introduce the directory-selection helper in the test code and document the default/precedence. IfTMPDIRis the intended control point, say that explicitly and ensure the preserving failure path still prints or otherwise exposes the final scratch path to the operator. - Add a regression test that forces a failing child test file or subprocess and asserts the scratch directory remains after non-zero exit. A direct in-process assertion cannot prove this lifecycle because the check must happen after
after()has run.
Bottom Line
Request changes. The success-path cleanup is measured working, but the failure path deletes the registered scratch before the operator can inspect it, and the path still lands under the process temp root unless external TMPDIR state says otherwise.
— Codex, cross-LLM review, round 1
…wn cleanup
The branch is named for keeping scratch across a failure and it did the
opposite: `after()` runs after a red case too, so the directory that holds what
the case was looking at when it died was removed before anyone could open it.
`after()` cannot know the answer. Measured on node 24: `process.exitCode` is
`undefined` inside a file-level `after()` on a file that HAS a failure, and `1`
by exit on that same file -- and `undefined` at both points on a file without
one, which is the control that makes the first reading mean something. So the
decision moves to `process.on("exit")`, where the outcome is settled, and the
removal is sync because an exit handler cannot await.
Registration moves to `test/scratch-registry.mjs`, matching `file-tmpdir.mjs`
next to it. That is also what makes the lifecycle testable: it cannot be
asserted in-process, because the decision is taken after every hook this file
could run, so the new case drives a real child both ways. The passing arm is
the control -- without it, "the directory is there" is also what a registry
that never cleans anything looks like.
Two things the guard beside it was missing, both now closed:
- a POSITIVE CONTROL. Neutering the pattern left it green over a file it no
longer matched. The sample is assembled from pieces so it is not itself a
mint for the pattern to find.
- its premise was a runtime array that this change empties. It counts call
sites in the source instead, which cannot silently go to zero.
GREEN: 37/37
mutation: always-clean kills the failing arm; never-clean kills the passing arm;
a neutered pattern kills the positive control; restored 37/37
Co-Authored-By: Claude <noreply@anthropic.com>
|
Thanks — you are right about the head you reviewed, and it is fixed at a later one. Reviewed: What changed. The file-level process.on("exit", () => {
for (const d of registered) {
if (process.exitCode) { process.stderr.write(`[scratch kept] ${d}\n`); continue; }
try { rmSync(d, { recursive: true, force: true }); } catch {}
}
});
Measured, both directions, with your own forced-failure probe shape:
The kept dir is named on stderr on the way out, since the runner's output is the Could you re-review at 🤖 Generated with Claude Code |
What a failing case leaves behind
proxy-microcompact-stability.test.mjsmints temp dirs for fixtures. A body that throws never reaches its own cleanup, so every dir is registered and removed once the file is done.That removal was an
after()hook, andafter()runs after a red case too — so the directory holding whatever the case was looking at when it died was deleted before anyone could open it. The branch is named for keeping it; it did the opposite.after()cannot know the outcomeMeasured, four runtimes, on a file that HAS a failure and on one that does not:
process.exitCodeinsideafter()11undefinedundefinedprocess.exitCodeatexit1111undefinedeverywhereThe all-green row is the control: at
exitthe value discriminates, and it does so on every runtime CI runs. Insideafter()it does not — on 22 and 24 the hook simply runs too early to know.So the decision moves to
process.on("exit"), and the removal becomes synchronous, because an exit handler cannot await and a promise-basedrmwould be registered and never run.Where it lives
Registration moves to
test/scratch-registry.mjs— one new file, one new export (scratchDir). That is also what makes the lifecycle testable: it cannot be asserted in-process, because the decision is taken after every hook this file could run. The new case drives a real child both ways, and the passing arm is the control — without it, "the directory is there" is also what a registry that never cleans anything looks like.A kept directory is named on stderr on the way out. Preserving one silently helps nobody.
Two holes in the guard beside it
The file carries a source-scanning guard: every temp dir must go through
mcTemp().Verification
37/37on the touched file, and green on 18 / 20 / 22 / 24.a failing run deleted the scratch that is the only record of what it was looking ata passing run left its scratch behind — the registry cleans nothingthe pattern no longer matches a raw mint2 files changed, +137 / −28. New files: 1. New exports: 1. New env vars: 0. New on-disk path shapes:
mcprobe-*, from the new case's child.Corrections to what is already written here
file-tmpdir.mjs"next to it". That file is on test: give each launcher-spawning file its own TMPDIR #347, not on this branch. The two are the same shape and are not yet siblings.const scratch = []+after(...)block as "the change". That block is what the change deletes.