fix(learning): quarantine corrupt archive files instead of overwriting them#679
Merged
Merged
Conversation
…g them A schema-invalid quality_archive.json was silently replaced on the next save(), destroying every archived entry (57 entries lost on 2026-06-10 when a maintenance script wrote lastUpdated as a number). load() now renames the invalid file to <path>.corrupt-<timestamp> and logs at error level before continuing with an empty cache, so the data stays recoverable. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
simongonzalezdc
added a commit
that referenced
this pull request
Jun 10, 2026
…e cycle (#686) ## Problem The audit's last standing open: the AutonomousGardener is dormant. Deeper than "no daemon" — its outputs went nowhere: `DreamQueue` was **in-memory only**, `dequeue()` had **zero callers**, so the per-gen "Queued dream recombination task" receipts and every gardener dream evaporated at process exit. Daemonizing a planner whose plans are discarded would be dead-code theater. ## Fix — wire the loop end-to-end 1. **DreamQueue opt-in persistence** (`persistPath`): load on construct, save on every mutation; corrupt files **quarantined** (#679 norm), never overwritten; persisted `running` tasks reclaimed to `queued` (consumer-crash recovery). Without the option: in-memory, byte-for-byte previous behavior. 2. **`sinter garden tend [budget]`**: bounded, non-interactive gardener pass for daemons — runs `AutonomousGardener.cycle()` until the budget exhausts (LLM-free), then persists the DreamPlanner plan to `~/.sinter/dreams/queue.json`, deduped against already-queued pairings. Listed in `--help`. 3. **Cycle consumption**: gen #1 of each self-improve cycle dequeues the top dream and generates from a descriptor-derived theme (`dreamThemeFromTask`: averaged source descriptors → decisive axis poles → e.g. "chaotic, dense forms recombined from its own archive lineage"), keeping #664 domain routing. `complete`/`fail` persisted; the 0.65 archive floor still rules. 4. **Daemon**: runs `garden tend` (seconds, LLM-free) before each hourly cycle. ## Verification - **Live**: `garden tend` → 1 gardener cycle over 67 real cells, 8 dreams persisted. Cycle run 1: gen failed → task persisted `failed`. Cycle run 2: hydra gen consumed `dream-2` (logged `dream=dream-2-…`), scored 0.35 → task persisted `completed` with parent lineage. Queue file: `{failed:1, completed:1, queued:6}`. - **Tests**: DreamQueue persistence 6/6 (round-trip, crash reclaim, completion persistence, id continuity, corrupt-quarantine, in-memory default); domains helpers 17/17 incl. `dreamThemeFromTask` pole naming/averaging/null paths. - `node --check` + `bash -n` on all touched scripts; eslint clean on DreamQueue. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Sinter Test <liminal@example.test> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
QualityArchive.load()validates the archive JSON withsafeJsonParse(ArchiveDataSchema). On validation failure it logged 'starting fresh' and continued with an empty cache — and the nextsave()then overwrote the on-disk file, silently destroying all archived entries.This caused real data loss on 2026-06-10: 57 entries wiped because a maintenance script wrote
lastUpdatedas a number instead of an ISO string (recovered from backup).Fix
When
load()finds an existing file that fails parse or schema validation, it now renames it to<path>.corrupt-<timestamp>(quarantine) before starting fresh, and logs at error level with the quarantine path. If the rename itself fails, it logs loudly that the file WILL be overwritten on next save.Both failure modes route through the same
safeJsonParse → nullbranch, so invalid JSON syntax and schema violations are both quarantined. The ENOENT path (no file yet) is unchanged and creates no quarantine files.Tests
4 new behavioral tests in
test/learning/QualityArchive.test.ts(real temp-dir fs, no mocks — the behavior under test is file preservation on disk):lastUpdatedas number, the exact incident shape): original preserved byte-for-byte under the quarantine name, cache starts fresh, subsequentsave()does not touch the quarantineTDD red→green verified: quarantine tests failed before the fix, pass after. Pre-commit related suite: 915 tests / 63 files green.
pnpm buildOK.🤖 Generated with Claude Code