feat(D3c): chaptered narrative (story arcs instead of FIFO trim) - #39
Conversation
Additive, backward-compatible: legacy identities (no chapters key) load with an empty list. A Chapter is a compact sealed span of narrative -- groundwork for replacing the FIFO 800-char trim with story arcs.
update_narrative now seals the open narrative into a Chapter (compact summary with date span + entry count) before it would overflow MAX_NARRATIVE, and the new entry starts a fresh chapter. Long-run history is preserved as arcs instead of silently losing the oldest lines. Chapter count is capped at MAX_CHAPTERS (oldest dropped); indices stay monotonic. Tests: no seal below threshold, overflow seals + conserves total entry count, open narrative stays bounded, indices monotonic under churn.
render_preamble now surfaces recent chapter summaries as a 'Story so far' section (above the current 'Recent history'), so an agent's long-run arc -- not just the last 400 chars -- informs goal generation and pursuit. Adds a Narrative & chapters note to docs/guide/persona.md. Tests: no Story section without chapters; sealed chapter summaries render.
…bound Review follow-ups: - Side effect: chaptering made identity.narrative the *open* chapter only, so life_summary.py silently dropped sealed history from biographies. Add AgentIdentity.full_narrative() (chapter summaries + open narrative) and use it in the life summary. - Copilot: a single entry longer than MAX_NARRATIVE bypassed sealing and bloated the open narrative. Cap an over-long entry so the open narrative is now strictly <= MAX_NARRATIVE. Tightened the bound test and added a lone-oversized-entry test + full_narrative tests.
…iew) - Greptile: chapter summaries were just 'Ch1 (dates): N entries' with no semantic content. Carry goal text -- the first goal (theme) and, if different, the last (arc) -- so 'Story so far' actually informs the agent. - Greptile: narrative entries now stamp %Y-%m-%d (was %m-%d), so a chapter span across a year boundary no longer renders reversed (12-31–01-01). - Test: chapter summary includes goal text.
There was a problem hiding this comment.
Pull request overview
Implements D3c “chaptered narrative” for AgentIdentity, replacing FIFO trimming with sealing overflowing narrative into compact Chapter summaries and surfacing those summaries in prompts and life summaries.
Changes:
- Add
Chaptermodel +chaptersfield onAgentIdentity, plusfull_narrative()for retrieving the complete story (chapters + open narrative). - Update narrative maintenance to seal the open narrative into chapters on overflow (strict
MAX_NARRATIVEbound, capped chapter list). - Render chapter summaries in the identity preamble (“Story so far”) and include full narrative in life summaries; add tests/docs for the new behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
tests/agents/test_identity.py |
Adds coverage for chapter model round-trip, sealing behavior, full_narrative(), and preamble rendering. |
src/hive/world/life_summary.py |
Switches life summaries to include the full narrative (chapters + open narrative). |
src/hive/agents/identity.py |
Introduces chapters, sealing logic, preamble rendering updates, and full narrative composition. |
docs/guide/persona.md |
Documents narrative chaptering and how it appears in prompts. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
| Filename | Overview |
|---|---|
| src/hive/agents/identity.py | Core change: adds Chapter model, _seal_chapter, full_narrative(), and update_narrative rewrite. Logic is correct — bounds hold, indices are monotonic, defensive narrative-clear on empty-lines path is present, and newline normalization prevents multi-line entries from corrupting per-line counting. |
| tests/agents/test_identity.py | New test file (175 lines) covering all major paths: sealing threshold, MAX_NARRATIVE bound, oversized single entry, chapter-index monotonicity, MAX_CHAPTERS trim, goal-text in summaries, newline normalisation, full_narrative composition, and preamble rendering with and without chapters. |
| src/hive/world/life_summary.py | One-line fix: passes identity.full_narrative() instead of identity.narrative to include sealed chapter summaries in life biographies — correct fix for the regression noted in previous reviews. |
| docs/guide/persona.md | Adds a short 'Narrative & chapters' section explaining the story-arc design. Documentation accurately reflects the implementation. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["update_narrative(agent_id, goal_text, outcome)"] --> B["load identity from disk"]
B --> C{identity found?}
C -- No --> Z["return (no-op)"]
C -- Yes --> D["normalize newlines in goal_text / outcome"]
D --> E["build entry string [YYYY-MM-DD] goal: outcome"]
E --> F{len entry > MAX_NARRATIVE?}
F -- Yes --> G["truncate entry to MAX_NARRATIVE-1 + ellipsis"]
F -- No --> H{narrative non-empty AND narrative+entry+1 > MAX_NARRATIVE?}
G --> H
H -- Yes --> I["_seal_chapter(identity)"]
H -- No --> J["append entry to narrative"]
I --> I1["collect non-empty lines from narrative"]
I1 --> I2{any lines?}
I2 -- No --> I3["narrative = '' defensive clear, return"]
I2 -- Yes --> I4["compute date span started/ended"]
I4 --> I5["extract first_goal / last_goal via _entry_goal()"]
I5 --> I6["build Chapter summary"]
I6 --> I7["append Chapter to identity.chapters"]
I7 --> I8{len chapters > MAX_CHAPTERS?}
I8 -- Yes --> I9["trim: keep last MAX_CHAPTERS"]
I8 -- No --> I10["identity.narrative = ''"]
I9 --> I10
I10 --> J
J --> K["save identity to disk"]
Reviews (2): Last reviewed commit: "fix(D3c): normalize newlines in entries ..." | Re-trigger Greptile
… test Review follow-ups (PR for D3c): - Copilot (bug): goal_text/outcome with newlines (LLM output) split one entry across lines, inflating entry_count and breaking date/goal extraction. Normalize newlines to spaces so an entry is always one line. - Greptile: _seal_chapter now clears narrative on its empty early-return path, so a whitespace-only narrative can't grow past MAX_NARRATIVE. - Greptile: add a test for the MAX_CHAPTERS trim path (oldest dropped, indices stay monotonic) + a newline-normalization test.
* fix(events): split JSONL on newline only, not str.splitlines() str.splitlines() also breaks on \r, \v, \f, NEL (U+0085) and Unicode line separators U+2028/U+2029 -- all legal *unescaped* inside JSON strings. A single event whose text (e.g. ASSISTANT_MESSAGE/TOOL_RESULT) carried one was one physical line with no \n, but splitlines() shredded it, so replay() raised a spurious 'corruption' error (whole session failed) and stream() crashed. Split on \n only and rstrip \r for CRLF tolerance. Test: a record containing NEL round-trips through replay without shredding. * fix(events): fsync parent dir on new session file (durability) With fsync=True, content was fsync'd but a newly-created session file's directory entry wasn't durably linked, so a crash could lose a just-created (content-fsync'd) file. fsync the parent dir once on file creation; guarded for filesystems that don't support directory fsync. * fix(config): drop dead HIVE_MAX_TURNS / HIVE_SESSION_TIMEOUT env mappings These mapped to model.max_turns / model.session_timeout, which ModelConfig does not define -- pydantic silently ignored them, so setting either env var was a no-op. Removed the dead rows (the remaining mappings all target real fields). * fix(daemon): fresh identity in checkpoint + drop duplicate crisis line - update_narrative reloads/saves its own identity copy, so the in-memory identity passed to checkpoint.save was stale (missing the just-appended entry/chapter). Reload before snapshotting. - Skip the derived mood line when in crisis: the suffering fragment already emits the crisis directive, so 'overwhelmed' duplicated it in the prompt. * fix(store): mark only the nudges just read as delivered (by id) get_pending_nudges did SELECT (delivered=0) then UPDATE delivered=1 for the whole agent, so a nudge inserted between the two statements (e.g. by another process) was marked delivered without being returned -- silently lost. Now it updates only the nudge_ids actually read. Test: a nudge added after a read is still delivered exactly once on the next read. * docs(changelog): cover mood, chaptered narrative, narrative-in-prompt + F1 in 0.5.4 The 0.5.4 entry listed only the durability work, but #34/#35/#38/#39 all merged into this unreleased version. Add Added entries (mood model, chaptered narrative, narrative-in-prompt, F1 coverage) and a Fixed section for the review follow-ups. Mirrored in docs/changelog.md. * test(store): exercise the actual nudge SELECT-vs-UPDATE race (Greptile) The prior test only validated sequential delivery -- the old WHERE delivered=0 sweep passed it too. New test commits a competing nudge from a separate connection in the window between the SELECT and the UPDATE (via a sync execute wrapper that preserves aiosqlite's await/async-with duality). It fails against the old sweep (n2 marked delivered and lost) and passes with the by-id UPDATE.
D3c — chaptered narrative. (Supersedes #37, which auto-closed when its base #35 merged; rebased onto
main, history preserved as 5 small commits.)Replaces the FIFO 800-char narrative trim with story arcs: the open narrative seals into compact
Chaptersummaries on overflow (carrying goal text + an unambiguous%Y-%m-%ddate span), and the preamble shows a "Story so far" section.AgentIdentity.full_narrative()keeps life-summary biographies complete.Includes the review fixes from #37: strict
MAX_NARRATIVEbound (lone oversized entry capped); life-summary regression fixed; richer chapter summaries + unambiguous dates (Greptile).Verified: 1013 tests pass, ruff/format/mypy/mkdocs clean.