Skip to content

feat(D3c): chaptered narrative (story arcs instead of FIFO trim) - #39

Merged
chiruu12 merged 6 commits into
mainfrom
feat/chaptered-narrative
Jun 2, 2026
Merged

feat(D3c): chaptered narrative (story arcs instead of FIFO trim)#39
chiruu12 merged 6 commits into
mainfrom
feat/chaptered-narrative

Conversation

@chiruu12

@chiruu12 chiruu12 commented Jun 2, 2026

Copy link
Copy Markdown
Owner

D3c — chaptered narrative. (Supersedes #37, which auto-closed when its base #35 merged; rebased onto main, history preserved as 5 small commits.)

Replaces the FIFO 800-char narrative trim with story arcs: the open narrative seals into compact Chapter summaries on overflow (carrying goal text + an unambiguous %Y-%m-%d date span), and the preamble shows a "Story so far" section. AgentIdentity.full_narrative() keeps life-summary biographies complete.

Includes the review fixes from #37: strict MAX_NARRATIVE bound (lone oversized entry capped); life-summary regression fixed; richer chapter summaries + unambiguous dates (Greptile).

Verified: 1013 tests pass, ruff/format/mypy/mkdocs clean.

chiruu12 added 5 commits June 2, 2026 15:31
Additive, backward-compatible: legacy identities (no chapters key) load
with an empty list. A Chapter is a compact sealed span of narrative --
groundwork for replacing the FIFO 800-char trim with story arcs.
update_narrative now seals the open narrative into a Chapter (compact
summary with date span + entry count) before it would overflow
MAX_NARRATIVE, and the new entry starts a fresh chapter. Long-run history
is preserved as arcs instead of silently losing the oldest lines. Chapter
count is capped at MAX_CHAPTERS (oldest dropped); indices stay monotonic.

Tests: no seal below threshold, overflow seals + conserves total entry
count, open narrative stays bounded, indices monotonic under churn.
render_preamble now surfaces recent chapter summaries as a 'Story so far'
section (above the current 'Recent history'), so an agent's long-run arc --
not just the last 400 chars -- informs goal generation and pursuit. Adds a
Narrative & chapters note to docs/guide/persona.md.

Tests: no Story section without chapters; sealed chapter summaries render.
…bound

Review follow-ups:
- Side effect: chaptering made identity.narrative the *open* chapter only,
  so life_summary.py silently dropped sealed history from biographies.
  Add AgentIdentity.full_narrative() (chapter summaries + open narrative)
  and use it in the life summary.
- Copilot: a single entry longer than MAX_NARRATIVE bypassed sealing and
  bloated the open narrative. Cap an over-long entry so the open narrative
  is now strictly <= MAX_NARRATIVE. Tightened the bound test and added a
  lone-oversized-entry test + full_narrative tests.
…iew)

- Greptile: chapter summaries were just 'Ch1 (dates): N entries' with no
  semantic content. Carry goal text -- the first goal (theme) and, if
  different, the last (arc) -- so 'Story so far' actually informs the agent.
- Greptile: narrative entries now stamp %Y-%m-%d (was %m-%d), so a chapter
  span across a year boundary no longer renders reversed (12-31–01-01).
- Test: chapter summary includes goal text.
Copilot AI review requested due to automatic review settings June 2, 2026 10:02

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Implements D3c “chaptered narrative” for AgentIdentity, replacing FIFO trimming with sealing overflowing narrative into compact Chapter summaries and surfacing those summaries in prompts and life summaries.

Changes:

  • Add Chapter model + chapters field on AgentIdentity, plus full_narrative() for retrieving the complete story (chapters + open narrative).
  • Update narrative maintenance to seal the open narrative into chapters on overflow (strict MAX_NARRATIVE bound, capped chapter list).
  • Render chapter summaries in the identity preamble (“Story so far”) and include full narrative in life summaries; add tests/docs for the new behavior.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
tests/agents/test_identity.py Adds coverage for chapter model round-trip, sealing behavior, full_narrative(), and preamble rendering.
src/hive/world/life_summary.py Switches life summaries to include the full narrative (chapters + open narrative).
src/hive/agents/identity.py Introduces chapters, sealing logic, preamble rendering updates, and full narrative composition.
docs/guide/persona.md Documents narrative chaptering and how it appears in prompts.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/hive/agents/identity.py
@greptile-apps

greptile-apps Bot commented Jun 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

Replaces the previous FIFO 800-character trim on AgentIdentity.narrative with a chaptered story-arc system: when the open narrative overflows it is sealed into a compact Chapter model (carrying a date span and goal theme) rather than dropping the oldest lines. The full history is composed back for life-summary biographies via full_narrative(), and the LLM preamble gains a "Story so far" section showing the five most-recent chapter summaries alongside the current open window.

  • Chapter is a new Pydantic model appended to AgentIdentity.chapters; legacy identities (no chapters field) deserialise cleanly with an empty list.
  • _seal_chapter computes monotonically incrementing index values, builds a semantically rich summary (date span + first→last goal arc), and trims to MAX_CHAPTERS=20 by dropping the oldest when the cap is exceeded.
  • 175 new tests covering sealing, bounds, monotonicity, MAX_CHAPTERS trim, newline normalisation, preamble rendering, and full-narrative composition.

Confidence Score: 5/5

Safe to merge. The sealing logic is correct, all size bounds hold, and legacy identities without chapters deserialise cleanly.

The update_narrative rewrite correctly enforces MAX_NARRATIVE at every code path: oversized single entries are capped before the sealing check, the sealing condition accounts for the newline separator, and _seal_chapter always clears the narrative (including the defensive empty-lines path). Chapter indices are monotonically increasing even after MAX_CHAPTERS trimming. The full_narrative() fix in life_summary.py correctly restores the biography regression. The 175-test suite covers all key invariants including the previously-missing MAX_CHAPTERS trim path.

No files require special attention.

Important Files Changed

Filename Overview
src/hive/agents/identity.py Core change: adds Chapter model, _seal_chapter, full_narrative(), and update_narrative rewrite. Logic is correct — bounds hold, indices are monotonic, defensive narrative-clear on empty-lines path is present, and newline normalization prevents multi-line entries from corrupting per-line counting.
tests/agents/test_identity.py New test file (175 lines) covering all major paths: sealing threshold, MAX_NARRATIVE bound, oversized single entry, chapter-index monotonicity, MAX_CHAPTERS trim, goal-text in summaries, newline normalisation, full_narrative composition, and preamble rendering with and without chapters.
src/hive/world/life_summary.py One-line fix: passes identity.full_narrative() instead of identity.narrative to include sealed chapter summaries in life biographies — correct fix for the regression noted in previous reviews.
docs/guide/persona.md Adds a short 'Narrative & chapters' section explaining the story-arc design. Documentation accurately reflects the implementation.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A["update_narrative(agent_id, goal_text, outcome)"] --> B["load identity from disk"]
    B --> C{identity found?}
    C -- No --> Z["return (no-op)"]
    C -- Yes --> D["normalize newlines in goal_text / outcome"]
    D --> E["build entry string [YYYY-MM-DD] goal: outcome"]
    E --> F{len entry > MAX_NARRATIVE?}
    F -- Yes --> G["truncate entry to MAX_NARRATIVE-1 + ellipsis"]
    F -- No --> H{narrative non-empty AND narrative+entry+1 > MAX_NARRATIVE?}
    G --> H
    H -- Yes --> I["_seal_chapter(identity)"]
    H -- No --> J["append entry to narrative"]
    I --> I1["collect non-empty lines from narrative"]
    I1 --> I2{any lines?}
    I2 -- No --> I3["narrative = '' defensive clear, return"]
    I2 -- Yes --> I4["compute date span started/ended"]
    I4 --> I5["extract first_goal / last_goal via _entry_goal()"]
    I5 --> I6["build Chapter summary"]
    I6 --> I7["append Chapter to identity.chapters"]
    I7 --> I8{len chapters > MAX_CHAPTERS?}
    I8 -- Yes --> I9["trim: keep last MAX_CHAPTERS"]
    I8 -- No --> I10["identity.narrative = ''"]
    I9 --> I10
    I10 --> J
    J --> K["save identity to disk"]
Loading

Reviews (2): Last reviewed commit: "fix(D3c): normalize newlines in entries ..." | Re-trigger Greptile

Comment thread src/hive/agents/identity.py
Comment thread tests/agents/test_identity.py
… test

Review follow-ups (PR for D3c):
- Copilot (bug): goal_text/outcome with newlines (LLM output) split one
  entry across lines, inflating entry_count and breaking date/goal
  extraction. Normalize newlines to spaces so an entry is always one line.
- Greptile: _seal_chapter now clears narrative on its empty early-return
  path, so a whitespace-only narrative can't grow past MAX_NARRATIVE.
- Greptile: add a test for the MAX_CHAPTERS trim path (oldest dropped,
  indices stay monotonic) + a newline-normalization test.
@chiruu12
chiruu12 merged commit f1db830 into main Jun 2, 2026
6 checks passed
chiruu12 added a commit that referenced this pull request Jun 2, 2026
* fix(events): split JSONL on newline only, not str.splitlines()

str.splitlines() also breaks on \r, \v, \f, NEL (U+0085) and Unicode line
separators U+2028/U+2029 -- all legal *unescaped* inside JSON strings. A
single event whose text (e.g. ASSISTANT_MESSAGE/TOOL_RESULT) carried one
was one physical line with no \n, but splitlines() shredded it, so replay()
raised a spurious 'corruption' error (whole session failed) and stream()
crashed. Split on \n only and rstrip \r for CRLF tolerance.

Test: a record containing NEL round-trips through replay without shredding.

* fix(events): fsync parent dir on new session file (durability)

With fsync=True, content was fsync'd but a newly-created session file's
directory entry wasn't durably linked, so a crash could lose a just-created
(content-fsync'd) file. fsync the parent dir once on file creation; guarded
for filesystems that don't support directory fsync.

* fix(config): drop dead HIVE_MAX_TURNS / HIVE_SESSION_TIMEOUT env mappings

These mapped to model.max_turns / model.session_timeout, which ModelConfig
does not define -- pydantic silently ignored them, so setting either env
var was a no-op. Removed the dead rows (the remaining mappings all target
real fields).

* fix(daemon): fresh identity in checkpoint + drop duplicate crisis line

- update_narrative reloads/saves its own identity copy, so the in-memory
  identity passed to checkpoint.save was stale (missing the just-appended
  entry/chapter). Reload before snapshotting.
- Skip the derived mood line when in crisis: the suffering fragment already
  emits the crisis directive, so 'overwhelmed' duplicated it in the prompt.

* fix(store): mark only the nudges just read as delivered (by id)

get_pending_nudges did SELECT (delivered=0) then UPDATE delivered=1 for the
whole agent, so a nudge inserted between the two statements (e.g. by another
process) was marked delivered without being returned -- silently lost. Now
it updates only the nudge_ids actually read. Test: a nudge added after a
read is still delivered exactly once on the next read.

* docs(changelog): cover mood, chaptered narrative, narrative-in-prompt + F1 in 0.5.4

The 0.5.4 entry listed only the durability work, but #34/#35/#38/#39 all
merged into this unreleased version. Add Added entries (mood model,
chaptered narrative, narrative-in-prompt, F1 coverage) and a Fixed section
for the review follow-ups. Mirrored in docs/changelog.md.

* test(store): exercise the actual nudge SELECT-vs-UPDATE race (Greptile)

The prior test only validated sequential delivery -- the old
WHERE delivered=0 sweep passed it too. New test commits a competing nudge
from a separate connection in the window between the SELECT and the UPDATE
(via a sync execute wrapper that preserves aiosqlite's await/async-with
duality). It fails against the old sweep (n2 marked delivered and lost) and
passes with the by-id UPDATE.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants