release: v0.5.4 -- durability & dependency hardening (C3/C4/C5/E3) (#33)

* feat(C5): config-gated event-log fsync + partial-line tolerance

- EventLog(fsync=True) flushes + os.fsync()'s each append for crash
  durability; defaults off to protect the hot heartbeat write path.
- Gated by HiveConfig.event_log_fsync (env HIVE_EVENT_LOG_FSYNC); daemon
  threads it into its EventLog.
- replay() tolerates a torn/partial last line; stream() consumes only
  complete (newline-terminated) lines, leaving a partial tail for the
  next poll. Mirrors the TFIDFBackend partial-line pattern.
- Document the append-only guarantee on EventLog.
- Tests: fsync round-trip, partial-last-line replay, env-var bool parse.

* feat(C3): ON DELETE CASCADE on agent child tables + delete_agent

- Add ON DELETE CASCADE to every child-table FK to agents (sessions,
  goals, nudges, schedules, sub_agents, tasks, alarms) in _SCHEMA, and a
  user_version 1->2 table-rebuild migration to add it to existing DBs.
- New HiveStore.delete_agent(): deletes an agent and cascades to all its
  child rows (no more orphans). It is the one place FK enforcement is
  turned on.
- Centralize connections behind _connect(); foreign_keys stays OFF by
  default (preserves writing child rows for non-persisted agents, a valid
  standalone-toolkit pattern) and is opt-in via _connect(foreign_keys=True).
- initialize() runs schema+migrations with FKs off so the table rebuild's
  DROP/RENAME can't trip referential checks.
- Tests: cascade delete, missing-agent returns False, v1->v2 migration
  round-trip preserving data with cascade then live.

* feat(C4): WAL journaling + explicit busy timeout for concurrency

- initialize() switches the DB to WAL (persistent in the header), so
  readers and a single writer no longer lock each other out across the
  daemon's concurrent cycles and other processes (MCP server, CLI).
- Every connection sets an explicit 5s busy_timeout so contention queues
  instead of erroring with 'database is locked'.
- Deliberately NOT pooling/reusing a single connection: aiosqlite's
  worker thread is non-daemon, so a long-lived shared connection without
  teardown at all ~20 call sites would block interpreter exit. WAL
  delivers the concurrency win without that lifecycle hazard.
- Tests: WAL is enabled persistently for fresh connections; 40 concurrent
  writers complete with no lock errors. Stress harness: 7/7, 1200
  concurrent writes 0 db-locked.

* build(E3): cap fast-moving deps at next major

Add conservative upper bounds so a breaking major can't silently enter
on a fresh install, while keeping minimums loose for downstream compat:
- anthropic >=0.40,<1   (0.x pre-1.0)
- httpx     >=0.27,<1
- openai    >=1.30,<3   (currently 2.x; block v3)
- mcp       >=1.27,<2
- pydantic  >=2.5,<3

Stable/slow-moving deps (typer, rich, aiosqlite, pyyaml, etc.) keep
minimum-only pins. Lockfile refreshed.

* release: v0.5.4 -- durability & dependency hardening

Bump 0.5.3 -> 0.5.4. Bundles C5 (event-log fsync), C3 (FK cascades +
delete_agent), C4 (WAL + busy timeout), E3 (dep upper bounds).

- CHANGELOG.md + docs/changelog.md: 0.5.4 entry.
- docs/guide/architecture.md: WAL/cascade/fsync notes in Performance &
  Persistence.
- docs/getting-started/cli-quickstart.md: event_log_fsync config option +
  HIVE_EVENT_LOG_FSYNC env var.
- uv.lock synced to 0.5.4.

* fix: address Greptile review on PR #33 (migration crash-safety + log corruption)

P1 -- migration crash window (store.py): a crash after DROP TABLE {table}
but before the RENAME left {table} gone and {table}_new holding the only
data; a re-run's executescript(_SCHEMA) recreated {table} empty, masking
the loss, then the rebuild dropped {table}_new -> data loss. Add
_recover_interrupted_rebuilds() that finishes an interrupted swap (renames
the orphaned {table}_new back) and runs BEFORE _SCHEMA, so the dropped
table can't be silently recreated empty. _migration_2 reverts to the
simple rebuild (recovery handled upstream). Regression test simulates the
exact mid-swap on-disk state and asserts no data loss.

P2 -- over-broad except in events.py: replay() caught Exception for every
line, silently hiding mid-log corruption. Now it tolerates a parse error
ONLY on a torn final line (file not newline-terminated) and re-raises
otherwise; the catch is narrowed to ValueError (covers JSON + pydantic
ValidationError). stream() only ever sees complete lines, so its broad
catch is removed -- malformed complete lines now surface. Regression test
asserts mid-log corruption raises.

963 tests pass; ruff/format/mypy clean.

* docs: address review observations on PR #33

- Changelog (both copies): the '1200 concurrent writes' figure came from
  the local (gitignored) stress harness, not the committed test (40
  writers). Reword to cite both so the claim is reproducible from the repo.
- pyproject: clarify the anthropic<1 cap -- anthropic is still 0.x (breaks
  at minor bumps), so <1 mainly guards the eventual 1.0; note to revisit
  when 1.0 ships. No pin change (caps still include all locked versions).

Doc/comment only -- no code or dependency-resolution change.

Author: chiruu12

CHANGELOG.md

@@ -1,5 +1,36 @@
 # Changelog
 
+## [0.5.4] — 2026-06-01
+
+Durability & dependency-hardening release. All changes are additive and
+backward compatible; existing databases upgrade automatically on first open.
+
+### Added
+- **Event-log fsync durability (C5)** — `EventLog(fsync=True)` flushes and
+  `os.fsync()`s every append so a power/OS crash cannot lose an acknowledged
+  event. Gated by the new `event_log_fsync` config option (env
+  `HIVE_EVENT_LOG_FSYNC`), default off to protect the hot heartbeat write path;
+  the daemon honors it. Reads now tolerate a torn/partial last line.
+- **`HiveStore.delete_agent()`** — deletes an agent and, via the new cascade,
+  all of its child rows in one call.
+
+### Changed
+- **FK cascades (C3)** — every child table's foreign key to `agents`
+  (`sessions`, `goals`, `nudges`, `schedules`, `sub_agents`, `tasks`, `alarms`)
+  now declares `ON DELETE CASCADE`. A `user_version` 1→2 migration rebuilds
+  existing tables to add it (data preserved). Foreign-key enforcement is opt-in
+  per operation (on for `delete_agent`), so writing child rows for
+  not-yet-persisted agents keeps working.
+- **WAL journaling (C4)** — the store now runs in WAL mode (set persistently on
+  initialize) with an explicit 5s busy timeout, so readers and a writer no
+  longer lock each other out across the daemon's concurrent cycles and other
+  processes (MCP server, CLI). Verified with concurrent writers and no
+  "database is locked" errors (40 in the test suite; 1200 in the local stress
+  harness).
+- **Dependency upper bounds (E3)** — fast-moving deps are capped below their
+  next major (`anthropic<1`, `httpx<1`, `openai<3`, `mcp<2`, `pydantic<3`) so a
+  breaking release can't silently enter; minimums stay loose.
+
 ## [0.5.3] — 2026-06-01
 
 ### Fixed

docs/changelog.md

@@ -1,5 +1,18 @@
 # Changelog
 
+## [0.5.4] -- 2026-06-01
+
+Durability & dependency-hardening release. All changes are additive and backward compatible; existing databases upgrade automatically on first open.
+
+### Added
+- **Event-log fsync durability (C5)**: `EventLog(fsync=True)` flushes and `os.fsync()`s every append so a power/OS crash can't lose an acknowledged event. Gated by the `event_log_fsync` config option (env `HIVE_EVENT_LOG_FSYNC`), default off to protect the hot heartbeat write path; the daemon honors it. Reads tolerate a torn/partial last line.
+- **`HiveStore.delete_agent()`**: deletes an agent and, via cascade, all of its child rows in one call.
+
+### Changed
+- **FK cascades (C3)**: every child table's foreign key to `agents` (`sessions`, `goals`, `nudges`, `schedules`, `sub_agents`, `tasks`, `alarms`) now declares `ON DELETE CASCADE`; a `user_version` 1->2 migration rebuilds existing tables to add it (data preserved). FK enforcement is opt-in per operation, so writing child rows for not-yet-persisted agents keeps working.
+- **WAL journaling (C4)**: the store runs in WAL mode (persistent) with a 5s busy timeout, so readers and a writer don't lock each other out across concurrent cycles and other processes. Verified with concurrent writers and no "database is locked" errors (40 in the test suite; 1200 in the local stress harness).
+- **Dependency upper bounds (E3)**: fast-moving deps capped below their next major (`anthropic<1`, `httpx<1`, `openai<3`, `mcp<2`, `pydantic<3`); minimums stay loose.
+
 ## [0.5.3] -- 2026-06-01
 
 ### Fixed

docs/getting-started/cli-quickstart.md

@@ -111,6 +111,8 @@ economy:
 suffering:
   threshold_crisis: 0.90
   max_stressors: 5
+
+event_log_fsync: false  # fsync every event-log append (crash-durable, slower)
 ```
 
-Override with environment variables: `HIVE_HEARTBEAT`, `HIVE_DEFAULT_MODEL`, `HIVE_STARTING_BALANCE`.
+Override with environment variables: `HIVE_HEARTBEAT`, `HIVE_DEFAULT_MODEL`, `HIVE_STARTING_BALANCE`, `HIVE_EVENT_LOG_FSYNC`.

docs/guide/architecture.md

@@ -116,6 +116,18 @@ src/hive/
   SQLite's `PRAGMA user_version`. Schema changes are ordered migration steps applied in
   a single transaction on `initialize()`, so an older database upgrades in place without
   data loss.
+- **WAL journaling + cascades.** The store runs in WAL mode (set once on `initialize()`,
+  persistent in the DB header) with a 5s busy timeout, so readers and a writer don't lock
+  each other out across concurrent agent cycles or other processes (the MCP server, the
+  CLI). Child tables (`sessions`, `goals`, `tasks`, ...) declare `ON DELETE CASCADE` on
+  their foreign key to `agents`, so `HiveStore.delete_agent()` removes an agent and all
+  of its rows together. FK enforcement is opt-in per operation, so writing child rows for
+  a standalone agent that was never persisted to `agents` still works.
+- **Crash-durable event log.** The JSONL session log is append-only by construction.
+  Setting `event_log_fsync: true` (env `HIVE_EVENT_LOG_FSYNC`) flushes and `fsync()`s
+  every append so an acknowledged event survives a power/OS crash; it defaults to off
+  because that is one `fsync` per event on the hot write path. Readers tolerate a torn
+  last line from an interrupted append.
 
 ## Extension Points
 

pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "hive-agent"
-version = "0.5.3"
+version = "0.5.4"
 description = "Local-first agent OS. Spawn persistent AI agents that collaborate, write code, and use tools autonomously."
 authors = [{ name = "chiruu12" }]
 license = { text = "MIT" }
@@ -21,15 +21,19 @@ classifiers = [
 dependencies = [
     "typer>=0.12",
     "rich>=13.0",
-    "httpx>=0.27",
+    # Upper bounds on fast-moving deps cap the next major so a breaking release
+    # can't silently enter; minimums stay loose for downstream compatibility.
+    # anthropic is still 0.x (breaking changes land in minor bumps), so its <1
+    # cap mainly guards the eventual 1.0 -- revisit/loosen when 1.0 ships.
+    "httpx>=0.27,<1",
     "aiosqlite>=0.20",
     "pyyaml>=6.0",
-    "pydantic>=2.5",
-    "anthropic>=0.40",
-    "openai>=1.30",
+    "pydantic>=2.5,<3",
+    "anthropic>=0.40,<1",
+    "openai>=1.30,<3",
     "python-dotenv>=1.0",
     "python-dateutil>=2.8",
-    "mcp>=1.27",
+    "mcp>=1.27,<2",
     "beautifulsoup4>=4.14.3",
     "tqdm>=4.67.3",
 ]

src/hive/__init__.py

@@ -1,6 +1,6 @@
 """Hive - Autonomous agent OS."""
 
-__version__ = "0.5.3"
+__version__ = "0.5.4"
 
 from hive.agents.existence import ExistenceLoop
 from hive.agents.goal_strategy import Goal, GoalContext, GoalStrategy

src/hive/config.py

@@ -1,6 +1,7 @@
 """Central configuration — all tunables in one place."""
 
 import os
+from collections.abc import Callable
 from pathlib import Path
 from typing import Any
 
@@ -38,6 +39,11 @@ def get_env(key: str, default: str = "") -> str:
     return dot.get(key) or os.environ.get(key, default)
 
 
+def _parse_bool(value: str) -> bool:
+    """Parse a truthy env-var string (bool('false') is True, so we can't use it)."""
+    return value.strip().lower() in {"1", "true", "yes", "on"}
+
+
 class SufferingConfig(BaseModel):
     threshold_prominent: float = 0.35
     threshold_constrained: float = 0.55
@@ -163,6 +169,8 @@ class HiveConfig(BaseModel):
     model: ModelConfig = ModelConfig()
     profiles_dir: str = ""
     logs_dir: str = "logs"
+    # fsync every event-log append for crash durability (one fsync per event).
+    event_log_fsync: bool = False
 
     @classmethod
     def load(cls, hive_dir: Path | None = None) -> "HiveConfig":
@@ -176,7 +184,7 @@ def load(cls, hive_dir: Path | None = None) -> "HiveConfig":
                     file_data = yaml.safe_load(f) or {}
                 data.update(file_data)
 
-        env_map = {
+        env_map: dict[str, tuple[str, str | None, Callable[[str], Any]]] = {
             "HIVE_HEARTBEAT": ("daemon", "heartbeat", int),
             "HIVE_MAX_RETRIES": ("daemon", "max_retries", int),
             "HIVE_DEFAULT_MODEL": ("model", "default_model", str),
@@ -185,6 +193,7 @@ def load(cls, hive_dir: Path | None = None) -> "HiveConfig":
             "HIVE_STARTING_BALANCE": ("economy", "starting_balance", float),
             "HIVE_PROFILES_DIR": ("profiles_dir", None, str),
             "HIVE_LOGS_DIR": ("logs_dir", None, str),
+            "HIVE_EVENT_LOG_FSYNC": ("event_log_fsync", None, _parse_bool),
         }
 
         for env_key, (section, field, cast) in env_map.items():

src/hive/daemon/loop.py

@@ -68,7 +68,7 @@ def __init__(
         self._economy_enabled = cfg.economy.enabled
         self._running = False
         self._store = HiveStore(hive_dir / "hive.db")
-        self._events = EventLog(hive_dir)
+        self._events = EventLog(hive_dir, fsync=cfg.event_log_fsync)
 
         world = None
         if self._economy_enabled:

src/hive/memory/events.py

@@ -2,6 +2,7 @@
 
 import asyncio
 import json
+import os
 from collections.abc import AsyncIterator
 from datetime import UTC, datetime
 from enum import StrEnum
@@ -45,11 +46,21 @@ def from_jsonl(cls, line: str) -> "HiveEvent":
 
 
 class EventLog:
-    """Append-only JSONL event stream for agent sessions."""
+    """Append-only JSONL event stream for agent sessions.
 
-    def __init__(self, hive_dir: Path):
+    Each session is one file opened only in append mode ("a"); events are never
+    rewritten or removed in place, so the log is append-only by construction.
+
+    With ``fsync=True`` every append is flushed and ``os.fsync``'d before
+    returning, so a power/OS crash cannot lose an acknowledged event. This costs
+    one fsync per event, so it defaults to off (it runs off the event loop via a
+    worker thread, but the syscall is still on the per-event write path).
+    """
+
+    def __init__(self, hive_dir: Path, fsync: bool = False):
         self._sessions_dir = hive_dir / "sessions"
         self._sessions_dir.mkdir(parents=True, exist_ok=True)
+        self._fsync = fsync
 
     def _session_path(self, agent_id: str, session_id: str) -> Path:
         agent_dir = self._sessions_dir / agent_id
@@ -64,16 +75,34 @@ async def append(self, event: HiveEvent) -> None:
     def _write_line(self, path: Path, line: str) -> None:
         with open(path, "a") as f:
             f.write(line)
+            if self._fsync:
+                f.flush()
+                os.fsync(f.fileno())
 
     async def replay(self, agent_id: str, session_id: str) -> list[HiveEvent]:
         path = self._session_path(agent_id, session_id)
         if not path.exists():
             return []
         text = await asyncio.to_thread(path.read_text)
+        if not text:
+            return []
+        # A complete append always ends in "\n"; if the file doesn't, the final
+        # line is a torn/half-written record we tolerate. Earlier lines (and a
+        # final line on a newline-terminated file) must parse -- a failure there
+        # is real corruption and is surfaced, not silently dropped.
+        ends_clean = text.endswith("\n")
+        lines = text.splitlines()
         events = []
-        for line in text.strip().splitlines():
-            if line.strip():
+        for idx, line in enumerate(lines):
+            if not line.strip():
+                continue
+            try:
                 events.append(HiveEvent.from_jsonl(line))
+            except ValueError:
+                # pydantic ValidationError / JSON errors subclass ValueError.
+                if idx == len(lines) - 1 and not ends_clean:
+                    continue  # torn final write -- expected, skip it
+                raise
         return events
 
     async def stream(self, agent_id: str) -> AsyncIterator[HiveEvent]:
@@ -91,11 +120,20 @@ async def stream(self, agent_id: str) -> AsyncIterator[HiveEvent]:
 
         while True:
             text = await asyncio.to_thread(path.read_text)
-            lines = text[offset:].strip().splitlines()
-            for line in lines:
-                if line.strip():
+            chunk = text[offset:]
+            # Only consume up to the last newline; a trailing partial line (an
+            # in-progress append) is left for the next poll once fully flushed.
+            nl = chunk.rfind("\n")
+            if nl != -1:
+                complete = chunk[: nl + 1]
+                for line in complete.splitlines():
+                    if not line.strip():
+                        continue
+                    # These are complete (newline-terminated) lines, so a parse
+                    # error is real corruption -- let it surface rather than
+                    # silently dropping events.
                     yield HiveEvent.from_jsonl(line)
-            offset = len(text)
+                offset += len(complete)
             await asyncio.sleep(0.3)
 
     async def list_sessions(self, agent_id: str) -> list[str]: