Skip to content

Retention and compaction policy for a 3GB history database #53

Description

@khaliqgant

Problem

ai-history.db reached 2.8GB with a 149MB WAL, on a volume that hit 98% full. The disk-full condition is what truncated .sync-state.json and started a multi-day outage (#44).

There is currently no retention, pruning, or compaction policy — the database only grows. Size has second-order effects: a large WAL takes longer to checkpoint, checkpoints hold locks longer, and long lock holds are what break sync (#49).

Proposal

  • Retention policy for history / session_events (age-based, or keep-N-per-project), with an explicit opt-in — this is the user's recall data, so deletion must be deliberate
  • ai-hist compact / vacuum command for reclaiming space after pruning
  • Report size and growth in ai-hist doctor (partly done in feat(ai-hist): read-only reads, lock diagnostics, and write guardrails #46: size, WAL, free space)
  • Consider whether session_events full-text indexes need the same retention as the rows

Related: #42 (repeated re-scan/backfill of large histories), #44, #46

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions