Fast Sessions

[cristinaponcela]

We want to move to a place where we can support SQLite session storage, though jsonl file format will still be default. We have 3 main concerns right now regarding sessions, and the speed at which we load/ search them. Clanker assisted:

Context

Sessions live under .pi/agent/sessions/<encoded-cwd>/...jsonl (and nit: foo-bar gets encoded to foo/bar see #4877). SessionManager.list() / listAll() fully parse every session file via buildSessionInfo(). This is suboptimal.

SessionInfo contains fields firstMessage, allMessagesText, which are unbounded and blow up latency when resuming sessions (see #5556).

buildSessionContext() doesn't stop at the last compaction and instead walks the entire path (applies the latest compaction semantics afterward, only to construct the final output messages but not to restrict parsing).

3 separate problems:

1. unecessary parsing for listing and searching
2. searching adds latency to resume
3. unnecessary parsing beyond compaction

1. Session listing metadata should not require full-file parse

We should:

• optimize SessionManager.list() / listAll() (or at least bound firstMessage and give some sort of search budget to allMessages?)
• avoid loading allMessagesText eagerly

Possible approaches:

• Only parse metadata lazily:
  • header
  • latest session_info
  • first user message
  • last activity timestamp
• Defer allMessagesText to on-demand
• Sort by file stat.mtime / derived modified desc before deeper work

Breakage surface:

• SessionInfo shape if allMessagesText becomes optional
• Any caller assuming list() returns fully searchable content immediately

What gets better:

• --resume / list rendering becomes faster
• listAll() scales better

2. Search in pi -r should be deferred and ordered by recency first

• Right now we fully prepare search body on selector open
• We should prioritize most recent sessions immediately
• Search only when user types (on-demand)

Possible approaches:

• on open:
  • load cheap metadata only
  • sort desc by modified
• on query:
  • hydrate searchable text incrementally
  • search newest-first
  • maybe stop early for first N matches, then continue in background
• optionally cache search text in memory for the selector lifetime

Breakage surface:

• mostly UX semantics, not file/API
• search result ordering may change
• fuzzy/relevance behavior may change

What gets better:

• Ctrl/interactive resume should feel instant
• pi -r no longer blocks on global search body construction
• most users get recent sessions fast, which I would guess is usually what they want

3. Compaction/runtime context reconstruction should not require replaying more than necessary

Current problem:

• buildSessionContext() walks the active branch and uses firstKeptEntryId.
• Because sessions are tree-flattened, append-only jsonl, file order is append order, not guaranteed semantic branch order.
• We can't depend on last compaction entry bc we need other values in state, which is why we walk the path. We'd need the session-level state that currently comes from replaying entries before that point, e.g.:
  • effective model
  • effective thinking level
  • the exact kept context slice, or enough to rebuild it directly
  • which entries/messages are still live after compaction
  • the active leaf/branch lineage from that point onward, or enough ancestry info to follow only the post-compaction branch

Possible approaches:

A. Keep current API, add checkpoints, or add state values to compaction entries

We could add periodic snapshot/checkpoint entries that serialize resolved context state after compaction (or add these values to compaction itself).

Example:

• compaction entry
• optional checkpoint entry containing:
  • latest thinking level
  • latest model
  • kept message ids
  • maybe summarized branch state

Pros:

• no need to reinterpret old branch prefix every time
• append-only compatible

Cons:

• new entry type
• more complexity
• still not ideal for arbitrary search

Breakage:

• additive if new entry type is ignored by old readers, higher if adding to compaction
• low if buildSessionContext() is updated compatibly

B. Introduce compaction generation ids

Each entry after a compaction gets generation: n, incremented on compaction.

Pros:

• quick filtering of pre/post-compaction entries

Cons:

• does not solve branch ordering by itself
• schema change on every new entry
• old files need fallback
• callers may start depending on it

Breakage:

• medium API/schema break if exposed as required
• migration burden

C. Persist a branch-local resolved context cache

Not in jsonl schema initially; use sidecar cache keyed by:

• session path
• leaf id
• latest compaction id
• file size/mtime

Pros:

• zero jsonl schema break
• good interim step before sqlite
• easy to invalidate

Cons:

• cache invalidation logic
• less portable than pure jsonl

Breakage:

• minimal, internal only

What gets better:

• session open/resume can avoid reconstructing from scratch every time
• preserves current append-only tree model
• creates a bridge to sqlite without redesigning semantics twice

##API breakage summary

Low breakage

• recent-first loading
• deferred search
• internal caches
• sidecar metadata files

Medium breakage

• making SessionInfo.allMessagesText optional/lazy
• adding new session entry types like checkpoint

High breakage

• replacing firstKeptEntryId semantics/ adding more info to compaction entries
• requiring generation counters on entries
• relying on file order for logical order
• changing flattened jsonl assumptions

An honorable mention xD:

A fourth concern is async persistence compatibility. Supporting SQLite cleanly likely means a number of session storage operations can no longer remain synchronous or fire-and-forget. Today, parts of session creation, append/persist, listing/loading, branching/forking, deletion/rename, and any codepaths that assume immediate on-disk visibility would need to support async writes or become fully async themselves. That expands the surface area beyond storage internals into callers of SessionManager and related resume/session-switch flows, because methods that currently return values immediately may need to return promises instead. This is an API break.

[Rohit-Bhardwaj10]

hey @cristinaponcela i would to have a workaround this , can you assign this to me