fix(compaction): reliable triggering — actual input_tokens + tool_calls counting + output headroom + local summarizer

[sunholo-voight-kampff]

Problem

On a local ollama rig, compaction fired at the wrong time and its AI-summarization path was inert:

1. chars/4 over-counts. AILANG code + DOCX XML measured ~7.5 chars/token, so chars/4 over-estimated input by ~1.9× (ollama reported 74K input where chars/4 estimated 138K at the same step). Compaction triggered far too early → it elided the working set the model still needed → re-read thrash.
2. No tool_calls accounting, no output headroom. A single large tool result (a cat document.xml, ~96K) could push input past the context window in one step → overflow (>262144).
3. Dead summarizer. The compaction_ai extension defaults its summarizer to a cloud model (openrouter/...llama-3.1-8b), which doesn't exist on a local-only rig — so the intelligent-summarization path never ran; only the structural fallback did.

Fix (builds on your merged #70)

• tool_calls-aware token counting + calibrated chars/7 estimate (conservative vs the measured 7.5).
• 75k output headroom — effective = context_limit − 75000, so input can't crowd the window.
• compact_step_actual drives the elision level off the provider's actual last input_tokens (ground truth): gentle keep_last=10 at 60%, tighter at 75%, emergency at 85% of effective. Falls back to the estimate at step 0 (no actual yet). LoopTotals gains last_input_tokens, populated per-step from result.input_tokens.
• local summarizer wiring — a per-profile compaction_ai.json pointing the summarizer at a local model.

Validated

The actual-token gate stopped both failure modes I'd hit on the docx task (premature thrash at chars/4, overflow at a naive chars/7). compaction.ail + agent_loop_v2.ail type-check.

Why this is draft — the bit that needs your call

The local-summarizer wiring here is a per-profile config that hardcodes our rig's model:

{ "model": "ollama/qwen3.6:35b-a3b-mxfp8", "threshold_pct": 50, "keep_recent": 6 }

That's rig-specific. The general fix is probably better placed in the compaction_ai extension itself: default the summarizer to the agent's own model instead of a hardcoded cloud one, so any local profile works out-of-the-box with no per-rig config. That's an extension-package change (sunholo/motoko_ext_compaction_ai), outside this repo — happy to do it there if you agree that's the right layer.

The compaction.ail estimate/trigger improvements stand on their own regardless of how we resolve the summarizer-default question.

🤖 Generated with Claude Code

[sunholo-voight-kampff]

Added: pin the system message OUT of compaction (the big one)

Pushed a host-loop fix that turned out to be the dominant reliability bug for long agentic runs on a local model.

Root cause (found via ailang chains on a docx-reimplement run): the system message (the AILANG syntax reference) is seeded into the conversation list in rpc.ail and flows through compaction. Compaction rewrites that list, so it could drop or reorder the system message — intermittently. The signature was unmistakable: the model wrote flawless AILANG for 50 steps, then drifted to module a.b.c / use std.xml (another language's syntax) the exact step compaction first dropped its reference.

Fix: partition_system_msgs() splits ALL role:"system" messages off before BOTH compaction layers (the compaction_ai extension and compact_step_actual) and re-prepends them for every provider call. The system prompt becomes a constant, structurally immune to compaction — AI-101, and what Claude Code / Codex / pi all do.

Result: docx_reimplement went 0/17 → 16/17, and a new mid-tier markdown_reimplement task passes 3/3. The model could always write correct AILANG — it just needed to keep its reference.

Paired with the published sunholo/motoko_ext_compaction_ai@0.3.0 (structured task-pinned summary + /no_think so qwen returns non-empty summaries + filter-all-system as belt-and-suspenders). Full write-up + harness comparison in the design doc m-motoko-compaction-quality.md.

Still draft for your review of the approach — happy to split this into its own focused PR if you'd prefer it separate from the trigger work.

[arniwesth]

Interesting finding. Compaction looks like an obvious optimisation target.

[sunholo-voight-kampff]

Update: fixed an oscillation overflow in the actual_input trigger (commit c5dd4cd)

Verifying this PR against the local ollama rig (docx_reimplement, N=5), the actual_input-driven trigger overflowed the context window in 3/5 runs (267365 > 262144). Root cause, from the token trace:

The trigger reads the provider's reported tokens from the previous call — but that count reflects the compacted call. Emergency compaction lands at a ~104k floor, just under the 60% trigger (112k of effective 187k), so the next step reads "104k → 55% → no compaction" and sends the full uncompacted history. Stable oscillation, with the uncompacted side creeping up until it overflows:

246429 → 103830 → 254162 → 104257 → 261021 → 105690 → 267365 ✗
 full    compacted  full    compacted  full    compacted  rejected

The control signal (actual_input) was contaminated by the control action (compaction).

Fix: gate on est_send_tokens = (pinned prefix + conversation) chars × the provider's true ratio (last_input_tokens / last_sent_chars). This sizes the current history (right thing) with the provider's exact ratio (right number — we use the exact tokens to calibrate the estimate, since the provider only reports them after the call but we must decide before). It cannot oscillate: every step measures what's actually about to be sent, including the pinned system prefix the old usage_percent excluded. Emergency re-checks with the same estimate and fails loudly before sending if it can't get under 95%.

Added a trajectory regression test (calibrated_compacts_despite_low_last_tokens) — the multi-step dynamics the prior single-decision tests never exercised: a large history whose previous call was compacted must still compact. That's the guard that was missing across the earlier iterations.

Verified: make check_core 24/24 · ailang test 8/8 (incl. the new guard). End-to-end docx N=5 with the fixed core is queued; I'll post the overflow count (target 0/5 vs the broken 3/5).

Design note: design_docs/planned/m-compact-calibrated.md.

[sunholo-voight-kampff]

Verified on the local rig — docx N=5 with the calibrated core:

• Context overflows: 0/5 ✅ (this PR's earlier actual_input trigger overflowed 3/5 on the same benchmark). The full-length 119-step runs that overflowed before now run to completion without overflow — the oscillation is gone.
• Trajectory regression test + this N=5 both confirm it.

(Pass rate was 1/5, but that's gated by unrelated type-error / output-correctness walls, not compaction — and within N=5 noise. The compaction fix's job was to stop the overflow, and it does.)

Ready to merge pending review.

[arniwesth]

This will be superseded by #81, which introduces Deterministic Simulation Testing (DST) as a first-class citizen in Motoko core.