fix(context): cap tokenizer input to prevent quadratic hang

[harsh20048]

Problem

estimateTokens() in src/agent/context.ts passes unbounded text to the js-tiktoken BPE encoder. BPE cost is superlinear in the length of an unbroken run of a single character, so pathological input stalls the event loop indefinitely.

Same length, different content (cl100k_base):

chars	repeated "x"	prose
5,000	1,218 ms	1 ms
20,000	18,385 ms	3 ms
50,000	113,194 ms	6 ms

Normal text is completely unaffected — this only bites on long unbroken runs.

It hangs the test suite

src/__tests__/context-hardening.test.ts:166 builds a 500,000-char string, which extrapolates to hours. vitest's 30s testTimeout cannot fire, because the spin is synchronous and blocks the event loop. vitest run therefore never terminates — one worker sits at ~100% CPU indefinitely.

It is reachable in production

estimateTurnTokens() tokenises tc.result before truncateToolResult() is applied, and the exec tool returns stdout uncapped (src/agent/tools.ts:143). A base64 blob, minified asset, or long separator bar in tool output can freeze the agent loop.

Fix

Sample a 2,000-char prefix and scale linearly, keeping the existing Math.max() floor against the character heuristic.

• Worst case: unbounded → ~187 ms
• Measured error on realistic text: 0.2% (12,003 actual vs 12,030 estimated over 60k chars)

Verification

• src/__tests__/context-hardening.test.ts: hangs indefinitely → 29 passed in 1.14s
• Full suite: 64 files / 1,643 tests pass, and now actually completes (~174s)
• No new tests needed — the existing suite already covered this; it just could never finish

Reproduction:

const { getEncoding } = require("js-tiktoken");
const enc = getEncoding("cl100k_base");
console.time("50k"); enc.encode("x".repeat(50_000)); console.timeEnd("50k");

🤖 Generated with Claude Code