feat(output-shaper): learn verbosity per-user + counterfactual savings + dashboard

Phase 2 of output-token reduction, built on the Phase 1 shaper:

- learn --verbosity: mine Claude Code transcripts for behavioral signals
  (interrupts, length-adaptive fast-skips, echo ratio), recommend a verbosity
  level (heuristic prior + optional --llm-judge), and seed the savings baseline.
  Real data: 11% interrupt / 26% fast-skip -> L3, high confidence.
- Counterfactual savings estimator (output_savings.py): per-stratum synthetic
  control (estimated) + A/B holdout (measured), signed-delta aggregate with a
  propagated 95% CI. Conversation-stable arm assignment (clean A/B + cache-safe).
- AIMD verbosity controller: additive-increase / fast-back-off state machine
  with hysteresis; live signal emission gated off by default.
- Shaper resolves the learned level (env > controller > learned > default);
  recording rides the existing transforms_applied label channel through the
  outcome funnel, so all response paths feed the ledger with no RequestOutcome
  changes.
- CLI: headroom output-savings (reduction % + CI, measured vs estimated).
- Dashboard: 'Output Tokens Saved' hero card (count, %, CI, measured/estimated).
- Docs: simple-words user guide + design doc with the counterfactual methodology.
- Tests: 94 new (estimator stats, signal extraction, controller, CLI), all green;
  44 existing outcome/dashboard tests still pass.

Opt-in: HEADROOM_OUTPUT_SHAPER=1; HEADROOM_OUTPUT_HOLDOUT=0.1 for a measured number.

Author: chopratejas

README.md

@@ -52,6 +52,7 @@ Headroom compresses everything your AI agent reads — tool outputs, logs, RAG c
 - **MCP server** — `headroom_compress`, `headroom_retrieve`, `headroom_stats` for any MCP client
 - **Cross-agent memory** — shared store across Claude, Codex, Gemini, auto-dedup
 - **`headroom learn`** — mines failed sessions, writes corrections to `CLAUDE.md` / `AGENTS.md`
+- **Output token reduction** — trims what the model *writes back* (not just what you send): drops ceremony/restated code and skips deep "thinking" on routine steps. See [Output token reduction](#output-token-reduction-cut-what-the-model-writes-back).
 - **Reversible (CCR)** — originals are cached for retrieval on demand
 
 ## How it works (30 seconds)
@@ -123,6 +124,54 @@ Granular extras: `[proxy]`, `[mcp]`, `[ml]`, `[code]`, `[memory]`, `[relevance]`
 
 Reproduce: `python -m headroom.evals suite --tier 1` · [Full benchmarks & methodology](https://headroom-docs.vercel.app/docs/benchmarks)
 
+## Output token reduction (cut what the model writes back)
+
+Everything above shrinks the prompt you **send**. But you also pay for every
+token the model **writes back** — and on Opus-class models output costs 5× input.
+A lot of that output is waste: "Great, let me…" preambles, re-printing code you
+just showed it, and deep "thinking" on routine steps like reading a file.
+
+Headroom can trim that too, from the proxy, without you changing any code:
+
+- **Verbosity steering** — appends a short "be terse, don't restate context"
+  note to the end of the system prompt (so your prompt cache still hits).
+- **Effort routing** — when a turn is just the model resuming after a tool result
+  (a file read, a passing test), it dials the model's thinking effort down. New
+  questions and errors keep full effort.
+
+Turn it on:
+
+```bash
+export HEADROOM_OUTPUT_SHAPER=1     # off by default
+headroom proxy --port 8787
+```
+
+**Learn the right terseness for you.** People don't *say* how terse they want
+answers — they *show* it (they interrupt long replies, or move on before they
+could have read them). `headroom learn --verbosity` reads your past sessions and
+picks the level automatically:
+
+```bash
+headroom learn --verbosity            # preview what it found (dry run)
+headroom learn --verbosity --apply    # save it; the proxy uses it from now on
+```
+
+**See how many output tokens you saved.** Output savings are *counterfactual* —
+we never see what the model *would* have written — so Headroom reports an honest
+**estimate with a confidence range**, never a made-up number:
+
+```bash
+headroom output-savings
+# Reduction: 31.7%  (95% CI 27.7% … 35.7%)   [estimated]
+```
+
+Want a *measured* number instead of an estimate? Leave 10% of conversations
+unshaped as a control group: `export HEADROOM_OUTPUT_HOLDOUT=0.1`. The dashboard
+shows an **Output Tokens Saved** card next to input compression, labelled
+`measured` or `estimated` with the confidence band.
+
+→ Full write-up incl. the measurement methodology: [`docs/proposals/output-token-reduction.md`](docs/proposals/output-token-reduction.md)
+
 <a href="https://www.star-history.com/?repos=chopratejas%2Fheadroom&type=date&legend=top-left">
  <picture>
    <img alt="Star History Chart" src="https://api.star-history.com/chart?repos=chopratejas/headroom&type=date&legend=top-left" />

docs/output-token-reduction-guide.md

@@ -0,0 +1,136 @@
+# Output Token Reduction — User Guide
+
+A plain-English guide to cutting the tokens the model **writes back**.
+
+## Why this exists
+
+Headroom normally shrinks the prompt you **send**. This feature shrinks what the
+model **returns**. That matters because:
+
+- Output tokens cost **5× more** than input on Opus-class models.
+- A lot of model output is waste: "Great, let me help with that…" intros,
+  re-printing code you already showed it, restating tool results, and long
+  internal "thinking" even on trivial steps.
+
+You don't change any code. It runs in the Headroom proxy.
+
+## Turn it on
+
+```bash
+export HEADROOM_OUTPUT_SHAPER=1     # off by default
+headroom proxy --port 8787
+```
+
+That's it. Two things now happen on every request:
+
+1. **Verbosity steering** — a short "be terse, don't restate context" instruction
+   is added to the **end** of the system prompt. (The end, so your prompt cache
+   still works.)
+2. **Effort routing** — if a turn is just the model continuing after a tool ran
+   (e.g. it read a file and there were no errors), Headroom turns the model's
+   "thinking effort" down for that one turn. Real questions and error-handling
+   turns keep full effort.
+
+## The verbosity dial (levels 0–4)
+
+| Level | What the model is told | Good for |
+|------:|------------------------|----------|
+| 0 | (off) | disable steering |
+| 1 | Skip the intro/outro chit-chat | people who read everything |
+| 2 | Also: don't restate code/output already on screen | **default** — safe |
+| 3 | Also: conclusions only, skip the reasoning | people who skim |
+| 4 | Bare minimum, fragments OK | maximum savings, terse |
+
+Set it by hand if you want:
+
+```bash
+export HEADROOM_VERBOSITY_LEVEL=3
+```
+
+Or — better — let Headroom learn it from your habits (next section).
+
+## Let Headroom pick the level for you
+
+People rarely *say* "be brief." They *show* it: they interrupt long answers, or
+reply so fast they couldn't have read the whole thing. `headroom learn
+--verbosity` reads your past sessions and picks a level from those signals.
+
+```bash
+# Preview what it found (doesn't change anything)
+headroom learn --verbosity
+
+# Save it — the proxy uses this level from now on
+headroom learn --verbosity --apply
+```
+
+Example output:
+
+```
+Verbosity — headroom
+  Interrupts:  29  (11% of turns)        ← push-back signal
+  Fast-skips:  31 / 119 long answers (26% unread)   ← strongest signal
+  >> Recommended verbosity level: 3 (confidence: high)
+```
+
+Add `--llm-judge` to have an LLM double-check the level (needs an API key).
+
+## See how much you saved
+
+Here's the honest part. We **can't directly measure** output savings — we never
+see what the model *would* have written without our nudge. So Headroom reports an
+**estimate with a confidence range**, never a fake exact number:
+
+```bash
+headroom output-savings
+```
+
+```
+Output-token reduction
+  Method:    ESTIMATED (synthetic control)
+  Requests:  1,240 shaped
+  Saved:     ~410,000 output tokens
+  Reduction: 28.0%   (95% CI 24.1% … 31.9%)
+```
+
+- **ESTIMATED** = compared against a baseline of your past (unshaped) sessions.
+- **MEASURED** = the gold standard, if you opt into a holdout (below).
+
+### Want a *measured* number?
+
+Leave a slice of traffic unshaped as a control group:
+
+```bash
+export HEADROOM_OUTPUT_HOLDOUT=0.1     # 10% of conversations stay unshaped
+```
+
+Now `headroom output-savings` compares shaped vs unshaped directly and reports a
+**measured** reduction. The trade-off: you give up the savings on that 10%.
+
+## On the dashboard
+
+Open `http://localhost:8787/dashboard`. Next to the input-compression card
+you'll see an **Output Tokens Saved** card showing the token count, the percent,
+a `measured`/`estimated` badge, and the confidence range.
+
+## FAQ
+
+**Will this make answers worse?**
+At level 2 (default), no — in our tests the model finds the same bugs and writes
+the same fixes; it just stops re-printing code and skipping the "let me…" intro.
+Levels 3–4 are terser by design; that's why learning the level per user matters.
+
+**Does it break prompt caching?**
+No. The steering text is added at the *end* of the system prompt and is
+byte-stable, so your cached prefix is untouched.
+
+**Is it safe with extended thinking / tool loops?**
+Yes. It never disables thinking outright (that can error), it only lowers effort
+on routine turns, and it never adds settings the model doesn't support.
+
+**How do I turn it off?**
+Unset `HEADROOM_OUTPUT_SHAPER` (or set it to `0`) and restart the proxy. You can
+also send `x-headroom-bypass: true` on a request to skip it for that call.
+
+---
+
+Deep dive (design + the counterfactual math): [`proposals/output-token-reduction.md`](proposals/output-token-reduction.md)