Skip to content

Commit b3a559b

Browse files
authored
fix(savings): cap ledger retention at 30 days (#1985)
## Description The durable savings ledger (`headroom savings`) retained up to 365 days of history with an unbounded-sounding "All time" window. Long-lived installs accumulate an ever-growing `~/.headroom/savings_events.jsonl`, and `--days` had no upper bound so a caller could request an arbitrarily large lookback. This caps retention at 30 days everywhere it's read, shrinks the compaction threshold to match, and renames the "All time" window to reflect what it actually is now: `Last 30 days`. ## Type of Change - [x] Bug fix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/savings_ledger.py`: `DEFAULT_RETENTION_DAYS` 365 → 30; add `MAX_RETENTION_DAYS = 30` and hard-clamp the lookback inside `aggregate_savings` so no caller (CLI or programmatic) can read back further than 30 days, regardless of the `retention_days` argument passed in. - `headroom/savings_ledger.py`: report window `all_time` → `last_30_days` (the bucket is exactly 30-day-bounded now, so it doubles as the lifetime view too). `_COMPACT_SIZE_BYTES` 8 MiB → 1 MiB, since a 30-day-bounded ledger should never need to grow large. - `headroom/cli/savings.py`: `--days` is now `click.IntRange(min=1, max=30)` (was unbounded); help text states the max. Window label `"All time"` → `"Last 30 days"`, and the label column width bumped 11 → 12 so the longer label stays aligned with the other rows' progress bars. - `tests/test_savings_ledger.py`: updated window-label assertions; added a hard-cap regression test (`retention_days=365` passed explicitly still excludes a 60-day-old event) and a `--days` range-rejection test (31/60/365 all rejected). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ ruff check headroom/savings_ledger.py headroom/cli/savings.py tests/test_savings_ledger.py All checks passed! $ ruff format --check headroom/savings_ledger.py headroom/cli/savings.py tests/test_savings_ledger.py 3 files already formatted $ mypy headroom --ignore-missing-imports Success: no issues found in 409 source files $ pytest tests/test_savings_ledger.py -q ............ss.... [100%] 16 passed, 2 skipped in 6.11s ``` (ruff `0.15.17`, mypy `1.20.2` — pinned to match `.github/workflows/ci.yml`'s `lint` job. Full multi-shard suite left to CI; ran the full touched-module suite locally.) ## Real Behavior Proof - Environment: macOS (Darwin 25.5.0), Python 3.13.14, local `uv` venv; branch built and installed via `uv tool install --force`. - Exact command / steps: ran `headroom savings` against a ledger holding multiple models' events (claude-opus-4-8, claude-sonnet-5, claude-haiku-4-5) recorded across the retention window, then ran `headroom savings --days 60` to exercise the new upper bound. - Observed result: all three windows (Today / Last 7 days / Last 30 days) populate and are each bounded to at most 30 days; cost-avoided breaks down per model; `--days 60` is rejected by the new `1..30` range instead of silently accepted. - Not tested: Windows/macOS native-wrapper e2e jobs — left to CI. ```text $ headroom savings Today █████░░░░░░░░░░░ 33.8% saved 8,702,348 / 25,781,326 tokens $25.5830 Last 7 days ██████░░░░░░░░░░ 36.3% saved 11,289,737 / 31,072,254 tokens $34.8287 Last 30 days ██████░░░░░░░░░░ 38.2% saved 14,449,516 / 37,821,634 tokens $48.5385 Cost avoided per model: claude-opus-4-8 $33.0494 claude-sonnet-5 $15.2989 claude-haiku-4-5-20251001 $0.1902 $ headroom savings --days 60 Usage: headroom savings [OPTIONS] Try 'headroom savings --help' for help. Error: Invalid value for '--days': 60 is not in the range 1<=x<=30. ``` - Not tested: Windows/macOS native-wrapper e2e jobs — left to CI. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the style guidelines of this project - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A — CLI text output only, see Real Behavior Proof above.
1 parent 82af5cd commit b3a559b

3 files changed

Lines changed: 49 additions & 18 deletions

File tree

headroom/cli/savings.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,7 @@ def _window_line(label: str, window: dict[str, Any]) -> str:
4040
before = int(window.get("tokens_before", 0) or 0)
4141
cost = float(window.get("cost_usd", 0.0) or 0.0)
4242
return (
43-
f"{label:<11} {_bar(pct)} {pct:5.1f}% "
43+
f"{label:<12} {_bar(pct)} {pct:5.1f}% "
4444
f"saved {_tokens(saved)} / {_tokens(before)} tokens {_money(cost)}"
4545
)
4646

@@ -49,10 +49,10 @@ def _window_line(label: str, window: dict[str, Any]) -> str:
4949
@click.option("--json", "as_json", is_flag=True, help="Emit the raw report as JSON.")
5050
@click.option(
5151
"--days",
52-
type=click.IntRange(min=1),
52+
type=click.IntRange(min=1, max=savings_ledger.MAX_RETENTION_DAYS),
5353
default=savings_ledger.DEFAULT_RETENTION_DAYS,
5454
show_default=True,
55-
help="Retention/lookback window for the ledger, in days.",
55+
help=f"Retention/lookback window for the ledger, in days (max {savings_ledger.MAX_RETENTION_DAYS}).",
5656
)
5757
@click.option("--reset", is_flag=True, help="Delete the savings ledger and start fresh.")
5858
def savings(as_json: bool, days: int, reset: bool) -> None:
@@ -87,7 +87,7 @@ def savings(as_json: bool, days: int, reset: bool) -> None:
8787
click.echo("")
8888
click.echo(_window_line("Today", report.windows["today"]))
8989
click.echo(_window_line("Last 7 days", report.windows["last_7_days"]))
90-
click.echo(_window_line("All time", report.windows["all_time"]))
90+
click.echo(_window_line("Last 30 days", report.windows["last_30_days"]))
9191

9292
if report.by_model:
9393
click.echo("")

headroom/savings_ledger.py

Lines changed: 16 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -48,14 +48,18 @@
4848
SCHEMA_VERSION = 1
4949
UNKNOWN = "unknown"
5050

51-
DEFAULT_RETENTION_DAYS = 365
51+
# Report windows never look back further than 30 days, and events older than
52+
# this are pruned from disk, keeping the JSONL small.
53+
MAX_RETENTION_DAYS = 30
54+
DEFAULT_RETENTION_DAYS = 30
5255
# Blended input price ($/token) used only when litellm cannot price the model.
5356
# Mirrors the ~$3 / 1M input-token assumption the MCP stats path already uses.
5457
DEFAULT_FALLBACK_INPUT_COST_PER_TOKEN = 3.0 / 1_000_000
5558

5659
# Disk hygiene: compact the ledger once it grows past this size. Retention is
5760
# also enforced on read, so accuracy never depends on compaction having run.
58-
_COMPACT_SIZE_BYTES = 8 * 1024 * 1024
61+
# Small because retention is only 30 days — the file should never be large.
62+
_COMPACT_SIZE_BYTES = 1 * 1024 * 1024
5963

6064

6165
def _utc_now() -> datetime:
@@ -286,6 +290,9 @@ def aggregate_savings(
286290
"""Aggregate the durable ledger into lifetime / windowed / per-dimension views."""
287291

288292
now = now or _utc_now()
293+
# Hard-cap the lookback at 30 days regardless of caller input (0/None means
294+
# "use the cap", never "unbounded"), keeping the report bounded and small.
295+
retention_days = max(1, min(retention_days or MAX_RETENTION_DAYS, MAX_RETENTION_DAYS))
289296
events = _read_events(path, retention_days=retention_days, now=now)
290297

291298
# "Today" is local-calendar-day; the 7-day window is a rolling 168h.
@@ -294,7 +301,9 @@ def aggregate_savings(
294301
)
295302
week_cutoff = now - timedelta(days=7)
296303

297-
all_time = _Bucket()
304+
# All retained events are within the 30-day cap, so this bucket doubles as
305+
# both the "Last 30 days" window and the (now 30-day-bounded) lifetime view.
306+
windowed = _Bucket()
298307
today = _Bucket()
299308
last_7 = _Bucket()
300309
by_model: dict[str, _Bucket] = {}
@@ -309,7 +318,7 @@ def aggregate_savings(
309318
except (TypeError, ValueError):
310319
cost = 0.0
311320

312-
all_time.add(saved=saved, before=before, cost=cost)
321+
windowed.add(saved=saved, before=before, cost=cost)
313322
if ts >= today_cutoff:
314323
today.add(saved=saved, before=before, cost=cost)
315324
if ts >= week_cutoff:
@@ -328,11 +337,11 @@ def aggregate_savings(
328337
return SavingsReport(
329338
path=str(_resolve_path(path)),
330339
schema_version=SCHEMA_VERSION,
331-
lifetime=all_time.to_dict(),
340+
lifetime=windowed.to_dict(),
332341
windows={
333342
"today": today.to_dict(),
334343
"last_7_days": last_7.to_dict(),
335-
"all_time": all_time.to_dict(),
344+
"last_30_days": windowed.to_dict(),
336345
},
337346
by_model=model_rows,
338347
by_client=_ranked(by_client, "client"),
@@ -383,6 +392,7 @@ def _maybe_compact(target: Path) -> None:
383392

384393
__all__ = [
385394
"SCHEMA_VERSION",
395+
"MAX_RETENTION_DAYS",
386396
"DEFAULT_RETENTION_DAYS",
387397
"DEFAULT_FALLBACK_INPUT_COST_PER_TOKEN",
388398
"SavingsReport",

tests/test_savings_ledger.py

Lines changed: 29 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -66,7 +66,7 @@ def test_breakdowns_aggregate_by_dimension(monkeypatch, tmp_path):
6666
assert clients["proxy"]["tokens_saved"] == 1400
6767

6868

69-
def test_windows_today_week_alltime(monkeypatch, tmp_path):
69+
def test_windows_today_week_last30(monkeypatch, tmp_path):
7070
_events_env(monkeypatch, tmp_path)
7171
now = datetime(2026, 6, 17, 12, 0, tzinfo=UTC)
7272
L.record_savings_event(
@@ -84,32 +84,35 @@ def test_windows_today_week_alltime(monkeypatch, tmp_path):
8484
tokens_after=700,
8585
model=None,
8686
client="c",
87-
timestamp=now - timedelta(days=30),
87+
timestamp=now - timedelta(days=20),
8888
)
8989
report = L.aggregate_savings(now=now)
9090
assert report.windows["today"]["tokens_saved"] == 500
9191
assert report.windows["last_7_days"]["tokens_saved"] == 500 + 400
92-
assert report.windows["all_time"]["tokens_saved"] == 500 + 400 + 300
93-
assert report.windows["all_time"]["calls"] == 3
92+
assert report.windows["last_30_days"]["tokens_saved"] == 500 + 400 + 300
93+
assert report.windows["last_30_days"]["calls"] == 3
9494
# 500 saved out of 1000 before today
9595
assert report.windows["today"]["savings_percent"] == pytest.approx(50.0)
9696

9797

98-
def test_retention_excludes_old_events(monkeypatch, tmp_path):
98+
def test_retention_hard_capped_at_30_days(monkeypatch, tmp_path):
9999
_events_env(monkeypatch, tmp_path)
100100
now = datetime(2026, 6, 17, 12, 0, tzinfo=UTC)
101101
L.record_savings_event(
102102
tokens_before=1000, tokens_after=500, model=None, client="c", timestamp=now
103103
)
104+
# 60 days old: within the requested 365-day window but past the 30-day cap.
104105
L.record_savings_event(
105106
tokens_before=1000,
106107
tokens_after=500,
107108
model=None,
108109
client="c",
109-
timestamp=now - timedelta(days=400),
110+
timestamp=now - timedelta(days=60),
110111
)
112+
# Caller asks for 365 days, but retention is hard-capped at 30.
111113
report = L.aggregate_savings(now=now, retention_days=365)
112114
assert report.lifetime["calls"] == 1
115+
assert report.windows["last_30_days"]["calls"] == 1
113116

114117

115118
def test_appends_do_not_clobber_and_survive_restart(monkeypatch, tmp_path):
@@ -184,15 +187,15 @@ def test_cli_renders_sections_and_json(monkeypatch, tmp_path):
184187
assert result.exit_code == 0
185188
# No redundant top-line headline; the windows lead the output.
186189
assert "cost avoided" not in result.output
187-
assert "Today" in result.output and "All time" in result.output
190+
assert "Today" in result.output and "Last 30 days" in result.output
188191
assert "Savings by client" in result.output and "claude-code" in result.output
189192
assert "Per-repo totals" not in result.output
190193

191194
result_json = runner.invoke(savings, ["--json"])
192195
assert result_json.exit_code == 0
193196
payload = json.loads(result_json.output)
194197
assert payload["lifetime"]["tokens_saved"] == 700
195-
assert payload["windows"]["all_time"]["calls"] == 1
198+
assert payload["windows"]["last_30_days"]["calls"] == 1
196199
assert "by_repo" not in payload
197200

198201

@@ -279,3 +282,21 @@ def test_proxy_record_request_appends_ledger_event(tmp_path, monkeypatch):
279282
assert "claude-code" in clients
280283
assert "proxy" in clients
281284
assert any(row["model"] == "gpt-4o" for row in report.by_model)
285+
286+
287+
# --------------------------------------------------------------------------- #
288+
# retention cap + stale-schema reset
289+
# --------------------------------------------------------------------------- #
290+
291+
292+
@pytest.mark.parametrize("bad_days", ["31", "60", "365"])
293+
def test_cli_days_flag_capped_at_30(monkeypatch, tmp_path, bad_days):
294+
pytest.importorskip("click")
295+
from click.testing import CliRunner
296+
297+
from headroom.cli.savings import savings
298+
299+
_events_env(monkeypatch, tmp_path)
300+
result = CliRunner().invoke(savings, ["--days", bad_days])
301+
assert result.exit_code != 0
302+
assert "30" in result.output # IntRange error mentions the allowed max

0 commit comments

Comments
 (0)