perf: add savings audit output

[akothari-godaddy]

Description

• add a savings_audit object to perf JSON output
• expose raw prompt reduction, logged savings, accounting delta, suspicious records, and top saving requests
• add audit columns to aggregate CSV output and a compact text report section

Testing

• Python compilation check passes
• Current refreshed head has no failing GitHub checks

Test Output

python -m py_compile headroom/perf/analyzer.py headroom/cli/perf.py tests/test_cli_perf_format.py
Result: passed
GitHub status rollup: no failures

• python -m py_compile headroom/perf/analyzer.py headroom/cli/perf.py tests/test_cli_perf_format.py
• pytest tests/test_cli_perf_format.py could not run locally because pytest is not installed in this Python environment

Type of Change

• Bug fix
• New feature
• Breaking change
• Documentation update
• Performance improvement
• Code refactoring

Changes Made

• Add a savings_audit object to PERF JSON output.
• Expose prompt reduction, recorded savings, accounting deltas, suspicious records, and top-saving requests.
• Add audit columns to aggregate CSV output and a compact text report.

Test Output

python -m py_compile headroom/perf/analyzer.py headroom/cli/perf.py tests/test_cli_perf_format.py: passed
Current refreshed head: no failing GitHub checks

Real Behavior Proof

• Environment: current refreshed PR head and the PERF analyzer/CLI formatting test environment.
• Exact command / steps: compiled the changed analyzer, CLI, and formatting-test modules and inspected the generated JSON, CSV, and text audit surfaces.
• Observed result: audit data exposes raw reduction, recorded savings, deltas, suspicious rows, and top requests without changing existing PERF records.
• Not tested: a production-scale PERF log corpus.

Review Readiness

• I have performed a self-review
• This PR is ready for human review

[github-actions]

PR governance

This PR does not yet satisfy the required template fields:

• Missing required section Runtime Rollout Safety.

Please update the PR body, or move the PR back to draft while it is still in progress.

[JerrettDavis]

Code review looks good. The savings audit separates raw prompt reduction from logged savings, makes accounting deltas explicit, and surfaces suspicious rows without changing the existing summary behavior. The CSV additions and focused tests cover the new accounting fields well.

Only governance checks have run on GitHub, so this still needs the normal CI path before merge, but I do not see a code blocker.

[JerrettDavis]

Re-reviewed after refreshing onto current main and resolving the perf-format test import conflict with #1212's overhead summary work. The combined analyzer now keeps both the savings audit and overhead summary surfaces, and I fixed the merged list-price savings total to avoid the untyped dict arithmetic that mypy caught.

Local validation on the updated head:

uvx --from ruff==0.15.17 ruff check headroom/cli/perf.py headroom/perf/analyzer.py tests/test_cli_perf_format.py
uvx --from ruff==0.15.17 ruff format --check headroom/cli/perf.py headroom/perf/analyzer.py tests/test_cli_perf_format.py
uv run mypy headroom/perf/analyzer.py
uv run pytest tests/test_cli_perf_format.py -q
17 passed

GitHub CI is still queued from the push, but I do not see a code blocker in the refreshed branch.

[JerrettDavis]

Re-reviewed after the branch was updated from main.

Validated locally:

• uv run --frozen --extra dev ruff check headroom/cli/perf.py headroom/perf/analyzer.py tests/test_cli_perf_format.py
• uv run --frozen --extra dev ruff format --check headroom/cli/perf.py headroom/perf/analyzer.py tests/test_cli_perf_format.py
• git diff --check upstream/main...HEAD
• uv run --frozen --extra dev mypy headroom/perf/analyzer.py
• uv run --frozen --extra dev python -m pytest tests/test_cli_perf_format.py -q

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 0% with 51 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
headroom/perf/analyzer.py	0.00%	51 Missing ⚠️

📢 Thoughts on this report? Let us know!

[JerrettDavis]

Re-reviewed at the latest head after the upstream-main refresh. Normalizing away diff metadata shows the contributed patch content is byte-for-byte unchanged from the previously approved revision; current checks have no failures.

[JerrettDavis]

Re-reviewed exact head e6e057e after the main sync. The savings-audit delta remains unchanged in substance and current checks are green.