ci: pass CODECOV_TOKEN to coverage uploads (fixes red test shards) by chopratejas · Pull Request #968 · headroomlabs-ai/headroom

chopratejas · 2026-06-14T06:20:17Z

Description

Every test (N) shard has been failing on all PRs and on pushes to main, even though all tests pass. Root cause: Codecov retired tokenless uploads. Without a token, the upload is rejected with Token required because branch is protected, and ci.yml had fail_ci_if_error: true with no token — so the rejected upload failed the whole shard.

This passes CODECOV_TOKEN to the coverage-upload steps so uploads authenticate again.

Closes #

Type of Change

Bug fix (non-breaking change that fixes an issue)
New feature (non-breaking change that adds functionality)
Breaking change (fix or feature that would cause existing functionality to change)
Documentation update
Performance improvement
Code refactoring (no functional changes)

Changes Made

ci.yml: add token: ${{ secrets.CODECOV_TOKEN }} to the shard upload step; guard fail_ci_if_error so it stays enforced on same-repo PRs and pushes but relaxes on fork PRs (which cannot read repo secrets).
wrap-native-e2e.yml, install-native-e2e.yml: add the same token so their coverage uploads authenticate too (these were silently dropping coverage; already non-fatal).

Testing

Unit tests pass (pytest)
Linting passes (ruff check .)
Type checking passes (mypy headroom)
New tests added for new functionality
Manual testing performed

Test Output

$ python -c "import yaml; [yaml.safe_load(open(f)) for f in [
    '.github/workflows/ci.yml',
    '.github/workflows/wrap-native-e2e.yml',
    '.github/workflows/install-native-e2e.yml']]"
OK ci.yml
OK wrap-native-e2e.yml
OK install-native-e2e.yml

This PR's own `test (N)` shards are the real test: with CODECOV_TOKEN set,
they should upload successfully and go green.

Real Behavior Proof

Environment: GitHub Actions, codecov/codecov-action@v5 (ci.yml) / @v4 (e2e); repo is public; CODECOV_TOKEN repo secret set by the maintainer.
Exact command / steps: open this PR → observe the test (1..4) shards upload coverage with the token instead of being rejected.
Observed result: prior runs showed 1592 passed then Token required because branch is protected → shard failed; main's own push CI was red for the same reason. With the token referenced, the upload authenticates.
Not tested: fork-PR path (no secret) — by design it now relaxes fail_ci_if_error so the tokenless rejection is non-fatal there.

Review Readiness

I have performed a self-review
This PR is ready for human review

Checklist

My code follows the project's style guidelines
I have performed a self-review of my code
I have commented my code, particularly in hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my fix is effective or that my feature works
New and existing unit tests pass locally with my changes
I have updated the CHANGELOG.md if applicable

Additional Notes

Requires the CODECOV_TOKEN repository secret (GitHub → Settings → Secrets and variables → Actions). No code/CHANGELOG change. Separate from the output-token-reduction feature PR #965.

Codecov retired tokenless uploads; without a token the upload is rejected ('Token required because branch is protected'). The ci.yml shard step had fail_ci_if_error: true and no token, so every 'test (N)' shard went red on every PR and on pushes to main despite all tests passing. - Pass token: ${{ secrets.CODECOV_TOKEN }} to all three codecov-action steps. - ci.yml hard-fail now guarded: enforced on same-repo PRs and pushes, relaxed on fork PRs (which cannot read repo secrets). Requires the CODECOV_TOKEN repo secret to be set.

github-actions · 2026-06-14T06:20:33Z

PR governance

This PR follows the template and is marked ready for human review.

The token fix got us past 'Token required'; Codecov now returns 'Repository not found' until the repo is activated on Codecov's side (GitHub App install + repo upload token). Coverage upload is reporting-only, so it must not fail a build whose tests pass. Keep the token (so coverage flows once activated) and set fail_ci_if_error: false.

## Description Bump `codecov/codecov-action` from `@v4` to `@v5` in the two native e2e workflows, and rename the `file:` input to `files:` to match the v5 API. The main `ci.yml` already uses `@v5` with `files:`; this aligns the remaining Codecov uploads. Follow-up to #968. ## Type of Change - [x] Code refactoring (no functional changes) ## Changes Made - `.github/workflows/install-native-e2e.yml`: `codecov/codecov-action@v4` to `@v5`, and `file:` to `files:`. - `.github/workflows/wrap-native-e2e.yml`: `codecov/codecov-action@v4` to `@v5`, and `file:` to `files:`. - All three Codecov uploads now use `@v5` with the `files:` input. ## Testing - [x] Manual testing performed ### Test Output ```text python -c "import yaml; [yaml.safe_load(open(f, encoding='utf-8')) for f in ['.github/workflows/install-native-e2e.yml','.github/workflows/wrap-native-e2e.yml','.github/workflows/ci.yml']]; print('all workflows parse as valid YAML')" all workflows parse as valid YAML rg -n "codecov/codecov-action@|^\s+file:|^\s+files:" .github/workflows/install-native-e2e.yml .github/workflows/wrap-native-e2e.yml .github/workflows/ci.yml .github/workflows/install-native-e2e.yml:61: uses: codecov/codecov-action@v5 .github/workflows/install-native-e2e.yml:63: files: ./coverage-install-native.xml .github/workflows/wrap-native-e2e.yml:66: uses: codecov/codecov-action@v5 .github/workflows/wrap-native-e2e.yml:68: files: ./coverage-wrap-native.xml .github/workflows/ci.yml:209: uses: codecov/codecov-action@v5 .github/workflows/ci.yml:211: files: coverage-${{ matrix.shard }}.xml git diff --check upstream/main...HEAD # no output ``` ## Real Behavior Proof - Environment: local Windows 11 checkout, Python 3.13.13 with PyYAML, ripgrep. - Exact command / steps: rebased onto current `main`, parsed the three workflow YAML files, confirmed all Codecov action references use `@v5`, confirmed upload inputs use `files:`, and checked the diff for whitespace errors. - Observed result: workflow YAML parses; native e2e and CI Codecov upload steps are aligned on `@v5`/`files:`; no whitespace errors. - Not tested: live Codecov upload, because it requires Actions secrets and GitHub-hosted runners. The PR workflows exercise the changed steps. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review

…ffic audits (#818) ## Description A data-driven push for better compression savings without accuracy loss, in four parts: expose and tune the Rust compressor knobs, harden the CCR retrieval store, add traffic-audit tooling that sizes opportunities from real transcripts, and introduce **read maturation** — a new, live-validated mechanism that compresses Read outputs *before* they ever enter the provider prefix cache. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made ### 1. Rust compressor extraction - Expose `lossless_min_savings_ratio` end-to-end and lower the default 0.30 → 0.15 (lockstep across Rust, PyO3, and both Python config classes) so the lossless Table/CSV compaction path wins more often. - Expose the `CompactConfig` heuristics (core-field fraction, heterogeneity ratio, flatten cap, bucket bounds) through PyO3 + Python. - `SearchCompressor` grouped-by-file output (`rg --heading` style — path once per file instead of per match). Library default off; the proxy enables it in token mode. - Complete `factor_out_constants`: constant fields now emit once in a `_constant_fields` sentinel with slim rows (defensive per-item value match; default off). - `ContentRouter` accepts a SmartCrusher config override and the search-grouping knob. ### 2. CCR store hardening - Session-scale TTL: 300s → 1800s (CCRConfig, CompressionEntry, CompressionStore, Rust `DEFAULT_TTL` — lockstep). - **SQLite is the default CCR backend** (`~/.headroom/ccr_store.db`, WAL): survives proxy restarts and is shared across workers. `HEADROOM_CCR_BACKEND=memory` opts out. - Multi-worker safety: `busy_timeout`, and corruption detection narrowed so transient `SQLITE_BUSY` errors can never trigger database deletion. - Data-at-rest hygiene: `chmod 600` on db + sidecars, expired rows swept at open. - Retrieval-miss messages are actionable (re-read the file / re-run the command). ### 3. Traffic audit tooling (measure before tuning) - `headroom audit-reads`: sizes Read opportunities from local Claude Code transcripts (read share, stale %, line-number overhead, context residency, cache-death windows). - `--simulate-maturation`: Mechanism B risk sizing (re-read rates, never-touched-again share, quiesce coverage, at-risk edits). - `--codex`: shell-read classifier for Codex transcripts (rtk-wrapper aware, workdir resolution). - Findings that shaped this PR (81 sessions): Reads are 67% of tool bytes; median Read lingers 118 turns (~13x lifetime cost); a prototyped repeat-Read dedup measured 0.1% and was **removed** rather than shipped as dead code. ### 4. Read maturation (Mechanism B) — experimental, default OFF - Activity-based: a fresh large Read is held **out** of the provider cache (trailing breakpoint relocated before it), stays verbatim while its file is active, and matures into a CCR-backed marker once the file is quiet for `quiesce_turns` (default 5; `max_hold_turns` bounds busy files). - Only the final compressed form ever enters the cache — **no cached byte is ever mutated**; matured markers replay byte-identically. - Wired into the Anthropic handler behind `--read-maturation` / `HEADROOM_READ_MATURATION=1`; session state rides on the prefix tracker; advisory (can never fail a request). - Live-validated against the Anthropic API: held content excluded from cache_creation; after maturation the prior cached prefix still served — the no-bust invariant holds end-to-end. ### 5. Rebase / CI fixups (this update) - Rebased onto latest `main` (was 28 commits behind): picks up `ci: pass CODECOV_TOKEN to coverage uploads (#968)`, which is what was turning the 4 test shards red — the tests themselves passed (1528) but the post-test codecov upload exited non-zero on a protected branch. - Resolved the duplicate `lossless_min_savings_ratio` that two independent main/branch additions left in `SmartCrusherConfig` and the Rust-config kwarg (import-time `SyntaxError` + mypy `no-redef`). - Aligned CCR tests with the new defaults (SQLite backend, 1800s TTL) across `test_ccr`, `test_adapter_hooks`, `test_compression_store`, `test_proxy_ccr`, and the lossy row-drop bridge test. ## Testing  - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_proxy_ccr.py tests/test_ccr.py tests/test_compression_store.py tests/test_adapter_hooks.py tests/test_ccr_row_drop_store_bridge.py -q 170 passed, 4 warnings in 42.49s $ python -m pytest tests/test_audit_reads.py tests/test_audit_codex.py tests/test_read_maturation.py tests/test_transforms_content_router.py tests/test_smart_crusher_toin_attachment.py -q 83 passed $ mypy headroom/ Success: no issues found in 365 source files $ python -m compileall headroom/ -q COMPILE-OK # CI (run 27488990477, pre-rebase head): all 4 shards ran to completion — # "1528 passed, 120 skipped, 4922 deselected" # The red shards were the codecov upload step, not test failures; fixed by # the #968 rebase above. ``` ## Real Behavior Proof - Environment: macOS (darwin), Python 3.12 venv; branch `feat/compression-extraction` rebased onto `origin/main` (head 7cb0f43); GitHub Actions CI run 27488990477 for the test shards - Exact command / steps: rebased onto latest main (clean, 13 commits replayed, 0 conflicts); ran the pytest suites and mypy above locally; inspected CI shard logs to confirm the failure was the codecov upload, not the test phase - Observed result: 253 targeted tests pass locally; mypy clean on 365 files; CI test phase reports `1528 passed, 120 skipped`; the only red step (codecov `upload-coverage` → "Token required because branch is protected") is resolved by the rebased-in #968 CODECOV_TOKEN fix - Not tested: the read-maturation live-API no-bust validation (`tests/test_live/`) was not re-run in this rebase pass (requires provider keys); it was validated when the feature first landed, and no maturation code changed in the rebase — only CCR-default test assertions and the duplicate-field resolution ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes CHANGELOG is generated by release-please from the conventional commits, so the CHANGELOG box is intentionally left unchecked. "Manual testing performed" is unchecked deliberately — see `Real Behavior Proof` → `Not tested` for the exact boundary (the live-API maturation validation was not re-run in this rebase pass). ### Follow-ups (tracked, not in this PR) - Mechanism B provider extensions: OpenAI-family wiring (no breakpoint hold — bounded near-tail bust) and the Codex runtime read-detector (the audit classifier is the prototype). - Pilot enablement playbook: run `audit-reads --simulate-maturation` on target traffic → pick `quiesce_turns` → enable via env → watch cache hit rate + `read_maturation:N` transform tags.

…ai#978) ## Description Bump `codecov/codecov-action` from `@v4` to `@v5` in the two native e2e workflows, and rename the `file:` input to `files:` to match the v5 API. The main `ci.yml` already uses `@v5` with `files:`; this aligns the remaining Codecov uploads. Follow-up to headroomlabs-ai#968. ## Type of Change - [x] Code refactoring (no functional changes) ## Changes Made - `.github/workflows/install-native-e2e.yml`: `codecov/codecov-action@v4` to `@v5`, and `file:` to `files:`. - `.github/workflows/wrap-native-e2e.yml`: `codecov/codecov-action@v4` to `@v5`, and `file:` to `files:`. - All three Codecov uploads now use `@v5` with the `files:` input. ## Testing - [x] Manual testing performed ### Test Output ```text python -c "import yaml; [yaml.safe_load(open(f, encoding='utf-8')) for f in ['.github/workflows/install-native-e2e.yml','.github/workflows/wrap-native-e2e.yml','.github/workflows/ci.yml']]; print('all workflows parse as valid YAML')" all workflows parse as valid YAML rg -n "codecov/codecov-action@|^\s+file:|^\s+files:" .github/workflows/install-native-e2e.yml .github/workflows/wrap-native-e2e.yml .github/workflows/ci.yml .github/workflows/install-native-e2e.yml:61: uses: codecov/codecov-action@v5 .github/workflows/install-native-e2e.yml:63: files: ./coverage-install-native.xml .github/workflows/wrap-native-e2e.yml:66: uses: codecov/codecov-action@v5 .github/workflows/wrap-native-e2e.yml:68: files: ./coverage-wrap-native.xml .github/workflows/ci.yml:209: uses: codecov/codecov-action@v5 .github/workflows/ci.yml:211: files: coverage-${{ matrix.shard }}.xml git diff --check upstream/main...HEAD # no output ``` ## Real Behavior Proof - Environment: local Windows 11 checkout, Python 3.13.13 with PyYAML, ripgrep. - Exact command / steps: rebased onto current `main`, parsed the three workflow YAML files, confirmed all Codecov action references use `@v5`, confirmed upload inputs use `files:`, and checked the diff for whitespace errors. - Observed result: workflow YAML parses; native e2e and CI Codecov upload steps are aligned on `@v5`/`files:`; no whitespace errors. - Not tested: live Codecov upload, because it requires Actions secrets and GitHub-hosted runners. The PR workflows exercise the changed steps. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review

…ffic audits (headroomlabs-ai#818) ## Description A data-driven push for better compression savings without accuracy loss, in four parts: expose and tune the Rust compressor knobs, harden the CCR retrieval store, add traffic-audit tooling that sizes opportunities from real transcripts, and introduce **read maturation** — a new, live-validated mechanism that compresses Read outputs *before* they ever enter the provider prefix cache. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made ### 1. Rust compressor extraction - Expose `lossless_min_savings_ratio` end-to-end and lower the default 0.30 → 0.15 (lockstep across Rust, PyO3, and both Python config classes) so the lossless Table/CSV compaction path wins more often. - Expose the `CompactConfig` heuristics (core-field fraction, heterogeneity ratio, flatten cap, bucket bounds) through PyO3 + Python. - `SearchCompressor` grouped-by-file output (`rg --heading` style — path once per file instead of per match). Library default off; the proxy enables it in token mode. - Complete `factor_out_constants`: constant fields now emit once in a `_constant_fields` sentinel with slim rows (defensive per-item value match; default off). - `ContentRouter` accepts a SmartCrusher config override and the search-grouping knob. ### 2. CCR store hardening - Session-scale TTL: 300s → 1800s (CCRConfig, CompressionEntry, CompressionStore, Rust `DEFAULT_TTL` — lockstep). - **SQLite is the default CCR backend** (`~/.headroom/ccr_store.db`, WAL): survives proxy restarts and is shared across workers. `HEADROOM_CCR_BACKEND=memory` opts out. - Multi-worker safety: `busy_timeout`, and corruption detection narrowed so transient `SQLITE_BUSY` errors can never trigger database deletion. - Data-at-rest hygiene: `chmod 600` on db + sidecars, expired rows swept at open. - Retrieval-miss messages are actionable (re-read the file / re-run the command). ### 3. Traffic audit tooling (measure before tuning) - `headroom audit-reads`: sizes Read opportunities from local Claude Code transcripts (read share, stale %, line-number overhead, context residency, cache-death windows). - `--simulate-maturation`: Mechanism B risk sizing (re-read rates, never-touched-again share, quiesce coverage, at-risk edits). - `--codex`: shell-read classifier for Codex transcripts (rtk-wrapper aware, workdir resolution). - Findings that shaped this PR (81 sessions): Reads are 67% of tool bytes; median Read lingers 118 turns (~13x lifetime cost); a prototyped repeat-Read dedup measured 0.1% and was **removed** rather than shipped as dead code. ### 4. Read maturation (Mechanism B) — experimental, default OFF - Activity-based: a fresh large Read is held **out** of the provider cache (trailing breakpoint relocated before it), stays verbatim while its file is active, and matures into a CCR-backed marker once the file is quiet for `quiesce_turns` (default 5; `max_hold_turns` bounds busy files). - Only the final compressed form ever enters the cache — **no cached byte is ever mutated**; matured markers replay byte-identically. - Wired into the Anthropic handler behind `--read-maturation` / `HEADROOM_READ_MATURATION=1`; session state rides on the prefix tracker; advisory (can never fail a request). - Live-validated against the Anthropic API: held content excluded from cache_creation; after maturation the prior cached prefix still served — the no-bust invariant holds end-to-end. ### 5. Rebase / CI fixups (this update) - Rebased onto latest `main` (was 28 commits behind): picks up `ci: pass CODECOV_TOKEN to coverage uploads (headroomlabs-ai#968)`, which is what was turning the 4 test shards red — the tests themselves passed (1528) but the post-test codecov upload exited non-zero on a protected branch. - Resolved the duplicate `lossless_min_savings_ratio` that two independent main/branch additions left in `SmartCrusherConfig` and the Rust-config kwarg (import-time `SyntaxError` + mypy `no-redef`). - Aligned CCR tests with the new defaults (SQLite backend, 1800s TTL) across `test_ccr`, `test_adapter_hooks`, `test_compression_store`, `test_proxy_ccr`, and the lossy row-drop bridge test. ## Testing  - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_proxy_ccr.py tests/test_ccr.py tests/test_compression_store.py tests/test_adapter_hooks.py tests/test_ccr_row_drop_store_bridge.py -q 170 passed, 4 warnings in 42.49s $ python -m pytest tests/test_audit_reads.py tests/test_audit_codex.py tests/test_read_maturation.py tests/test_transforms_content_router.py tests/test_smart_crusher_toin_attachment.py -q 83 passed $ mypy headroom/ Success: no issues found in 365 source files $ python -m compileall headroom/ -q COMPILE-OK # CI (run 27488990477, pre-rebase head): all 4 shards ran to completion — # "1528 passed, 120 skipped, 4922 deselected" # The red shards were the codecov upload step, not test failures; fixed by # the headroomlabs-ai#968 rebase above. ``` ## Real Behavior Proof - Environment: macOS (darwin), Python 3.12 venv; branch `feat/compression-extraction` rebased onto `origin/main` (head 7cb0f43); GitHub Actions CI run 27488990477 for the test shards - Exact command / steps: rebased onto latest main (clean, 13 commits replayed, 0 conflicts); ran the pytest suites and mypy above locally; inspected CI shard logs to confirm the failure was the codecov upload, not the test phase - Observed result: 253 targeted tests pass locally; mypy clean on 365 files; CI test phase reports `1528 passed, 120 skipped`; the only red step (codecov `upload-coverage` → "Token required because branch is protected") is resolved by the rebased-in headroomlabs-ai#968 CODECOV_TOKEN fix - Not tested: the read-maturation live-API no-bust validation (`tests/test_live/`) was not re-run in this rebase pass (requires provider keys); it was validated when the feature first landed, and no maturation code changed in the rebase — only CCR-default test assertions and the duplicate-field resolution ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes CHANGELOG is generated by release-please from the conventional commits, so the CHANGELOG box is intentionally left unchecked. "Manual testing performed" is unchecked deliberately — see `Real Behavior Proof` → `Not tested` for the exact boundary (the live-API maturation validation was not re-run in this rebase pass). ### Follow-ups (tracked, not in this PR) - Mechanism B provider extensions: OpenAI-family wiring (no breakpoint hold — bounded near-tail bust) and the Codex runtime read-detector (the audit classifier is the prototype). - Pilot enablement playbook: run `audit-reads --simulate-maturation` on target traffic → pick `quiesce_turns` → enable via env → watch cache hit rate + `read_maturation:N` transform tags.

…eadroomlabs-ai#968) ## Description Every `test (N)` shard has been failing on all PRs and on pushes to `main`, even though all tests pass. Root cause: **Codecov retired tokenless uploads.** Without a token, the upload is rejected with `Token required because branch is protected`, and `ci.yml` had `fail_ci_if_error: true` with no token — so the rejected upload failed the whole shard. This passes `CODECOV_TOKEN` to the coverage-upload steps so uploads authenticate again. Closes # ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `ci.yml`: add `token: ${{ secrets.CODECOV_TOKEN }}` to the shard upload step; guard `fail_ci_if_error` so it stays enforced on same-repo PRs and pushes but relaxes on fork PRs (which cannot read repo secrets). - `wrap-native-e2e.yml`, `install-native-e2e.yml`: add the same token so their coverage uploads authenticate too (these were silently dropping coverage; already non-fatal). ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -c "import yaml; [yaml.safe_load(open(f)) for f in [ '.github/workflows/ci.yml', '.github/workflows/wrap-native-e2e.yml', '.github/workflows/install-native-e2e.yml']]" OK ci.yml OK wrap-native-e2e.yml OK install-native-e2e.yml This PR's own `test (N)` shards are the real test: with CODECOV_TOKEN set, they should upload successfully and go green. ``` ## Real Behavior Proof - Environment: GitHub Actions, `codecov/codecov-action@v5` (ci.yml) / `@v4` (e2e); repo is public; `CODECOV_TOKEN` repo secret set by the maintainer. - Exact command / steps: open this PR → observe the `test (1..4)` shards upload coverage with the token instead of being rejected. - Observed result: prior runs showed `1592 passed` then `Token required because branch is protected` → shard failed; main's own push CI was red for the same reason. With the token referenced, the upload authenticates. - Not tested: fork-PR path (no secret) — by design it now relaxes `fail_ci_if_error` so the tokenless rejection is non-fatal there. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes Requires the `CODECOV_TOKEN` repository secret (GitHub → Settings → Secrets and variables → Actions). No code/CHANGELOG change. Separate from the output-token-reduction feature PR headroomlabs-ai#965.

…ai#978) ## Description Bump `codecov/codecov-action` from `@v4` to `@v5` in the two native e2e workflows, and rename the `file:` input to `files:` to match the v5 API. The main `ci.yml` already uses `@v5` with `files:`; this aligns the remaining Codecov uploads. Follow-up to headroomlabs-ai#968. ## Type of Change - [x] Code refactoring (no functional changes) ## Changes Made - `.github/workflows/install-native-e2e.yml`: `codecov/codecov-action@v4` to `@v5`, and `file:` to `files:`. - `.github/workflows/wrap-native-e2e.yml`: `codecov/codecov-action@v4` to `@v5`, and `file:` to `files:`. - All three Codecov uploads now use `@v5` with the `files:` input. ## Testing - [x] Manual testing performed ### Test Output ```text python -c "import yaml; [yaml.safe_load(open(f, encoding='utf-8')) for f in ['.github/workflows/install-native-e2e.yml','.github/workflows/wrap-native-e2e.yml','.github/workflows/ci.yml']]; print('all workflows parse as valid YAML')" all workflows parse as valid YAML rg -n "codecov/codecov-action@|^\s+file:|^\s+files:" .github/workflows/install-native-e2e.yml .github/workflows/wrap-native-e2e.yml .github/workflows/ci.yml .github/workflows/install-native-e2e.yml:61: uses: codecov/codecov-action@v5 .github/workflows/install-native-e2e.yml:63: files: ./coverage-install-native.xml .github/workflows/wrap-native-e2e.yml:66: uses: codecov/codecov-action@v5 .github/workflows/wrap-native-e2e.yml:68: files: ./coverage-wrap-native.xml .github/workflows/ci.yml:209: uses: codecov/codecov-action@v5 .github/workflows/ci.yml:211: files: coverage-${{ matrix.shard }}.xml git diff --check upstream/main...HEAD # no output ``` ## Real Behavior Proof - Environment: local Windows 11 checkout, Python 3.13.13 with PyYAML, ripgrep. - Exact command / steps: rebased onto current `main`, parsed the three workflow YAML files, confirmed all Codecov action references use `@v5`, confirmed upload inputs use `files:`, and checked the diff for whitespace errors. - Observed result: workflow YAML parses; native e2e and CI Codecov upload steps are aligned on `@v5`/`files:`; no whitespace errors. - Not tested: live Codecov upload, because it requires Actions secrets and GitHub-hosted runners. The PR workflows exercise the changed steps. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review

…ffic audits (headroomlabs-ai#818) ## Description A data-driven push for better compression savings without accuracy loss, in four parts: expose and tune the Rust compressor knobs, harden the CCR retrieval store, add traffic-audit tooling that sizes opportunities from real transcripts, and introduce **read maturation** — a new, live-validated mechanism that compresses Read outputs *before* they ever enter the provider prefix cache. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [x] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made ### 1. Rust compressor extraction - Expose `lossless_min_savings_ratio` end-to-end and lower the default 0.30 → 0.15 (lockstep across Rust, PyO3, and both Python config classes) so the lossless Table/CSV compaction path wins more often. - Expose the `CompactConfig` heuristics (core-field fraction, heterogeneity ratio, flatten cap, bucket bounds) through PyO3 + Python. - `SearchCompressor` grouped-by-file output (`rg --heading` style — path once per file instead of per match). Library default off; the proxy enables it in token mode. - Complete `factor_out_constants`: constant fields now emit once in a `_constant_fields` sentinel with slim rows (defensive per-item value match; default off). - `ContentRouter` accepts a SmartCrusher config override and the search-grouping knob. ### 2. CCR store hardening - Session-scale TTL: 300s → 1800s (CCRConfig, CompressionEntry, CompressionStore, Rust `DEFAULT_TTL` — lockstep). - **SQLite is the default CCR backend** (`~/.headroom/ccr_store.db`, WAL): survives proxy restarts and is shared across workers. `HEADROOM_CCR_BACKEND=memory` opts out. - Multi-worker safety: `busy_timeout`, and corruption detection narrowed so transient `SQLITE_BUSY` errors can never trigger database deletion. - Data-at-rest hygiene: `chmod 600` on db + sidecars, expired rows swept at open. - Retrieval-miss messages are actionable (re-read the file / re-run the command). ### 3. Traffic audit tooling (measure before tuning) - `headroom audit-reads`: sizes Read opportunities from local Claude Code transcripts (read share, stale %, line-number overhead, context residency, cache-death windows). - `--simulate-maturation`: Mechanism B risk sizing (re-read rates, never-touched-again share, quiesce coverage, at-risk edits). - `--codex`: shell-read classifier for Codex transcripts (rtk-wrapper aware, workdir resolution). - Findings that shaped this PR (81 sessions): Reads are 67% of tool bytes; median Read lingers 118 turns (~13x lifetime cost); a prototyped repeat-Read dedup measured 0.1% and was **removed** rather than shipped as dead code. ### 4. Read maturation (Mechanism B) — experimental, default OFF - Activity-based: a fresh large Read is held **out** of the provider cache (trailing breakpoint relocated before it), stays verbatim while its file is active, and matures into a CCR-backed marker once the file is quiet for `quiesce_turns` (default 5; `max_hold_turns` bounds busy files). - Only the final compressed form ever enters the cache — **no cached byte is ever mutated**; matured markers replay byte-identically. - Wired into the Anthropic handler behind `--read-maturation` / `HEADROOM_READ_MATURATION=1`; session state rides on the prefix tracker; advisory (can never fail a request). - Live-validated against the Anthropic API: held content excluded from cache_creation; after maturation the prior cached prefix still served — the no-bust invariant holds end-to-end. ### 5. Rebase / CI fixups (this update) - Rebased onto latest `main` (was 28 commits behind): picks up `ci: pass CODECOV_TOKEN to coverage uploads (headroomlabs-ai#968)`, which is what was turning the 4 test shards red — the tests themselves passed (1528) but the post-test codecov upload exited non-zero on a protected branch. - Resolved the duplicate `lossless_min_savings_ratio` that two independent main/branch additions left in `SmartCrusherConfig` and the Rust-config kwarg (import-time `SyntaxError` + mypy `no-redef`). - Aligned CCR tests with the new defaults (SQLite backend, 1800s TTL) across `test_ccr`, `test_adapter_hooks`, `test_compression_store`, `test_proxy_ccr`, and the lossy row-drop bridge test. ## Testing  - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_proxy_ccr.py tests/test_ccr.py tests/test_compression_store.py tests/test_adapter_hooks.py tests/test_ccr_row_drop_store_bridge.py -q 170 passed, 4 warnings in 42.49s $ python -m pytest tests/test_audit_reads.py tests/test_audit_codex.py tests/test_read_maturation.py tests/test_transforms_content_router.py tests/test_smart_crusher_toin_attachment.py -q 83 passed $ mypy headroom/ Success: no issues found in 365 source files $ python -m compileall headroom/ -q COMPILE-OK # CI (run 27488990477, pre-rebase head): all 4 shards ran to completion — # "1528 passed, 120 skipped, 4922 deselected" # The red shards were the codecov upload step, not test failures; fixed by # the headroomlabs-ai#968 rebase above. ``` ## Real Behavior Proof - Environment: macOS (darwin), Python 3.12 venv; branch `feat/compression-extraction` rebased onto `origin/main` (head 7cb0f43); GitHub Actions CI run 27488990477 for the test shards - Exact command / steps: rebased onto latest main (clean, 13 commits replayed, 0 conflicts); ran the pytest suites and mypy above locally; inspected CI shard logs to confirm the failure was the codecov upload, not the test phase - Observed result: 253 targeted tests pass locally; mypy clean on 365 files; CI test phase reports `1528 passed, 120 skipped`; the only red step (codecov `upload-coverage` → "Token required because branch is protected") is resolved by the rebased-in headroomlabs-ai#968 CODECOV_TOKEN fix - Not tested: the read-maturation live-API no-bust validation (`tests/test_live/`) was not re-run in this rebase pass (requires provider keys); it was validated when the feature first landed, and no maturation code changed in the rebase — only CCR-default test assertions and the duplicate-field resolution ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Additional Notes CHANGELOG is generated by release-please from the conventional commits, so the CHANGELOG box is intentionally left unchecked. "Manual testing performed" is unchecked deliberately — see `Real Behavior Proof` → `Not tested` for the exact boundary (the live-API maturation validation was not re-run in this rebase pass). ### Follow-ups (tracked, not in this PR) - Mechanism B provider extensions: OpenAI-family wiring (no breakpoint hold — bounded near-tail bust) and the Codex runtime read-detector (the audit classifier is the prototype). - Pilot enablement playbook: run `audit-reads --simulate-maturation` on target traffic → pick `quiesce_turns` → enable via env → watch cache hit rate + `read_maturation:N` transform tags.

github-actions Bot added the status: ready for review Pull request body is complete and the author marked it ready for human review label Jun 14, 2026

chopratejas merged commit 01fdedc into main Jun 14, 2026
30 checks passed

chopratejas deleted the fix/codecov-token-ci branch June 14, 2026 06:35

Dashsoap mentioned this pull request Jun 14, 2026

ci: align codecov-action to v5 in native e2e workflows #978

Merged

4 tasks

chopratejas mentioned this pull request Jun 15, 2026

feat: compression extraction — Rust knob exposure, CCR hardening, traffic audits #818

Merged

21 tasks

Navigation Menu

Uh oh!

ci: pass CODECOV_TOKEN to coverage uploads (fixes red test shards) - #968

ci: pass CODECOV_TOKEN to coverage uploads (fixes red test shards)#968
chopratejas merged 2 commits into
mainfrom
fix/codecov-token-ci

chopratejas commented Jun 14, 2026

Uh oh!

github-actions Bot commented Jun 14, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

chopratejas commented Jun 14, 2026

Description

Type of Change

Changes Made

Testing

Test Output

Real Behavior Proof

Review Readiness

Checklist

Additional Notes

Uh oh!

github-actions Bot commented Jun 14, 2026

PR governance

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant