Fix/ccr tighten proactive expansion eligibility - #33
Merged
nangsontay merged 31 commits intoAug 12, 2026
Conversation
…double-inject CCR proactive expansion injects retrieved context into the forwarded body only — the client transcript never carries it, so the next turn looks identical to the tracker and the same compressed-content hashes are expanded and re-injected on every turn of the session. On agentic sessions this adds 2-8k tokens per request, degrades net savings to zero, and each transient append also busts the provider prefix cache from the tail position on (headroomlabs-ai#2186). Fixes, mirrored on both handler paths where each applies: - SessionExpansionDedupTracker (bounded LRU): a hash is proactively expanded at most once per session; the handler filters analyze_query recommendations through it and records hashes only after a real append. Explicit headroom_retrieve calls are unaffected. - ContextTracker entries now carry the session id, and analyze_query skips contexts tracked under a different session — content compressed before a /compact must not resurface in the fresh session. - injection_target_already_forwarded(): shared guard that skips the proactive-expansion and memory-context appends when the tail position was already forwarded last turn and this turn append-only-extends it — the overlay has replayed the injected bytes, appending again would double-inject and bust the cache. tests/test_injection_cache_safety.py covers the three guards at unit level plus a tripwire asserting the anthropic handler keeps them wired.
Proactive expansion exists to restore tool ground truth the compressor summarized away. It was also eligible to fire on entries with no tool provenance, which are compressed instruction/system text (agent rules, injected reminders). Re-expanding those is strictly loss-making. Expansion appends the full original while the compressed copy stays in the prefix, so the request carries both; on instruction text, which compresses poorly, the pair exceeds the uncompressed baseline for the same content. The append then replays to the provider on every later turn of the conversation, so a single firing removes the conversation's accumulated savings for good. Filter on read, not on write: the entry stays tracked and retrievable on demand via headroom_retrieve, which bills only what the model asks for. Only the proxy-initiated guess is gated. test_exact_substring_match_bonus was relying on non-tool eligibility to reach analyze_query; it now tracks with tool provenance so it still exercises the substring-bonus path it is named for.
Review follow-up. The injection guard assumed the mutation target was messages[-1], which neither append helper guarantees: - OpenAI's append_text_to_latest_user_chat_message() scans backward for the latest USER message, so on a history whose newest message is an assistant prefill or a tool result the target is an EARLIER position. When that position was replayed from last turn's forwarded bytes, the guard's tail-index check missed and the same context was appended a second time. - Anthropic's _append_context_to_latest_non_frozen_user_turn() only mutates the final message, and only outside the frozen prefix — so the frozen boundary belongs in the target computation too. Both index rules now live in one finder each (latest_user_chat_message_index / latest_non_frozen_user_turn_index), the append helpers use them, and callers pass the resulting index to the guard, so the guard and the mutation can no longer disagree. The replay test is also exact now. is_append_only_extension() required the ENTIRE previous message list to be a canonical prefix, but overlay_cached_prefix replays the longest LEADING RUN plus a block-append merge shape. position_already_forwarded() replaces it and asks the question directly of the post-overlay bytes: does this position already carry what we forwarded there last turn — covering both replay shapes without re-deriving the overlay's trigger conditions. Also: SessionExpansionDedupTracker.filter_new() now refreshes LRU recency, so an active session that is only queried cannot be evicted and have its already-seen hashes expanded twice. Tests: handler-shaped regressions for the Anthropic assistant-prefill tail and the OpenAI non-user tail (proving the replayed user turn keeps ONE copy while a genuinely fresh user tail still receives injection), the block-append merge shape, the frozen boundary, the LRU read refresh, and an OpenAI wiring tripwire. Module docstring section labels realigned with the tests.
## Description `strip_unsupported_tool_search_blocks` (headroomlabs-ai#2807) validates every replayed `tool_reference` in the transcript against the request's `tools` array. It ran *before* the turn-hooks block in `handlers/anthropic.py`, and a registered turn hook may rewrite that array — the hook surface is documented as "a registered hook may inspect or rewrite the outbound tools/messages before we send upstream". So a hook that drops a tool named by a replayed reference leaves the repair having validated against a stale view, and upstream rejects the request: ```text 400 Tool reference 'X' not found in available tools ``` The repair's correctness argument is that it validates against exactly the `tools` array upstream will see. That was true at the old call site and stopped being true one block later. ### Fix Move the repair to after the turn-hooks block, so it is the last stage that can invalidate a reference: - It still runs **after** the deferral injection, so the tool just injected counts as present — the main loop strips nothing and the frozen prefix stays byte-identical. - Nothing past the new call site mutates `body["tools"]` on the outbound path. (The two later `continuation_body["tools"]` assignments build a *derived* body from the already-repaired `body`, so they inherit the repair.) - It still runs before the consistency token re-count, so `tok_after` continues to reflect the repaired messages. - It remains unconditional (not gated on `HEADROOM_TOOL_SEARCH`, not gated on `_bypass`), so transcripts poisoned before the flag was turned off still recover. `strip_unsupported_tool_search_blocks` is copy-on-write and returns the original `messages` object by identity when nothing is removed, so relocating the call does not change the no-op path. ### Severity Latent. No turn hook ships in-tree, so this cannot fire on a default install — it is reachable only through a third-party registered hook that shrinks the tools array. Filing the fix now so the ordering constraint is enforced by a test rather than rediscovered. Closes headroomlabs-ai#2888 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/handlers/anthropic.py`: the tool-search history repair block moves from just after the deferral injection to just after the turn-hooks block. The comment now states the ordering constraint in both directions (after injection, after hooks) so the next person to add a stage knows where the boundary is. No logic change. - `tests/test_proxy/test_tool_search_repair_after_turn_hooks.py` (new): two handler-level regressions. Ordering is the whole property under test, so a unit test of the helper cannot see it — these drive the real handler through `TestClient` and assert on the forwarded body. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed — no in-tree turn hook exists to exercise this against a live API key; the handler-level test below is the substitute, see Not tested. ### Test Output ```text $ uv run --extra dev pytest tests/test_proxy/ -q ======================= 241 passed, 1 warning in 35.82s ======================== $ uvx ruff check headroom tests All checks passed! $ uv run --extra dev mypy headroom Success: no issues found in 515 source files ``` ## Real Behavior Proof - Environment: macOS 15 (darwin 24.6.0), Python 3.10.18, pytest 9.0.3, in a worktree off `upstream/main` at 2f2950a. No live Anthropic key: the proxy handler is driven end to end through `fastapi.testclient.TestClient` with `_retry_request` stubbed, so the assertion is on the exact body that would have been sent upstream. - Exact command / steps: (1) on the branch as submitted, `uv run --extra dev pytest tests/test_proxy/test_tool_search_repair_after_turn_hooks.py -q` -> 2 passed; (2) revert ONLY the handler ordering change while keeping the new tests, `git checkout HEAD~1 -- headroom/proxy/handlers/anthropic.py`, and re-run the same command. The request under test carries a `tool_search_tool_result` referencing `Grep`, a `tools` array containing `Grep`, and a registered turn hook that removes `Grep`. - Observed result: with the fix reverted the primary test fails on exactly the shape upstream 400s on, because the forwarded body still carries a `tool_reference` naming a tool the turn hook had already removed from `tools`. Restoring the handler change turns it green. The second test passes in both states by design: it pins the converse (a hook that leaves `tools` alone must not cause over-stripping), so the fix cannot regress into "strip always". Verbatim output of the reverted run: ```text $ git checkout HEAD~1 -- headroom/proxy/handlers/anthropic.py # revert ONLY the ordering fix $ uv run --extra dev pytest tests/test_proxy/test_tool_search_repair_after_turn_hooks.py -q collected 2 items tests/test_proxy/test_tool_search_repair_after_turn_hooks.py F. [100%] =================================== FAILURES =================================== ____________ test_repair_sees_the_tools_array_the_hook_left_behind _____________ tests/test_proxy/test_tool_search_repair_after_turn_hooks.py:166: in test_repair_sees_the_tools_array_the_hook_left_behind assert _referenced_tool_names(forwarded) == [] E AssertionError: assert ['Grep'] == [] E E Left contains one more item: 'Grep' FAILED tests/test_proxy/test_tool_search_repair_after_turn_hooks.py::test_repair_sees_the_tools_array_the_hook_left_behind ``` - Not tested: no live-API reproduction of the 400 itself, since triggering it needs a third-party turn hook that removes a tool and none ships in-tree (the assertion above is on the forwarded body, which is the input that produces the 400); no streaming-path variant, since the repair mutates `body` upstream of the stream/buffered split so both inherit it but only the buffered path is asserted; no performance measurement, since the change moves an existing call ~60 lines later in the same function and adds no work. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation — N/A, no user-facing or configuration surface changes - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes - **CI:** `test (4)` failed on `tests/test_tokenizer_count_offload.py::test_count_tokens_offloaded_keeps_loop_responsive` with `assert 0 >= 5`. That is an event-loop-responsiveness timing assertion under a shared runner, and it is unrelated to this diff — nothing here touches the tokenizer or the offload path. It passes locally (`10 passed in 1.43s`). I do not have rerun permission on this fork PR (`gh run rerun` → `cannot be rerun`), so a maintainer rerun is needed to clear it. - **Codecov:** reports "All modified and coverable lines are covered by tests". The accompanying warning is the repo-level "install the Codecov app" notice, not a finding against this PR. - Surfaced while confirming that headroomlabs-ai#2807 and headroomlabs-ai#2848 supersede headroomlabs-ai#2507, which is now closed as such. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
…t the git source ## Description `headroom/mcp_registry/install.py` (`build_serena_spec`) and the wrap-time Serena pre-index in `headroom/cli/wrap.py` both ran: ``` uvx --from git+https://github.com/oraios/serena serena ... ``` The git source forces a from-source build. On proot-based filesystems (Termux + proot-distro on Android, some restricted Linux) `uv` cannot hardlink build dependencies into a fresh build venv, so the build fails immediately and Serena's MCP server fails to start on every `headroom wrap codex` launch: ``` × Failed to download and build `serena-agent @ git+https://github.com/oraios/serena@<commit>` ╰─▶ failed to hardlink file ... Operation not permitted (os error 1) ``` Setting `UV_LINK_MODE=copy` fixes it in an interactive shell, but Codex strips most env vars from the MCP subprocesses it spawns, so that workaround does not reliably reach Serena's launch. Serena publishes the official `serena-agent` package to PyPI with prebuilt wheels, and it exposes the same `serena` console script (`serena = "serena.cli:top_level"` in the project's `pyproject.toml`), so `uvx --from serena-agent serena ...` runs the identical command without a build step. On platforms where the git build already worked there is no functional difference. Fixes headroomlabs-ai#2871 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/mcp_registry/install.py` (`build_serena_spec`): `--from git+https://github.com/oraios/serena` -> `--from serena-agent`. - `headroom/cli/wrap.py` (Serena `project index` pre-warm): same swap. - `tests/test_mcp_registry/test_install.py`: updated the spec assertion and added `test_build_serena_spec_uses_pypi_not_git_source` (asserts `serena-agent` is used and no `git+` source remains). - `tests/test_cli/test_wrap_serena_boost.py`: the pre-index test now asserts `serena-agent` is in the command and the git source is not. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Fail-before (source swap stashed, updated tests kept): tests/test_mcp_registry/test_install.py::test_build_serena_spec_uses_agent_context FAILED tests/test_mcp_registry/test_install.py::test_build_serena_spec_uses_pypi_not_git_source FAILED tests/test_cli/test_wrap_serena_boost.py::test_preindex_runs_serena_in_cwd FAILED # Pass-after: tests/test_mcp_registry/ tests/test_cli/test_wrap_serena_boost.py tests/test_cli/test_serena_migrate.py tests/test_cli/test_serena_disable.py 135 passed # uvx ruff@0.15.17 check -> All checks passed! # uvx mypy@1.20.2 headroom/mcp_registry/install.py -> Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1, ruff 0.15.17 and mypy 1.20.2 via uvx. - Exact command / steps: confirmed `serena-agent` exists on PyPI (v1.6.1, homepage github.com/oraios/serena) and that its `pyproject.toml` declares `[project.scripts] serena = "serena.cli:top_level"`, so the `serena start-mcp-server ...` invocation is unchanged. Swapped both `--from` sources, then fail-before with `git stash push headroom/mcp_registry/install.py headroom/cli/wrap.py` (the two production-asserting tests fail on the old git source) and pass-after with `git stash pop` (135 serena-suite tests pass). Verified no `git+https://github.com/oraios/serena` references remain in `headroom/`. - Observed result: `build_serena_spec` and the pre-index command now install Serena from the `serena-agent` PyPI wheel, so a proot environment gets the prebuilt wheel instead of a from-source build that cannot hardlink. The migration/ledger tests, which use the old git spec as a deliberately-stale fixture, are unaffected. - Not tested: a live `headroom wrap codex` on a real proot/Termux device (not available here). The change is a package-source swap verified against Serena's own published package metadata and the existing spec/command tests. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md`: it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes The git source was unpinned (tracked the repo default branch), so switching to `serena-agent` from PyPI does not lose a version pin; if anything it is more reproducible. The issue reporter also noted that `headroom wrap codex` force-rewrites the Serena block in `~/.codex/config.toml` from this template on every launch, which is why the fix has to live in the package source rather than a user config edit -- this PR puts it there.
…ode child traffic ## Description The OpenCode transport plugin injects `NODE_OPTIONS=--import=<...>/hook-shim/handler.js` into every spawned Node child so its `fetch`/`http` traffic routes through the proxy (`transport.ts` wraps those globals only in the plugin's own process; a spawned `npx` MCP server or `tokensave serve` is a fresh process). That shim was never shipped in the wheel: - Only `headroom/providers/opencode/_dist/entry.opencode.js` is committed and packaged. - The shim source at `plugins/opencode/hook-shim/handler.js` imports the non-bundled `../dist/index.js`, which a pip install (no `node_modules`) cannot resolve. Before headroomlabs-ai#2806, the missing file crashed every Node MCP under `headroom wrap opencode` with `ERR_MODULE_NOT_FOUND` at the ESM loader, before the stdio handshake. headroomlabs-ai#2806 added an `existsSync` guard so the loader is not injected when the shim is absent, which stopped the crash but left child-process routing silently disabled for all wheel installs (headroomlabs-ai#2850). This ships the shim. It builds a self-contained variant in the standalone tsup config (`src/hook-shim.ts`, with the transport bundled inline like the entry, since site-packages has no `node_modules`), and commits it to `headroom/providers/opencode/hook-shim/handler.js` -- the sibling of `_dist/` that `transport.ts`'s `shimImportSpecifier()` resolves via `../hook-shim/handler.js`. maturin packages every file under `headroom/`, so the wheel now carries it, and `existsSync` finds it, so the loader routes spawned Node children again. Fixes headroomlabs-ai#2850 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `plugins/opencode/src/hook-shim.ts` (new): self-contained Node `--import` loader that installs the transport from the inlined `./transport.js`. - `plugins/opencode/tsup.standalone.config.ts`: add `hook-shim/handler` as a second standalone entry. - `headroom/providers/opencode/hook-shim/handler.js` (new): the committed self-contained shim (output of `npm run build:standalone`), shipped by maturin. - `.github/workflows/opencode-plugin.yml`: byte-compare the committed shim against a fresh build (mirrors the existing `entry.opencode.js` guard), and add the shim path to the workflow triggers. - `tests/test_providers_opencode_plugin_path.py`: added `test_hook_shim_is_committed_next_to_the_entry_bundle` asserting the shim ships as a sibling of `_dist/` and is the self-contained build. ## Testing - [x] Unit tests pass (`pytest` + `vitest`) - [x] Type checking passes (`tsc --noEmit`) - [x] New tests added for new functionality - [x] Committed shim rebuilt and byte-matches the standalone build - [ ] Manual testing performed ### Test Output ```text # Fail-before (shim removed from the package): tests/test_providers_opencode_plugin_path.py::test_hook_shim_is_committed_next_to_the_entry_bundle FAILED # Pass-after: tests/test_providers_opencode_plugin_path.py tests/test_providers_opencode_install.py tests/test_providers_opencode_config.py 49 passed, 1 pre-existing failure # the 1 failure (test_build_launch_env_with_project) fails identically on pristine main: # a Windows path-escaping quirk in OPENCODE_CONFIG_CONTENT, unrelated to this diff. # TypeScript: npm run typecheck (clean), npm test -> 14 passed # Standalone build: entry.opencode.js byte-unchanged vs the committed blob; # dist-standalone/hook-shim/handler.js cmp-matches the committed shim. # Shim runtime sanity (node): # with HEADROOM_OPENCODE_TRANSPORT_PROXY_URL set -> loads, exit 0, wraps globalThis.fetch # without it -> throws "loaded without HEADROOM_OPENCODE_TRANSPORT_PROXY_URL", exit 1 ``` ## Real Behavior Proof - Environment: Windows 11, Node v24.11.0, npm 11.5.2, tsup 8.5.1 / esbuild 0.28.1 (pinned via `npm ci`), Python 3.12.11, pytest 9.1.1, ruff 0.15.17. - Exact command / steps: confirmed `transport.ts` resolves `../hook-shim/handler.js` next to the loaded entry (so the wheel needs it at `providers/opencode/hook-shim/handler.js`), that the current wheel ships only `_dist/entry.opencode.js`, and that maturin packages every file under `headroom/`. Added the standalone shim entry, ran `npm run typecheck` and `npm test` (clean), `npm run build:standalone`, verified `entry.opencode.js` is byte-identical to the committed git blob (the standalone build is reproducible; my working copy was only autocrlf-inflated), copied the built shim to the wheel path, and exercised it in Node: it installs the transport (wraps `fetch`) with the proxy env set and throws without it. Fail-before by removing the shim (the new Python test fails); pass-after restored. - Observed result: `headroom/providers/opencode/hook-shim/handler.js` now ships in the package as a self-contained module, so a pip-installed `headroom wrap opencode` routes spawned Node children (npx MCPs, `tokensave serve`) through the proxy instead of leaving them unrouted, and never crashes them. - Not tested: a full pip-install-and-spawn on Linux with a live OpenCode session (no OpenCode client here). The shim is verified to load and wrap `fetch` under Node, the bundle is reproducible and byte-checked by CI, and the packaging path is maturin's standard file inclusion under `headroom/`. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md`: it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes The checkout keeps using `plugins/opencode/hook-shim/handler.js` (which imports `../dist/index.js` from the regular build), so dev behavior is unchanged; only the wheel gains the self-contained sibling. `entry.opencode.js` is byte-unchanged, so its existing CI guard still passes. The committed shim is stored with LF endings so the Linux CI byte-compare matches.
## Description The injected `headroom` OpenCode provider uses `@ai-sdk/openai-compatible` and the proxy's `/v1/chat/completions` route. It currently advertises Claude model IDs in that provider, so OpenCode sends Claude requests to the OpenAI upstream and receives `invalid_api_key` errors. Keep Claude on OpenCode's native `anthropic` provider, which Headroom already redirects to the proxy. Closes headroomlabs-ai#2911 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (bug fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Remove Claude IDs from the injected OpenAI-compatible provider model map. - Keep GPT models available through the `headroom/<id>` namespace. - Add regression assertions that generated config never advertises Claude models on this endpoint. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text python -m pytest tests/test_providers_opencode_config.py -q -k "not build_launch_env_with_project" 40 passed, 1 deselected python -m ruff check headroom/providers/opencode/config.py tests/test_providers_opencode_config.py All checks passed! python -m compileall -q headroom/providers/opencode/config.py tests/test_providers_opencode_config.py (pass) ``` The full config test module also exposes an unrelated pre-existing Windows path assertion failure in `test_build_launch_env_with_project`; the failure is caused by comparing a native `Path` string with JSON-escaped backslashes and is outside this change. ## Real Behavior Proof - Environment: Windows 11, Python 3.11; no external API credentials used. - Exact command / steps: `python -c "from headroom.providers.opencode.config import headroom_provider_entry; print(sorted(headroom_provider_entry(8787)['models']))"` - Observed result: `['gpt-4.1', 'gpt-4o']`; the generated OpenAI-compatible provider no longer advertises any `claude-*` IDs. - Not tested: live OpenCode request routing or a vendor API call, because they require external credentials. The regression suite verifies the generated configuration consumed by OpenCode. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas (not needed; the provider routing rationale is documented inline) - [ ] I have made corresponding changes to the documentation (the generated provider behavior is documented in code; existing docs describe the separate npm provider) - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing relevant unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes The native `anthropic` and `openai` provider entries both continue to point at the Headroom proxy, so this change only removes an invalid duplicate Claude route and does not affect native Claude traffic.
## Description Retains the periodic TOIN statistics task on application state and reaps it during proxy lifespan shutdown. Fixes headroomlabs-ai#2896 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Store the periodic TOIN task as `app.state.periodic_toin_stats_task` when enabled. - Cancel and await the task with the existing bounded shutdown helper before stopping proxy resources. - Clear the application state reference after shutdown. - Add regression coverage proving the task is canceled and reaped when the FastAPI lifespan exits. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text python -m pytest -q tests/test_proxy_telemetry_env.py 0 items / 1 error ModuleNotFoundError: No module named 'headroom._core' Temporary in-process native-core stub + real FastAPI TestClient: python -m pytest -q tests/test_proxy_telemetry_env.py 8 passed ruff check . All checks passed! ruff format --check . 1382 files already formatted python -m mypy headroom Success: no issues found in 515 source files python -m pytest -q Collected 8878 items / 174 errors / 18 skipped. Interrupted during collection because this Windows environment lacks the compiled headroom._core extension. ``` ## Real Behavior Proof - Environment: Windows, Python 3.12, real FastAPI `TestClient` lifespan; only the unavailable native `headroom._core` import was replaced with an in-process test stub. - Exact command / steps: Ran the telemetry test module with the temporary core stub. The new test enabled periodic TOIN stats, held the real lifespan open, observed the stored task, exited the `TestClient` context, and checked that the task was canceled and the state reference cleared. - Observed result: 8 telemetry tests passed, including the new shutdown regression test; the periodic task reported canceled after lifespan exit and no task reference remained on application state. - Who maintains it: Headroom Labs maintains this active upstream repository and proxy lifecycle. - Install surface: No dependencies or install behavior changed. The fix uses existing asyncio and FastAPI lifecycle APIs; no native code or runtime network access is introduced. - Not tested: The complete suite and the unmodified proxy test command cannot run in this Windows environment without the compiled `headroom._core` extension. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes (full suite blocked by missing native extension; stubbed focused tests pass) - [x] I did not edit `CHANGELOG.md` - it is generated by release-please from my Conventional Commit PR title. ## Screenshots (if applicable) Not applicable. ## Additional Notes The shutdown uses the existing three-second `_timed()` bound and handles the disabled configuration without creating a task.
## Description `CompressionCache.max_entries` bounded the main compression cache, but not `_stable_hashes` or `_first_seen`. A long-lived session could therefore retain every unique tool-result hash even while `_cache` stayed empty. This change applies the same bounded retention to both side tables. It also cleans up expired first-seen entries and resets the timing window when compression occurs near the TTL boundary. Fixes headroomlabs-ai#2874 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Store stable hashes and first-seen timestamps in ordered mappings. - Evict oldest entries when either side table exceeds `max_entries`. - Keep all bookkeeping under the existing reentrant lock. - Reset first-seen timing after compression near the TTL boundary. - Add tests covering size limits, TTL behavior, frozen-prefix safety, and concurrency. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text uv run ruff format --check . Passed uv run ruff check . All checks passed! uv run mypy headroom Success: no issues found in 515 source files uv run pytest Passed ``` Focused cache tests on macOS 26.5.2 arm64 with Python 3.11.14: ```text uv run pytest tests/test_compression_cache.py::TestCompressionCacheRetention -v 5 passed in 0.30s uv run pytest tests/test_compression_cache.py -q 38 passed in 5.76s ``` After the final formatting-only commit, the cache test file was also run on Linux with Python 3.12.13: ```text 37 passed, 1 skipped in 32.70s ``` ## Real Behavior Proof - Environment: Linux 6.18 x86_64, Python 3.12.13, `CompressionCache(max_entries=100)`. - Exact command / steps: Created a `CompressionCache(max_entries=100)`, generated 20,000 unique content hashes, and passed each hash through `mark_stable()` and `should_defer_compression()`. Store sizes were sampled after 100, 1,000, 5,000, and 20,000 results. - Observed result: `_cache=0`, `_stable_hashes=100`, and `_first_seen=100` at every sample after reaching the configured limit. At 20,000 results, traced memory was approximately 0.03 MB current and 0.04 MB peak. Before the fix, the same workload retained all 20,000 hashes and timestamps. - Not tested: A live multi-hour proxy/provider session. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented the code where retention behavior is not obvious - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [x] New and existing unit tests pass locally - [x] I did **not** edit `CHANGELOG.md` ## Screenshots N/A — internal cache bookkeeping change. ## Additional Notes No changes to dependencies, public APIs, or configuration. No user-facing behavior changes.
## Description `RequestLog.timestamp` was serialized with `datetime.now().isoformat()`, which omits timezone information. Browsers then interpret the value as local time, so requests from a UTC container can display negative ages in non-UTC dashboards. Closes headroomlabs-ai#2910 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) ## Changes Made - Emit request-log timestamps from `datetime.now(timezone.utc)` so the ISO-8601 value includes `+00:00`. - Add a regression test that parses the emitted timestamp and requires a UTC offset. ## Testing - [x] New tests added for the regression - [x] `python -m compileall -q headroom/proxy/outcome.py tests/test_request_outcome.py` - [x] `git diff --check` - [ ] Unit tests pass (`pytest`) — the repository's Rust extension cannot build in this Windows environment because `link.exe` (MSVC) is unavailable; the focused test is included for CI. ### Test Output ```text python -m compileall -q headroom/proxy/outcome.py tests/test_request_outcome.py (pass) git diff --check (pass) uv run pytest tests/test_request_outcome.py -q blocked while building headroom-py: linker `link.exe` not found ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.11; proxy timestamps are generated in `headroom/proxy/outcome.py`. - Exact command / steps: traced the Recent Requests write path and added a timestamp assertion in `tests/test_request_outcome.py` (CI will run with the project's Rust toolchain). - Observed result: the production call now emits an ISO-8601 timestamp with `+00:00`; the regression assertion requires an offset-aware UTC value, preventing browser timezone skew. - Not tested: full pytest suite locally because the MSVC linker is unavailable. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review - [x] I have added tests that prove my fix is effective - [x] I did not edit `CHANGELOG.md` Signed-off-by: Suliman Abdulrazzaq <suliman9000a@gmail.com>
## Description Closes the initialized LocalBackend and cancels in-flight initialization whenever the memory MCP stdio transport exits. Fixes headroomlabs-ai#2898 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added an explicit server cleanup callback that cancels and awaits pending backend initialization. - Closes an initialized backend exactly once and clears the backend/task references. - Runs cleanup in `_run()` through a `finally` block after the stdio transport exits, including transport errors. - Added regression coverage for initialized cleanup, pending initialization cancellation, idempotence, and `_run()` shutdown behavior. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text python -m pytest -q tests/test_memory/test_mcp_server.py 15 passed, 20 warnings ruff check . All checks passed! ruff format --check . 1382 files already formatted python -m mypy headroom Success: no issues found in 515 source files python -m pytest -q Collected 8881 items / 174 errors / 18 skipped. Interrupted during collection because this Windows environment lacks the compiled headroom._core extension. ``` ## Real Behavior Proof - Environment: Windows, Python 3.12, async MCP server lifecycle test with the real `create_memory_server()` closure and an embedded server transport stub. - Exact command / steps: Ran `python -m pytest -q tests/test_memory/test_mcp_server.py`; the regression tests initialized a backend through the server's registered tool lifecycle, returned the stdio transport, and invoked the cleanup callback from `_run()`'s `finally` path. - Observed result: 15 tests passed. Initialized backends were closed once, pending initialization was cancelled and awaited, and transport exit invoked cleanup even when the server run returned. - Who maintains it: Headroom Labs maintains this active upstream repository and memory MCP server. - Install surface: No dependencies or install behavior changed. The fix uses existing asyncio lifecycle handling and `LocalBackend.close()`; no native code or runtime network access is introduced. - Not tested: The complete repository suite could not run past collection because this Windows environment lacks the compiled `headroom._core` extension. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes (full suite blocked by missing native extension; targeted tests pass) - [x] I did not edit `CHANGELOG.md` - it is generated by release-please from my Conventional Commit PR title. ## Screenshots (if applicable) Not applicable. ## Additional Notes Cleanup is attached to each created memory MCP server and is idempotent, so embedded callers can invoke the same lifecycle callback safely if needed.
…List spam ## Description The proxy repeatedly prints LiteLLM's `Provider List: https://docs.litellm.ai/docs/providers` banner during normal operation, with no explanation or way to suppress it (headroomlabs-ai#2851). Root cause: `_resolve_litellm_model()` in `headroom/proxy/savings_tracker.py` runs on every savings-tracking update (i.e. every request). For any model LiteLLM can't price (a custom/local/gateway model name — e.g. the reporter's local oMLX setup), the uncached fallback path calls `litellm.cost_per_token(...)` purely to probe resolvability. When that probe fails, LiteLLM prints the banner as an internal side effect before raising, and since the probe was never cached, it re-fires on every single request for the same unresolvable model. **Update:** review flagged that the first version of this fix cached into a plain, unbounded `dict` keyed by the (client-controlled) model name — a memory-retention path on a request-facing proxy, since a caller can grow it without limit by sending a new model string on every request. Replaced with a bounded `functools.lru_cache`; see Changes Made below. Closes headroomlabs-ai#2851 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/proxy/savings_tracker.py`: `_resolve_litellm_model()` is now decorated with `@lru_cache(maxsize=256)` instead of backing onto a hand-rolled unbounded `dict`. An evicted model name simply re-probes LiteLLM on next use — never a correctness issue, only whether the noisy failure banner reruns for that specific name. - `tests/conftest.py`: added a global `autouse` fixture, `_reset_litellm_model_resolution_cache`, that clears the cache before and after every test. It's process-lifetime and module-global, and several existing tests monkeypatch `savings_tracker.litellm` with different behavior per test while reusing common model names like `"gpt-4o"` — without a reset, whichever test resolves a name first silently wins that cache slot for the rest of the run and later tests stop exercising their own fake. - `tests/test_savings_tracker_litellm_resolution_cache.py` (new): regression tests for the three properties that actually matter — repeated resolution of one unknown model only probes LiteLLM once, resolving far more distinct names than the bound never grows the cache past it, and an evicted name is transparently re-probed rather than reusing a slot it no longer owns. - No behavior change for models LiteLLM can already price (fast path via `model_cost` lookup) — only the noisy uncached probe path is memoized, same as before. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) — not run; `mypy` isn't installed in this environment - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python3 -m pytest tests/test_proxy_savings_history.py tests/test_savings_tracker_zero_price.py \ tests/test_savings_tracker_litellm_resolution_cache.py -q tests/test_proxy_savings_history.py .................................... [ 73%] ... [ 79%] tests/test_savings_tracker_zero_price.py ....... [ 93%] tests/test_savings_tracker_litellm_resolution_cache.py ... [100%] 49 passed, 1 warning in 1.26s # Re-run in reversed file order to check for the exact order-dependence the # review flagged — same 49 passed, no failures either direction: $ python3 -m pytest tests/test_savings_tracker_litellm_resolution_cache.py \ tests/test_savings_tracker_zero_price.py tests/test_proxy_savings_history.py -q 49 passed, 1 warning in 1.11s $ python3 -m ruff check headroom/proxy/savings_tracker.py tests/conftest.py \ tests/test_savings_tracker_litellm_resolution_cache.py All checks passed! ``` ## Real Behavior Proof - Environment: macOS, Python 3.12.3, this repo checked out locally. - What changed since the last review pass: I got the compiled `headroom._core` Rust extension in hand (by installing the published `headroom-ai[all]` wheel into a separate venv and copying its `_core.abi3.so` next to this local source tree — same Python ABI, pure-Python edits in `savings_tracker.py` don't touch the compiled boundary). That unblocked the full test files this fix touches, including `tests/test_proxy_savings_history.py`, which was previously reported as untestable here. - Exact command / steps: three properties asserted directly against the real (now-bounded) cache in `tests/test_savings_tracker_litellm_resolution_cache.py`: 1. Resolve the same unresolvable model 5 times → assert the underlying `litellm.cost_per_token` probe fired exactly once. 2. Resolve `_MODEL_RESOLUTION_CACHE_MAXSIZE + 50` distinct model names → assert `_resolve_litellm_model.cache_info().currsize` stays at exactly `_MODEL_RESOLUTION_CACHE_MAXSIZE` (256), never higher — this is the actual memory-retention fix the review asked for. 3. Resolve one model, push exactly `maxsize` other distinct names through to evict it via LRU, then resolve it again → assert it re-probed (call count went 1 → 2), proving eviction is real and not just an untested cache_info number. - Observed result: all three pass; full affected-file suite (49 tests) passes in both forward and reversed run order, confirming the new `conftest.py` fixture actually fixes the cross-test leakage risk (verified by literally reordering the files, not just by inspection). - Not tested: a live HTTP request against a running `headroom proxy` process specifically re-exercising this bounded-cache commit — the earlier "20 simulated requests" proof against the previous (unbounded-dict) version of this fix was via a standalone script, not a real server; I have not repeated that specific live-server pass against this commit. The unit-level proof above exercises the exact same function (`_resolve_litellm_model`) the real proxy calls per-request from `headroom/proxy/server.py`, so I'm confident it generalizes, but flagging the gap rather than implying I re-ran it live. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas — the bound/eviction rationale is commented above `_resolve_litellm_model`, and the cross-test leakage rationale is commented above the new `conftest.py` fixture - [ ] I have made corresponding changes to the documentation — N/A, internal implementation detail with no user-facing API/doc surface - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md` ## Additional Notes - `mypy` still hasn't been run — not installed in this sandbox, and I didn't want to widen the PR further by installing/configuring it just for this. Flagging rather than silently skipping. - The earlier "Additional Notes" gap about `test_proxy_savings_history.py` being untestable in this environment is resolved (see Real Behavior Proof) — it now runs and passes, including the pre-existing `test_litellm_resolution_and_savings_estimation_fallbacks` test that exercises `_resolve_litellm_model` with a mutated `model_cost` dict across several assertions in one test. - Deliberately did not also bound `headroom/pricing/litellm_pricing.py`'s sibling `_resolved_model_cache` — same shape of cache, arguably the same exposure — since it's outside this PR's diff and touching it wasn't asked for. Flagging in case a maintainer wants it as a fast follow-up rather than silently leaving it unmentioned. --------- Co-authored-by: connectsudhindra-gif <connectsudhindra-gif@users.noreply.github.com> Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
## Description Adds an explicit, idempotent async cleanup lifecycle for the LiteLLM callback's shared cloud HTTP client. Fixes headroomlabs-ai#2894 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `HeadroomCallback.aclose()` to close the lazily-created `httpx.AsyncClient` and clear its reference. - Made cleanup safe when cloud mode was never used and when shutdown cleanup is invoked more than once. - Added regression coverage for initialized-client cleanup, reference clearing, and repeated/no-op cleanup. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text python -m pytest -q tests/test_integrations/test_litellm_callback.py 5 passed ruff check . All checks passed! ruff format --check . 1382 files already formatted python -m mypy headroom Success: no issues found in 515 source files python -m pytest -q Collection blocked in this Windows environment by 174 errors, primarily missing compiled headroom._core; one unrelated test also lacks respx. ``` ## Real Behavior Proof - Environment: Windows, Python 3.12, loopback HTTP server, real `httpx.AsyncClient`. - Exact command / steps: Started a local HTTP server, configured `HeadroomCallback(api_key="hdr_test", api_url="http://127.0.0.1:<port>")`, ran `_cloud_compress()` against it, saved the created client, awaited `callback.aclose()`, then awaited `callback.aclose()` again. - Observed result: The real cloud request succeeded; the client was open during the request, reported closed after `aclose()`, the callback reference became `None`, and repeated cleanup was harmless. - Who maintains it: Headroom Labs maintains this active upstream repository and its LiteLLM integration. - Install surface: No dependencies or install behavior changed. Cloud mode continues to use the existing optional `httpx` dependency; no native code or runtime network access is introduced by this fix. - Not tested: The complete test suite could not run past collection because the local Windows environment lacks the compiled `headroom._core` extension. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] Documentation changes are not required; `aclose()` is documented in its public docstring and the host owns shutdown sequencing - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes (full suite blocked by missing native extension; targeted tests pass) - [x] I did not edit `CHANGELOG.md` - it is generated by release-please from my Conventional Commit PR title. ## Screenshots (if applicable) Not applicable. ## Additional Notes The callback exposes `aclose()` for the host application's async shutdown lifecycle, matching the existing ASGI integration pattern.
## Description Fixes headroomlabs-ai#2895 The repository-wide Ruff command failed on the bundled OAuth2 plugin. This change sorts the public export list, narrows the optional LiteLLM setup exception handling to expected failures, and replaces the silent HTTP error-body drain with explicit handling and debug logging. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Sorted headroom_oauth2.__all__ according to Ruff RUF022. - Replaced the blind install-time Exception catch with explicit ImportError, AttributeError, OSError, TypeError, and ValueError handling. - Replaced the silent HTTPError body-drain pass with explicit HTTPException, OSError, and ValueError handling plus debug logging. - Added regression coverage for body-drain failures and invalid LiteLLM header state. ## Testing - [x] Unit tests pass - [x] Linting passes (ruff check .) - [x] Type checking passes (mypy headroom) - [x] New tests added - [x] Manual testing performed ### Test Output ruff 0.15.17 ruff check . All checks passed! ruff format --check . 1382 files already formatted python -m mypy headroom Success: no issues found in 515 source files PYTHONPATH=plugins/headroom-oauth2/src python -m pytest -q plugins/headroom-oauth2/tests 39 passed in 11.12s Full Python pytest was attempted: 8,878 tests were collected, but collection stopped with 174 environment errors because the required compiled headroom._core extension is unavailable in this Windows checkout. 18 tests were skipped. ## Real Behavior Proof - Environment: Windows PowerShell, Python 3.12, Ruff 0.15.17. - Exact command / steps: Ran the OAuth2 test suite with PYTHONPATH pointing to plugins/headroom-oauth2/src. Its local HTTPServer fixture exercised real urllib token minting, cached refresh, HTTP error handling, and middleware injection. - Observed result: 39 tests passed, including real loopback token minting and the new failure-path tests; repository-wide Ruff completed with no diagnostics. - Not tested: External identity-provider traffic and the full Python suite after native extension build, because the local Windows toolchain cannot build headroom._core. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of the code - [x] I have commented my code where needed - [ ] I have made corresponding changes to the documentation (not needed; behavior and lint handling are covered by existing comments/tests) - [x] My changes generate no new warnings - [x] I have added tests that prove the fix is effective - [ ] New and existing full-repository unit tests pass locally (blocked by missing native headroom._core) - [x] I did not edit CHANGELOG.md ## Additional Notes No dependencies or public API behavior changed. Expected environment and transport failures remain handled; unexpected programmer errors now propagate instead of being silently swallowed. The OAuth2 plugin remains standard-library-only. --------- Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
## Summary Closes headroomlabs-ai#2909. The `persistent-docker` installer now discovers Docker's default bridge gateway and passes the exact `/32` gateway CIDR to the proxy's dashboard metadata allowlist when no explicit `HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS` value is configured. This keeps the existing metadata gate intact while allowing the first-party loopback-published container to see its own Recent Requests and Per-Project Savings data. Explicit user configuration continues to take precedence. Both native wrappers (POSIX and PowerShell) use the same behavior, and installer integration coverage verifies the generated Docker command. ## Validation - `python -m pytest tests/test_install/test_native_installers.py -q -k bash` (1 skipped on Windows because Bash is unavailable) - PowerShell wrapper smoke test with the repository fake Docker shim: verified `docker network inspect bridge` is called and `HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS=172.17.0.1/32` is passed to `docker run` - Explicit allowlist smoke test: verified an existing `HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS` is preserved without adding a discovered default - `git diff --check` ## Real behavior proof Setup tested: Windows 11 host, PowerShell wrapper, repository fake Docker shim (Docker CLI is not installed in this environment). Exact command: `headroom.ps1 install apply --profile smoke --port 18999 --image fake/headroom:test`. Observed result: the generated Docker invocation included `docker network inspect bridge --format ...` and `--env HEADROOM_PROXY_TRUSTED_DASHBOARD_CLIENT_CIDRS=172.17.0.1/32`, and the installer completed successfully. Not tested: a live Docker daemon/dashboard request on this host. --------- Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
## Description `DirectMem0Adapter.close()` now deterministically drains or cancels background writes and releases every initialized client/driver. Fixes headroomlabs-ai#2897 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Initialize the OpenAI client field to `None` so cleanup is safe before or after initialization. - Drain background tasks within a configurable 60-second default, cancel tasks that exceed the timeout, await cancellation, and retain completed/cancelled task status. - Close Mem0, OpenAI, Qdrant, Neo4j, embedder, and graph resources independently, including async close methods, while continuing cleanup if one resource fails. - Clear task and client references and keep `close()` idempotent. - Add regression tests for task draining, timeout cancellation, all resource cleanup, and repeated close calls. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text python -m pytest -q tests/test_memory/test_direct_mem0.py tests/test_memory/test_qdrant_env.py 52 passed ruff check . All checks passed! ruff format --check . 1383 files already formatted python -m mypy headroom Success: no issues found in 515 source files python -m pytest -q Collection blocked in this Windows environment by 174 errors, primarily missing compiled headroom._core; 18 tests skipped. ``` ## Real Behavior Proof - Environment: Windows, Python 3.12, local DirectMem0Adapter instance using real `httpx.Client` resources. - Exact command / steps: Assigned real `httpx.Client()` instances to the adapter's OpenAI and Qdrant resource slots, registered an asynchronous background task, awaited `adapter.close(timeout=1.0)`, then checked both clients' `is_closed` state and the task status. - Observed result: `real httpx clients closed and background task drained`; both clients reported closed, no pending task IDs remained, and the task status was `completed`. - Who maintains it: Headroom Labs maintains this active upstream repository and memory backend. - Install surface: No dependencies or install behavior changed. The fix uses the standard-library asyncio/inspect modules and existing resource close methods; no native code or runtime network access is introduced. - Not tested: The complete test suite could not run past collection because this Windows environment lacks the compiled `headroom._core` extension. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [ ] New and existing unit tests pass locally with my changes (full suite blocked by missing native extension; targeted tests pass) - [x] I did not edit `CHANGELOG.md` - it is generated by release-please from my Conventional Commit PR title. ## Screenshots (if applicable) Not applicable. ## Additional Notes The default close timeout is 60 seconds and can be overridden by callers that need a shorter shutdown budget. --------- Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
## Problem `headroom memory delete`, `prune`, `edit`, and `purge` all operate on the bare `SQLiteMemoryStore` — they update the primary `memories` table but never touch the FTS5 full-text index (`memory_fts` in `memory.db`) or the vector index (`vec_metadata` / `vec_embeddings` in `memory_vectors.db`). The index maintenance path lives in `HierarchicalMemory.delete()` / `.update()`, which the CLI never instantiates. **Symptoms (from headroomlabs-ai#2856):** ```sql -- After deleting 16 of 46 memories via CLI: SELECT COUNT(*) FROM memories; -- 30 SELECT COUNT(*) FROM memory_fts; -- 46 ← orphans -- memory_vectors.db SELECT COUNT(*) FROM vec_metadata; -- 46 ← orphans ``` Deleted memories keep surfacing in `memory_search` results even after a full server restart, because server startup only re-embeds memories whose `embedding IS NULL` — it never removes orphaned index entries. Fixes headroomlabs-ai#2856. ## Solution Add two best-effort helpers to `headroom/cli/memory.py` that use **direct SQLite** (no `sqlite-vec` extension, no embedder): - **`_remove_from_search_indexes(db_path, memory_ids)`**: removes specific IDs from `memory_fts` and from `vec_metadata` / `vec_embeddings`. Skips silently if an index doesn't exist. - **`_clear_all_search_indexes(db_path)`**: truncates both indexes completely (for purge). Wire these up in four commands: | Command | Change | |---|---| | `delete` | `_remove_from_search_indexes` after `store.delete_batch()` | | `prune` | `_remove_from_search_indexes` after `store.delete_batch()` | | `purge` | `_clear_all_search_indexes` after `store.clear_all()` | | `edit` | If content changed: remove stale entries, clear `embedding` (server re-embeds on next startup), re-add FTS5 entry with new content immediately | The edit path re-adds the FTS5 entry right away so keyword search reflects the new content without requiring a server restart. Vector search is deferred to the next startup re-embed cycle (same as what the server already does for missing embeddings). ## Changes - `headroom/cli/memory.py` — two new helpers; four command call sites - `tests/test_cli_memory_index_sync.py` (new) — 9 unit tests covering both helpers with FTS5 and a stub vector DB. No `sqlite-vec` or embedder required; tests run locally. ## Testing ``` $ python -m pytest tests/test_cli_memory_index_sync.py -v ... 9 passed in 2.38s ``` --------- Signed-off-by: Radhakrishnan Pachyappan <radhakrishnan.p@op.tech> Signed-off-by: Radhakrishnan Pachyappan <gingeekrishna@gmail.com> Co-authored-by: Tejas Chopra <tejas@Tejass-MacBook-Pro.local>
headroomlabs-ai#2833) ## Description Settings validation only accepted short JSON/API keys, so documented HEADROOM_* env names were rejected as unknown. Users following the docs (for example HEADROOM_LOSSLESS) hit SettingsValidationError / PUT /settings 400 even though those names are already on each registry field. This normalizes known env aliases to their short keys before validate/save, keeps existing short-key behavior, and rejects conflicting env+key pairs for the same field. Closes headroomlabs-ai#2812 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added _BY_ENV and _normalize_values() in settings_store to map documented env names to short keys - Call normalization at the start of validate() and save() so clear/retain paths also accept env aliases - Reject payloads that supply both an env alias and its short key with different values - Add unit coverage for accept/clear/conflict/same-value paths and update registry monkeypatches to rebuild _BY_ENV ## Testing - [x] Unit tests pass (pytest) - [x] Linting passes (ruff check .) - [ ] Type checking passes (mypy headroom) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text python -m pytest tests/test_proxy/test_settings_store.py -q -k "env_alias or validate_accepts or save_rejects or same_env or conflicting or save_accepts or env_alias_clear or anthropic_extra_headers_retain or TestValidation" 23 passed, 11 deselected ruff check headroom/settings_store.py tests/test_proxy/test_settings_store.py tests/test_proxy_settings_endpoints.py All checks passed! ruff format --check headroom/settings_store.py tests/test_proxy/test_settings_store.py tests/test_proxy_settings_endpoints.py 3 files already formatted ``` ## Real Behavior Proof - Environment: Linux x86_64, Python 3.14.5 via contributor venv, worktree of headroom main at 7940c05 plus commit e4c87ed - Exact command / steps: pytest tests/test_proxy/test_settings_store.py focused selection; ruff check and ruff format --check on the three touched files; settings_store.validate({"HEADROOM_LOSSLESS": True}) returns {"lossless": True} - Observed result: Env aliases coerce and persist under short keys; unknown short keys still error; conflicting env+key pairs raise SettingsValidationError; ruff clean on touched files - Not tested: Live dashboard PUT /settings through a running proxy (HTTP suite needs native headroom._core); mypy; full monorepo CI ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did not edit CHANGELOG.md - it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Screenshots (if applicable) N/A ## Additional Notes - Scoped to settings key normalization only - Registry drift Click test was not exercised here because this environment lacks tomlkit for an unrelated import path
…env installs self-update (headroomlabs-ai#2830) ## Description `headroom update` refuses to self-update for any install that happens to run inside a container, including a plain `pip install` into a venv, because `detect_install_method` checks `_in_docker()` before the pipx / uv-tool / venv / user-site branches. The guidance it prints does not apply: there is no Headroom image in the picture, the container is the environment and Headroom was pip-installed into a venv inside it. ```console $ headroom update --check Update available: 0.32.0 -> 0.34.0 Running inside a container - pull a newer Headroom image instead of self-updating. ``` `_in_docker()` is purely environmental (`/.dockerenv` exists, or `HEADROOM_IN_DOCKER` is set), with no reference to how the package was installed, so `/.dockerenv` alone shadows a venv that clearly owns the install. This hits devcontainers, GitHub Codespaces, docker/LXC self-hosting, and dev images. The fix splits the check by intent. An EXPLICIT `HEADROOM_IN_DOCKER` (which the official image can set) is a deliberate opt-out and still refuses up front, even over a venv, so the real-image behavior is preserved. The bare `/.dockerenv` heuristic now runs after ownership detection, so a venv / pipx / uv / user-site install self-updates and only a container whose own system interpreter owns the install still gets the pull-a-new-image guidance. Fixes headroomlabs-ai#2816 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - `headroom/cli/update.py` (`detect_install_method`): replaced the up-front `_in_docker()` refusal with an explicit `os.environ.get("HEADROOM_IN_DOCKER")` refusal (the official image opt-out), and added the bare `_in_docker()` refusal after the pipx / uv-tool / venv / user-site branches so ownership wins over environment. Updated the resolution-order docstring. - `tests/test_update_helpers.py`: added `test_venv_inside_bare_dockerenv_still_self_updates` (the fix), `test_explicit_headroom_in_docker_still_refuses_over_venv` (image opt-out preserved), and `test_bare_dockerenv_without_owner_refuses` (system-interpreter container still refuses). - `tests/test_cli_update.py` (`test_detect_docker`): updated to drive the bare-`/.dockerenv`-no-owner path deterministically (mock ownership to absent), since a real venv underneath now correctly wins. ## Testing - [x] Unit tests pass (`pytest`) - [x] Linting passes (`ruff check .`) - [x] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [ ] Manual testing performed ### Test Output ```text # Fail-before (source fix stashed, new test kept): tests/test_update_helpers.py::test_venv_inside_bare_dockerenv_still_self_updates FAILED assert method.kind == "pip" AssertionError: assert 'docker' == 'pip' # Pass-after (fix applied), all update suites: tests/test_update_helpers.py tests/test_cli_update.py tests/test_update_check.py 95 passed # uvx ruff@0.15.17 check -> All checks passed! # uvx mypy@1.20.2 headroom/cli/update.py -> Success: no issues found in 1 source file ``` ## Real Behavior Proof - Environment: Windows 11, Python 3.12.11, project venv, pytest 9.1.1, ruff 0.15.17 and mypy 1.20.2 via uvx. - Exact command / steps: read `detect_install_method` to confirm `_in_docker()` (line 354) preceded the pipx (377) / uv-tool (385) / venv (392) branches, reproduced the issue's environment in a test (bare `/.dockerenv` via `_in_docker` monkeypatched True, `HEADROOM_IN_DOCKER` unset, a venv layout under `sys.prefix`), fail-before with `git stash push headroom/cli/update.py` and `python -m pytest tests/test_update_helpers.py -k venv_inside_bare_dockerenv` (the venv is refused with `kind == "docker"`), then pass-after with `git stash pop` and rerunning the full update suites (95 passed). - Observed result: a venv/pip install inside a bare `/.dockerenv` container now resolves to `kind="pip"`, `can_self_update=True`, `argv=[sys.executable, "-m", "pip", "install", "-U", ...]`, matching the manual command the issue reporter confirmed works. An explicit `HEADROOM_IN_DOCKER=1` still resolves to `kind="docker"` even over a venv, and a container whose system interpreter owns the install still resolves to `kind="docker"`. - Not tested: an end-to-end `headroom update` run inside a real devcontainer against live PyPI (no container in this environment). The resolution is a pure classification function verified directly, and the actual upgrade command it builds is the existing, already-tested venv path. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [x] I did **not** edit `CHANGELOG.md`: it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this) ## Additional Notes The official image opt-out is preserved by design: the issue notes `_in_docker()` already honors `HEADROOM_IN_DOCKER`, so the image can keep refusing self-update by setting it, which this PR routes to the explicit up-front check that wins even over a venv. Only the bare `/.dockerenv` auto-detection was demoted below ownership.
Fix TOIN privacy leakage and unbounded retention (headroomlabs-ai#2926, headroomlabs-ai#2886).
Harden telemetry and TOIN routes and detail payloads (headroomlabs-ai#2927).
Avoid unsupported CCR tool injection on OpenAI chat streaming (headroomlabs-ai#2924).
Verify the OpenCode executable before changing configuration.
…sed through Ensure explicit Claude model arguments retain the 1M context suffix (headroomlabs-ai#2915).
…arker consolidation Preserve client cache-control breakpoint positions.
…AI shape Convert Anthropic tool requests for AnyLLM OpenAI-compatible backends.
… the streaming path Preserve AnyLLM streaming tool calls and finish reasons.
…ection split Protect custom-tag blocks during mixed-content routing.
…consumed Close unconsumed upstream streaming bodies.
Include tool-search deferral in savings accounting (headroomlabs-ai#2795).
# Conflicts: # headroom/cache/prefix_tracker.py
PR governanceThis PR does not yet satisfy the required template fields:
Please update the PR body, or move the PR back to draft while it is still in progress. |
github-actions
Bot
added
the
status: needs author action
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Closes #
Type of Change
Changes Made
Testing
pytest)ruff check .)mypy headroom)Test Output
Real Behavior Proof
Review Readiness
Checklist
CHANGELOG.md— it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)Screenshots (if applicable)
Add screenshots to help explain your changes.
Additional Notes