Merge branch 'main' into jd/architecture-slice-45

Author: JerrettDavis

.github/workflows/ci.yml

@@ -10,6 +10,8 @@ name: CI
 #
 # Notes: CPU-only torch everywhere (no CUDA stack); test shards run HF_HUB_OFFLINE.
 # Multi-version (3.10/3.11/3.13) coverage on main is a planned follow-up.
+# Windows wheel (win_amd64) built separately — builds the Rust ext just like the
+# Linux wheel, then uploads as a separate artifact for downstream consumption.
 
 on:
   push:
@@ -118,6 +120,31 @@ jobs:
           path: dist/*.whl
           retention-days: 1
 
+  build-wheel-windows:
+    needs: changes
+    if: needs.changes.outputs.code == 'true'
+    runs-on: windows-latest
+    timeout-minutes: 45
+    steps:
+      - uses: actions/checkout@v6
+      - uses: actions/setup-python@v6
+        with:
+          python-version: ${{ env.PY_VERSION }}
+      - uses: dtolnay/rust-toolchain@stable
+      - uses: Swatinem/rust-cache@v2
+        with:
+          workspaces: ". -> target"
+      - name: Build wheel (fast CI cargo profile)
+        shell: bash
+        run: |
+          python -m pip install --upgrade pip maturin
+          maturin build --profile ci --out dist --interpreter "python${{ env.PY_VERSION }}"
+      - uses: actions/upload-artifact@v7
+        with:
+          name: headroom-wheel-windows
+          path: dist/*.whl
+          retention-days: 1
+
   prefetch-model:
     needs: changes
     if: needs.changes.outputs.code == 'true'

CHANGELOG.md

@@ -32,6 +32,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ### Bug Fixes
 
+* **proxy/openai:** stop PRE_SEND from reintroducing `tools: []` after the direct #728 fix. The OpenAI request handler now mirrors the existing `tools or _original_tools is not None` body-write guard during PRE_SEND write-back, so providers that reject empty tool arrays no longer see a tools field when the client omitted it, while explicit client `tools: []` remains preserved ([#1983](https://github.com/headroomlabs-ai/headroom/issues/1983)).
 * **proxy/openai:** keep the exact Responses function name `terminal` resident during OpenAI tool-search deferral so cache-mode optimization stops forwarding `terminal.terminal` and triggering the reserved-namespace 400 on Codex Responses ([#1946](https://github.com/headroomlabs-ai/headroom/issues/1946)).
 * **proxy/openai:** thread the savings-profile kwargs into the live `/v1/chat/completions` compression path. The chat handler called `openai_pipeline.apply()` without `proxy_pipeline_kwargs(config)`, so `HEADROOM_SAVINGS_PROFILE=agent-90` (and the individual `compress_user_messages`/`target_ratio`/`min_tokens_to_compress`/... knobs) were silently dropped — OpenAI-compatible clients like OpenCode kept protecting user messages and missed the configured profile. Both the token-mode and non-token chat branches now pass the profile kwargs, matching `handlers/anthropic.py` and the dedicated OpenAI compress endpoint ([#1534](https://github.com/headroomlabs-ai/headroom/issues/1534)).
 * **proxy:** forward Codex Desktop `/v1/responses` posts byte-faithfully so they stop returning upstream `400 {"detail":"Bad Request"}`. `handle_openai_responses` decoded the inbound body to inspect it but always re-serialized a canonical body on the way out, and it never stripped the inbound `content-encoding` header — so a `content-encoding: zstd` Codex Desktop request was forwarded as already-decoded JSON still advertising `zstd`, and the upstream ChatGPT Codex endpoint rejected it. The handler now keeps the original decoded bytes and forwards them verbatim whenever nothing (compression or memory injection) mutated the request, and drops the stale `content-encoding` header, mirroring the byte-faithful passthrough the chat and Anthropic paths already use ([#1542](https://github.com/headroomlabs-ai/headroom/issues/1542)).
@@ -40,8 +41,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 * **install:** stop leaking a file descriptor on every `headroom install start`. `start_detached_agent()` opened the agent log file and handed it to `subprocess.Popen` but never closed the parent's copy, so each call leaked one fd (and pinned the log file open against rotation). The parent now closes its copy in a `try/finally` once the child has inherited it — the close also runs if `Popen` raises ([#1554](https://github.com/headroomlabs-ai/headroom/issues/1554)).
 * **memory/sync:** stop the Codex AGENTS.md sync adapter from erasing previously-synced memories on every export. `sync_export` hands each adapter only the *delta* (memories the agent lacks), but `CodexAdapter.write_memories` rebuilt its whole managed section from just that delta — so each sync overwrote the section with only the new items, thrashing the file between disjoint subsets and never accumulating. It now merges the delta into the facts already present (deduped), matching the additive contract the ClaudeCode adapter already follows.
 * **memory/sync:** stop the Claude Code sync adapter from clobbering distinct memories that share a first line. `write_memories` derived each file name from the first line of the content only (`headroom_{slug}.md`), so two different DB memories whose first lines slugify identically wrote to the same file and the second silently overwrote the first — and because the loser never landed on disk, the next sync re-exported it, ping-ponging the pair forever. When the slug is already taken by a *different* memory (distinct `headroom_id`) the file name is now disambiguated with a content-hash suffix; an update to the same memory still rewrites its slug file in place, so existing file names are unchanged.
+* **transforms/code:** stop raising `ValueError` on common language hints and fence tags. `CodeAwareCompressor.compress()` built the language with `CodeLanguage(language.lower())`, which only accepts the exact enum values (`python`/`javascript`/`typescript`/…). A markdown ` ```js ` / ` ```ts ` / ` ```py ` fence tag (or any caller passing an alias) raised `ValueError` — crashing direct callers, and inside the content router the error was swallowed so those blocks silently skipped code-aware compression. A new `coerce_language` helper maps the common aliases to their canonical language and returns `UNKNOWN` (never raises) for unrecognized tags, falling back to content-based detection.
 * **cli/proxy:** honor `HEADROOM_MIN_TOKENS=0` / `HEADROOM_MAX_ITEMS=0`. The Click `proxy` command built these with `_get_env_int_optional(name) or 500`/`or 50`, so an explicit `0` — a legitimate value (`min_tokens_to_crush=0` means "crush every item") — was treated as falsy and silently replaced with the default. The `headroom proxy` argparse path already preserved `0` via `_get_env_int`, so the two entry points disagreed. The Click path now uses the same None-checking helper.
 * **proxy:** strip the inbound `Content-Encoding`/`Transfer-Encoding` request headers on the Anthropic `/v1/messages` and OpenAI `/v1/chat/completions` paths before forwarding upstream. `read_request_json_with_bytes` already decompresses the inbound body (zstd/gzip/deflate/br), so the bytes forwarded upstream are plain JSON — but these two handlers left the original `content-encoding` header in place, so a client (or an edge proxy like a Cloudflare Worker) that sent a compressed body got its request rejected with upstream HTTP 400 because the provider tried to decompress already-decoded JSON. The `/v1/responses` handler already carried this fix (#1542); it is now applied to the messages and chat paths too.
+* **models:** fix the model registry's prefix fallback silently returning the wrong context window. `ModelRegistry.get` accepted any registered name as a `str.startswith` prefix and returned the *first* match, so `gpt-4-32k-0613` resolved to `gpt-4` (8192) instead of `gpt-4-32k` (32768), and unregistered ids like `gpt-4.1`/`gpt-4.5` inherited `gpt-4`'s 8192-token window — making the proxy think a nearly-empty context was almost full and compress far too aggressively. The fallback now requires the registered name to end at a version boundary in the query (so `gpt-4.1` no longer matches `gpt-4`) and picks the longest qualifying name (so `gpt-4-32k-0613` → `gpt-4-32k`).
 * **proxy:** include the system prompt, tools, and the response-shaping request fields in the SemanticCache key. `_compute_key` hashed only `{model, messages}`, so two non-streaming requests with identical messages but a different top-level `system` prompt, tool set, sampling config, or output-shaping field collided on one key and the second caller was served the first's cached response — generated under different request semantics, in the default config (`cache_enabled` defaults on). The key now folds the request fields that shape generation — `temperature`/`top_p`/`top_k`/`max_tokens`/`stop`, plus OpenAI `tool_choice`/`response_format`/`parallel_tool_calls`/`seed`/`presence_penalty`/`frequency_penalty`/`logit_bias`/`n`/`logprobs`/`top_logprobs`/`reasoning_effort`/`verbosity`/`modalities` and Anthropic `thinking`/`tool_choice`/`output_config` — canonicalizing `system`/`tools` so a moved `cache_control` breakpoint does not fragment it, and the handlers snapshot the fields once at the cache read and reuse them at write so a body mutated by the pipeline cannot diverge the key. Non-streaming path only.
 * **learn (verbosity):** `--verbosity --apply --all` now aggregates the savings baseline across every project instead of overwriting it per project (last-project-wins), which previously left the output shaper with a tiny, unrepresentative baseline. The applied verbosity level comes from the project with the most samples ([#1288](https://github.com/headroomlabs-ai/headroom/pull/1288)).
 * **proxy/anthropic:** restore token-mode compression on continued Claude Code turns with a frozen prefix and deferred CCR tool injection. Token mode now runs request-side compression even when the client did not pre-register `headroom_retrieve`, relying on the existing marker-triggered injection override to keep emitted CCR markers redeemable ([#1487](https://github.com/headroomlabs-ai/headroom/issues/1487)).

headroom/cache/compression_feedback.py

@@ -33,6 +33,8 @@
 from dataclasses import dataclass, field
 from typing import TYPE_CHECKING, Any
 
+from .compression_strategy_outcomes import CompressionStrategyOutcomes
+
 if TYPE_CHECKING:
     from .compression_store import CompressionStore, RetrievalEvent
 
@@ -94,28 +96,33 @@ def search_rate(self) -> float:
 
     def strategy_retrieval_rate(self, strategy: str) -> float:
         """Get retrieval rate for a specific compression strategy."""
-        compressions = self.strategy_compressions.get(strategy, 0)
-        if compressions == 0:
-            return 0.0
-        retrievals = self.strategy_retrievals.get(strategy, 0)
-        return retrievals / compressions
+        return self.strategy_outcomes.retrieval_rate(strategy)
 
     def best_strategy(self) -> str | None:
         """Find the strategy with lowest retrieval rate (most successful)."""
-        if not self.strategy_compressions:
-            return None
+        return self.strategy_outcomes.best_strategy()
 
-        best = None
-        best_rate = 1.0
+    @property
+    def strategy_outcomes(self) -> CompressionStrategyOutcomes:
+        """Strategy outcome view backed by this pattern's public counters."""
+        return CompressionStrategyOutcomes(
+            compressions=self.strategy_compressions,
+            retrievals=self.strategy_retrievals,
+        )
 
-        for strategy in self.strategy_compressions:
-            rate = self.strategy_retrieval_rate(strategy)
-            # Only consider strategies with enough samples
-            if self.strategy_compressions[strategy] >= 3 and rate < best_rate:
-                best_rate = rate
-                best = strategy
+    def record_strategy_compression(self, strategy: str) -> None:
+        """Record strategy compression outcome."""
+        outcomes = self.strategy_outcomes
+        outcomes.record_compression(strategy)
+        self.strategy_compressions = outcomes.compressions
+        self.strategy_retrievals = outcomes.retrievals
 
-        return best
+    def record_strategy_retrieval(self, strategy: str) -> None:
+        """Record strategy retrieval outcome."""
+        outcomes = self.strategy_outcomes
+        outcomes.record_retrieval(strategy)
+        self.strategy_compressions = outcomes.compressions
+        self.strategy_retrievals = outcomes.retrievals
 
 
 @dataclass
@@ -235,15 +242,7 @@ def record_compression(
 
             # Track strategy usage
             if strategy:
-                pattern.strategy_compressions[strategy] = (
-                    pattern.strategy_compressions.get(strategy, 0) + 1
-                )
-
-                # CRITICAL FIX: When truncating strategy dicts, keep them in sync
-                # to prevent desync between compressions and retrievals.
-                # Both dicts must have the same keys for accurate retrieval rate calculation.
-                if len(pattern.strategy_compressions) > 50:
-                    self._truncate_strategy_dicts(pattern)
+                pattern.record_strategy_compression(strategy)
 
             # Track signature hash for TOIN correlation
             if tool_signature_hash:
@@ -291,14 +290,7 @@ def record_retrieval(
 
             # Track strategy retrievals (for success rate calculation)
             if strategy:
-                pattern.strategy_retrievals[strategy] = (
-                    pattern.strategy_retrievals.get(strategy, 0) + 1
-                )
-
-                # CRITICAL FIX: When truncating strategy dicts, keep them in sync
-                # to prevent desync between compressions and retrievals.
-                if len(pattern.strategy_retrievals) > 50:
-                    self._truncate_strategy_dicts(pattern)
+                pattern.record_strategy_retrieval(strategy)
 
             # Track query patterns
             if event.query:
@@ -318,40 +310,11 @@ def record_retrieval(
                 self._extract_field_hints(pattern, event.query)
 
     def _truncate_strategy_dicts(self, pattern: LocalToolPattern) -> None:
-        """Truncate strategy_compressions and strategy_retrievals in sync.
-
-        CRITICAL FIX: Both dicts must have the same keys for accurate retrieval
-        rate calculation. When truncating, we keep the union of top strategies
-        from both dicts, then truncate both to the same key set.
-        """
-        # Get top 40 strategies from each dict (using 40 to allow union to stay under 50)
-        top_compressions = {
-            k
-            for k, _ in sorted(
-                pattern.strategy_compressions.items(),
-                key=lambda x: x[1],
-                reverse=True,
-            )[:40]
-        }
-        top_retrievals = {
-            k
-            for k, _ in sorted(
-                pattern.strategy_retrievals.items(),
-                key=lambda x: x[1],
-                reverse=True,
-            )[:40]
-        }
-
-        # Keep union of top strategies from both
-        keys_to_keep = top_compressions | top_retrievals
-
-        # Truncate both dicts to same keys
-        pattern.strategy_compressions = {
-            k: v for k, v in pattern.strategy_compressions.items() if k in keys_to_keep
-        }
-        pattern.strategy_retrievals = {
-            k: v for k, v in pattern.strategy_retrievals.items() if k in keys_to_keep
-        }
+        """Truncate strategy counters using the shared strategy outcome domain."""
+        outcomes = pattern.strategy_outcomes
+        outcomes.prune()
+        pattern.strategy_compressions = outcomes.compressions
+        pattern.strategy_retrievals = outcomes.retrievals
 
     def _extract_field_hints(self, pattern: LocalToolPattern, query: str) -> None:
         """Extract potential field names from search queries.

headroom/cache/compression_strategy_outcomes.py

@@ -0,0 +1,99 @@
+"""Strategy outcome accounting for local compression feedback."""
+
+from __future__ import annotations
+
+from dataclasses import dataclass, field
+
+
+@dataclass
+class CompressionStrategyOutcomes:
+    """Track compression and retrieval outcomes by compression strategy."""
+
+    compressions: dict[str, int] = field(default_factory=dict)
+    retrievals: dict[str, int] = field(default_factory=dict)
+    max_strategies: int = 50
+    top_strategies_per_counter: int = 40
+    minimum_samples_for_recommendation: int = 3
+
+    def record_compression(self, strategy: str) -> None:
+        """Record one compression for a strategy."""
+        self.compressions[strategy] = self.compressions.get(strategy, 0) + 1
+        self.prune()
+
+    def record_retrieval(self, strategy: str) -> None:
+        """Record one retrieval for a strategy."""
+        self.retrievals[strategy] = self.retrievals.get(strategy, 0) + 1
+        self.prune()
+
+    def retrieval_rate(self, strategy: str) -> float:
+        """Return the retrievals-per-compression rate for one strategy."""
+        compressions = self.compressions.get(strategy, 0)
+        if compressions == 0:
+            return 0.0
+        return self.retrievals.get(strategy, 0) / compressions
+
+    def best_strategy(self) -> str | None:
+        """Return the sampled strategy with the lowest retrieval rate."""
+        best = None
+        best_rate = 1.0
+
+        for strategy, compression_count in self.compressions.items():
+            if compression_count < self.minimum_samples_for_recommendation:
+                continue
+
+            rate = self.retrieval_rate(strategy)
+            if rate < best_rate:
+                best = strategy
+                best_rate = rate
+
+        return best
+
+    def prune(self) -> None:
+        """Bound counters while preserving the highest-signal strategies."""
+        if (
+            len(self.compressions) <= self.max_strategies
+            and len(self.retrievals) <= self.max_strategies
+        ):
+            return
+
+        keys_to_keep = self._keys_to_keep()
+        self.compressions = {
+            strategy: count
+            for strategy, count in self.compressions.items()
+            if strategy in keys_to_keep
+        }
+        self.retrievals = {
+            strategy: count
+            for strategy, count in self.retrievals.items()
+            if strategy in keys_to_keep
+        }
+
+    def _keys_to_keep(self) -> set[str]:
+        top_compressions = self._top_keys(self.compressions)
+        top_retrievals = self._top_keys(self.retrievals)
+        candidate_keys = top_compressions | top_retrievals
+
+        if len(candidate_keys) <= self.max_strategies:
+            return candidate_keys
+
+        ranked_keys = sorted(
+            candidate_keys,
+            key=lambda strategy: (
+                self.compressions.get(strategy, 0) + self.retrievals.get(strategy, 0),
+                self.compressions.get(strategy, 0),
+                self.retrievals.get(strategy, 0),
+                strategy,
+            ),
+            reverse=True,
+        )
+        return set(ranked_keys[: self.max_strategies])
+
+    def _top_keys(self, counts: dict[str, int]) -> set[str]:
+        return {
+            strategy
+            for strategy, _ in sorted(
+                counts.items(),
+                key=lambda item: (item[1], item[0]),
+                reverse=True,
+            )[: self.top_strategies_per_counter]
+        }