Skip to content

refactor(proxy): isolate semantic cache key policy - #1964

Merged
JerrettDavis merged 3 commits into
mainfrom
jd/architecture-slice-21
Jul 12, 2026
Merged

refactor(proxy): isolate semantic cache key policy#1964
JerrettDavis merged 3 commits into
mainfrom
jd/architecture-slice-21

Conversation

@JerrettDavis

Copy link
Copy Markdown
Collaborator

Description

Extracts proxy semantic response-cache key normalization and hashing into a pure semantic_cache_key_policy module. SemanticCache keeps ownership of storage, locking, TTL, and LRU behavior while the deterministic cache-key formula is directly tested as a standalone policy.

Closes #

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Performance improvement
  • Code refactoring (no functional changes)

Changes Made

  • Added headroom.proxy.semantic_cache_key_policy with recursive cache_control stripping and semantic cache key hashing.
  • Updated SemanticCache._compute_key to delegate to the pure key policy while preserving its existing private wrapper contract.
  • Added direct policy tests for recursive annotation stripping, key stability, response-shaping distinctions, breakpoint movement, and wrapper parity.
  • Included the current LiteLLM callback signature compatibility shim required for repo-wide mypy on main-based slices.

Testing

  • Unit tests pass (pytest)
  • Linting passes (ruff check .)
  • Type checking passes (mypy headroom)
  • New tests added for new functionality
  • Manual testing performed

Test Output

python -m pytest tests/test_semantic_cache_key_policy.py tests/test_proxy_semantic_cache_key.py tests/test_litellm_callback.py -q
39 passed in 6.34s

python -m ruff check .
All checks passed!

python -m ruff format --check .
1095 files already formatted

python -m mypy headroom --ignore-missing-imports
Success: no issues found in 409 source files

gitleaks protect --staged --no-banner --redact
no leaks found

Real Behavior Proof

  • Environment: Windows, Python 3.13.13, clean worktree based on headroomlabs/main.
  • Exact command / steps: targeted pytest, ruff, format check, repo-wide mypy, staged gitleaks scan.
  • Observed result: semantic cache key policy/cache/callback tests pass; static checks pass; no staged secrets detected.
  • Not tested: live proxy cache traffic; this slice preserves the existing cache wrapper and only moves pure key policy.

Review Readiness

  • I have performed a self-review
  • This PR is ready for human review

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes
  • I have updated the CHANGELOG.md if applicable

Screenshots (if applicable)

N/A

Additional Notes

Documentation and changelog updates are not applicable for this internal architecture slice. PR-specific GHAS checks will be monitored after opening.

@github-actions

Copy link
Copy Markdown
Contributor

PR governance

This PR follows the template and is marked ready for human review.

@github-actions github-actions Bot added the status: ready for review Pull request body is complete and the author marked it ready for human review label Jul 10, 2026
@JerrettDavis
JerrettDavis force-pushed the jd/architecture-slice-21 branch from 25659dd to 96f3989 Compare July 11, 2026 00:11
@JerrettDavis
JerrettDavis marked this pull request as draft July 11, 2026 05:00
@JerrettDavis

Copy link
Copy Markdown
Collaborator Author

Marking this back to draft during PR maintenance: the branch is currently conflicted against main, and gh pr update-branch cannot update it automatically. It should come back to ready-for-review only after manual conflict resolution/rebase and fresh CI on the new head.

@JerrettDavis
JerrettDavis marked this pull request as ready for review July 11, 2026 05:00
@JerrettDavis
JerrettDavis merged commit 2f53a18 into main Jul 12, 2026
28 checks passed
@JerrettDavis
JerrettDavis deleted the jd/architecture-slice-21 branch July 12, 2026 02:35
@chopratejas chopratejas mentioned this pull request Jul 12, 2026
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: ready for review Pull request body is complete and the author marked it ready for human review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant