Skip to content

test: track active LiteLLM DeepSeek pricing - #3161

Merged
JerrettDavis merged 1 commit into
headroomlabs-ai:mainfrom
JerrettDavis:fix/litellm-upstream-deepseek-pricing
Aug 21, 2026
Merged

test: track active LiteLLM DeepSeek pricing#3161
JerrettDavis merged 1 commit into
headroomlabs-ai:mainfrom
JerrettDavis:fix/litellm-upstream-deepseek-pricing

Conversation

@JerrettDavis

Copy link
Copy Markdown
Collaborator

Description

Keep the LiteLLM DeepSeek V4 integration tests compatible with upstream-owned pricing entries. LiteLLM now publishes these models directly, so Headroom correctly preserves upstream values instead of installing its fallback values; the tests must validate the active entry rather than require fallback prices.

Related: #3157

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Documentation update
  • Performance improvement
  • Code refactoring (no functional changes)

Changes Made

  • Validate that active upstream DeepSeek V4 price entries contain positive input and output prices.
  • Compare cost_per_token results with the active LiteLLM model-cost entry.
  • Preserve the existing fallback-price and non-overwrite coverage.

Testing

  • Unit tests pass (pytest)
  • Linting passes (ruff check .)
  • Type checking passes (mypy headroom)
  • New tests added for new functionality
  • Manual testing performed

Test Output

python -m pytest tests/test_providers/test_deepseek.py -q
20 passed in 4.63s

ruff check tests/test_providers/test_deepseek.py
All checks passed!

ruff format --check tests/test_providers/test_deepseek.py
1 file already formatted

pre-commit: Ruff alignment, merge-conflict check, Ruff, Ruff format, and mypy all passed

Real Behavior Proof

  • Environment: Windows, Python 3.13.13, LiteLLM model-cost data available.
  • Exact command / steps: python -m pytest tests/test_providers/test_deepseek.py -q
  • Observed result: all 20 DeepSeek provider and pricing tests pass against the active LiteLLM entries.
  • Not tested: provider API calls; this change only concerns local pricing metadata assertions.

Runtime Rollout Safety

  • Rollout-managed feature(s): None.
  • Minimum rollout channel: N/A.
  • Stable/default behavior changed: No runtime behavior changes.
  • Kill switch / disable path: N/A.
  • Unsafe override required: No.
  • Qualification impact: Restores deterministic CI coverage for upstream-owned pricing entries.
  • Rollback path: Revert this test-only commit.

Review Readiness

  • I have performed a self-review
  • This PR is ready for review

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have commented my code where needed
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • Existing tests prove the fix is effective
  • New and existing unit tests pass locally with my changes
  • I did not edit CHANGELOG.md — it is generated by release-please from my Conventional Commit PR title (a CI guard enforces this)

Screenshots (if applicable)

N/A — test-only change.

Additional Notes

Documentation changes are not applicable because runtime behavior and public APIs are unchanged.

@github-actions

Copy link
Copy Markdown
Contributor

PR governance

This PR does not yet satisfy the required template fields:

  • Check This PR is ready for human review or convert the PR back to draft.

Please update the PR body, or move the PR back to draft while it is still in progress.

@github-actions github-actions Bot added the status: needs author action Pull request body or readiness checklist still needs author updates label Aug 21, 2026
@codecov-commenter

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

@JerrettDavis
JerrettDavis merged commit 1bea0ea into headroomlabs-ai:main Aug 21, 2026
33 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

status: needs author action Pull request body or readiness checklist still needs author updates

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants