feat(admin): add experiment-level cost exclusions - #1132
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Admin-managed list of experiments whose trials' spend is ignored by cost accounting, the experiment-level sibling of the cost-excluded LLM keys: new cost_excluded_experiments table + migration, a not_excluded_experiment_filter folded into first_party_spend_filter and the quota inflight predicates, an /admin/cost-excluded-experiments router, and an admin Costs-tab card grouped with the keys card. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
a707442 to
86980bd
Compare
Oddish previewCommit:
Vercel deployment URL: https://oddish-odrwiupm5.oddish.app Plan:
This comment is updated by the PR Preview workflow. |
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 85a81e8. Configure here.
| TrialModel.superseded_by_trial_id.is_(None), | ||
| TrialModel.status.in_(_INFLIGHT_TRIAL_STATUSES), | ||
| not_excluded_llm_key_filter(), | ||
| not_excluded_experiment_filter(), |
There was a problem hiding this comment.
Quota cancel skips experiment exclusion
High Severity
_active_trial_predicates still only applies not_excluded_llm_key_filter, so trials in cost-excluded experiments are cancelled when a quota fires even though their spend is omitted from settled sums and inflight reservation. That breaks parity with excluded LLM keys, which this path already preserves.
Reviewed by Cursor Bugbot for commit 85a81e8. Configure here.




Extends cost-exclusion handling to experiments, allowing admins to register experiments (by name or id) whose trials are excluded from cost accounting, matching the existing LLM API key cost-exclusion feature. Implementation adds a new
cost_excluded_experimentstable and folds anot_excluded_experiment_filterinto the shared cost-basis and quota-reservation predicates. The admin dashboard gains a Cost-excluded experiments card grouped with the existing Cost-excluded LLM keys card, backed by a mirrored/admin/cost-excluded-experimentsAPI.🤖 Generated with Claude Code
Note
Medium Risk
Changes shared quota and cost-basis predicates used for billing enforcement; mistakes could mis-attribute spend, though behavior is covered by router and DB integration tests and mirrors the existing key-exclusion pattern.
Overview
Adds experiment-level cost exclusions alongside the existing LLM-key list: operators can register experiments (by id or unambiguous name) whose home trials are omitted from shared cost accounting.
Data & accounting: New
cost_excluded_experimentstable (soft-delete, one live row per experiment) andnot_excluded_experiment_filter()wired intofirst_party_spend_filter()and quota inflight predicates so excluded spend does not appear on admin cost dashboards or count toward caps/reservations; removing an entry restores spend. Experiment pages still show raw trial cost.API & UI: Operator-only
GET/POST/DELETE /admin/cost-excluded-experiments, Next.js proxies, and a Cost-excluded experiments card on the admin Costs tab next to cost-excluded keys. Docs note this as a deployment-wide operator control.Reviewed by Cursor Bugbot for commit 85a81e8. Bugbot is set up for automated code reviews on this repo. Configure here.