Skip to content

feat(cost,budgets): impute token cost for subscription runs and enforce it - #11104

Open
trelmitt wants to merge 2 commits into
paperclipai:masterfrom
trelmitt:feat/imputed-token-cost-and-budgets
Open

feat(cost,budgets): impute token cost for subscription runs and enforce it#11104
trelmitt wants to merge 2 commits into
paperclipai:masterfrom
trelmitt:feat/imputed-token-cost-and-budgets

Conversation

@trelmitt

@trelmitt trelmitt commented Aug 8, 2026

Copy link
Copy Markdown

Thinking Path

  • Paperclip is the open source control plane that people use to run companies of AI agents.
  • Cost and budgets are the control plane's safety layer. Runs write cost_events, and budget_policies enforce spend limits.
  • Many agents run on subscription plans, for example Claude Max. These runs report cost_cents = 0.
  • So cost views and budget metrics read zero even when the runs use millions of tokens. Hard-stops never fire for this usage.
  • This means the "safe autonomy, no hidden token burn" promise is not enforced for subscription runs.
  • This pull request imputes a token cost from a calibratable per-model price map, shows it in the cost views, and adds a budget metric that enforces on it.
  • The benefit is that subscription usage becomes visible and can trip the same soft and hard budget gates as billed spend.

Linked Issues or Issue Description

No public GitHub issue exists. The change is described below with the feature template fields.

Related pull requests
I searched the PR list and found related work. Refs #3330 (open) and #4430 (closed) — both impute an API-equivalent cost for subscription-included runs; the imputation approach overlaps. Refs #10871 (open) — reports token usage where subscription billing zeroes spend; complementary. This PR's distinct piece is the effective_cents budget metric that enforces on the imputed cost, so soft and hard budget gates pause a scope on subscription usage — which the above PRs do not do. Maintainers may prefer to consolidate the imputation half with #3330.

Subsystem affected
Cost accounting and budgets (server/src/services/costs.ts, server/src/services/budgets.ts, @paperclipai/shared).

Problem or motivation
Subscription runs report cost_cents = 0. Every cost view and budget metric reads zero even while the runs consume many tokens. A billed-cents hard-stop can never pause a subscription workload, so spend is ungoverned.

Proposed solution
Add a calibratable per-model price map with imputeCostCents and effectiveCostCents helpers. Surface effectiveCostCents in the cost views. Add an effective_cents budget metric and enforce it in computeObservedAmount, evaluateCostEvent, and getInvocationBlock.

Alternatives considered
Store an imputed cost column at write time. This was rejected because it needs a migration and freezes prices in old rows. Imputing at query time keeps prices editable and avoids schema churn.

Additional context
billed_cents behavior is unchanged and stays the default metric. The price table holds documented default rates that an operator can tune as provider pricing changes.

What Changed

  • Add packages/shared/src/model-pricing.ts: a per-model price map plus imputeCostCents() and effectiveCostCents() (one source of truth for imputation). Exported from the package index.
  • Add server/src/services/cost-imputation-sql.ts: a SQL sum expression generated from the same price map, so the views and budgets share one definition.
  • Update server/src/services/costs.ts: report effectiveCostCents in summary, byAgent, byAgentModel, and byProject. No schema change.
  • Add an effective_cents value to BUDGET_METRICS. Update computeObservedAmount, evaluateCostEvent, and all three getInvocationBlock scopes to enforce it. The three per-scope lookups now route through one firstExceededHardStopPolicy helper.
  • Add tests: price math, a real-SQL subscription imputation test, and an effective_cents hard-stop that pauses an agent and blocks new work.

Verification

  • pnpm --filter @paperclipai/shared build — passes.
  • pnpm --filter @paperclipai/server typecheck — passes.
  • npx vitest run packages/shared/src/model-pricing.test.ts server/src/__tests__/costs-service.test.ts server/src/__tests__/budgets-service.test.ts — 32 tests pass.
  • End-to-end (embedded Postgres): a subscription run of 1M input + 1M cached + 1M output opus tokens at cost_cents = 0 returns effectiveCostCents = 9150 from byAgent and summary, and trips a 500-cent effective_cents agent hard-stop (agent paused, hard incident raised, new work blocked).

Risks

  • Low risk. billed_cents is unchanged and remains the default, so existing policies behave exactly as before.
  • effective_cents is opt-in per policy. No migration runs.
  • The price table holds default rates. If a rate is stale, imputed cost drifts from the true rate. The table is a single, commented file that an operator edits.

Model Used

Claude Opus 4.8 (claude-opus-4-8), Anthropic. Extended thinking, tool use, and code execution, run through an agentic CLI harness. The model read the code, wrote the change, and ran the tests and typecheck locally.

Checklist

  • I have included a thinking path that traces from project context to this change
  • I have specified the model used (with version and capability details)
  • I have checked ROADMAP.md and confirmed this PR does not duplicate planned core work
  • I have searched GitHub for duplicate or related PRs and linked them above
  • I have either (a) linked existing issues with Fixes: # / Closes # / Refs # OR (b) described the issue in-PR following the relevant issue template
  • I have not referenced internal/instance-local Paperclip issues or links (only public GitHub #NNN / github.com/paperclipai/paperclip URLs)
  • My branch name describes the change (e.g. docs/..., fix/...) and contains no internal Paperclip ticket id or instance-derived details
  • I have run tests locally and they pass
  • I have added or updated tests where applicable
  • I have updated relevant documentation to reflect my changes
  • I have considered and documented any risks above
  • All Paperclip CI gates are green
  • Greptile is 5/5 with no open P2s, recommendations, or follow-ups
  • I will address all Greptile and reviewer comments before requesting merge

🤖 Generated with Claude Code

…ce it

Subscription runs (e.g. Claude Max) report cost_cents=0, so every cost view and
budget metric read zero even while millions of tokens were consumed. This makes
that usage visible and governable.

- add a calibratable per-model price map + imputeCostCents/effectiveCostCents
  helpers in @paperclipai/shared (one source of truth for imputation)
- surface effectiveCostCents in the cost views (summary, byAgent, byAgentModel,
  byProject) via a SQL expression generated from that same map (no migration)
- add an effective_cents budget metric; computeObservedAmount, evaluateCostEvent,
  and all three getInvocationBlock scopes enforce it, so a $0 subscription run
  trips the same soft/hard budget gates as billed spend
- tests: model-pricing math, real-SQL subscription imputation, and an
  effective_cents hard-stop pausing an agent + blocking new work

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@commitperclip

commitperclip Bot commented Aug 8, 2026

Copy link
Copy Markdown

✅ All checks passing — ready for Greptile review and maintainer approval.

— commitperclip

@trelmitt

trelmitt commented Aug 8, 2026

Copy link
Copy Markdown
Author

Done — searched the PR list and found the related work: linked #3330 and #4430 (subscription-cost imputation) and #10871 (subscription token-usage dashboard) under Related pull requests in the description, and checked the dedup-search box. This PR's distinct piece is the effective_cents budget metric that enforces on the imputed cost; happy to consolidate the imputation half with #3330 if you prefer.

🤖 Addressed by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant