Skip to content

Commit de03060

Browse files
missBergclaude
andcommitted
docs: correct QuotaPolicy cost-expression examples and clientSelector scope
The cost-expression examples used uint*int and uint*double arithmetic (e.g. "output_tokens * 6", "cached_input_tokens * 0.1"), which CEL rejects since the token variables are unsigned integers — the controller would skip the expression and silently stop enforcing that model's quota. Use unsigned literals and integer division (6u, / 10u), and document that fractional weights require an explicit uint(double(...)) cast. Also correct the clientSelectors note: QuotaPolicy only honors the headers matcher; sourceCIDR/methods/path/queryParams are accepted by the embedded EG type but not applied to quota buckets. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Erica Hughberg <erica.sundberg.90@gmail.com>
1 parent 79e51b5 commit de03060

1 file changed

Lines changed: 11 additions & 7 deletions

File tree

site/docs/capabilities/traffic/quota-policy.md

Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -134,18 +134,22 @@ following variables are available in a `costExpression`:
134134
perModelQuotas:
135135
- modelName: gpt-4
136136
quota:
137-
# Cached input tokens cost 10% of regular input tokens;
138-
# output tokens cost 6x regular input tokens.
139-
costExpression: "input_tokens + cached_input_tokens * 0.1 + output_tokens * 6"
137+
# Cached input tokens count as 1/10 of a regular input token;
138+
# output tokens count 6x.
139+
costExpression: "input_tokens + cached_input_tokens / 10u + output_tokens * 6u"
140140
defaultBucket:
141141
limit: 50000
142142
duration: "1h"
143143
```
144144

145145
:::tip
146146
Use a custom cost expression when token types have significantly different costs with your provider —
147-
for example, output tokens are typically more expensive than input tokens. The expression must
148-
evaluate to a non-negative integer.
147+
for example, output tokens are typically more expensive than input tokens.
148+
149+
The token variables are **unsigned integers**, so numeric literals must carry a `u` suffix (for
150+
example `output_tokens * 6u`) and the expression must evaluate to a non-negative integer. Integer
151+
division truncates (`cached_input_tokens / 10u`); for an exact fractional weight, cast through
152+
floating point — for example `uint(double(cached_input_tokens) * 0.1)`.
149153
:::
150154

151155
### Bucket Mode
@@ -215,7 +219,7 @@ bucketRules:
215219

216220
Supported header match types are `Exact`, `Distinct`, and `RegularExpression`.
217221

218-
`clientSelectors` are Envoy Gateway [`RateLimitSelectCondition`](https://gateway.envoyproxy.io/docs/api/extension_types/#ratelimitselectcondition) values. Besides `headers`, a selector can also match on `sourceCIDR` (client IP range), `methods` (HTTP method), `path`, and `queryParams`. Refer to the Envoy Gateway rate limit API reference for the full set of selector options.
222+
`clientSelectors` reuse the Envoy Gateway [`RateLimitSelectCondition`](https://gateway.envoyproxy.io/docs/api/extension_types/#ratelimitselectcondition) type, but QuotaPolicy currently applies only the `headers` matcher. The other fields on that type (`sourceCIDR`, `methods`, `path`, `queryParams`) are accepted by the schema but **not yet honored** for quota buckets.
219223

220224
### Shadow Mode
221225

@@ -284,7 +288,7 @@ spec:
284288
perModelQuotas:
285289
- modelName: gpt-4
286290
quota:
287-
costExpression: "input_tokens + cached_input_tokens * 0.1 + output_tokens * 6"
291+
costExpression: "input_tokens + cached_input_tokens / 10u + output_tokens * 6u"
288292
mode: Shared
289293
defaultBucket:
290294
limit: 10000

0 commit comments

Comments
 (0)