Skip to content

Commit 786b4b9

Browse files
authored
api: update quota policy docs (#2261)
**Description** The failover logic depends on an Envoyproxy commit that has not been officially released. Marking that feature as a todo and will enable it once the changes are stable. Signed-off-by: achoo30 <achoo30@bloomberg.net>
1 parent 1d2f496 commit 786b4b9

3 files changed

Lines changed: 10 additions & 7 deletions

File tree

api/v1alpha1/quota_policy.go

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,10 @@ import (
1212
)
1313

1414
// QuotaPolicy specifies token quota configuration for inference services.
15-
// Providing a list of backends in the AIGatewayRouteRule allows failover to a different service
16-
// if token quota for a service had been exceeded.
15+
// Generates rate limit configuration and tracks quota usage.
16+
// Reject requests with 429 once all related quota to that request has been exceeded.
17+
//
18+
// TODO: Waiting on next release of Envoyproxy that will support routing based on non-exceeded quotas.
1719
//
1820
// +genclient
1921
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

manifests/charts/ai-gateway-crds-helm/templates/aigateway.envoyproxy.io_quotapolicies.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -28,10 +28,10 @@ spec:
2828
name: v1alpha1
2929
schema:
3030
openAPIV3Schema:
31-
description: |-
31+
description: |
3232
QuotaPolicy specifies token quota configuration for inference services.
33-
Providing a list of backends in the AIGatewayRouteRule allows failover to a different service
34-
if token quota for a service had been exceeded.
33+
Generates rate limit configuration and tracks quota usage.
34+
Reject requests with 429 once all related quota to that request has been exceeded.
3535
properties:
3636
apiVersion:
3737
description: |-

site/docs/api/api.mdx

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -481,8 +481,9 @@ MCPRouteList contains a list of MCPRoute.
481481
- [QuotaPolicyList](#github.qkg1.top-envoyproxy-ai-gateway-api-v1alpha1-quotapolicylist)
482482

483483
QuotaPolicy specifies token quota configuration for inference services.
484-
Providing a list of backends in the AIGatewayRouteRule allows failover to a different service
485-
if token quota for a service had been exceeded.
484+
Generates rate limit configuration and tracks quota usage.
485+
Reject requests with 429 once all related quota to that request has been exceeded.
486+
486487

487488
##### Fields
488489

0 commit comments

Comments
 (0)