feat(gateway): spend/age trust tiers + USD spend caps

Regular (kind=default, non-enterprise) orgs are now governed by a single
trust tier ladder (T0-T4) qualified by account age OR lifetime usage
spend. The tier drives both the per-endpoint RPM multiplier (replacing
the old $1k/$10k/$50k usage-spend multiplier) and new daily/monthly USD
spend caps.

Caps are Redis UTC-bucket counters checked pre-request (429 at/over cap)
and incremented post-request in the insertLog chokepoint. Tier uses the
already-cached lifetime usage spend + org.createdAt, so no schema change.
Dev/chat plans keep flat limits; enterprise and free models are exempt.
Fail-open on Redis errors; caps default-off under test runners.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: steebchen

.env.example

@@ -111,14 +111,27 @@ TIMEOUT_MS=5000
 # GATEWAY_RATE_LIMIT_CHATPLAN_AUDIO_SPEECH_RPM=30
 # GATEWAY_RATE_LIMIT_CHATPLAN_VIDEOS_RPM=12
 
-# Spend-tier thresholds (USD lifetime spend) and the multiplier each grants on
-# top of the per-path base limit for regular orgs (defaults shown).
-# GATEWAY_RATE_LIMIT_TIER_1_THRESHOLD=1000
-# GATEWAY_RATE_LIMIT_TIER_1_MULTIPLIER=2
-# GATEWAY_RATE_LIMIT_TIER_2_THRESHOLD=10000
-# GATEWAY_RATE_LIMIT_TIER_2_MULTIPLIER=4
-# GATEWAY_RATE_LIMIT_TIER_3_THRESHOLD=50000
-# GATEWAY_RATE_LIMIT_TIER_3_MULTIPLIER=10
+# Unified trust tiers for regular orgs. An org gets the highest tier whose
+# age OR lifetime-usage-spend threshold is met; the tier sets the per-path RPM
+# multiplier AND the daily/monthly USD spend caps below. Defaults shown; every
+# value is overridable as GATEWAY_SPEND_TIER_<N>_<FIELD> for N in 0..4 and FIELD
+# in AGE_DAYS, SPEND_USD, RPM_MULTIPLIER, DAILY_CAP_USD, MONTHLY_CAP_USD.
+# GATEWAY_SPEND_TIER_1_AGE_DAYS=7
+# GATEWAY_SPEND_TIER_1_SPEND_USD=10
+# GATEWAY_SPEND_TIER_1_RPM_MULTIPLIER=2
+# GATEWAY_SPEND_TIER_1_DAILY_CAP_USD=100
+# GATEWAY_SPEND_TIER_1_MONTHLY_CAP_USD=1000
+# GATEWAY_SPEND_TIER_2_AGE_DAYS=30
+# GATEWAY_SPEND_TIER_2_SPEND_USD=100
+# GATEWAY_SPEND_TIER_3_AGE_DAYS=60
+# GATEWAY_SPEND_TIER_3_SPEND_USD=1000
+# GATEWAY_SPEND_TIER_4_AGE_DAYS=90
+# GATEWAY_SPEND_TIER_4_SPEND_USD=5000
+
+# Global switch for the daily/monthly USD spend caps (default enabled, but off
+# under test runners). Set to "false" to disable the caps only; the RPM tier
+# multiplier above is unaffected.
+# GATEWAY_SPEND_CAPS_ENABLED=true
 
 # How long (seconds) to cache an organization's lifetime spend used for tier
 # selection. Defaults to 900 (15 minutes).

apps/docs/content/resources/rate-limits.mdx

@@ -38,17 +38,23 @@ The default limits (requests per minute, per organization) are:
 	limits. [Contact us](mailto:contact@llmgateway.io) about enterprise plans.
 </Callout>
 
-### Higher Limits as You Scale
+### Trust Tiers (account age or spend)
 
-For regular (pay-as-you-go) organizations, the endpoint limits above automatically increase based on your **lifetime spend**:
+For regular (pay-as-you-go) organizations, limits scale with a **trust tier**. An organization qualifies for a tier when **either** its account is old enough **or** its lifetime usage spend is high enough — whichever gets it to the higher tier. The tier raises the per-endpoint RPM limits **and** the daily/monthly USD spend caps below.
 
-| Lifetime spend | Limit multiplier |
-| -------------- | ---------------- |
-| $1,000+        | 2×               |
-| $10,000+       | 4×               |
-| $50,000+       | 10×              |
+| Tier | Qualifies (age **or** spend) | RPM multiplier | Daily cap | Monthly cap |
+| ---- | ---------------------------- | -------------- | --------- | ----------- |
+| 0    | new / $0                     | 1×             | $5        | $50         |
+| 1    | 7 days **or** $10            | 2×             | $100      | $1,000      |
+| 2    | 30 days **or** $100          | 4×             | $500      | $5,000      |
+| 3    | 60 days **or** $1,000        | 10×            | $5,000    | $50,000     |
+| 4    | 90 days **or** $5,000        | 20×            | $15,000   | $200,000    |
 
-For example, an organization that has spent more than $10,000 gets a 4× multiplier, raising the chat completions limit from 600 to 2,400 requests per minute. The multiplier applies to every endpoint.
+For example, an org past 30 days old (or with $100+ of usage) is Tier 2: chat completions rises from 600 to 2,400 RPM, with a $500/day and $5,000/month spend ceiling.
+
+### Daily & Monthly Spend Caps
+
+Regular organizations also have hard **USD spend ceilings** — a daily and a monthly cap set by the trust tier above — so a brand-new account has a tight dollar velocity limit that rises as it ages or spends. Only real paid usage counts; **free models are exempt**, as are **enterprise** (no caps) and dev/chat plan orgs (which have their own plan limits). When a cap is reached, requests return `429` until the counter resets (UTC midnight for daily, first of the month for monthly).
 
 ### Dev and Chat Plans
 

apps/gateway/src/api.spec.ts

@@ -1,5 +1,6 @@
 import { afterAll, beforeAll, describe, expect, test, vi } from "vitest";
 
+import { redisClient } from "@llmgateway/cache";
 import { db, eq, tables } from "@llmgateway/db";
 import { logger } from "@llmgateway/logger";
 
@@ -1434,6 +1435,55 @@ describe("api", () => {
 		);
 	});
 
+	test("/v1/chat/completions returns 429 when the org is over its daily spend cap", async () => {
+		await harness.setProjectMode("credits");
+		await harness.setOrganizationCredits("100");
+		await db
+			.update(tables.organization)
+			.set({ retentionLevel: "none" })
+			.where(eq(tables.organization.id, "org-id"));
+
+		await db.insert(tables.apiKey).values({
+			id: "token-id-spend-cap",
+			token: "real-token-spend-cap",
+			projectId: "project-id",
+			description: "Test API Key",
+			createdBy: "user-id",
+		});
+
+		const now = new Date();
+		const dayKey = `${now.getUTCFullYear()}-${String(
+			now.getUTCMonth() + 1,
+		).padStart(2, "0")}-${String(now.getUTCDate()).padStart(2, "0")}`;
+		const counterKey = `spend_cap:daily:org-id:${dayKey}`;
+
+		process.env.GATEWAY_SPEND_CAPS_ENABLED = "true";
+		// Well above any tier's daily cap so this holds regardless of the seeded
+		// org's age/spend tier.
+		await redisClient.set(counterKey, "1000000");
+		try {
+			const res = await app.request("/v1/chat/completions", {
+				method: "POST",
+				headers: {
+					"Content-Type": "application/json",
+					Authorization: "Bearer real-token-spend-cap",
+				},
+				body: JSON.stringify({
+					model: "gpt-4o-mini",
+					messages: [{ role: "user", content: "Hello!" }],
+				}),
+			});
+
+			expect(res.status).toBe(429);
+			const json = await res.json();
+			expect(json.error.type).toBe("rate_limit_error");
+			expect(json.error.message).toContain("spend limit");
+		} finally {
+			delete process.env.GATEWAY_SPEND_CAPS_ENABLED;
+			await redisClient.del(counterKey);
+		}
+	});
+
 	test("/v1/embeddings hybrid fallback requires credits", async () => {
 		await harness.setProjectMode("hybrid");
 		await harness.setOrganizationCredits("0");

apps/gateway/src/chat/chat.ts

@@ -81,6 +81,7 @@ import {
 import { getResponsesContext } from "@/lib/responses-context.js";
 import { getResolvedRoutingConfig } from "@/lib/routing-config-loader.js";
 import { getNoFallbackRoutingMetadata } from "@/lib/routing-metadata.js";
+import { checkSpendLimit } from "@/lib/spend-limit.js";
 import {
 	createCombinedSignal,
 	createStreamingCombinedSignal,
@@ -4664,6 +4665,18 @@ chat.openapi(completions, async (c) => {
 			});
 		}
 
+		// Per-org daily/monthly USD spend caps. Applies to regular pay-as-you-go
+		// orgs only (kind/enterprise/enabled gates live inside checkSpendLimit);
+		// free models are exempt.
+		if (!((finalModelInfo ?? modelInfo) as ModelDefinition).free) {
+			const spendLimit = await checkSpendLimit(organization);
+			if (!spendLimit.allowed) {
+				throw new HTTPException(429, {
+					message: `Organization ${organization.id} has reached its ${spendLimit.period} spend limit of $${spendLimit.limit}. Try again later or contact support to raise your limit.`,
+				});
+			}
+		}
+
 		if (usedProvider === "llmgateway") {
 			throw new HTTPException(400, {
 				message:

apps/gateway/src/embeddings/embeddings.ts

@@ -36,6 +36,7 @@ import { extractApiToken } from "@/lib/extract-api-token.js";
 import { createFailedKeyTracker } from "@/lib/failed-key-tracker.js";
 import { throwIamException, validateRequestModelAccess } from "@/lib/iam.js";
 import { calculateDataStorageCost, insertLog } from "@/lib/logs.js";
+import { checkSpendLimit } from "@/lib/spend-limit.js";
 import { createCombinedSignal, isTimeoutError } from "@/lib/timeout-config.js";
 
 import { getProviderHeaders } from "@llmgateway/actions";
@@ -264,12 +265,24 @@ function getAvailableCredits(
 	};
 }
 
-function assertCreditsAvailableForEmbedding(
+async function assertCreditsAvailableForEmbedding(
 	organization: InferSelectModel<typeof tables.organization>,
 	modelDef: ModelDefinition,
 	insufficientCreditsMessage: string,
 	devPlanCreditLimitMessage: (renewalDate: string) => string,
 ) {
+	// Per-org daily/monthly USD spend caps, checked even when the org has
+	// credits (a funded org can still hit its cap). Free models are exempt; the
+	// kind/enterprise/enabled gates live inside checkSpendLimit.
+	if (!modelDef.free) {
+		const spendLimit = await checkSpendLimit(organization);
+		if (!spendLimit.allowed) {
+			throw new HTTPException(429, {
+				message: `Organization ${organization.id} has reached its ${spendLimit.period} spend limit of $${spendLimit.limit}. Try again later or contact support to raise your limit.`,
+			});
+		}
+	}
+
 	const {
 		devPlanCreditsRemaining,
 		chatPlanCreditsRemaining,
@@ -712,7 +725,7 @@ embeddings.openapi(createEmbeddings, async (c): Promise<any> => {
 			}
 			usedToken = providerKey.token;
 		} else if (retryProject.mode === "credits") {
-			assertCreditsAvailableForEmbedding(
+			await assertCreditsAvailableForEmbedding(
 				retryOrganization,
 				modelDef,
 				`Organization ${retryOrganization.id} has insufficient credits`,
@@ -737,7 +750,7 @@ embeddings.openapi(createEmbeddings, async (c): Promise<any> => {
 			if (providerKey) {
 				usedToken = providerKey.token;
 			} else {
-				assertCreditsAvailableForEmbedding(
+				await assertCreditsAvailableForEmbedding(
 					retryOrganization,
 					modelDef,
 					"No API key set for provider and organization has insufficient credits",

apps/gateway/src/lib/logs.ts

@@ -8,6 +8,8 @@ import {
 import { recordChatCompletionMetrics } from "@llmgateway/instrumentation";
 import { logger } from "@llmgateway/logger";
 
+import { recordSpend } from "./spend-limit.js";
+
 import type { InferInsertModel } from "@llmgateway/db";
 
 /**
@@ -328,6 +330,11 @@ export async function insertLog(
 		errorType,
 	});
 
+	// Maintain per-org daily/monthly spend-cap counters. Single DRY chokepoint
+	// for every request path; only increments when cost > 0 and swallows its own
+	// Redis errors so logging is never blocked.
+	await recordSpend(logData.organizationId, logData.cost ?? 0);
+
 	if (options?.syncInsert) {
 		await db.insert(log).values(logData as LogData);
 		return 1;

apps/gateway/src/lib/org-rate-limit.spec.ts

@@ -4,8 +4,8 @@ import {
 	baseLimitEnvVar,
 	checkOrgRateLimit,
 	getBaseLimit,
+	getOrgSpendTier,
 	getPlanClass,
-	getSpendTierMultiplier,
 	isOrgRateLimitEnabled,
 	PATH_RATE_LIMITS,
 	resolvePathRateLimit,
@@ -54,12 +54,8 @@ const ENV_KEYS = [
 	"GATEWAY_RATE_LIMIT_CHAT_COMPLETIONS_RPM",
 	"GATEWAY_RATE_LIMIT_DEV_CHAT_COMPLETIONS_RPM",
 	"GATEWAY_RATE_LIMIT_CHATPLAN_CHAT_COMPLETIONS_RPM",
-	"GATEWAY_RATE_LIMIT_TIER_1_THRESHOLD",
-	"GATEWAY_RATE_LIMIT_TIER_2_THRESHOLD",
-	"GATEWAY_RATE_LIMIT_TIER_3_THRESHOLD",
-	"GATEWAY_RATE_LIMIT_TIER_1_MULTIPLIER",
-	"GATEWAY_RATE_LIMIT_TIER_3_MULTIPLIER",
-	"GATEWAY_RATE_LIMIT_TIER_0_MULTIPLIER",
+	"GATEWAY_SPEND_TIER_1_SPEND_USD",
+	"GATEWAY_SPEND_TIER_1_RPM_MULTIPLIER",
 	"GATEWAY_RATE_LIMIT_WINDOW_SECONDS",
 ];
 
@@ -125,34 +121,68 @@ describe("isOrgRateLimitEnabled", () => {
 	});
 });
 
-describe("getSpendTierMultiplier", () => {
-	it("returns tier 0 below the first threshold", () => {
-		expect(getSpendTierMultiplier(0)).toEqual({ tier: 0, multiplier: 1 });
-		expect(getSpendTierMultiplier(999)).toEqual({ tier: 0, multiplier: 1 });
+describe("getOrgSpendTier", () => {
+	const NOW = Date.UTC(2026, 0, 15);
+	const daysAgo = (n: number) => {
+		const offsetMs = n * 86_400_000;
+		return new Date(NOW - offsetMs);
+	};
+
+	it("classifies a brand-new $0 org as T0", () => {
+		expect(getOrgSpendTier({ createdAt: daysAgo(0) }, 0, NOW)).toMatchObject({
+			tier: 0,
+			rpmMultiplier: 1,
+			dailyCapUsd: 5,
+			monthlyCapUsd: 50,
+		});
+	});
+
+	it("qualifies by account age alone (spend $0)", () => {
+		expect(getOrgSpendTier({ createdAt: daysAgo(8) }, 0, NOW).tier).toBe(1);
+		expect(getOrgSpendTier({ createdAt: daysAgo(31) }, 0, NOW).tier).toBe(2);
+		expect(getOrgSpendTier({ createdAt: daysAgo(61) }, 0, NOW).tier).toBe(3);
+		expect(getOrgSpendTier({ createdAt: daysAgo(90) }, 0, NOW).tier).toBe(4);
 	});
 
-	it("returns tier 1 at $1k", () => {
-		expect(getSpendTierMultiplier(1_000)).toEqual({ tier: 1, multiplier: 2 });
-		expect(getSpendTierMultiplier(9_999)).toEqual({ tier: 1, multiplier: 2 });
+	it("qualifies by lifetime spend alone on a brand-new org", () => {
+		const org = { createdAt: daysAgo(0) };
+		expect(getOrgSpendTier(org, 10, NOW).tier).toBe(1);
+		expect(getOrgSpendTier(org, 100, NOW).tier).toBe(2);
+		expect(getOrgSpendTier(org, 1_000, NOW).tier).toBe(3);
+		expect(getOrgSpendTier(org, 5_000, NOW).tier).toBe(4);
+		expect(getOrgSpendTier(org, 1_000_000, NOW).tier).toBe(4);
 	});
 
-	it("returns tier 2 at $10k", () => {
-		expect(getSpendTierMultiplier(10_000)).toEqual({ tier: 2, multiplier: 4 });
+	it("takes the highest of the age-or-spend qualifiers", () => {
+		// 31 days old (age → T2) but $1,000 lifetime spend (spend → T3) => T3
+		expect(getOrgSpendTier({ createdAt: daysAgo(31) }, 1_000, NOW).tier).toBe(
+			3,
+		);
 	});
 
-	it("returns tier 3 at $50k", () => {
-		expect(getSpendTierMultiplier(50_000)).toEqual({ tier: 3, multiplier: 10 });
-		expect(getSpendTierMultiplier(1_000_000)).toEqual({
-			tier: 3,
-			multiplier: 10,
+	it("treats thresholds as inclusive", () => {
+		expect(getOrgSpendTier({ createdAt: daysAgo(7) }, 0, NOW).tier).toBe(1);
+		expect(getOrgSpendTier({ createdAt: daysAgo(6) }, 9, NOW).tier).toBe(0);
+	});
+
+	it("exposes the caps and multiplier for the resolved tier", () => {
+		expect(
+			getOrgSpendTier({ createdAt: daysAgo(0) }, 5_000, NOW),
+		).toMatchObject({
+			tier: 4,
+			rpmMultiplier: 20,
+			dailyCapUsd: 15_000,
+			monthlyCapUsd: 200_000,
 		});
 	});
 
-	it("respects env overrides for thresholds and multipliers", () => {
-		process.env.GATEWAY_RATE_LIMIT_TIER_1_THRESHOLD = "500";
-		process.env.GATEWAY_RATE_LIMIT_TIER_1_MULTIPLIER = "3";
-		expect(getSpendTierMultiplier(500)).toEqual({ tier: 1, multiplier: 3 });
-		expect(getSpendTierMultiplier(499)).toEqual({ tier: 0, multiplier: 1 });
+	it("respects env overrides for tier thresholds and values", () => {
+		process.env.GATEWAY_SPEND_TIER_1_SPEND_USD = "5";
+		process.env.GATEWAY_SPEND_TIER_1_RPM_MULTIPLIER = "3";
+		expect(getOrgSpendTier({ createdAt: daysAgo(0) }, 5, NOW)).toMatchObject({
+			tier: 1,
+			rpmMultiplier: 3,
+		});
 	});
 });