Summary
TokenFirewall enforces global token limits but has no per-user budget system. Multiple users sharing an application can have one heavy user exhaust the quota and block everyone else.
Problem Statement
Without per-user budgets, a single user making rapid requests can consume the entire token allowance, degrading service for all other users with no fair-use enforcement.
Proposed Solution
Add a userBudget option to the middleware:
app.use(tokenFirewall({
provider: 'openai',
userBudget: {
extractUserId: (req) => req.user?.id,
monthlyTokenLimit: 100_000,
rollover: {
enabled: true,
maxCarryover: 50_000,
},
onBudgetExceeded: (req, res, info) => {
res.status(429).json({
error: 'Monthly token budget exceeded',
resetAt: info.resetAt,
used: info.used,
limit: info.limit,
});
},
},
}));
Budget keys in the storage adapter follow: tokenfirewall:budget:{userId}:{YYYY-MM}.
Rollover Calculation
At the start of each billing period: bonus = min(previousUnused, maxCarryover). New period effective limit = monthlyTokenLimit + bonus.
Storage
Add a StorageAdapter interface with MemoryAdapter (default) and RedisAdapter (uses atomic INCRBY for thread safety).
Acceptance Criteria
Summary
TokenFirewall enforces global token limits but has no per-user budget system. Multiple users sharing an application can have one heavy user exhaust the quota and block everyone else.
Problem Statement
Without per-user budgets, a single user making rapid requests can consume the entire token allowance, degrading service for all other users with no fair-use enforcement.
Proposed Solution
Add a
userBudgetoption to the middleware:Budget keys in the storage adapter follow:
tokenfirewall:budget:{userId}:{YYYY-MM}.Rollover Calculation
At the start of each billing period:
bonus = min(previousUnused, maxCarryover). New period effective limit =monthlyTokenLimit + bonus.Storage
Add a
StorageAdapterinterface withMemoryAdapter(default) andRedisAdapter(uses atomicINCRBYfor thread safety).Acceptance Criteria
maxCarryoverinto the next month.