Skip to content

[Feature] Add Per-User Token Budget with Monthly Rollover Support #88

Description

@anshul23102

Summary

TokenFirewall enforces global token limits but has no per-user budget system. Multiple users sharing an application can have one heavy user exhaust the quota and block everyone else.

Problem Statement

Without per-user budgets, a single user making rapid requests can consume the entire token allowance, degrading service for all other users with no fair-use enforcement.

Proposed Solution

Add a userBudget option to the middleware:

app.use(tokenFirewall({
  provider: 'openai',
  userBudget: {
    extractUserId: (req) => req.user?.id,
    monthlyTokenLimit: 100_000,
    rollover: {
      enabled: true,
      maxCarryover: 50_000,
    },
    onBudgetExceeded: (req, res, info) => {
      res.status(429).json({
        error: 'Monthly token budget exceeded',
        resetAt: info.resetAt,
        used: info.used,
        limit: info.limit,
      });
    },
  },
}));

Budget keys in the storage adapter follow: tokenfirewall:budget:{userId}:{YYYY-MM}.

Rollover Calculation

At the start of each billing period: bonus = min(previousUnused, maxCarryover). New period effective limit = monthlyTokenLimit + bonus.

Storage

Add a StorageAdapter interface with MemoryAdapter (default) and RedisAdapter (uses atomic INCRBY for thread safety).

Acceptance Criteria

  • Each user has an independent token counter.
  • A user who exceeds their budget receives 429; other users are unaffected.
  • Rollover carries unused tokens up to maxCarryover into the next month.
  • Redis adapter uses atomic increments to prevent race conditions.
  • Budget reset timestamp is included in the 429 response.
  • Unit tests cover: budget enforcement, rollover calculation, Redis adapter behavior.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions