Skip to content

Commit 817542a

Browse files
committed
Clarify readme
1 parent e76889d commit 817542a

1 file changed

Lines changed: 19 additions & 126 deletions

File tree

README.md

Lines changed: 19 additions & 126 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,14 @@
11
# AIMD Bucket
22

3-
A TypeScript/JavaScript implementation of an **AIMD (Additive Increase Multiplicative Decrease)** rate limiting token bucket with adaptive rate adjustment. This library is ideal for clients in distributed systems that need to discover and adapt to unknown server-side rate limits dynamically.
3+
A TypeScript/JavaScript implementation of an **AIMD (Additive Increase Multiplicative Decrease)** rate limiting token bucket with adaptive rate adjustment. This library is ideal for clients in distributed systems that need to discover and adapt to unknown remote system limits dynamically.
44

55
You can create a bucket with some configured defaults and boundaries, and then ask for tokens from it. You report if each token was successful in doing a unit of work, or if it encountered an error or a server side rate limit. The bucket will then start brokering tokens faster or slower depending on the outcomes you report. The bucket adjusts the rate limit using the smae simple adaptive limiting algorithm used in TCP: AIMD.
66

7+
## Example uses cases
8+
9+
- throttle your outgoing requests to a system that starts breaking under load to not break it
10+
- throttle your outgoing requests to a rate limited API from 3 different processes all competing for that rate limit
11+
712
## Installation
813

914
```bash
@@ -19,9 +24,12 @@ pnpm add aimd-bucket
1924
```typescript
2025
import { AIMDBucket } from "aimd-bucket";
2126

22-
// Create a bucket with default settings
27+
// Create a bucket with default settings (starts with unlimited rate)
2328
const bucket = new AIMDBucket();
2429

30+
// Or create a bucket with a conservative initial rate
31+
const conservativeBucket = new AIMDBucket({ initialRate: 10 });
32+
2533
// Acquire a token before making a request
2634
const token = await bucket.acquire();
2735

@@ -47,13 +55,13 @@ The `AIMDBucket` constructor accepts a configuration object with the following o
4755

4856
```typescript
4957
interface AIMDBucketConfig {
50-
initialRate?: number; // Initial rate limit (tokens per second), default: 10
51-
maxRate?: number; // Maximum rate limit, default: 100
58+
initialRate?: number; // Initial rate limit (tokens per second), default: maxRate
59+
maxRate?: number; // Maximum rate limit, default: Infinity
5260
minRate?: number; // Minimum rate limit, default: 1
5361
increaseDelta?: number; // Amount to increase rate by on success, default: 1
5462
decreaseMultiplier?: number; // Multiplier to decrease rate by on failure, default: 0.5
5563
failureThreshold?: number; // Failure threshold (0-1) that triggers decrease, default: 0.2
56-
tokenTimeoutMs?: number; // Token timeout in milliseconds, default: 30000
64+
tokenReturnTimeoutMs?: number; // Token timeout in milliseconds, default: 30000
5765
windowMs?: number; // Sliding window duration for rate decisions, default: 30000
5866
}
5967
```
@@ -68,128 +76,11 @@ const bucket = new AIMDBucket({
6876
increaseDelta: 2, // Increase by 2 on success
6977
decreaseMultiplier: 0.8, // Decrease to 80% on failure
7078
failureThreshold: 0.1, // Decrease rate if >10% failures
71-
tokenTimeoutMs: 60000, // Tokens expire after 60 seconds
79+
tokenReturnTimeoutMs: 60000, // Tokens expire after 60 seconds
7280
windowMs: 60000, // Use 60-second window for rate decisions
7381
});
7482
```
7583

76-
## Usage Patterns
77-
78-
### Basic API Rate Limiting
79-
80-
```typescript
81-
import { AIMDBucket } from "aimd-bucket";
82-
83-
const bucket = new AIMDBucket({ initialRate: 10 });
84-
85-
async function makeApiCall() {
86-
const token = await bucket.acquire();
87-
88-
try {
89-
const response = await fetch("https://api.example.com/endpoint");
90-
91-
if (response.ok) {
92-
token.success();
93-
return await response.json();
94-
} else if (response.status === 429) {
95-
token.rateLimited();
96-
throw new Error("Rate limited");
97-
} else {
98-
token.failure();
99-
throw new Error(`API error: ${response.status}`);
100-
}
101-
} catch (error) {
102-
token.failure();
103-
throw error;
104-
}
105-
}
106-
```
107-
108-
### Concurrent Request Handling
109-
110-
```typescript
111-
async function makeConcurrentRequests(count: number) {
112-
const promises = Array.from({ length: count }, async () => {
113-
const token = await bucket.acquire();
114-
115-
try {
116-
const response = await fetch("https://api.example.com/endpoint");
117-
token.success();
118-
return response.json();
119-
} catch (error) {
120-
token.failure();
121-
throw error;
122-
}
123-
});
124-
125-
return Promise.all(promises);
126-
}
127-
```
128-
129-
### Monitoring and Statistics
130-
131-
```typescript
132-
// Get current statistics
133-
const stats = bucket.getStatistics();
134-
console.log("Current rate:", stats.currentRate, "tokens/sec");
135-
console.log("Success rate:", (stats.successRate * 100).toFixed(1) + "%");
136-
console.log("Total tokens issued:", stats.tokensIssued);
137-
console.log("Recent failures:", stats.failureCount);
138-
139-
// Monitor rate changes
140-
setInterval(() => {
141-
const currentRate = bucket.getCurrentRate();
142-
console.log(`Current rate: ${currentRate} tokens/sec`);
143-
}, 5000);
144-
```
145-
146-
### OpenTelemetry Observability
147-
148-
The AIMD Bucket automatically emits OpenTelemetry spans when tokens are not immediately available and require waiting. This provides valuable observability into rate limiting behavior without any additional configuration.
149-
150-
#### Emitted Spans
151-
152-
**Span Name**: `token-bucket.wait`
153-
154-
**When Emitted**: Only when `acquire()` cannot immediately provide a token and the request must wait for capacity to become available.
155-
156-
**Attributes**:
157-
158-
- `token_bucket.current_rate` (number): Current rate limit in tokens per second
159-
- `token_bucket.available_tokens` (number): Number of tokens currently available
160-
- `token_bucket.pending_requests` (number): Number of requests currently waiting for tokens
161-
162-
**Example Usage**:
163-
164-
```typescript
165-
import { trace } from "@opentelemetry/api";
166-
167-
// The span is automatically created when waiting is required
168-
const token = await bucket.acquire(); // May create a span if tokens aren't immediately available
169-
170-
// You can access the current span context if needed
171-
const currentSpan = trace.getActiveSpan();
172-
if (currentSpan) {
173-
console.log("Current span:", currentSpan.name);
174-
}
175-
```
176-
177-
**Note**: No spans are emitted for immediate token acquisition when capacity is available. Spans are only created when there's an actual wait period, providing focused observability on rate limiting bottlenecks.
178-
179-
### Graceful Shutdown
180-
181-
```typescript
182-
// Shutdown the bucket gracefully
183-
await bucket.shutdown();
184-
185-
// All pending acquire() calls will be rejected
186-
try {
187-
await bucket.acquire(); // This will throw an error
188-
} catch (error) {
189-
console.log("Bucket is shut down");
190-
}
191-
```
192-
19384
## Token Lifecycle
19485

19586
Each token must be completed exactly once with one of these methods:
@@ -199,16 +90,18 @@ Each token must be completed exactly once with one of these methods:
19990
- `token.rateLimited()` - Request was rate limited (429 status)
20091
- `token.timeout()` - Request timed out
20192

93+
Successful responses will allow the rate limit to increase, and failed responses will force the rate limit to decrease.
94+
20295
### Token States
20396

20497
```typescript
20598
const token = await bucket.acquire();
20699

207-
console.log(token.isCompleted()); // false
208-
console.log(token.isExpired()); // false
100+
console.log(token.isCompleted); // false
101+
console.log(token.isExpired); // false
209102

210103
token.success();
211-
console.log(token.isCompleted()); // true
104+
console.log(token.isCompleted); // true
212105

213106
// Cannot complete the same token twice
214107
token.success(); // Throws error: "Token has already been completed"

0 commit comments

Comments
 (0)