Skip to content

Commit 86c90ef

Browse files
authored
Feat/259 (#355)
* tooltip * display * redis * soroban
1 parent ebf878a commit 86c90ef

15 files changed

Lines changed: 2093 additions & 24 deletions

File tree

.github/workflows/rust-wasm.yml

Lines changed: 37 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,4 +83,40 @@ jobs:
8383
uses: actions/upload-artifact@v4
8484
with:
8585
name: wasm-artifacts
86-
path: artifacts/wasm
86+
path: artifacts/wasm
87+
88+
testnet-deploy:
89+
name: Testnet deploy & smoke test
90+
runs-on: ubuntu-latest
91+
needs: wasm-build
92+
# Only run on push to main — not on pull requests
93+
if: github.event_name == 'push' && github.ref == 'refs/heads/main'
94+
steps:
95+
- name: Checkout
96+
uses: actions/checkout@v4
97+
98+
- name: Download WASM artifacts
99+
uses: actions/download-artifact@v4
100+
with:
101+
name: wasm-artifacts
102+
path: artifacts/wasm
103+
104+
- name: Verify vault WASM is present
105+
run: ls -la artifacts/wasm/
106+
107+
- name: Install Stellar CLI
108+
uses: stellar/stellar-cli@v23.0.1
109+
110+
- name: Deploy contract and run smoke test
111+
env:
112+
TESTNET_SECRET_KEY: ${{ secrets.TESTNET_SECRET_KEY }}
113+
TESTNET_TOKEN_ADDRESS: ${{ secrets.TESTNET_TOKEN_ADDRESS }}
114+
GIT_SHA: ${{ github.sha }}
115+
run: bash contracts/vault/scripts/smoke-test.sh
116+
117+
- name: Upload deployment artifact
118+
uses: actions/upload-artifact@v4
119+
with:
120+
name: testnet-deployment
121+
path: deployment.json
122+
retention-days: 7
Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
{"specId": "1e0f4317-5877-48ed-83d6-8c0622026c5f", "workflowType": "requirements-first", "specType": "feature"}
Lines changed: 307 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,307 @@
1+
# Design Document: Redis Rate Limiting
2+
3+
## Overview
4+
5+
This design replaces the current in-memory `express-rate-limit` store in the YieldVault backend with a Redis-backed store. The migration introduces per-wallet-address keying, per-endpoint limit configuration, RFC 6585-compliant 429 responses, and a fail-open fallback for Redis unavailability.
6+
7+
The existing `rateLimiter.ts` module is refactored into a `RateLimiterFactory` that constructs per-endpoint middleware instances. Each instance shares a single Redis client but uses a key prefix derived from the endpoint path, preventing counter collisions across routes.
8+
9+
**Key design decisions:**
10+
11+
- **`rate-limit-redis`** is used as the Redis store adapter for `express-rate-limit`. It is maintained by the same organisation as `express-rate-limit`, supports both `node-redis` and `ioredis`, and uses an atomic Lua script internally to prevent race conditions on counter increment + expiry-set.
12+
- **`ioredis`** is chosen as the Redis client over `node-redis` because it provides built-in reconnection logic, a `lazyConnect` option that avoids blocking startup, and a well-typed TypeScript API. The npm page for `ioredis` notes it is in maintenance mode and recommends `node-redis` for new projects; however, `ioredis` remains fully functional and its reconnection model is simpler to configure for the fail-open requirement.
13+
- **Fixed window** strategy is used (the default in `express-rate-limit`). It is simpler to reason about, sufficient for the stated abuse-prevention goal, and avoids the memory overhead of sliding-window per-key state.
14+
- **Fail-open** on Redis unavailability: the middleware skips enforcement rather than returning 503, preserving API availability during Redis outages.
15+
16+
---
17+
18+
## Architecture
19+
20+
```mermaid
21+
flowchart TD
22+
Client -->|HTTP request| Express
23+
Express -->|/api/v1| VersionMiddleware
24+
VersionMiddleware -->|route match| EndpointLimiter["Per-Endpoint Rate Limiter\n(express-rate-limit)"]
25+
EndpointLimiter -->|sendCommand| RedisStore["RedisStore\n(rate-limit-redis)"]
26+
RedisStore -->|EVAL Lua script| Redis[(Redis)]
27+
Redis -->|counter + TTL| RedisStore
28+
RedisStore -->|allow / deny| EndpointLimiter
29+
EndpointLimiter -->|429 or next()| RouteHandler
30+
EndpointLimiter -.->|Redis unreachable| FailOpen["Fail-open:\npass request through"]
31+
```
32+
33+
The `RateLimiterFactory` module owns the Redis client lifecycle. It is imported once at application startup and exports pre-built middleware instances for each endpoint. The `index.ts` file replaces the single `apiLimiter` import with the per-endpoint limiters.
34+
35+
---
36+
37+
## Components and Interfaces
38+
39+
### `RateLimiterFactory` (`backend/src/rateLimiter.ts`)
40+
41+
Replaces the current single-export module. Responsibilities:
42+
43+
1. Create and manage the `ioredis` client.
44+
2. Expose a `createLimiter(config: EndpointLimiterConfig): RequestHandler` factory function.
45+
3. Export pre-built limiter instances for each configured endpoint.
46+
4. Export a `getRedisClient()` accessor for health-check use.
47+
48+
```typescript
49+
interface EndpointLimiterConfig {
50+
/** Route prefix used as Redis key prefix, e.g. '/api/v1/vault/deposits' */
51+
routePrefix: string;
52+
/** Maximum requests per window */
53+
max: number;
54+
/** Window duration in milliseconds */
55+
windowMs: number;
56+
}
57+
58+
/**
59+
* Extracts the rate-limit key from a request.
60+
* Priority: walletAddress (body) > x-wallet-address (header) > x-api-key (header) > IP > 'unknown'
61+
*/
62+
function extractRateLimitKey(req: Request): string;
63+
64+
/**
65+
* Constructs the Redis key for a given route prefix and identifier.
66+
* Format: `rl:{routePrefix}:{identifier}`
67+
*/
68+
function buildRedisKey(routePrefix: string, identifier: string): string;
69+
70+
/**
71+
* Creates an express-rate-limit middleware instance backed by Redis.
72+
* Falls back to in-memory store when Redis is unavailable.
73+
*/
74+
function createLimiter(config: EndpointLimiterConfig): RequestHandler;
75+
76+
// Pre-built exports
77+
export const depositsLimiter: RequestHandler;
78+
export const summaryLimiter: RequestHandler;
79+
export const defaultLimiter: RequestHandler;
80+
```
81+
82+
### `RedisClientManager`
83+
84+
An internal singleton that wraps `ioredis`. It:
85+
86+
- Connects lazily (`lazyConnect: true`) so startup is not blocked if Redis is unavailable.
87+
- Emits structured log messages on `connect`, `reconnecting`, and `error` events.
88+
- Exposes `isReady(): boolean` for the fail-open check.
89+
- Truncates/hashes wallet addresses before logging (production mode).
90+
91+
### `index.ts` changes
92+
93+
- Remove the single `app.use('/api/v1', apiLimiter)` call.
94+
- Apply `depositsLimiter` to `POST /api/v1/vault/deposits`.
95+
- Apply `summaryLimiter` to `GET /api/v1/vault/summary`.
96+
- Apply `defaultLimiter` to the remaining `/api/v1` routes.
97+
98+
---
99+
100+
## Data Models
101+
102+
### Redis Key Schema
103+
104+
```
105+
rl:{routePrefix}:{identifier}
106+
```
107+
108+
Examples:
109+
- `rl:/api/v1/vault/deposits:GABC...XYZ` — wallet-keyed deposit counter
110+
- `rl:/api/v1/vault/summary:192.168.1.1` — IP-keyed summary counter
111+
- `rl:/api/v1/vault/deposits:unknown` — fallback key when no identifier is available
112+
113+
The `rate-limit-redis` library stores a single integer counter per key with a TTL equal to the window duration. The Lua script it uses performs an atomic `INCR` + conditional `EXPIRE` in a single round-trip.
114+
115+
### Environment Variables
116+
117+
| Variable | Default | Description |
118+
|---|---|---|
119+
| `REDIS_URL` | _(none)_ | Redis connection URL. If absent, falls back to in-memory store. |
120+
| `DEPOSITS_RATE_LIMIT_MAX` | `10` | Max requests per window for `/vault/deposits`. |
121+
| `DEPOSITS_RATE_LIMIT_WINDOW_MS` | `60000` | Window duration (ms) for `/vault/deposits`. |
122+
| `SUMMARY_RATE_LIMIT_MAX` | `30` | Max requests per window for `/vault/summary`. |
123+
| `SUMMARY_RATE_LIMIT_WINDOW_MS` | `60000` | Window duration (ms) for `/vault/summary`. |
124+
| `API_RATE_LIMIT_MAX_REQUESTS` | `30` | Default max requests per window (all other endpoints). |
125+
| `API_RATE_LIMIT_WINDOW_MS` | `60000` | Default window duration (ms). |
126+
127+
### 429 Response Body
128+
129+
```json
130+
{
131+
"error": "Rate limit exceeded",
132+
"status": 429,
133+
"message": "Too many requests. Please try again in {N} seconds.",
134+
"retryAfter": 42
135+
}
136+
```
137+
138+
### Rate Limit Response Headers (all `/api/v1` responses)
139+
140+
| Header | Description |
141+
|---|---|
142+
| `RateLimit-Limit` | Configured max requests for the endpoint |
143+
| `RateLimit-Remaining` | Requests remaining in current window (0 on 429) |
144+
| `RateLimit-Reset` | UTC epoch second when the window resets |
145+
| `Retry-After` | Seconds until window reset (429 responses only) |
146+
147+
---
148+
149+
## Correctness Properties
150+
151+
*A property is a characteristic or behavior that should hold true across all valid executions of a system — essentially, a formal statement about what the system should do. Properties serve as the bridge between human-readable specifications and machine-verifiable correctness guarantees.*
152+
153+
### Property 1: Key extraction uses wallet address when present
154+
155+
*For any* request containing a `walletAddress` field in the JSON body or an `x-wallet-address` header, the `extractRateLimitKey` function SHALL return that wallet address as the rate-limit key.
156+
157+
**Validates: Requirements 2.1**
158+
159+
---
160+
161+
### Property 2: Key extraction falls back to IP when no wallet address is present
162+
163+
*For any* request that contains neither a `walletAddress` body field nor an `x-wallet-address` header, the `extractRateLimitKey` function SHALL return the client IP address (or `'unknown'` when IP is absent) as the rate-limit key.
164+
165+
**Validates: Requirements 2.2, 7.1**
166+
167+
---
168+
169+
### Property 3: Distinct wallet addresses have independent counters
170+
171+
*For any* two distinct wallet addresses A and B, incrementing the counter for A SHALL NOT change the counter value for B.
172+
173+
**Validates: Requirements 2.3**
174+
175+
---
176+
177+
### Property 4: Redis key contains endpoint prefix
178+
179+
*For any* wallet address and endpoint route prefix, the Redis key produced by `buildRedisKey` SHALL contain the route prefix as a component, ensuring keys for different endpoints never collide.
180+
181+
**Validates: Requirements 2.4**
182+
183+
---
184+
185+
### Property 5: Requests beyond the limit receive 429 with required headers and body
186+
187+
*For any* endpoint with configured limit N and any wallet address, after N+1 requests within the same window, the (N+1)th response SHALL have status 429, include `Retry-After`, `RateLimit-Limit`, `RateLimit-Remaining` (= 0), and `RateLimit-Reset` headers, and a JSON body containing `error`, `status`, `message`, and `retryAfter` fields.
188+
189+
**Validates: Requirements 4.1, 4.2, 4.3, 4.4, 4.5, 4.6**
190+
191+
---
192+
193+
### Property 6: Requests within the limit include rate-limit headers
194+
195+
*For any* request to a rate-limited endpoint that returns HTTP 200, the response SHALL include `RateLimit-Limit`, `RateLimit-Remaining`, and `RateLimit-Reset` headers.
196+
197+
**Validates: Requirements 4.7**
198+
199+
---
200+
201+
### Property 7: Counter initialises to 1 on first request in a window
202+
203+
*For any* wallet address with no prior requests in the current window, after exactly one request the `RateLimit-Remaining` header SHALL equal `N - 1` (where N is the configured limit), confirming the counter was initialised to 1.
204+
205+
**Validates: Requirements 5.1**
206+
207+
---
208+
209+
### Property 8: Rate-limit log entries contain required fields without exposing full wallet address
210+
211+
*For any* rate-limited request with any wallet address, the structured log entry emitted SHALL contain the endpoint path, the current counter value, and the window reset time, and in production mode SHALL NOT contain the full wallet address string.
212+
213+
**Validates: Requirements 6.1, 6.4**
214+
215+
---
216+
217+
### Property 9: Environment variable overrides are applied
218+
219+
*For any* valid positive integer value set as `DEPOSITS_RATE_LIMIT_MAX` or `API_RATE_LIMIT_MAX_REQUESTS`, the corresponding limiter's `max` configuration SHALL equal that value.
220+
221+
**Validates: Requirements 3.4**
222+
223+
---
224+
225+
## Error Handling
226+
227+
### Redis Unavailable at Startup
228+
229+
When `REDIS_URL` is set but the Redis server is unreachable at startup:
230+
231+
1. `ioredis` with `lazyConnect: true` does not throw during client construction.
232+
2. The first `sendCommand` call from `rate-limit-redis` will fail.
233+
3. The `store` option in `express-rate-limit` does not have a built-in skip-on-error mechanism; the `skip` callback is used to detect Redis unavailability and bypass the store entirely.
234+
4. A structured error log is emitted: `{ level: 'error', event: 'redis_unavailable', host, port, reason }`.
235+
5. All requests pass through (fail-open).
236+
237+
### Redis Disconnects During Operation
238+
239+
`ioredis` emits an `error` event on connection loss. The `RedisClientManager` listens for this event and sets an internal `redisAvailable` flag to `false`. The `skip` callback in each limiter checks this flag and bypasses enforcement while Redis is down. When `ioredis` reconnects (it retries automatically), the flag is reset to `true` and enforcement resumes.
240+
241+
### Invalid Environment Variables
242+
243+
If `DEPOSITS_RATE_LIMIT_MAX` or similar variables are set to non-numeric values, `parseInt(..., 10)` returns `NaN`. The config loader uses `Number.isFinite(parsed) ? parsed : DEFAULT` to fall back to compiled-in defaults.
244+
245+
### Missing `REDIS_URL`
246+
247+
When `REDIS_URL` is not set, the factory skips Redis client creation entirely and constructs limiters with the default in-memory store. A `warn`-level log is emitted: `{ level: 'warn', event: 'redis_not_configured', message: 'REDIS_URL not set; using in-memory rate limit store' }`.
248+
249+
---
250+
251+
## Testing Strategy
252+
253+
### Dual Testing Approach
254+
255+
Unit tests cover specific examples, edge cases, and error conditions. Property-based tests verify universal properties across generated inputs. Both are needed for comprehensive coverage.
256+
257+
### Property-Based Testing Library
258+
259+
**`fast-check`** with **`@fast-check/jest`** is used for property-based tests. It integrates natively with Jest (the existing test runner), supports TypeScript, and provides rich arbitraries for strings, integers, and records. Each property test is configured to run a minimum of **100 iterations**.
260+
261+
Tag format for each property test:
262+
```
263+
// Feature: redis-rate-limiting, Property {N}: {property_text}
264+
```
265+
266+
### Unit Tests
267+
268+
Located in `backend/src/__tests__/rateLimiter.test.ts`:
269+
270+
- `extractRateLimitKey` with body wallet address, header wallet address, API key fallback, IP fallback, and `unknown` fallback.
271+
- `buildRedisKey` with various route prefixes and identifiers.
272+
- Config loading: valid env vars, non-numeric env vars, absent env vars.
273+
- 429 response body shape.
274+
- Fail-open when Redis is unavailable (mock `ioredis` to throw).
275+
- Redis connection/disconnection log events.
276+
- Wallet address truncation in production log output.
277+
278+
### Property-Based Tests
279+
280+
Located in `backend/src/__tests__/rateLimiter.property.test.ts`:
281+
282+
| Property | Arbitraries | Assertion |
283+
|---|---|---|
284+
| P1: Key extraction — wallet address | `fc.string()` for wallet address, request variants | extracted key === wallet address |
285+
| P2: Key extraction — IP fallback | requests without wallet address, `fc.ipV4()` | extracted key === IP |
286+
| P3: Independent counters | two distinct wallet addresses, `fc.nat()` for counts | counter A unchanged after incrementing B |
287+
| P4: Redis key prefix | `fc.string()` for prefix and identifier | key contains prefix |
288+
| P5: 429 headers and body | `fc.integer({min:1, max:20})` for limit N | after N+1 requests: status=429, all headers present, body fields present |
289+
| P6: 200 includes rate-limit headers | `fc.integer({min:1, max:29})` for request count within limit | all three RateLimit-* headers present |
290+
| P7: Counter initialises to 1 | `fc.string()` for wallet address | RateLimit-Remaining = N-1 after first request |
291+
| P8: Log fields and PII masking | `fc.string()` for wallet address | log contains required fields; full address absent in production |
292+
| P9: Env var override | `fc.integer({min:1, max:1000})` for limit value | limiter.max === env var value |
293+
294+
### Integration Tests
295+
296+
Located in `backend/src/__tests__/rateLimiter.integration.test.ts` (requires a running Redis instance, skipped in CI without `REDIS_URL`):
297+
298+
- Counter persists across client reconnection (Requirement 1.2).
299+
- Window expiry resets counter (Requirements 5.2, 5.3).
300+
- Two simulated instances share the same counter (Requirement 1.3).
301+
302+
### Existing Tests
303+
304+
The existing `api.test.ts` rate-limiting tests are updated to:
305+
- Use per-endpoint limiters instead of the global `apiLimiter`.
306+
- Assert `Retry-After` header presence on 429 responses.
307+
- Assert the new JSON body shape (`retryAfter` field).

0 commit comments

Comments
 (0)