Redis Async Proxy

Asynchronous proxy caching server built with FastAPI, Uvicorn, redis.asyncio, and httpx.

The /fetch endpoint receives an absolute URL, checks Redis, and returns:

• X-Cache: HIT when the JSON payload is served from Redis.
• X-Cache: MISS when the proxy fetches the upstream URL, stores it in Redis, and returns it.
• X-Cache: BYPASS when Redis is unavailable and the proxy fails through directly to the upstream API.

Concurrent identical cache misses are coalesced with an async single-flight task, so only one upstream HTTP request is made while the other callers await the same result.

The service also includes Redis-backed fixed-window rate limiting, manual cache invalidation, cache stats, and production-oriented Docker packaging.

Structure

.
├── app/
│   ├── __init__.py
│   ├── config.py
│   ├── main.py
│   ├── rate_limiter.py
│   ├── redis_state.py
│   ├── responses.py
│   └── singleflight.py
├── .dockerignore
├── .env.example
├── Dockerfile
├── docker-compose.yml
├── README.md
└── requirements.txt

Setup

python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt

Start Redis locally:

docker compose up -d redis

Copy or edit the environment file:

Copy-Item .env.example .env

Run the API:

uvicorn app.main:app --reload

Try the sample endpoint:

curl.exe -i "http://127.0.0.1:8000/fetch?url=https%3A%2F%2Fjsonplaceholder.typicode.com%2Ftodos%2F1"

Call it twice within 60 seconds: the first response should include X-Cache: MISS; the next should include X-Cache: HIT.

Invalidate one URL manually:

curl.exe -X POST "http://127.0.0.1:8000/cache/invalidate" `
  -H "Content-Type: application/json" `
  -d "{\"url\":\"https://jsonplaceholder.typicode.com/todos/1\"}"

Read cache and Redis memory stats:

curl.exe "http://127.0.0.1:8000/cache/stats"

Run the full containerized stack:

docker compose up --build

The Compose Redis service starts with --maxmemory 100mb and --maxmemory-policy allkeys-lru.

Environment

Variable	Default	Description
APP_NAME	Redis Async Proxy	FastAPI application title.
LOG_LEVEL	INFO	Python logging level.
REDIS_URL	redis://localhost:6379/0	Redis connection URL.
REDIS_RETRY_INTERVAL_SECONDS	5.0	Short cache-layer backoff after Redis errors.
CACHE_TTL_SECONDS	60	Cache TTL for upstream JSON payloads.
CACHE_KEY_PREFIX	async-proxy-cache	Prefix for Redis keys.
RATE_LIMIT_ENABLED	true	Enables Redis-backed fixed-window rate limiting.
RATE_LIMIT_REQUESTS	10	Max requests per client IP per window.
RATE_LIMIT_WINDOW_SECONDS	60	Rate-limit fixed window duration.
RATE_LIMIT_KEY_PREFIX	async-proxy-rate-limit	Prefix for rate-limit Redis keys.
RATE_LIMIT_FAIL_OPEN	true	Allows traffic if Redis is unavailable, preserving fail-through behavior.

Core Architecture & Technical Highlights

This proxy is engineered for high-throughput production environments where third-party API token costs, latency, and downstream reliability are critical constraints. It implements several advanced backend and network engineering patterns:

1. Request Coalescing (SingleFlight Pattern)

To eliminate the Cache Stampede (Thundering Herd) problem, the proxy integrates an asynchronous runtime tracking layer. When multiple concurrent requests target the exact same expired or missing URL simultaneously:

• Only one upstream HTTP request is dispatched.
• All other duplicate concurrent requests are put on hold (await) inside the event loop.
• Once the primary request returns, the JSON payload is propagated to all waiting clients and cached in Redis. This reduces upstream overhead by up to 99% during traffic spikes.

2. SSRF Protection & Network Isolation

Allowing arbitrary URL fetching poses severe Server-Side Request Forgery (SSRF) risks. This application enforces strict runtime security boundaries:

• Before initiating any TCP handshake via httpx, the target URL hostname is dynamically resolved to its IP address.
• The application intercepts and inspects the IP; if it belongs to local, private, or loopback subnets (e.g., 127.0.0.1, 10.0.0.0/8, 192.168.0.0/16), the request is immediately aborted before any external data is transmitted.

3. Asynchronous Distributed Rate Limiting

Traffic management is handled via an optimized atomic fixed-window rate limiter utilizing redis.asyncio:

• Identifies clients safely via real IP detection (supporting multi-tier proxy routing headers).
• Features a customizable Fail-Open mechanism (RATE_LIMIT_FAIL_OPEN): if the Redis cluster encounters a network drop or outage, the rate limiter gracefully bypasses itself to guarantee high availability (HA) for legitimate users.

4. Smart Cache Tracking & Resilient Failures

Every response injects custom execution headers (X-Cache: HIT | MISS | BYPASS) to allow real-time client-side tracing. The engine implements a graceful circuit breaker: if Redis goes completely offline, the system automatically switches to direct upstream fetch mode, ensuring zero application downtime. | RATE_LIMIT_EXCLUDED_PATHS | /health | Comma-separated paths excluded from rate limiting. | | TRUST_PROXY_HEADERS | false | Uses X-Forwarded-For/X-Real-IP when the app is behind a trusted proxy. | | HTTP_TIMEOUT_SECONDS | 10.0 | Upstream HTTP timeout. | | USER_AGENT | redis-async-proxy/0.2.0 | User-Agent sent to upstream APIs. |