PRODUCTION_FEATURES.md

Production Features Implementation

This document describes three production-ready features added to the Stellar Goal Vault backend:

1. API Key Authentication Middleware

Overview

Implements request-level authentication using API keys for production deployments. Protects write operations and sensitive endpoints while allowing public read access to certain endpoints.

Configuration

Set the API_KEYS environment variable with comma-separated valid API keys:

API_KEYS=key1,key2,key3

Usage

Include the API key in the Authorization header using Bearer token format:

curl -H "Authorization: Bearer your-api-key" https://api.example.com/api/campaigns

Public Endpoints (No Authentication Required)

• GET /api/health - Health check
• GET /api/config - Client configuration
• GET /api/stats - Global statistics
• GET /api/leaderboard - Top contributors
• GET /api/open-issues - GitHub issues

Protected Endpoints (Require Authentication)

• POST /api/campaigns - Create campaign
• POST /api/campaigns/:id/pledges - Add pledge
• POST /api/campaigns/:id/pledges/reconcile - Reconcile on-chain pledge
• POST /api/campaigns/:id/claim - Claim campaign
• POST /api/campaigns/:id/refund - Refund contributor
• GET /api/campaigns/:id/pledges - List pledges
• GET /api/campaigns/:id/contributors - Get contributors
• GET /api/campaigns/:id/history - Get campaign history

Error Responses

{
  "success": false,
  "error": {
    "code": "UNAUTHORIZED",
    "message": "Missing or invalid Authorization header. Use format: Bearer <api-key>",
    "requestId": "uuid"
  }
}

Implementation Details

• File: src/middleware/apiKeyAuth.ts
• Middleware: apiKeyAuthMiddleware
• Only enabled in production (NODE_ENV=production)
• Development mode allows all requests if API_KEYS is not set

---

2. Redis Cache Layer

Overview

Implements a distributed caching layer using Redis for production deployments. Caches GET request responses to reduce database load and improve API response times.

Configuration

Set the REDIS_URL environment variable:

REDIS_URL=redis://localhost:6379
# or with authentication
REDIS_URL=redis://:password@host:port

Features

• Automatic Cache Management: GET requests are automatically cached with configurable TTL
• Cache Invalidation: Cache is automatically invalidated on write operations
• Graceful Degradation: API continues to work if Redis is unavailable
• Production-Only: Cache is only enabled in production (NODE_ENV=production)

Cache Configuration

Default TTL: 300 seconds (5 minutes)

Customize TTL in src/middleware/cacheMiddleware.ts:

app.use(cacheMiddleware(600)); // 10 minutes

Cache Headers

Responses include cache status headers:

• X-Cache: HIT - Response served from cache
• X-Cache: MISS - Response generated fresh and cached

Cached Endpoints

All GET endpoints are cached:

• GET /api/campaigns - Campaign list
• GET /api/campaigns/:id - Campaign details
• GET /api/campaigns/:id/pledges - Campaign pledges
• GET /api/campaigns/:id/contributors - Contributor summary
• GET /api/campaigns/:id/history - Campaign history
• GET /api/stats - Global statistics
• GET /api/leaderboard - Top contributors

Cache Invalidation

Cache is automatically cleared when:

• New campaign is created
• New pledge is added
• Campaign is claimed
• Contributor is refunded

Implementation Details

• Files:
  • src/services/cache.ts - Redis client and cache operations
  • src/middleware/cacheMiddleware.ts - Express middleware for caching
• Functions:
  • initRedisCache() - Initialize Redis connection
  • getCacheValue(key) - Retrieve cached value
  • setCacheValue(key, value, ttl) - Store value in cache
  • deleteCacheValue(key) - Remove cached value
  • clearCachePattern(pattern) - Clear cache by pattern
  • isCacheAvailable() - Check cache availability

Error Handling

• Cache failures are logged but don't affect API functionality
• If Redis is unavailable, API continues to work without caching
• Connection errors are automatically logged

---

3. Concurrent Pledge Race Condition Tests

Overview

Comprehensive test suite for detecting and validating behavior under concurrent pledge operations. Tests ensure data consistency and proper handling of race conditions.

Test File

src/services/campaignStore.concurrent.test.ts

Test Cases

1. Concurrent Pledges Without Race Conditions

Tests that multiple concurrent pledges from different contributors are all recorded correctly.

- 4 concurrent pledges of 250 each
- Expected: All pledges recorded, total = 1000

2. Over-Pledging Prevention

Tests behavior when concurrent pledges exceed campaign target.

- 3 concurrent pledges of 300 each (total 900, target 500)
- Expected: All pledges recorded (no hard cap), total = 900

3. Per-Contributor Limits

Tests enforcement of per-contributor pledge limits under concurrent conditions.

- 2 concurrent pledges of 150 each from same contributor (limit 200)
- Expected: Both pledges recorded (race condition), total = 300
- Note: This demonstrates a known race condition

4. High Concurrent Load

Tests data consistency under heavy concurrent load.

- 20 concurrent pledges of 50 each
- Expected: All pledges recorded, total = 1000, no data corruption

5. Concurrent Claim and Pledge

Tests interaction between claim and pledge operations.

- Concurrent claim and pledge on expired campaign
- Expected: Both operations succeed, campaign claimed, pledge recorded

6. Duplicate Concurrent Pledges

Tests handling of duplicate pledges from same contributor.

- 3 concurrent identical pledges from same contributor
- Expected: All pledges recorded (no deduplication at this level)

Running Tests

# Run all tests
npm test

# Run only concurrent tests
npm test -- campaignStore.concurrent.test.ts

# Run with coverage
npm test -- --coverage

Known Race Conditions

The tests document the following race conditions:

1. Per-Contributor Limit Race Condition

   • When multiple pledges from the same contributor are submitted concurrently, the limit check may not see previous pledges
   • Result: Contributor can exceed their limit
   • Mitigation: Implement database-level constraints or use transactions

2. Campaign Funding Cap Race Condition

   • When pledges are submitted concurrently, the total can exceed the target
   • Result: Campaign can be over-funded
   • Mitigation: Implement atomic operations or use database locks

Implementation Details

• Uses Vitest for testing
• Isolated SQLite database per test
• Async/await for concurrent operations
• Promise.all() for parallel execution
• Comprehensive assertions on final state

Recommendations for Production

1. Database Transactions: Wrap pledge operations in transactions
2. Optimistic Locking: Add version fields to campaigns
3. Distributed Locks: Use Redis for cross-instance coordination
4. Event Sourcing: Record all operations for audit trail
5. Monitoring: Track pledge success/failure rates

---

Environment Variables

Required for Production

NODE_ENV=production
API_KEYS=key1,key2,key3
REDIS_URL=redis://localhost:6379

Optional

# Cache TTL in seconds (default: 300)
CACHE_TTL=600

# Redis connection timeout
REDIS_TIMEOUT=5000

# Log level
LOG_LEVEL=info

---

Deployment Checklist

• Set NODE_ENV=production
• Generate and configure API_KEYS
• Set up Redis instance and configure REDIS_URL
• Run concurrent tests to verify behavior
• Monitor cache hit rates and Redis performance
• Set up alerts for authentication failures
• Configure log aggregation for cache errors
• Test API key rotation procedure
• Document API key management process

---

Performance Considerations

Cache Performance

• Hit Rate: Monitor X-Cache headers to track hit rate
• TTL Tuning: Adjust TTL based on data freshness requirements
• Memory: Monitor Redis memory usage
• Eviction: Configure Redis eviction policy (e.g., allkeys-lru)

Authentication Performance

• Overhead: API key validation adds minimal overhead (~1ms)
• Scaling: Stateless design allows horizontal scaling
• Key Rotation: No downtime required for key rotation

Concurrency Performance

• Database: SQLite WAL mode supports concurrent reads
• Writes: Concurrent writes may cause contention
• Scaling: Consider PostgreSQL for higher concurrency

---

Troubleshooting

Cache Not Working

1. Check REDIS_URL is set and Redis is running
2. Check NODE_ENV=production
3. Review logs for Redis connection errors
4. Verify Redis credentials and network access

Authentication Failures

1. Verify API key is in API_KEYS environment variable
2. Check Authorization header format: Bearer <key>
3. Ensure NODE_ENV=production for authentication to be active
4. Review logs for authentication attempts

Race Conditions

1. Review concurrent test results
2. Monitor database lock contention
3. Consider implementing optimistic locking
4. Use database transactions for critical operations