tekdi · sourav-dev-hub · Jul 30, 2025
diff --git a/API_CACHING_RECOMMENDATIONS.md b/API_CACHING_RECOMMENDATIONS.md
@@ -0,0 +1,236 @@
+# API Caching Recommendations for User Microservice
+
+## Executive Summary
+
+This document provides a comprehensive analysis of all API endpoints in the User Microservice and identifies opportunities for implementing caching to improve performance, reduce database load, and enhance user experience. The recommendations are categorized by priority level and include specific caching strategies for each endpoint.
+
+---
+
+## High Priority Caching Opportunities
+
+### 1. **User Management Endpoints**
+
+#### **GET `/read/:userId`** - Get User Details
+- **Why Cache**: Frequently accessed user profile data that rarely changes
+- **Cache Strategy**: Redis with 15-30 minute TTL
+- **Cache Key**: `user:profile:{userId}:{tenantId}`
+- **Benefits**: Reduces database load for profile views, improves response time
+- **Invalidation**: On user updates, role changes, custom field modifications
+
+#### **POST `/list`** - Search/List Users  
+- **Why Cache**: Search results are expensive queries with filters, pagination
+- **Cache Strategy**: Redis with 5-10 minute TTL
+- **Cache Key**: `users:search:{hash(searchParams)}:{tenantId}`
+- **Benefits**: Improves search performance, reduces complex query load
+- **Invalidation**: Time-based expiry, user creation/updates
+
+#### **GET `/presigned-url`** - File Upload URLs
+- **Why Cache**: S3 presigned URL generation can be cached temporarily
+- **Cache Strategy**: In-memory cache with 5-10 minute TTL
+- **Cache Key**: `presigned:{filename}:{foldername}:{fileType}`
+- **Benefits**: Reduces AWS API calls
+- **Invalidation**: Short TTL-based expiry
+
+### 2. **Cohort Management Endpoints**
+
+#### **GET `/cohort/cohortHierarchy/:cohortId`** - Cohort Hierarchy
+- **Why Cache**: Complex hierarchical data structures, expensive joins
+- **Cache Strategy**: Redis with 30-60 minute TTL
+- **Cache Key**: `cohort:hierarchy:{cohortId}:{academicYearId}`
+- **Benefits**: Significant performance improvement for nested data retrieval
+- **Invalidation**: Cohort updates, member changes, hierarchy modifications
+
+#### **POST `/cohort/search`** - Search Cohorts
+- **Why Cache**: Filtered cohort lists accessed frequently by admins/teachers
+- **Cache Strategy**: Redis with 15-20 minute TTL
+- **Cache Key**: `cohorts:search:{hash(searchParams)}:{tenantId}:{academicYearId}`
+- **Benefits**: Faster dashboard loading, improved admin experience
+- **Invalidation**: Cohort creation/updates, time-based expiry
+
+#### **GET `/cohort/mycohorts/:userId`** - User's Cohorts
+- **Why Cache**: Frequently accessed for navigation, role-based content
+- **Cache Strategy**: Redis with 20-30 minute TTL
+- **Cache Key**: `user:cohorts:{userId}:{tenantId}:{academicYearId}`
+- **Benefits**: Faster app navigation, improved user experience
+- **Invalidation**: Cohort membership changes, role updates
+
+### 3. **Fields and Forms Configuration**
+
+#### **POST `/fields/search`** - Search Fields
+- **Why Cache**: Metadata that changes infrequently, used for form generation
+- **Cache Strategy**: Redis with 60-120 minute TTL
+- **Cache Key**: `fields:search:{hash(searchParams)}:{tenantId}`
+- **Benefits**: Faster form rendering, reduced metadata queries
+- **Invalidation**: Field configuration changes
+
+#### **GET `/fields/formFields`** - Form Custom Fields
+- **Why Cache**: Form configurations rarely change, used repeatedly
+- **Cache Strategy**: Redis with 2-4 hour TTL
+- **Cache Key**: `form:fields:{context}:{contextType}:{tenantId}`
+- **Benefits**: Significantly faster form loading
+- **Invalidation**: Field configuration updates
+
+#### **POST `/fields/options/read`** - Field Options
+- **Why Cache**: Dropdown/select options that are relatively static
+- **Cache Strategy**: Redis with 1-2 hour TTL
+- **Cache Key**: `field:options:{fieldName}:{controllingfieldfk}:{context}`
+- **Benefits**: Faster form field population
+- **Invalidation**: Option updates, dependency changes
+
+
+## Medium Priority Caching Opportunities
+
+### 5. **Cohort Members Management**
+
+#### **GET `/cohortmember/read/:cohortId`** - Cohort Members List
+- **Why Cache**: Member lists accessed frequently by instructors
+- **Cache Strategy**: Redis with 15-20 minute TTL
+- **Cache Key**: `cohort:members:{cohortId}:{tenantId}:{academicYearId}`
+- **Benefits**: Faster class roster loading
+- **Invalidation**: Member additions/removals, enrollment changes
+
+#### **POST `/cohortmember/list`** - Search Cohort Members
+- **Why Cache**: Filtered member searches for attendance, grading
+- **Cache Strategy**: Redis with 10-15 minute TTL
+- **Cache Key**: `cohort:members:search:{hash(searchParams)}:{tenantId}`
+- **Benefits**: Improved search performance for large cohorts
+- **Invalidation**: Member updates, enrollment changes
+
+### 6. **Tenant and Academic Year Data**
+
+#### **GET `/tenant/read`** - All Tenants
+- **Why Cache**: Tenant list rarely changes, used for configuration
+- **Cache Strategy**: Redis with 2-4 hour TTL
+- **Cache Key**: `tenants:all`
+- **Benefits**: Faster admin interface loading
+- **Invalidation**: Tenant creation/updates
+
+#### **POST `/tenant/search`** - Search Tenants
+- **Why Cache**: Administrative searches that are relatively static
+- **Cache Strategy**: Redis with 60-90 minute TTL
+- **Cache Key**: `tenants:search:{hash(searchParams)}`
+- **Benefits**: Improved admin dashboard performance
+- **Invalidation**: Tenant updates
+
+#### **GET `/academicyears/:id`** - Academic Year Details
+- **Why Cache**: Academic year data is stable once set
+- **Cache Strategy**: Redis with 4-6 hour TTL
+- **Cache Key**: `academicyear:{id}`
+- **Benefits**: Faster context loading across the application
+- **Invalidation**: Academic year updates (rare)
+
+#### **POST `/academicyears/list`** - Academic Years List
+- **Why Cache**: List of academic years changes infrequently
+- **Cache Strategy**: Redis with 2-3 hour TTL
+- **Cache Key**: `academicyears:list:{tenantId}`
+- **Benefits**: Faster dropdown/selection loading
+- **Invalidation**: Academic year creation/updates
+
+
+## Low Priority Caching Opportunities
+
+### 8. **Authentication and Session Management**
+
+#### **GET `/auth/`** - Get User by Auth Token
+- **Why Cache**: User session data validation
+- **Cache Strategy**: Redis with 10-15 minute TTL
+- **Cache Key**: `auth:user:{tokenHash}:{tenantId}`
+- **Benefits**: Reduced database calls for token validation
+- **Invalidation**: Token refresh, user updates
+
+### 9. **Form and Automatic Member Configuration**
+
+#### **GET `/form/read`** - Form Data
+- **Why Cache**: Form configurations are relatively stable
+- **Cache Strategy**: Redis with 1-2 hour TTL
+- **Cache Key**: `form:data:{context}:{contextType}:{tenantId}`
+- **Benefits**: Faster form rendering
+- **Invalidation**: Form configuration updates
+
+#### **GET `/automaticMember`** - All Automatic Members
+- **Why Cache**: Configuration data that changes infrequently
+- **Cache Strategy**: Redis with 60-90 minute TTL
+- **Cache Key**: `automatic:members:all`
+- **Benefits**: Faster configuration loading
+- **Invalidation**: Configuration updates
+
+---
+
+## Implementation Strategy
+
+### Cache Technologies Recommended:
+
+1. **Redis** - Primary caching layer for most endpoints
+   - Supports complex data structures
+   - Excellent for multi-server deployments
+   - Built-in TTL and eviction policies
+
+2. **In-Memory Cache** - For temporary, high-frequency data
+   - File upload URLs
+   - Session tokens
+   - Quick lookup data
+
+3. **Browser/CDN Caching** - For static responses
+   - Static form configurations
+   - Public metadata
+   - File downloads
+
+### Cache Invalidation Strategies:
+
+1. **Time-based (TTL)** - For data with predictable change patterns
+2. **Event-based** - For data that changes based on user actions
+3. **Manual invalidation** - For critical data requiring immediate consistency
+4. **Cache-aside pattern** - For complex, expensive queries
+
+### Monitoring and Metrics:
+
+- Cache hit/miss ratios
+- Response time improvements
+- Database load reduction
+- Memory usage patterns
+- Cache invalidation frequency
+
+---
+
+## Expected Performance Benefits
+
+### Database Load Reduction:
+- **High Priority implementations**: 40-60% reduction in database queries
+- **Medium Priority implementations**: 20-30% additional reduction
+- **Overall**: Up to 70-80% reduction in repetitive database calls
+
+### Response Time Improvements:
+- **User profile queries**: 200-500ms → 10-50ms
+- **Search operations**: 500-2000ms → 50-200ms
+- **Form loading**: 300-800ms → 20-100ms
+- **Authorization checks**: 100-300ms → 5-20ms
+
+### Scalability Benefits:
+- Support for 3-5x more concurrent users
+- Reduced database connection pool pressure
+- Better handling of traffic spikes
+- Improved overall system responsiveness
+
+---
+
+## Implementation Priority
+
+### Phase 1 (Immediate - High Impact):
+1. User profile caching (`/read/:userId`)
+2. User roles and privileges caching
+3. Form fields configuration caching
+4. Cohort hierarchy caching
+
+### Phase 2 (Short-term - Medium Impact):
+1. Search result caching
+2. Cohort members caching
+3. Tenant and academic year data
+4. Field options caching
+
+### Phase 3 (Long-term - System Optimization):
+1. Authentication token caching
+2. Location and configuration data
+3. Fine-tuning and monitoring improvements
+4. Advanced cache warming strategies
+
+This comprehensive caching strategy will significantly improve the User Microservice's performance, reduce infrastructure costs, and provide a better user experience across all user types (students, teachers, administrators). 
diff --git a/caching.md b/caching.md
@@ -0,0 +1,86 @@
+# API Caching Recommendations for User Microservice
+
+## Executive Summary
+
+This document provides a comprehensive analysis of all API endpoints in the User Microservice and identifies opportunities for implementing caching to improve performance, reduce database load, and enhance user experience. The recommendations are categorized by priority level and include specific caching strategies for each endpoint.
+
+---
+
+## Technical Context for the User Microservice
+
+### 1. Service Architecture & Caching Layer
+
+The User Microservice is built with **NestJS 10** and deployed as a containerised workload in Kubernetes.  A custom, _pluggable_ caching layer lives under `src/cache` and is registered **globally** during application bootstrap.  All controllers, services, guards, and pipes can therefore inject `CacheService` directly without additional module wiring.
+
+| File / Dir | Responsibility |
+|------------|----------------|
+| `src/cache/cache.module.ts` | Detects environment variables and selects an appropriate strategy at runtime. |
+| `src/cache/cache.service.ts` | Thin wrapper that exposes a unified API—`get`, `set`, `del`, `reset`—regardless of the underlying store. |
+| `src/cache/strategies/` | Strategy implementations: **In-Memory**, **Redis**, **File-system**, **Multi-layer**, and **NoCache**. |
+
+The **MultiCacheStrategy** can fan-out writes to Redis _and_ hold a short-lived in-memory replica, delivering sub-millisecond reads for hot keys while still providing cross-pod consistency.
+
+### 2. Environment Variables
+
+The behaviour of the cache layer is driven entirely by configuration so that the same container image can run in development, staging, and production:
+
+| Variable | Allowed Values | Default | Purpose |
+|----------|----------------|---------|---------|
+| `CACHE_STRATEGY` | `redis`, `inmemory`, `file`, `multi`, `none` | `inmemory` | Selects strategy; `none` or `CACHE_ENABLED=false` disables caching altogether. |
+| `CACHE_ENABLED` | `true` \| `false` | `true` | Master switch that forces **NoCacheStrategy** when `false`. |
+| `CACHE_TTL` | Integer seconds | `3600` | Global fallback TTL when none is supplied programmatically. |
+| `REDIS_HOST` / `REDIS_PORT` | host / port | `localhost` / `6379` | Connection details for Redis strategy. |
+| `FILE_CACHE_PATH` | Path | `./cache/files` | Storage path used by File strategy. |
+
+**Example `.env` for production**
+```ini
+CACHE_STRATEGY=redis
+REDIS_HOST=redis-master
+REDIS_PORT=6379
+CACHE_TTL=900
+```
+
+### 3. Using `CacheService` in Business Logic
+
+```ts
+import { Injectable } from "@nestjs/common";
+import { CacheService } from "src/cache/cache.service";
+import { UserRepository } from "src/user/user.repository";
+import { UserDto } from "src/user/dto/user-response.dto";
+
+@Injectable()
+export class UserService {
+  constructor(
+    private readonly cache: CacheService,
+    private readonly repo: UserRepository,
+  ) {}
+
+  async getUserProfile(userId: string, tenantId: string): Promise<UserDto> {
+    const cacheKey = `user:profile:${userId}:${tenantId}`;
+
+    // 1. Fast path — return from cache if present
+    const cached = await this.cache.get<UserDto>(cacheKey);
+    if (cached) return cached;
+
+    // 2. Fallback to database
+    const entity = await this.repo.findById(userId);
+    const dto = this.mapToDto(entity);
+
+    // 3. Store in cache for 30 minutes (1 800 seconds)
+    await this.cache.set(cacheKey, dto, 1800);
+    return dto;
+  }
+
+  private mapToDto(/* … */) /* … */ {}
+}
+```
+
+### 4. Invalidation Guidelines
+
+1. **Write-through**: Any service that mutates state must immediately `del` the affected keys or overwrite them with fresh values.
+2. **Bulk changes** (e.g. role assignment, CSV imports) should publish a **Kafka** event that downstream consumers use to clear or rebuild their portion of the key-space.
+3. **Scheduled flushes**: Use `CacheService.reset()` only during maintenance windows; routine operations should rely on TTL and targeted invalidation.
+
+These practices ensure the recommendations in the next sections can be implemented safely without risking stale data.
+
+---