Skip to content

Commit 9ea2167

Browse files
NotYuShengclaude
NotYuSheng
and
claude
committed
fix: Add Redis authentication support with conditional secret mounting
- Add conditional REDIS_PASSWORD environment variable mounting from redis-secrets - Update Redis deployment to handle authentication dynamically with config and CLI - Fix startup probe to support authentication like liveness/readiness probes - Update README.md with complete service table including all microservices - Add comprehensive security and architecture documentation 🤖 Generated with [Claude Code](https://claude.ai/code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent fe78b73 commit 9ea2167

8 files changed

Lines changed: 882 additions & 20 deletions

‎HPA-CONFIGURATION.md‎

Lines changed: 145 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,145 @@
1+
# Horizontal Pod Autoscaler (HPA) Configuration
2+
3+
## Overview
4+
5+
HPA automatically scales the number of pods based on CPU and memory utilization metrics to handle varying workloads efficiently.
6+
7+
## Services WITH HPA Enabled (6 services)
8+
9+
| **Service** | **Min Replicas** | **Max Replicas** | **CPU Target** | **Memory Target** | **Scaling Strategy** |
10+
|-------------|------------------|------------------|----------------|-------------------|---------------------|
11+
| chat-service | 2 | 10 | 70% | 80% | High demand during chat interactions |
12+
| docling-translation-service | 2 | 10 | 70% | 80% | Scales with translation workload |
13+
| pdf-processor-service | 2 | 8 | 70% | 80% | Main orchestrator - handles all requests |
14+
| pdf-extraction-service | 2 | 10 | 70% | 80% | CPU-intensive PDF processing |
15+
| pdf-renderer-service | 2 | 10 | 70% | 80% | Scales with rendering requests |
16+
| nginx | 2 | 10 | 70% | 80% | API gateway - handles all external traffic |
17+
18+
## Services WITH HPA Disabled (8 services)
19+
20+
| **Service** | **Status** | **Min/Max Replicas** | **Reason for Disabling** |
21+
|-------------|-----------|---------------------|--------------------------|
22+
| redis | `enabled: false` | 1/3 | **Stateful service** - Redis clustering is complex, typically single instance |
23+
| chromadb | `enabled: false` | 1/3 | **Vector database** - Stateful, requires careful scaling coordination |
24+
| frontend | `enabled: false` | 1/3 | **Streamlit app** - Single instance sufficient, stateful sessions |
25+
| cleaner | `enabled: false` | N/A | **Background worker** - Single scheduled job, no need to scale |
26+
| minio | `enabled: false` | 1/3 | **Object storage** - Single-node deployment, stateful storage |
27+
| embedder-service | `enabled: false` | 1/5 | Not explicitly stated - likely low/predictable load |
28+
| metadata-service | `enabled: false` | 1/3 | Disabled by default - likely infrequent usage |
29+
| image-captioner-service | `enabled: false` | 1/3 | Disabled by default for VLM service - expensive operations |
30+
31+
## HPA Configuration Details
32+
33+
### Standard Thresholds
34+
- **CPU Target**: 70% utilization (most services)
35+
- **Memory Target**: 80% utilization (most services)
36+
- **Scaling Frequency**: Every 15 seconds (Kubernetes default)
37+
- **Cooldown Period**: 3-5 minutes between scale events
38+
39+
### Replica Strategy
40+
- **High Availability**: Minimum 2 replicas for critical services
41+
- **Load Distribution**: Maximum 8-10 replicas for reasonable resource bounds
42+
- **Cost Optimization**: Services start at minimum replicas during low load
43+
44+
## Service Categories
45+
46+
### **1. Auto-Scaling Services (Traffic-Dependent)**
47+
**Services**: nginx, chat-service, pdf-processor-service
48+
- Handle user traffic directly
49+
- Need to scale with concurrent users
50+
- High availability requirements
51+
52+
### **2. Processing Services (Workload-Dependent)**
53+
**Services**: pdf-extraction-service, docling-translation-service, pdf-renderer-service
54+
- Scale based on processing queue depth
55+
- CPU/Memory intensive operations
56+
- Can have bursts during bulk processing
57+
58+
### **3. Stateful Services (No Scaling)**
59+
**Services**: redis, chromadb, minio
60+
- Maintain persistent state
61+
- Complex clustering requirements
62+
- Typically single instance or manual scaling
63+
64+
### **4. Utility Services (Low/Predictable Load)**
65+
**Services**: embedder-service, metadata-service, image-captioner-service, frontend, cleaner
66+
- Predictable or infrequent usage patterns
67+
- Either expensive operations or simple functionality
68+
- Manual scaling sufficient
69+
70+
## Monitoring and Tuning
71+
72+
### Key Metrics to Watch
73+
- **Pod CPU/Memory utilization** across services
74+
- **Request latency** during scaling events
75+
- **Queue depth** for processing services
76+
- **Scale-up/scale-down frequency**
77+
78+
### Common Tuning Scenarios
79+
80+
**Scale Too Aggressively:**
81+
- Increase CPU/Memory thresholds (70% → 80%)
82+
- Reduce maximum replicas
83+
84+
**Scale Too Slowly:**
85+
- Decrease thresholds (70% → 60%)
86+
- Reduce cooldown periods (advanced configuration)
87+
88+
**Resource Waste:**
89+
- Lower minimum replicas for less critical services
90+
- Implement custom metrics (requests/second vs CPU)
91+
92+
## Production Recommendations
93+
94+
### **Consider Enabling HPA For:**
95+
1. **embedder-service**: If document ingestion has spikes
96+
2. **metadata-service**: If metadata generation becomes frequent
97+
98+
### **Advanced Configuration:**
99+
```yaml
100+
# Example: Custom metrics for API services
101+
metrics:
102+
- type: Pods
103+
pods:
104+
metric:
105+
name: http_requests_per_second
106+
target:
107+
type: AverageValue
108+
averageValue: "100"
109+
```
110+
111+
### **Multi-Metric Scaling:**
112+
Most services use both CPU and Memory metrics. HPA scales based on whichever metric suggests more replicas needed.
113+
114+
## Troubleshooting
115+
116+
### Common Issues
117+
1. **HPA not scaling**: Check if metrics-server is running
118+
2. **Frequent scaling**: Adjust thresholds or cooldown periods
119+
3. **Resource requests missing**: HPA requires resource requests to be defined
120+
4. **Scale-down stuck**: Check PodDisruptionBudget settings
121+
122+
### Debug Commands
123+
```bash
124+
# Check HPA status
125+
kubectl get hpa -n omnipdf
126+
127+
# Detailed HPA information
128+
kubectl describe hpa chat-service-hpa -n omnipdf
129+
130+
# Check current CPU/Memory usage
131+
kubectl top pods -n omnipdf
132+
```
133+
134+
## Cost Optimization
135+
136+
### Current Resource Allocation
137+
- **Minimum pods running**: ~13 pods (assuming all minimums)
138+
- **Maximum pods possible**: ~63 pods (if all services scale to max)
139+
- **Typical running state**: 15-20 pods under normal load
140+
141+
### Optimization Strategies
142+
1. **Profile actual usage** before adjusting min/max replicas
143+
2. **Use Vertical Pod Autoscaler (VPA)** for right-sizing resources
144+
3. **Consider cluster autoscaler** for node-level scaling
145+
4. **Implement graceful degradation** for non-critical services during high load

‎README.md‎

Lines changed: 134 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -10,30 +10,117 @@ The following port mappings are used across the OmniPDF microservices in both de
1010
1111
| Service | Description | Port |
1212
|---------------------------|-----------------------------------------------------------|--------|
13+
| **Frontend Services** | | |
1314
| Streamlit Frontend | Web UI for user interaction | 8501 |
14-
| Nginx API Gateway | Proxies file uploads to PDF Processor | 8080 |
15-
| PDF Processor Service | Main coordinator for processing and routing | 8000 |
16-
| Chat Service | Retrieves context chunks and queries LLM | 8000 |
17-
| PDF Extraction Service | Extracts tables and images from PDFs | 8000 |
18-
| Docling Translate Service | Translates text fields in docling-style JSON | 8000 |
19-
| PDF Renderer Service | Overlays translated tables and text onto the original PDF | 8000 |
20-
| Embedder Service | Chunks + embeds PDF text and stores in ChromaDB | 8000 |
21-
| vLLM LLM Server | LLM backend for chat, translation, captions, summaries | 1234 |
22-
| Redis | In-memory session store | 6379 |
23-
| ChromaDB | Temporary in-memory vector store | 5100 |
24-
| S3-Compatible Store | Object storage (e.g., MinIO S3 API) | 9000 |
25-
| MinIO Console | MinIO web-based Admin UI | 9001 |
15+
| Nginx API Gateway | Proxies requests and handles file uploads | 8080 |
16+
| **Core Processing Services** | | |
17+
| PDF Processor Service | Main orchestrator - coordinates all processing workflows | 8000 |
18+
| PDF Extraction Service | Extracts tables and images from PDFs using docling | 8000 |
19+
| Docling Translation Service | Translates text fields in docling-format JSON | 8000 |
20+
| PDF Renderer Service | Renders translated content onto original PDFs | 8000 |
21+
| Embedder Service | Chunks text and creates embeddings for vector storage | 8000 |
22+
| Chat Service | RAG chat interface using retrieved context chunks | 8000 |
23+
| Image Captioner Service | AI image captioning for extracted images using VLM | 8000 |
24+
| Metadata Service | Document metadata and word cloud generation | 8000 |
25+
| Cleaner | Background cleanup of expired sessions and files | N/A |
26+
| **AI/ML Services** | | |
27+
| vLLM Text Model | Text-only LLM (Llama 3.1, Qwen2.5) for chat/translation | 8000 |
28+
| vLLM Vision-Language Model | Multimodal VLM (Llava, Qwen2-VL) for image captioning | 8000 |
29+
| **Data Services** | | |
30+
| Redis | Session storage and caching | 6379 |
31+
| ChromaDB | Vector database for embeddings | 8000 |
32+
| MinIO | S3-compatible object storage for files | 9000 |
33+
| MinIO Console | MinIO web-based Admin UI | 9001 |
34+
35+
## Architecture
36+
37+
OmniPDF follows a **microservices architecture** with **centralized orchestration**:
38+
39+
- **pdf-processor-service**: Main hub that coordinates all processing workflows
40+
- **Processing services**: Specialized services for extraction, translation, rendering, embedding, and chat
41+
- **Data layer**: Redis (sessions), ChromaDB (vectors), MinIO (files)
42+
- **AI/ML layer**: vLLM text and vision-language models
2643

2744
## Deployment Environments
2845

29-
OmniPDF supports multiple deployment environments with different orchestration methods:
46+
OmniPDF supports multiple deployment environments with **Kubernetes + Helm**:
3047

3148
- **Development**: Docker Compose for local development
32-
- **Pre-staging**: CodeReady Containers (CRC) with local OpenShift registry
33-
- **Staging**: Offline OpenShift Container Platform (OCP) with internal registries
34-
- **Production**: Offline OpenShift Container Platform (OCP) with internal registries
49+
- **Pre-staging**: CodeReady Containers (CRC) with Helm charts and local OpenShift registry
50+
- **Staging**: Offline OpenShift Container Platform (OCP) with Helm deployment
51+
- **Production**: Offline OpenShift Container Platform (OCP) with Helm deployment
3552

36-
**Note**: Staging and production environments are offline/disconnected and cannot access external registries. All container images must be pre-mirrored to internal registries.
53+
**Container Registry Patterns**:
54+
- **Development**: Local Docker images
55+
- **Pre-staging**: `default-route-openshift-image-registry.apps-crc.testing/omnipdf/SERVICE_NAME`
56+
- **Staging/Production**: Internal/disconnected registries (images must be pre-mirrored)
57+
58+
## Quick Start
59+
60+
### Development (Docker Compose)
61+
```bash
62+
# Start all services
63+
docker compose up --build
64+
65+
# Start with GPU support (for LLM services)
66+
docker compose -f docker-compose.gpu.yml up --build
67+
```
68+
69+
### Kubernetes/OpenShift (Helm)
70+
```bash
71+
# Deploy individual service
72+
helm install chat-service ./helm/chat-service --namespace omnipdf
73+
74+
# Deploy all services
75+
for service in helm/*/; do
76+
service_name=$(basename "$service")
77+
helm install "$service_name" "$service" --namespace omnipdf
78+
done
79+
80+
# Deploy RBAC (service accounts and permissions)
81+
helm install omnipdf-rbac ./helm/rbac --namespace omnipdf
82+
```
83+
84+
## Security Features
85+
86+
OmniPDF implements **defense-in-depth security** with multiple layers:
87+
88+
### Service Account & RBAC
89+
- **Individual service accounts** for each service with per-service secret isolation
90+
- **RBAC roles** with principle of least privilege:
91+
- `orchestrator-role`: pdf-processor (full coordination access)
92+
- `individual-service-roles`: Each service accesses only its own secrets
93+
- `cleaner-role`: Full data store access for cleanup operations
94+
- **Complete audit trail** for inter-service communication
95+
96+
### NetworkPolicy (Zero-Trust)
97+
- **Network segmentation** between services
98+
- **Ingress/egress controls** based on service communication matrix
99+
- **DNS and HTTPS egress** allowed for external services
100+
- **Pod selector-based** traffic rules
101+
102+
### HPA (Horizontal Pod Autoscaler)
103+
- **6 services** with auto-scaling enabled (nginx, chat-service, pdf-processor, etc.)
104+
- **CPU/Memory thresholds**: 70% CPU, 80% Memory
105+
- **High availability**: Minimum 2 replicas for critical services
106+
- **Resource optimization**: Scale from 2-10 replicas based on load
107+
108+
## Security Configuration
109+
110+
```bash
111+
# Enable NetworkPolicy for production
112+
helm upgrade chat-service ./helm/chat-service \
113+
--set networkPolicy.enabled=true \
114+
--namespace omnipdf
115+
116+
# Check service account permissions
117+
kubectl auth can-i get secrets \
118+
--as=system:serviceaccount:omnipdf:chat-service \
119+
-n omnipdf
120+
121+
# Monitor HPA status
122+
kubectl get hpa -n omnipdf
123+
```
37124

38125
## CRC (OpenShift Local) Setup
39126

@@ -93,6 +180,36 @@ oc describe node crc | grep -A 10 "Allocated resources"
93180
crc config view
94181
```
95182

183+
## Documentation
184+
185+
### Architecture & Design
186+
- [Service Communication Matrix](service-communication-matrix.md): Complete ingress/egress patterns for all services
187+
- [Service Account Matrix](service-account-matrix.md): RBAC permissions and secret access patterns
188+
189+
### Security & Operations
190+
- [Service Account Setup](SERVICE-ACCOUNT-SETUP.md): RBAC implementation and deployment guide
191+
- [Secret Management](SECRET-MANAGEMENT.md): Per-service secret isolation strategy
192+
- [HPA Configuration](HPA-CONFIGURATION.md): Auto-scaling configuration and monitoring
193+
194+
### Development
195+
- [CLAUDE.md](CLAUDE.md): Development commands, testing, and architecture details
196+
197+
## Testing
198+
199+
```bash
200+
# Run all service unit tests (206+ tests across 7 services)
201+
./scripts/test-all-services.sh
202+
203+
# Run tests for individual service
204+
./scripts/test-single-service.sh chat_service
205+
206+
# Security scanning with Trivy
207+
./scripts/scan_with_trivy.sh
208+
209+
# Lint all Helm charts
210+
for chart in helm/*/; do helm lint "$chart"; done
211+
```
212+
96213
## Development Workflow
97214

98215
This project uses a `Makefile` to simplify common Helm and Kubernetes operations.

0 commit comments

Comments
 (0)