Feature/helm setup

CLAUDE.md

@@ -0,0 +1,141 @@
+# CLAUDE.md
+
+This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+
+## Project Overview
+
+OmniPDF is a microservices-based PDF analyzer with translation, summarization, captioning and RAG capabilities. The system consists of 8 main services orchestrated via Docker Compose and deployable with Helm/Kubernetes.
+
+## Core Architecture
+
+### Service Structure
+- **pdf_processor_service** (port 8000): Main coordinator for processing and routing
+- **chat_service** (port 8001): RAG-based conversational interface with LLM integration
+- **pdf_extraction_service** (port 8002): Extracts tables and images from PDFs
+- **docling_translation_service** (port 8003): Translates docling-style JSON content
+- **embedder_service** (port 8005): Text chunking and embedding with ChromaDB storage
+- **nginx** (port 8080): API gateway proxying file uploads
+- **cleaner**: Background cleanup service
+
+### Dependencies
+- **Redis** (port 6379): Session management
+- **ChromaDB** (port 5100): Vector storage for embeddings
+- **MinIO** (ports 9000/9001): S3-compatible object storage
+- **vLLM** (port 1234): LLM backend server
+
+### Project Structure
+```
+├── {service_name}/           # Each service follows this pattern:
+│   ├── main.py              # FastAPI app with router includes
+│   ├── routers/             # API endpoints (health.py always present)
+│   ├── models/              # Pydantic models and business logic
+│   ├── utils/               # Service-specific utilities
+│   ├── Dockerfile           # Container definition
+│   └── example.env          # Environment template
+├── shared_utils/            # Common utilities across services
+│   ├── openai_client.py     # OpenAI/vLLM client wrapper
+│   ├── chroma_client.py     # ChromaDB async client
+│   ├── redis.py             # Redis connection utilities
+│   └── s3_utils.py          # MinIO/S3 operations
+└── helm/                    # Kubernetes deployment
+    └── chat-service/        # Example Helm chart structure
+```
+
+All services use FastAPI with consistent structure: main.py imports routers, routers handle endpoints, models contain business logic.
+
+## Common Commands
+
+### Development with Docker Compose
+```bash
+# Start all services
+docker-compose up -d
+
+# Start with GPU support
+docker-compose -f docker-compose.gpu.yml up -d
+
+# View logs for specific service
+docker-compose logs -f chat_service
+
+# Rebuild and restart service
+docker-compose up -d --build chat_service
+```
+
+### Helm/Kubernetes Operations
+```bash
+# Get help with available commands
+make help
+
+# Install single service (defaults to dev environment)
+make install CHART_NAME=chat-service ENV=staging
+
+# Install all services
+make install-all ENV=prod
+
+# Upgrade service
+make upgrade CHART_NAME=chat-service ENV=prod
+
+# Check service status
+make status CHART_NAME=chat-service
+
+# Port forward to local machine
+make port-forward CHART_NAME=chat-service LOCAL_PORT=8001 REMOTE_PORT=8000
+
+# Lint Helm chart
+make lint CHART_NAME=chat-service
+
+# Uninstall service
+make uninstall CHART_NAME=chat-service
+```
+
+### Environment Configuration
+- Services use `.env` files (see `example.env` in each service directory)
+- Helm uses environment-specific values: `values-{dev,staging,prestaging,prod}.yaml`
+- Default namespace: `omnipdf`
+- Use hyphens (not underscores) in chart names for Kubernetes compliance
+
+## Key Implementation Details
+
+### Service Communication
+- Services communicate via HTTP APIs using shared utilities
+- All services expose `/health` endpoint for monitoring
+- nginx serves as API gateway for external requests
+- Internal service discovery uses container names in Docker Compose
+
+### Data Flow
+1. Files uploaded via nginx (8080) → pdf_processor_service (8000)
+2. pdf_processor_service orchestrates other services as needed
+3. embedder_service chunks text → ChromaDB for vector storage
+4. chat_service queries ChromaDB + LLM for RAG responses
+
+### Shared Utilities
+- `openai_client.py`: Configurable OpenAI-compatible client (works with vLLM)
+- `chroma_client.py`: Async ChromaDB client with environment-based config
+- All shared utilities use environment variables for configuration
+
+## Code Standards
+
+### File Organization
+- Follow the established `models/`, `routers/`, `utils/` structure within services
+- Place cross-service utilities in `shared_utils/`
+- Each service must have a health router
+
+### Code Review Process
+- All changes require peer review before merge
+- Focus on correctness, clarity, structure, and security
+- Use comment labels: `nit:`, `suggest:`, `blocking:`
+- Final review by designated senior reviewer before merge
+
+### Kubernetes Naming
+- Use hyphens (not underscores) in resource names
+- Follow RFC 1123 naming conventions
+- Chart names should match service names with hyphens
+
+## Testing and Quality
+
+### Service Testing
+- Each service should be tested independently
+- Use the `/health` endpoints for basic connectivity testing
+- Helm charts include test connections via `test-connection.yaml`
+
+### Port Assignments (Development Only)
+Development ports are documented in README.md. Production deployments should use proper routing layers (Ingress, Service Mesh, etc.).

Makefile

@@ -0,0 +1,122 @@
+# Makefile for Helm chart management in OmniPDF
+
+# Default namespace and configurable chart
+NAMESPACE ?= omnipdf
+CHART_NAME ?= example-service
+CHART_DIR ?= helm/$(CHART_NAME)
+ENV ?= dev
+VALUES_FILE ?= $(CHART_DIR)/values-$(ENV).yaml
+
+# Default port for port-forwarding (override as needed)
+LOCAL_PORT ?= 8000
+REMOTE_PORT ?= 8000
+
+.PHONY: help install install-all upgrade upgrade-all uninstall uninstall-all lint status port-forward
+
+help:
+	@echo "Makefile commands for Helm chart management:"
+	@echo ""
+	@echo "Single-service commands:"
+	@echo "  make install                Install chart (CHART_NAME, ENV)"
+	@echo "                              e.g. make install CHART_NAME=chat-service ENV=staging"
+	@echo "  make upgrade                Upgrade chart (CHART_NAME, ENV)"
+	@echo "                              e.g. make upgrade CHART_NAME=embedder-service ENV=prod"
+	@echo "  make uninstall              Uninstall chart (CHART_NAME)"
+	@echo "                              e.g. make uninstall CHART_NAME=pdf-processor"
+	@echo "  make lint                   Run helm lint on chart (CHART_NAME)"
+	@echo "                              e.g. make lint CHART_NAME=embedder-service"
+	@echo "  make status                 Show status of Helm release (CHART_NAME)"
+	@echo "                              e.g. make status CHART_NAME=chat-service"
+	@echo "  make port-forward           Port-forward a pod to local machine"
+	@echo "                              e.g. make port-forward CHART_NAME=chat-service LOCAL_PORT=8000 REMOTE_PORT=8000"
+	@echo ""
+	@echo "Multi-service commands:"
+	@echo "  make install-all            Install all charts under ./helm/ (ENV)"
+	@echo "                              e.g. make install-all ENV=prod"
+	@echo "  make upgrade-all            Upgrade all charts under ./helm/ (ENV)"
+	@echo "                              e.g. make upgrade-all ENV=staging"
+	@echo "  make uninstall-all          Uninstall all charts under ./helm/"
+	@echo ""
+	@echo "Environment Variables:"
+	@echo "  ENV                         Environment (dev, staging, prestaging, prod) - defaults to 'dev'"
+	@echo ""
+	@echo "⚠️ IMPORTANT:"
+	@echo "  Avoid underscores (_) in CHART_NAME or release names."
+	@echo "  Use hyphens (-) instead to follow Kubernetes naming conventions (RFC 1123)."
+	@echo "  Example: use chat-service ✅, not chat_service ❌"
+
+## Install a single Helm chart
+install:
+	helm upgrade --install $(CHART_NAME) $(CHART_DIR) \
+		--namespace $(NAMESPACE) \
+		--create-namespace \
+		--values $(VALUES_FILE)
+
+## Upgrade a single Helm chart
+upgrade:
+	helm upgrade $(CHART_NAME) $(CHART_DIR) \
+		--namespace $(NAMESPACE) \
+		--values $(VALUES_FILE)
+
+## Uninstall a single Helm chart
+uninstall:
+	helm uninstall $(CHART_NAME) \
+		--namespace $(NAMESPACE)
+
+## Run lint check on a chart
+lint:
+	helm lint $(CHART_DIR)
+
+## Show release status and pod info
+status:
+	@echo "=== Helm Release Status ==="
+	helm status $(CHART_NAME) -n $(NAMESPACE)
+	@echo ""
+	@echo "=== Pod Status ==="
+	kubectl get pods -n $(NAMESPACE) -l "app.kubernetes.io/name=$(CHART_NAME),app.kubernetes.io/instance=$(CHART_NAME)"
+
+## Install all Helm charts in ./helm/
+install-all:
+	@echo "Installing all Helm charts under ./helm/..."
+	@for dir in helm/*/; do \
+		CHART=$$(basename $$dir); \
+		echo "Installing chart: $$CHART"; \
+		helm upgrade --install $$CHART helm/$$CHART \
+			--namespace $(NAMESPACE) \
+			--create-namespace \
+			--values helm/$$CHART/values-$(ENV).yaml; \
+	done
+
+## Upgrade all Helm charts in ./helm/
+upgrade-all:
+	@echo "Upgrading all Helm charts under ./helm/..."
+	@for dir in helm/*/; do \
+		CHART=$$(basename $$dir); \
+		echo "Upgrading chart: $$CHART"; \
+		helm upgrade --install $$CHART helm/$$CHART \
+			--namespace $(NAMESPACE) \
+			--create-namespace \
+			--values helm/$$CHART/values-$(ENV).yaml; \
+	done
+
+## Uninstall all Helm charts in ./helm/
+uninstall-all:
+	@echo "Uninstalling all Helm charts under ./helm/..."
+	@for dir in helm/*/; do \
+		CHART=$$(basename $$dir); \
+		echo "Uninstalling chart: $$CHART"; \
+		helm uninstall $$CHART \
+			--namespace $(NAMESPACE); \
+	done
+
+## Port-forward a running pod (default 8000:8000)
+port-forward:
+ifeq ($(CHART_NAME),example-service)
+	@echo "ERROR: CHART_NAME must be specified. Example usage:"
+	@echo "  make port-forward CHART_NAME=chat-service LOCAL_PORT=3000 REMOTE_PORT=8000"
+	@exit 1
+else
+	kubectl --namespace $(NAMESPACE) port-forward \
+	  $$(kubectl get pod -n $(NAMESPACE) -l "app.kubernetes.io/name=$(CHART_NAME),app.kubernetes.io/instance=$(CHART_NAME)" -o jsonpath="{.items[0].metadata.name}") \
+	  $(LOCAL_PORT):$(REMOTE_PORT)
+endif

README.md

@@ -23,3 +23,13 @@ The following port mappings are used across the OmniPDF microservices **for deve
 | ChromaDB                  | Temporary in-memory vector store                          | 5100   |
 | S3-Compatible Store       | Object storage (e.g., MinIO S3 API)                       | 9000   |
 | MinIO Console             | MinIO web-based Admin UI                                  | 9001   |
+
+## Development Workflow
+
+This project uses a `Makefile` to simplify common Helm and Kubernetes operations.
+
+To get started, run:
+
+```bash
+make help
+```

docker-compose.yml

@@ -95,7 +95,7 @@ services:
       - "8000"
 
   minio:
-    image: minio/minio
+    image: minio/minio:RELEASE.2024-01-16T16-07-38Z
     container_name: minio
     ports:
       - "9000:9000"   # S3 API