Deployment: Kubernetes Helm chart + admin docs - #1712
Draft
Mtze wants to merge 9 commits into
Draft
Conversation
Add a reusable Helm chart (charts/prompt) to deploy PROMPT on Kubernetes: umbrella + one subchart per phase sharing a prompt-common library chart, and an infrastructure subchart for CloudNativePG, SeaweedFS, Gateway API + cert-manager TLS, and optional in-cluster Keycloak. Every stateful dependency is pluggable between in-cluster and external. Adds a comprehensive admin guide under docs/admin/kubernetes. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
A review of the Helm chart found that a `helm install` could not reach a running state, plus a cluster of credential, data-retention and parity defects. Nothing in CI rendered the chart, so none of it surfaced. Install blockers: - Backends set `runAsNonRoot: true`, but the Go images build on distroless static (the root variant, no `USER`), so the kubelet refused every one of them. The chart now supplies UID/GID 65532 via `global.podSecurity`. - Frontends dropped all capabilities while running stock `nginx` as root on port 80, which crash-loops. Capabilities are now dropped for backends only; the chart targets the `baseline` Pod Security Standard. - `infrastructure.realmConfigMap` was only a boolean gate: the ConfigMap was never read, so the rendered `KeycloakRealmImport` shipped an empty realm and every login would fail with `invalid_client`. Its spec takes a full inline realm representation, which cannot come from values, so the resource is gone and the procedure is documented instead. - `S3_SECRET_KEY` defaulted to empty while `S3_ENDPOINT` pointed at SeaweedFS, which fatals core-server. It is now required, as is `S3_ACCESS_KEY` in external mode. - The default `template` phase pulled an image no workflow builds. Replaced with the `example` phase, which Compose does serve. Credentials and state: - External PostgreSQL fell through to `randAlphaNum` when no password was set, rewriting the secret on every render without restarting pods, and ignored the configured user. Host, user and password are now required, and `lookup` runs only in in-cluster mode. - `helm uninstall` deleted the CloudNativePG cluster and with it the Postgres PVCs, which the docs promised persist. The cluster and role secrets now carry `helm.sh/resource-policy: keep`. - In-cluster Keycloak with external PostgreSQL referenced a secret that is never created; it now fails at render time with an explanation. - `wait-for-db` ran `nc -z` against the pooler, which succeeds long before the database and role exist, so migrations ran against a missing database. It now authenticates with `psql`. - Config and secret changes never reached running pods. Pod templates carry `checksum/appconfig`, `checksum/appsecrets` and, for backends, `checksum/db`; the S3 gateway carries the secret checksum. - `SSL_MODE` was unsettable, so external managed PostgreSQL was dialled with `sslmode=disable`. Added `global.postgresql.sslMode`, defaulting to `require` off-cluster. Parity and hardening: - S3 CORS was gated on the rate-limiting toggle, so disabling the rate limit silently broke every browser upload. Split into `gateway.provider` and `gateway.rateLimiting.enabled`, and dropped the never-rendered `burst`. - Added a NetworkPolicy for the unauthenticated SeaweedFS master, volume and filer, which were reachable from every namespace. - `INTRO_COURSE_HOST` and `DEVOPS_CHALLENGE_HOST` pointed at the apex host although those remotes come from separate deployments; they are now configurable and empty by default. CI now lints the chart, renders every mode combination through kubeconform, and runs `charts/prompt/ci/assert-render.sh`, which asserts the properties above including the negative cases. This is render-time only: it cannot cover what the kubelet decides at container start, so a manual install check on a real cluster is still required before the PR ships. The pre-commit `check-yaml` hook now skips `charts/*/templates/`: Go templates are not YAML, so any chart template edit failed the hook.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
✨ What is the change?
Adds a reusable Helm chart (
charts/prompt) to deploy PROMPT 2.0 on Kubernetes, plus a comprehensive admin guide underdocs/admin/kubernetes/.Chart structure:
prompt-commonlibrary chart (Deployment/Service/HTTPRoute + helpers) so the ~14 near-identical components stay DRY.infrastructuresubchart: CloudNativePGCluster(one logical database per phase, managed roles, PgBouncer pooler, optional WAL + scheduled backups), SeaweedFS object storage, Gateway API + Envoy Gateway with per-listener cert-manager TLS, and optional in-cluster Keycloak (operator + realm import).envFrom) and per-phase DB Secrets mirror the existing.envcontract, so no application code changes are needed.Every stateful dependency is pluggable between an in-cluster bundle (self-contained installs) and an external managed service (production):
postgresql.mode,objectStorage.mode,keycloak.mode.📌 Reason for the change
Enables institutions running Kubernetes to deploy PROMPT alongside the existing Docker Compose path. Defaults favour TUM AET but the chart is built to be reusable by other chairs.
Design hardening folded in from review: Helm
lookupso DB passwords survive upgrades, roles declared on the CNPG cluster, connection pooling, per-listener TLS (no multi-SAN blast radius), Envoy rate-limit gated behind a toggle, S3 CORS + Host-header preservation for presigned URLs, DB-wait init containers.🧪 How to Test
helm lint charts/prompt(passes).helm template prompt charts/promptrenders cleanly; verify counts (15 Deployments, 6 Databases, 3 SeaweedFS StatefulSets, per-listener Gateway).--set global.postgresql.mode=external,--set global.objectStorage.mode=external,--set global.keycloak.mode=in-cluster, and--set <phase>.enabled=false.cd docs && yarn buildsucceeds with the new admin pages.helm install, then verify TLS issuance, login, and a presigned upload (seedocs/admin/kubernetes).✅ PR Checklist
helm lint/helm template/ docsyarn build)🤖 Generated with Claude Code