|
| 1 | +--- |
| 2 | +title: Architecture |
| 3 | +nav_title: Architecture |
| 4 | +--- |
| 5 | + |
| 6 | +This page explains how `serge` is put together — its components, the execution |
| 7 | +backends that run the LLM work, and the two deployment flavors (**Docker** and |
| 8 | +**Kubernetes**). For the step-by-step review lifecycle see |
| 9 | +[How it works](how-it-works.md); for the trust model see |
| 10 | +[Security](security.md) and [Security architecture](security-architecture.md). |
| 11 | + |
| 12 | +## The big picture |
| 13 | + |
| 14 | +serge is an **orchestrator**. Its long-lived process owns the durable state and |
| 15 | +the credentials — the SQLite job store, the GitHub App private key, the OAuth / |
| 16 | +session secrets, and LLM keys — and decides *when* a review or task runs and |
| 17 | +*what* gets published. The actual LLM loop (fetch diff → prompt → tool calls → |
| 18 | +draft) can either run **in that same process** or be pushed out into a |
| 19 | +short-lived **runner pod**, depending on how you deploy. Either way, the model |
| 20 | +only ever *proposes* output; serge validates it and publishes. |
| 21 | + |
| 22 | +```text |
| 23 | + ┌───────────────────────────────────────────┐ |
| 24 | + GitHub ── comment ──▶│ serge (orchestrator: reviewbot-web/-app) │ |
| 25 | + / webhook / OIDC │ • trigger gate • SQLite job store │ |
| 26 | + │ • GitHub App auth • provider configs │ |
| 27 | + │ • draft/publish • per-job secrets │ |
| 28 | + └───────────────┬─────────────────────────────┘ |
| 29 | + │ run the LLM loop … |
| 30 | + ┌────────────────┼─────────────────────┐ |
| 31 | + ▼ ▼ ▼ |
| 32 | + in-process docker runner kubernetes pod |
| 33 | + (same process) (docker run) (one Job / request) |
| 34 | +``` |
| 35 | + |
| 36 | +## Entry points |
| 37 | + |
| 38 | +serge ships three entry points around one shared core (`reviewbot/reviewer.py`): |
| 39 | + |
| 40 | +| Command | Role | Where it runs | |
| 41 | +| ------- | ---- | ------------- | |
| 42 | +| `reviewbot-action` | [GitHub Action](github-action.md) runner; reads `GITHUB_EVENT_PATH` | GitHub's ephemeral runner | |
| 43 | +| `reviewbot-app` | Flask [GitHub App webhook](github-app.md) server | Your server | |
| 44 | +| `reviewbot-web` | FastAPI [staged web app](web-app.md) + `POST /webhook` + write-capable [`POST /tasks`](tasks-flow.md) | Your server | |
| 45 | + |
| 46 | +The **Action** is self-contained: it runs on GitHub's disposable runner with a |
| 47 | +job-scoped token, so the runner itself is the isolation boundary. The |
| 48 | +**webhook** and **web app** are the persistent, hosted deployments — everything |
| 49 | +below about execution backends and deployment flavors is about those. |
| 50 | + |
| 51 | +## Core components |
| 52 | + |
| 53 | +| Component | File | Responsibility | |
| 54 | +| --------- | ---- | -------------- | |
| 55 | +| Reviewer core | `reviewbot/reviewer.py` | Annotate diff, prompt the LLM, validate inline comments against real diff positions, publish or draft | |
| 56 | +| Trigger gate | `reviewbot/triggers.py` | Decide whether an event should start a review (event type, author association, trigger phrase, PR state) | |
| 57 | +| Tools + sandbox | `reviewbot/tools.py`, `sandbox.py` | Read-only repo browse tools and repo helper tools, each executing subprocess wrapped in bubblewrap | |
| 58 | +| Clone cache | `reviewbot/clone_cache.py` | Shallow checkout of the PR head with an upstream `.ai/` overlay; standalone (self-contained) clones for pods | |
| 59 | +| Store | `reviewbot/store.py` | Embedded SQLite: jobs, drafts, provider configs, cached installation tokens | |
| 60 | +| GitHub auth | `reviewbot/github_auth.py` | Mint short-lived installation tokens from the App private key | |
| 61 | +| Launchers | `reviewbot/launcher.py`, `k8s_sandbox.py` | Build the runner spec and start a docker container or a Kubernetes Job | |
| 62 | +| Runners | `reviewbot/task_runner.py`, `review_runner.py` | The in-pod entry points that do the whole loop and stream results back | |
| 63 | + |
| 64 | +## Execution backends |
| 65 | + |
| 66 | +serge has **two independent backend selectors** — one for write-capable |
| 67 | +[tasks](tasks-flow.md) and one for PR reviews. Both default to `inprocess`, so a |
| 68 | +fresh install runs the entire loop inside serge's own thread pool with **no |
| 69 | +Docker and no Kubernetes**. |
| 70 | + |
| 71 | +| Setting | Env var | Values | Default | |
| 72 | +| ------- | ------- | ------ | ------- | |
| 73 | +| Task execution | `TASK_EXECUTION` | `inprocess` \| `docker` \| `kubernetes` | `inprocess` | |
| 74 | +| Review execution | `REVIEW_EXECUTION` | `inprocess` \| `docker` \| `kubernetes` | `inprocess` | |
| 75 | + |
| 76 | +- **`inprocess`** — the loop runs in a worker thread inside serge. Simplest; |
| 77 | + this is the legacy path and the default. |
| 78 | +- **`docker`** — serge does `docker run` on a runner image per request, blocks |
| 79 | + on the worker thread, and finalizes inline. |
| 80 | +- **`kubernetes`** — serge creates one short-lived Kubernetes **Job per |
| 81 | + request**, watches it out-of-band, and finalizes when the runner calls back. |
| 82 | + |
| 83 | +The `docker` and `kubernetes` backends share the **same runner image, spec |
| 84 | +format, and HTTP callback** — they differ only in how the runner process is |
| 85 | +started. Inline PR-comment follow-ups always stay in-process regardless of the |
| 86 | +setting. |
| 87 | + |
| 88 | +> Kubernetes is never mandatory. The `kubernetes` client is imported lazily, the |
| 89 | +> pod watcher no-ops when there are no pod tasks, and every k8s-only feature is |
| 90 | +> gated — so a Docker or in-process deployment never needs the cluster libraries. |
| 91 | +
|
| 92 | +### Sandboxing vs. execution backend |
| 93 | + |
| 94 | +Two other knobs control how *subprocesses within a review/task* are sandboxed — |
| 95 | +they are separate from the execution backend above: |
| 96 | + |
| 97 | +- **`HELPER_SANDBOX`** (`require` \| `auto` \| `off`) wraps the read-only helper |
| 98 | + tools, the `.ai/context-script`, and the pip-install hook in **bubblewrap**. |
| 99 | + Production sets `require`. See [Security architecture](security-architecture.md). |
| 100 | +- **`TASK_SANDBOX_BACKEND`** (`bwrap` \| `docker` \| `kubernetes` \| `auto`) |
| 101 | + wraps the in-loop [normalize](tasks-normalize.md) command when tasks run |
| 102 | + in-process. |
| 103 | + |
| 104 | +Inside a **runner pod** both are forced `off`: the ephemeral pod plus its |
| 105 | +allowlist-egress firewall *is* the isolation boundary, so there is no nested |
| 106 | +sandbox. |
| 107 | + |
| 108 | +## Pods (the Kubernetes backend) |
| 109 | + |
| 110 | +When `TASK_EXECUTION=kubernetes` (and, for reviews, `reviewPods=true` → |
| 111 | +`REVIEW_EXECUTION=kubernetes`), serge becomes a pure orchestrator and all LLM |
| 112 | +work happens in per-request pods. |
| 113 | + |
| 114 | +### Lifecycle of a pod |
| 115 | + |
| 116 | +```text |
| 117 | + request ─▶ serge mints installation token + callback token |
| 118 | + │ |
| 119 | + ├─ build spec (request + GitHub token + LLM settings + callback) |
| 120 | + ├─ create Job (batch/v1, backoffLimit 0, ttlSecondsAfterFinished 300) |
| 121 | + ├─ create per-job Secret (task.json, ownerRef→Job) → returns immediately |
| 122 | + │ |
| 123 | + ▼ |
| 124 | + runner pod: read /etc/serge/task.json |
| 125 | + ├─ clone repo (standalone, self-contained) via the egress proxy |
| 126 | + ├─ run the LLM loop (tools, in-process normalize for tasks) |
| 127 | + ├─ POST each event ──────────────▶ serge callback (SSE relay) |
| 128 | + └─ POST terminal result ──────────▶ serge: finalize + publish/draft |
| 129 | + │ |
| 130 | + serge Job watcher (every 5s, k8s only): reconciles crashes, reaps the Job |
| 131 | +``` |
| 132 | + |
| 133 | +1. **Launch (non-blocking).** serge mints a short-lived GitHub installation |
| 134 | + token and a per-job callback bearer token, serializes them plus the request |
| 135 | + and resolved LLM settings into a **spec**, creates the Job, then the Secret, |
| 136 | + and returns without parking a thread. |
| 137 | +2. **Per-job Secret.** The spec is projected read-only into the pod at |
| 138 | + `/etc/serge/task.json`. The Secret carries an `ownerReference` back to the |
| 139 | + Job, so it is garbage-collected with the Job even on a crash. |
| 140 | +3. **The runner pod** (`reviewbot-task-runner`) dispatches on `request_type`: |
| 141 | + `review` → `review_runner.run`, otherwise `task_runner.run`. It rebuilds a |
| 142 | + `Config` that has **no App private key and no web-auth env**, clones into its |
| 143 | + own `emptyDir` (no shared PVC), runs the loop, and POSTs events and a terminal |
| 144 | + payload back over the authenticated callback. A task runner *publishes* its PR |
| 145 | + itself; a review runner only ships an encoded **draft** — serge publishes on |
| 146 | + human approval (or auto-publishes for webhook reviews). |
| 147 | +4. **Watcher + idempotent finalize.** An in-process asyncio watcher polls the |
| 148 | + Job status every 5s (only when there are Kubernetes pod tasks). Finalization |
| 149 | + is guarded so it happens **exactly once** whether it's triggered by the happy- |
| 150 | + path callback or by the watcher detecting a crashed/timed-out pod. |
| 151 | +5. **Reaping.** serge deletes the Job (and its Secret) on completion; |
| 152 | + `ttlSecondsAfterFinished` is the backstop if a delete is missed. |
| 153 | + |
| 154 | +### The Pods page |
| 155 | + |
| 156 | +The web app exposes a **Pods** admin page (`/admin/pods`, admin-gated) that lists |
| 157 | +running pods — joining live Kubernetes pods with serge's tracked tasks, so both |
| 158 | +orphaned pods (after a serge restart) and just-launched ones appear — and offers |
| 159 | +a **kill** action (`POST /admin/pods/kill`) as the manual stop for a runaway Job. |
| 160 | + |
| 161 | +### Network isolation |
| 162 | + |
| 163 | +A runner pod holds a live GitHub token and LLM key **while running arbitrary |
| 164 | +repository build code** (e.g. `make style`). Its egress is therefore locked down |
| 165 | +by two Kubernetes `NetworkPolicy` objects and a forward proxy: |
| 166 | + |
| 167 | +- The **task-pod NetworkPolicy** denies all egress except to the `serge-egress` |
| 168 | + proxy, kube-dns, and serge's own callback Service. |
| 169 | +- **`serge-egress`** is a small tinyproxy Deployment (built in-house) that |
| 170 | + `CONNECT`-allowlists only GitHub + the configured LLM hosts and denies |
| 171 | + everything else. serge injects it as the pod's `HTTPS_PROXY`/`HTTP_PROXY`; |
| 172 | + `NO_PROXY` keeps the in-cluster callback off the proxy. serge keeps the |
| 173 | + proxy's allowlist in sync with the configured LLM providers. |
| 174 | + |
| 175 | +The residual risks (DNS egress, token tunneling through legitimate GitHub |
| 176 | +requests) are documented in [Security](security.md#per-task-pod-network-firewall). |
| 177 | + |
| 178 | +## Deployment flavors |
| 179 | + |
| 180 | +serge runs in two shapes. Pick based on scale and isolation needs. |
| 181 | + |
| 182 | +### Flavor 1 — Docker |
| 183 | + |
| 184 | +A single container running `reviewbot-web` (`Dockerfile` at the repo root: |
| 185 | +Python 3.11 + bubblewrap + the `[web]` extra, uvicorn on `$PORT`). The embedded |
| 186 | +SQLite store lives on a mounted volume. This is the simplest hosted deployment — |
| 187 | +the one that maps directly to the original EC2 host. |
| 188 | + |
| 189 | +```text |
| 190 | + ┌──────────────────────── docker host / VM ────────────────────────┐ |
| 191 | + │ serge container (reviewbot-web) │ |
| 192 | + │ • uvicorn :8080 • bubblewrap sandbox (HELPER_SANDBOX) │ |
| 193 | + │ • jobs.db on a mounted volume │ |
| 194 | + │ │ |
| 195 | + │ reviews/tasks: TASK_EXECUTION=inprocess (or =docker for the │ |
| 196 | + │ normalize/runner container on the same docker socket) │ |
| 197 | + └───────────────────────────────────────────────────────────────────┘ |
| 198 | +``` |
| 199 | + |
| 200 | +- **Backends:** `inprocess` by default. Optionally `docker` for tasks/reviews |
| 201 | + (docker-out-of-docker via a mounted socket) or `TASK_SANDBOX_BACKEND=docker` |
| 202 | + for the normalize step. |
| 203 | +- **Isolation:** per-subprocess bubblewrap (`HELPER_SANDBOX=require`) is the |
| 204 | + boundary between fork code and host secrets. |
| 205 | +- **State:** one container, one SQLite file on a volume. No orchestration. |
| 206 | +- **Good for:** a single team, one or a few repositories, straightforward ops. |
| 207 | + |
| 208 | +### Flavor 2 — Kubernetes (Helm) |
| 209 | + |
| 210 | +The [Helm chart](https://github.com/huggingface/serge/tree/main/deploy) in |
| 211 | +`deploy/helm` packages serge for a cluster. Because SQLite is a single writer, |
| 212 | +the Deployment runs **one replica** with a `Recreate` strategy and an RWO PVC. |
| 213 | +When the pod-per-task/-review backend is enabled, the chart also renders the |
| 214 | +egress proxy, the network policies, and the RBAC serge needs to manage Jobs. |
| 215 | + |
| 216 | +The diagram below is generated straight from the chart with |
| 217 | +[KubeDiagrams](https://github.com/philippemerle/KubeDiagrams) (see |
| 218 | +[Regenerating the diagram](#regenerating-the-diagram)): |
| 219 | + |
| 220 | +![serge Kubernetes architecture]({{ "/assets/architecture-k8s.png" | relative_url }}) |
| 221 | + |
| 222 | +What the chart creates: |
| 223 | + |
| 224 | +| Resource | Purpose | |
| 225 | +| -------- | ------- | |
| 226 | +| `Deployment serge` (1 replica, `Recreate`) | The orchestrator (`reviewbot-web`) | |
| 227 | +| `Service serge` + `Ingress` | In-cluster + external access (ALB in prod) | |
| 228 | +| `PVC serge-data` + `ConfigMap serge` | SQLite store + non-secret runtime env | |
| 229 | +| `ServiceAccount` + `Role`/`RoleBinding serge-task-runner` | Lets serge create Jobs + per-job Secrets and read the egress Service | |
| 230 | +| `Deployment/Service/ConfigMap serge-egress` | The allowlisting forward proxy for runner pods | |
| 231 | +| `NetworkPolicy serge-egress` | Proxy may reach :443 + kube-dns only | |
| 232 | +| `NetworkPolicy serge-task-pod` | Runner pods may reach only the proxy, kube-dns, and serge's callback | |
| 233 | + |
| 234 | +Runner pods themselves are **not** in the chart — serge creates them at runtime, |
| 235 | +one per request, as `batch/v1` Jobs. |
| 236 | + |
| 237 | +- **Backends:** `taskExecution.kubernetes.enabled=true` turns on task pods; |
| 238 | + `reviewPods=true` also runs reviews in pods. |
| 239 | +- **Isolation:** the ephemeral pod + the egress allowlist. Secrets never sit |
| 240 | + next to arbitrary fork code on the long-lived host. |
| 241 | +- **Good for:** many repositories, heavier toolchains (the runner image carries |
| 242 | + the repo's build/lint stack), and horizontal isolation per request. |
| 243 | + |
| 244 | +See [Deploying serge](https://github.com/huggingface/serge/tree/main/deploy) and |
| 245 | +the production values in `deploy/helm/env/prod.yaml`. |
| 246 | + |
| 247 | +#### Regenerating the diagram |
| 248 | + |
| 249 | +The Kubernetes diagram is reproducible from the chart: |
| 250 | + |
| 251 | +```bash |
| 252 | +pip install KubeDiagrams # provides `kube-diagrams` |
| 253 | +brew install graphviz helm # `dot` + `helm` |
| 254 | + |
| 255 | +deploy/scripts/gen-arch-diagram.sh # → docs/assets/architecture-k8s.png |
| 256 | +``` |
| 257 | + |
| 258 | +The script runs `helm template` with the pod backend enabled and pipes the |
| 259 | +manifests through `kube-diagrams`. |
| 260 | + |
| 261 | +## Data and persistence |
| 262 | + |
| 263 | +| Data | Where | Notes | |
| 264 | +| ---- | ----- | ----- | |
| 265 | +| Jobs, drafts, provider configs, cached tokens | SQLite (`WEB_STORE_PATH`) | Single writer → 1 replica; on a PVC/volume so it survives restarts | |
| 266 | +| Clone cache | `WEB_CLONE_CACHE_DIR` | Shallow bare-clone cache; point at durable storage in production | |
| 267 | +| Runner pod checkout | pod `emptyDir` | Self-contained standalone clone; evaporates with the pod | |
| 268 | +| Secrets (App key, OAuth, LLM keys) | env / Kubernetes Secret | Never enter a sandbox or runner pod's git remote | |
| 269 | + |
| 270 | +## Related pages |
| 271 | + |
| 272 | +- [How it works](how-it-works.md) — the review lifecycle step by step. |
| 273 | +- [Tasks flow](tasks-flow.md) — the write-capable `POST /tasks` path. |
| 274 | +- [Configuration](configuration.md) — every env var, including the backend and |
| 275 | + pod settings. |
| 276 | +- [Security](security.md) / [Security architecture](security-architecture.md) — |
| 277 | + the trust model and the sandbox/egress boundaries. |
0 commit comments