|
| 1 | +# Application template exposure & ingress wiring |
| 2 | + |
| 3 | +The `application` template provisions a 3-tier app (frontend + backend + |
| 4 | +Postgres) and exposes **only the frontend**, behind oauth2-proxy, on a public |
| 5 | +URL. This doc explains how that URL is wired to your cluster's ingress |
| 6 | +controller and how to configure it. |
| 7 | + |
| 8 | +See also: [credentials.md](credentials.md) (the `cloud-credentials` Secret the |
| 9 | +OIDC client secret is bridged from). |
| 10 | + |
| 11 | +## The exposure chain |
| 12 | + |
| 13 | +For each `Application` instance the kro RGD materializes (see |
| 14 | +[install/templates/application.yaml](../install/templates/application.yaml)): |
| 15 | + |
| 16 | +``` |
| 17 | +public host (fqdn) |
| 18 | + │ |
| 19 | + ▼ |
| 20 | +Ingress ── ingressClassName: <KEDGE_INGRESS_CLASS> (host = expose.fqdn) |
| 21 | + │ |
| 22 | + ▼ |
| 23 | +Service <name>-oauth : 4180 |
| 24 | + │ |
| 25 | + ▼ |
| 26 | +oauth2-proxy ── OIDC gate (authenticated users only) |
| 27 | + │ --redirect-url=https://<fqdn>/oauth2/callback |
| 28 | + ▼ |
| 29 | +Service <name>-frontend : <frontendPort> |
| 30 | +``` |
| 31 | + |
| 32 | +The backend and Postgres tiers are **never** exposed — they are reachable only |
| 33 | +via in-cluster DNS. |
| 34 | + |
| 35 | +There is **no `tls:` block** on the generated Ingress. The default design |
| 36 | +terminates TLS at the edge (Cloudflare Tunnel): the `cloudflare` ingress |
| 37 | +controller reads the Ingress `host` and creates the DNS record + tunnel route |
| 38 | +itself. oauth2-proxy runs with `--reverse-proxy=true --cookie-secure=true` and |
| 39 | +trusts `X-Forwarded-*` from that edge. If you front this with a different |
| 40 | +controller you own TLS/DNS yourself — see "Using a non-Cloudflare controller". |
| 41 | + |
| 42 | +## The two knobs |
| 43 | + |
| 44 | +Both are environment variables on the **provider serve** container. |
| 45 | + |
| 46 | +| Env var | Default | What it does | |
| 47 | +|---|---|---| |
| 48 | +| `KEDGE_INGRESS_CLASS` | `cloudflare` | Substituted for the reserved `${kedge.ingressClass}` token in a Template's `backendConfig` before the kro RGD is authored. Ends up as the Ingress `spec.ingressClassName`. kro never sees the token — substitution happens in [backend/kro/rgd.go](../backend/kro/rgd.go) (`substituteTokens`). | |
| 49 | +| `KEDGE_APP_BASE_DOMAIN` | _(unset)_ | The DNS zone apps are served under, e.g. `apps.example.com`. **Also gates the Application instance controller** — see below. | |
| 50 | + |
| 51 | +### `KEDGE_APP_BASE_DOMAIN` gates the whole feature |
| 52 | + |
| 53 | +The Application instance controller is **opt-in**. It starts only when BOTH |
| 54 | +`KEDGE_APP_BASE_DOMAIN` and a runtime kubeconfig (`KRO_KUBECONFIG`, or the |
| 55 | +in-cluster runtime) are present |
| 56 | +([application_controller.go:37](../application_controller.go#L37)). Without the |
| 57 | +base domain it logs `application controller: disabled` and Application |
| 58 | +instances never get an fqdn. |
| 59 | + |
| 60 | +The controller computes the public hostname as |
| 61 | +([apps/host.go:58](../apps/host.go#L58)): |
| 62 | + |
| 63 | +``` |
| 64 | +<hostnamePrefix | name>-<tenantHash>.<KEDGE_APP_BASE_DOMAIN> |
| 65 | +``` |
| 66 | + |
| 67 | +and stamps it onto `spec.expose.fqdn`. The RGD then reads |
| 68 | +`${schema.spec.expose.fqdn}` for the Ingress `host`, the oauth2-proxy |
| 69 | +`--redirect-url`, and the reported `status.url`. The tenant must NOT set `fqdn` |
| 70 | +or `credentialsSecretName` by hand — the controller owns both. |
| 71 | + |
| 72 | +So to turn on app exposure you must set **`KEDGE_APP_BASE_DOMAIN`**; |
| 73 | +`KEDGE_INGRESS_CLASS` only changes *which* controller fulfils the Ingress. |
| 74 | + |
| 75 | +A wildcard DNS record `*.<KEDGE_APP_BASE_DOMAIN>` (or per-app records created by |
| 76 | +your controller, as Cloudflare Tunnel does) must resolve to your ingress edge. |
| 77 | + |
| 78 | +## Configuring it — by deploy mode |
| 79 | + |
| 80 | +### Legacy chart mode (`operator.enabled=false`) |
| 81 | + |
| 82 | +Set the two values; the chart's |
| 83 | +[deployment.yaml](../deploy/chart/templates/deployment.yaml) renders them onto |
| 84 | +the serve container as `KEDGE_APP_BASE_DOMAIN` / `KEDGE_INGRESS_CLASS`: |
| 85 | + |
| 86 | +```yaml |
| 87 | +# values.yaml |
| 88 | +application: |
| 89 | + baseDomain: apps.example.com # empty → feature disabled |
| 90 | + ingressClass: cloudflare # swap to your controller's class |
| 91 | +``` |
| 92 | +
|
| 93 | +```sh |
| 94 | +helm upgrade --install infrastructure \ |
| 95 | + oci://ghcr.io/faroshq/charts/kedge-infrastructure-provider --version 0.0.13 \ |
| 96 | + --set application.baseDomain=apps.example.com \ |
| 97 | + --set application.ingressClass=nginx \ |
| 98 | + ... |
| 99 | +``` |
| 100 | + |
| 101 | +### Operator mode (`operator.enabled=true`) |
| 102 | + |
| 103 | +**This is the mode the production install runs in.** In operator mode the serve |
| 104 | +Deployment is built in code by `EnsureProviderServe` |
| 105 | +([operator/serve.go](../operator/serve.go)), not by the chart's |
| 106 | +`deployment.yaml`, so the exposure config travels through the |
| 107 | +`InfrastructureProvider` CR. Set the `operator.application.*` values; the chart's |
| 108 | +[operator-config.yaml](../deploy/chart/templates/operator-config.yaml) renders |
| 109 | +them into `spec.application` on the CR, and the operator stamps the two env vars |
| 110 | +onto the serve container: |
| 111 | + |
| 112 | +```yaml |
| 113 | +# values.yaml |
| 114 | +operator: |
| 115 | + enabled: true |
| 116 | + application: |
| 117 | + baseDomain: apps.example.com # empty → feature disabled |
| 118 | + ingressClass: cloudflare # empty → in-binary default "cloudflare" |
| 119 | +``` |
| 120 | +
|
| 121 | +```sh |
| 122 | +helm upgrade --install infrastructure \ |
| 123 | + oci://ghcr.io/faroshq/charts/kedge-infrastructure-provider --version 0.0.13 \ |
| 124 | + -n kedge-prod-infrastructure-operator --create-namespace \ |
| 125 | + --set operator.enabled=true \ |
| 126 | + --set-file operator.providerKubeconfig=./kedge/provider-infrastructure.kubeconfig \ |
| 127 | + --set operator.application.baseDomain=apps.example.com \ |
| 128 | + --set operator.application.ingressClass=cloudflare \ |
| 129 | + ... |
| 130 | +``` |
| 131 | + |
| 132 | +Equivalently, edit the CR directly: |
| 133 | + |
| 134 | +```yaml |
| 135 | +apiVersion: infrastructure.kedge.faros.sh/v1alpha1 |
| 136 | +kind: InfrastructureProvider |
| 137 | +spec: |
| 138 | + application: |
| 139 | + baseDomain: apps.example.com |
| 140 | + ingressClass: cloudflare |
| 141 | +``` |
| 142 | +
|
| 143 | +The operator re-reconciles the serve Deployment on the next pass (≤2 min), which |
| 144 | +sets `KEDGE_APP_BASE_DOMAIN` / `KEDGE_INGRESS_CLASS` and rolls the pods. Leaving |
| 145 | +`baseDomain` empty keeps the Application controller disabled; leaving |
| 146 | +`ingressClass` empty falls back to the in-binary `cloudflare` default. |
| 147 | + |
| 148 | +> Do **not** `kubectl set env` the serve Deployment by hand — the operator |
| 149 | +> overwrites its `spec` on every reconcile (`existing.Spec = want.Spec`), so the |
| 150 | +> change is reverted. Configure it on the CR instead. |
| 151 | + |
| 152 | +## Writing exposure into your own templates |
| 153 | + |
| 154 | +Any Template's `backendConfig` can defer the controller choice to platform |
| 155 | +config by writing the reserved token in the Ingress: |
| 156 | + |
| 157 | +```yaml |
| 158 | +- id: ingress |
| 159 | + template: |
| 160 | + apiVersion: networking.k8s.io/v1 |
| 161 | + kind: Ingress |
| 162 | + spec: |
| 163 | + ingressClassName: ${kedge.ingressClass} # ← substituted at RGD-author time |
| 164 | + rules: |
| 165 | + - host: ${schema.spec.expose.fqdn} # ← stamped by the controller |
| 166 | + ... |
| 167 | +``` |
| 168 | + |
| 169 | +`${kedge.ingressClass}` is the only `${kedge.*}` token today. It is replaced by |
| 170 | +a plain string substitution on the backendConfig JSON before kro parses it, so |
| 171 | +it is safe to use anywhere a class name is valid. kro's own `${...}` references |
| 172 | +(`${schema.spec.*}`, `${someResource.metadata.name}`) pass through untouched. |
| 173 | + |
| 174 | +## Using a non-Cloudflare controller |
| 175 | + |
| 176 | +`KEDGE_INGRESS_CLASS=nginx` (or `traefik`, etc.) only changes |
| 177 | +`spec.ingressClassName`. The shipped `application` template assumes edge TLS and |
| 178 | +edge-managed DNS (the Cloudflare model), so for a generic in-cluster controller |
| 179 | +you also need to: |
| 180 | + |
| 181 | +- Provide DNS for `*.<baseDomain>` → your ingress LB. |
| 182 | +- Terminate TLS yourself — add a `tls:` block to the Ingress (e.g. cert-manager |
| 183 | + with an `cert-manager.io/cluster-issuer` annotation). The shipped template has |
| 184 | + none; fork `install/templates/application.yaml` to add it, or author your own |
| 185 | + template. |
| 186 | + |
| 187 | +oauth2-proxy already trusts forwarded headers (`--reverse-proxy=true`) and sets |
| 188 | +secure cookies, so it works behind any TLS-terminating ingress. |
| 189 | +</content> |
0 commit comments