Caddy TLS overlay for HTTPS (Option B from #64) — validated end-to-end, proposed upstreaming plan

[alexskr]

Context

This issue tracks the implementation and upstreaming plan for Option B from #64 — opt-in HTTPS via an external/sidecar TLS termination layer, rather than baking HTTPS into the dev compose by default.

The approach is now validated end-to-end against a staging instance running the same topology as production:

• Dev default stays plain HTTP (docker compose up) — the "keep dev simple" preference is preserved.
• HTTPS is an opt-in overlay (docker-compose.tls.yml) applied on top of the same base compose. Two new commands cover the dev path:

make dev-certs      # one-time per machine: mkcert install + locally-trusted cert
make dev-https-up   # bring the stack up behind Caddy/TLS

• The same overlay covers single-host prod (auto-letsencrypt). LB-fronted prod (Stanford LB, AWS ELB, etc.) just doesn't include the overlay.

Implementation lives at https://github.com/alexskr/webprotege-deploy (squashed into main on the fork).

What changed, by repo

webprotege-deploy — the overlay

• caddy/Caddyfile — single config; env vars select mkcert (dev) vs auto-letsencrypt (single-host prod).
• docker-compose.tls.yml — Caddy sidecar; strips the nginx host-port binding; redirects JVM back-channel Keycloak calls to the internal bridge (see architecture note).
• bin/dev-setup-certs — mkcert installer + cert generator.
• Makefile — dev-up / dev-certs / dev-https-up / prod-up convenience targets.
• Base-compose fixes that any externally-TLS-terminated deploy needs (independent of dev/prod):
  • Dropped hostname: ${SERVER_HOST} from webprotege-nginx — it shadowed public DNS on the user-defined bridge, breaking back-channel HTTPS to the public hostname.
  • KC_HOSTNAME set as a full URL (incl. scheme) — bare hostname caused Keycloak to emit http:// URLs in discovery on the internal bridge.
  • Bind-mounted Tomcat server.xml with a RemoteIpValve on webprotege-gwt-ui-server so the legacy Keycloak adapter honors X-Forwarded-Proto.

webprotege-nginx — https://github.com/alexskr/webprotege-nginx

• Trust an upstream proxy's X-Forwarded-Proto instead of hardcoding $scheme. Backward-compatible: falls back to $scheme when the header is absent (i.e. dev).
• Needed by any deployment with TLS terminated in front of it.

webprotege-keycloak — https://github.com/alexskr/webprotege-keycloak

• Entrypoint honors a PUBLIC_SCHEME variable so the realm frontendUrl, the webprotege client's redirect URIs, and the OIDC discovery doc all advertise the correct scheme.
• Relaxes the master-realm sslRequired to NONE in HTTP dev so the admin console stays reachable (left at the stock external when PUBLIC_SCHEME=https).

webprotege-gwt-api-gateway — https://github.com/alexskr/webprotege-gwt-api-gateway/tree/feature/remove-hardcoded-url-defaults

• Drops the deploy-specific Keycloak URLs hardcoded into BOOT-INF/classes/application.yaml, so the gateway takes its URLs from the environment.
• This is what lets the overlay keep all JVM→Keycloak back-channel calls on the internal bridge (see below).
• Already tracked upstream as Remove hardcoded webprotege-local.edu from application.yaml webprotege-gwt-api-gateway#45.

Architecture note — front/back-channel split

The piece that keeps the dev setup simple:

The base compose already splits Keycloak front-channel from back-channel for the api-gateway:

• Front-channel (browser-facing): issuer-uri, authorization-uri → public ${PUBLIC_SCHEME}://${SERVER_HOST}/keycloak/...
• Back-channel (server-to-server): token-uri, jwk-set-uri, user-info-uri → internal http://webprotege-keycloak:8080/keycloak/...

The TLS overlay extends that so all JVM→Keycloak back-channel traffic stays on the internal bridge:

• webprotege-gwt-api-gateway: blanks SPRING_SECURITY_OAUTH2_*_ISSUER_URI so Spring skips the boot-time OIDC discovery fetch.
• webprotege-gwt-ui-server: KEYCLOAK_AUTH_URL → bridge.
• webprotege-user-management-service / webprotege-event-history-service: webprotege.keycloak.serverUrl → bridge.

Net effect: browser flows stay fully HTTPS through Caddy, but no JVM service does back-channel HTTPS — so nothing needs the dev cert in its truststore. JWT signature validation (via the bridge JWKS) is unchanged; only issuer-claim matching is dropped in dev, and only in dev (prod with a real LE cert keeps full validation).

Proposed upstreaming plan

The fork patches are bridging dependencies; I'd like to land them upstream and drop the fork pins in the deploy overlay. Proposed order:

1. webprotege-nginx: trust X-Forwarded-Proto (small, backward-compatible, unblocks all external-TLS deployments).
2. webprotege-keycloak: PUBLIC_SCHEME support + master-realm sslRequired relax in HTTP dev.
3. webprotege-gwt-api-gateway: remove the hardcoded application.yaml URL defaults (Remove hardcoded webprotege-local.edu from application.yaml webprotege-gwt-api-gateway#45).
4. webprotege-deploy: the Caddy overlay + tooling (depends on 1–3 being released so the overlay can point at protegeproject/* tags).

Open questions for the team

1. Order / OK to proceed. Does the order above work, and any objections to me opening these PRs now?
2. webprotege-nginx patch shape. Do you prefer the fix as a forked-image change, or as a mountable nginx config snippet so the stock image stays untouched?
3. Reviewer ownership for the api-gateway PR (Remove hardcoded webprotege-local.edu from application.yaml webprotege-gwt-api-gateway#45)?

Notes

• The new-account NPE seen on first load right after registration (AccessToken$Access.getRoles() returning null; a page reload works) is independent of the TLS work and is being looked at separately.

Refs #64.