Skip to content

Commit d67663c

Browse files
mjudeikisclaude
andcommitted
feat(infra): sandbox-runner images as schema defaults (drop ${kedge.*} tokens)
The sandbox-runner template was the lone outlier using ${kedge.*} env-substitution tokens for its container images (Phase 2a) — every other template declares configurable images as spec.schema fields with sane defaults and hardcodes fixed sidecar images. That outlier broke in practice: an unset KEDGE_SANDBOX_RUNNER_IMAGE substituted to "" and kro authored a Deployment/Job with `image: ""` (Required value). Make sandbox-runner follow the same convention as simple-webapp / database / redis / application: - spec.runnerImage is a schema field with a sane default (ghcr.io/faroshq/kedge-sandbox-runner:dev); the runner Deployment uses ${schema.spec.runnerImage}. Overridable per instance. - the control-token Job image is hardcoded bitnami/kubectl, like every other template's control job (tokenGeneratorImage field dropped). Remove all the now-dead image-token machinery: - kro backend: drop the sandbox image tokens (only the exposure Gateway tokens remain). - infra chart: drop KEDGE_SANDBOX_*_IMAGE env, the install guard, and the sandboxRunner values block; operator passthrough. - App Studio: sandboxRunnerValues supplies only {projectRef}; drop the image env helpers, chart guard/env/values, README rows. Document the convention so this doesn't recur: - new providers/infrastructure/docs/template-conventions.md (when to use a schema default vs a hardcoded literal vs a ${kedge.*} token, with the failure modes that motivated it). - AGENTS.md §9 guardrail pointing at it. Also: dev postgres images default to Google's Docker Hub mirror (mirror.gcr.io/library/postgres:16-alpine) — Docker Hub has been dropping pulls with "unexpected EOF". Preview routing is unchanged (it was already schema-based on main). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent 5efe47c commit d67663c

18 files changed

Lines changed: 120 additions & 184 deletions

File tree

AGENTS.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -390,6 +390,14 @@ whole point.
390390
build/test and `go.work` awareness; they are not in the root `./...`.
391391
- Before merging: `make verify` is the full gate
392392
(boilerplate + codegen + vet + lint + build + test).
393+
- **Infrastructure templates declare configurable inputs (container images,
394+
versions, sizes) as `spec.schema` fields with sane defaults** — never via
395+
`${kedge.*}` env-substitution tokens. Fixed sidecar images (e.g. the
396+
control-token `kubectl` job) are hardcoded literals. `${kedge.*}` tokens are
397+
reserved for genuinely platform-global values with no universal default (only
398+
the exposure Gateway parent today). A missing env must never be able to
399+
produce an empty/invalid field. See
400+
[`providers/infrastructure/docs/template-conventions.md`](providers/infrastructure/docs/template-conventions.md).
393401

394402
---
395403

Makefile

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -678,7 +678,9 @@ KUERY_RUNTIME_KUBECONFIG ?= $(KCP_DATA_DIR)/kuery-runtime.kubeconfig
678678
# …) diverges from SQLite and passing on SQLite has shipped real query bugs.
679679
# kuery-db-up starts a throwaway container matching KUERY_DEV_DATABASE_URL.
680680
KUERY_POSTGRES_CONTAINER ?= kedge-kuery-postgres
681-
KUERY_POSTGRES_IMAGE ?= postgres:16-alpine
681+
# Pulled from Google's Docker Hub mirror (same official image, more reliable
682+
# pulls — Docker Hub has been dropping them with "unexpected EOF").
683+
KUERY_POSTGRES_IMAGE ?= mirror.gcr.io/library/postgres:16-alpine
682684
KUERY_POSTGRES_PORT ?= 55433
683685
KUERY_POSTGRES_DATA_DIR ?= $(KCP_DATA_DIR)/kuery-postgres
684686
KUERY_POSTGRES_USER ?= kuery
@@ -862,7 +864,7 @@ APP_STUDIO_SANDBOX_RUNNER_IMAGE ?= $(SANDBOX_RUNNER_IMAGE)
862864
APP_STUDIO_SANDBOX_TOKEN_GENERATOR_IMAGE ?= $(SANDBOX_TOKEN_GENERATOR_IMAGE)
863865
APP_STUDIO_DEV_DATABASE_URL ?= postgres://appstudio:appstudio@localhost:55432/appstudio?sslmode=disable
864866
APP_STUDIO_POSTGRES_CONTAINER ?= kedge-app-studio-postgres
865-
APP_STUDIO_POSTGRES_IMAGE ?= postgres:16-alpine
867+
APP_STUDIO_POSTGRES_IMAGE ?= mirror.gcr.io/library/postgres:16-alpine
866868
APP_STUDIO_POSTGRES_PORT ?= 55432
867869
APP_STUDIO_POSTGRES_DATA_DIR ?= $(KCP_DATA_DIR)/app-studio-postgres
868870
APP_STUDIO_POSTGRES_USER ?= appstudio

providers/app-studio/README.md

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,8 +52,6 @@ Environment variables consumed by the binary:
5252
| `KEDGE_HUB_TOKEN` | Bearer token for the heartbeat |
5353
| `KEDGE_PROVIDER_NAME` | CatalogEntry name (default `app-studio`) |
5454
| `KEDGE_PROVIDER_KUBECONFIG` | Provider kubeconfig (kcp front-proxy host + TLS only) |
55-
| `APP_STUDIO_SANDBOX_RUNNER_IMAGE` | Runner image passed to new `SandboxRunner` resources; use an immutable digest outside local development |
56-
| `APP_STUDIO_SANDBOX_TOKEN_GENERATOR_IMAGE` | kubectl-capable token-generator image passed to new `SandboxRunner` resources; use an immutable digest outside local development |
5755
| `APP_STUDIO_PREVIEW_BASE_DOMAIN` | Optional DNS zone for companion Sandbox preview routing. |
5856
| `APP_STUDIO_PREVIEW_HTTPROUTE_PARENT_GATEWAY_NAME` | Gateway resource name to attach each preview `HTTPRoute` to. |
5957
| `APP_STUDIO_PREVIEW_HTTPROUTE_PARENT_GATEWAY_NAMESPACE` | Namespace of the parent Gateway (defaults to `kedge-preview`). |

providers/app-studio/api/deployment_defaults.go

Lines changed: 6 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -67,17 +67,15 @@ func defaultSandboxRunnerBinding(projectName string) aiv1alpha1.ProjectProviderB
6767
}
6868
}
6969

70+
// sandboxRunnerValues are the project-scoped fields App Studio supplies on a
71+
// SandboxRunner binding. The runner image is NOT one of them: the sandbox-runner
72+
// template declares spec.runnerImage as a schema field with a sane default (the
73+
// web-app convention), so App Studio doesn't pass it. See
74+
// providers/infrastructure/docs/template-conventions.md.
7075
func sandboxRunnerValues(projectName string) map[string]any {
71-
values := map[string]any{
76+
return map[string]any{
7277
"projectRef": projectName,
7378
}
74-
if image := sandboxRunnerImage(); image != "" {
75-
values["runnerImage"] = image
76-
}
77-
if image := sandboxTokenGeneratorImage(); image != "" {
78-
values["tokenGeneratorImage"] = image
79-
}
80-
return values
8179
}
8280

8381
func previewHTTPRouteEnabled() bool {
@@ -132,14 +130,6 @@ func previewBackendServicePort() int64 {
132130
return port
133131
}
134132

135-
func sandboxRunnerImage() string {
136-
return envValue("APP_STUDIO_SANDBOX_RUNNER_IMAGE")
137-
}
138-
139-
func sandboxTokenGeneratorImage() string {
140-
return envValue("APP_STUDIO_SANDBOX_TOKEN_GENERATOR_IMAGE")
141-
}
142-
143133
func envValue(key string) string {
144134
return strings.TrimSpace(os.Getenv(key))
145135
}

providers/app-studio/api/deployment_defaults_test.go

Lines changed: 9 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -12,31 +12,19 @@ package api
1212

1313
import "testing"
1414

15-
func TestSandboxRunnerValuesOmitImagesWithoutConfiguration(t *testing.T) {
16-
t.Setenv("APP_STUDIO_SANDBOX_RUNNER_IMAGE", "")
17-
t.Setenv("APP_STUDIO_SANDBOX_TOKEN_GENERATOR_IMAGE", "")
15+
// App Studio supplies only projectRef on a SandboxRunner binding. The runner
16+
// image is a schema field with a sane default on the sandbox-runner template
17+
// (the web-app convention), so App Studio never sets it — not even from the
18+
// legacy env.
19+
func TestSandboxRunnerValuesSuppliesOnlyProjectRef(t *testing.T) {
20+
t.Setenv("APP_STUDIO_SANDBOX_RUNNER_IMAGE", "ghcr.io/faroshq/kedge-sandbox-runner@sha256:runner")
21+
t.Setenv("APP_STUDIO_SANDBOX_TOKEN_GENERATOR_IMAGE", "registry.example.com/kubectl@sha256:token")
1822

1923
values := sandboxRunnerValues("demo")
2024
if got := values["projectRef"]; got != "demo" {
2125
t.Fatalf("projectRef = %q, want demo", got)
2226
}
23-
if _, ok := values["runnerImage"]; ok {
24-
t.Fatal("runnerImage must not default to a mutable development image")
25-
}
26-
if _, ok := values["tokenGeneratorImage"]; ok {
27-
t.Fatal("tokenGeneratorImage must not default to a mutable development image")
28-
}
29-
}
30-
31-
func TestSandboxRunnerValuesUseConfiguredImages(t *testing.T) {
32-
t.Setenv("APP_STUDIO_SANDBOX_RUNNER_IMAGE", " ghcr.io/faroshq/kedge-sandbox-runner@sha256:runner ")
33-
t.Setenv("APP_STUDIO_SANDBOX_TOKEN_GENERATOR_IMAGE", " registry.example.com/kubectl@sha256:token ")
34-
35-
values := sandboxRunnerValues("demo")
36-
if got := values["runnerImage"]; got != "ghcr.io/faroshq/kedge-sandbox-runner@sha256:runner" {
37-
t.Fatalf("runnerImage = %q, want configured digest", got)
38-
}
39-
if got := values["tokenGeneratorImage"]; got != "registry.example.com/kubectl@sha256:token" {
40-
t.Fatalf("tokenGeneratorImage = %q, want configured digest", got)
27+
if len(values) != 1 {
28+
t.Fatalf("sandboxRunnerValues = %#v, want only projectRef (image is a template schema default)", values)
4129
}
4230
}

providers/app-studio/deploy/chart/templates/deployment.yaml

Lines changed: 0 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -1,26 +1,11 @@
11
{{- $catalogEntryConfigMap := and .Values.catalogEntry.enabled .Values.catalogEntry.renderAsConfigMap -}}
2-
{{- $sandboxRunnerImage := .Values.sandboxRunner.runnerImage | default "" -}}
3-
{{- $sandboxTokenGeneratorImage := .Values.sandboxRunner.tokenGeneratorImage | default "" -}}
42
{{- $previewGatewayEnabled := .Values.previewGateway.enabled | default false -}}
53
{{- $previewGatewayBaseDomain := .Values.previewGateway.baseDomain | default "" -}}
64
{{- $previewGatewayFeatureEnabled := and $previewGatewayEnabled (ne (trim $previewGatewayBaseDomain) "") -}}
75
{{- $previewGatewayServiceName := printf "%s-preview-gateway" (include "appstudio.fullname" .) | trunc 63 | trimSuffix "-" -}}
86
{{- if and $previewGatewayFeatureEnabled (not .Values.previewTokenSecretRef.name) -}}
97
{{- fail "previewGateway is enabled with baseDomain, but previewTokenSecretRef.name is required for shared preview tokens" -}}
108
{{- end -}}
11-
{{- /*
12-
App Studio creates a SandboxRunner for every project's development environment,
13-
and the SandboxRunner spec requires both images. Require them unconditionally so
14-
a misconfigured install fails here instead of producing invalid SandboxRunner
15-
resources at runtime. (Image ownership moves to the infrastructure provider in a
16-
later phase; until then App Studio still passes them through.)
17-
*/ -}}
18-
{{- if not $sandboxRunnerImage -}}
19-
{{- fail "sandboxRunner.runnerImage is required; use an immutable digest reference" -}}
20-
{{- end -}}
21-
{{- if not $sandboxTokenGeneratorImage -}}
22-
{{- fail "sandboxRunner.tokenGeneratorImage is required; use an immutable digest reference" -}}
23-
{{- end -}}
249
apiVersion: apps/v1
2510
kind: Deployment
2611
metadata:
@@ -106,12 +91,6 @@ spec:
10691
# calling user via the forwarded bearer token.
10792
- name: KEDGE_PROVIDER_KUBECONFIG
10893
value: /var/run/secrets/kedge/kedge-provider-kubeconfig
109-
# Always set: the SandboxRunner spec requires both images, and the
110-
# template above fails the install when either is unset.
111-
- name: APP_STUDIO_SANDBOX_RUNNER_IMAGE
112-
value: {{ $sandboxRunnerImage | quote }}
113-
- name: APP_STUDIO_SANDBOX_TOKEN_GENERATOR_IMAGE
114-
value: {{ $sandboxTokenGeneratorImage | quote }}
11594
{{- if $previewGatewayFeatureEnabled }}
11695
- name: APP_STUDIO_PREVIEW_BASE_DOMAIN
11796
value: {{ $previewGatewayBaseDomain | quote }}

providers/app-studio/deploy/chart/values.yaml

Lines changed: 4 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -39,14 +39,10 @@ providerKubeconfig:
3939
# infrastructure provider as subresources on the SandboxRunner instance, reached
4040
# through the hub as the calling user. See docs/app-studio-runtime-decoupling.md.
4141

42-
# Images passed into infrastructure-backed SandboxRunner resources. App Studio
43-
# creates a SandboxRunner for every project, and its spec requires both images,
44-
# so the chart fails the install when either is unset (independent of the
45-
# runtime kubeconfig). They are empty by default so installs do not silently
46-
# trust mutable development tags — set both to immutable digest references.
47-
sandboxRunner:
48-
runnerImage: ""
49-
tokenGeneratorImage: ""
42+
# The SandboxRunner runner image is NOT configured here — the sandbox-runner
43+
# template declares it as a schema field with a sane default (the web-app
44+
# convention). App Studio passes only the project reference. See
45+
# providers/infrastructure/docs/template-conventions.md.
5046

5147
# Optional preview route deployment settings and companion infrastructure binding.
5248
# If baseDomain is empty, App Studio does not add the SandboxPreviewHTTPRoute

providers/infrastructure/backend/kro/backend.go

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -56,9 +56,10 @@ type Backend struct {
5656
// tokens are the platform-config values substituted for reserved ${kedge.*}
5757
// placeholders in a Template's backendConfig before the RGD is authored —
5858
// platform-wide settings that belong on the backend, not in per-tenant data.
59-
// See substituteTokens in rgd.go. Today: the exposure-layer Gateway parent
60-
// (${kedge.gatewayName}/${kedge.gatewayNamespace}) and the sandbox runner
61-
// images (${kedge.sandboxRunnerImage}/${kedge.sandboxTokenGeneratorImage}).
59+
// See substituteTokens in rgd.go. The ONLY tokens are the exposure-layer
60+
// Gateway parent (${kedge.gatewayName}/${kedge.gatewayNamespace}); per-instance
61+
// inputs like container images are schema fields with defaults, not tokens
62+
// (see providers/infrastructure/docs/template-conventions.md).
6263
tokens map[string]string
6364
}
6465

@@ -82,10 +83,11 @@ const (
8283
//
8384
// - KEDGE_GATEWAY_NAME / KEDGE_GATEWAY_NAMESPACE — the exposure-layer Gateway
8485
// parent (defaults "cloudflare-tunnel" / "cfgate-system").
85-
// - KEDGE_SANDBOX_RUNNER_IMAGE / KEDGE_SANDBOX_TOKEN_GENERATOR_IMAGE — the
86-
// images materialized into SandboxRunner workloads. The platform owns these
87-
// (App Studio no longer injects them); deployments should pin immutable
88-
// digests. Empty when unset — the chart guards against that at install.
86+
//
87+
// Per-instance inputs (container images, etc.) are NOT env tokens — templates
88+
// declare them as schema fields with sane defaults (e.g. SandboxRunner's
89+
// spec.runnerImage), the same convention every other template follows. See
90+
// providers/infrastructure/docs/template-conventions.md.
8991
func New(runtime dynamic.Interface) *Backend {
9092
gatewayName := os.Getenv("KEDGE_GATEWAY_NAME")
9193
if gatewayName == "" {
@@ -96,10 +98,8 @@ func New(runtime dynamic.Interface) *Backend {
9698
gatewayNamespace = DefaultGatewayNamespace
9799
}
98100
tokens := map[string]string{
99-
gatewayNameToken: gatewayName,
100-
gatewayNamespaceToken: gatewayNamespace,
101-
sandboxRunnerImageToken: os.Getenv("KEDGE_SANDBOX_RUNNER_IMAGE"),
102-
sandboxTokenGeneratorToken: os.Getenv("KEDGE_SANDBOX_TOKEN_GENERATOR_IMAGE"),
101+
gatewayNameToken: gatewayName,
102+
gatewayNamespaceToken: gatewayNamespace,
103103
}
104104
return &Backend{runtime: runtime, tokens: tokens}
105105
}

providers/infrastructure/backend/kro/backend_test.go

Lines changed: 4 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -162,27 +162,12 @@ func TestSubstituteTokensLeavesKroRefs(t *testing.T) {
162162
}
163163
}
164164

165-
func TestSubstituteTokensSandboxImages(t *testing.T) {
166-
in := []byte(`{"runner":"${kedge.sandboxRunnerImage}","token":"${kedge.sandboxTokenGeneratorImage}"}`)
167-
tokens := map[string]string{
168-
sandboxRunnerImageToken: "ghcr.io/faroshq/kedge-sandbox-runner@sha256:abc",
169-
sandboxTokenGeneratorToken: "docker.io/bitnami/kubectl@sha256:def",
170-
}
171-
out := string(substituteTokens(in, tokens))
172-
want := `{"runner":"ghcr.io/faroshq/kedge-sandbox-runner@sha256:abc","token":"docker.io/bitnami/kubectl@sha256:def"}`
173-
if out != want {
174-
t.Errorf("substituteTokens = %s, want %s", out, want)
175-
}
176-
}
177-
178-
// testTokens is the platform-config token map the backend builds from env,
179-
// with the gateway defaults and placeholder sandbox images, for buildRGD tests.
165+
// testTokens is the platform-config token map the backend builds from env (the
166+
// exposure-layer Gateway parent), for buildRGD tests.
180167
func testTokens() map[string]string {
181168
return map[string]string{
182-
gatewayNameToken: DefaultGatewayName,
183-
gatewayNamespaceToken: DefaultGatewayNamespace,
184-
sandboxRunnerImageToken: "ghcr.io/faroshq/kedge-sandbox-runner:test",
185-
sandboxTokenGeneratorToken: "docker.io/bitnami/kubectl:test",
169+
gatewayNameToken: DefaultGatewayName,
170+
gatewayNamespaceToken: DefaultGatewayNamespace,
186171
}
187172
}
188173

providers/infrastructure/backend/kro/rgd.go

Lines changed: 0 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -30,11 +30,6 @@ import (
3030
const (
3131
gatewayNameToken = "${kedge.gatewayName}"
3232
gatewayNamespaceToken = "${kedge.gatewayNamespace}"
33-
// sandboxRunnerImageToken / sandboxTokenGeneratorToken carry the images the
34-
// SandboxRunner workload + its control-token Job run. The platform owns them
35-
// (App Studio no longer injects them into the instance spec).
36-
sandboxRunnerImageToken = "${kedge.sandboxRunnerImage}"
37-
sandboxTokenGeneratorToken = "${kedge.sandboxTokenGeneratorImage}"
3833
)
3934

4035
const (

0 commit comments

Comments
 (0)