Skip to content

Commit 002afb2

Browse files
mjudeikisclaude
andauthored
feat(app-studio): drop runtime kubeconfig; call infra data-plane (Phase 3) (#368)
The payoff phase of decoupling App Studio from the runtime cluster (see docs/app-studio-runtime-decoupling.md §5). App Studio no longer holds a kubeconfig to the cluster where SandboxRunner workloads run. The live development data plane is served by the infrastructure provider as subresources on the SandboxRunner instance (Phase 1), reached through the hub as the requesting user. New api/dataplane_client.go calls: {hub}/services/providers/infrastructure/dataplane/clusters/{cluster}/ sandboxrunners/{name}/{verb} forwarding the caller's bearer token; the infra provider authorizes the request as that user and owns the runtime credential + control-token injection. Handlers rewritten: - sync/restart -> POST .../{sync,restart} - logs -> GET .../log (streamed) - preview ready -> GET .../proxy/ (503 + k8s Status body => starting) - status -> unchanged (reads the CR status; control plane) Removed from App Studio: - runtimeConfig/runtimeClient/runtimeDynamic + SetRuntimeConfig - loadRuntimeConfig + APP_STUDIO_RUNTIME_KUBECONFIG + chart wiring - the per-instance control-token Secret read (infra injects it) - the explicit runtime Namespace delete on teardown — deleting the SandboxRunner CR lets the kro template GC the namespace - the imperative preview ReferenceGrant create — now materialized by the sandbox-runner template (new previewReferenceGrant RGD resource) runtimeTargetForProject (reads the CR to resolve preview/control refs) and signed preview-URL minting stay — both are control-plane / product logic. App Studio still passes the runner images through until image ownership lands (kept to avoid a cross-phase ordering trap with #366). Tests: new dataplane_client tests (URL shape, token forwarding, hub/ cluster guards); the sync + preview-readiness tests rewritten to drive a fake hub data-plane endpoint; obsolete runtime-namespace and ReferenceGrant tests removed. go build + go test ./... green; both provider charts render. Deploy ordering: requires Phase 1 (#367) live for the data plane and Phase 2a (#366) for image ownership; the ReferenceGrant RGD resource ships here so preview routing has no gap when App Studio stops writing it. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent cb7031e commit 002afb2

14 files changed

Lines changed: 385 additions & 705 deletions

docs/app-studio-runtime-decoupling.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -131,7 +131,7 @@ The `/services/providers/infrastructure` prefix is the hub backend proxy; the pr
131131
132132
Today the runtime-cluster client lives in the kro backend / operator. The data-plane handler runs in the provider **serve** process, which already gets the runtime kubeconfig mounted (the operator wires it). Add the runtime client to the server `Deps` so the handler can read the control Secret and reach the service-proxy. No new credential is introduced — it is the credential the provider already owns.
133133
134-
## 5. App Studio after cutover
134+
## 5. App Studio after cutover — **DONE (Phase 3)**
135135
136136
- **Delete** `runtimeConfig` / `runtimeClient` / `runtimeDynamic`, `loadRuntimeConfig`, `APP_STUDIO_RUNTIME_KUBECONFIG`, and the `runtimeKubeconfig` chart wiring.
137137
- **Rewrite** `api/development_runtime.go` / `api/development_sync.go` handlers to call the VW subresources as the tenant user (forwarding the caller's bearer token over App Studio's existing tenant kcp client) instead of the runtime cluster.

docs/app-studio-sandbox-runtime.md

Lines changed: 13 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -38,24 +38,19 @@ Known limitations:
3838
Before promoting this beyond local/dev use, add explicit runtime isolation,
3939
quota defaults, image provenance controls, and network policy hardening.
4040

41-
## Runtime kubeconfig RBAC
42-
43-
`APP_STUDIO_RUNTIME_KUBECONFIG` must be scoped to only the runtime data-plane
44-
operations App Studio performs after validating deterministic `SandboxRunner`
45-
refs. The credential should not be cluster-admin. A minimal role needs:
46-
47-
```yaml
48-
rules:
49-
- apiGroups: [""]
50-
resources: ["secrets", "endpoints"]
51-
verbs: ["get"]
52-
- apiGroups: [""]
53-
resources: ["services/proxy"]
54-
verbs: ["get", "create"]
55-
- apiGroups: [""]
56-
resources: ["namespaces"]
57-
verbs: ["delete"]
58-
```
41+
## Runtime data plane (no App Studio runtime kubeconfig)
42+
43+
App Studio no longer holds a kubeconfig to the runtime cluster. The live
44+
data-plane operations (sync, restart, logs, preview readiness) are served by the
45+
**infrastructure provider** as subresources on the `SandboxRunner` instance —
46+
the provider owns the runtime-cluster credential and the control-token
47+
injection. App Studio calls those subresources through the hub as the requesting
48+
user, who is authorized by their own RBAC on the instance. The runtime namespace
49+
is garbage-collected by the kro template when the `SandboxRunner` instance is
50+
deleted, and the preview `ReferenceGrant` is materialized by that template too.
51+
See [`app-studio-runtime-decoupling.md`](./app-studio-runtime-decoupling.md) for
52+
the full design (including BYO compute, where a workspace can be backed by a
53+
different infrastructure provider / runtime cluster).
5954

6055
## Capability Boundary
6156

providers/app-studio/README.md

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -52,7 +52,6 @@ Environment variables consumed by the binary:
5252
| `KEDGE_HUB_TOKEN` | Bearer token for the heartbeat |
5353
| `KEDGE_PROVIDER_NAME` | CatalogEntry name (default `app-studio`) |
5454
| `KEDGE_PROVIDER_KUBECONFIG` | Provider kubeconfig (kcp front-proxy host + TLS only) |
55-
| `APP_STUDIO_RUNTIME_KUBECONFIG` | Kubernetes kubeconfig for the runtime cluster that runs `SandboxRunner` pods |
5655
| `APP_STUDIO_SANDBOX_RUNNER_IMAGE` | Runner image passed to new `SandboxRunner` resources; use an immutable digest outside local development |
5756
| `APP_STUDIO_SANDBOX_TOKEN_GENERATOR_IMAGE` | kubectl-capable token-generator image passed to new `SandboxRunner` resources; use an immutable digest outside local development |
5857
| `APP_STUDIO_PREVIEW_BASE_DOMAIN` | Optional DNS zone for companion Sandbox preview routing. |
@@ -112,9 +111,11 @@ preview URLs by minting signed, host-based URLs from the companion
112111

113112
Infrastructure owns the resource composition: the `sandbox-runner` Template uses
114113
KRO to create the runtime namespace, PVC, Deployment, Service, control Secret,
115-
and network policy. App Studio uses `APP_STUDIO_RUNTIME_KUBECONFIG` only for
116-
runtime data-plane calls after validating that `SandboxRunner` status refs still
117-
point at the deterministic runner-owned namespace, Service, and Secret.
114+
and network policy. App Studio holds **no** kubeconfig to the runtime cluster:
115+
the live data plane (logs, file sync, restart, preview readiness) is served by
116+
the infrastructure provider as subresources on the `SandboxRunner` instance,
117+
which App Studio calls through the hub as the requesting user. See
118+
[`docs/app-studio-runtime-decoupling.md`](../../docs/app-studio-runtime-decoupling.md).
118119

119120
When `APP_STUDIO_PREVIEW_BASE_DOMAIN` is set, App Studio also creates a
120121
`SandboxPreviewHTTPRoute` infrastructure resource beside each `SandboxRunner`.
Lines changed: 163 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,163 @@
1+
/*
2+
Copyright 2026 The Faros Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
*/
10+
11+
package api
12+
13+
import (
14+
"context"
15+
"fmt"
16+
"io"
17+
"net/http"
18+
"strings"
19+
"time"
20+
)
21+
22+
// App Studio no longer holds a kubeconfig to the runtime cluster. The live
23+
// development data plane (logs, file sync, restart, preview readiness) is now
24+
// served by the infrastructure provider as subresources on the SandboxRunner
25+
// instance, reached through the hub backend proxy:
26+
//
27+
// {hub}/services/providers/infrastructure/dataplane/clusters/{cluster}/sandboxrunners/{name}/{verb}
28+
//
29+
// The caller's bearer token is forwarded as-is; the infra provider authorizes
30+
// the request as that caller (a tenant-scoped GET on the instance) and owns the
31+
// runtime-cluster credential. See docs/app-studio-runtime-decoupling.md.
32+
const (
33+
infraDataPlaneProvider = "infrastructure"
34+
sandboxRunnersResource = "sandboxrunners"
35+
36+
dataPlaneVerbLog = "log"
37+
dataPlaneVerbSync = "sync"
38+
dataPlaneVerbRestart = "restart"
39+
dataPlaneVerbProxy = "proxy"
40+
41+
dataPlaneCallTimeout = 30 * time.Second
42+
)
43+
44+
// sandboxDataPlaneURL composes the hub URL for a SandboxRunner data-plane verb.
45+
// tail (with leading slash) is appended after the verb — used only by the open
46+
// "proxy" verb; the control verbs leave it empty.
47+
func (s *Server) sandboxDataPlaneURL(clusterID, runnerName, verb, tail string) string {
48+
u := strings.TrimRight(s.hubBase, "/") +
49+
fmt.Sprintf("/services/providers/%s/dataplane/clusters/%s/%s/%s/%s",
50+
infraDataPlaneProvider, clusterID, sandboxRunnersResource, runnerName, verb)
51+
if tail != "" {
52+
u += tail
53+
}
54+
return u
55+
}
56+
57+
// newSandboxDataPlaneRequest builds a data-plane request authenticated as the
58+
// caller (the same bearer token the caller authenticated to App Studio with).
59+
func (s *Server) newSandboxDataPlaneRequest(ctx context.Context, method string, id identity, runnerName, verb, tail string, body io.Reader) (*http.Request, error) {
60+
if strings.TrimSpace(s.hubBase) == "" {
61+
return nil, fmt.Errorf("hub URL is not configured; cannot reach the infrastructure data plane")
62+
}
63+
if strings.TrimSpace(id.clusterID) == "" {
64+
return nil, fmt.Errorf("no workspace cluster on request; cannot address the sandbox runner")
65+
}
66+
req, err := http.NewRequestWithContext(ctx, method, s.sandboxDataPlaneURL(id.clusterID, runnerName, verb, tail), body)
67+
if err != nil {
68+
return nil, err
69+
}
70+
if token := strings.TrimSpace(id.token); token != "" {
71+
req.Header.Set("Authorization", "Bearer "+token)
72+
}
73+
return req, nil
74+
}
75+
76+
// sandboxDataPlaneClient returns an HTTP client for data-plane calls, honoring
77+
// the same TLS-skip knob the MCP client uses for hub-internal addressing.
78+
func (s *Server) sandboxDataPlaneClient(timeout time.Duration) *http.Client {
79+
return &http.Client{Timeout: timeout, Transport: projectMCPTransport(s.mcpInsecureSkipTLSVerify)}
80+
}
81+
82+
// sandboxDataPlanePost sends a POST verb (sync, restart) and returns the body +
83+
// status code. The caller maps non-2xx to an error so the runner's own response
84+
// surfaces to the UI.
85+
func (s *Server) sandboxDataPlanePost(ctx context.Context, id identity, runnerName, verb string, payload []byte) ([]byte, int, error) {
86+
callCtx, cancel := context.WithTimeout(ctx, dataPlaneCallTimeout)
87+
defer cancel()
88+
req, err := s.newSandboxDataPlaneRequest(callCtx, http.MethodPost, id, runnerName, verb, "", strings.NewReader(string(payload)))
89+
if err != nil {
90+
return nil, 0, err
91+
}
92+
req.Header.Set("Content-Type", "application/json")
93+
resp, err := s.sandboxDataPlaneClient(dataPlaneCallTimeout).Do(req)
94+
if err != nil {
95+
return nil, 0, fmt.Errorf("sandbox data plane %s: %w", verb, err)
96+
}
97+
defer func() { _ = resp.Body.Close() }()
98+
body, err := io.ReadAll(io.LimitReader(resp.Body, 16<<20))
99+
if err != nil {
100+
return nil, resp.StatusCode, err
101+
}
102+
return body, resp.StatusCode, nil
103+
}
104+
105+
// sandboxDataPlaneStream proxies a streaming GET verb (logs) straight to w,
106+
// copying the upstream status and content type.
107+
func (s *Server) sandboxDataPlaneStream(ctx context.Context, id identity, runnerName, verb string, w http.ResponseWriter) error {
108+
req, err := s.newSandboxDataPlaneRequest(ctx, http.MethodGet, id, runnerName, verb, "", nil)
109+
if err != nil {
110+
return err
111+
}
112+
// No client timeout: log streams are long-lived; ctx cancellation (request
113+
// close) ends them.
114+
resp, err := s.sandboxDataPlaneClient(0).Do(req)
115+
if err != nil {
116+
return fmt.Errorf("sandbox data plane %s: %w", verb, err)
117+
}
118+
defer func() { _ = resp.Body.Close() }()
119+
if ct := resp.Header.Get("Content-Type"); ct != "" {
120+
w.Header().Set("Content-Type", ct)
121+
}
122+
w.WriteHeader(resp.StatusCode)
123+
flusher, _ := w.(http.Flusher)
124+
buf := make([]byte, 32<<10)
125+
for {
126+
n, readErr := resp.Body.Read(buf)
127+
if n > 0 {
128+
if _, writeErr := w.Write(buf[:n]); writeErr != nil {
129+
return writeErr
130+
}
131+
if flusher != nil {
132+
flusher.Flush()
133+
}
134+
}
135+
if readErr == io.EOF {
136+
return nil
137+
}
138+
if readErr != nil {
139+
return readErr
140+
}
141+
}
142+
}
143+
144+
// sandboxDataPlaneProbe performs a GET against the open proxy verb (path tail)
145+
// and returns the upstream status + a bounded body, for preview readiness.
146+
func (s *Server) sandboxDataPlaneProbe(ctx context.Context, id identity, runnerName, tail string) (int, []byte, error) {
147+
callCtx, cancel := context.WithTimeout(ctx, previewReadinessProbeTimeout)
148+
defer cancel()
149+
req, err := s.newSandboxDataPlaneRequest(callCtx, http.MethodGet, id, runnerName, dataPlaneVerbProxy, tail, nil)
150+
if err != nil {
151+
return 0, nil, err
152+
}
153+
resp, err := s.sandboxDataPlaneClient(previewReadinessProbeTimeout).Do(req)
154+
if err != nil {
155+
return 0, nil, err
156+
}
157+
defer func() { _ = resp.Body.Close() }()
158+
body, err := io.ReadAll(io.LimitReader(resp.Body, 1<<20))
159+
if err != nil {
160+
return resp.StatusCode, nil, err
161+
}
162+
return resp.StatusCode, body, nil
163+
}
Lines changed: 53 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,53 @@
1+
/*
2+
Copyright 2026 The Faros Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
*/
10+
11+
package api
12+
13+
import (
14+
"context"
15+
"net/http"
16+
"testing"
17+
)
18+
19+
func TestSandboxDataPlaneURL(t *testing.T) {
20+
s := &Server{hubBase: "https://hub.example/"}
21+
got := s.sandboxDataPlaneURL("root:kedge:orgs:acme", "kedge-sandbox-abc", dataPlaneVerbLog, "")
22+
want := "https://hub.example/services/providers/infrastructure/dataplane/clusters/root:kedge:orgs:acme/sandboxrunners/kedge-sandbox-abc/log"
23+
if got != want {
24+
t.Fatalf("sandboxDataPlaneURL = %q, want %q", got, want)
25+
}
26+
// The open proxy verb appends the caller tail after the verb.
27+
gotProxy := s.sandboxDataPlaneURL("c1", "r1", dataPlaneVerbProxy, "/assets/app.js")
28+
wantProxy := "https://hub.example/services/providers/infrastructure/dataplane/clusters/c1/sandboxrunners/r1/proxy/assets/app.js"
29+
if gotProxy != wantProxy {
30+
t.Fatalf("proxy URL = %q, want %q", gotProxy, wantProxy)
31+
}
32+
}
33+
34+
func TestNewSandboxDataPlaneRequestRequiresHubAndCluster(t *testing.T) {
35+
id := identity{clusterID: "c1", token: "tok"}
36+
// No hub base configured.
37+
if _, err := (&Server{}).newSandboxDataPlaneRequest(context.Background(), http.MethodGet, id, "r1", dataPlaneVerbLog, "", nil); err == nil {
38+
t.Fatal("expected error when hubBase is unset")
39+
}
40+
// No cluster on the request.
41+
s := &Server{hubBase: "https://hub.example"}
42+
if _, err := s.newSandboxDataPlaneRequest(context.Background(), http.MethodGet, identity{token: "tok"}, "r1", dataPlaneVerbLog, "", nil); err == nil {
43+
t.Fatal("expected error when clusterID is empty")
44+
}
45+
// Happy path forwards the caller's bearer token.
46+
req, err := s.newSandboxDataPlaneRequest(context.Background(), http.MethodGet, id, "r1", dataPlaneVerbLog, "", nil)
47+
if err != nil {
48+
t.Fatalf("unexpected error: %v", err)
49+
}
50+
if got := req.Header.Get("Authorization"); got != "Bearer tok" {
51+
t.Fatalf("Authorization = %q, want Bearer tok", got)
52+
}
53+
}

0 commit comments

Comments
 (0)