Skip to content

Commit cb7031e

Browse files
mjudeikisclaude
andauthored
feat(infra): data-plane subresource proxy handler (Phase 1) (#367)
Second phase of decoupling App Studio from the runtime cluster (see docs/app-studio-runtime-decoupling.md). The infrastructure provider now serves a workload instance's declared data-plane verbs as subresources, so consumers reach a workload's logs/sync/restart/preview without holding a runtime-cluster credential themselves. App Studio's cutover (dropping its runtime kubeconfig) is Phase 3. Transport decision (doc §6.1): the provider's own serve mux behind the hub backend proxy — the same path /mcp already uses — rather than a kcp APIExport subresource (unproven for custom verbs on CRD-backed resources). The workspace is addressed explicitly in the URL: /dataplane/clusters/<ws>/<resource>/<name>/<verb>[/<tail>] reached via /services/providers/infrastructure/dataplane/... with the caller's bearer token forwarded as-is. Per request the handler: 1. authorizes + fetches the instance AS THE CALLER via the tenant client (a tenant-scoped GET; the caller's 403/404 is the gate — no provider-wide credential decides access), 2. resolves the template's dataPlane contract (Phase 0 resolver), 3. enforces the per-verb method allowlist, 4. reads the per-instance control token from the runtime cluster, 5. reverse-proxies to the runtime Service via services/proxy, injecting X-Sandbox-Control-Token and stripping the caller's Authorization; httputil.ReverseProxy handles WebSocket/stream upgrades (preview, log follow) when upgrade/stream is declared. A "status" verb is served straight from the instance status with no runtime hop. The runtime credential is resolved exactly like the kro backend (KRO_KUBECONFIG, else in-cluster); the data plane is disabled (503) when absent, preserving the REST-only/dev flow. Unit-tested with fakes + an httptest runtime backend: control-verb proxy (path/token-injection/auth-stripping), preview caller-path append, status-from-CR, missing-token 401, authz 403/404 forwarding, method 405, namespace-escape 409, unknown verb, nil-deps 503, path parsing, and contract decode. Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
1 parent ba17e5c commit cb7031e

11 files changed

Lines changed: 1012 additions & 7 deletions

File tree

docs/app-studio-runtime-decoupling.md

Lines changed: 7 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -110,14 +110,14 @@ A generic resolver turns `(instance, endpointName) → {serviceNamespace, servic
110110
### 4.1 Request shape
111111

112112
```
113-
GET /services/providers/infrastructure/vw/clusters/{ws}/
114-
apis/infrastructure.kedge.faros.sh/v1alpha1/sandboxrunners/{name}/log
113+
GET /services/providers/infrastructure/dataplane/clusters/{ws}/sandboxrunners/{name}/log
115114
GET …/sandboxrunners/{name}/proxy/{path...}
116115
POST …/sandboxrunners/{name}/sync
117116
POST …/sandboxrunners/{name}/restart
117+
…/sandboxrunners/{name}/status (served from the instance status; no runtime hop)
118118
```
119119
120-
(The exact prefix depends on the transport vehicle chosen in §6; the handler logic is identical either way.)
120+
The `/services/providers/infrastructure` prefix is the hub backend proxy; the provider's serve mux sees `/dataplane/clusters/{ws}/{resource}/{name}/{verb}[/{tail}]` (§6.1). The `{ws}` segment carries colons (`root:kedge:orgs:acme`) and is a single path segment.
121121
122122
### 4.2 Per-request flow
123123
@@ -148,16 +148,17 @@ After this, App Studio's `SandboxRunner` values shrink to roughly `{projectRef}`
148148
149149
## 6. Open decisions
150150
151-
### 6.1 VW transport vehicle — **spike in Phase 1**
151+
### 6.1 VW transport vehicle — **decided in Phase 1**
152152
153153
The "subresource on the instance" semantics can be realized two ways:
154154
155155
| Vehicle | Pros | Cons |
156156
|---|---|---|
157157
| **kcp APIExport subresource** on the instance kind | Purest model; reachable via the normal bound-resource API path App Studio already uses; no per-workspace URL mapping | Unproven that kcp APIExport supports arbitrary custom subresources on CRD-backed resources |
158-
| **Dedicated VW builder** (`spec.virtualWorkspace` → `/services/providers/infrastructure/vw/…`) | **Proven in-repo** by `edges_proxy_builder.go` (proxy + upgrades + WebSocket through the hub) | Hub must map workspace → provider (the binding supplies this); slightly more wiring |
158+
| **Provider serve mux behind the hub backend proxy** (`/services/providers/infrastructure/dataplane/…`, workspace in the path) | **Proven in-repo** — the provider already serves `/mcp` this way, and `edges_proxy_builder.go` shows proxy + upgrades + WebSocket through the hub; no kcp-VW machinery | Workspace addressed explicitly in the URL rather than implied by the bound resource; the handler re-authorizes against the instance itself |
159159
160-
Recommendation: budget for the **dedicated VW builder**; it is the de-risked path and the handler logic is identical. Confirm the APIExport-subresource option early — if kcp supports it, prefer it.
160+
**Decision (Phase 1):** the **provider serve mux** vehicle. The handler mounts at `dataplane.PathPrefix` (`/dataplane/`) on the provider's existing HTTP server and is reached through the hub backend proxy with the caller's bearer token forwarded as-is. The URL carries the workspace explicitly —
161+
`/dataplane/clusters/<ws>/<resource>/<name>/<verb>[/<tail>]` — and the handler authorizes by fetching the instance **as the caller** (a tenant-scoped GET; 403/404 is the gate). This keeps the k8s-native subresource *semantics* while avoiding the unproven kcp-APIExport-subresource path. BYO compute still falls out of the binding: App Studio resolves which provider backs a workspace and routes there; a future migration to a true APIExport subresource can swap the transport without touching the resolver or the handler logic.
161162
162163
### 6.2 Move runner config ownership to infra
163164
Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
/*
2+
Copyright 2026 The Faros Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package dataplane
18+
19+
import (
20+
"context"
21+
"encoding/json"
22+
"fmt"
23+
"strings"
24+
25+
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
26+
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
27+
"k8s.io/apimachinery/pkg/runtime/schema"
28+
"k8s.io/client-go/dynamic"
29+
30+
infrav1alpha1 "github.com/faroshq/provider-infrastructure/apis/v1alpha1"
31+
)
32+
33+
// ContractGetter returns the data-plane contract for an instance resource
34+
// (the lowercase plural, e.g. "sandboxrunners"). Returns a nil contract with no
35+
// error when the template declares no data plane, and an error when no template
36+
// owns the resource.
37+
type ContractGetter interface {
38+
For(ctx context.Context, resource string) (*infrav1alpha1.TemplateDataPlane, error)
39+
}
40+
41+
var templateGVR = schema.GroupVersionResource{
42+
Group: "infrastructure.kedge.faros.sh",
43+
Version: "v1alpha1",
44+
Resource: "templates",
45+
}
46+
47+
// TemplateContractGetter resolves a resource's data-plane contract by reading
48+
// Templates from the provider workspace with the provider's own (platform)
49+
// kcp client. Templates are platform-owned and cluster-scoped, so reading them
50+
// with the provider credential is correct — the caller's RBAC is enforced
51+
// separately, on the instance (see handler.go).
52+
type TemplateContractGetter struct {
53+
templates dynamic.NamespaceableResourceInterface
54+
}
55+
56+
// NewTemplateContractGetter builds a getter over the provider's dynamic client.
57+
// Returns nil when client is nil (REST-only/dev serve), which the handler
58+
// surfaces as "data plane unavailable".
59+
func NewTemplateContractGetter(client dynamic.Interface) *TemplateContractGetter {
60+
if client == nil {
61+
return nil
62+
}
63+
return &TemplateContractGetter{templates: client.Resource(templateGVR)}
64+
}
65+
66+
// For lists Templates and returns the dataPlane of the one whose
67+
// spec.instanceCRD.resource matches. Not cached: the Template set is tiny and a
68+
// stale contract would silently mis-route a proxy, so we always read fresh.
69+
func (g *TemplateContractGetter) For(ctx context.Context, resource string) (*infrav1alpha1.TemplateDataPlane, error) {
70+
resource = strings.TrimSpace(resource)
71+
if resource == "" {
72+
return nil, fmt.Errorf("empty instance resource")
73+
}
74+
list, err := g.templates.List(ctx, metav1.ListOptions{})
75+
if err != nil {
76+
return nil, fmt.Errorf("listing templates: %w", err)
77+
}
78+
for i := range list.Items {
79+
tmpl := &list.Items[i]
80+
got, _, _ := unstructured.NestedString(tmpl.Object, "spec", "instanceCRD", "resource")
81+
if strings.TrimSpace(got) != resource {
82+
continue
83+
}
84+
return dataPlaneFromTemplate(tmpl)
85+
}
86+
return nil, fmt.Errorf("no template owns resource %q", resource)
87+
}
88+
89+
// dataPlaneFromTemplate extracts and decodes spec.dataPlane from a Template's
90+
// unstructured form. Returns (nil, nil) when the template declares no data
91+
// plane.
92+
func dataPlaneFromTemplate(tmpl *unstructured.Unstructured) (*infrav1alpha1.TemplateDataPlane, error) {
93+
raw, found, err := unstructured.NestedMap(tmpl.Object, "spec", "dataPlane")
94+
if err != nil {
95+
return nil, fmt.Errorf("template %q spec.dataPlane is malformed: %w", tmpl.GetName(), err)
96+
}
97+
if !found || len(raw) == 0 {
98+
return nil, nil
99+
}
100+
encoded, err := json.Marshal(raw)
101+
if err != nil {
102+
return nil, fmt.Errorf("template %q spec.dataPlane re-encode: %w", tmpl.GetName(), err)
103+
}
104+
var contract infrav1alpha1.TemplateDataPlane
105+
if err := json.Unmarshal(encoded, &contract); err != nil {
106+
return nil, fmt.Errorf("template %q spec.dataPlane decode: %w", tmpl.GetName(), err)
107+
}
108+
return &contract, nil
109+
}
Lines changed: 186 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,186 @@
1+
/*
2+
Copyright 2026 The Faros Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package dataplane
18+
19+
import (
20+
"context"
21+
"encoding/json"
22+
"errors"
23+
"net/http"
24+
"strings"
25+
26+
apierrors "k8s.io/apimachinery/pkg/api/errors"
27+
"k8s.io/apimachinery/pkg/apis/meta/v1/unstructured"
28+
)
29+
30+
// PathPrefix is where the handler is mounted on the provider's serve mux. It is
31+
// reached through the hub backend proxy at
32+
// /services/providers/infrastructure/dataplane/... with the caller's bearer
33+
// token forwarded as-is and X-Kedge-* identity headers injected.
34+
const PathPrefix = "/dataplane/"
35+
36+
// InstanceGetter authorizes and fetches a workload instance AS THE CALLER. The
37+
// implementation builds a tenant-scoped client from (workspace, token) and does
38+
// a GET — so a 403/404 from the caller's RBAC is the access gate for the whole
39+
// data plane. No provider-wide credential is consulted here.
40+
type InstanceGetter interface {
41+
Get(ctx context.Context, workspace, token, resource, name string) (*unstructured.Unstructured, error)
42+
}
43+
44+
// Handler serves a template's declared data-plane verbs as subresources on a
45+
// workload instance:
46+
//
47+
// /dataplane/clusters/<ws>/<resource>/<name>/<verb>[/<caller-path...>]
48+
//
49+
// e.g. /dataplane/clusters/root:kedge:orgs:acme/sandboxrunners/kedge-sandbox-…/log
50+
//
51+
// It authorizes the caller against the instance, resolves the verb to a runtime
52+
// target via the template contract, and reverse-proxies to the runtime cluster
53+
// the provider owns. Consumers therefore never hold a runtime credential.
54+
type Handler struct {
55+
instances InstanceGetter
56+
contracts ContractGetter
57+
runtime Runtime
58+
}
59+
60+
// NewHandler wires the handler. Any nil dependency makes the data plane report
61+
// 503 (the serve process runs without a runtime/kcp config in dev).
62+
func NewHandler(instances InstanceGetter, contracts ContractGetter, runtime Runtime) *Handler {
63+
return &Handler{instances: instances, contracts: contracts, runtime: runtime}
64+
}
65+
66+
// request is the parsed addressing of a data-plane call.
67+
type request struct {
68+
workspace string
69+
resource string
70+
name string
71+
verb string
72+
callerPath string // remaining path beyond the verb (open-proxy tail)
73+
}
74+
75+
func (h *Handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
76+
if h == nil || h.instances == nil || h.contracts == nil || h.runtime == nil {
77+
http.Error(w, "data plane unavailable on this provider", http.StatusServiceUnavailable)
78+
return
79+
}
80+
81+
id := identityFromRequest(r)
82+
if id.token == "" {
83+
http.Error(w, "no bearer token — cannot act on the caller's behalf", http.StatusUnauthorized)
84+
return
85+
}
86+
87+
req, ok := parsePath(r.URL.Path)
88+
if !ok {
89+
http.Error(w, "bad data-plane path; want /dataplane/clusters/<ws>/<resource>/<name>/<verb>", http.StatusBadRequest)
90+
return
91+
}
92+
93+
// 1. Authorize + fetch the instance as the caller. RBAC is the gate.
94+
instance, err := h.instances.Get(r.Context(), req.workspace, id.token, req.resource, req.name)
95+
if err != nil {
96+
writeKubeError(w, err)
97+
return
98+
}
99+
100+
// 2. Resolve the template's data-plane contract for this resource.
101+
contract, err := h.contracts.For(r.Context(), req.resource)
102+
if err != nil {
103+
http.Error(w, err.Error(), http.StatusNotFound)
104+
return
105+
}
106+
if contract == nil {
107+
http.Error(w, "resource "+req.resource+" exposes no data plane", http.StatusNotFound)
108+
return
109+
}
110+
111+
// 3. Method allowlist for the verb.
112+
if !MethodAllowed(contract, req.verb, r.Method) {
113+
http.Error(w, "method "+r.Method+" not allowed for verb "+req.verb, http.StatusMethodNotAllowed)
114+
return
115+
}
116+
117+
// 4. Resolve the verb to a concrete runtime target (namespace-confined).
118+
target, err := Resolve(contract, instance, req.verb)
119+
if err != nil {
120+
http.Error(w, err.Error(), http.StatusConflict)
121+
return
122+
}
123+
124+
// 5a. A status verb is served straight from the instance status — no hop.
125+
if target.FromStatus {
126+
writeInstanceStatus(w, instance)
127+
return
128+
}
129+
130+
// 5b. Reverse-proxy to the runtime Service the provider owns.
131+
serveProxy(w, r, h.runtime, target, req.callerPath)
132+
}
133+
134+
// parsePath parses /dataplane/clusters/<ws>/<resource>/<name>/<verb>[/<tail...>].
135+
// The workspace segment may itself contain colons (root:kedge:orgs:acme); it is
136+
// a single path segment, so no escaping is needed.
137+
func parsePath(p string) (request, bool) {
138+
rest := strings.TrimPrefix(p, PathPrefix)
139+
if rest == p {
140+
return request{}, false
141+
}
142+
rest = strings.TrimPrefix(rest, "clusters/")
143+
parts := strings.SplitN(rest, "/", 5)
144+
// parts: [ws, resource, name, verb, tail?]
145+
if len(parts) < 4 {
146+
return request{}, false
147+
}
148+
req := request{
149+
workspace: strings.TrimSpace(parts[0]),
150+
resource: strings.TrimSpace(parts[1]),
151+
name: strings.TrimSpace(parts[2]),
152+
verb: strings.TrimSpace(parts[3]),
153+
}
154+
if req.workspace == "" || req.resource == "" || req.name == "" || req.verb == "" {
155+
return request{}, false
156+
}
157+
if len(parts) == 5 {
158+
req.callerPath = "/" + parts[4]
159+
}
160+
return req, true
161+
}
162+
163+
// writeInstanceStatus returns the instance's status subobject as JSON.
164+
func writeInstanceStatus(w http.ResponseWriter, instance *unstructured.Unstructured) {
165+
status, _, _ := unstructured.NestedMap(instance.Object, "status")
166+
if status == nil {
167+
status = map[string]any{}
168+
}
169+
w.Header().Set("Content-Type", "application/json")
170+
_ = json.NewEncoder(w).Encode(status)
171+
}
172+
173+
// writeKubeError maps a Kubernetes API error from the authorize/fetch step to
174+
// an HTTP status, so a caller's 403/404 surfaces faithfully rather than as 500.
175+
func writeKubeError(w http.ResponseWriter, err error) {
176+
var statusErr *apierrors.StatusError
177+
if errors.As(err, &statusErr) {
178+
code := int(statusErr.ErrStatus.Code)
179+
if code == 0 {
180+
code = http.StatusBadGateway
181+
}
182+
http.Error(w, statusErr.ErrStatus.Message, code)
183+
return
184+
}
185+
http.Error(w, err.Error(), http.StatusBadGateway)
186+
}

0 commit comments

Comments
 (0)