feat(infra): template images as schema defaults + kro-acceptance e2e/CI (#369)

* feat(infra): sandbox-runner images as schema defaults (drop ${kedge.*} tokens)

The sandbox-runner template was the lone outlier using ${kedge.*}
env-substitution tokens for its container images (Phase 2a) — every
other template declares configurable images as spec.schema fields with
sane defaults and hardcodes fixed sidecar images. That outlier broke in
practice: an unset KEDGE_SANDBOX_RUNNER_IMAGE substituted to "" and kro
authored a Deployment/Job with `image: ""` (Required value).

Make sandbox-runner follow the same convention as simple-webapp / database
/ redis / application:

  - spec.runnerImage is a schema field with a sane default
    (ghcr.io/faroshq/kedge-sandbox-runner:dev); the runner Deployment uses
    ${schema.spec.runnerImage}. Overridable per instance.
  - the control-token Job image is hardcoded bitnami/kubectl, like every
    other template's control job (tokenGeneratorImage field dropped).

Remove all the now-dead image-token machinery:
  - kro backend: drop the sandbox image tokens (only the exposure Gateway
    tokens remain).
  - infra chart: drop KEDGE_SANDBOX_*_IMAGE env, the install guard, and the
    sandboxRunner values block; operator passthrough.
  - App Studio: sandboxRunnerValues supplies only {projectRef}; drop the
    image env helpers, chart guard/env/values, README rows.

Document the convention so this doesn't recur:
  - new providers/infrastructure/docs/template-conventions.md (when to use a
    schema default vs a hardcoded literal vs a ${kedge.*} token, with the
    failure modes that motivated it).
  - AGENTS.md §9 guardrail pointing at it.

Also: dev postgres images default to Google's Docker Hub mirror
(mirror.gcr.io/library/postgres:16-alpine) — Docker Hub has been dropping
pulls with "unexpected EOF".

Preview routing is unchanged (it was already schema-based on main).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(infra): e2e that every template's RGD is accepted by kro + CI gate

Unit tests only check buildRGD's output; they miss whether kro actually
ACCEPTS the authored graph — which is exactly the class of bug the
sandbox-runner template hit (an integer field fed a string, an
includeWhen that wasn't a standalone CEL expression). Add an e2e that
closes that gap for every seeded template.

  - providers/infrastructure/backend/kro/e2e_test.go (build tag `e2e`):
    for each install/templates/*.yaml, build the real RGD via buildRGD
    and assert kro reports GraphAccepted=True. Reuses the package's
    decodeTemplate/buildRGD/testTokens. Skips when KUBECONFIG is unset,
    so it never runs in the unit pass.

  - make e2e-infrastructure: stands up a throwaway kind cluster with the
    Gateway API CRDs (templates declare HTTPRoute/ReferenceGrant) and
    STANDALONE kro (multicluster.enabled=false — no kcp needed; graph
    validation is independent of the watch source), then runs the test.
    Split into -up / -run / -down for reuse.

  - .github/workflows/infrastructure-templates-e2e.yaml: runs it on PRs
    touching the templates, the kro backend, or the provider apis.

Validated end to end: a fresh kind + standalone kro accepts all five
templates (application, database, redis-cache, sandbox-runner,
simple-webapp).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* test(infra): e2e also creates an instance of each template

The template e2e proved only that kro ACCEPTS each authored RGD
(GraphAccepted=True). Extend it to also CREATE a sample instance per
template and assert kro reconciles it WITHOUT an apply error.

This is the exact failure that motivated the schema-default image
convention ("apply results contain errors: ... image: Required value"):
RGD acceptance alone never exercised it, because the empty-image fault
only surfaces when kro applies a child Deployment. The test now catches
it.

Per template: build an instance from sampleValues when present, else a
minimal valid spec (e2eMinimalSpecs); retry Create until the fresh CRD
is servable; poll status conditions and fail on any apply-error marker
(apply results contain errors / is invalid / Required value / failed to
apply / admission webhook). Readiness is out of scope — images need not
pull in CI; apply validates the spec.

Validated end-to-end on a fresh standalone kro kind cluster (ok 20.7s,
all five templates).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* ci(infra): reflect instance-creation coverage in template e2e workflow

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* test(infra): verify kro creates each template's child objects

The e2e proved kro accepted the RGD and the instance reported no apply
error, but never checked the runtime cluster for the objects kro is
supposed to create. Add that: per template, after the instance
reconciles, List every child the RGD declares (matched by kro's
kro.run/instance-id + node-id labels) and assert the always-on ones
exist; resources gated by includeWhen (e.g. the preview HTTPRoute) are
verified only when present.

This caught a real gap: application's HTTPRoute references
spec.expose.fqdn, a field the platform stamps onto the instance ("do
NOT set" in the sample) — with no controller in the standalone-kro e2e,
the route could never render. The test now stands in for the platform
via e2ePlatformStamped (supplies expose.fqdn + credentialsSecretName),
so the HTTPRoute is created and verified.

Instance names are made per-run-unique (a runID nonce; the 16-hex form
satisfies sandbox-runner's name pattern) so re-running against a reused
cluster can't collide with a prior run's not-yet-GC'd child objects.

Validated on a fresh standalone kro cluster: all five templates create
their declared objects (ok 27.6s) — application now incl. HTTPRoute×2.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* docs: name the provider-isolation rule (no cross-provider backend access)

Providers must not reach into another provider's backend layer (its
runtime/target clusters and credentials, databases, internal Services,
controllers, kro RGDs). All cross-provider interaction goes through the
owning provider's published API: its kcp APIExport resources (control
plane) and virtual-workspace subresources (data plane), invoked as the
tenant user and routed by binding — never by a hardcoded backend URL.

The rule was already practiced everywhere (the App Studio -> infrastructure
runtime-decoupling is the worked example) but never named. Make it explicit
and link the docs that embody it to one canonical statement:

- docs/providers.md: new "Provider isolation (the cross-provider boundary)"
  section + a Security bullet.
- docs/provider-connectivity-contract.md: new Contract 3 (provider-to-
  provider) alongside the existing platform-facing contracts 1 and 2, with
  a conformance table and checklist item.
- AGENTS.md: a guardrail bullet so new work follows it by default.
- Reinforcement notes (linked, not duplicated) in infrastructure-,
  kuery-, mcp-architecture, app-studio-runtime-decoupling,
  app-studio-sandbox-runtime, and the infrastructure/app-studio READMEs.

Docs only; no code or behavior change.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

Author: mjudeikis

.github/workflows/infrastructure-templates-e2e.yaml

@@ -0,0 +1,65 @@
+name: Infrastructure Template E2E
+
+# Every seeded infrastructure Template must author a kro ResourceGraphDefinition
+# that kro ACCEPTS, and a sample instance of it must reconcile WITHOUT an apply
+# error. Unit tests only check buildRGD's output; this stands up a real
+# (standalone) kro on kind and, per template, asserts GraphAccepted=True and then
+# creates a sample instance that kro applies cleanly — catching the failures unit
+# tests miss (string-vs-int fields, non-standalone includeWhen expressions, an
+# image that resolved to "" surfacing as "Required value" only at apply time).
+# See providers/infrastructure/backend/kro/e2e_test.go.
+on:
+  pull_request:
+    branches: [main]
+    paths:
+      - 'providers/infrastructure/install/templates/**'
+      - 'providers/infrastructure/backend/kro/**'
+      - 'providers/infrastructure/apis/**'
+      - '.github/workflows/infrastructure-templates-e2e.yaml'
+      - 'Makefile'
+  push:
+    branches: [main]
+    paths:
+      - 'providers/infrastructure/install/templates/**'
+      - 'providers/infrastructure/backend/kro/**'
+      - 'providers/infrastructure/apis/**'
+      - '.github/workflows/infrastructure-templates-e2e.yaml'
+  workflow_dispatch:
+
+permissions:
+  contents: read
+  packages: read
+
+jobs:
+  template-e2e:
+    name: Templates accepted + instances apply
+    runs-on: ubuntu-latest
+    timeout-minutes: 25
+    steps:
+      - uses: actions/checkout@v4
+
+      - uses: actions/setup-go@v5
+        with:
+          go-version: v1.26.1
+
+      - name: Install kind
+        uses: helm/kind-action@v1
+        with:
+          install_only: true
+
+      - name: Install Helm
+        uses: azure/setup-helm@v3
+        with:
+          token: ${{ secrets.GITHUB_TOKEN }}
+
+      # The kro fork chart + image live in ghcr under this org; logging in lets
+      # helm/kind pull them whether they are public or org-private.
+      - name: Log in to GHCR for the kro chart
+        run: echo "${{ secrets.GITHUB_TOKEN }}" | helm registry login ghcr.io -u ${{ github.actor }} --password-stdin
+
+      - name: Bring up kro + run the template e2e
+        run: make e2e-infrastructure
+
+      - name: Tear down the e2e cluster
+        if: always()
+        run: make e2e-infrastructure-down

AGENTS.md

@@ -390,6 +390,26 @@ whole point.
   build/test and `go.work` awareness; they are not in the root `./...`.
 - Before merging: `make verify` is the full gate
   (boilerplate + codegen + vet + lint + build + test).
+- **Infrastructure templates declare configurable inputs (container images,
+  versions, sizes) as `spec.schema` fields with sane defaults** — never via
+  `${kedge.*}` env-substitution tokens. Fixed sidecar images (e.g. the
+  control-token `kubectl` job) are hardcoded literals. `${kedge.*}` tokens are
+  reserved for genuinely platform-global values with no universal default (only
+  the exposure Gateway parent today). A missing env must never be able to
+  produce an empty/invalid field. See
+  [`providers/infrastructure/docs/template-conventions.md`](providers/infrastructure/docs/template-conventions.md).
+- **Providers are isolated; never reach into another provider's backend.** A
+  provider's backend layer (its runtime/target clusters and their
+  credentials, databases, internal Services, controllers, kro RGDs) is
+  private to it. A provider must not hold a second credential into another
+  provider's cluster/DB/service, call its internal endpoints directly, or
+  hardcode its backend topology. Cross-provider access goes **only** through
+  the other provider's published `APIExport` resources + virtual-workspace
+  subresources, invoked **as the tenant user** and routed by binding (not by
+  a backend URL). This is what makes BYO compute work and bounds blast
+  radius. See [`docs/providers.md` §"Provider isolation"](docs/providers.md#provider-isolation-the-cross-provider-boundary)
+  and contract 3 in
+  [`docs/provider-connectivity-contract.md`](docs/provider-connectivity-contract.md).
 
 ---
 

Makefile

@@ -678,7 +678,9 @@ KUERY_RUNTIME_KUBECONFIG ?= $(KCP_DATA_DIR)/kuery-runtime.kubeconfig
 # …) diverges from SQLite and passing on SQLite has shipped real query bugs.
 # kuery-db-up starts a throwaway container matching KUERY_DEV_DATABASE_URL.
 KUERY_POSTGRES_CONTAINER ?= kedge-kuery-postgres
-KUERY_POSTGRES_IMAGE ?= postgres:16-alpine
+# Pulled from Google's Docker Hub mirror (same official image, more reliable
+# pulls — Docker Hub has been dropping them with "unexpected EOF").
+KUERY_POSTGRES_IMAGE ?= mirror.gcr.io/library/postgres:16-alpine
 KUERY_POSTGRES_PORT ?= 55433
 KUERY_POSTGRES_DATA_DIR ?= $(KCP_DATA_DIR)/kuery-postgres
 KUERY_POSTGRES_USER ?= kuery
@@ -862,7 +864,7 @@ APP_STUDIO_SANDBOX_RUNNER_IMAGE ?= $(SANDBOX_RUNNER_IMAGE)
 APP_STUDIO_SANDBOX_TOKEN_GENERATOR_IMAGE ?= $(SANDBOX_TOKEN_GENERATOR_IMAGE)
 APP_STUDIO_DEV_DATABASE_URL ?= postgres://appstudio:appstudio@localhost:55432/appstudio?sslmode=disable
 APP_STUDIO_POSTGRES_CONTAINER ?= kedge-app-studio-postgres
-APP_STUDIO_POSTGRES_IMAGE ?= postgres:16-alpine
+APP_STUDIO_POSTGRES_IMAGE ?= mirror.gcr.io/library/postgres:16-alpine
 APP_STUDIO_POSTGRES_PORT ?= 55432
 APP_STUDIO_POSTGRES_DATA_DIR ?= $(KCP_DATA_DIR)/app-studio-postgres
 APP_STUDIO_POSTGRES_USER ?= appstudio
@@ -1322,6 +1324,14 @@ KRO_IMAGE_TAG ?= v0.0.1-mc.7
 KRO_NAMESPACE ?= kro-system
 KRO_SEED_DIR ?= providers/infrastructure/examples/rgds
 
+# --- Infrastructure template e2e (RGD acceptance against a real kro) ---
+# A throwaway kind cluster running STANDALONE kro (no kcp) — enough to validate
+# that every seeded Template authors a kro graph kro accepts. See
+# providers/infrastructure/backend/kro/e2e_test.go.
+E2E_KRO_KIND_NAME ?= kedge-kro-e2e
+E2E_KRO_KUBECONFIG ?= $(CURDIR)/.kedge-kro-e2e.kubeconfig
+GATEWAY_API_VERSION ?= v1.2.1
+
 ## Bring up the management kro cluster + install kro (fork w/ multicluster) +
 ## register the cluster as a self-member + seed RGDs. Idempotent: re-running
 ## just helm-upgrades the chart and re-applies the RGDs.
@@ -1428,6 +1438,33 @@ dev-kro-down: ## Delete the kedge-kro kind cluster + kubeconfig file
 	-kind delete cluster --name $(KRO_KIND_NAME)
 	-rm -f $(KRO_KIND_KUBECONFIG)
 
+e2e-infrastructure-up: ## Bring up a throwaway kind cluster with Gateway API CRDs + standalone kro for the template e2e
+	@command -v kind >/dev/null || { echo "kind not found; install: brew install kind"; exit 1; }
+	@command -v helm >/dev/null || { echo "helm not found; install: brew install helm"; exit 1; }
+	@if ! kind get clusters | grep -qx "$(E2E_KRO_KIND_NAME)"; then \
+		echo ">>> creating kind cluster $(E2E_KRO_KIND_NAME)"; \
+		kind create cluster --name $(E2E_KRO_KIND_NAME) --kubeconfig $(E2E_KRO_KUBECONFIG); \
+	else \
+		kind get kubeconfig --name $(E2E_KRO_KIND_NAME) > $(E2E_KRO_KUBECONFIG); \
+	fi
+	@echo ">>> installing Gateway API CRDs ($(GATEWAY_API_VERSION)) — templates declare HTTPRoute/ReferenceGrant"
+	KUBECONFIG=$(E2E_KRO_KUBECONFIG) kubectl apply -f https://github.com/kubernetes-sigs/gateway-api/releases/download/$(GATEWAY_API_VERSION)/standard-install.yaml
+	@echo ">>> installing standalone kro (no kcp) into $(KRO_NAMESPACE)"
+	KUBECONFIG=$(E2E_KRO_KUBECONFIG) helm upgrade --install kro $(KRO_CHART) \
+		--version $(KRO_CHART_VERSION) -n $(KRO_NAMESPACE) --create-namespace \
+		--set image.repository=$(KRO_IMAGE_REPO) --set image.tag=$(KRO_IMAGE_TAG) \
+		--set multicluster.enabled=false --wait --timeout 5m
+
+e2e-infrastructure-down: ## Delete the infra-template e2e kind cluster + kubeconfig
+	-kind delete cluster --name $(E2E_KRO_KIND_NAME)
+	-rm -f $(E2E_KRO_KUBECONFIG)
+
+e2e-infrastructure-run: ## Run the infra template RGD-acceptance e2e against $(E2E_KRO_KUBECONFIG)
+	cd providers/infrastructure && KUBECONFIG=$(E2E_KRO_KUBECONFIG) \
+		go test -tags e2e -count=1 -timeout 15m ./backend/kro/ -run TestE2E -v
+
+e2e-infrastructure: e2e-infrastructure-up e2e-infrastructure-run ## Bring up kro + run the infra template e2e (then `make e2e-infrastructure-down`)
+
 # --- In-cluster variants (no separate kind cluster) ---
 # Used by Tiltfile.cluster, which already manages a kind cluster for kcp.
 # Installing kro INTO that same cluster lets kro reach kcp via in-cluster

docs/app-studio-runtime-decoupling.md

@@ -60,6 +60,14 @@ App Studio keeps the **product** concerns: the Project capability contract, the
 
 ## 2. Design principles
 
+> This work is the reference implementation of the platform-wide
+> [provider-isolation rule](./providers.md#provider-isolation-the-cross-provider-boundary)
+> (contract 3 in [`provider-connectivity-contract.md`](./provider-connectivity-contract.md)):
+> App Studio's `APP_STUDIO_RUNTIME_KUBECONFIG` is a direct credential into
+> another provider's backend — exactly what the rule forbids — and this
+> design replaces it with calls to the infrastructure provider's published
+> subresources.
+
 1. **The runtime cluster has exactly one owner: the infrastructure provider.** Whoever materializes a workload also serves its data plane. No other component holds a runtime credential.
 2. **Data-plane verbs are subresources on the workload instance.** `sandboxrunners/{name}/log` is to a `SandboxRunner` what `pods/log` is to a Pod. This keeps the model k8s-native and routable by binding.
 3. **Routing is binding-driven, never URL-driven.** App Studio never resolves a provider backend URL. It calls a subresource on a resource it is already bound to; kcp routes to the provider that exports it. This is the BYO mechanism.

docs/app-studio-sandbox-runtime.md

@@ -50,7 +50,10 @@ is garbage-collected by the kro template when the `SandboxRunner` instance is
 deleted, and the preview `ReferenceGrant` is materialized by that template too.
 See [`app-studio-runtime-decoupling.md`](./app-studio-runtime-decoupling.md) for
 the full design (including BYO compute, where a workspace can be backed by a
-different infrastructure provider / runtime cluster).
+different infrastructure provider / runtime cluster). This is the platform
+[provider-isolation rule](./providers.md#provider-isolation-the-cross-provider-boundary):
+App Studio never reaches into the infrastructure provider's backend — only its
+published `SandboxRunner` CR and subresources.
 
 ## Capability Boundary
 

docs/infrastructure-architecture.md

@@ -42,6 +42,7 @@ Same user, same workspace, two different namespaces. The instance is real, the p
 3. **Backends implement a narrow interface, not a data model.** Each backend is responsible for taking a `Template` and serving instances of the corresponding per-template CRD. *How* it does that — RGDs, terraform modules, cloud SDK calls — is its own business.
 4. **kcp is the source of truth for identity.** Tenant = workspace logical-cluster name, embedded in the URL. No headers. No hashing.
 5. **The provider binary owns the platform layer + the backends it ships with.** Today that's one backend (kro). The binary stays small; complexity lives in backends behind the interface.
+6. **The backend layer is private to this provider; other providers reach it only through published APIs.** The runtime/target clusters a backend drives, the credentials to them, the kro RGDs, the internal Services — none of it is reachable by another provider. A consumer (e.g. App Studio) interacts with infrastructure-owned workloads **only** through the `Template`/instance CRs (control plane) and their VW subresources (data plane: `sandboxrunners/{name}/log`, `…/proxy/…`), as the tenant user. No other component holds a credential into the runtime cluster. This is the platform-wide [provider-isolation rule](./providers.md#provider-isolation-the-cross-provider-boundary); the worked example is [`app-studio-runtime-decoupling.md`](./app-studio-runtime-decoupling.md).
 
 ## 3. Architecture
 

docs/kuery-provider-architecture.md

@@ -116,6 +116,12 @@ cluster-name prefix (`{tenantCluster}/…`) before forwarding to the engine. One
 isolation enforced at the single choke point. `KEDGE_DEV_ALLOW_TENANT_QUERY` mirrors the
 infrastructure provider's dev escape hatch.
 
+Kuery's relationship to the **edge providers** also follows the platform
+[provider-isolation rule](./providers.md#provider-isolation-the-cross-provider-boundary):
+it consumes `Edge` CRs (a tenant-scoped permission claim) and reaches the clusters
+through the hub's **edges-proxy** as the caller — it never holds a credential into an
+edge provider's backend.
+
 ### MCP tools (the primary consumer)
 
 The provider serves `/mcp` + `/mcp/sse` (proxied at `/services/providers/kuery/mcp`) and

docs/mcp-architecture.md

@@ -172,6 +172,11 @@ Two consequences:
   [`providers/infrastructure/tenant/client.go`](https://github.com/faroshq/kedge/blob/main/providers/infrastructure/tenant/client.go): the tenant client is
   built per-(tenant, caller) from the request token; the provider's own
   credentials are never used for tenant work.
+- **Federation routes to published endpoints, not backends.** The aggregator
+  reaches each provider through its registered `BackendURL`/`/mcp` surface
+  with the caller's identity forwarded — never into the provider's runtime
+  cluster, DB, or internal Services. This is the cross-provider half of the
+  platform [provider-isolation rule](./providers.md#provider-isolation-the-cross-provider-boundary).
 
 ## Adding a new integration
 

docs/provider-connectivity-contract.md

@@ -1,10 +1,17 @@
 # Provider connectivity contract — how providers connect to the platform
 
-This doc pins down the two contracts every provider should follow for *how it
-reaches data*: the **UI data path** (contract 1) and the **API access path**
-(contract 2). It explains the hub plumbing that enforces them, how the portal
-authenticates, which providers conform today, and where the deliberate
-exceptions are.
+This doc pins down the contracts every provider should follow for *how it
+reaches data*: the **UI data path** (contract 1), the **platform API access
+path** (contract 2), and the **provider-to-provider path** (contract 3). It
+explains the hub plumbing that enforces them, how the portal authenticates,
+which providers conform today, and where the deliberate exceptions are.
+
+Contracts 1 and 2 are about a provider reaching the **platform** (kcp / the
+GraphQL gateway). Contract 3 is about a provider reaching **another
+provider** — and the rule there is that it does so *only* through that
+provider's published API, never into its backend. See
+[`providers.md` §"Provider isolation"](./providers.md#provider-isolation-the-cross-provider-boundary)
+for the principle in full; this doc gives the concrete access mechanics.
 
 It is the companion to [`providers.md`](./providers.md) (the provider plane
 overview), [`provider-scoping.md`](./provider-scoping.md) (Global/Org/Personal
@@ -14,11 +21,14 @@ scoping), and [`security.md`](./security.md) (auth setup).
 
 ## Restore-from-reboot summary
 
-- There are **two** legitimate data paths, not one. UI data flows through the
+- There are **three** legitimate data paths. UI data flows through the
   hub's **GraphQL gateway** (contract 1); backend/controller code reaches kcp
   either as a **non-privileged provider ServiceAccount via an
   APIExportEndpointSlice** *or* as the **caller using their forwarded bearer
-  token**, scoped to the tenant workspace (contract 2).
+  token**, scoped to the tenant workspace (contract 2); and one provider
+  reaches **another** provider only through that provider's **published
+  APIExport resources + VW subresources, as the caller** — never into its
+  backend (contract 3).
 - The hub exposes providers through **two different proxies** with **different
   token handling**: the UI proxy forwards **no token**, the backend proxy
   forwards the caller's `Authorization` header **as-is**.
@@ -58,6 +68,27 @@ Both 2a and 2b are admin-free. New providers should pick one (or use 2a for
 controllers and 2b for request-driven endpoints, like `code` and
 `infrastructure` do) and never construct a kcp-admin / root client.
 
+**Contract 3 — provider-to-provider path.** When provider A needs something
+provider B owns, A goes through **B's published API**, not B's backend:
+
+- **B's `APIExport` resources** — A binds B's `APIExport` (an `APIBinding`
+  in the tenant workspace) and reads/writes B's CRs over the normal
+  `/clusters/...` path. This is the control-plane channel (spec/status).
+- **B's VW subresources** on those resources (e.g.
+  `sandboxrunners/{name}/log`, `…/proxy/{path}`) — for data-plane verbs
+  (streams, proxies, shells) that aren't plain CRUD. B serves them against
+  *its* backend; A never sees it.
+
+Both are invoked **as the caller** (forwarded bearer token, scoped to the
+workspace — same identity as 2b) and **routed by binding, never by a
+backend URL**. A must never hold a credential into B's runtime cluster /
+database / internal Service, call B's internal endpoints directly, or encode
+B's backend topology in its own config. The owning provider is the single
+holder of its backend credential. This is the access mechanics behind
+[`providers.md` §"Provider isolation"](./providers.md#provider-isolation-the-cross-provider-boundary);
+the canonical example is the App Studio → infrastructure data-plane
+decoupling ([`app-studio-runtime-decoupling.md`](./app-studio-runtime-decoupling.md)).
+
 ---
 
 ## The hub plumbing (and what it does with the token)
@@ -215,6 +246,22 @@ keeps only the CA, and authenticates with the **caller's** bearer token against
 
 ---
 
+## Contract 3 conformance — provider-to-provider via published API only
+
+| Interaction | Mechanism | Verdict |
+|----------|-----------|---------|
+| `app-studio` → `infrastructure` (sandbox data plane) | **Target:** `SandboxRunner` CR (control plane) + `sandboxrunners/{name}/{log,proxy,sync,restart}` VW subresources (data plane), called as the tenant user | ✅ Conforms (after runtime-decoupling) |
+| `app-studio` → runtime cluster (legacy) | held `APP_STUDIO_RUNTIME_KUBECONFIG`, a direct second credential into infrastructure's runtime cluster | ❌ The violation being removed — see [`app-studio-runtime-decoupling.md`](./app-studio-runtime-decoupling.md) |
+| `kuery` → `Edge`s | reads bound `Edge` CRs / engages clusters through the hub's edges-proxy as the caller, never the edge provider's backend | ✅ Conforms |
+
+The shape that conforms: address a **bound resource** (or a subresource on
+it), authenticate as the **caller**, and let the binding decide which
+provider's backend answers. The shape that doesn't: a kubeconfig / DSN / URL
+in provider A's config that points straight at provider B's cluster, DB, or
+internal Service.
+
+---
+
 ## Known divergences (and why)
 
 1. **Built-ins are privileged by construction.** `mcp`, `kubernetesedges`, and
@@ -249,6 +296,10 @@ keeps only the CA, and authenticates with the **caller's** bearer token against
       drop the provider's own credential.
 - [ ] Never trust inbound `X-Kedge-*` headers in the provider — the backend
       proxy strips and re-injects them; treat them as hub-asserted only.
+- [ ] To reach **another provider**, bind its `APIExport` and call its CRs /
+      VW subresources as the caller (contract 3). Never hold a credential
+      into another provider's backend (runtime cluster, DB, internal
+      Service) or hardcode its backend URL.
 
 ---