feat(zoneproxy): implement xds changes and identity #15870
feat(zoneproxy): implement xds changes and identity #15870lukidzi wants to merge 47 commits intokumahq:masterfrom
Conversation
Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>
Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>
Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>
Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>
Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>
…one proxy Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>
Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>
… support-k8s-controller
Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>
Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>
Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>
Reviewer Checklist🔍 Each of these sections need to be checked by the reviewer of the PR 🔍:
|
Signed-off-by: Ilya Lobkov <ilya.lobkov@konghq.com>
Signed-off-by: Lukasz Dziedziak <lukidzi@gmail.com>
… into xds-config-zone-proxy
There was a problem hiding this comment.
Pull request overview
Implements xDS support for mesh-scoped “zone proxy” listeners embedded in regular Dataplane resources (zone ingress/egress), including endpoint resolution updates and workload-identity-based mTLS handling for egress.
Changes:
- Added
ZoneProxyListenerGeneratorto generate listeners/CDS/EDS for embedded zone ingress/egress listeners and wired it into the default proxy profile. - Extended topology/mesh context building to support MeshZoneAddress prioritization and zone-egress instance resolution (including SANs for workload identity).
- Added test framework deployments and multizone E2E coverage for embedded zone proxy connectivity/metrics.
Reviewed changes
Copilot reviewed 33 out of 33 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| test/server/cmd/echo.go | Simplifies Prometheus metrics handler wiring for the echo test server. |
| test/framework/universal_controlplane.go | Adjusts dataplane token generation to omit tags when service is empty. |
| test/framework/universal_cluster.go | Adds helper to start kuma-dp for a regular Dataplane manifest with embedded listeners and expose Envoy admin tunnel. |
| test/framework/k8s_controlplane.go | Adjusts dataplane token generation to omit tags when service is empty. |
| test/framework/k8s_cluster.go | Adds generic Envoy admin tunnel getter for arbitrary apps in a namespace. |
| test/framework/deployments/zoneproxy/universal.go | Adds Universal deployment for embedded zone ingress/egress dataplane listeners. |
| test/framework/deployments/zoneproxy/kubernetes.go | Adds Kubernetes deployment (Deployment+Service) for embedded zone ingress/egress dataplane listeners. |
| test/framework/deployments/zoneproxy/deployment.go | Introduces cross-env install wrapper for zoneproxy deployments. |
| test/framework/dataplane_template.go | Extends dataplane template rendering to support a networking.listeners block. |
| test/e2e_env/multizone/multizone_suite_test.go | Registers new multizone E2E suite for MeshProxy connectivity. |
| test/e2e_env/multizone/meshproxy/connectivity.go | Adds multizone E2E test validating cross-zone routing via embedded zone ingress/egress and stats verification. |
| pkg/xds/topology/outbound_test.go | Updates endpoint-map tests for new BuildEdsEndpointMap signature and zone-egress instance handling. |
| pkg/xds/topology/outbound.go | Adds endpoint-map helpers for embedded zone ingress/egress and updates EDS map logic to use MeshZoneAddress + resolved zone-egress instances. |
| pkg/xds/topology/ingress_outbound_test.go | Updates ingress endpoint map tests to pass resolved zone-egress instances. |
| pkg/xds/sync/ingress_proxy_builder.go | Renames mesh resource container type used by ingress proxy builder. |
| pkg/xds/sync/dataplane_proxy_builder.go | Populates Proxy.DataplaneZoneListeners when dataplane has embedded zone proxy listeners. |
| pkg/xds/generator/zone_proxy_listener_generator_test.go | Adds golden tests for embedded zone ingress/egress generator behavior (incl. WI/no-WI cases). |
| pkg/xds/generator/zone_proxy_listener_generator.go | Implements generator for embedded zone ingress/egress listeners (incl. WI-based egress mTLS). |
| pkg/xds/generator/testdata/zone-proxy-listener/ingress-meshservice-zone.envoy.golden.yaml | Golden output for ingress listener exposing a local MeshService. |
| pkg/xds/generator/testdata/zone-proxy-listener/ingress-meshservice-filtered.envoy.golden.yaml | Golden output verifying non-local MeshServices are filtered out. |
| pkg/xds/generator/testdata/zone-proxy-listener/egress-no-identity.envoy.golden.yaml | Golden output verifying egress is no-op without WorkloadIdentity. |
| pkg/xds/generator/testdata/zone-proxy-listener/egress-meshexternalservice.envoy.golden.yaml | Golden output for egress listener + cluster with WI mTLS. |
| pkg/xds/generator/proxy_template.go | Adds the new zone-proxy listener generator to the default proxy generator chain. |
| pkg/xds/generator/ingress_generator_test.go | Updates ingress generator tests to use renamed MeshProxyResources type. |
| pkg/xds/context/resources.go | Adds accessor for MeshZoneAddress resources in mesh local resources. |
| pkg/xds/context/mesh_context_builder.go | Resolves zone-egress instances (incl. SANs) and builds shared endpoint maps for embedded zone proxy listeners. |
| pkg/xds/context/context.go | Extends MeshContext with resolved zone-egress instances (SANs) and shared endpoint maps; adds ZoneEgressSANs(). |
| pkg/plugins/runtime/gateway/cluster_generator.go | Updates upstream TLS context creation call signature. |
| pkg/plugins/policies/core/xds/meshroute/clusters.go | Refactors UpstreamTLSContext API and adds WI-mode handling for MeshExternalService via zone-egress SAN validation. |
| pkg/core/xds/types.go | Adds ZoneEgressInstance, DataplaneZoneListeners, and renames mesh resource container type to MeshProxyResources. |
| pkg/core/resources/apis/mesh/dataplane_overview_helpers.go | Updates dataplane status computation to account for embedded zone proxy listener readiness. |
| pkg/core/naming/contextual.go | Adds contextual listener naming helpers for embedded zone ingress/egress listeners. |
| api/mesh/v1alpha1/dataplane_helpers.go | Adds helpers to detect embedded zone proxy listeners and list ready zone ingress/egress listeners. |
There was a problem hiding this comment.
This is quite a big PR, I tried to review this as carefully as I can. Nothing obviously wrong stands out maybe apart from #15870 (comment) .
Mostly questions / small things, if you we conclude that there's nothing that needs to change I'll change this to 👍
| egressSANs := meshCtx.ZoneEgressSANs() | ||
| if len(egressSANs) == 0 { | ||
| continue | ||
| } |
There was a problem hiding this comment.
If WorkloadIdentity is on but ZoneEgressSANs() returns empty, every MeshExternalService cluster gets silently skipped. resolveZoneEgresses already logs errors for trust domain / SPIFFE ID failures. On the client side I don't think there's anything connecting those CP logs to the missing clusters. Is it worth adding a warning here?
| // Only expose services local to this zone. | ||
| localMS := &meshservice_api.MeshServiceResourceList{} | ||
| for _, ms := range meshResources.MeshServices().GetItems() { | ||
| if lbls := ms.GetMeta().GetLabels(); lbls == nil || lbls[mesh_proto.ZoneTag] == "" || lbls[mesh_proto.ZoneTag] == cp.Zone { |
There was a problem hiding this comment.
MeshService already has IsLocalMeshService() but it checks origin, not zone. Could we add something like IsLocalToZone(zone string) bool that combines both checks? Would make this condition more readable. WDYT?
| Resource: builders.Mesh().WithName("default").WithMeshServicesEnabled(mesh_proto.Mesh_MeshServices_Exclusive).Build(), | ||
| DataplaneZoneEgressEndpointMap: core_xds.EgressEndpointMap{ | ||
| unifiedSvcName: core_xds.EgressEndpointGroup{ | ||
| Protocol: core_meta.ProtocolHTTP, |
There was a problem hiding this comment.
are we testing TCP anywhere?
| trustZone2 := getMeshTrust(multizone.KubeZone2.Name(), Config.KumaNamespace) | ||
| installTrustToGlobal(trustZone2, multizone.KubeZone2.Name()) | ||
|
|
||
| trustZone4 := getMeshTrust(multizone.UniZone1.Name()) |
|
|
||
| // GetEnvoyAdminTunnel creates or returns an Envoy admin tunnel for any named | ||
| // app in a given namespace. | ||
| func (c *K8sCluster) GetEnvoyAdminTunnel(appName, namespace string) envoy_admin.Tunnel { |
There was a problem hiding this comment.
GetZoneEgressEnvoyTunnel and GetZoneIngressEnvoyTunnel right above do the same thing with hardcoded values. Could be one-liners wrapping this. Could be a separate PR.
|
|
||
| // GetAppEnvoyTunnel returns the Envoy admin tunnel for a named app (e.g. a | ||
| // zone proxy deployed via CreateDataplaneProxy). | ||
| func (c *UniversalCluster) GetAppEnvoyTunnel(name string) envoy_admin.Tunnel { |
There was a problem hiding this comment.
probably similar situation as in test/framework/k8s_cluster.go
…e time Signed-off-by: Ilya Lobkov <ilya.lobkov@konghq.com>
Signed-off-by: Ilya Lobkov <ilya.lobkov@konghq.com>
Motivation
This PR implements xDS configuration support for mesh-scoped zone proxies: zone ingress and egress listeners embedded directly in a regular Dataplane resource.
Implementation information
BuildDataplaneZoneIngressEndpointMap: local MeshService/MeshMultiZoneService endpoints for the ingress listenerBuildDataplaneZoneEgressEndpointMap: MeshExternalService endpoints reachable from the zone for the egress listenerSupporting documentation
Close: https://github.com/Kong/kong-mesh/issues/9265
Close: https://github.com/Kong/kong-mesh/issues/9215
Close: https://github.com/Kong/kong-mesh/issues/9249