Skip to content

Commit 6f5aacc

Browse files
mjudeikis-botOpenClaw Botmjudeikis
authored
test: e2e tests for status.url, k8s proxy access, and SSH edge URL (#73)
* test: add e2e tests for status.url, k8s proxy, and SSH edge URL - Add EdgeURLSet: verifies edge.status.url populated for kubernetes-type - Add K8sProxyAccess: verifies kubectl works against status.url - Add SSHEdgeURLSet: verifies status.url set for server-type edges - Fix AgentEdgeJoin: use status.url for proxy access assertion - Add GetEdgeURL + KubectlWithURL framework helpers - Fix mount_reconciler: set SSH URL for server-type edges instead of clearing - Sync CRD YAML: update URL field description to match current types Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * test: fix e2e tests - verified passing locally Fixes all failures in the SSH and standalone e2e suites. ## Root causes fixed ### 1. Port conflict: KEDGE_HUB_API_SERVER_PORT (framework/cluster.go) Port 6443 was occupied by kcp on this host. Add KEDGE_HUB_API_SERVER_PORT env-var support so can be told to use a different API server port (e.g. 6444). All SetupClusters* variants now forward the flag. ### 2. SSH WebSocket URL double-concatenation (pkg/cli/cmd/ssh.go) buildSSHWebSocketURL was prepending the hub base path on top of the full status.URL, producing nonsense URLs like: wss://hub:8443/services/edges-proxyhttps://hub:8443/.../ssh/ssh Now it simply parses status.URL (already the complete URL) and converts the scheme to wss. ### 3. Cluster name mismatch for static-token agents (tunneler.go / agent.go) extractClusterNameFromToken returned "default" for non-JWT static tokens. The hub's mount_reconciler embeds the real kcp cluster name (e.g. "1tww43gelbj45g0k") in status.URL; the agent was registering its reverse-tunnel under the key edges/default/... causing the SSH and k8s proxy handlers to return 502. Fix: add SplitBaseAndCluster(hubURL) which extracts the cluster from the kubeconfig server URL (set by static-token login as https://hub:8443/clusters/<id>). Priority: explicit --cluster flag > URL-embedded cluster > JWT claim > "default" fallback. Also add --cluster flag to for the Docker container case. ### 4. ServerProcess uses hub kubeconfig (test/e2e/framework/server_process.go) SSH server-mode agents were started with bare --hub-url (no cluster path). Add HubKubeconfig field: when set, use --hub-kubeconfig + --tunnel-url so the agent gets the cluster-scoped server URL and SplitBaseAndCluster picks up the correct cluster ID. ### 5. DialSSH hardcoded /clusters/default (framework/ssh_client.go) Look up edge.Status.URL (which has the correct cluster) then convert https→wss instead of constructing the URL from scratch with a hardcoded "default" cluster path. ### 6. VirtualWorkload siteSelector → edgeSelector (cases/multisite.go) PlacementSpec.EdgeSelector is the actual field name; siteSelector caused strict-decoding 400 errors for LabelBasedScheduling, WorkloadIsolation, SiteFailoverIsolation, SiteReconnect tests. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * test: add functional e2e tests for workload deployment and k8s proxy - WorkloadDeployment: VirtualWorkload → Placement → resource on edge cluster - K8sProxyWrite: kubectl apply via edges-proxy creates resource on edge - K8sProxyExec: kubectl exec via edges-proxy (tests upgrade/hijack path) Also fix a bug in edgesHandleK8sUpgrade: the request URL path was not being rewritten to the /k8s/... form before forwarding to the agent, causing the agent's mux to return 404 for exec/port-forward requests. Now we call extractEdgeK8sPath before r.Write(deviceConn). All tested locally against real kind clusters. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix: rename TestOIDCUserCanListSites → TestOIDCUserCanListEdges, use kedge edge list kedge site list was removed; the OIDC test must use kedge edge list. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix: rename --site-name → --edge-name, PlacementSpec.SiteName → EdgeName Addresses PR #73 review comment. Sites are gone; use edge terminology: - CLI flag --site-name → --edge-name (deprecated alias kept) - PlacementSpec.SiteName json:siteName → EdgeName json:edgeName - All scheduler/aggregator/workload reconciler references updated - Test framework updated Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix(e2e): use edgeName json field in WaitForPlacement kubectl query .spec.siteName was renamed to .spec.edgeName when PlacementSpec was migrated from Site to Edge CRD. Update both WaitForPlacement and WaitForNoPlacement custom-columns queries to match. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * refactor: remove all stale site/siteName references - Drop --site-name and --mode deprecated CLI flags (no backward compat) - Remove AgentModeSite, AgentMode, AgentModeServer compat constants/types - Rename kedge.faros.sh/site label -> kedge.faros.sh/edge in scheduler and agent workload reconciler - Rename siteName fields/params -> edgeName throughout agent and test packages - Fix kedge get placements: column SITE->EDGE, spec.siteName->spec.edgeName - Fix WaitForPlacement/WaitForNoPlacement: custom-columns SITE->EDGE - Fix Helm dev plugin examples: siteName->edgeName, site-kubeconfig->edge-kubeconfig - Update 'kedge site list/get' examples -> 'kedge edge list/get' Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * refactor: replace remaining site references with edge throughout codebase Update all remaining "site" references from the old Site CRD concept: - Rename test data edge names (site-1 → edge-1, etc.) - Rename functions SiteFailoverIsolation → EdgeFailoverIsolation, etc. - Rename constants msSite1 → msEdge1, etc. - Fix user-visible strings in dev plugin (site kubeconfig → edge kubeconfig) - Update all comments and package docs - Rename test functions NoMatchingSites → NoMatchingEdges, etc. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * chore: regenerate CRD manifests after virtualworkload doc update Update generated CRDs and APIResourceSchema to reflect description changes: 'across sites' -> 'across edges'. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> --------- Co-authored-by: OpenClaw Bot <bot@openclaw.ai> Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt>
1 parent eea5d12 commit 6f5aacc

46 files changed

Lines changed: 1333 additions & 390 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

apis/kedge/v1alpha1/types_placement.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -23,7 +23,7 @@ import (
2323

2424
// +genclient
2525
// +kubebuilder:subresource:status
26-
// +kubebuilder:printcolumn:name="Edge",type="string",JSONPath=".spec.siteName"
26+
// +kubebuilder:printcolumn:name="Edge",type="string",JSONPath=".spec.edgeName"
2727
// +kubebuilder:printcolumn:name="Phase",type="string",JSONPath=".status.phase"
2828
// +kubebuilder:printcolumn:name="Ready",type="integer",JSONPath=".status.readyReplicas"
2929
// +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp"
@@ -49,7 +49,7 @@ type PlacementList struct {
4949
// PlacementObjSpec defines the desired state of a Placement.
5050
type PlacementObjSpec struct {
5151
WorkloadRef corev1.ObjectReference `json:"workloadRef"`
52-
SiteName string `json:"siteName"`
52+
EdgeName string `json:"edgeName"`
5353
Replicas *int32 `json:"replicas,omitempty"`
5454
}
5555

apis/kedge/v1alpha1/types_virtualworkload.go

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ const (
3131
VirtualWorkloadPhaseUnknown VirtualWorkloadPhase = "Unknown"
3232
)
3333

34-
// PlacementStrategy defines how workloads are placed across sites.
34+
// PlacementStrategy defines how workloads are placed across edges.
3535
type PlacementStrategy string
3636

3737
const (
@@ -48,7 +48,7 @@ const (
4848
// +kubebuilder:printcolumn:name="Age",type="date",JSONPath=".metadata.creationTimestamp"
4949
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object
5050

51-
// VirtualWorkload describes a workload to be deployed across sites.
51+
// VirtualWorkload describes a workload to be deployed across edges.
5252
type VirtualWorkload struct {
5353
metav1.TypeMeta `json:",inline"`
5454
metav1.ObjectMeta `json:"metadata,omitempty"`

cmd/kedge-agent/main.go

Lines changed: 3 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -35,7 +35,7 @@ func main() {
3535

3636
cmd := &cobra.Command{
3737
Use: "kedge-agent",
38-
Short: "Kedge agent - connects a site to the hub via reverse tunnel",
38+
Short: "Kedge agent - connects an edge to the hub via reverse tunnel",
3939
RunE: func(cmd *cobra.Command, args []string) error {
4040
ctx, cancel := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
4141
defer cancel()
@@ -54,18 +54,12 @@ func main() {
5454
cmd.Flags().StringVar(&opts.HubContext, "hub-context", "", "Kubeconfig context for hub cluster")
5555
cmd.Flags().StringVar(&opts.TunnelURL, "tunnel-url", "", "Hub tunnel URL (defaults to hub URL)")
5656
cmd.Flags().StringVar(&opts.Token, "token", "", "Bootstrap token")
57-
cmd.Flags().StringVar(&opts.SiteName, "site-name", "", "Name of this site")
57+
cmd.Flags().StringVar(&opts.EdgeName, "edge-name", "", "Name of this edge")
5858
cmd.Flags().StringVar(&opts.Kubeconfig, "kubeconfig", "", "Path to target cluster kubeconfig")
5959
cmd.Flags().StringVar(&opts.Context, "context", "", "Kubeconfig context to use")
60-
cmd.Flags().StringToStringVar(&opts.Labels, "labels", nil, "Labels for this site (key=value pairs)")
60+
cmd.Flags().StringToStringVar(&opts.Labels, "labels", nil, "Labels for this edge (key=value pairs)")
6161
cmd.Flags().StringVar((*string)(&opts.Type), "type", string(agent.AgentTypeKubernetes),
6262
"Edge type: 'kubernetes' (k8s cluster) or 'server' (bare-metal/systemd host)")
63-
// --mode is a deprecated alias for --type; kept for backward compatibility.
64-
cmd.Flags().StringVar((*string)(&opts.Mode), "mode", "", //nolint:staticcheck
65-
"Deprecated: use --type. Agent mode: 'site' (→ kubernetes) or 'server'")
66-
if err := cmd.Flags().MarkDeprecated("mode", "use --type instead (kubernetes|server)"); err != nil {
67-
klog.Warning("failed to mark --mode as deprecated: ", err)
68-
}
6963
cmd.Flags().BoolVar(&opts.InsecureSkipTLSVerify, "insecure-skip-tls-verify", false, "Skip TLS certificate verification for hub connection (dev/test only)")
7064
cmd.Flags().IntVar(&opts.SSHProxyPort, "ssh-proxy-port", 22, "Local SSH daemon port to proxy connections to")
7165
cmd.Flags().StringVar(&opts.SSHUser, "ssh-user", "", "SSH username for server-type edges (default: current user)")

config/crds/kedge.faros.sh_edges.yaml

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -117,8 +117,10 @@ spec:
117117
properties:
118118
URL:
119119
description: |-
120-
WorkspaceURL is the virtual workspace URL for this edge.
121-
Only set for type=kubernetes edges.
120+
URL is the proxy URL path for accessing this edge via the hub.
121+
Format: /clusters/{cluster}/apis/kedge.faros.sh/v1alpha1/edges/{name}
122+
For kubernetes edges, append /k8s/ for K8s API proxy.
123+
For server edges, append /ssh for SSH WebSocket terminal.
122124
type: string
123125
connected:
124126
description: Connected indicates whether the edge agent currently

config/crds/kedge.faros.sh_placements.yaml

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,7 @@ spec:
1515
scope: Namespaced
1616
versions:
1717
- additionalPrinterColumns:
18-
- jsonPath: .spec.siteName
18+
- jsonPath: .spec.edgeName
1919
name: Edge
2020
type: string
2121
- jsonPath: .status.phase
@@ -52,11 +52,11 @@ spec:
5252
spec:
5353
description: PlacementObjSpec defines the desired state of a Placement.
5454
properties:
55+
edgeName:
56+
type: string
5557
replicas:
5658
format: int32
5759
type: integer
58-
siteName:
59-
type: string
6060
workloadRef:
6161
description: ObjectReference contains enough information to let you
6262
inspect or modify the referred object.
@@ -102,7 +102,7 @@ spec:
102102
type: object
103103
x-kubernetes-map-type: atomic
104104
required:
105-
- siteName
105+
- edgeName
106106
- workloadRef
107107
type: object
108108
status:

config/crds/kedge.faros.sh_virtualworkloads.yaml

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -32,7 +32,7 @@ spec:
3232
name: v1alpha1
3333
schema:
3434
openAPIV3Schema:
35-
description: VirtualWorkload describes a workload to be deployed across sites.
35+
description: VirtualWorkload describes a workload to be deployed across edges.
3636
properties:
3737
apiVersion:
3838
description: |-
@@ -119,7 +119,7 @@ spec:
119119
x-kubernetes-map-type: atomic
120120
strategy:
121121
description: PlacementStrategy defines how workloads are placed
122-
across sites.
122+
across edges.
123123
type: string
124124
type: object
125125
replicas:

config/kcp/apiexport-kedge.faros.sh.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -65,12 +65,12 @@ spec:
6565
resources:
6666
- group: kedge.faros.sh
6767
name: edges
68-
schema: v260228-0f5e1a9.edges.kedge.faros.sh
68+
schema: v260228-eea5d12.edges.kedge.faros.sh
6969
storage:
7070
crd: {}
7171
- group: kedge.faros.sh
7272
name: placements
73-
schema: v260227-f675e28.placements.kedge.faros.sh
73+
schema: v260301-c08d8fa.placements.kedge.faros.sh
7474
storage:
7575
crd: {}
7676
- group: kedge.faros.sh
@@ -80,7 +80,7 @@ spec:
8080
crd: {}
8181
- group: kedge.faros.sh
8282
name: virtualworkloads
83-
schema: v260227-f675e28.virtualworkloads.kedge.faros.sh
83+
schema: v260301-1a00c73.virtualworkloads.kedge.faros.sh
8484
storage:
8585
crd: {}
8686
status: {}

config/kcp/apiresourceschema-edges.kedge.faros.sh.yaml

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
apiVersion: apis.kcp.io/v1alpha1
22
kind: APIResourceSchema
33
metadata:
4-
name: v260228-0f5e1a9.edges.kedge.faros.sh
4+
name: v260228-eea5d12.edges.kedge.faros.sh
55
spec:
66
group: kedge.faros.sh
77
names:
@@ -113,8 +113,10 @@ spec:
113113
properties:
114114
URL:
115115
description: |-
116-
WorkspaceURL is the virtual workspace URL for this edge.
117-
Only set for type=kubernetes edges.
116+
URL is the proxy URL path for accessing this edge via the hub.
117+
Format: /clusters/{cluster}/apis/kedge.faros.sh/v1alpha1/edges/{name}
118+
For kubernetes edges, append /k8s/ for K8s API proxy.
119+
For server edges, append /ssh for SSH WebSocket terminal.
118120
type: string
119121
connected:
120122
description: Connected indicates whether the edge agent currently has

config/kcp/apiresourceschema-placements.kedge.faros.sh.yaml

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
apiVersion: apis.kcp.io/v1alpha1
22
kind: APIResourceSchema
33
metadata:
4-
name: v260227-f675e28.placements.kedge.faros.sh
4+
name: v260301-c08d8fa.placements.kedge.faros.sh
55
spec:
66
group: kedge.faros.sh
77
names:
@@ -12,7 +12,7 @@ spec:
1212
scope: Namespaced
1313
versions:
1414
- additionalPrinterColumns:
15-
- jsonPath: .spec.siteName
15+
- jsonPath: .spec.edgeName
1616
name: Edge
1717
type: string
1818
- jsonPath: .status.phase
@@ -48,11 +48,11 @@ spec:
4848
spec:
4949
description: PlacementObjSpec defines the desired state of a Placement.
5050
properties:
51+
edgeName:
52+
type: string
5153
replicas:
5254
format: int32
5355
type: integer
54-
siteName:
55-
type: string
5656
workloadRef:
5757
description: ObjectReference contains enough information to let you
5858
inspect or modify the referred object.
@@ -98,7 +98,7 @@ spec:
9898
type: object
9999
x-kubernetes-map-type: atomic
100100
required:
101-
- siteName
101+
- edgeName
102102
- workloadRef
103103
type: object
104104
status:

config/kcp/apiresourceschema-virtualworkloads.kedge.faros.sh.yaml

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
apiVersion: apis.kcp.io/v1alpha1
22
kind: APIResourceSchema
33
metadata:
4-
name: v260227-f675e28.virtualworkloads.kedge.faros.sh
4+
name: v260301-1a00c73.virtualworkloads.kedge.faros.sh
55
spec:
66
group: kedge.faros.sh
77
names:
@@ -28,7 +28,7 @@ spec:
2828
type: date
2929
name: v1alpha1
3030
schema:
31-
description: VirtualWorkload describes a workload to be deployed across sites.
31+
description: VirtualWorkload describes a workload to be deployed across edges.
3232
properties:
3333
apiVersion:
3434
description: |-
@@ -115,7 +115,7 @@ spec:
115115
x-kubernetes-map-type: atomic
116116
strategy:
117117
description: PlacementStrategy defines how workloads are placed
118-
across sites.
118+
across edges.
119119
type: string
120120
type: object
121121
replicas:

0 commit comments

Comments
 (0)