Skip to content

Commit 55198fc

Browse files
mjudeikis-botmjudeikis-botmjudeikis
authored
refactor: drop kedge-agent binary; use kedge CLI in agent container image (#141)
* refactor: drop kedge-agent binary; use kedge CLI in agent container image - cmd/kedge-agent/ removed; agent container now runs kedge agent run - Dockerfile.agent builds cmd/kedge/, ENTRYPOINT [/kedge, agent, run] - kedge agent run already accepts --hub-kubeconfig for SA-backed auth - e2e framework Agent.Start() updated to use bin/kedge agent run - Makefile build-agent now builds bin/kedge (alias for build-kedge) - dev-run-edge updated to use kedge agent run with correct flag names - Closes #140 Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix: use --hub-insecure-skip-tls-verify in Helm chart and agent join template kedge-agent used --insecure-skip-tls-verify but kedge agent run uses --hub-insecure-skip-tls-verify. After dropping kedge-agent in favour of the kedge CLI binary, the Helm chart deployment template and the Deployment generated by 'kedge agent join' were passing the old flag name, causing CrashLoopBackOff: 'unknown flag: --insecure-skip-tls-verify'. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix: load hub kubeconfig from in-cluster Secret on pod restart When the agent runs as a Kubernetes Deployment with a bootstrap join token, the flow is: 1. First boot: connect with token → hub sends back kubeconfig 2. Agent saves kubeconfig to in-cluster Secret (SaveKubeconfigToSecret) then calls os.Exit(1) to force a pod restart 3. Second boot (token still in Deployment args): runAgentForeground must load the Secret BEFORE falling back to the bootstrap token Previously step 3 only checked the local filesystem (ephemeral in a container) and the durable token config, meaning it always fell back to the bootstrap token — which the hub had already cleared — causing 'invalid join token' rejections on every restart. Fix: add LoadKubeconfigFromSecret to incluster.go and call it at the top of runAgentForeground when IsInCluster() is true. If the Secret exists the kubeconfig is written to a temp file and used directly, overriding the bootstrap token in the Deployment args. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix: restore --hub-insecure-skip-tls-verify in agentJoinKubernetes Deployment template The cp in the previous commit accidentally re-introduced --insecure-skip-tls-verify (old kedge-agent flag) in the token-mode and kubeconfig-mode Deployment args generated by 'kedge agent join --type kubernetes'. This caused the Deployment pod to crash immediately with 'unknown flag'. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> --------- Co-authored-by: mjudeikis-bot <ai@faros.sh> Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt>
1 parent c584751 commit 55198fc

8 files changed

Lines changed: 78 additions & 245 deletions

File tree

‎Makefile‎

Lines changed: 9 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -49,16 +49,17 @@ ldflags: ## Print ldflags for goreleaser
4949

5050
all: build
5151

52-
build: build-kedge build-hub build-agent
52+
build: build-kedge build-hub
5353

5454
build-kedge:
5555
go build $(GOFLAGS) -ldflags "$(LDFLAGS)" -o $(BINDIR)/kedge ./cmd/kedge/
5656

5757
build-hub:
5858
go build $(GOFLAGS) -ldflags "$(LDFLAGS)" -o $(BINDIR)/kedge-hub ./cmd/kedge-hub/
5959

60-
build-agent:
61-
go build $(GOFLAGS) -ldflags "$(LDFLAGS)" -o $(BINDIR)/kedge-agent ./cmd/kedge-agent/
60+
# build-agent is an alias for build-kedge: the agent container image now ships
61+
# the kedge CLI binary (cmd/kedge/) with ENTRYPOINT [/kedge, agent, run].
62+
build-agent: build-kedge
6263

6364
test:
6465
go test $(shell go list ./... | grep -v '/test/e2e')
@@ -156,12 +157,12 @@ TYPE ?= kubernetes
156157
dev-edge-create: build-kedge ## Create an Edge resource: TYPE=kubernetes (default) or TYPE=server
157158
PATH=$(CURDIR)/$(BINDIR):$$PATH BINDIR=$(CURDIR)/$(BINDIR) hack/scripts/dev-edge-setup.sh $(DEV_EDGE_NAME) $(TYPE) "env=dev,provider=local"
158159

159-
dev-run-edge: build-agent ## Run the edge agent: TYPE=kubernetes (default) or TYPE=server
160+
dev-run-edge: build-kedge ## Run the edge agent: TYPE=kubernetes (default) or TYPE=server
160161
@test -f .env.edge || (echo "Run 'make dev-edge-create [TYPE=$(TYPE)]' first"; exit 1)
161162
ifeq ($(TYPE),server)
162-
$(BINDIR)/kedge-agent \
163+
$(BINDIR)/kedge agent run \
163164
--hub-url=https://localhost:8443 \
164-
--insecure-skip-tls-verify \
165+
--hub-insecure-skip-tls-verify \
165166
--token=$(KEDGE_EDGE_JOIN_TOKEN) \
166167
--tunnel-url=https://localhost:8443 \
167168
--edge-name=$(DEV_EDGE_NAME) \
@@ -172,9 +173,9 @@ ifeq ($(TYPE),server)
172173
--ssh-password=password
173174
else
174175
hack/scripts/ensure-kind-cluster.sh
175-
$(BINDIR)/kedge-agent \
176+
$(BINDIR)/kedge agent run \
176177
--hub-url=https://localhost:8443 \
177-
--insecure-skip-tls-verify \
178+
--hub-insecure-skip-tls-verify \
178179
--token=$(KEDGE_EDGE_JOIN_TOKEN) \
179180
--tunnel-url=https://localhost:8443 \
180181
--edge-name=$(DEV_EDGE_NAME) \

‎cmd/kedge-agent/incluster.go‎

Lines changed: 0 additions & 101 deletions
This file was deleted.

‎cmd/kedge-agent/main.go‎

Lines changed: 0 additions & 126 deletions
This file was deleted.

‎deploy/Dockerfile.agent‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,10 +18,10 @@ RUN CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} go build \
1818
-X github.com/faroshq/faros-kedge/pkg/cli/cmd.version=${VERSION} \
1919
-X github.com/faroshq/faros-kedge/pkg/cli/cmd.gitCommit=${GIT_COMMIT} \
2020
-X github.com/faroshq/faros-kedge/pkg/cli/cmd.buildDate=${BUILD_DATE}" \
21-
-o /kedge-agent ./cmd/kedge-agent/
21+
-o /kedge ./cmd/kedge/
2222

2323
FROM alpine:3.21
2424
RUN apk add --no-cache ca-certificates
25-
COPY --from=builder /kedge-agent /kedge-agent
25+
COPY --from=builder /kedge /kedge
2626
USER 65534:65534
27-
ENTRYPOINT ["/kedge-agent"]
27+
ENTRYPOINT ["/kedge", "agent", "run"]

‎deploy/charts/kedge-agent/templates/deployment.yaml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -50,7 +50,7 @@ spec:
5050
- --hub-url={{ .Values.agent.hub.url }}
5151
{{- end }}
5252
{{- if .Values.agent.hub.insecureSkipTLSVerify }}
53-
- --insecure-skip-tls-verify
53+
- --hub-insecure-skip-tls-verify
5454
{{- end }}
5555
{{- if .Values.agent.tunnelURL }}
5656
- --tunnel-url={{ .Values.agent.tunnelURL }}

‎pkg/agent/incluster.go‎

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -71,6 +71,32 @@ func decodeKubeconfigB64(kubeconfigB64 string) (string, error) {
7171
return string(b), nil
7272
}
7373

74+
// LoadKubeconfigFromSecret reads the hub kubeconfig from the in-cluster Secret.
75+
// Returns ("", nil) when the Secret does not exist yet (first boot before token exchange).
76+
func LoadKubeconfigFromSecret(edgeName string) (string, error) {
77+
cs, err := newInClusterKubernetesClient()
78+
if err != nil {
79+
return "", err
80+
}
81+
secretName := AgentKubeconfigSecretName(edgeName)
82+
secret, err := cs.CoreV1().Secrets(inClusterNamespace).Get(
83+
context.Background(),
84+
secretName,
85+
metav1.GetOptions{},
86+
)
87+
if apierrors.IsNotFound(err) {
88+
return "", nil
89+
}
90+
if err != nil {
91+
return "", fmt.Errorf("getting kubeconfig secret: %w", err)
92+
}
93+
data, ok := secret.Data[kubeconfigSecretKey]
94+
if !ok || len(data) == 0 {
95+
return "", nil
96+
}
97+
return string(data), nil
98+
}
99+
74100
// SaveKubeconfigToSecret writes the hub kubeconfig to the in-cluster Secret so
75101
// that it survives a pod restart.
76102
func SaveKubeconfigToSecret(edgeName, kubeconfigData string) error {

‎pkg/cli/cmd/agent.go‎

Lines changed: 33 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -83,6 +83,37 @@ func runAgentForeground(ctx context.Context, opts *agent.Options) error {
8383
// try to load a previously saved kubeconfig or durable token from disk.
8484
// This allows the agent to reconnect after the first successful join
8585
// without requiring the operator to re-supply the bootstrap join token.
86+
if opts.HubKubeconfig == "" && opts.EdgeName != "" {
87+
// In-cluster mode: check the kubeconfig Secret FIRST. On the initial
88+
// run the Secret does not exist yet so we fall through to token-based
89+
// bootstrap. After the first successful join the hub kubeconfig is
90+
// persisted in the Secret (see SaveKubeconfigToSecret / os.Exit(1)
91+
// restart dance). On subsequent restarts we load the Secret here so
92+
// the agent never tries to re-use the already-cleared bootstrap token.
93+
if agent.IsInCluster() {
94+
kubeconfigData, err := agent.LoadKubeconfigFromSecret(opts.EdgeName)
95+
if err != nil {
96+
logger.Info("Could not load in-cluster kubeconfig Secret (will try other sources)", "err", err)
97+
} else if kubeconfigData != "" {
98+
// Write to a temp file so the rest of the startup path can use it.
99+
tmpFile, err := os.CreateTemp("", "kedge-agent-kubeconfig-*")
100+
if err != nil {
101+
return fmt.Errorf("creating temp kubeconfig file: %w", err)
102+
}
103+
if _, err := tmpFile.WriteString(kubeconfigData); err != nil {
104+
return fmt.Errorf("writing temp kubeconfig: %w", err)
105+
}
106+
if closeErr := tmpFile.Close(); closeErr != nil {
107+
return fmt.Errorf("closing temp kubeconfig file: %w", closeErr)
108+
}
109+
logger.Info("Using hub kubeconfig from in-cluster Secret", "edgeName", opts.EdgeName, "secret", agent.AgentKubeconfigSecretName(opts.EdgeName))
110+
opts.HubKubeconfig = tmpFile.Name()
111+
opts.Token = "" // Secret kubeconfig takes precedence over bootstrap token.
112+
opts.UsingSavedKubeconfig = true
113+
}
114+
}
115+
}
116+
86117
if opts.HubKubeconfig == "" && opts.EdgeName != "" {
87118
// Prefer a saved kubeconfig (from a previous join-token exchange) over
88119
// the bootstrap token. This allows the agent to reconnect after restart
@@ -342,7 +373,7 @@ metadata:
342373
deployArgs := fmt.Sprintf("--hub-url=%s --edge-name=%s --type=kubernetes --token=%s",
343374
hubURL, opts.EdgeName, opts.Token)
344375
if opts.InsecureSkipTLSVerify {
345-
deployArgs += " --insecure-skip-tls-verify"
376+
deployArgs += " --hub-insecure-skip-tls-verify"
346377
}
347378
if opts.Cluster != "" {
348379
deployArgs += " --cluster=" + opts.Cluster
@@ -404,7 +435,7 @@ stringData:
404435
// kedge-agent is a standalone binary; flags are passed directly (no subcommands).
405436
deployArgs := fmt.Sprintf("--hub-kubeconfig=/etc/kedge/hub.kubeconfig --edge-name=%s --type=kubernetes", opts.EdgeName)
406437
if opts.InsecureSkipTLSVerify {
407-
deployArgs += " --insecure-skip-tls-verify"
438+
deployArgs += " --hub-insecure-skip-tls-verify"
408439
}
409440
if opts.Cluster != "" {
410441
deployArgs += " --cluster=" + opts.Cluster

0 commit comments

Comments
 (0)