Skip to content

Commit 2c1fece

Browse files
mjudeikis-botMangirdas Judeikismjudeikis
authored
feat(e2e): multi-site agent support with --agent-count flag (#32) (#55)
* feat(e2e): multi-site agent support with --agent-count flag (#32) - Add --agent-count flag to 'kedge dev create/delete' (default 1, preserves existing behaviour; N>1 creates kind clusters with -1..-N suffixes) - Extend ClusterEnv with AgentClusters []AgentClusterInfo so the e2e framework can hand each test a kubeconfig per agent cluster - Add RunCmd / KubectlWithConfig package-level helpers to framework/client.go - Add test/e2e/cases/multisite.go with 6 feature builders: TwoAgentsJoin - both agents connect, both sites Ready (PASSES) LabelBasedScheduling - VW region selector → site-1 only WorkloadIsolation - site-1 VW has no placement on site-2 SiteFailoverIsolation - site-1 offline, site-2 placement intact SiteReconnect - site-1 reconnects, placement reappears SiteListAccuracyUnderChurn - site list reflects disconnect/reconnect - VW-scheduler tests auto-skip via requireVirtualWorkloadScheduler() probe when the embedded-KCP scheduler is not functional (avoids cascade failures) - Reuse authenticated client stored in context to avoid double Login in Teardown - Register all 6 tests in standalone, external_kcp, and oidc suites Closes #32 Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix: apply APIResourceSchemas before APIExport to fix VW scheduler (#56) Root cause: confighelpers.Bootstrap applied files in alphabetical order, so apiexport-kedge.faros.sh.yaml (which references the VirtualWorkload schema v260219-7bf8d83) was applied BEFORE apiresourceschema-virtualworkloads.kedge.faros.sh.yaml. KCP accepted the APIExport with an unresolvable schema reference, leaving the VirtualWorkload API unavailable until a later reconcile. The multicluster-provider's wildcard informer saw 'server could not find the requested resource' on initial LIST, retried with exponential backoff, and by the time it finally synced the e2e test's 2-minute WaitForPlacement window had expired. Fix: sort bootstrap files by dependency tier inside createResourcesFromFS: tier 0: apiresourceschema-* (must exist before APIExport references them) tier 1: apiexport-* (reference tier-0 schemas) tier 2: everything else (workspaces, etc.) Within each tier files are applied alphabetically; the ordering is transparent to callers of Bootstrap. Also: - Remove the requireVirtualWorkloadScheduler() skip probe from multisite tests; the scheduler is expected to work now that the bootstrap is fixed. - Add V(4) reconcile trace log to scheduler.Reconcile for future debugging. Closes #56 Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix: preserve site labels so VW scheduler can match them Two bugs caused the VirtualWorkload scheduler to never produce Placements: 1. agent.registerSite() overwrote existing site labels with the agent's own (initially-empty) label map. Any labels set via 'kedge site create --labels' were silently wiped on the agent's first reconcile. Fix: merge agent labels into existing labels rather than replace. 2. e2e framework launched agents without --labels, so agents had no labels to preserve even if the merge logic were correct. Fix: Agent.WithLabels() + pass matching labels in startMultisiteAgents. Also adds site-label + siteSelector diagnostic logging to the scheduler reconciler (Info level) so failures surface immediately in test runs. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix(scheduler): use mcr context for cluster name + RequeueAfter 30s for site reconnect - Use mccontext.ClusterFrom(ctx) in mapSiteToVirtualWorkloads (canonical cluster name from multicluster-runtime context, fall back to kcp annotation) - Add RequeueAfter: 30s so VWs are re-evaluated periodically, ensuring placements reappear after a site reconnects even if the watch event for the status-only Site update is missed Fixes TestSiteReconnect failure in standalone e2e suite. * chore: remove diagnostic logging from scheduler reconciler Revert the temporary info-level logging added to diagnose #56. Root cause (labels wiped by agent, and VW scheduler not triggered due to bootstrap ordering) is now fixed and confirmed by passing e2e tests. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix: add bin/ to PATH in CI so OIDC exec plugin can find kedge The OIDC kubeconfig returned by the hub uses an exec credential plugin (kedge get-token) instead of a static bearer token. When the test framework runs kubectl with this kubeconfig, kubectl spawns the plugin process -- but bin/kedge is not on PATH in the GitHub Actions runner, causing every kubectl invocation to fail with exec-plugin-not-found. The result: ExtractSiteKubeconfig polls for 5 minutes, never succeeds, and all multisite tests fail with 'timed out after 5m0s waiting for condition' in the OIDC suite. Fix: add $(pwd)/bin to GITHUB_PATH after make build in all e2e jobs so that the kedge binary is resolvable by the exec plugin. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix: use stored OIDC client in TwoAgentsJoin assess step In the OIDC suite, clusterEnv.HubKubeconfig is the kind admin kubeconfig (never overwritten by kedge login, unlike the standalone suite). Creating a KedgeClient from it points at the kind Kubernetes API which has no Sites CRD, so WaitForSiteReady polls forever and fails after 3 minutes. All other multisite assess functions already use ctx.Value(msClientKey{}) to get the authenticated client stored by startMultisiteAgents. Apply the same pattern to TwoAgentsJoin. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * fix(e2e): remove multisite tests from OIDC suite (static-token agents incompatible with Dex mode) * fix: remove redundant UpdateStatus in registerSite The initial UpdateStatus call used a bare Site object (no resourceVersion) which always failed with 'metadata.resourceVersion: Invalid value: 0'. It was also redundant: the heartbeat reporter's first sendHeartbeat() fires immediately on start and patches status.phase=Ready via MergePatch, which doesn't require resourceVersion. Remove the broken call; let the reporter handle all status updates. Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> * Revert "fix(e2e): remove multisite tests from OIDC suite (static-token agents incompatible with Dex mode)" This reverts commit 18dc324. --------- Co-authored-by: Mangirdas Judeikis <mjudeikis@gmail.com> Co-authored-by: Mangirdas Judeikis <mangirdas@judeikis.lt> Co-authored-by: mjudeikis-bot <mjudeikis-bot@users.noreply.github.com>
1 parent 30cb01f commit 2c1fece

13 files changed

Lines changed: 921 additions & 69 deletions

File tree

‎.github/workflows/e2e.yaml‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -38,6 +38,9 @@ jobs:
3838
- name: Build binaries
3939
run: make build
4040

41+
- name: Add bin/ to PATH
42+
run: echo "$(pwd)/bin" >> $GITHUB_PATH
43+
4144
- name: Build hub Docker image
4245
run: docker build -f deploy/Dockerfile.hub -t ghcr.io/faroshq/kedge-hub:test .
4346

@@ -92,6 +95,9 @@ jobs:
9295
- name: Build binaries
9396
run: make build
9497

98+
- name: Add bin/ to PATH
99+
run: echo "$(pwd)/bin" >> $GITHUB_PATH
100+
95101
- name: Build hub Docker image
96102
run: |
97103
docker build -f deploy/Dockerfile.hub \
@@ -145,6 +151,9 @@ jobs:
145151
- name: Build binaries
146152
run: make build
147153

154+
- name: Add bin/ to PATH
155+
run: echo "$(pwd)/bin" >> $GITHUB_PATH
156+
148157
- name: Build hub Docker image
149158
run: |
150159
docker build -f deploy/Dockerfile.hub \

‎pkg/agent/agent.go‎

Lines changed: 9 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -213,35 +213,21 @@ func (a *Agent) registerSite(ctx context.Context, client *kedgeclient.Client) er
213213
return fmt.Errorf("creating site: %w", err)
214214
}
215215
} else {
216-
// Update existing site labels
216+
// Merge agent labels into existing site labels; don't wipe labels the
217+
// agent doesn't own (e.g. user-assigned routing labels).
217218
logger.Info("Updating Site", "name", a.opts.SiteName)
218-
existing.Labels = a.opts.Labels
219+
if existing.Labels == nil {
220+
existing.Labels = make(map[string]string)
221+
}
222+
for k, v := range a.opts.Labels {
223+
existing.Labels[k] = v
224+
}
219225
_, err := client.Sites().Update(ctx, existing, metav1.UpdateOptions{})
220226
if err != nil {
221227
return fmt.Errorf("updating site: %w", err)
222228
}
223229
}
224230

225-
// Update status to connected
226-
now := metav1.Now()
227-
statusSite := &kedgev1alpha1.Site{
228-
TypeMeta: metav1.TypeMeta{
229-
APIVersion: kedgev1alpha1.SchemeGroupVersion.String(),
230-
Kind: "Site",
231-
},
232-
ObjectMeta: metav1.ObjectMeta{
233-
Name: a.opts.SiteName,
234-
},
235-
Status: kedgev1alpha1.SiteStatus{
236-
Phase: kedgev1alpha1.SitePhaseConnected,
237-
TunnelConnected: true,
238-
LastHeartbeatTime: &now,
239-
},
240-
}
241-
_, err = client.Sites().UpdateStatus(ctx, statusSite, metav1.UpdateOptions{})
242-
if err != nil {
243-
logger.Error(err, "Failed to update site status to connected")
244-
}
245-
231+
// Status is set to Ready by the heartbeat reporter immediately on start.
246232
return nil
247233
}

‎pkg/cli/cmd/dev/plugin/create.go‎

Lines changed: 34 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -79,6 +79,11 @@ type DevOptions struct {
7979
// configures the hub to use it instead of embedded kcp.
8080
WithExternalKCP bool
8181
KCPHTTPSPort int // host port for the kcp NodePort mapping (default 7443)
82+
83+
// AgentCount controls how many agent kind clusters to create.
84+
// Default is 1 (single agent cluster named AgentClusterName).
85+
// When > 1, clusters are named AgentClusterName-1, AgentClusterName-2, …
86+
AgentCount int
8287
}
8388

8489
// fallbackAssetVersion is used when unable to fetch the latest version
@@ -107,6 +112,7 @@ func NewDevOptions(streams genericclioptions.IOStreams) *DevOptions {
107112
Streams: streams,
108113
HubClusterName: "kedge-hub",
109114
AgentClusterName: "kedge-agent",
115+
AgentCount: 1,
110116
ChartPath: "deploy/charts/kedge-hub",
111117
AgentChartPath: "oci://ghcr.io/faroshq/charts/kedge-agent",
112118
ChartVersion: fallbackAssetVersion,
@@ -137,6 +143,7 @@ func (o *DevOptions) AddCmdFlags(cmd *cobra.Command) {
137143
cmd.Flags().IntVar(&o.DexHTTPPort, "dex-http-port", 5556, "Host port for the Dex NodePort mapping (default 5556)")
138144
cmd.Flags().BoolVar(&o.WithExternalKCP, "with-external-kcp", false, "Deploy kcp via Helm into the hub kind cluster instead of using embedded kcp")
139145
cmd.Flags().IntVar(&o.KCPHTTPSPort, "kcp-https-port", 7443, "Host port for the kcp front-proxy NodePort mapping (default 7443)")
146+
cmd.Flags().IntVar(&o.AgentCount, "agent-count", 1, "Number of agent kind clusters to create (default 1). When > 1, clusters are named <agent-cluster-name>-1, -2, …")
140147
}
141148

142149
// Complete completes the options
@@ -258,6 +265,21 @@ func redText(text string) string {
258265
return "\033[31m" + text + "\033[0m"
259266
}
260267

268+
// agentClusterNames returns the list of agent cluster names derived from
269+
// AgentClusterName and AgentCount.
270+
// - count == 1 → ["<AgentClusterName>"] (preserves backwards-compat naming)
271+
// - count > 1 → ["<AgentClusterName>-1", "<AgentClusterName>-2", …]
272+
func (o *DevOptions) agentClusterNames() []string {
273+
if o.AgentCount <= 1 {
274+
return []string{o.AgentClusterName}
275+
}
276+
names := make([]string, o.AgentCount)
277+
for i := range names {
278+
names[i] = fmt.Sprintf("%s-%d", o.AgentClusterName, i+1)
279+
}
280+
return names
281+
}
282+
261283
func (o *DevOptions) runWithColors(ctx context.Context) error {
262284
// Display experimental warning header with red "EXPERIMENTAL"
263285
fmt.Fprintf(o.Streams.ErrOut, "kedge Development Environment Setup\n\n") // nolint:errcheck
@@ -275,9 +297,11 @@ func (o *DevOptions) runWithColors(ctx context.Context) error {
275297
return err
276298
}
277299

278-
// Create agent cluster (no kedge installed, just a plain cluster)
279-
if err := o.createCluster(ctx, o.AgentClusterName, agentClusterConfig, false); err != nil {
280-
return err
300+
// Create agent cluster(s) (no kedge installed, just plain clusters).
301+
for _, agentName := range o.agentClusterNames() {
302+
if err := o.createCluster(ctx, agentName, agentClusterConfig, false); err != nil {
303+
return err
304+
}
281305
}
282306

283307
hubIP, err := o.getClusterIPAddress(ctx, o.HubClusterName, o.KindNetwork)
@@ -290,11 +314,13 @@ func (o *DevOptions) runWithColors(ctx context.Context) error {
290314
_, _ = fmt.Fprint(o.Streams.ErrOut, "kedge dev environment is ready!\n\n")
291315

292316
// Configuration
293-
fmt.Fprint(o.Streams.ErrOut, "Configuration:\n") // nolint:errcheck
294-
fmt.Fprintf(o.Streams.ErrOut, " Hub cluster kubeconfig: %s.kubeconfig\n", o.HubClusterName) // nolint:errcheck
295-
fmt.Fprintf(o.Streams.ErrOut, " Agent cluster kubeconfig: %s.kubeconfig\n", o.AgentClusterName) // nolint:errcheck
296-
fmt.Fprint(o.Streams.ErrOut, " kedge server URL: https://kedge.localhost:8443\n") // nolint:errcheck
297-
fmt.Fprint(o.Streams.ErrOut, " Static auth token: dev-token\n") // nolint:errcheck
317+
fmt.Fprint(o.Streams.ErrOut, "Configuration:\n") // nolint:errcheck
318+
fmt.Fprintf(o.Streams.ErrOut, " Hub cluster kubeconfig: %s.kubeconfig\n", o.HubClusterName) // nolint:errcheck
319+
for _, agentName := range o.agentClusterNames() {
320+
fmt.Fprintf(o.Streams.ErrOut, " Agent cluster kubeconfig: %s.kubeconfig\n", agentName) // nolint:errcheck
321+
}
322+
fmt.Fprint(o.Streams.ErrOut, " kedge server URL: https://kedge.localhost:8443\n") // nolint:errcheck
323+
fmt.Fprint(o.Streams.ErrOut, " Static auth token: dev-token\n") // nolint:errcheck
298324
if hubIP != "" {
299325
fmt.Fprintf(o.Streams.ErrOut, " Hub cluster IP (for agent): %s\n", hubIP) // nolint:errcheck
300326
}

‎pkg/cli/cmd/dev/plugin/delete.go‎

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -33,9 +33,11 @@ func (o *DevOptions) RunDelete() error {
3333
return err
3434
}
3535

36-
// Delete agent cluster
37-
if err := o.deleteCluster(o.AgentClusterName); err != nil {
38-
return err
36+
// Delete all agent cluster(s).
37+
for _, agentName := range o.agentClusterNames() {
38+
if err := o.deleteCluster(agentName); err != nil {
39+
return err
40+
}
3941
}
4042

4143
// Also clean up the site kubeconfig if it exists

‎pkg/hub/controllers/scheduler/reconciler.go‎

Lines changed: 20 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,7 @@ package scheduler
1919
import (
2020
"context"
2121
"fmt"
22+
"time"
2223

2324
corev1 "k8s.io/api/core/v1"
2425
apierrors "k8s.io/apimachinery/pkg/api/errors"
@@ -32,6 +33,7 @@ import (
3233
kedgev1alpha1 "github.com/faroshq/faros-kedge/apis/kedge/v1alpha1"
3334

3435
mcbuilder "sigs.k8s.io/multicluster-runtime/pkg/builder"
36+
mccontext "sigs.k8s.io/multicluster-runtime/pkg/context"
3537
mchandler "sigs.k8s.io/multicluster-runtime/pkg/handler"
3638
mcmanager "sigs.k8s.io/multicluster-runtime/pkg/manager"
3739
mcreconcile "sigs.k8s.io/multicluster-runtime/pkg/reconcile"
@@ -45,6 +47,7 @@ type Reconciler struct {
4547
// SetupWithManager registers the scheduler controller with the multicluster manager.
4648
func SetupWithManager(mgr mcmanager.Manager) error {
4749
r := &Reconciler{mgr: mgr}
50+
klog.Info("Registering VirtualWorkload scheduler controller")
4851
return mcbuilder.ControllerManagedBy(mgr).
4952
Named(controllerName).
5053
For(&kedgev1alpha1.VirtualWorkload{}).
@@ -55,9 +58,11 @@ func SetupWithManager(mgr mcmanager.Manager) error {
5558
// Reconcile handles a single VirtualWorkload reconciliation across workspaces.
5659
func (r *Reconciler) Reconcile(ctx context.Context, req mcreconcile.Request) (ctrl.Result, error) {
5760
logger := klog.FromContext(ctx).WithValues("key", req.NamespacedName, "cluster", req.ClusterName)
61+
logger.V(4).Info("Reconciling VirtualWorkload")
5862

5963
cl, err := r.mgr.GetCluster(ctx, req.ClusterName)
6064
if err != nil {
65+
logger.Error(err, "Failed to get cluster", "clusterName", req.ClusterName)
6166
return ctrl.Result{}, fmt.Errorf("getting cluster %s: %w", req.ClusterName, err)
6267
}
6368
c := cl.GetClient()
@@ -68,12 +73,14 @@ func (r *Reconciler) Reconcile(ctx context.Context, req mcreconcile.Request) (ct
6873
if apierrors.IsNotFound(err) {
6974
return ctrl.Result{}, nil
7075
}
76+
logger.Error(err, "Failed to get VirtualWorkload")
7177
return ctrl.Result{}, err
7278
}
7379

7480
// List all Sites in this workspace
7581
var siteList kedgev1alpha1.SiteList
7682
if err := c.List(ctx, &siteList); err != nil {
83+
logger.Error(err, "Failed to list Sites")
7784
return ctrl.Result{}, fmt.Errorf("listing sites: %w", err)
7885
}
7986

@@ -83,7 +90,7 @@ func (r *Reconciler) Reconcile(ctx context.Context, req mcreconcile.Request) (ct
8390
return ctrl.Result{}, fmt.Errorf("matching sites: %w", err)
8491
}
8592
selected := SelectSites(matched, vw.Spec.Placement.Strategy)
86-
logger.V(4).Info("Scheduling", "matched", len(matched), "selected", len(selected))
93+
logger.V(4).Info("Scheduling", "sites", len(siteList.Items), "matched", len(matched), "selected", len(selected))
8794

8895
// List existing placements for this VW
8996
var placementList kedgev1alpha1.PlacementList
@@ -158,14 +165,24 @@ func (r *Reconciler) Reconcile(ctx context.Context, req mcreconcile.Request) (ct
158165
}
159166
}
160167

161-
return ctrl.Result{}, nil
168+
// Requeue periodically so site reconnects are picked up even if the watch
169+
// event is missed (e.g. status-only changes may not always fire the mapper).
170+
return ctrl.Result{RequeueAfter: 30 * time.Second}, nil
162171
}
163172

164173
// mapSiteToVirtualWorkloads re-enqueues all VirtualWorkloads in the same
165174
// workspace whenever a Site changes.
166175
func (r *Reconciler) mapSiteToVirtualWorkloads(ctx context.Context, obj client.Object) []reconcile.Request {
167-
cl, err := r.mgr.GetCluster(ctx, obj.GetAnnotations()["kcp.io/cluster"])
176+
// Prefer the cluster name from the multicluster-runtime context (canonical),
177+
// fall back to the kcp annotation if the context value is absent.
178+
clusterKey, ok := mccontext.ClusterFrom(ctx)
179+
if !ok {
180+
clusterKey = obj.GetAnnotations()["kcp.io/cluster"]
181+
}
182+
klog.V(2).InfoS("mapSiteToVirtualWorkloads", "site", obj.GetName(), "cluster", clusterKey)
183+
cl, err := r.mgr.GetCluster(ctx, clusterKey)
168184
if err != nil {
185+
klog.V(2).InfoS("mapSiteToVirtualWorkloads: GetCluster failed", "cluster", clusterKey, "err", err)
169186
return nil
170187
}
171188
var vwList kedgev1alpha1.VirtualWorkloadList

‎pkg/util/confighelpers/confighelpers.go‎

Lines changed: 41 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,9 @@ import (
2929
"errors"
3030
"fmt"
3131
"io"
32+
"io/fs"
33+
"sort"
34+
"strings"
3235
"time"
3336

3437
apierrors "k8s.io/apimachinery/pkg/api/errors"
@@ -75,7 +78,8 @@ const annotationCreateOnly = "bootstrap.kcp.io/create-only"
7578
// Bootstrap reads all YAML files from an embed.FS and applies them to the
7679
// cluster using a discovery-based REST mapper. It retries continuously until
7780
// all resources are successfully applied or the context is cancelled.
78-
func Bootstrap(ctx context.Context, discoveryClient discovery.DiscoveryInterface, dynamicClient dynamic.Interface, fs embed.FS, opts ...Option) error {
81+
// Files are applied in dependency order: APIResourceSchemas before APIExports.
82+
func Bootstrap(ctx context.Context, discoveryClient discovery.DiscoveryInterface, dynamicClient dynamic.Interface, embedFS embed.FS, opts ...Option) error {
7983
cache := memory.NewMemCacheClient(discoveryClient)
8084
mapper := restmapper.NewDeferredDiscoveryRESTMapper(cache)
8185

@@ -87,7 +91,7 @@ func Bootstrap(ctx context.Context, discoveryClient discovery.DiscoveryInterface
8791
}
8892

8993
return wait.PollUntilContextCancel(ctx, time.Second, true, func(ctx context.Context) (bool, error) {
90-
if err := createResourcesFromFS(ctx, dynamicClient, mapper, fs, transformers...); err != nil {
94+
if err := createResourcesFromFS(ctx, dynamicClient, mapper, embedFS, transformers...); err != nil {
9195
klog.FromContext(ctx).V(2).Info("Failed to bootstrap resources, retrying", "err", err)
9296
cache.Invalidate()
9397
return false, nil
@@ -96,19 +100,46 @@ func Bootstrap(ctx context.Context, discoveryClient discovery.DiscoveryInterface
96100
})
97101
}
98102

103+
// bootstrapFileOrder returns a sort-key that ensures APIResourceSchemas are
104+
// applied before APIExports (which reference them), and both before any other
105+
// resources. Within each tier files are ordered alphabetically.
106+
func bootstrapFileOrder(name string) int {
107+
switch {
108+
case strings.HasPrefix(name, "apiresourceschema-"):
109+
return 0
110+
case strings.HasPrefix(name, "apiexport-"):
111+
return 1
112+
default:
113+
return 2
114+
}
115+
}
116+
99117
// createResourcesFromFS reads all YAML files from an embed.FS and applies them.
100-
func createResourcesFromFS(ctx context.Context, client dynamic.Interface, mapper meta.RESTMapper, fs embed.FS, transformers ...TransformFileFunc) error {
101-
files, err := fs.ReadDir(".")
118+
// Files are applied in dependency order: APIResourceSchemas first, then
119+
// APIExports (which reference schemas), then all other resources.
120+
func createResourcesFromFS(ctx context.Context, client dynamic.Interface, mapper meta.RESTMapper, embedFS embed.FS, transformers ...TransformFileFunc) error {
121+
entries, err := embedFS.ReadDir(".")
102122
if err != nil {
103123
return err
104124
}
105125

126+
files := make([]fs.DirEntry, 0, len(entries))
127+
for _, e := range entries {
128+
if !e.IsDir() {
129+
files = append(files, e)
130+
}
131+
}
132+
sort.Slice(files, func(i, j int) bool {
133+
oi, oj := bootstrapFileOrder(files[i].Name()), bootstrapFileOrder(files[j].Name())
134+
if oi != oj {
135+
return oi < oj
136+
}
137+
return files[i].Name() < files[j].Name()
138+
})
139+
106140
var errs []error
107141
for _, f := range files {
108-
if f.IsDir() {
109-
continue
110-
}
111-
if err := createResourceFromFS(ctx, client, mapper, f.Name(), fs, transformers...); err != nil {
142+
if err := createResourceFromFS(ctx, client, mapper, f.Name(), embedFS, transformers...); err != nil {
112143
errs = append(errs, err)
113144
}
114145
}
@@ -117,8 +148,8 @@ func createResourcesFromFS(ctx context.Context, client dynamic.Interface, mapper
117148

118149
// createResourceFromFS reads a single YAML file (possibly multi-document),
119150
// applies transformers, and upserts each resource.
120-
func createResourceFromFS(ctx context.Context, client dynamic.Interface, mapper meta.RESTMapper, filename string, fs embed.FS, transformers ...TransformFileFunc) error {
121-
raw, err := fs.ReadFile(filename)
151+
func createResourceFromFS(ctx context.Context, client dynamic.Interface, mapper meta.RESTMapper, filename string, embedFS embed.FS, transformers ...TransformFileFunc) error {
152+
raw, err := embedFS.ReadFile(filename)
122153
if err != nil {
123154
return fmt.Errorf("could not read %s: %w", filename, err)
124155
}

0 commit comments

Comments
 (0)