Skip to content

Commit d3e94e3

Browse files
v47v47
committed
multicluster-runtime plugin
1 parent a9cf539 commit d3e94e3

20 files changed

Lines changed: 2282 additions & 0 deletions

File tree

‎docs/book/src/plugins/extending/multicluster-runtime-plugin.md‎

Lines changed: 165 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,165 @@
1+
# multicluster-runtime/v1-alpha Plugin
2+
3+
The `multicluster-runtime/v1-alpha` plugin adds
4+
[sigs.k8s.io/multicluster-runtime](https://github.com/kubernetes-sigs/multicluster-runtime)
5+
support to a Kubebuilder project. Instead of reconciling objects in a single cluster, your
6+
controller will reconcile objects across all clusters registered with the chosen provider.
7+
8+
This plugin is built into the `kubebuilder` binary — no separate installation is required.
9+
10+
## When to use this plugin
11+
12+
Use `multicluster-runtime/v1-alpha` when you need to:
13+
14+
- Manage resources across **multiple Kubernetes clusters** from a single operator binary
15+
- React to cluster lifecycle events (clusters joining or leaving the fleet)
16+
- Run **fleet-wide controllers** while still using familiar controller-runtime patterns
17+
18+
## Prerequisites
19+
20+
- Kubebuilder v4+
21+
- Go 1.22+
22+
23+
## Project initialization
24+
25+
Chain `multicluster-runtime/v1-alpha` after `go/v4`:
26+
27+
```bash
28+
kubebuilder init \
29+
--plugins go/v4,multicluster-runtime/v1-alpha \
30+
--domain example.com \
31+
--repo github.com/example/myop \
32+
--provider kubeconfig
33+
```
34+
35+
The plugin rewrites `cmd/main.go` to use `mcmanager.New(...)` instead of `ctrl.NewManager(...)`.
36+
37+
## Provider selection guide
38+
39+
The `--provider` flag (default: `kubeconfig`) controls how clusters are discovered.
40+
41+
| Provider | Flag value | Use case |
42+
|---|---|---|
43+
| **Kubeconfig secrets** | `kubeconfig` | Dynamic fleet — clusters join/leave at runtime by creating kubeconfig Secrets |
44+
| **Namespace** | `namespace` | Single cluster, namespace-per-tenant; each namespace is treated as a "cluster" |
45+
| **Cluster API** | `cluster-api` | Fleet managed by [Cluster API](https://cluster-api.sigs.k8s.io/) controllers |
46+
| **File** | `file` | Static cluster list; one kubeconfig file per cluster in a directory (great for CI) |
47+
48+
### Kubeconfig provider (default)
49+
50+
```bash
51+
kubebuilder init --plugins go/v4,multicluster-runtime/v1-alpha \
52+
--domain example.com --repo github.com/example/myop \
53+
--provider kubeconfig
54+
```
55+
56+
Add a cluster at runtime by creating a Secret with the kubeconfig:
57+
58+
```yaml
59+
apiVersion: v1
60+
kind: Secret
61+
metadata:
62+
name: my-cluster
63+
namespace: default
64+
labels:
65+
# label recognized by the kubeconfig provider
66+
multicluster.x-k8s.io/cluster-name: my-cluster
67+
type: Opaque
68+
data:
69+
kubeconfig: <base64-encoded-kubeconfig>
70+
```
71+
72+
### Namespace provider
73+
74+
```bash
75+
kubebuilder init --plugins go/v4,multicluster-runtime/v1-alpha \
76+
--domain example.com --repo github.com/example/myop \
77+
--provider namespace
78+
```
79+
80+
The generated `cmd/main.go` starts the provider and manager concurrently using `errgroup`.
81+
82+
### File provider
83+
84+
```bash
85+
kubebuilder init --plugins go/v4,multicluster-runtime/v1-alpha \
86+
--domain example.com --repo github.com/example/myop \
87+
--provider file \
88+
--kubeconfig-dir /etc/kubeconfig
89+
```
90+
91+
Place one kubeconfig file per cluster in `--kubeconfig-dir`. Each file name becomes the
92+
cluster name.
93+
94+
## Create a multicluster controller
95+
96+
```bash
97+
kubebuilder create api \
98+
--plugins go/v4,multicluster-runtime/v1-alpha \
99+
--group foo --version v1 --kind Foo \
100+
--controller --resource
101+
```
102+
103+
The generated controller (`internal/controller/foo_controller.go`) uses:
104+
105+
- `mcreconcile.Request` — carries `ClusterName` in addition to the usual `NamespacedName`
106+
- `mcbuilder.ControllerManagedBy(mgr)` — watches objects across all registered clusters
107+
- `mcmanager.Manager` — the multicluster-aware manager type
108+
109+
### Using `req.ClusterName`
110+
111+
```go
112+
func (r *FooReconciler) Reconcile(ctx context.Context, req mcreconcile.Request) (ctrl.Result, error) {
113+
log := log.FromContext(ctx).WithValues("cluster", req.ClusterName)
114+
115+
// Fetch the object from the correct cluster's cache.
116+
foo := &foov1.Foo{}
117+
if err := r.Get(ctx, req.NamespacedName, foo); err != nil {
118+
return ctrl.Result{}, client.IgnoreNotFound(err)
119+
}
120+
121+
log.Info("Reconciling Foo", "name", foo.Name)
122+
// ... your business logic ...
123+
return ctrl.Result{}, nil
124+
}
125+
```
126+
127+
## Webhooks
128+
129+
Webhooks register with the **local cluster's** API server — they do not need multicluster
130+
changes. You can scaffold them normally:
131+
132+
```bash
133+
kubebuilder create webhook \
134+
--plugins go/v4,multicluster-runtime/v1-alpha \
135+
--group foo --version v1 --kind Foo \
136+
--defaulting --programmatic-validation
137+
```
138+
139+
The `multicluster-runtime/v1-alpha` plugin does not modify webhook files. The webhook
140+
scaffolding is handled entirely by `go/v4` and the output is identical to a single-cluster
141+
project. Webhooks register with the **local** cluster's API server and do not need
142+
multicluster changes.
143+
144+
## Switching providers
145+
146+
Use `kubebuilder edit` to replace the provider in an existing project:
147+
148+
```bash
149+
kubebuilder edit --plugins multicluster-runtime/v1-alpha --provider namespace
150+
```
151+
152+
This rewrites `cmd/main.go` while preserving all `// +kubebuilder:scaffold:*` markers so
153+
that future `kubebuilder create api` and `kubebuilder create webhook` commands still work.
154+
155+
## Plugin chain note
156+
157+
This plugin is designed to run **after** `go/v4`. The plugin chain `go/v4,multicluster-runtime/v1-alpha`
158+
means:
159+
160+
1. `go/v4` scaffolds the standard project structure
161+
2. `multicluster-runtime/v1-alpha` rewrites `cmd/main.go` to use the multicluster manager
162+
163+
If `go/v4` is absent from the chain, the scaffolded `cmd/main.go` will not compile
164+
because the standard project structure (`api/`, `internal/controller/`, `Makefile`, etc.)
165+
will be missing. Always chain `go/v4` first.

‎internal/cli/cmd/cmd.go‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ import (
3636
grafanav1alpha "sigs.k8s.io/kubebuilder/v4/pkg/plugins/optional/grafana/v1alpha"
3737
helmv1alpha "sigs.k8s.io/kubebuilder/v4/pkg/plugins/optional/helm/v1alpha" //nolint:staticcheck // Deprecated
3838
helmv2alpha "sigs.k8s.io/kubebuilder/v4/pkg/plugins/optional/helm/v2alpha"
39+
multiclusterruntimev1alpha "sigs.k8s.io/kubebuilder/v4/pkg/plugins/optional/multicluster-runtime/v1alpha"
3940
)
4041

4142
// Run bootstraps & runs the CLI
@@ -78,6 +79,7 @@ func Run() {
7879
&helmv1alpha.Plugin{},
7980
&helmv2alpha.Plugin{},
8081
&autoupdatev1alpha.Plugin{},
82+
&multiclusterruntimev1alpha.Plugin{},
8183
),
8284
cli.WithPlugins(externalPlugins...),
8385
cli.WithDefaultPlugins(cfgv3.Version, gov4Bundle),

‎pkg/plugins/optional/multicluster-runtime/v1alpha/api.go‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
/*
2+
Copyright 2026 The Kubernetes Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package v1alpha
18+
19+
import (
20+
"fmt"
21+
22+
"sigs.k8s.io/kubebuilder/v4/pkg/config"
23+
"sigs.k8s.io/kubebuilder/v4/pkg/machinery"
24+
"sigs.k8s.io/kubebuilder/v4/pkg/model/resource"
25+
"sigs.k8s.io/kubebuilder/v4/pkg/plugin"
26+
"sigs.k8s.io/kubebuilder/v4/pkg/plugins/optional/multicluster-runtime/v1alpha/scaffolds"
27+
)
28+
29+
var _ plugin.CreateAPISubcommand = &createAPISubcommand{}
30+
31+
type createAPISubcommand struct {
32+
config config.Config
33+
resource *resource.Resource
34+
}
35+
36+
func (p *createAPISubcommand) UpdateMetadata(cliMeta plugin.CLIMetadata, subcmdMeta *plugin.SubcommandMetadata) {
37+
subcmdMeta.Description = `Overwrite the controller scaffolded by go/v4 with a multicluster-aware version.
38+
39+
The generated controller uses:
40+
- mcreconcile.Request (carries ClusterName alongside NamespacedName)
41+
- mcbuilder.ControllerManagedBy(mgr) (watches across all registered clusters)
42+
- mcmanager.Manager (multicluster manager type)`
43+
subcmdMeta.Examples = fmt.Sprintf(` %[1]s create api \
44+
--plugins go/v4,%[2]s \
45+
--group foo --version v1 --kind Foo --controller --resource`, cliMeta.CommandName, plugin.KeyFor(Plugin{}))
46+
}
47+
48+
func (p *createAPISubcommand) InjectConfig(c config.Config) error {
49+
p.config = c
50+
return nil
51+
}
52+
53+
func (p *createAPISubcommand) InjectResource(res *resource.Resource) error {
54+
p.resource = res
55+
return nil
56+
}
57+
58+
func (p *createAPISubcommand) Scaffold(fs machinery.Filesystem) error {
59+
if p.resource == nil || !p.resource.HasController() {
60+
return nil
61+
}
62+
s := scaffolds.NewAPIScaffolder(p.config, *p.resource)
63+
s.InjectFS(fs)
64+
if err := s.Scaffold(); err != nil {
65+
return fmt.Errorf("failed to scaffold api: %w", err)
66+
}
67+
return nil
68+
}

‎pkg/plugins/optional/multicluster-runtime/v1alpha/api_test.go‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
/*
2+
Copyright 2026 The Kubernetes Authors.
3+
4+
Licensed under the Apache License, Version 2.0 (the "License");
5+
you may not use this file except in compliance with the License.
6+
You may obtain a copy of the License at
7+
8+
http://www.apache.org/licenses/LICENSE-2.0
9+
10+
Unless required by applicable law or agreed to in writing, software
11+
distributed under the License is distributed on an "AS IS" BASIS,
12+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
See the License for the specific language governing permissions and
14+
limitations under the License.
15+
*/
16+
17+
package v1alpha
18+
19+
import (
20+
. "github.com/onsi/ginkgo/v2"
21+
. "github.com/onsi/gomega"
22+
"github.com/spf13/afero"
23+
24+
"sigs.k8s.io/kubebuilder/v4/pkg/config"
25+
cfgv3 "sigs.k8s.io/kubebuilder/v4/pkg/config/v3"
26+
"sigs.k8s.io/kubebuilder/v4/pkg/machinery"
27+
"sigs.k8s.io/kubebuilder/v4/pkg/model/resource"
28+
)
29+
30+
const controllerPath = "internal/controller/bar_controller.go"
31+
32+
var _ = Describe("createAPISubcommand", func() {
33+
var (
34+
subCmd *createAPISubcommand
35+
cfg config.Config
36+
)
37+
38+
BeforeEach(func() {
39+
subCmd = &createAPISubcommand{}
40+
cfg = cfgv3.New()
41+
_ = cfg.SetRepository("github.com/example/myop")
42+
_ = cfg.SetDomain("example.com")
43+
Expect(subCmd.InjectConfig(cfg)).To(Succeed())
44+
})
45+
46+
Context("InjectConfig", func() {
47+
It("should store the config", func() {
48+
Expect(subCmd.config).To(Equal(cfg))
49+
})
50+
})
51+
52+
Context("InjectResource", func() {
53+
It("should store the resource", func() {
54+
res := &resource.Resource{
55+
GVK: resource.GVK{Group: "foo", Version: "v1", Kind: "Bar"},
56+
}
57+
Expect(subCmd.InjectResource(res)).To(Succeed())
58+
Expect(subCmd.resource).To(Equal(res))
59+
})
60+
})
61+
62+
Context("Scaffold", func() {
63+
var memFS machinery.Filesystem
64+
65+
BeforeEach(func() {
66+
memFS = machinery.Filesystem{FS: afero.NewMemMapFs()}
67+
})
68+
69+
It("should be a no-op when resource is nil", func() {
70+
subCmd.resource = nil
71+
Expect(subCmd.Scaffold(memFS)).To(Succeed())
72+
})
73+
74+
It("should be a no-op when resource has no controller", func() {
75+
subCmd.resource = &resource.Resource{
76+
GVK: resource.GVK{Group: "foo", Version: "v1", Kind: "Bar"},
77+
Controller: false,
78+
}
79+
Expect(subCmd.Scaffold(memFS)).To(Succeed())
80+
})
81+
82+
It("should write a controller when resource has a controller", func() {
83+
subCmd.resource = &resource.Resource{
84+
GVK: resource.GVK{Group: "foo", Version: "v1", Kind: "Bar"},
85+
Plural: "bars",
86+
Path: "github.com/example/myop/api/v1",
87+
Controller: true,
88+
}
89+
Expect(subCmd.Scaffold(memFS)).To(Succeed())
90+
91+
exists, err := afero.Exists(memFS.FS, controllerPath)
92+
Expect(err).NotTo(HaveOccurred())
93+
Expect(exists).To(BeTrue())
94+
95+
content, err := afero.ReadFile(memFS.FS, controllerPath)
96+
Expect(err).NotTo(HaveOccurred())
97+
Expect(string(content)).To(ContainSubstring("mcreconcile.Request"))
98+
Expect(string(content)).To(ContainSubstring("mcbuilder.ControllerManagedBy"))
99+
Expect(string(content)).To(ContainSubstring("mcmanager.Manager"))
100+
Expect(string(content)).To(ContainSubstring(`"sigs.k8s.io/multicluster-runtime/pkg/reconcile"`))
101+
})
102+
})
103+
})

0 commit comments

Comments
 (0)