ci(e2e): wire multi-controller workflow and drop RBAC patch-and-restore (4/4) - #1209
Closed
lionelvillard wants to merge 5 commits into
Closed
ci(e2e): wire multi-controller workflow and drop RBAC patch-and-restore (4/4)#1209lionelvillard wants to merge 5 commits into
lionelvillard wants to merge 5 commits into
Conversation
Adds a vanilla Kubernetes namespace-scoped overlay that mirrors the existing namespace-scoped/openshift overlay (without the OCP-specific Thanos and user-workload-monitoring patches). Wires the new overlay into the kustomize-build CI check. Prep work for the dedicated dual-controller e2e job; subsequent PRs repoint the secondary-controller testdata to this overlay so both controllers can run namespace-scoped from the start, eliminating the ClusterRoleBinding name collisions between the primary cluster-scoped and secondary namespace-scoped deployments. Refs llm-d#1205 Signed-off-by: Lionel Villard <villard@us.ibm.com>
- Move the "Dual namespace-scoped controllers isolation" Context out of
smoke_test.go into a new multi_controller_test.go labeled
Label("multi-controller") so it can be selected by ginkgo label-filter.
- Repoint test/e2e/testdata/secondary-controller/kustomization.yaml to the
new namespace-scoped/kubernetes overlay added in the prior PR; the
--watch-namespace patch is dropped because the namespace-scoped component
already injects it.
- Add make targets test-e2e-multi-controller and
test-e2e-multi-controller-with-setup mirroring the smoke targets.
The patch-and-restore RBAC workaround stays in place for now; a later PR
removes it once the dedicated CI job runs the suite in a fresh kind
cluster.
Refs llm-d#1205
Signed-off-by: Lionel Villard <villard@us.ibm.com>
Adds .github/workflows/ci-e2e-multi-controller.yaml with workflow_dispatch trigger and a placeholder step. The job name 'e2e-multi-controller' is fixed so branch-protection rules can reference it ahead of the full implementation landing in a follow-up PR. Refs llm-d#1205 Signed-off-by: Lionel Villard <villard@us.ibm.com>
- Replace the stub ci-e2e-multi-controller.yaml workflow with the real pull_request job. It mirrors e2e-tests-smoke (kind setup, local image build, kind-emulator env), but installs the primary controller from the new namespace-scoped/kubernetes overlay (via the WVA_OVERLAY env hook added below) and runs only the multi-controller label suite. - Add a WVA_OVERLAY override in deploy/lib/infra_wva.sh and deploy/lib/cleanup.sh so install/uninstall can target an alternative overlay without changing the default openshift/kubernetes selection. - Remove the kubectl patch-and-restore workarounds and the secondary ClusterRoleBinding bookkeeping from multi_controller_test.go. With both controllers running namespace-scoped from a fresh kind cluster, the cluster-scoped binding collisions described in the original TODO no longer occur. Refs llm-d#1205 Signed-off-by: Lionel Villard <villard@us.ibm.com>
Contributor
|
This PR is marked as stale after 21d of inactivity. After an additional 14d of inactivity (7d to become rotten, then 7d more), it will be closed. To prevent this PR from being closed, add a comment or remove the |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
.github/workflows/ci-e2e-multi-controller.yaml(added in PR 3) with a fullpull_requestjob mirroringe2e-tests-smoke. The job installs the primary controller from the newnamespace-scoped/kubernetesoverlay so both controllers run namespace-scoped from a fresh kind cluster.WVA_OVERLAYenv override todeploy/lib/infra_wva.shanddeploy/lib/cleanup.shso install/uninstall can target a non-default overlay without changing the default openshift/kubernetes selection.kubectl patch ... clusterrolebindingand per-deployment secondary CRB bookkeeping fromtest/e2e/multi_controller_test.go— the TODO it referenced is resolved by the dedicated job.This is PR 4 of 4 for #1205. Stacked on PR 2 (#1207) and PR 3 (#1208) — the diff against
mainwill include their commits until they merge. Please review/merge #1206 → #1207 → #1208 first.Open question for reviewers
Even with both controllers namespace-scoped, the existing
namespace-scoped/kubernetesandnamespace-scoped/openshiftoverlays still emit the same fiveClusterRoleBindingnames fromconfig/base/. If the secondary install'skubectl applyoverwrites them, the primary controller will lose RBAC. The fresh-cluster CI job runs the primary install first and the secondary second, so the last apply (secondary) wins for cluster-scoped resources. If a follow-up shows that cluster-scoped binding collisions still cause flakes, the right fix is to scope those bindings out of the namespace-scoped overlay (move them under thenamespace-scopedcomponent asRoleBindings, or skip them entirely when the overlay is namespace-scoped). I left that out of this PR per the issue's explicit scope ("eliminates the RBAC patch-and-restore workarounds entirely — no shared ClusterRoleBinding names means no conflict") and welcome guidance.Test plan
go build ./...succeedsbin/kustomize build config/overlays/namespace-scoped/kubernetessucceedse2e-multi-controllerjob goes green on this PRe2e-tests-smokestill passes (no regression fromWVA_OVERLAYoverride defaulting unchanged)Refs #1205