test(e2e): nightly workflow + heavy scenarios (HA, failure-status, upgrade) (#4202)

[wrkode]

What

Add a Tier-2 nightly e2e suite for the heavier scenarios too slow for the
per-PR Tier-1 job (ADR-13 E4). Closes kairos-io/kairos#4202.

Gated by //go:build e2e && nightly: the per-PR job (-tags e2e) excludes
them; the nightly job (-tags "e2e nightly") compiles + runs everything. The
existing Tier-1 files and ci.yml's e2e job are untouched.

Scenarios (test/e2e/nightly_*_test.go)

• pre-membership failure-status — a worker reconcile against an unreachable
  endpoint fails loud within the DefaultBudget Total=8m ceiling (not the 14m exec
  backstop) and writes Phase=Failed / Outcome=failure / Membership=uninitialized / Reason=ControlPlaneUnreachable / terminal=false / lastAction=wait-for-control-plane — the #4099-1 never-hang proof.
• multi-control-plane stacked-etcd HA — mint-join --role controlplane
  (asserting the upload-certs/cert-key keystone), cp2 control-plane join converges,
  2 control-plane-role-labeled nodes, 2 etcd members.
• kubeadm-layer in-place minor upgrade — swap the higher-minor toolchain into
  the running lower-minor container, pin kubernetesVersion=higher, run a real
  kubeadm upgrade apply; asserts lastAction=upgrade-apply (no no-op) and
  manifest + kubeletVersion at the higher minor. Self-skips (never fakes) when
  the higher-minor node image isn't staged.

Workflow / tooling

• .github/workflows/nightly.yml — cron: 0 3 * * * + workflow_dispatch; builds
  the lower (and, where a supported higher adjacent minor exists, higher) node
  images and runs the suite. Actions SHA-pinned (matching #4203); per-job least
  privilege; 150m timeout; leaked-container prune on failure.
• Makefile — e2e-nightly target (-tags "e2e nightly", 90m timeout, upgrade-to
  env knobs).
• test/e2e/nodecontainer.go — CopyInto + extractBinariesFromImage helpers
  (argv-only, bounded, t.Cleanup-reaped) for the upgrade binary swap.

Validation

• failure-status and HA run GREEN end-to-end locally (privileged systemd node
  containers, real kubeadm): failure surfaced at 8m5s == the budget ceiling
  (exit 1); HA reached 2 CP nodes + 2 etcd members.
• upgrade validated by the first nightly run / workflow_dispatch (it stages a
  second node image; self-skips locally without it).
• gofmt clean; both tag sets compile; untagged build/test unaffected; nightly.yml
  valid with zero floating action tags; asserted status/action/budget constants
  (wait-for-control-plane, ControlPlaneUnreachable, upgrade-apply, 8m/20m)
  verified against internal/*.

[wrkode]

Update after the first nightly (workflow_dispatch) run:

• failure-status and multi-CP HA passed on every leg (and locally); all Tier-1 scenarios passed.
• The in-container kubeadm-layer upgrade scenario failed: kubeadm upgrade apply preflight requires a genuinely functional pod-scheduling cluster (Ready node + an upgrade-health-check Job that must schedule and complete) — the property least specific to the provider and already validated on the libvirt VM upgrade smoke.
• Per ADR-13-B1 (staff-architect ratified), the VM-only upgrade coverage boundary is extended to the kubeadm-layer in-place upgrade, and the in-container upgrade scenario is dropped (commit e4afb4e). #4202's acceptance is met by failure-status + HA in-container and the kubeadm upgrade on the VM boundary.

Re-verified after removal: HA still green end-to-end (2 CP nodes, 2 etcd members); go vet -tags "e2e nightly" clean.