Skip to content

Switch container base images to bci-nano and remove tini#5346

Merged
thardeck merged 5 commits into
mainfrom
switch_to_nano_image
Jun 25, 2026
Merged

Switch container base images to bci-nano and remove tini#5346
thardeck merged 5 commits into
mainfrom
switch_to_nano_image

Conversation

@thardeck

@thardeck thardeck commented Jun 24, 2026

Copy link
Copy Markdown
Collaborator

Switch the fleet-controller and fleet-agent container images from bci-busybox to bci-nano to reduce image size and CVE exposure.

Both images no longer ship tini. The controller and agent now run the Fleet binaries directly as PID 1, which keeps signal handling correct without an init wrapper. The original controller-side tini usage was only needed when log.sh was the entrypoint; that shell wrapper has been removed, and the remaining Fleet processes do not spawn child processes that require zombie reaping.

Because bci-nano has no shell, log.sh is removed. Gitjob containers now use TerminationMessageFallbackToLogsOnError to preserve failure diagnostics by surfacing container logs as the termination message on non-zero exit. Their commands now invoke fleet directly instead of going through tini --.

Also wait for the ChartMuseum health endpoint before uploading charts in the E2E test environment setup to make the process more reliable.

Refers #5407

@thardeck thardeck self-assigned this Jun 24, 2026
@thardeck thardeck force-pushed the switch_to_nano_image branch from 702d6f4 to 7655ec7 Compare June 24, 2026 12:40
Wait for the ChartMuseum health endpoint before posting charts.
@thardeck thardeck force-pushed the switch_to_nano_image branch from 7655ec7 to b818523 Compare June 24, 2026 15:07
@thardeck thardeck added this to the v2.15.0 milestone Jun 24, 2026
@thardeck thardeck marked this pull request as ready for review June 24, 2026 15:31
@thardeck thardeck requested a review from a team as a code owner June 24, 2026 15:31
Copilot AI review requested due to automatic review settings June 24, 2026 15:31
@thardeck thardeck added this to Fleet Jun 24, 2026
@thardeck thardeck moved this to 👀 In review in Fleet Jun 24, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Switches the fleet-controller and fleet-agent container images from bci-busybox to bci-nano, adding tini as PID 1 for proper signal handling and removing the log.sh wrapper. It also adjusts GitJob pod specs to preserve termination diagnostics via TerminationMessageFallbackToLogsOnError, and hardens E2E infra setup by waiting for ChartMuseum readiness before uploading charts.

Changes:

  • Update both Dockerfiles to use bci-nano and install tini; remove log.sh from the image build.
  • Update GitJob container commands to run via tini -- and use TerminationMessageFallbackToLogsOnError for better failure diagnostics.
  • Improve E2E stability: wait for ChartMuseum health endpoint; adjust signal-handling tests to terminate pods via deletion; tweak dev Helm install script and agent chart args/command behavior.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
package/log.sh Removes the shell-based logging wrapper (no shell in bci-nano).
package/Dockerfile.agent Switches base image to bci-nano, downloads/copies tini, sets ENTRYPOINT to tini.
package/Dockerfile Switches base image to bci-nano, removes copying of log.sh, uses tini entrypoint.
internal/cmd/controller/gitops/reconciler/gitjob.go Runs GitJob containers under tini and uses fallback termination messages for logs on error.
internal/cmd/controller/gitops/reconciler/gitjob_test.go Updates expectations for tini command and termination message policy.
e2e/testenv/infra/cmd/setup.go Waits for ChartMuseum readiness; improves load balancer IP readiness handling.
e2e/single-cluster/signals_test.go Changes termination mechanism to pod deletion and updates termination-state probing logic.
dev/setup-fleet Changes Helm wait behavior for local dev setup.
charts/fleet-agent/templates/deployment.yaml Switches from command: to args: to avoid overriding the image entrypoint.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread dev/setup-fleet Outdated
Comment thread package/Dockerfile
Comment thread charts/fleet-agent/templates/deployment.yaml Outdated
Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
@thardeck thardeck force-pushed the switch_to_nano_image branch from b818523 to 2e5957d Compare June 24, 2026 16:09
@thardeck thardeck marked this pull request as draft June 24, 2026 16:20
@thardeck thardeck requested a review from Copilot June 24, 2026 17:47

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 6 comments.

Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
thardeck added 2 commits June 25, 2026 07:47
and drop log.sh dependency for git jobs.

Run gitjob containers through tini instead of log.sh wrappers.
Remove log.sh from the main image to support bci-nano which does not
have a shell.

Preserve gitjob error messages by using termination message
fallback-to-logs for gitjob containers.
Add tini to the fleet-agent image and use it as entrypoint
for stable PID 1 behavior on bci-nano.
@thardeck thardeck force-pushed the switch_to_nano_image branch from 2e5957d to a51ce80 Compare June 25, 2026 05:49
@thardeck thardeck marked this pull request as ready for review June 25, 2026 05:49
@thardeck thardeck force-pushed the switch_to_nano_image branch from a51ce80 to fc964b0 Compare June 25, 2026 06:15
Comment thread package/Dockerfile.agent Outdated
@thardeck thardeck force-pushed the switch_to_nano_image branch from fc964b0 to 480606b Compare June 25, 2026 07:37
@thardeck thardeck changed the title Switch container base images to bci-nano Switch container base images to bci-nano and remove tini Jun 25, 2026
@thardeck thardeck requested a review from Copilot June 25, 2026 08:08

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 5 comments.

Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
Comment thread e2e/single-cluster/signals_test.go
@thardeck thardeck force-pushed the switch_to_nano_image branch from 94b09ca to 08ffda3 Compare June 25, 2026 10:22
thardeck added 2 commits June 25, 2026 12:24
Go programs handle SIGTERM correctly as PID 1, and no Fleet binary
spawns child processes (exec.Command is used only in test helpers).
The original reason for tini on the controller was log.sh, a shell
wrapper that spawned fleetcontroller as a child; log.sh was removed
when the image switched to bci-nano. The agent never had tini on main.

Remove the tini downloader stage from both Dockerfiles, drop the
ENTRYPOINT, and change gitjob pod specs to use fleet directly as the
command instead of wrapping it with tini.

Add simple tests to not forget to update the helm charts in the future.
@thardeck thardeck force-pushed the switch_to_nano_image branch from 08ffda3 to cfd0be3 Compare June 25, 2026 10:24
@thardeck thardeck requested review from aruiz14 and weyfonk June 25, 2026 10:26
@thardeck thardeck dismissed aruiz14’s stale review June 25, 2026 15:27

Alejandro is on vacation and verified the changes via phone but could not log into github from there.

@thardeck thardeck merged commit 5f97dfc into main Jun 25, 2026
24 checks passed
@thardeck thardeck deleted the switch_to_nano_image branch June 25, 2026 15:28
@github-project-automation github-project-automation Bot moved this from 👀 In review to ✅ Done in Fleet Jun 25, 2026
@thardeck thardeck moved this from ✅ Done to Needs QA review in Fleet Jun 26, 2026
@mmartin24 mmartin24 self-assigned this Jul 7, 2026
@thardeck thardeck moved this from Needs QA review to ✅ Done in Fleet Jul 7, 2026
@thardeck thardeck removed this from the v2.15.0 milestone Jul 7, 2026
@thardeck

thardeck commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

I have created a ticket to track this: #5407

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

5 participants