Summary
Deploy reports failure at the image pruning step even though the container is healthy and serving traffic. The root cause is that Kamal's pruning logic is internally contradictory: it intentionally keeps stopped containers but then tries to remove the images they reference.
Steps to reproduce
- Deploy an app with Kamal (2.10.1) multiple times
- Stopped containers accumulate (Kamal keeps the 5 most recent via
tail -n +6)
- The image prune step tries to remove images still referenced by those kept containers
docker rmi fails with exit code 1, and the deploy is marked as failed
INFO Running docker ps -q -a --filter label=service=myapp ... | tail -n +6 | while read container_id; do docker rm $container_id; done
INFO Finished in 0.440 seconds with exit status 0 (successful).
INFO Running docker image ls --filter label=service=myapp --format '{{.ID}} {{.Repository}}:{{.Tag}}' | grep -v -w "$(docker container ls -a --format '{{.Image}}\|' ...)" | while read image tag; do docker rmi $tag; done
ERROR (SSHKit::Command::Failed): Exception while executing on host: docker exit status: 1
The container is running and healthy at this point — the deploy succeeded, but Kamal reports failure.
Root cause
Two issues in lib/kamal/commands/prune.rb:
-
Contradictory pruning logic: The container prune keeps the 5 most recent stopped containers (tail -n +6), but the image prune then tries to remove images those containers reference. The grep -v filter is supposed to exclude referenced images but doesn't catch all of them.
-
No error tolerance: The while read image tag; do docker rmi $tag; done loop has no error handling. If any single docker rmi fails, the entire deploy is marked as failed — even though the app is healthy and serving traffic.
Suggested fix
Either:
- Fix the grep filter so it properly excludes images referenced by the kept stopped containers
- Or remove stopped containers before pruning images so there are no references blocking removal
- And add
|| true to the docker rmi loop so image pruning is best-effort and never fails the deploy
Environment
- Kamal 2.10.1
- Docker 28.x
- Single-server deploy with local registry
Summary
Deploy reports failure at the image pruning step even though the container is healthy and serving traffic. The root cause is that Kamal's pruning logic is internally contradictory: it intentionally keeps stopped containers but then tries to remove the images they reference.
Steps to reproduce
tail -n +6)docker rmifails with exit code 1, and the deploy is marked as failedThe container is running and healthy at this point — the deploy succeeded, but Kamal reports failure.
Root cause
Two issues in
lib/kamal/commands/prune.rb:Contradictory pruning logic: The container prune keeps the 5 most recent stopped containers (
tail -n +6), but the image prune then tries to remove images those containers reference. Thegrep -vfilter is supposed to exclude referenced images but doesn't catch all of them.No error tolerance: The
while read image tag; do docker rmi $tag; doneloop has no error handling. If any singledocker rmifails, the entire deploy is marked as failed — even though the app is healthy and serving traffic.Suggested fix
Either:
|| trueto thedocker rmiloop so image pruning is best-effort and never fails the deployEnvironment