Skip to content

fix(docker): make the git-checkout volume writable by the runtime user - #1468

Merged
FelixTJDietrich merged 2 commits into
mainfrom
fix/prod-git-repos-ownership
Aug 21, 2026
Merged

fix(docker): make the git-checkout volume writable by the runtime user#1468
FelixTJDietrich merged 2 commits into
mainfrom
fix/prod-git-repos-ownership

Conversation

@FelixTJDietrich

Copy link
Copy Markdown
Collaborator

Description

The first practice review ever executed on staging failed like this:

Agent job failed: jobId=140efc9b…, error=Failed to lock CAS blob 0a603c06…
java.io.UncheckedIOException: Failed to lock CAS blob …
    at ContentAddressedStore.lockBlob(ContentAddressedStore.java:245)
Caused by: java.nio.file.AccessDeniedException: /data/git-repos/cas

A named volume is created root-owned, and one created by an earlier image keeps that image's uid. Staging's app_git-repos is owned by 100:101 at mode 0755 while the buildpack image runs as 1002:1001, so the agent cannot create its content-addressed store under the volume.

Nothing writes there until an agent job runs, which is why this never showed up before: the deployment reports healthy, the health indicator is green, and the failure appears only when someone finally enables reviews. The preview stack already carries a volume-init for exactly this reason — the production stack does not.

What changes

A one-shot volume-init that both application roles wait on, matching the preview stack:

find /data/git-repos ! -user 1002 -exec chown -h 1002:1001 {} + && chmod 0755 /data/git-repos

Two deliberate details:

  • ! -user 1002 — only entries with the wrong owner are touched, so an already-correct volume costs one metadata walk rather than a recursive rewrite of every deploy. Staging's tree is 6 GB across 216 repositories.
  • chown -h — the tree contains dangling symlinks (vendored framework directories with Versions/Current pointing at nothing). Without -h, chown follows them, fails with No such file or directory, and exits non-zero, which would make the init container fail the deploy. Verified on the live volume: it errored without -h, and reported zero remaining wrong-owner entries with it.

How to test

On the affected volume:

$ docker run --rm -v app_git-repos:/data alpine:3 \
    sh -c 'find /data ! -user 1002 -exec chown -h 1002:1001 {} + && chmod 0755 /data'
$ find /var/lib/docker/volumes/app_git-repos/_data ! -user 1002 | wc -l
0

The agent then creates /data/git-repos/cas (drwxr-xr-x 1002 1001) on the next job, and the AccessDeniedException is gone — confirmed on staging, where the following review reached evidence collection instead of failing at sandbox preparation.

Checklist

  • My changeset summary reads as an operator/user-facing note (it becomes the changelog entry) — see .changeset/README.md
  • If the operator must act on this change (new required env var, manual migration step), the changeset summary says how (**Operators:** …) and MIGRATION.md is updated

No new variable and no operator action — the init runs as part of the stack. MIGRATION.md is untouched.

@FelixTJDietrich
FelixTJDietrich requested a review from a team as a code owner August 21, 2026 16:37
@coderabbitai

coderabbitai Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

@FelixTJDietrich, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 10 minutes

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 80a5c8ab-de50-440e-a550-0ad8445c7d8a

📥 Commits

Reviewing files that changed from the base of the PR and between dae91e3 and 96ceb24.

📒 Files selected for processing (4)
  • .changeset/practice-review-writable-fabric.md
  • docker/compose.app.yaml
  • docker/preview/compose.app.yaml
  • docker/self-host/.env.example

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot added bug Something isn't working ci GitHub Actions, workflows, build pipeline changes infrastructure Docker, containers, and deployment infrastructure size:S This PR changes 10-29 lines, ignoring generated files. size:M This PR changes 30-99 lines, ignoring generated files. and removed size:S This PR changes 10-29 lines, ignoring generated files. labels Aug 21, 2026
FelixTJDietrich and others added 2 commits August 21, 2026 20:39
A named volume is created root-owned, and one created by an earlier image
keeps that image's uid. Staging's app_git-repos is owned by 100:101 at mode
0755 while the buildpack image runs as 1002:1001, so the agent could not
create its content-addressed store:

  Failed to lock CAS blob 0a603c06...
  Caused by: java.nio.file.AccessDeniedException: /data/git-repos/cas

Nothing writes there until an agent job runs, so this surfaced on the first
practice review rather than at startup — the deployment looked healthy and
reviews failed.

The preview stack already carries a volume-init for exactly this; the
production stack did not. Only entries with the wrong owner are touched, so
an already-correct volume costs a metadata walk. chown -h retargets
symlinks instead of following them: the tree contains dangling links, and
following those turns the fix into a wall of errors that exits non-zero.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EZADQeSx6zQNNqsdu7AAqZ
A new named volume is created root-owned, and one created by an earlier
image keeps that image's uid. Staging's is 100:101 at 0755 while the
buildpack image runs as 1002:1001, so the agent could not create its
content-addressed store:

  Failed to lock CAS blob 0a603c06...
  Caused by: java.nio.file.AccessDeniedException: /data/git-repos/cas

Nothing writes there until an agent job runs, so the deployment reported
healthy and the first practice review failed.

chown -Rh, not a find predicate: -h retargets the dangling symlinks the
tree contains instead of following them out of the volume, and the
predicate it replaces skipped the group entirely, so it did not do what it
claimed. The preview stack gets the same text and the same command — it had
drifted to chown -R, which has the symlink hole.

volume-init is a dependency of both application roles, so it runs on every
start: network_mode none because it needs no networking, a logging cap
because every sibling has one, and a digest-pinned base because an
unpinned tag in that position can block the stack from starting.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01EZADQeSx6zQNNqsdu7AAqZ
@FelixTJDietrich
FelixTJDietrich force-pushed the fix/prod-git-repos-ownership branch from 35e3313 to 96ceb24 Compare August 21, 2026 18:42
@FelixTJDietrich
FelixTJDietrich merged commit 0b4e14e into main Aug 21, 2026
31 checks passed
@FelixTJDietrich
FelixTJDietrich deleted the fix/prod-git-repos-ownership branch August 21, 2026 18:55
@github-actions

Copy link
Copy Markdown
Contributor

📚 Documentation Preview

Preview has been removed (PR closed)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working ci GitHub Actions, workflows, build pipeline changes infrastructure Docker, containers, and deployment infrastructure size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant