Skip to content

fix: auto repair Docker mount permissions - #1263

Merged
ZhuLinsen merged 4 commits into
mainfrom
codex/docker-auto-fix-mounted-permissions
May 11, 2026
Merged

fix: auto repair Docker mount permissions#1263
ZhuLinsen merged 4 commits into
mainfrom
codex/docker-auto-fix-mounted-permissions

Conversation

@ZhuLinsen

@ZhuLinsen ZhuLinsen commented May 11, 2026

Copy link
Copy Markdown
Owner

PR Type

  • fix
  • feat
  • refactor
  • docs
  • chore
  • test

Background And Problem

PR #1214 switched the Docker image to run as the non-root dsa user, which is a good security direction, but the default Docker / Compose examples still bind mount host data, logs, and reports directories. That made ordinary upgrades fail unless users manually fixed host-side UID/GID permissions.

PR #1262 made logs permission failures non-fatal, but it did not remove the manual permission burden for data and reports writes. This PR makes the default Docker startup path self-healing instead of asking users to run chown / chmod.

Scope Of Change

  • Add docker/entrypoint.sh that starts as root, creates /app/data, /app/logs, and /app/reports, checks whether dsa can already write them, and only repairs permissions when needed.
  • When repair is needed, the entrypoint fixes ownership/user-write permissions and then drops privileges to dsa via gosu before running the existing command.
  • The fast path avoids recursive chown / chmod when mounts are already writable; it also checks the configured SQLite database files and top-level mounted files to catch common root-owned leftovers from older images.
  • Update docker/Dockerfile to install gosu, copy the entrypoint, and use it before the existing CMD.
  • Update the runtime logging fallback text so it no longer tells normal users to manually chown directories.
  • Update Chinese and English deployment docs to say normal Docker / Compose deployments no longer require manual host-side chown / chmod.
  • Add regression coverage for the entrypoint contract and updated logging text.

Compatibility And Risk

  • Default Docker / Compose usage should become easier: bind-mounted data, logs, and reports are repaired automatically before the app runs as non-root.
  • Healthy mounts should not pay the recursive repair cost on each startup; the entrypoint first probes write access as dsa and skips repair when the mount is already usable.
  • The app process still runs as dsa; only the entrypoint starts as root long enough to create/check/repair mount permissions and then drops privileges.
  • Special environments can still need manual permission handling if they explicitly set --user / Compose user:, use read-only mounts, rootless Docker, NFS, or another storage backend that prevents chown / chmod from inside the container.

Verification Commands And Results

sh -n docker/entrypoint.sh
python -m py_compile main.py tests/test_docker_entrypoint.py tests/test_main_schedule_mode.py
python -m pytest tests/test_docker_entrypoint.py tests/test_main_schedule_mode.py -q
./scripts/ci_gate.sh flake8
./scripts/ci_gate.sh offline-tests
sh docker/entrypoint.sh sh -c 'test "$(id -u)" = "1000" && test "$(id -g)" = "1000"'

Results:

  • shell syntax: PASS
  • py_compile: PASS
  • targeted pytest: 21 passed
  • flake8 critical checks: PASS
  • offline tests: 1872 passed, 2 deselected, 166 subtests passed
  • local non-root entrypoint passthrough: PASS

Docker build was attempted twice:

docker build -f docker/Dockerfile -t dsa-permission-entrypoint-test .

Both attempts failed before building project layers because Docker Hub base image metadata requests timed out for python:3.11-slim-bookworm / node:20-slim. This is recorded as an environment/network gap rather than a code failure.

Documentation And Changelog

Updated:

  • docs/CHANGELOG.md
  • docs/full-guide.md
  • docs/full-guide_EN.md
  • docs/DEPLOY.md
  • docs/DEPLOY_EN.md

Rollback Plan

Revert this PR commit to restore the previous Dockerfile behavior. If rollback is needed after release, users can still apply the existing manual host directory permission workaround from prior documentation.

Checklist

  • 本 PR 有明确动机和业务价值 / This PR has a clear motivation and value
  • 已提供可复现的验证命令与结果 / Reproducible verification commands and results are included
  • 已评估兼容性与风险 / Compatibility and risk have been assessed
  • 已提供回滚方案 / A rollback plan is provided
  • 已同步更新相关文档与 docs/CHANGELOG.md / Relevant docs and docs/CHANGELOG.md are updated

@ZhuLinsen
ZhuLinsen marked this pull request as ready for review May 11, 2026 14:19
Copilot AI review requested due to automatic review settings May 11, 2026 14:19
@github-actions github-actions Bot added ai documentation Improvements or additions to documentation size/M testing labels May 11, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7fb10d73e9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docker/entrypoint.sh Outdated
Comment on lines +21 to +23
owner="$(stat -c '%u:%g' "$dir" 2>/dev/null || true)"
if [ "$owner" != "$APP_UID:$APP_GID" ]; then
if ! chown -R "$APP_UID:$APP_GID" "$dir"; then

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Repair nested mount ownership even when top directory matches

When a bind-mounted directory itself is already owned by 1000:1000 but contains root-owned files from a previous run, restore, or manual copy (for example /app/data/stock_analysis.db), this guard skips the recursive chown. The following chmod -R u+rwX only changes the owner permission bits, so root-owned children remain unwritable after the process drops to dsa, and the advertised automatic permission repair still leaves database/report writes failing. Consider always applying the recursive ownership repair, or checking child ownership instead of only the mount point.

Useful? React with 👍 / 👎.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes the default Docker startup path “self-healing” after the switch to non-root execution by introducing a root entrypoint that repairs bind-mounted directory permissions for /app/data, /app/logs, and /app/reports, then drops privileges to the dsa user before starting the app.

Changes:

  • Add a Docker entrypoint script that creates/fixes writable mount directories and then gosu-execs the app as dsa.
  • Update the Dockerfile to install gosu, wire the new entrypoint, and remove USER dsa so the entrypoint can start as root.
  • Update runtime logging guidance + deployment docs, and add regression tests for the entrypoint/Dockerfile contract.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
docker/entrypoint.sh New root entrypoint to repair mount dirs and drop privileges via gosu.
docker/Dockerfile Installs gosu, copies/execs entrypoint, removes USER dsa.
main.py Updates the file-logging fallback warning text to reflect the new Docker behavior.
tests/test_docker_entrypoint.py Adds tests asserting entrypoint syntax + Dockerfile/entrypoint contract.
tests/test_main_schedule_mode.py Updates expectation for the changed logging fallback message.
docs/full-guide.md Updates CN guide to state mounts are auto-repaired at startup and when it may not apply.
docs/full-guide_EN.md Same as above for EN guide.
docs/DEPLOY.md Updates CN deploy guide permissions section to reflect auto-repair entrypoint.
docs/DEPLOY_EN.md Adds EN deploy guide permissions section reflecting auto-repair entrypoint.
docs/CHANGELOG.md Adds an Unreleased fix entry describing the new Docker permission auto-repair.

Comment thread docker/entrypoint.sh Outdated
Comment on lines +29 to +30
warn "WARN: unable to make $dir writable for $APP_USER; check read-only, rootless, or NFS mount permissions if writes fail."
fi
Comment thread docker/entrypoint.sh Outdated
Comment on lines +22 to +30
if [ "$owner" != "$APP_UID:$APP_GID" ]; then
if ! chown -R "$APP_UID:$APP_GID" "$dir"; then
warn "WARN: unable to set ownership for $dir; check read-only, rootless, or NFS mount permissions if writes fail."
fi
fi

if ! chmod -R u+rwX "$dir"; then
warn "WARN: unable to make $dir writable for $APP_USER; check read-only, rootless, or NFS mount permissions if writes fail."
fi
Comment thread tests/test_docker_entrypoint.py Outdated
Comment on lines +23 to +27
def test_docker_entrypoint_repairs_ownership_and_user_permissions() -> None:
entrypoint = (REPO_ROOT / "docker" / "entrypoint.sh").read_text(encoding="utf-8")

assert 'chown -R "$APP_UID:$APP_GID" "$dir"' in entrypoint
assert 'chmod -R u+rwX "$dir"' in entrypoint
@github-actions

github-actions Bot commented May 11, 2026

Copy link
Copy Markdown

🤖 自动审查报告

项目 结果
📊 变更文件 10 个
➕ 新增行数 278 行
➖ 删除行数 31 行
🔍 静态检查 ✅ 通过
🧠 AI 审查 ✅ 已完成

📁 修改的文件

  • 📝 docker/Dockerfile (+8/-4)
  • 🆕 docker/entrypoint.sh (+90/-0)
  • 📝 docs/CHANGELOG.md (+1/-0)
  • 📝 docs/DEPLOY.md (+5/-8)
  • 📝 docs/DEPLOY_EN.md (+8/-2)
  • 📝 docs/full-guide.md (+2/-7)
  • 📝 docs/full-guide_EN.md (+2/-7)
  • 📝 main.py (+2/-2)
  • 🆕 tests/test_docker_entrypoint.py (+159/-0)
  • 📝 tests/test_main_schedule_mode.py (+1/-1)

🧠 AI 代码审查意见

结论

Ready to Merge

该 PR 解决了 Docker 容器在非 root 用户下运行时,主机挂载目录权限导致的应用启动失败问题,通过引入一个自修复的 Docker 启动入口脚本,显著改善了用户体验。代码变更、文档更新和测试覆盖全面,风险评估充分,验证结果良好。

结构化审查结果

  • 必要性通过。本 PR 有明确的业务价值。它解决了 chore: implement non-root execution for Docker #1214fix: Docker 部署启动失败 (#1261) #1262 引入的 Docker 非 root 用户权限问题,使默认的 Docker/Compose 部署能够自适应地处理 datalogsreports 目录的权限,避免用户手动 chown/chmod,极大地简化了部署流程并提升了用户体验。
  • 关联性通过。PR 描述清晰地阐述了问题的背景和动机,明确指出了问题源于先前 PR (chore: implement non-root execution for Docker #1214, fix: Docker 部署启动失败 (#1261) #1262) 引入的非 root 用户运行模式,提供了足够的上下文信息,无需额外 Issue 关联。
  • 类型fix, docs, test
    • fix: 核心功能是修复 Docker 容器的权限管理问题。
    • docs: 更新了 DEPLOY.mdfull-guide.md 及其英文版本,移除了手动权限修改的说明,并解释了自动修复机制。
    • test: 添加了新的 test_docker_entrypoint.py 来验证 entrypoint.sh 的行为,并更新了 test_main_schedule_mode.py 中的日志断言。
    • 类型判定与实际变更内容完全匹配。
  • 描述完整性完整。PR 描述非常详细和专业,包含了背景、范围、兼容性与风险、验证命令与结果、文档更新以及回滚方案,且所有 Checklist 项均已勾选。验证命令和结果(包括本地运行 ci_gate.sh)提供了充分的证据,CI 检查状态也已通过。
  • 风险级别
    • 关键风险:新的 docker/entrypoint.sh 脚本在容器启动时以 root 用户身份运行,然后通过 gosu 降权为 dsa 用户。该脚本设计了权限探测机制,仅在需要时进行 chown/chmod,避免了不必要的开销。对特殊部署环境(如显式 --user、只读挂载、rootless Docker、NFS)的兼容性风险在文档和 PR 描述中均有明确说明和警示,用户可以根据自身情况进行调整。新的测试文件 test_docker_entrypoint.pyentrypoint.sh 的关键逻辑(包括修复成功和失败情况)进行了模拟测试,验证了其健壮性。
    • 总体而言,风险得到了充分的识别和缓解,并且有相应的测试覆盖。

必改项 (无)

未发现任何阻断性问题。

建议项 (1 条)

  1. docker/entrypoint.sh 文件是本次 PR 的核心变更,但未出现在提供的 diff 列表中。虽然 PR 描述详细介绍了其作用,且 tests/test_docker_entrypoint.py 对其逻辑进行了很好的测试,但如果能直接审查 entrypoint.sh 的具体内容,将有助于进一步确认脚本的实现细节和潜在的 shell 脚本最佳实践。考虑到本次审查的限制是基于提供的 diff,这不是一个阻断项,但作为建议,未来在关键 shell 脚本引入时,可考虑在 PR 描述中附带其内容或确保其包含在 diff 中。

💡 提示: 请确保代码已通过本地测试,并遵循项目代码规范。

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ba9132dcfd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docker/Dockerfile
|| python -c "import sys; sys.exit(0)"

# 启动入口先修复 bind mount 目录权限,再降权为 dsa 执行应用
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Keep documented exec runs from writing as root

This entrypoint drops privileges only for the initial container command; docker-compose exec starts a new process without going through the entrypoint. Since this commit also removes the Dockerfile USER dsa, the documented docker-compose -f ./docker/docker-compose.yml exec stock-analyzer python main.py --no-notify command now runs as the image default root unless users add -u dsa (Docker documents USER as setting the default user, while Compose exec has a separate --user override). A manual analysis run can therefore create root-owned database/log/report files on the bind mounts, and the already-running scheduled process as dsa can fail to update them until a restart repairs ownership.

Useful? React with 👍 / 👎.

@ZhuLinsen ZhuLinsen left a comment

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

评审结论

  • 必要性:通过;非 root 镜像与默认 bind mount 之间的权限冲突会影响普通 Docker/Compose 升级路径,本 PR 有明确修复价值。
  • 是否有对应 issue:无;描述中引用了 PR #1214 / #1262 作为背景,但未检测到 Fixes/Closes/Refs 形式的 issue 关联。
  • PR 类型:fix;同时包含 docs/test,因为改动了 Docker 启动行为、部署文档和回归测试。
  • description 完整性:完整;已覆盖背景、改动范围、兼容性风险、验证结果、文档更新和回滚方案。
  • 是否可直接合入:可;当前 CI 为 success,改动同步更新了 docs/CHANGELOG.md 和中英文部署文档,未发现阻断性行为风险;mergeable_state=blocked 更像分支保护/权限状态,不单独构成阻断。

🤖 此回复由 OpenReview Bot 自动生成,仅供参考。如有疑问请 @维护者。

@ZhuLinsen
ZhuLinsen merged commit 8a0b093 into main May 11, 2026
10 checks passed
Anyone878 pushed a commit to Anyone878/daily_stock_analysis that referenced this pull request May 17, 2026
ZhuLinsen added a commit that referenced this pull request Jun 22, 2026
efinance writes its search cache (search-cache.json) into its own
package directory (site-packages/efinance/data), which is root-owned
in the image. Since the container runs as the non-root dsa user
(uid 1000), every A-share fetch failed with PermissionError, forcing
EfinanceFetcher (the default priority-0 source) to fail and fall back.

Pre-create and chown the efinance data dir to dsa at build time. The
path is resolved dynamically to avoid hardcoding the Python version.
This is image-internal static content, so the fix belongs at build
time rather than the runtime mount-repair in the entrypoint (#1263),
which only covers user-mounted volumes.

Fixes #1748

Co-authored-by: gang.wu@ximalaya.com <gang.wu@ximalaya.com>
Co-authored-by: mumu <42829555+ZhuLinsen@users.noreply.github.com>
EchoingFootsteps pushed a commit to EchoingFootsteps/daily_stock_analysis that referenced this pull request Jul 4, 2026
EchoingFootsteps pushed a commit to EchoingFootsteps/daily_stock_analysis that referenced this pull request Jul 4, 2026
efinance writes its search cache (search-cache.json) into its own
package directory (site-packages/efinance/data), which is root-owned
in the image. Since the container runs as the non-root dsa user
(uid 1000), every A-share fetch failed with PermissionError, forcing
EfinanceFetcher (the default priority-0 source) to fail and fall back.

Pre-create and chown the efinance data dir to dsa at build time. The
path is resolved dynamically to avoid hardcoding the Python version.
This is image-internal static content, so the fix belongs at build
time rather than the runtime mount-repair in the entrypoint (ZhuLinsen#1263),
which only covers user-mounted volumes.

Fixes ZhuLinsen#1748

Co-authored-by: gang.wu@ximalaya.com <gang.wu@ximalaya.com>
Co-authored-by: mumu <42829555+ZhuLinsen@users.noreply.github.com>
bmwu pushed a commit to bmwu/daily_stock_analysis that referenced this pull request Aug 24, 2026
bmwu pushed a commit to bmwu/daily_stock_analysis that referenced this pull request Aug 24, 2026
efinance writes its search cache (search-cache.json) into its own
package directory (site-packages/efinance/data), which is root-owned
in the image. Since the container runs as the non-root dsa user
(uid 1000), every A-share fetch failed with PermissionError, forcing
EfinanceFetcher (the default priority-0 source) to fail and fall back.

Pre-create and chown the efinance data dir to dsa at build time. The
path is resolved dynamically to avoid hardcoding the Python version.
This is image-internal static content, so the fix belongs at build
time rather than the runtime mount-repair in the entrypoint (ZhuLinsen#1263),
which only covers user-mounted volumes.

Fixes ZhuLinsen#1748

Co-authored-by: gang.wu@ximalaya.com <gang.wu@ximalaya.com>
Co-authored-by: mumu <42829555+ZhuLinsen@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ai documentation Improvements or additions to documentation size/L size/M testing

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants