chore: sync release-3.5 branch with main - #508
Conversation
--userns=keep-id alone doesn't override the base image's USER directive, so the process runs as UID 1001 (default) and can't write to host-owned files in the mounted volume. Adding --user ensures the correct UID. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…hub-io#493) ## Summary - The `pkg-gen` pre-commit hook fails with `PermissionError: [Errno 13] Permission denied: 'distribution/config.yaml'` - `--userns=keep-id` alone doesn't override the base image's `USER` directive (UID 1001), so the build container can't write to host-owned files in the mounted volume - Adding `--user=$(id -u):$(id -g)` alongside `--userns=keep-id` ensures the process runs as the host user ## Test plan - [x] `pre-commit run pkg-gen --all-files` passes locally with podman ## Summary by CodeRabbit * **Bug Fixes** * Improved container startup handling across Podman and Docker environments. * Fixed container user configuration so the correct user-flag arguments are now passed during launch. * **Chores** * Regenerated locked dependency manifests with updated pinned versions and hashes. Approved-by: nathan-weinberg Approved-by: eoinfennessy
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
OGX v1.2.0 requires TLS by default. Pass --insecure so the server starts without TLS certificates configured. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Doug Edgar <dedgar@redhat.com>
Signed-off-by: Doug Edgar <dedgar@redhat.com>
…datahub-io#494) ## Summary - Cherry-picks `4e63ac67` from `release-3.5` to align `main` with the version bump to `v1.2.0+rhaiv.0` ## Test plan - CI should pass — this is the same change already merged on `release-3.5` ## Summary by CodeRabbit * **New Release** * Updated the distribution to OGX version `1.2.0+rhaiv.0` (including updated image metadata). * **Updates** * Refreshed pinned dependencies for document processing, AI, and observability/Prometheus instrumentation. * Run invocation now consistently uses `ogx run --insecure` in all entrypoint execution paths. * **Documentation** * Updated distribution README release information to `1.2.0+rhaiv.0`. * **Tests** * Expanded integration test skipping for specific OpenAI streaming cases. Approved-by: eoinfennessy Approved-by: rhdedgar
The vLLM container image pull was failing with "no space left on device" on GitHub Actions runners. Add the repo's free-disk-space composite action early in the build-test job to reclaim space before the heavy image build and container pulls. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
Upstream OGX PR #6264 introduced a bug in the vLLM provider's
anthropic_messages() method: it constructs the URL as
`{base_url}/v1/messages` but base_url already contains `/v1`,
resulting in `http://localhost:8000/v1/v1/messages` (404).
Skip the test until fixed upstream. The Messages API is still
covered by the dedicated messages-vllm.yml workflow.
ogx-ai/ogx#6290
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
…io#497) ## Summary - The `build-test` job was failing with `no space left on device` when pulling the vLLM container image ([failed run](https://github.com/opendatahub-io/ogx-distribution/actions/runs/29271038404/job/86888475345?pr=488)) - Add the repo's existing `free-disk-space` composite action early in the `build-test` job to reclaim runner disk space before the image build and container pulls - Skip the Messages API smoke test for local vLLM due to an upstream OGX bug where the vLLM provider constructs a double `/v1/v1/messages` URL path ([ogx-ai/ogx#6290](ogx-ai/ogx#6290)) ## Test plan - [ ] CI `build-test` job passes without hitting disk space errors - [ ] Messages API smoke test is skipped with a clear log message referencing the upstream bug 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by CodeRabbit - **Chores** - Improved build reliability by freeing disk space during distribution builds. - Updated pinned dependency versions and security hashes. - **Tests** - Temporarily skipped the Messages API smoke test due to current gateway and local vLLM routing limitations. Approved-by: rhdedgar Approved-by: eoinfennessy
Bumps [actions/stale](https://github.com/actions/stale) from 10.3.0 to 10.4.0. - [Release notes](https://github.com/actions/stale/releases) - [Changelog](https://github.com/actions/stale/blob/main/CHANGELOG.md) - [Commits](actions/stale@eb5cf3a...1e223db) --- updated-dependencies: - dependency-name: actions/stale dependency-version: 10.4.0 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.2.0 to 8.3.2. - [Release notes](https://github.com/astral-sh/setup-uv/releases) - [Commits](astral-sh/setup-uv@fac544c...11f9893) --- updated-dependencies: - dependency-name: astral-sh/setup-uv dependency-version: 8.3.2 dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com>
Add _FILE variant support for 18 secret environment variables in the container entrypoint. When a _FILE-suffixed variable (e.g. OPENAI_API_KEY_FILE) points to a mounted file, its contents are read into the base variable at startup. This avoids exposing secrets through /proc/1/environ and subprocess environments in Kubernetes deployments. Add a pre-commit sync check in build.py that detects secret fields in build.yaml (by field name heuristic) and verifies they all have corresponding _FILE entries in entrypoint.sh, so new providers with secrets cannot be added without extending _FILE support. Add auto-generated documentation in distribution/README.md with the supported variable list and a Kubernetes Pod spec example. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
The verify_file_secrets_sync function imports yaml.safe_load but pyyaml was not listed in the pkg-gen hook's additional_dependencies, causing CI to fail with ModuleNotFoundError. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
The pkg-gen hook uses `language: script`, which doesn't support `additional_dependencies`. Move pyyaml to the `uv run --with` command in run.sh where the build actually executes. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ub-io#486) ## Summary - Add `_FILE` variant support for 18 secret env vars in the container entrypoint — when e.g. `OPENAI_API_KEY_FILE` points to a mounted file, its contents are read into `OPENAI_API_KEY` at startup, avoiding exposure via `/proc/1/environ` and subprocess environments - Add a pre-commit sync check in `build/build.py` that auto-detects secret fields in `build/build.yaml` by field name heuristic and fails the build if any are missing from the entrypoint's `_FILE` resolution list - Add auto-generated documentation in `distribution/README.md` with supported variable list and Kubernetes Pod spec example ## Test plan - [ ] `shellcheck distribution/entrypoint.sh` passes - [ ] `bash tests/test_file_secrets.sh` — 8 unit tests covering: file→env resolution, `_FILE` cleanup, base variable preservation, mutual exclusion error, missing file error, trailing newline stripping, special characters, noop - [ ] `pre-commit run --all-files` passes (sync check + docs regeneration) - [ ] Verify sync check: temporarily add a secret field to `build/build.yaml` without updating `entrypoint.sh` → `build/build.py` errors with a message naming the missing var 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by CodeRabbit * **New Features** * Added support for providing container secrets through mounted files using `_FILE` environment variables. * Resolves `_FILE` values at startup, exports the secret content, and unsets the `_FILE` variables. * Validates and fails on conflicting base and `_FILE` values, plus missing or non-regular files. * **Documentation** * Added a “Mounting Secrets as Files” section with Kubernetes examples and supported variable mappings. * **Tests** * Added `entrypoint.sh` `_FILE` secret resolution tests covering success, error cases, and file content handling. Approved-by: Artemon-line Approved-by: cdoern
Upstream OGX redacts known field names (api_key, api_token, password) but any provider whose secret uses a different field name would appear in plaintext in the uploaded log artifacts. Add a defense-in-depth scrub step that replaces actual secret env-var values with ***REDACTED*** in all log files before the upload-artifact step. Also add a pre-commit hook (check-secret-scrub) that ensures the scrub list stays in sync with smoke.sh — it greps for secret-looking env vars (KEY, TOKEN, PASSWORD, SECRET, CREDENTIAL) passed to the container and fails if any are missing from the workflow's scrub list. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
Add scrub steps before upload-artifact in the three responses workflows (OpenAI, Vertex AI, vLLM MaaS) to match the pattern in redhat-distro-container.yml. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move the inline Python scrubber into tests/scrub_secrets.sh and call it from all workflows that upload artifacts. Each workflow passes its own glob pattern and secret env var names as arguments, eliminating the duplicated Python snippet. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…ahub-io#488) ## Summary - Add a defense-in-depth scrub step to the CI workflow that replaces actual secret env-var values with `***REDACTED***` in all log files before the `upload-artifact` step - Add a `check-secret-scrub` pre-commit hook that ensures the scrub list stays in sync with `smoke.sh` — greps for secret-looking env vars (`KEY`, `TOKEN`, `PASSWORD`, `SECRET`, `CREDENTIAL`) passed to the container and fails if any are missing from the workflow's scrub list ## Test plan - [x] `pre-commit run check-secret-scrub --all-files` passes - [x] Temporarily removing a var from the scrub list causes the hook to fail with a clear error message - [ ] CI run uploads log artifacts with redacted values 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by CodeRabbit * **New Features** * Added automatic redaction of sensitive values from generated log and test-result artifacts before they’re uploaded. * Introduced CI secret-scrubbing steps across multiple response-test workflows. * Added a pre-commit hook to validate consistency between local smoke-test secret inputs and CI scrubbing configuration. * **Bug Fixes** * Reduced the risk of accidentally exposing tokens, passwords, credentials, and related secret values in uploaded artifacts. Approved-by: Artemon-line Approved-by: cdoern
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Replace GNU-only `grep -P` (Perl regex) with `grep -oE` and `sed` so the pre-commit hook works on macOS where `-P` is not available. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
The monolithic build/build.py is decomposed into: - build/common.py (shared BuildConfig) - build/gen_config.py (config.yaml generation) - build/gen_lockfile.py (lock file compilation, Linux-only) - build/gen_containerfile.py (Containerfile generation) - build/verify_secrets.py (entrypoint.sh secret sync verification) Each script uses inline PEP 723 metadata for uv run. The gen-lockfile pre-commit hook is removed (will move to a cron workflow); CI workflows call uv run directly since runners are already Linux. A platform guard in gen_lockfile.py directs macOS/Windows users to run_gen_lockfile.sh. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
All local pre-commit hooks now use `uv run` (language: system) so setup-uv is the only runtime dependency. setup-uv provides Python for pre-commit itself. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Replace raw subprocess.run() with the existing _run() helper for consistent failure diagnostics across all subprocess calls. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Constrain PEP 723 inline dependencies to major-version ranges to limit exposure to unexpected breaking changes or supply chain risk. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
PEP 686 (UTF-8 mode default) only applies from Python 3.15, but these scripts declare requires-python >= 3.12. On Windows with 3.12-3.14, open()/read_text()/write_text() default to the locale encoding (typically cp1252), not UTF-8. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…pendatahub-io#499) ## Summary - Decomposes the monolithic `build/build.py` into single-responsibility scripts: `gen_config.py`, `gen_lockfile.py`, `gen_containerfile.py`, `verify_secrets.py`, and a shared `common.py` (BuildConfig) - Removes the `gen-lockfile` pre-commit hook (lockfile generation will move to a cron workflow) - CI workflows call `uv run build/gen_lockfile.py` directly instead of using the container wrapper (`run_gen_lockfile.sh`), since runners are already Linux - Adds a Linux platform guard to `gen_lockfile.py` — macOS/Windows users are directed to `./build/run_gen_lockfile.sh` - Fixes POSIX-incompatible `grep -P` in `check_secret_scrub_list.sh` ## Test plan - [x] Verify `uv run build/gen_lockfile.py` fails with a clear message on macOS - [x] Verify `pre-commit run --all-files` succeeds without the lockfile hook - [ ] Verify CI build/test workflow passes on Linux runners 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by CodeRabbit * **Build and Distribution** * Improved generation of distribution configuration, container definitions, and pip lockfiles using dedicated build scripts. * Added automated verification that secret environment variables match the runtime secret handling. * Updated container metadata/config labels and regenerated artifacts; included a dependency pin update (asgiref 3.12.0). * **CI and Testing** * Updated CI workflows to run artifact generation directly and to use faster tooling for pre-commit. * Enhanced secret-list parsing for more reliable checks. * **Documentation** * Refreshed README, CLAUDE, and CI/testing instructions to match the streamlined regeneration workflow. Approved-by: nathan-weinberg Approved-by: skamenan7
…datahub-io#496) Bumps [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) from 8.2.0 to 8.3.2. <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/astral-sh/setup-uv/commit/11f9893b081a58869d3b5fccaea48c9e9e46f990"><code>11f9893</code></a> chore: roll up Dependabot updates (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/948">#948</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/f79855603231e1609d02bec6956bd0e05cbc46b5"><code>f798556</code></a> docs: update version references to v8.3.1 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/946">#946</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/e80544d808267c93733c3fd1e2c8c65e0c8707d6"><code>e80544d</code></a> chore: update known checksums for 0.11.28 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/947">#947</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/f98e06938123ccabd21905ea5d0069192241f9f1"><code>f98e069</code></a> Change update-docs PR labels from 'update-docs' to 'documentation' (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/945">#945</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/cd462639a967553a16241af35461402a96978d48"><code>cd46263</code></a> chore: update known checksums for 0.11.27 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/944">#944</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/11245c7e122cd1c2297e8115d1e43fe1570f6270"><code>11245c7</code></a> docs: update version references to v8.3.0 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/939">#939</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/d31148d669074a8d0a63714ba94f3201e7020bc3"><code>d31148d</code></a> Strip environment markers from detected uv dependency pins (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/938">#938</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/17c398959b4611a88929fabb5c563a8e43a0ff60"><code>17c3989</code></a> Fix cache keys for Python version ranges (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/937">#937</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/3cc3c11fdf511cab39136b7c946d973d4ad0df20"><code>3cc3c11</code></a> chore(deps): roll up Dependabot updates (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/936">#936</a>)</li> <li><a href="https://github.com/astral-sh/setup-uv/commit/9225f843d7a9f80a757cf25ef48901fda69ba4bc"><code>9225f84</code></a> chore(deps): bump release-drafter/release-drafter from 7.3.1 to 7.4.0 (<a href="https://redirect.github.com/astral-sh/setup-uv/issues/924">#924</a>)</li> <li>Additional commits viewable in <a href="https://github.com/astral-sh/setup-uv/compare/fac544c07dec837d0ccb6301d7b5580bf5edae39...11f9893b081a58869d3b5fccaea48c9e9e46f990">compare view</a></li> </ul> </details> <br /> Approved-by: nathan-weinberg Approved-by: eoinfennessy
…ahub-io#495) Bumps [actions/stale](https://github.com/actions/stale) from 10.3.0 to 10.4.0. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/actions/stale/releases">actions/stale's releases</a>.</em></p> <blockquote> <h2>v10.4.0</h2> <h2>What's Changed</h2> <h3>Bug Fix</h3> <ul> <li>Fixed <code>only-issue-types</code> validation by <a href="https://github.com/trueberryless"><code>@trueberryless</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1338">actions/stale#1338</a></li> </ul> <h3>Dependency Updates</h3> <ul> <li>Bump undici to 6.27.0 via override, clean up stale license files, and version to 10.4.0. by <a href="https://github.com/dependabot"><code>@dependabot</code></a> in <a href="https://redirect.github.com/actions/stale/pull/1342">actions/stale#1342</a></li> </ul> <h2>New Contributors</h2> <ul> <li><a href="https://github.com/trueberryless"><code>@trueberryless</code></a> made their first contribution in <a href="https://redirect.github.com/actions/stale/pull/1338">actions/stale#1338</a></li> </ul> <p><strong>Full Changelog</strong>: <a href="https://github.com/actions/stale/compare/v10.3.0...v10.4.0">https://github.com/actions/stale/compare/v10.3.0...v10.4.0</a></p> </blockquote> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/actions/stale/commit/1e223db275d687790206a7acac4d1a11bd6fe629"><code>1e223db</code></a> Bump undici to 6.27.0 via override, clean up stale license files, and version...</li> <li><a href="https://github.com/actions/stale/commit/9461cb10066d1553762bac6a02599ab8c26b14dd"><code>9461cb1</code></a> fix: <code>only-issue-types</code> does not affect PRs (<a href="https://redirect.github.com/actions/stale/issues/1338">#1338</a>)</li> <li>See full diff in <a href="https://github.com/actions/stale/compare/eb5cf3af3ac0a1aa4c9c45633dd1ae542a27a899...1e223db275d687790206a7acac4d1a11bd6fe629">compare view</a></li> </ul> </details> <br /> Approved-by: nathan-weinberg Approved-by: eoinfennessy
## Summary - Bumps the vLLM CPU base image from `v0.23.0` to `v0.25.0` in `vllm/Containerfile` ## Test plan - [ ] CI builds the vLLM CPU container image successfully - [ ] Smoke tests pass against the new vLLM version - [ ] Integration tests pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) ## Summary by CodeRabbit * **Chores** * Updated the application’s container image to use the newer vLLM CPU runtime version 0.25.0. * Existing model setup and serving behavior remain unchanged. Approved-by: eoinfennessy Approved-by: EleanorWho
Signed-off-by: Nathan Weinberg <nweinber@redhat.com>
# What does this PR do? ## Test Plan Approved-by: rhdedgar Approved-by: derekhiggins
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (4)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Central YAML (base), Organization UI (inherited) Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
|
|
This pull request has merge conflicts that must be resolved before it can be merged. @nathan-weinberg please rebase it. https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork |
|
Tick the box to add this pull request to the merge queue (same as
|
c220d19
into
opendatahub-io:release-3.5
What does this PR do?
Test Plan