[ci] Switch to the CI-infra-hosted PyPI index proxy - #65687
Conversation
The mirror fleet now hosts the index proxy itself, at <mirror>/_pypi/simple: index pages come back with every files.pythonhosted.org URL rewritten to the mirror, over HTTPS on a public CA. That removes the reason the job-local proxy existed, so this deletes it and its whole support structure: - ci/pypi_proxy_profile.sh probes the hosted index and exports one URL for every consumer: pip/uv on the agent, docker builds (RAYCI_IMAGE_PIP_INDEX_URL, no --add-host and no trusted-host since the address is neither loopback nor plain HTTP), and nested containers. No local process, port, venv, or interpreter hunt. - ci/pypi_proxy_agent.sh and ci/ray_ci/macos/pypi_proxy.sh become thin shims sourcing the shared profile script. - ci/pypi_index_proxy.py and ci/install_pypi_proxy.sh are deleted; the forge and manylinux layers now install only the profile.d hook and the bazel downloader helper (in /etc/rayci), and forge drops the 3.12 interpreter that existed only for the proxy venv. Fail-open semantics are unchanged: the probe decides per step, and an agent that cannot reach the hosted index resolves from public PyPI with nothing exported. The bazel downloader rewrite is also unchanged; it needs only the mirror, and the index probe implies mirror reachability. Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Thomas Desrosiers <thomas@anyscale.com>
…dex does not Everything here forwarded values that nothing sets since the switch to the mirror-hosted HTTPS index; each leg existed for the old proxy's non-loopback plain-HTTP addresses: - The trusted-host graph: PIP_TRUSTED_HOST/UV_INSECURE_HOST passthrough in .bazelrc and ci/ray_ci/container.py, the read-and-forward blocks in linux_container.py and byod/build.py, ARG/ENV in eight Dockerfiles, RAYCI_IMAGE_PIP_TRUSTED_HOST in twelve wanda specs, and the tests pinning those lists. The hosted index is HTTPS on a public CA. - PIP_EXTRA_INDEX_URL/UV_EXTRA_INDEX_URL forwarding: never set by anything; the real extra indexes live inside the deplock files. - byod/build.py's --add-host rayci.localhost: nothing resolves that name any more. - The baked ENV PIP_INDEX_URL/UV_INDEX_URL becomes ARG-only in the same eight Dockerfiles: ARG is visible to every RUN in the stage, which is all the bake ever bought, and not persisting it means images no longer carry a CI-only index into runtime -- byod images run on Anyscale clusters outside the CI VPCs, where the baked value could never resolve. That also deletes the profile's reset branch, which existed only to neutralize the bake. - The one-writer cleanups: linux_container.py and byod/build.py read only RAYCI_IMAGE_PIP_INDEX_URL, and bazel_mirror_downloader.sh takes its mirror argument without a dead default chain. Kept, each with a live consumer: the per-step probe and fail-open, the RAYCI_PYPI_MIRROR_URL override (hostname migrations happen), and RAYCI_IMAGE_PIP_INDEX_URL (docker builds inherit nothing from the step environment). Co-Authored-By: Claude <noreply@anthropic.com> Signed-off-by: Thomas Desrosiers <thomas@anyscale.com>
1d5aea4 to
5f03567
Compare
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request simplifies the Ray CI PyPI caching mirror integration by replacing the local rewriting proxy with a centralized, mirror-hosted index. This change eliminates the need for a local proxy process, its installer, and complex trusted-host configurations across various environments (Buildkite agents, macOS, and Docker containers). Feedback on the changes suggests improving the robustness of the diagnostic host resolution in ci/pypi_proxy_profile.sh by dynamically stripping any protocol prefix (e.g., http:// or https://) rather than hardcoding https://.
| # mirror is deployed and this fleet has no route to it". getent is Linux-only; | ||
| # on macOS the address line is simply absent. | ||
| local resolved | ||
| resolved="$(getent hosts "${mirror#https://}" 2>/dev/null | awk '{print $1}' | paste -sd, -)" |
There was a problem hiding this comment.
To make the diagnostic host resolution more robust and support mirrors configured with http:// (e.g., in local development or test environments), we should strip any protocol prefix using ${mirror#*://} instead of hardcoding ${mirror#https://}. This is also consistent with how the host is extracted in ci/bazel_mirror_downloader.sh.
| resolved="$(getent hosts "${mirror#https://}" 2>/dev/null | awk '{print $1}' | paste -sd, -)" | |
| resolved="$(getent hosts "${mirror#*://}" 2>/dev/null | awk '{print $1}' | paste -sd, -)" |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
Reviewed by Cursor Bugbot for commit 5f03567. Configure here.
| ENV PIP_TRUSTED_HOST=${RAYCI_IMAGE_PIP_TRUSTED_HOST} | ||
| ENV UV_INSECURE_HOST=${RAYCI_IMAGE_PIP_TRUSTED_HOST} | ||
| ARG PIP_INDEX_URL=${RAYCI_IMAGE_PIP_INDEX_URL:-https://pypi.org/simple} | ||
| ARG UV_INDEX_URL=${RAYCI_IMAGE_PIP_INDEX_URL:-https://pypi.org/simple} |
There was a problem hiding this comment.
CI bases drop inherited pip index
High Severity
Switching PIP_INDEX_URL/UV_INDEX_URL from ENV to ARG on the CI base images covers those builds only and no longer persists into derived images. Downstream Dockerfiles such as base.ml.Dockerfile, base.build.Dockerfile, core.build.Dockerfile, data.build.Dockerfile, and doc.build.Dockerfile install packages with pip/uv but declare no index build-arg, and their wanda specs do not pass RAYCI_IMAGE_PIP_INDEX_URL. Those builds previously inherited the baked index and will now resolve from public PyPI again, re-exposing the files.pythonhosted.org 502 failures this work was meant to avoid. The ARG-only choice fits BYOD images that run outside CI; these bases are CI-only and are the root of that install tree.
Additional Locations (1)
Reviewed by Cursor Bugbot for commit 5f03567. Configure here.
1 participant
CI infra runs the PyPI index proxy on the mirror fleet now, so this repo no longer needs to start its own copy in every job. Switching to the hosted endpoint deletes the local proxy and everything that existed to reach it:
ci/pypi_index_proxy.py,ci/install_pypi_proxy.sh, and the per-image interpreter/venv installs are deleted.ci/pypi_proxy_profile.shstays as the single decision file: probe the hosted index once per step, export the index variables on success, export nothing on failure and the step resolves from public PyPI as before..bazelrcandcontainer.pypassthroughs, build-arg forwarding inlinux_container.pyandbyod/build.py, ARG/ENV in eight Dockerfiles, build_args in twelve wanda specs). It existed because the local proxy's addresses were non-loopback plain HTTP, which pip refuses without a named trusted host; the hosted endpoint is HTTPS.PIP_EXTRA_INDEX_URL/UV_EXTRA_INDEX_URLforwarding is deleted (nothing ever set them; the real extra indexes live inside the deplock files), as is BYOD's--add-host rayci.localhost(nothing resolves that name).ENV PIP_INDEX_URL/UV_INDEX_URL; the value arrives asARG, which covers the build and does not persist: BYOD images run outside CI, where a baked CI index can never resolve.🤖 Generated with Claude Code