[ci] Pin test_runtime_env_cache_with_pip_check to PyPI (#65713)

## What

Sets `PIP_INDEX_URL` to PyPI for the runtime_env in
`test_runtime_env_cache_with_pip_check`, so the pip versions that test
pins can resolve. One key added to one dict; every assertion and every
pip version is unchanged.

## Why

The test asks for pip **20.2.3** and **21.3.1** by design — it covers
`runtime_env` `pip_version` handling, `pip_check` conflict detection,
and cache-keying on both. Both releases predate PEP 691 (added in pip
22.2), so they can only read an index page served as `text/html`.

CI now points `PIP_INDEX_URL` at the mirror-hosted index (#65687). The
mirror serves both representations, but caches entries keyed on URL
alone while PyPI answers `Vary: Accept` — so an entry warmed by a
JSON-speaking client (uv, or any modern pip) is handed back to these,
which skip it:

```
WARNING: Skipping page .../simple/idna/ because the GET request got Content-Type:
application/vnd.pypi.simple.v1+json. The only supported Content-Type is text/html
ERROR: Could not find a version that satisfies the requirement idna==2.4 (from versions: none)
```

The resolve then fails before it can produce the dependency conflict the
test asserts on.

## Why not just upgrade the pip

There is no pip here to upgrade. The runtime_env agent log shows the
venv is seeded correctly and then downgraded on request:

```
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, ...)
  added seed packages: pip==26.0.1, setuptools==82.0.1
INFO pip.py:171 -- Installing pip with version ==20.2.3
  Found existing installation: pip 26.0.1
  Uninstalling pip-26.0.1: Successfully installed pip-20.2.3
```

The old pip is the input under test — and in production, an input from
users.

Raising the pins is not available either, because the two versions
deliberately straddle pip's resolver change. pip's conflict-detecting
resolver became the default in **pip 20.3**, and `pip_check` runs `pip
check` *after* install (`pip.py:183-196`):

| step | pip_version | pip_check | resolver | what it proves |
|---|---|---|---|---|
| 1 | `==20.2.3` | False | legacy | install **succeeds** despite the
conflict → `assert ray.get(...) is True` |
| 2 | `==21.3.1` | False | new | install **fails at resolve** → asserts
`"The conflict is caused by:"` |
| 3 | `==20.2.3` | True | legacy | install succeeds, then **`pip
check`** catches it |

Step 2's pin could move to any modern pip. Steps 1 and 3 cannot: on a
modern pip the resolver refuses the install outright, so step 1's
assertion breaks and step 3 becomes unreachable — "install succeeds,
then `pip check` catches the conflict" cannot be exercised if the
install never succeeds, which drops Ray's `_check_pip` conflict path
from coverage entirely.

The asymmetry also carries the cache assertions: misses are detected
*behaviourally*, via the same packages producing a different outcome
under a different `pip_version`. Two modern pins would fail identically,
leaving no way to tell a cache hit from a miss.

Setting the index per runtime_env keeps everything intact:

- `env_vars` win over the inherited environment — `RuntimeEnvPipManager`
does `os.environ.copy()` then `.update(runtime_env.env_vars())`
(`pip.py:145-146`), and `_pip_env` is used for all three pip steps
(bootstrap, install, check).
- `env_vars` do **not** feed `_get_pip_hash()`, which takes only the pip
dict — so the cache-hit/miss behaviour this test exercises is unchanged.
The three variants still differ only in `pip_version` / `pip_check`.

## Scope

Only the one test needs it. Its siblings in the same file are untouched
and passing:

| test | pip_version | needs change? |
|---|---|---|
| `test_runtime_env_with_pip_config` | `==24.1.2`, `<24.2, >19` | no —
reads either representation |
| `test_runtime_env_with_conflict_pip_version` | `<19,>19` | no —
asserts a failure it gets anyway |
| `test_runtime_env_cache_with_pip_check` | `==20.2.3`, `==21.3.1` |
**yes** |

## This is a stopgap

The durable fix is for the mirror to vary its cache entries on `Accept`.
`runtime_env` documents arbitrary user-chosen pip versions, so no pin in
this repo protects a user who asks for an old one against a
`Vary`-ignoring cache. Filed separately; this unblocks #65687.

## Testing

The failure needs a JSON-warmed cache entry on the CI mirror, so it is
not reproducible locally — the mirror is split-horizon and unreachable
outside the CI VPCs. Evidence is from the CI run itself, plus source
verification of the three preconditions above:

```
# The failure, premerge 72421 (on #65687), core: wheel tests
//python/ray/tests:test_runtime_env_conda_and_pip_5
  test_runtime_env_cache_with_pip_check   FAILED
  test_runtime_env_with_pip_config        PASSED
  test_runtime_env_with_conflict_pip_version PASSED

# Root cause, from the job's failed_test_logs artifact
runtime_env_setup-01000000.log:
  added seed packages: pip==26.0.1
  INFO pip.py:171 -- Installing pip with version ==20.2.3
  WARNING: Skipping page .../simple/idna/ ... only supported Content-Type is text/html

# Preconditions verified in source
pip.py:145-146   env_vars override the inherited environment
pip.py:272,280,288  _pip_env used for bootstrap, install and check
pip.py:42        _get_pip_hash takes only the pip dict, so env_vars stay out of the cache key

# Lint
$ pre-commit run --files python/ray/tests/test_runtime_env_conda_and_pip_5.py
  ruff / black / pydoclint / semgrep / import order ... Passed
```

The real verification is premerge on this PR, and then #65687 rerunning
on top of it.

## Not a duplicate

`gh pr list --state open --search "test_runtime_env_conda_and_pip"` and
`"runtime_env pip_version PIP_INDEX_URL"` both return nothing. #65687 is
the change that surfaces this and does not touch `python/ray/tests`;
#65634 (merged) fixed the bazel-side `whl_library` pip, a different pip
on a different path.

## AI assistance

AI assistance was used: the root-cause analysis from the Buildkite
artifacts, the PEP 691 / cache-negotiation reasoning, and verifying the
three preconditions in `pip.py`. Every changed line was reviewed and
every command above was run and its output confirmed before requesting
review.


PREMERGE TEST:
https://buildkite.com/ray-project/premerge/builds/72430/list

Signed-off-by: elliot-barn <elliot.barnwell@anyscale.com>

Author: elliot-barn

python/ray/tests/test_runtime_env_conda_and_pip_5.py

@@ -70,7 +70,18 @@ def test_runtime_env_cache_with_pip_check(start_cluster):
             "packages": conflict_packages,
             "pip_version": "==20.2.3",
             "pip_check": False,
-        }
+        },
+        # Resolve from PyPI rather than whatever PIP_INDEX_URL this process
+        # inherited. The pip versions below are the point of the test, and both
+        # predate PEP 691 (added in pip 22.2), so they can only read an index page
+        # served as text/html. CI points pip at a caching mirror that keys entries
+        # on URL alone while PyPI answers `Vary: Accept`, so an entry warmed by a
+        # JSON-speaking client is then handed to these, and the resolve fails with
+        # "from versions: none" rather than the conflict this asserts on.
+        # runtime_env env_vars win over the inherited environment (pip.py
+        # _pip_env), and do not feed the pip hash, so the cache behaviour under
+        # test is unchanged.
+        "env_vars": {"PIP_INDEX_URL": "https://pypi.org/simple"},
     }
 
     @ray.remote