Skip to content

Commit c0b5cd1

Browse files
committed
linux_job_v3: refresh the HF cache unattended, and honour the label off a PR
The label is read from github.event.pull_request.labels, which only exists on a pull_request event, so the ciflow tag run sitting next to a labelled pull request ignores it, writes at the read-only /mnt/hf_cache mount, and dies on any model the seeded cache does not carry: OSError: [Errno 30] Read-only file system: '/mnt/hf_cache/hub/models--...' Same for a push to main after that pull request merges. Resolve the labels when the event does not carry them, in one request: a ciflow tag names its pull request, as pytorch's filter_test_configs.py also relies on, otherwise ask which pull requests contain the commit. A schedule or workflow_dispatch run now refreshes unconditionally, matching pytorch's _linux-test.yml. That is what keeps the shared cache seeded; without it the only way to add a model is to label a pull request, which is also the only way to notice one was needed. The read-only mount stays the default otherwise, so nothing changes for a run that does not ask for a refresh. Authored with Claude Code.
1 parent d1ad90a commit c0b5cd1

1 file changed

Lines changed: 39 additions & 1 deletion

File tree

.github/workflows/linux_job_v3.yml

Lines changed: 39 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -223,9 +223,47 @@ jobs:
223223
# mount; the sync step below pushes it back to S3. Otherwise leave the mount.
224224
- name: Resolve HF cache mode
225225
shell: bash
226+
env:
227+
GH_TOKEN: ${{ github.token }}
228+
REPO: ${{ github.repository }}
229+
EVENT_PR: ${{ github.event.pull_request.number }}
230+
EVENT_HAS_LABEL: ${{ contains(github.event.pull_request.labels.*.name, 'ci-refresh-hf-cache') }}
226231
run: |
227232
set -euo pipefail
228-
if [[ "${{ contains(github.event.pull_request.labels.*.name, 'ci-refresh-hf-cache') }}" == "true" ]]; then
233+
234+
# Reads a GitHub REST path, authenticated if the token can, otherwise
235+
# anonymously. Never fails the step: no answer means no refresh.
236+
gh_get() {
237+
curl -fsSL -H "Accept: application/vnd.github+json" \
238+
-H "Authorization: Bearer ${GH_TOKEN}" "https://api.github.com/$1" 2>/dev/null \
239+
|| curl -fsSL -H "Accept: application/vnd.github+json" \
240+
"https://api.github.com/$1" 2>/dev/null \
241+
|| true
242+
}
243+
244+
REFRESH=0
245+
if [[ "${GITHUB_EVENT_NAME}" == "schedule" || "${GITHUB_EVENT_NAME}" == "workflow_dispatch" ]]; then
246+
# An unattended refresh, matching pytorch's _linux-test.yml. This is
247+
# what keeps the shared cache seeded: without it the only way to add a
248+
# model is for someone to label a pull request.
249+
REFRESH=1
250+
elif [[ "${EVENT_HAS_LABEL}" == "true" ]]; then
251+
REFRESH=1
252+
elif [[ -z "${EVENT_PR}" ]]; then
253+
# A push or tag run carries no labels. A ciflow tag names its pull
254+
# request, as pytorch's filter_test_configs.py also relies on; the
255+
# fallback costs the same request but a much larger response.
256+
if [[ "${GITHUB_REF_NAME:-}" =~ ^ciflow/[^/]+/([0-9]+)$ ]]; then
257+
LABELS=$(gh_get "repos/${REPO}/issues/${BASH_REMATCH[1]}/labels")
258+
else
259+
LABELS=$(gh_get "repos/${REPO}/commits/${GITHUB_SHA}/pulls")
260+
fi
261+
case "${LABELS}" in
262+
*'"ci-refresh-hf-cache"'*) REFRESH=1 ;;
263+
esac
264+
fi
265+
266+
if [[ "${REFRESH}" == "1" ]]; then
229267
{
230268
echo "HF_CACHE_REFRESH=1"
231269
echo "TRANSFORMERS_OFFLINE=0"

0 commit comments

Comments
 (0)