Job-level instrumentation: make cache-package install async and add a minimal synchronous shell-based cache restore

[plengauer]

Summary

In the job-level instrumentation, cache restoration currently blocks on installing the npm packages from the job-level folder's package.json (which provides @actions/cache) before it can restore. This npm install is observed to take up to ~20s. Since those packages are needed later anyway, the proposal is:

1. Make the npm install of the cache-restoration packages asynchronous / background, the same way other setup tasks are already dispatched, since the installed package is only needed later in the job (e.g. for saving the cache at the end).
2. Add a minimal, synchronous, pure-shell cache restore (curl + tar, both already present on the runner) that runs immediately without waiting on npm, so the job can proceed with a warm cache while the install continues in the background.

This removes the npm install from the critical path of cache restoration while keeping the real package available by the time it's actually required.

Motivation

The install is a hard dependency-resolution step that currently gates restore. The restore itself only needs a couple of HTTP calls plus a tar extraction — none of which require Node or the npm dependency tree. Decoupling the two means the up-to-~20s install overlaps with useful work instead of blocking it.

Proposed sequence for the synchronous shell restore

The cache-restore protocol (GitHub Actions cache service v2; v1 was sunset Feb 2025) can be driven directly over HTTP. The general sequence:

Step 1 — Resolve the cache version (recommended, avoids reimplementing the hash).
The version is a content hash that the download request must match exactly. Rather than recomputing it in shell (it's a sha256 over the sorted paths + compression method + a version salt, and a mismatch causes silent permanent misses), it can be read directly from the public, documented, stable REST API:

GET https://api.github.com/repos/{owner}/{repo}/actions/caches?key={cache_key}

with a standard GitHub API token (actions: read). Each entry in the actions_caches array includes key, ref, size_in_bytes, and crucially version. This is the lookup that lets the shell path stay simple and correct.

Step 2 — Request a signed download URL (cache service v2, internal Twirp endpoint).

POST {ACTIONS_RESULTS_URL}/twirp/github.actions.results.api.v1.CacheService/GetCacheEntryDownloadURL
Authorization: Bearer {ACTIONS_RUNTIME_TOKEN}
Content-Type: application/json

{ "key": "<cache_key>", "restore_keys": [], "version": "<version from step 1>" }

Both ACTIONS_RESULTS_URL and ACTIONS_RUNTIME_TOKEN are injected into the job environment. Note this is a different credential from the REST API token in step 1. The response (fields observed as snake_case: ok, signed_download_url, matched_key) contains a pre-signed blob-storage URL. On a cache miss, ok is false.

Step 3 — Download the archive.
Plain unauthenticated GET on signed_download_url (it's pre-signed) → write to a temp file.

Step 4 — Extract.
tar with zstd (--use-compress-program="zstd -d --long=30") when zstd is available, else gzip (-z), preserving absolute paths (-P) as the archive was created with them.

Important caveats to bake into the implementation

• Behavioral parity: the official restoreCache() swallows all non-validation errors and simply doesn't restore (never fails the caller). The shell path should mirror this: any non-200 / cache-miss / download failure should be a clean no-op exit, not a job failure.
• Cold-cache / first run: the step-1 lookup returns no matching entry → clean no-op, matching restoreCache() returning undefined.
• Fallback: if the direct path fails for any reason (protocol drift, etc.), it should fall back to the async-installed @actions/cache once that install completes — so a protocol change costs the ~20s again rather than breaking the pipeline.
• Unverified detail worth confirming once: the exact wire field casing (signed_download_url / matched_key) and the version value should be confirmed against a real run with ACTIONS_STEP_DEBUG=true, which logs the actual request/response JSON. The REST version lookup in step 1 removes the largest source of uncertainty.
• This is the v2 internal service protocol. GitHub has not published the Twirp endpoint as a stable public contract (unlike the REST cache-management API used in step 1), so it can change without notice — hence the fallback requirement above.

Non-goals / scope

This issue intentionally describes the approach and research only; no implementation is included here.

[github-actions]

Analysis

This is a well-scoped performance enhancement for the job-level instrumentation. The key changes needed:

Implementation Plan

1. Make npm install async — move the npm install of @actions/cache packages to a background process (same pattern as other async setup tasks already in the instrumentation), coordinated so the package is ready before it's actually needed (cache save at end of job).

2. Add synchronous shell-based cache restore — using curl + tar (both already on runners), following the v2 cache service protocol:

   • REST API lookup to get version: GET https://api.github.com/repos/{owner}/{repo}/actions/caches?key={key}
   • Twirp GetCacheEntryDownloadURL POST with ACTIONS_RESULTS_URL + ACTIONS_RUNTIME_TOKEN
   • Unauthenticated GET on the signed URL → temp file
   • tar extraction with zstd (fallback to gzip), preserving absolute paths (-P)

3. Fallback — if shell restore fails for any reason, fall back silently to the npm-installed @actions/cache once ready; all failures must be non-fatal (match restoreCache() behavior of returning undefined on miss/error).

Key files to modify

• actions/instrument/ — job-level instrumentation scripts handling cache setup/restore

Verification

• Test with ACTIONS_STEP_DEBUG=true to confirm exact wire field names (signed_download_url, matched_key, version from REST)
• Confirm the ~20s improvement on a cold-start job run

Generated by Autotriage for #3891 · sonnet46 16.4 AIC · ⌖ 5.07 AIC · ⊞ 5.2K · ◷