Skip to content

fix(cicd): experiment with using GHA cache with sccache#762

Open
chet wants to merge 1 commit intoNVIDIA:mainfrom
chet:cicd-use-gha-cache
Open

fix(cicd): experiment with using GHA cache with sccache#762
chet wants to merge 1 commit intoNVIDIA:mainfrom
chet:cicd-use-gha-cache

Conversation

@chet
Copy link
Copy Markdown
Contributor

@chet chet commented Mar 31, 2026

Description

So I've been trying to figure out how to speed up build times, because they've gotten suuuper slow. One call out was we're not actually really using sccache, which I played around with in #733, but it didn't quite do what I wanted. I went this route because the default GHA cache size (if we wanted to use Github caching) is 10GB, unless you pay for more. So I had started an internal discussion (can't link it here), and I was pointed to some people who said we actually do pay for more GHA cache, meaning we CAN use it.

That said, we may burn through our bumped up cache size pretty, quick, BUT, consider this an experiment. We want to see:

  • How quickly we burn through the cache.
  • If we see immediately noticable benefits.
  • What we're leaving on the table by not wiring up sccache to use a cache.

Another catch 22 is this has to be merged for subsequent PRs to leverage it (I can't do an A/B like I did with Azure testing), so once this gets merged, I can watch subsequent PR performance to see if we're benefitting from it.

Note that I do set specific "fallback" environment variables here too -- if for some reason there is an error using the GH cache, sccache will just skip using the cache, to allow the build to proceed (albeit without leveraging the GH cache).

Signed-off-by: Chet Nichols III chetn@nvidia.com

Type of Change

  • Add - New feature or capability
  • Change - Changes in existing functionality
  • Fix - Bug fixes
  • Remove - Removed features or deprecated functionality
  • Internal - Internal changes (refactoring, tests, docs, etc.)

Related Issues (Optional)

Breaking Changes

  • This PR contains breaking changes

Testing

  • Unit tests added/updated
  • Integration tests added/updated
  • Manual testing performed
  • No testing required (docs, internal refactor, etc.)

Additional Notes

@chet chet requested review from a team as code owners March 31, 2026 20:36
@github-actions
Copy link
Copy Markdown

🔐 TruffleHog Secret Scan

No secrets or credentials found!

Your code has been scanned for 700+ types of secrets and credentials. All clear! 🎉

🔗 View scan details

🕐 Last updated: 2026-03-31 20:38:26 UTC | Commit: 4db1036

@chet chet force-pushed the cicd-use-gha-cache branch from 4db1036 to ebbe8a2 Compare March 31, 2026 21:05
So I've been trying to figure out how to speed up build times, because they've gotten suuuper slow. One call out was we're not actually really using `sccache`, which I played around with in NVIDIA#733, but it didn't quite do what I wanted. I went this route because the default GHA cache size (if we wanted to use Github caching) is 10GB, unless you pay for more. So I had started an internal discussion (can't link it here), and I was pointed to some people who said we actually do pay for more GHA cache, meaning we CAN use it.

That said, we may burn through our bumped up cache size pretty, quick, BUT, consider this an experiment. We want to see:
- How quickly we burn through the cache.
- If we see immediately noticable benefits.
- What we're leaving on the table by not wiring up `sccache` to use a cache.

Another catch 22 is this has to be merged for subsequent PRs to leverage it (I can't do an A/B like I did with Azure testing), so once this gets merged, I can watch subsequent PR performance to see if we're benefitting from it.

Signed-off-by: Chet Nichols III <chetn@nvidia.com>
@chet chet force-pushed the cicd-use-gha-cache branch from ebbe8a2 to 33241e4 Compare March 31, 2026 21:30
@chet
Copy link
Copy Markdown
Contributor Author

chet commented Mar 31, 2026

lol -- and i'm getting an error message (same as docker/build-push-action#1485)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant