Skip to content

[ZeroGPU] CUDA placement note#2368

Merged
cbensimon merged 2 commits intomainfrom
zerogpu-cuda-placement
Apr 9, 2026
Merged

[ZeroGPU] CUDA placement note#2368
cbensimon merged 2 commits intomainfrom
zerogpu-cuda-placement

Conversation

@cbensimon
Copy link
Copy Markdown
Collaborator

@cbensimon cbensimon commented Apr 9, 2026

Agents (also some human devs) reading https://huggingface.co/docs/hub/spaces-zerogpu tend to lazy-load models inside @spaces.GPU (despite the example code snippet), probably out of caution. Let's make it more explicit


Note

Low Risk
Low risk documentation-only change that clarifies where models should be moved to CUDA and why lazy-loading inside @spaces.GPU is inefficient.

Overview
Clarifies ZeroGPU guidance around @spaces.GPU by adding a prominent note that the decorator is effect-free outside ZeroGPU environments.

Adds a new Model loading section explicitly instructing users to move models to cuda at module import time (not inside @spaces.GPU) and explains that this works via CUDA emulation outside the decorated function, with real CUDA used inside it.

Reviewed by Cursor Bugbot for commit d98ed6b. Bugbot is set up for automated code reviews on this repo. Configure here.

@cbensimon cbensimon requested review from apolinario and pcuenca April 9, 2026 15:01
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link
Copy Markdown
Member

@pcuenca pcuenca left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Big +1 to the clarifications, I've seen this happen in the wild a few times.

@cbensimon cbensimon merged commit b29a034 into main Apr 9, 2026
3 checks passed
@cbensimon cbensimon deleted the zerogpu-cuda-placement branch April 9, 2026 17:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants