Skip to content

Add full support for set_grad_enabled - #2474

Closed
shino16 wants to merge 5 commits into
Lightning-AI:mainfrom
shino16:fix-2398
Closed

shino16 wants to merge 5 commits into
Lightning-AI:mainfrom
shino16:fix-2398

Conversation

@shino16

@shino16 shino16 commented Aug 27, 2025

Copy link
Copy Markdown
Collaborator

Fixes #2398. Thunder has support for torch.set_grad_enabled calls that come from torch.no_grad() context by managing cd.is_grad_enabled without actually calling torch.set_grad_enabled. As in #2398, when Thunder's graph splitter splits the region, torch.set_grad_enabled(True/False) appears in each segment independently without its counterpart. We should actually call it, and it should be done before connecting the computation to Autograd so that it will populate the grad_fn attribute appropriately.

@shino16

shino16 commented Aug 28, 2025

Copy link
Copy Markdown
Collaborator Author

Currently, the following does not work:

import torch, thunder

torch.set_grad_enabled(False)
def fn(x):
    with torch.enable_grad():
        return x.sin()

x = torch.randn((), requires_grad=True, device="cuda")
assert fn(x).grad_fn is not None
assert thunder.jit(fn)(x).grad_fn is not None  # AssertionError
assert thunder.dynamo.thunderfx(fn)(x).grad_fn is not None  # AssertionError

This is because:

  1. the epilogue calls unpack_trivial(t0), where t0 represents x.sin()
  2. the output of unpack_trivial is also t0
  3. autograd is disabled when tracing the epilogue
  4. so the output of unpack_trivial is tagged DETACHED_AUTOGRAD_GRAPH meaning it will be treated as constant

This maybe a similar issue to #2396 (comment).

@shino16 shino16 changed the title Treat set_grad_enabled faithfully before connecting to autograd Add full support for set_grad_enabled Aug 28, 2025
@shino16

shino16 commented Aug 28, 2025

Copy link
Copy Markdown
Collaborator Author

Let me close this PR as the purpose has somewhat changed. I'll reopen as a new PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[thunderfx] Bug when no_grad region is split between inductor and thunder

1 participant