You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
On torch 2.14.0 (and later), import thunder raises at module import:
File ".../thunder/__init__.py", line 30, in <module>
from thunder.core.jit_ext import thunder_general_jit
File ".../thunder/core/jit_ext.py", line 797, in <module>
@register_general_jit_lookaside(torch.autograd.function.Function.apply.__func__)
AttributeError: 'builtin_function_or_method' object has no attribute '__func__'
Why
pytorch/pytorch#190694 (4fee7d8a70cfdd5fe88cf949cd9e95f67af9c369, merged 2026-07-23) moved the autograd.Function.apply wrapper into C++. torch/autograd/function.py now reads
apply=_C._FunctionBase.__dict__["apply"]
so Function.apply is a builtin_function_or_method with no __func__. This is in the v2.14.0
tag (line 650); v2.13.0 still has def apply(cls, *args, **kwargs) (line 610).
(On 2.14.0 you will hit the align_as failure below first unless the named-tensor references are
guarded; with them guarded, the traceback above is what you get.)
Why CI has not caught it (as of thunder main @ 422f45b)
The latest lanes (11 jobs) never install: requirements/devel.txt asks for torchaudio==2.14.0, which does not exist, so pip fails before thunder is imported.
The nightly lanes (7 jobs) fail earlier, in thunder/torch/default_torch_ops.py:361, with AttributeError: type object 'Tensor' has no attribute 'align_as' — Remove named tensor pytorch/pytorch#173895
(00512145ec4) removed named tensors, and the module-level table evaluates torch.Tensor.{align_as,align_to,has_names,refine_names,rename} unguarded. Worth flagging on its
own: that removal is in released torch 2.13.0, not just nightlies
(test/test_namedtensor.py is present at v2.12.1 and gone at v2.13.0), so plain pip install torch==2.13.0 lightning-thunder; python -c "import thunder" already fails today.
The oldest lane (3 jobs, torch==2.7.1) passes — the only green pytester jobs in that run.
We hit the apply failure only because we already carry a local hasattr guard for those five
named-tensor attributes; that guard is what gets the import past default_torch_ops.py to jit_ext.py:797. Happy to send it as a separate PR if useful.
A re-key alone is not a fix
We measured two obvious one-line repairs — keying the lookaside on torch.autograd.function.Function.apply, and on torch._C._FunctionBase.__dict__["apply"]. Both
make the import succeed, and with both the lookaside never fires for a user subclass:
MyFunc.apply in _general_jit_lookaside_map: False
MyFunc.apply.__self__: <class '__main__.MyFunc'>
Function.apply.__self__: <class 'torch.autograd.function.Function'>
thunder.jit on a trivial torch.autograd.Function then dies in the opaque-function fallback:
thunder.core.interpreter.InterpreterError: Encountered exception KeyError: None while tracing ...
thunder/core/jit_ext.py:1365 in general_jit_lookaside -> register_operator_for_opaque_function(fn)
thunder/extend/__init__.py:416 in get_name -> mod = sys.modules[fn.__module__]
KeyError: None
(A plain function under thunder.jit compiles and runs fine in the same process, so this is
specific to autograd.Function.)
Under the new torch, every subclass's .apply is a distinct bound builtin, so the interpreter
needs to recognise a builtin_function_or_method whose __self__ is an autograd.Function
subclass and route it to _general_jit_torch_autograd_function_apply_lookaside, rather than
matching a single shared function object. Separately, get_name could give a clearer error when fn.__module__ is None.
Environment
lightning-thunder 0.2.7.dev0 (main @ 422f45b), CPython 3.14.7,
torch 2.14.0a0, aarch64. The jit_ext.py:797 line is also present at the latest release tag
(0.2.6), so this is not trunk-only.
What happens
On torch 2.14.0 (and later),
import thunderraises at module import:Why
pytorch/pytorch#190694 (
4fee7d8a70cfdd5fe88cf949cd9e95f67af9c369, merged 2026-07-23) moved theautograd.Function.applywrapper into C++.torch/autograd/function.pynow readsso
Function.applyis abuiltin_function_or_methodwith no__func__. This is in the v2.14.0tag (line 650); v2.13.0 still has
def apply(cls, *args, **kwargs)(line 610).Reproducer
(On 2.14.0 you will hit the
align_asfailure below first unless the named-tensor references areguarded; with them guarded, the traceback above is what you get.)
Why CI has not caught it (as of thunder main @ 422f45b)
latestlanes (11 jobs) never install:requirements/devel.txtasks fortorchaudio==2.14.0, which does not exist, so pip fails before thunder is imported.nightlylanes (7 jobs) fail earlier, inthunder/torch/default_torch_ops.py:361, withAttributeError: type object 'Tensor' has no attribute 'align_as'— Remove named tensor pytorch/pytorch#173895(
00512145ec4) removed named tensors, and the module-level table evaluatestorch.Tensor.{align_as,align_to,has_names,refine_names,rename}unguarded. Worth flagging on itsown: that removal is in released torch 2.13.0, not just nightlies
(
test/test_namedtensor.pyis present at v2.12.1 and gone at v2.13.0), so plainpip install torch==2.13.0 lightning-thunder; python -c "import thunder"already fails today.oldestlane (3 jobs, torch==2.7.1) passes — the only green pytester jobs in that run.We hit the
applyfailure only because we already carry a localhasattrguard for those fivenamed-tensor attributes; that guard is what gets the import past
default_torch_ops.pytojit_ext.py:797. Happy to send it as a separate PR if useful.A re-key alone is not a fix
We measured two obvious one-line repairs — keying the lookaside on
torch.autograd.function.Function.apply, and ontorch._C._FunctionBase.__dict__["apply"]. Bothmake the import succeed, and with both the lookaside never fires for a user subclass:
thunder.jiton a trivialtorch.autograd.Functionthen dies in the opaque-function fallback:(A plain function under
thunder.jitcompiles and runs fine in the same process, so this isspecific to
autograd.Function.)Under the new torch, every subclass's
.applyis a distinct bound builtin, so the interpreterneeds to recognise a
builtin_function_or_methodwhose__self__is anautograd.Functionsubclass and route it to
_general_jit_torch_autograd_function_apply_lookaside, rather thanmatching a single shared function object. Separately,
get_namecould give a clearer error whenfn.__module__isNone.Environment
lightning-thunder 0.2.7.dev0 (main @ 422f45b), CPython 3.14.7,
torch 2.14.0a0, aarch64. The
jit_ext.py:797line is also present at the latest release tag(
0.2.6), so this is not trunk-only.