Commit 924e16c
committed
Add a SymInt-free tensor specialization key for exact torch.Tensor args
_tensor_key wraps every size and stride element in _hashable_dim to
normalize torch.SymInt into a hashable (shape_env_id, expr) pair. That
wrap exists only for symbolic shapes, which appear on FakeTensors
during tracing -- yet concrete tensors paid for it on every kernel
call: two Python-level loops (sizes + strides) with an isinstance
check per dimension, plus rebuilding each as a fresh tuple. For the
default static_shapes=True path this was the bulk of per-tensor key
extraction (~0.6us each, x number of tensor args, every call).
Specialization-extractor dispatch is by exact type
(_specialization_extractors.get(type(obj))), so the torch.Tensor and
torch.nn.Parameter entries are only ever hit by objects whose
sizes/strides are guaranteed concrete ints. Route those to a new
_concrete_tensor_key that uses obj.size() (a torch.Size) and
obj.stride() directly as key components. Both are tuple subclasses:
they hash and compare identically to plain int tuples, so keys
produced by the fast path and the SymInt path for the same concrete
shape are interchangeable -- no cache invalidation, and the
specialization axes (dtype, shape, stride, _dynamo_static_indices,
int32/int64 index width, dynamic-shape buckets) are bit-identical.
Anything that can carry SymInts keeps the safe path:
* FakeTensor has its own dispatch entry -> _tensor_key (unchanged)
* torch.Tensor *subclasses* (e.g. the JAX-export adapter) miss the
exact-type dict and hit the isinstance fallback in
_specialization_key, which now routes to _tensor_key explicitly
(This is the same optimization as the Python-only part of upstream
PR #2611, independently arrived at from profiling.)
Verified: test_misc, test_specialize, test_torch_compile (306 passed)
-- the torch.compile suite exercises the FakeTensor/SymInt path.
Benchmark (B200, end-to-end wall time per call, add-style kernel,
N=4096 fp32, HELION_AUTOTUNE_EFFORT=none, steady state, 20k iters):
```
n_args | baseline | prev commit | this commit
-------+----------+-------------+------------
2 | 24.14 us | 16.39 us | 15.53 us
8 | 33.05 us | 24.53 us | 22.12 us
16 | 43.56 us | 34.82 us | 30.62 us
```
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
stack-info: PR: #2748, branch: yushangdi/stack/301 parent d739597 commit 924e16c
1 file changed
Lines changed: 40 additions & 4 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
352 | 352 | | |
353 | 353 | | |
354 | 354 | | |
355 | | - | |
356 | | - | |
| 355 | + | |
| 356 | + | |
| 357 | + | |
| 358 | + | |
357 | 359 | | |
358 | 360 | | |
359 | 361 | | |
| |||
1329 | 1331 | | |
1330 | 1332 | | |
1331 | 1333 | | |
| 1334 | + | |
| 1335 | + | |
| 1336 | + | |
| 1337 | + | |
| 1338 | + | |
| 1339 | + | |
| 1340 | + | |
| 1341 | + | |
| 1342 | + | |
| 1343 | + | |
| 1344 | + | |
| 1345 | + | |
| 1346 | + | |
| 1347 | + | |
| 1348 | + | |
| 1349 | + | |
| 1350 | + | |
| 1351 | + | |
| 1352 | + | |
| 1353 | + | |
| 1354 | + | |
| 1355 | + | |
| 1356 | + | |
| 1357 | + | |
| 1358 | + | |
| 1359 | + | |
| 1360 | + | |
| 1361 | + | |
| 1362 | + | |
| 1363 | + | |
1332 | 1364 | | |
1333 | 1365 | | |
1334 | 1366 | | |
| |||
1408 | 1440 | | |
1409 | 1441 | | |
1410 | 1442 | | |
1411 | | - | |
1412 | | - | |
| 1443 | + | |
| 1444 | + | |
| 1445 | + | |
| 1446 | + | |
| 1447 | + | |
| 1448 | + | |
1413 | 1449 | | |
1414 | 1450 | | |
1415 | 1451 | | |
| |||
0 commit comments