[PIPELINE] Support non-power-of-two TLE pipe capacities - #990
Draft
Prophet-Hongyi wants to merge 1 commit into
Draft
[PIPELINE] Support non-power-of-two TLE pipe capacities#990Prophet-Hongyi wants to merge 1 commit into
Prophet-Hongyi wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Allow TLE memory-backed descriptors and pipe state to use arbitrary positive
static capacities, including 3, 5, and 6.
The failure had two independent compiler layers:
buffered_tensor_typeandbarrier_typeinheritedtl.block_type, somemory descriptors incorrectly received Triton's register-tensor
power-of-two shape validation.
TleLowerPipeToNvwsinitialized all close tags through a distributedtensor<capacity x 1>. Even after fixing the Python type hierarchy, thatregister tensor reintroduced the same restriction during native lowering.
This change introduces a memory-descriptor base type with positive static
shape validation and initializes close tags one scalar memdesc slot at a time.
On H20, a real FlagGems FP8 einsum kernel forced to a six-stage typed pipe now
compiles and executes through the updated native library.
Review map
memory_descriptor_typeingpu/types.py: memory-backed shape,element type, size, and scalar properties are separated from register
tl.block_typevalidation.buffered_tensor_typeandbarrier_type: both adopt the new base;buffered-tensor equality also includes element type and storage, and
with_element_typreserves descriptor metadata.createPipeStateinTleLowerPipeToNvws.cpp: close tags remain inshared memory, but initialization no longer materializes a
tensor<capacity x 1>register value.Supported contract
tl.block_typecontinues to reject invalid register-block shapesfalsevalue, initialized per scalar slottl.block_typebehavior remain unchangedRoot cause and solution
Before this change, a six-stage memory buffer was rejected because its type was
modeled as a register block:
Changing only that Python inheritance was insufficient. Native pipe lowering
still built a distributed
tensor<6x1xi32>to initialize close tags, which wasagain subject to the register-block restriction.
The new representation keeps the distinction explicit:
The register-tensor validator is preserved for actual register blocks.
Validation
Current Draft source identity:
d817794488017d8bcbfc408d8b52128ab44f6537based ondf824d72097a31b4dca479ddfb7f480a470b7d66.The H20 and native gates below ran on pre-rebase commit
0dfeca539a7bf57551e25d6380f51bdf8060e6ce. Before publication, the singlefeature commit was replayed onto the current base and its stable patch ID
remained unchanged; the two intervening upstream commits do not touch this
PR's files. The device results are therefore patch-identical pre-rebase
evidence, not an exact-current-commit H20 rerun.
46 passed[6, BLOCK]SMEM slot lower and executetest_tle_lower_pipe_to_nvws.mlirtriton-optmemdesc<6x1>, emits six scalar stores, and does not createtensor<6x1>fp8_einsumstage-6 compile/executepro_b1,w4/s6/tile0traverses both the Python and native fixesThe native library used by the stage-6 gate had SHA-256
8ca85dfc84da45c2198c7a8743fb2864202fb7bc97eab2c72eab298717b1deda.The real-kernel gate selected six stages and produced finite output. It is an
expressibility/execution gate; it is not a stage-6 performance claim and does
not by itself establish full numerical equivalence for every workload.
Focused regression coverage — invariants exercised by the added tests
[6, 1, 64, 128]buffered descriptor construction succeeds while theequivalent
tl.block_typeconstruction remains rejected.(6, 1)and is not a block.[6, 64]SMEM, stores, reloads, and returnsexact integer values.
ttg.local_storeoperations andnvws.create_tokenwithnumBuffers = 6.Performance evidence
Baseline: current upstream rejects the tested capacity-6 descriptor or
pipe before the real kernel can run.
Evidence boundary: this is a compiler correctness and expressibility PR.
No latency in ms or throughput improvement is claimed; the performance-relevant result is that a
previously unavailable six-stage schedule becomes compilable for later,
independent tuning.
Files changed
python/triton/experimental/tle/language/gpu/types.py— memory-descriptor type hierarchy and equality.
python/test/tle/unit/test_tle.py— shape, validation, equality, and barrier unit tests.
python/test/tle/unit/test_tle_gpu_slot.py— H20 non-power-of-two SMEM slot execution test.
third_party/tle/dialect/lib/Transforms/TleLowerPipeToNvws.cpp— scalar close-tag slot initialization.
third_party/tle/test/GPU/test_tle_lower_pipe_to_nvws.mlir— native capacity-6 lowering regression.
Known limits
capacity 6 is optimal for any kernel or add it to an autotuning space.
focused on the affected type, slot, pipe-lowering, and real-kernel paths.
changes with their own correctness and performance evidence.