Skip to content

[KernelGen][Nvidia] Add transpose_ operator with Triton kernel - #5446

Open
ShawnsYing wants to merge 1 commit into
flagos-ai:masterfrom
ShawnsYing:pr/transpose_
Open

[KernelGen][Nvidia] Add transpose_ operator with Triton kernel#5446
ShawnsYing wants to merge 1 commit into
flagos-ai:masterfrom
ShawnsYing:pr/transpose_

Conversation

@ShawnsYing

Copy link
Copy Markdown
Contributor

Summary

Adds a Triton kernel for transpose_. In-place transpose: swaps two dimensions of the tensor, mutating it in place
and returning it. This is a zero-copy view operation that swaps the sizes and
strides of the two specified dimensions via as_strided, matching the semantics
of aten::transpose_.

Testing

  • Validated against reference on device via to_reference(inp, True)
  • Tested on: Nvidia, Tianshu, Muxi, Ascend, Hygon

Performance

Test command: pytest benchmark/test_transpose_.py --level core (NVIDIA H20)

transpose_ (in-place)

dtype Size Torch Latency (ms) Gems Latency (ms) Speedup
float16 [64, 64] 0.003424 0.003200 1.070
float16 [256, 512] 0.003424 0.003264 1.049
float16 [1024, 1024] 0.003328 0.003232 1.030
float16 [4096, 4096] 0.003232 0.003232 1.000
float16 [64, 512, 512] 0.003232 0.003296 0.981
float16 [128, 256, 64] 0.003328 0.003296 1.010
float16 [8, 16, 32, 64] 0.003232 0.003232 1.000
float32 [64, 64] 0.003232 0.003296 0.981
float32 [256, 512] 0.003264 0.003264 1.000
float32 [1024, 1024] 0.003264 0.003248 1.005
float32 [4096, 4096] 0.003264 0.003264 1.000
float32 [64, 512, 512] 0.003264 0.003232 1.010
float32 [128, 256, 64] 0.003232 0.003264 0.990
float32 [8, 16, 32, 64] 0.003264 0.003264 1.000
bfloat16 [64, 64] 0.003264 0.003264 1.000
bfloat16 [256, 512] 0.003264 0.003264 1.000
bfloat16 [1024, 1024] 0.003264 0.003232 1.010
bfloat16 [4096, 4096] 0.003264 0.003264 1.000
bfloat16 [64, 512, 512] 0.003264 0.003296 0.990
bfloat16 [128, 256, 64] 0.003264 0.003264 1.000
bfloat16 [8, 16, 32, 64] 0.003264 0.003264 1.000
Operator Arithmetic Mean Speedup
transpose_ (in-place) 1.006

Multi-backend Testing

Backend Accuracy Test Speedup (mean) Notes
Nvidia (H20) PASS (21 cases) 1.006 Primary
Tianshu N/A
Muxi N/A
Ascend N/A
Hygon N/A

Files Changed

  • src/flag_gems/ops/transpose_.py: Triton kernel implementation
  • tests/test_transpose_.py: Accuracy test
  • benchmark/test_transpose_.py: Performance benchmark
  • src/flag_gems/ops/__init__.py: Register import and __all__
  • src/flag_gems/__init__.py: Register to _FULL_CONFIG
  • conf/operators.yaml: Add operator entry (kind: Tensor, stage: alpha 5.4)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant