[KMCompiler][ttx] Optimize gelu with rowwise nomask kernels#364
[KMCompiler][ttx] Optimize gelu with rowwise nomask kernels#364YangLong114514 wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Code Review
This pull request optimizes the GELU forward pass on NPU by introducing specialized no-mask kernels (_gelu_fwd_nomask_kernel and _gelu_fwd_nomask_single_kernel) when dimensions align, and refactors grid and block size calculations. The feedback suggests reusing the cached get_num_cores utility from .utils instead of dynamically querying device properties on every grid calculation, and dynamically computing GELU_TANH_MAX_BLOCK_SIZE_M from the autotune configurations to prevent potential out-of-bounds bugs if the configurations are updated in the future.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
Description
The gelu operator has been optimized for the Ascend platform.
Changes
Updated the
BLOCK_SIZE_Ncalculation to use a capped power-of-two tile size. For GELU, the cap is set to1024to reduce UB pressure from the tanh approximation path.Added dynamic grid calculation based on the autotuned
BLOCK_SIZE_M, reducing empty programs for small shapes.Added no-mask forward kernels for fully divisible row/column tiles, removing mask construction and masked
tl.load/tl.storeoverhead.Added a no-mask single-tile forward kernel for shapes where
n_cols == BLOCK_SIZE_N, eliminating the inner column loop for common benchmark shapes.Performance
Using Ascend 910C and Triton 3.2.x of FlagTree:
Accuracy test
The correctness verification of the GeLU operator for all three data types, float32, float16, and bf16, passed in the Mojo correctness verification directory.
