[KernelGen][Iluvatar] Add log_normal_ vendor specialization with fused kernel

[yzw1128]

Summary

Optimize log_normal_ operator for Iluvatar backend by fusing Philox random number generation, Box-Muller normal sampling, and log-normal transformation into a single Triton kernel. This eliminates the temporary float32 buffer and halves the kernel launch count compared to the generic two-kernel path.

Key optimizations

• Single kernel launch instead of two (reduced launch overhead)
• No temporary float32 buffer (reduced memory bandwidth)
• exp2/log2-based evaluation with pre-multiplied constants and fused-multiply-add for faster transcendental evaluation than natural-base tl.exp/tl.log
• Iluvatar-tuned autotune configs with num_stages=1 and BLOCK_SIZE ranging from 512 to 32768

Correctness Verification

Tested with shape (10000, 1024) and default parameters (mean=1.0, std=2.0). Expected mean of log-normal distribution: exp(mean + std²/2) = 20.0855.

dtype	all_positive	sample_mean	error
torch.float32	True	20.17	0.43%
torch.bfloat16	True	19.99	0.46%
torch.float16	True	20.00	0.42%

All float16 log-normal values are positive, though some may overflow to inf for large std values due to float16 range limitation (~65504). This is an inherent numerical property of the log-normal distribution, not specific to this implementation.

Performance

Benchmarked on Iluvatar BI-V150 with 36 configurations (3 dtypes × 12 shapes, element counts from 64 to 1B+).

Verification parameters:

• CUDA_VISIBLE_DEVICES=6 python3 -m pytest benchmark/test_log_normal_.py -v -s
• 12 shapes across 1D/2D/3D: (1073741824,), (64,64), (4096,4096), (64,512,512), (1024,1024,1024), (268435456,), (10000,1), (10000,256), (10000,65536), (100,1,100), (100,256,100), (100,65536,100)
• 3 dtypes: torch.float16, torch.float32, torch.bfloat16
• Default parameters: mean=1.0, std=2.0

dtype	Geometric mean speedup
torch.float16	1.18
torch.float32	1.16
torch.bfloat16	1.12
Overall	1.15

All 36 test cases show speedup ≥ 1.0. The fused kernel shows consistent gains across all shapes and dtypes, with the greatest benefits on large tensors where the eliminated temporary buffer had the most impact.

Files Changed

File	Change
src/flag_gems/runtime/backend/_iluvatar/ops/log_normal_.py	NEW — fused Iluvatar kernel with autotune configs
src/flag_gems/runtime/backend/_iluvatar/ops/__init__.py	MODIFIED — register log_normal_ in Iluvatar backend ops (+2 lines)

[Dongxu-H]

/test | log_normal_:iluvatar

[Dongxu-H]

LGTM

[github-actions]

❌ On-demand test failed

Operator: log_normal_
Runner: iluvatar
Backend: iluvatar

The test failed to complete. Check the workflow run for details.

[Dongxu-H]

LGTM

[Dongxu-H]

/test | log_normal__vendor_specialization_with_fused_kernel:iluvatar

[github-actions]

❌ On-demand test failed

Operator: log_normal__vendor_specialization_with_fused_kernel
Runner: iluvatar
Backend: iluvatar

The test failed to complete. Check the workflow run for details.

[Dongxu-H]

LGTM