[FlagGems Operator Development Competition] Add leaky_relu, leaky_relu_, and leaky_relu.out

[Karl-XZ]

PR Category

• Operator Development Competition

Type of Change

• New operator implementation
• Tests
• Benchmark

Description

• Add aten::leaky_relu, aten::leaky_relu_, and aten::leaky_relu.out
• Add dtype-aware Triton fast path for contiguous CUDA tensors
• Redispatch unsupported or mismatched out= cases to preserve PyTorch behavior
• Add correctness tests and benchmark entries

Issue

• N/A

Progress

• Change is properly reviewed (1 reviewer required, 2 recommended).
• Change is responded to an issue.
• Change is fully covered by a UT.

Performance

Local device-side benchmark on RTX 3070:

• float16:
  • (4096, 4096): 1.13x
  • (64, 512, 512): 1.00x
  • (64, 64, 4096): 1.03x
• float32:
  • (64, 512, 512): 1.01x
  • (1024, 4096): 1.12x
• Validation:
  • python -m pre_commit run --files src/flag_gems/ops/leaky_relu.py src/flag_gems/__init__.py src/flag_gems/ops/__init__.py benchmark/test_unary_pointwise_perf.py tests/test_unary_pointwise_ops.py
  • python -m pytest tests/test_unary_pointwise_ops.py -m "leaky_relu or leaky_relu_ or leaky_relu_out" -q -x
  • python -m pytest benchmark/test_unary_pointwise_perf.py -m "leaky_relu" -q -x
  • python -m pytest benchmark/test_unary_pointwise_perf.py -k "leaky_relu_" -q -x

[CLAassistant]

All committers have signed the CLA.

[douxetpur]

Hi, thank you for your participation in the FlagOS Open Computing Competition 🙏

After review, we have chosen to move forward with a different implementation for this operator. As a result, this PR will be closed.

We appreciate your time and contribution, and hope to see more of your submissions in the future.