{term}KernelGenBench is a benchmark framework for evaluating {term}LLM and agent-based {term}Triton kernel generation across multiple hardware platforms. It is a component of FlagOS — a unified, open-source AI system software stack.
{term}KernelGenBench provides a standardized way to measure how effectively AI models can generate GPU kernel code. The generated {term}Triton kernels serve as drop-in replacements for production use, enabling direct evaluation of real-world applicability.
The benchmark addresses a critical gap in the AI ecosystem: while {term}LLM show promise in automating kernel development, there was no comprehensive way to evaluate their effectiveness across diverse {term}Operator sources and heterogeneous hardware platforms.
{term}KernelGenBench consists of two complementary sub-benchmarks:
| Sub-benchmark | Description |
|---|---|
{term}KernelGenBench-MS |
Multi-Source evaluation with 210 operators |
{term}KernelGenBench-MC |
Multi-Chip evaluation across 6 hardware platforms |
| Project | Description |
|---|---|
| awesome-LLM-driven-kernel-generation | Survey of AI-driven kernel generation |
| KernelGen | High-performance platform for automated {term}Triton kernel generation |