Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ KernelGenBench is a component of [FlagOS](https://flagos.io/) — a unified, ope

KernelGenBench is a benchmark framework for evaluating LLM and agent-based Triton kernel generation across multiple hardware platforms.

![KernelGenBench Overview](docs/_static/images/figures/overview.png)
![KernelGenBench Overview](docs/source/_static/images/figures/overview.png)

## Features

Expand Down
4 changes: 2 additions & 2 deletions README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

跨多种硬件平台评测 LLM 和 Agent 生成 Triton kernel 能力的基准框架。

![KernelGenBench 架构概览](docs/_static/images/figures/overview.png)
![KernelGenBench 架构概览](docs/source/_static/images/figures/overview.png)

## 特性

Expand Down Expand Up @@ -117,7 +117,7 @@ export OPENAI_BASE_URL=http://your-endpoint/v1 # 可选,自定义端点
| Claude Code | 92 / 0.86 | 89 / 0.18 | 93 / 0.80 | 88 / 0.87 | 96 / 0.89 | 83 / 0.83 |
| AKO4all | 89 / 1.00 | 84 / 0.30 | 88 / 1.09 | 88 / 1.08 | 86 / 1.12 | 80 / 1.07 |

![跨平台精度、加速比与生态开销](docs/_static/images/figures/figure_crossplatform.png)
![跨平台精度、加速比与生态开销](docs/source/_static/images/figures/figure_crossplatform.png)

*在异构芯片上开展算子生成任务会带来显著的经济成本和时间成本增加——非 NVIDIA 平台最高需要 2 倍的 token 消耗和运行时间。*

Expand Down
2 changes: 1 addition & 1 deletion docs/requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ sphinx-multiproject==1.0.0
sphinx-rtd-theme
sphinx-book-theme~=1.1
sphinx-autodoc2~=0.5.0
sphinx-design
sphinx-design>=0.8.0
sphinx-copybutton
sphinxext.rediraffe~=0.3.0
sphinxcontrib.mermaid~=0.7.1
Expand Down
File renamed without changes.
20 changes: 10 additions & 10 deletions docs/source/conf.py
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@
"sphinx_togglebutton",
]

templates_path = ["../_templates"]
templates_path = ["_templates"]
exclude_patterns = ["_build", "_includes"]
master_doc = "index"
default_role = "obj"
Expand Down Expand Up @@ -73,41 +73,41 @@
# -- HTML output -------------------------------------------------------------

html_theme = "sphinx_book_theme"
html_static_path = ["../_static"]
html_static_path = ["_static"]
html_css_files = ["css/custom.css"]
html_js_files = []
# html_js_files = []

# Logo and favicon configuration based on build language
# Read the Docs sets READTHEDOCS_LANGUAGE environment variable during builds
if CURRENT_LANGUAGE == "zh_CN":
html_logo = "../_static/images/logos/zh-logo.svg"
html_logo = "_static/images/logos/zh-logo.svg"
html_theme_options = {
"home_page_in_toc": True,
"use_download_button": False,
"repository_url": "https://github.com/flagos-ai/KernelGenBench",
"use_edit_page_button": True,
"use_repository_button": True,
"logo": {
"image_light": "../_static/images/logos/zh-logo.svg",
"image_dark": "../_static/images/logos/zh-logo-dark.svg",
"image_light": "_static/images/logos/zh-logo.svg",
"image_dark": "_static/images/logos/zh-logo-dark.svg",
},
}
else:
# Default to English configuration for all other languages
html_logo = "../_static/images/logos/en-logo.svg"
html_logo = "_static/images/logos/en-logo.svg"
html_theme_options = {
"home_page_in_toc": True,
"use_download_button": False,
"repository_url": "https://github.com/flagos-ai/KernelGenBench",
"use_edit_page_button": True,
"use_repository_button": True,
"logo": {
"image_light": "../_static/images/logos/en-logo.svg",
"image_dark": "../_static/images/logos/en-logo-dark.svg",
"image_light": "_static/images/logos/en-logo.svg",
"image_dark": "_static/images/logos/en-logo-dark.svg",
},
}

html_favicon = "../_static/images/logos/favicon.svg"
html_favicon = "_static/images/logos/favicon.svg"

htmlhelp_basename = "KernelGenBenchdoc"

Expand Down
24 changes: 0 additions & 24 deletions docs/source/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,10 +83,6 @@ Contributing guides, custom operators, and extending the framework.
:hidden:

overview/index.md
overview/what-is.md
overview/why-kernelgenbench.md
overview/capabilities.md
overview/results.md
```

```{toctree}
Expand All @@ -103,10 +99,6 @@ getting-started/index.md
:hidden:

features/index.md
features/multi-source.md
features/multi-chip.md
features/anti-hack.md
features/metrics.md
```

```{toctree}
Expand All @@ -115,15 +107,6 @@ features/metrics.md
:hidden:

operation-guide/index.md
operation-guide/llm-track/index.md
operation-guide/llm-track/commands.md
operation-guide/llm-track/parameters.md
operation-guide/llm-track/examples.md
operation-guide/agent-track/index.md
operation-guide/agent-track/setup.md
operation-guide/agent-track/methods.md
operation-guide/agent-track/commands.md
operation-guide/agent-track/cost-analysis.md
```

```{toctree}
Expand All @@ -132,9 +115,6 @@ operation-guide/agent-track/cost-analysis.md
:hidden:

reference/index.md
reference/datasets.md
reference/operators.md
reference/hardware.md
```

```{toctree}
Expand All @@ -143,9 +123,6 @@ reference/hardware.md
:hidden:

development/index.md
development/contributing.md
development/custom-operators.md
development/extending.md
```

```{toctree}
Expand All @@ -156,4 +133,3 @@ development/extending.md
faq/index.md
glossary/index.md
```

19 changes: 8 additions & 11 deletions docs/source/locale/zh_CN/LC_MESSAGES/overview/results.po
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: KernelGenBench Documentation \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-06-15 10:42+0800\n"
"POT-Creation-Date: 2026-06-17 13:36+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language: zh_CN\n"
Expand Down Expand Up @@ -38,11 +38,8 @@ msgid ""
msgstr "在 NVIDIA A100 上对来自三个来源(ATen、vLLM、cuBLAS)的 210 个算子进行评估。"

#: ../../source/overview/results.md:9
msgid ""
"![Multi-Source "
"Results](../../_static/images/figures/table_multi_source.png)"
msgstr ""
"![多源结果](../../_static/images/figures/table_multi_source.png)"
msgid "![Multi-Source Results](../_static/images/figures/table_multi_source.png)"
msgstr "![多源结果](../_static/images/figures/table_multi_source.png)"

#: ../../source/overview/results.md:11 ../../source/overview/results.md:35
msgid "Key Findings"
Expand Down Expand Up @@ -141,15 +138,14 @@ msgid "Cross-platform evaluation of 110 ATen operators on 6 hardware platforms."
msgstr "在 6 个硬件平台上对 110 个 ATen 算子进行跨平台评估。"

#: ../../source/overview/results.md:31
msgid "![Multi-Chip Results](../../_static/images/figures/table_multi_chip.png)"
msgstr "![多芯片结果](../../_static/images/figures/table_multi_chip.png)"
msgid "![Multi-Chip Results](../_static/images/figures/table_multi_chip.png)"
msgstr "![多芯片结果](../_static/images/figures/table_multi_chip.png)"

#: ../../source/overview/results.md:33
msgid ""
"![Cross-platform "
"Performance](../../_static/images/figures/figure_crossplatform.png)"
msgstr ""
"![跨平台性能](../../_static/images/figures/figure_crossplatform.png)"
"Performance](../_static/images/figures/figure_crossplatform.png)"
msgstr "![跨平台性能](../_static/images/figures/figure_crossplatform.png)"

#: ../../source/overview/results.md:33
msgid "Cross-platform Performance"
Expand Down Expand Up @@ -288,3 +284,4 @@ msgid ""
"Large-scale agent evaluations may consume billions of tokens. Plan your "
"budget accordingly."
msgstr "大规模智能体评估可能消耗数十亿 Token。请相应规划您的预算。"

20 changes: 11 additions & 9 deletions docs/source/locale/zh_CN/LC_MESSAGES/overview/what-is.po
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: KernelGenBench Documentation \n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-06-15 10:42+0800\n"
"POT-Creation-Date: 2026-06-17 13:36+0800\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language: zh_CN\n"
Expand All @@ -32,12 +32,13 @@ msgid ""
"[FlagOS](https://flagos.io/) — a unified, open-source AI system software "
"stack."
msgstr ""
"{term}`KernelGenBench` 是一个用于评估 {term}`LLM` 和智能体驱动的 {term}`Triton` 内核生成的基准框架,"
"支持多种硬件平台。它是 [FlagOS](https://flagos.io/) 的组成部分——一个统一的开源 AI 系统软件栈。"
"{term}`KernelGenBench` 是一个用于评估 {term}`LLM` 和智能体驱动的 {term}`Triton` "
"内核生成的基准框架,支持多种硬件平台。它是 [FlagOS](https://flagos.io/) 的组成部分——一个统一的开源 AI "
"系统软件栈。"

#: ../../source/overview/what-is.md:5
msgid "![KernelGenBench Overview](../../_static/images/figures/overview.png)"
msgstr "![KernelGenBench 概览](../../_static/images/figures/overview.png)"
msgid "![KernelGenBench Overview](../_static/images/figures/overview.png)"
msgstr "![KernelGenBench 概览](../_static/images/figures/overview.png)"

#: ../../source/overview/what-is.md:5
msgid "KernelGenBench Overview"
Expand All @@ -55,8 +56,8 @@ msgid ""
"{term}`Triton` kernels serve as drop-in replacements for production use, "
"enabling direct evaluation of real-world applicability."
msgstr ""
"{term}`KernelGenBench` 提供了一种标准化的方法来衡量 AI 模型生成 GPU 内核代码的有效性。"
"生成的 {term}`Triton` 内核可直接用于生产环境,从而实现对实际应用性的直接评估。"
"{term}`KernelGenBench` 提供了一种标准化的方法来衡量 AI 模型生成 GPU 内核代码的有效性。生成的 "
"{term}`Triton` 内核可直接用于生产环境,从而实现对实际应用性的直接评估。"

#: ../../source/overview/what-is.md:11
msgid "Problem Addressed"
Expand All @@ -70,8 +71,8 @@ msgid ""
"comprehensive way to evaluate their effectiveness across diverse "
"{term}`Operator` sources and heterogeneous hardware platforms."
msgstr ""
"该基准填补了 AI 生态系统中的一个关键空白:虽然 {term}`LLM` 在自动化内核开发方面展现出潜力,"
"但之前缺乏一种全面的方法来评估其在多样化 {term}`算子` 来源和异构硬件平台上的有效性。"
"该基准填补了 AI 生态系统中的一个关键空白:虽然 {term}`LLM` "
"在自动化内核开发方面展现出潜力,但之前缺乏一种全面的方法来评估其在多样化 {term}`算子` 来源和异构硬件平台上的有效性。"

#: ../../source/overview/what-is.md:15
msgid "Components"
Expand Down Expand Up @@ -136,3 +137,4 @@ msgstr "[KernelGen](https://github.com/flagos-ai/KernelGen)"
#, python-brace-format
msgid "High-performance platform for automated {term}`Triton` kernel generation"
msgstr "用于自动化 {term}`Triton` 内核生成的高性能平台"

6 changes: 3 additions & 3 deletions docs/source/overview/results.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Key findings from KernelGenBench evaluation experiments.

Evaluation of 210 operators from three sources (ATen, vLLM, cuBLAS) on NVIDIA A100.

![Multi-Source Results](../../_static/images/figures/table_multi_source.png)
![Multi-Source Results](../_static/images/figures/table_multi_source.png)

### Key Findings

Expand All @@ -28,9 +28,9 @@ Evaluation of 210 operators from three sources (ATen, vLLM, cuBLAS) on NVIDIA A1

Cross-platform evaluation of 110 ATen operators on 6 hardware platforms.

![Multi-Chip Results](../../_static/images/figures/table_multi_chip.png)
![Multi-Chip Results](../_static/images/figures/table_multi_chip.png)

![Cross-platform Performance](../../_static/images/figures/figure_crossplatform.png)
![Cross-platform Performance](../_static/images/figures/figure_crossplatform.png)

### Key Findings

Expand Down
2 changes: 1 addition & 1 deletion docs/source/overview/what-is.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

{term}`KernelGenBench` is a benchmark framework for evaluating {term}`LLM` and agent-based {term}`Triton` kernel generation across multiple hardware platforms. It is a component of [FlagOS](https://flagos.io/) — a unified, open-source AI system software stack.

![KernelGenBench Overview](../../_static/images/figures/overview.png)
![KernelGenBench Overview](../_static/images/figures/overview.png)

## Purpose

Expand Down
Loading