Skip to content

[KernelGen][MThreads] Add permute_copy Moore Threads specialized operator - #117

Merged
Dongxu-H merged 3 commits into
flagos-ai:masterfrom
Yukun-Cui:pr/mthreads-permute_copy
Jul 27, 2026
Merged

[KernelGen][MThreads] Add permute_copy Moore Threads specialized operator#117
Dongxu-H merged 3 commits into
flagos-ai:masterfrom
Yukun-Cui:pr/mthreads-permute_copy

Conversation

@Yukun-Cui

@Yukun-Cui Yukun-Cui commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

[KernelGen][MThreads] Add permute_copy Moore Threads specialized operator

Summary

Add a Moore Threads (MUSA) specialized Triton kernel for permute_copy, overriding the generic
implementation via runtime.replace_customized_ops(). Implements the permute copy operation.

Testing

  • Reused the existing upstream accuracy tests tests/test_permute_copy.py (-m permute_copy)
  • Validated against reference on the MUSA device; specialization confirmed active via the
    GEMS_MTHREADS PERMUTE_COPY debug log
  • Falls back to the generic implementation for unsupported dtype/device/shape
    (fp64/int64 are not supported on Moore Threads hardware)

Performance

Compared against the generic FlagGems implementation on Moore Threads (MUSA).

permute_copy

dtype Size Torch Latency (ms) Gems Latency (ms) Speedup
float16 64, 128 0.005640 0.002760 2.043x
float16 128, 256 0.002760 0.002720 1.015x
float16 32, 64, 128 0.004240 0.005240 0.809x
float32 64, 128 0.006000 0.002760 2.174x
float32 128, 256 0.002640 0.002760 0.957x
float32 32, 64, 128 0.005240 0.005320 0.985x
bfloat16 64, 128 0.005640 0.002760 2.043x
bfloat16 128, 256 0.002760 0.002760 1.000x
bfloat16 32, 64, 128 0.004240 0.005280 0.803x
Operator Arithmetic Mean Speedup
permute_copy 1.31x

Files Changed

  • src/flag_gems/runtime/backend/_mthreads/ops/permute_copy.py: Moore Threads Triton kernel + fallback
  • src/flag_gems/runtime/backend/_mthreads/ops/__init__.py: Register import and __all__

@Dongxu-H

Copy link
Copy Markdown
Collaborator

/test | permute_copy:mthreads

@github-actions

github-actions Bot commented Jul 25, 2026

Copy link
Copy Markdown

❌ On-demand test failed

Operator: permute_copy
Runner: mthreads
Backend: mthreads-musa520

The test failed to complete. Check the workflow run for details.

@Dongxu-H

Copy link
Copy Markdown
Collaborator

/test | permute_copy_moore_threads_specialized:mthreads

@github-actions

Copy link
Copy Markdown

❌ On-demand test failed

Operator: permute_copy_moore_threads_specialized
Runner: mthreads
Backend: mthreads-musa520

The test failed to complete. Check the workflow run for details.

@Dongxu-H Dongxu-H left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logger 命名不符合 MThreads 后端规范。当前使用的是主文件夹写法 logging.getLogger(name),应该改为 MThreads 后端专用写法:logger = logging.getLogger(fflag_gems.runtime.backend._mthreads.ops.{name[-1]})。

@Dongxu-H

Copy link
Copy Markdown
Collaborator

/test | permute_copy:mthreads

@Dongxu-H Dongxu-H left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Dongxu-H
Dongxu-H merged commit 3ad235e into flagos-ai:master Jul 27, 2026
11 checks passed
@Yukun-Cui

Copy link
Copy Markdown
Contributor Author

2026.07.27已被移出待特化列表

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants