Skip to content

[hot-fix] fix GroupedTopKRouterFL routing and MLA decode method name - #352

Open
tonyh168 wants to merge 2 commits into
flagos-ai:0.2.0-rc2from
tonyh168:hotfix/v0.2.0
Open

[hot-fix] fix GroupedTopKRouterFL routing and MLA decode method name#352
tonyh168 wants to merge 2 commits into
flagos-ai:0.2.0-rc2from
tonyh168:hotfix/v0.2.0

Conversation

@tonyh168

@tonyh168 tonyh168 commented Aug 7, 2026

Copy link
Copy Markdown

Summary

基于 v0.2.0 tag 的 hotfix,修复两个运行时 bug。共两个 commit,各修一处。

Bug 1 — GroupedTopKRouterFL 路由报错 (4b6c1b2)

vllm_fl/ops/fused_moe/router.pyGroupedTopKRouterFL._compute_routing
调用了 self._valid_grouping(router_logits),但上游 GroupedTopKRouter
并没有 _valid_grouping 方法 —— 它在 _compute_routing 内部用一个局部闭包
valid_grouping() 做校验。因此启用 grouped top-k 路由(DeepSeek-V2/V3、
GLM-MoE、混元等模型)时会抛 AttributeError

修复:移植 v0.3.0-dev 的做法,用内联的 valid_grouping() 闭包替换该调用
(判断逻辑:num_experts > num_expert_group 且能被整除)。仅移植该最小修复,
不含 v0.3.0-dev 中无关的 call_opCachedOp dispatch 重构。

```python
def valid_grouping() -> bool:
num_experts = router_logits.shape[-1]
if num_experts <= self.num_expert_group:
return False
return num_experts % self.num_expert_group == 0

if not valid_grouping():
...
```

Bug 2 — MLA decode 方法签名对不上 (2bac207)

vllm_fl/dispatch/backends/flaggems/impl/mla.pyMLAFLImpl 继承自
MLACommonImpl。vllm 0.20.0 把基类的 decode 覆写方法从 _forward_decode
重命名为 forward_mqa(参数签名完全相同)。FL 侧仍定义 _forward_decode,
导致该实现永远不会被调用,decode 时命中基类抽象的 forward_mqa → 抛
NotImplementedError

修复:将 MLAFLImpl._forward_decode 重命名为 forward_mqa,方法体不变。
(metax 后端用的是自带 vendored 的 MLA,不受影响,未改动。)

Test

  • 两个改动文件均通过 python -m py_compile
  • 确认 router.py 中不再存在 self._valid_grouping 调用
  • 确认 mla.py 中 decode 方法名为 forward_mqa

loopyt added 2 commits August 7, 2026 15:50
GroupedTopKRouterFL._compute_routing called self._valid_grouping(), which
does not exist on upstream GroupedTopKRouter (it uses an inner valid_grouping()
closure), raising AttributeError during grouped top-k routing. Port the inline
closure from v0.3.0-dev.
vllm 0.20.0 renamed the MLACommonImpl decode override from _forward_decode
to forward_mqa (same signature). MLAFLImpl still defined _forward_decode, so
its implementation was never dispatched and the abstract forward_mqa raised
NotImplementedError at decode. Rename to match the base class.
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


loopyt seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

@github-actions github-actions Bot added the core label Aug 7, 2026
@tonyh168 tonyh168 changed the title [hot-fix] 针对2.0.0 修复 flagrelease 发布中遇到的运行报错问题 [hot-fix] fix GroupedTopKRouterFL routing and MLA decode method name Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants