测试问题修复:🐛 修复 FP8 转换规则与 Gather 索引行为 - #704
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🧭 背景
paddle_to_torch规则承担 Paddle API 在 PyTorch 侧的等价执行。FP8 量化场景中的标度张量布局、UE8M0 指数打包和极值标度处理必须与 Paddle 保持一致;同时,gather的一维索引分支需要保留 PyTorch 对索引类型和输出形状的原生语义。🔎 问题定位
原有 FP8 规则在
128x128量化、转置标度和 UE8M0 打包组合下,未将块级标度扩展到覆盖行,导致标度形状和打包结果不一致。融合 SwigLU FP8 量化还会在零值或极小值输入、幂次标度及不同 CUDA 架构下产生与 Paddle 不一致的数值。GatherRule对一维索引逐项执行narrow后再堆叠,既未统一转换索引类型,也可能改变原生index_select的边界行为和空索引输出设备属性。🛠️ 实现方案
FP8 路径统一按实际行数分配并生成标度,在
128x128模式下先展开块级反量化标度再打包 UE8M0;融合量化抽取标度计算逻辑,显式处理零值、无穷大、幂次标度和 SM90 差异,并直接使用浮点指数位完成 UE8M0 编码。Gather 一维索引改为转换为torch.long后调用torch.index_select,交由 PyTorch 保持一致的语义。🔧 主要变更
1. 修正 FP8 标度与 UE8M0 打包
调整块级量化的标度张量形状和
128x128标度展开逻辑,修复转置布局下的行数计算。融合 SwigLU FP8 量化使用统一的量化标度计算,并按浮点指数位打包 UE8M0;同时修正融合堆叠量化的打包 reshape 维度2. 简化 Gather 一维索引转换
将逐索引
narrow/stack实现替换为torch.index_select,并在调用前将索引转换为torch.long📁 改动文件
tester/paddle_to_torch/rules.py