[KMCompiler][Hygon] Token-tiled fused_deepseek_v4_qnorm_rope_kv_rope_quant_insert

[cheersluvs]

A token-tiled Hygon override for
fused_deepseek_v4_qnorm_rope_kv_rope_quant_insert. Two files, both under
src/flag_gems/runtime/backend/_hygon/fused/ — no shared module and no generic
operator changed.

Stacked on #5321. Until that merges, the diff shown here also contains its
commits. The part belonging to this PR is exactly the two _hygon/ files; there
is no overlap between the two branches.

Performance

Measured on Hygon BW1000 against 1340.3 GB/s of achievable bandwidth
(512 MiB device-to-device copy — measured, not a spec figure):

shape	generic	tiled
32768 x 64	604.8 GB/s (45.1%)	1182.9 GB/s (88.3%)
32768 x 128	609.2 GB/s (45.5%)	1201.4 GB/s (89.6%)

The generic kernel gives each program one (token, head) slot at num_warps=1,
which on a 64-lane warp is 8 elements per lane, and that leaves more than half the
bandwidth unused here.

Through this repo's benchmark, --mode kernel --level core, all 22 shapes, against
vLLM's own portable Triton implementation of this operator as the baseline:

tokens	64 heads	128 heads
1-17	1.20-1.54x	1.27-1.54x
1024-2048	1.94-1.98x	1.96-2.30x
8192-131072	2.02-2.04x	2.99-3.22x

Two caveats on that table, because it is easy to read more into it than it says.
The figures at 128 heads are larger mostly because the baseline degrades there:
this operator holds 89.6% of achievable at both head counts while the baseline
falls from 45% to 29.6%, so the honest summary is "about 2x, more at 128 heads
because the baseline drops". And the 1-17 token rows dispatch to the generic
kernel, so that gain is one fused launch beating the baseline's two and has nothing
to do with tiling.

The baseline is not a vendor kernel: there is no vendor implementation of this
operator for this card. vllm._custom_ops registers 49 ops and none is this one,
and a scan of 2505 shared objects for the symbol found nothing. The vendor's
lightop does cover this stage — fused_rms_norm_rope_contiguous and
fuse_rmsnorm_rope_quant_qkv both take slot_mapping and kv_cache — but only
for a scalar-scale FP8 convention: kv_cache_scale is a single float, which
cannot carry seven per-block exponents per token, so that interface cannot express
this operator's UE8M0 blockwise layout whatever the kernel does internally.

What the optimisation does

Each program handles TPP=8 tokens of one slot, so every program is uniformly
all-Q or all-KV with no divergence, at num_warps=4: 16 elements per lane instead
of 8, in a 256-thread block instead of 64.

Two axes matter and neither is visible on its own, which is worth recording
because tuning them separately gives the wrong answer twice over. A full
TPP x num_warps sweep puts every optimum at TPP / num_warps = 2 — two tokens
per warp, so 16 elements per lane. Bandwidth by elements per lane is
237 / 408 / 604 / 906 / 1183 / 1077 / 1050 GB/s for 1 / 2 / 4 / 8 / 16 / 32 /
64. But elements per lane does not explain everything: TPP=1/warps=1 and
TPP=2/warps=2 are both 8 elements per lane and differ by 50% (604 vs 906 GB/s),
because the second has a wider program. Meanwhile sweeping num_warps alone at
TPP=1 shows 8 and 4 elements per lane tied at ~605, which invites the conclusion
that access width does not matter — at TPP=1 the block is only 512 elements and
there is nothing to widen into.

TPP=8 / num_warps=4 is one of the optimal points and is also what the MetaX C550
override in #5321 uses. Both parts have 64-lane warps, so the tuning transfers
directly.

Dispatch threshold

Below a threshold the generic kernel wins: a 256-thread block costs about 10 us
more to launch here (measured at one token: 80.5 us versus 90.2 us) and TPP=8
masks off most of every program when there are fewer than 8 tokens to fill it.

The threshold is a program count, not a token count. The crossover measured at
256 tokens for 64 heads and 128 tokens for 128 heads — 16640 and 16512 programs
respectively — so num_tokens * (num_heads + 1) is what the two agree on, and the
threshold is 16384. A flat 256-token threshold would forfeit the 1.16x-1.32x
available between 128 and 256 tokens at 128 heads.

Correctness

This override is verified against the torch reference, not against the generic
kernel, and the distinction turned out to matter. Under the stack on the machine
used for development (torch 2.10.0 / FlagTree 0.6.1a1+hcu3.6) the tiled and
generic kernels are bit-identical at every shape tried — 32 shapes, 10 repetitions
each, zero differing bytes and zero differing q elements. Under the combination
backends.yaml pins for this card, which is what CI installs (torch 2.9.0 /
FlagTree 0.5.1+hcu3.1), they are not: q differs on 5 of 16744448 elements by up
to 2 bf16 ULP at 511 tokens, and one FP8 cache byte differs at 1000 tokens.

That is expected rather than alarming, once stated precisely: a 2-D [TPP, 512]
tile assigns lanes differently than a 1-D 512 block, so the RMSNorm variance
rounds differently, and how that lowers is the compiler's choice. It is why
#5321's override test compares each implementation against the reference at the
tolerances the accuracy tests use, rather than requiring two Triton
implementations to agree more closely than either agrees with the oracle.

The accuracy suite is 60 passed / 0 failed / 0 skipped with the override
active.

That suite was also run with the threshold temporarily set to 0, forcing every
shape through the tiled path. This is the only way to exercise the tok_ok masking
at 1, 4 and 17 tokens, where 7 or more of each program's 8 slots are empty — at the
production threshold those shapes dispatch to the generic kernel and the masking is
never tested. It passes 60/60 there too.

One thing for reviewers to decide

The ~140-line tiled kernel is now duplicated between _metax/fused/ and
_hygon/fused/, identical in both. Both parts are 64-lane, so the two copies will
stay in sync by accident rather than by construction. The alternatives are a shared
module or one backend importing the other's, and neither looked obviously right in
this layout, so raising it rather than deciding it unilaterally. Happy to factor it
out whichever way you prefer.

[cheersluvs]

Folded into #5321, which now carries both backend overrides and the four test/benchmark fixes as one change. Nothing is lost: #5449's branch was stacked on #5321's, so its single commit has been fast-forwarded there. Closing to keep the review in one place.