perf(text2semantic): pass only active KV cache prefix to SDPA

[quantumxiaol]

Is this PR adding new feature or fix a BUG?

Performance optimization for the native PyTorch autoregressive inference path.

Problem

During cached generation, the physical KV cache is preallocated at
max_seq_len. At every decode step, the current implementation expands the
full-capacity K/V tensors with repeat_interleave and passes the full K/V
sequence to SDPA. For S2-Pro, this means expanding and passing a key sequence
length of 32768 to SDPA even when only a few hundred positions have been
populated.

This describes the tensors supplied to SDPA; it does not assume how a
particular SDPA backend handles the masked, unpopulated tail internally.

Change

This change keeps the physical KV cache at full capacity but limits attention
to the active prefix:

• prompt prefill uses kv_len=T;
• decode step i uses kv_len=T+i+1;
• the causal mask is sliced to the active K length;
• cached K/V are sliced before head expansion and SDPA.

The regular full-sequence forward() path used for training does not use the
inference KV cache and is unchanged. The new slicing is gated by
self.kv_cache is not None and is exercised by cached forward_generate()
inference.

Benchmark results

S2-Pro, FP16, native PyTorch CLI inference, prompt length 234,
max_seq_len=32768. Throughput is the median of three warm samples:

Mode	Full-capacity KV	Active-prefix KV	Speedup	Peak CUDA memory reserved (GB)
Eager	2.73 tok/s	12.57 tok/s	4.60x	17.33 -> 15.16
torch.compile	16.65 tok/s	32.74 tok/s	1.97x	15.72 -> 15.16

CUDA environment: Quadro GV100 32 GB, PyTorch 2.8.0+cu128, Triton 3.4.0.

Generated lengths differed slightly because FP16/kernel differences can change
stochastic sampling trajectories, so throughput is normalized by the number of
generated tokens.

The memory values are decimal GB, matching the existing log calculation
torch.cuda.max_memory_reserved() / 1e9.

The same change was also validated on Apple MPS (M4 Pro 48 GB, FP16,
max_seq_len=4096), where semantic generation improved from 4.08 to
8.31 frames/s (2.04x). The MPS service metric and CUDA CLI tok/s are not
directly comparable; only their within-device A/B ratios are reported.

Cold compile time is excluded from the throughput comparison. The existing
Bandwidth achieved log value is also excluded because it is derived from
model size and tok/s rather than measured hardware DRAM bandwidth.

Correctness and compatibility validation

• Upstream pre-commit hooks pass for both modified files.
• A CPU shape check confirmed that SDPA receives only the active K/V prefix
  while the physical KV cache retains full capacity.
• A CPU float32 comparison confirmed that active-prefix attention matches
  full-cache attention with a masked tail (rtol=1e-5, atol=1e-6).
• CUDA eager and torch.compile generation completed successfully.
• With TORCH_LOGS=recompiles, the compiled active-KV path recompiled once
  when kv_len first changed, not once per token; warm throughput remained
  stable at approximately 32.7 tok/s.
• Four representative CUDA outputs (full/active x eager/compiled) were decoded
  and manually listened to; no audible regression was observed.
• MPS end-to-end generation and manual listening completed successfully.

The benchmark results above were collected from
c4146e7.
The current draft contains only the two production source changes; the
standalone correctness tests used for validation remain available in that
commit.

Is this pull request related to any issue? If yes, please link the issue.

Closes #1310