feat: Slow-Fast inference support

[howard0su]

Summary

This branch adds the qwen35 SFI (Slow-Fast Inference) sparse full-attention path and fixes the benchmark/runtime plumbing needed to measure it correctly.

At a high level, single-token full-attention decode steps can now attend to a sparse set of cached KV positions instead of the full window, while periodic refresh steps still run dense attention to keep quality stable.

What changed

• add SFI gather plumbing to the qwen35 step graph
• enable sparse KV gather in qwen35 full-attention blocks during single-token/no-mask decode
• represent SFI gather indices as [budget, n_head_kv] so sparse KV gather matches ggml row-gather requirements
• initialize and refresh SFI selector state in the qwen35 runtime/test path
• seed decode from prefill last_tok correctly in the daemon fallback loop
• update qwen35 compress handling to use the current protocol and caller-provided drafter path/arch
• update benchmark defaults to repo-local, compatible draft model paths
• harden the Python benchmark client/harness so daemon exits or empty results fail loudly instead of being reported as valid runs
• update README examples/documentation to match the current runtime protocol

Why

Before this, the SFI benchmark path was not trustworthy:

• the qwen35 daemon still used an old compress protocol and a hardcoded drafter path
• benchmark defaults pointed at stale/incompatible draft paths
• broken runs could be reported as successful
• full SFI crashed at runtime because sparse gather indices had the wrong shape for 3D KV cache gathers

This branch fixes those issues and makes bench_sfi_ab.py exercise the real path end to end.

Benchmark

32K context, 1 sample:

Mode	Wall	Score	Gen	TTFT	Acc
baseline	46.94s	14.7s	5.0s	19.7s	1/1
refresh-only	45.88s	15.5s	4.9s	20.4s	1/1
sfi-full	45.83s	15.3s	4.4s	19.7s	1/1

vs baseline, full SFI:

• wall time: -2.4%
• generation time: -12.0%
• TTFT: no change
• accuracy: 1/1

How to test

 # baseline
 python3 scripts/bench_llm.py

 # full SFI, always sparse on eligible steps
 DFLASH27B_SFI_BUDGET=2048 python3 scripts/bench_llm.py

 # full SFI + periodic dense refresh
 DFLASH27B_SFI_BUDGET=2048 DFLASH27B_FA_REFRESH_INTERVAL=4096 \
 python3 scripts/bench_llm.py

[cubic-dev-ai]

2 issues found across 21 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="dflash/scripts/bench_sfi_ab.py">

<violation number="1" location="dflash/scripts/bench_sfi_ab.py:111">
P1: Benchmark subprocess failures are recorded but not enforced, so the script can report results and exit successfully after failed runs.</violation>
</file>

<file name="pflash/README.md">

<violation number="1" location="pflash/README.md:91">
P2: README weight-download step does not produce the GGUF draft artifact now required by updated commands.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
Re-trigger cubic}

[cubic-dev-ai]

2 issues found across 16 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="pflash/pflash/dflash_client.py">

<violation number="1">
P1: compress and generate no longer detect daemon death and can silently return partial/empty outputs on subprocess failure.</violation>
</file>

<file name="pflash/tests/bench_niah_cpp.py">

<violation number="1">
P2: Hardcoded absolute defaults make the benchmark fail in a normal checkout unless every path is overridden.</violation>
</file>

_{Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
Re-trigger cubic}

[howard0su]

After we fix RoPE bug, this PR doesn't show gain. close it.