PARETO-1 rung A: the K-quant path, and the instrument it is measured with - #413
Open
chrishayuk wants to merge 7 commits into
Open
PARETO-1 rung A: the K-quant path, and the instrument it is measured with#413chrishayuk wants to merge 7 commits into
chrishayuk wants to merge 7 commits into
Conversation
…g trusts `Q8_0`/`Q6_K`/`Q4_K` encoders existed but only Kimi's routed-expert and KDA bank compilers could reach them, so a dense model could not be compiled to a K-quant at all. This is that vocabulary, stated once. Planning a segment needs each tensor's encoded length BEFORE any bytes are written, so the compiler cannot encode and measure — which forces a geometry table, and a table is a second statement of a fact the codecs already make. So it is checked against them in both directions rather than believed: encode a real buffer, assert the byte count matches, then decode and assert the element count returns. Eligibility is the part worth reading twice. It is NOT `n_elements % block == 0`; it is `row_len % block == 0`, with each outer row framed independently, because that is how ggml lays a row out. `[2, 128]` separates the two: 256 elements, exactly one Q6_K super-block, so the flat rule admits it and the resulting block spans the end of row 0 and the start of row 1 under one shared scale. Nothing crashes — the bytes serialise, the table is self-consistent, the container loads, and the arm yields plausible behavioural numbers from semantically invalid bytes. The test asserts BOTH that the flat check accepts `[2,128]` and that `plan` refuses it, so the rule cannot be "simplified" back to the total. Each K-quant is its own `CodecIdentity` family rather than a revision of a shared one. `Q4_K` and `Q6_K` are different physical interpretations, so the invariant kept is "same family => the reader can interpret this layout", never "same quantisation lineage => vaguely related". That matters more as Q2_K/Q3_K/IQ variants arrive. `EncoderRecipe` gains a `source` field for an encoder this workspace does not own. Unlike a LARQL recipe name — which that type's docs rightly keep free of build ids — an external encoder's upstream revision does determine the chosen values, so it participates in equality and therefore in the reproducibility claim.
`compile_representation` refused every encoding but NVFP4, so the general
model-agnostic path could not produce a K-quant even though the encoders
existed. The refusal becomes a target dispatch.
NVFP4 is a split pack — codes, group scales, a tensor scale — whose
planner derives a layout from a 2-D shape. A K-quant is contiguous ggml
blocks along the row. They share this function's plan-then-write
skeleton and nothing else, so the choice is a `Target`, not a flag on one
path, and the shape question ("does the encoding FIT?") is asked of the
target rather than of `PackLayout` directly. The role policy keeps its
separate question ("should the encoding be SPENT here?") untouched.
The planner contract now documents the row-framing rule and why it
matters once a search enumerates scopes automatically: under the wrong
rule the optimiser discovers scopes that look valid, measures them, and
folds them into a Pareto curve that is quietly part fiction. A
representation error that fails loudly costs an afternoon; that one
costs the curve's credibility.
On the load side a K-quant decodes in `OperandStore::load`, not in
`widen`, following the precedent NVFP4 already set: the block count comes
from the operand's SHAPE, and `widen` sees only bytes and a dtype label.
Keeping it out of `widen` means the one function that answers "what does
this dtype mean as f32, from bytes alone" keeps meaning exactly that.
`encode_kquant` and `kquant_encoder_recipe` are derived in one place so
the bytes and the provenance describing them cannot disagree.
A compiled K-quant pack was unreachable at execution: the backend is what declares which representation a run wants, and no arm asked for one. Three arms, one per compilable encoding, rather than one arm plus a flag — `wanted_representation` is exhaustive on purpose, so a new backend is a compile error until someone states what it runs. That design earned its keep immediately: `lowered_formats` is a second exhaustive match over the same enum and the compiler refused the change until it was covered too. These arms decode a K-quant operand to f32 rather than running K-quant kernels. That is deliberate and is the right instrument for the question being asked — what behaviour a representation buys per stored byte — and the wrong one for how fast a Q4_K kernel is. Decoding removes kernel implementation quality as a confounder. Native K-quant kernels are a separate substrate experiment; this model has no Metal path at all.
… nothing
`decode(encode(x)) ~ x` passes just as happily when the encoder and the
decoder share ONE misunderstanding — a swapped nibble order, a scale and
a min the wrong way round, sub-scales read backwards. Every such pair
round-trips perfectly and is wrong against the rest of the world.
So these bytes come from llama.cpp's ggml. `fixtures/
ggml_kquant_golden.gen.c` links against it and dumps, for one frozen
512-value input, ggml's own block geometry, its numeric type ids, its
quantiser's bytes, and its own decode of them. Generated, never
transcribed — a transcription is the step that goes wrong.
Three properties a round-trip collapses into one reassuring pass, here
separated so each can fail on its own:
geometry ggml's blck_size/type_size confirm the table PASSES
(the table's numbers were read off this crate's
decoders, so those decoders cannot confirm it)
type ids ggml's ids confirm ours see 3576f34a
layout decode_larql(bytes_ggml) == decode_ggml(...) PASSES bit-for-bit
encoder encode_larql(x) == bytes_ggml DOES NOT HOLD
The last is recorded rather than asserted, because it is a legitimate
difference and not a defect: LARQL emits the ggml *representation* with
its own chosen values. Q8_0 differs in 10 of 544 bytes, Q6_K in 366 of
420, Q4_K in 94 of 288. Any comparison written against a
llama.cpp-derived artifact must therefore say which of those it means.
A fixture also has to be capable of failing, so that is asserted too: the
input is signed on both sides, non-monotone, and its two super-blocks
differ, and reversing a real Q6_K pack's bytes must change what comes
out. A ramp or a symmetric input would let a wrong layout still look
right.
…inned
LARQL's native K-quant encoders emit legal, correctly-laid-out bytes but
choose different values from ggml, and the gap is not uniform:
Q8_0 0.9967 Q4_K 1.0326 Q6_K 1.1146 reconstruction RMS ratio
It tracks how much scale search a format admits — Q8_0 has one scale per
32 values and nothing to search; Q6_K has 16 int8 sub-scales plus an f16
`d` and the most to gain. A mechanism, not noise.
That is a confound, not a curiosity. REPRESENT's claim is that it wins
the SELECTION race, not the codec race, and that is only isolable if the
codec is held constant against the ecosystem's. A matched-byte
comparison against a llama.cpp-derived artifact would otherwise measure
allocation MINUS a bit-width-dependent codec handicap, and a loss would
be unattributable between the two. Worse, the deficit is differential:
an artificially bad Q6_K point shrinks kl(Q4)-kl(Q6) and inflates
kl(Q6)-kl(Q8), biasing a step-versus-slope question toward one answer
for reasons that have nothing to do with the model.
So comparative artifacts are encoded by ggml itself, and the only
independent variable left is where precision is spent.
The dependency belongs to artifact PRODUCTION, never to consumption. Off
by default; the reader and runtime keep their own decoders and acquire
no llama.cpp dependency, which the foreign fixture already showed is
sound. And the control is architectural rather than statistical: the
feature being compiled in IS the switch, so there is no runtime flag to
set wrong, a campaign cannot accidentally call the native encoder, and a
future regression in `quantize_q6_k` cannot perturb an experiment that
never called it.
`CodecIdentity` is deliberately untouched — both encoders produce the
same codec family, and identity answers "what are these bytes and how
are they decoded". Which implementation chose the values goes to
`EncoderRecipe`, which already existed for exactly this distinction and
whose docs already say an encoder mismatch is not a refusal, only a
weakened reproducibility claim. An artifact can now say:
codec Q6_K
encoder kquant-ggml-reference-v1 @ <pinned upstream>
selection REPRESENT <policy/evidence id>
The native encoders stay: executable format documentation, a
self-contained fallback, differential-test subjects, and a research
target. Their RMS comparison is demoted from a research-eligibility gate
to a quality test for them — with a floor as well as a ceiling, so if
they ever reach parity the loose bound is tightened rather than left
recording a gap that has closed.
…ers to
PARETO-1 asks whether REPRESENT can contest Unsloth Dynamic 3.0 on
Qwen3.8-27B. The audit is what a first honest look found, with file:line
evidence for each claim, and it is deliberately unflattering in places.
Six gaps. Two are decisive and were not visible from the outside:
- at the Q3_XL and Q2_XL budgets REPRESENT has NO ACTION AT ALL, not
merely a weaker codec — though this narrowed on inspection, since
ggml decoders for Q2_K/Q3_K/Q5_K already exist and it is an
ENCODER-only gap;
- Qwen3.8's 48 Gated DeltaNet layers have no Metal kernel, so every
arm is CPU and there is NO throughput arm. A result here is
behaviour-per-byte only, and the bytes/token -> GPU ms -> tok/s
chain that most distinguishes REPRESENT's objective from Unsloth's
cannot be exercised on this model. Writing it up as though it could
would be a false claim, so the pre-registration excludes it.
The model is also dense, so two of the five Kimi constraint dimensions
— including the routed-mixture pair that BALANCED-SEARCH-2 found was
actually binding — do not exist, and a contract must be earned here
rather than transplanted.
The audit closes with why the instrument deserves trust. Three defects
were found at three boundaries — tensor geometry, encoder
implementation, foreign ABI identity — and all three shared one
pathology: internal agreement is not evidence of external correctness.
Each successive check defeats a class of shared misconception the
previous one is blind to. None changed the research hypothesis; all
three changed what the instrument can show.
The pre-registration is frozen outside the repository and named here by
sha256, so a later edit is visible as an edit rather than absorbed
silently. It records E1-E4, the stopping rule, the control, the band
construction, and two amendments written before any anchor arm existed
— which is the only reason they are evidence rather than
rationalisation.
chrishayuk
force-pushed
the
pareto-1/unsloth-headhead
branch
from
September 4, 2026 00:41
a54d0df to
b6b290a
Compare
The coverage gate failed on four files, all of them touched by this
branch, and the diagnosis is worth keeping rather than quietly fixing:
kquant.rs 65.59% with nineteen passing tests
represent/mod.rs 82.11%
operands.rs 88.34%
nvfp4_pack.rs 89.14%
The unit tests proved the CODEC. Nothing ran `compile_representation`
down the K-quant branch or `OperandStore::load` through the K-quant
decode, so the wiring between them was type-checked, reviewed, described
at length — and never once executed. Seven other CI checks were
satisfied by a path that had never run; the coverage floor was the only
observation that could tell "this is correct" from "this has ever
happened".
That is the same pathology as the three defects this branch already
records, arriving a fourth time in the review of the work itself:
components tested and code compiling is internal agreement.
So the fix is execution, not a policy baseline entry:
- a Q8_0 representation compiles, is smaller than its BF16 source,
declares a codec this build admits and an encoder recipe naming who
chose the values, then re-opens and decodes through the store;
- a Q6_K map compiles ONLY `down_proj`'s 256-wide rows and carries the
64-wide projections — the row-geometry rule acting as an eligibility
filter rather than a whole-object refusal, which is the `[2,128]`
rule in executable form on a real container;
- an uncompilable encoding is refused by a message listing the
vocabulary that actually exists rather than a stale literal.
Plus the branch guards the codec needs: a table row with no encoder
refuses instead of passing source bytes through under a name claiming
otherwise; a table that disagrees with its codec about a length is
caught at encode time rather than by a segment whose tensor table does
not describe its payload; each K-quant's codec identity is admitted by
this build and is not equal to another's.
kquant.rs 100.00% mod.rs 90.44% operands.rs 90.16% nvfp4 92.76%
One end-to-end test lifted three of those files together, which is what
a connected wiring gap looks like from the outside.
Also corrects `RepresentSpec::encoding`'s doc, stale since the dispatch
landed: it still said only NVFP4 was compilable.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #412 — based on
fix/ggml-type-idsso this diff contains only the K-quant work. After #412 merges this will be rebased onto main and retargeted (rebase + force-push, never close/reopen).What this unblocks
PARETO-1 asks whether REPRESENT can contest Unsloth Dynamic 3.0 on Qwen3.8-27B. The readiness audit found six gaps; this closes the one that blocked everything:
Q8_0/Q6_K/Q4_Kencoders existed but were reachable only through Kimi's routed-expert and KDA bank compilers, so a dense model could not be compiled to a K-quant at all.Six commits, each independently compilable
fix(models)feat(represent)feat(represent)feat(cli)test(represent)feat(represent)docs(pareto-1)Verified by checking each out and building it, not asserted.
The three defects found while building it
All at different boundaries, all sharing one pathology — internal agreement is not evidence of external correctness:
row_len % block == 0, notn_elements % block == 0.[2,128]is 256 elements — one whole Q6_K super-block — so the flat rule admits it and emits a block spanning two rows under one shared scale. Nothing crashes; the arm yields plausible behavioural numbers from semantically invalid bytes. Once a search enumerates scopes automatically that would fold fiction into a Pareto curve.Holding the codec constant
REPRESENT claims it wins the selection race, not the codec race — isolable only if the codec is constant against the ecosystem's. So comparative artifacts are encoded by ggml itself (
--features reference-encoder, off by default, production-side only; the reader and runtime keep their own decoders). The control is architectural: the feature being compiled in is the switch, so a campaign cannot accidentally call the native encoder and a future regression inquantize_q6_kcannot perturb an experiment that never called it.CodecIdentityis deliberately untouched — both encoders produce the same codec family. Which implementation chose the values goes toEncoderRecipe.Gates
Not claimed
No measurement. No throughput arm — Qwen3.8's 48 Gated DeltaNet layers have no Metal kernel, so this is behaviour-per-byte only.
PARITY-FLOOR-1stays open, so relative comparisons only. The pre-registration is frozen outside the repo and named by sha256 in the audit, so a later edit is visible as an edit.