Four surfaces state the rules and nothing says which wins: schema, reference model, normative spec and docs disagree

[imran-siddique]

TRACE states its rules across four surfaces and nothing says which one wins when they disagree. Four reports in three days found four separate instances, all from producers who did the reasonable thing and validated against one of them.

The four surfaces

Surface	Files
Normative spec	spec/trace-v0.2.md
JSON Schema	schema/trace-claim.json and its byte-identical copy src/agentrust_trace/schema/trace-v0.2.json
Reference model	src/agentrust_trace/models.py
Prose docs	docs/*.md, plus docs/rfcs/* which disclaim being normative

The four instances

• schema: the software-only description states a narrower rule than §3.1.1 mandates #232 (PR #234) - the schema's runtime.platform description states a narrower rule than §3.1.1's MUST. Schema description under-states the spec.
• trust-levels: the all-zero measurement convention is stated with two different scopes #240 (PR #242) - docs/trust-levels.md is stale on the all-zero measurement convention. Docs under-state the spec.
• schema: the subject pattern accepts identities the reference model refuses #244 - the schema's subject pattern is a prefix test where models.py:343 requires a full shape, so a record can be schema-conformant and refused by model_validate. Schema is looser than the model.
• spec: parent_record_hash's preimage is stated only in a document that binds nothing #245 - parent_record_hash appears zero times in the normative spec, and its preimage is stated only in a draft that says "Binds nothing". No normative surface covers it at all.

Reported by @chernistry (four of them) and independently in the same window by @mohdibrahimaiml in #241, where the published 0.9.0 schema rejects §3.1.2 references on additionalProperties. Five, if that one is counted, and it is the same shape.

Each has been answered on its own merits. This issue is about the thing underneath them.

Why answering them individually is not enough

Every instance was found by a producer validating their records against one surface and discovering that a different surface disagreed. The schema check passing is not evidence the model will accept the record. The model accepting is not evidence the record is spec-conformant. A reader of the docs gets a fourth answer.

Two consequences worth stating plainly:

1. Conformance is currently unfalsifiable. "Is this record TRACE-conformant" has no single answer, so a producer cannot be told they are wrong, and a verifier cannot be told it is being too strict.
2. The failure is asymmetric and silent. The schema is the surface producers vendor and check against, and in three of the four instances it was the loose one. So the checks people actually run are the ones least likely to catch the divergence.

What this issue is for

Deciding the precedence rule and writing it somewhere a producer will find it. Not deciding the four individual answers, which are already ruled.

The obvious candidate: the normative spec wins; the schema and the reference model are derived artifacts and any divergence from the spec is a bug in the derived artifact. That is what the four rulings did in practice, and it is what §3.1.1 versus the schema description implies. But it needs to be true operationally, not just declared:

• The schema and the model currently drift because nothing checks them against each other. A test that generates the boundary cases from the spec, or at minimum a parity test between schema/trace-claim.json and models.py, would have caught schema: the subject pattern accepts identities the reference model refuses #244 before a producer did. trace-spec#236 added tests/test_the_schema_and_the_models_agree.py in that direction; whether it covers the subject case is worth checking first.
• Where the normative spec says nothing, as in spec: parent_record_hash's preimage is stated only in a document that binds nothing #245, precedence cannot resolve it. That is a coverage gap, and the answer is to write the spec text, not to promote a draft.
• The two schema copies being byte-identical is maintained by hand today.

Not in scope here

The individual rulings, and the release-versus-main gap that #241 sits on. That one is real and separate: a fix on main is not a fix for anyone until it is tagged, which is why #241's reporter hit a schema that is correct in this repository and wrong in the package they installed.

[lywinged]

whether it covers the subject case is worth checking first

Checked. It does not, and the reason is structural rather than an omission, which I think matters for the precedence question.

MUTANTS in tests/test_the_schema_and_the_models_agree.py holds 24 values, 13 of them strings, and none begins with spiffe:// or did:. The old subject pattern was a prefix test, so every one of those failed it as well as failing the model's shape: both validators rejected, and they agreed by rejecting. Splitting them needs a value that passes the prefix and fails the shape, and the matrix had none, for any field. The sweep ran 672 records and was green, correctly.

Four such values are added in the PR for #244. Measured there: with the schema reverted, four failures, all on subject; with the model loosened instead, the same four with the direction flipped. No other path moves, so DECLARED_DIVERGENCES gains no row.

There is also a denominator for one leg of this issue. Resolving the model's aliases, DigestStr and retention's _DURATION_RE, the schema carries ten patterns and the model has a counterpart for each. Nine were already identical strings. subject was the only one that differed, and after the fix that set is empty. That is the schema-against-model leg only; it says nothing about spec-against-schema or docs-against-schema.

The docs leg is worth a line, because it has a mechanism already. docs/schema.md described subject with the prefix reading, and had been stale since 84df8ef ("security: pre-launch hardening", #49, 2026-06-18) tightened the model and left the schema on the old string.

tests/test_build_provenance_depth_doc.py parses the build_provenance table out of that same file and checks it against the schema, and tests/test_references_block.py does the same for the references table and for one row of the top-level field table. So that top-level table, the one that went stale, is parsed for exactly one row, on its type and required cells, and never on a description cell. Generalising that to the rest of the table is small, and it is the operational half you are asking for rather than a declaration.

PR: #248.

[imran-siddique]

Narrowing this with a measurement rather than an opinion.

#248 merged, closing #244, and @lywinged did more than the fix: they audited all ten of the schema's patterns against the model and report that subject was the only differing pair.

So the schema-versus-model surface, which produced two of this issue's five instances (#244 here and the 100.0 case raised on #206), is now clean rather than merely fixed in one place.

Two things from that PR belong on this issue:

The parity test was structurally blind, and that is the reusable lesson. tests/test_the_schema_and_the_models_agree.py already existed and did not catch #244, because subject's schema pattern was a prefix test: every value in the differential's matrix failed both surfaces, so the two validators "agreed by both rejecting". A differential that reports agreement is worth nothing unless its inputs can separate the two implementations. Any future parity work here needs discriminating inputs generated from the looser surface, not a fixed matrix.

The drift was dated. The two patterns parted at 84df8ef ("security: pre-launch hardening", #49, 2026-06-18), which tightened the model and left the schema on the old string. Ten weeks in which a producer validating against the published schema could emit a record the reference implementation refuses.

What remains open on this issue, now narrower than when I filed it:

1. Spec-versus-schema, which is schema: the software-only description states a narrower rule than §3.1.1 mandates #232 and trust-levels: the all-zero measurement convention is stated with two different scopes #240, both with PRs open.
2. Spec coverage gaps, where no surface states the rule at all, which is spec: parent_record_hash's preimage is stated only in a document that binds nothing #245's parent_record_hash.
3. The release gap, 0.9.0 schema rejects §3.1.2 references (additionalProperties: false) #241: the schema in agentrust-trace 0.9.0 on PyPI is not the schema in this repository, so a producer can be conformant to the artifact they installed and refused by current main. That one is not fixed by any parity test.
4. The precedence rule itself, still unwritten: which surface wins when they disagree.

Item 3 is the one that keeps producing reports, and it is a release rather than a code change.

[chernistry]

Four of the five instances are mine, so the producer's-eye view is the part I can
add. Short version: I agree the normative spec wins, and I do not think saying so
changes anything on its own.

Why precedence alone does not close this. Every one of the four I filed was
found the same way — validate against one surface, ship, discover another surface
disagrees. Declaring which surface wins tells a producer who was right after
the divergence exists. It does not tell them a divergence exists, and that is the
part that costs the week. The rule is worth writing down, but it is the cheap
half.

@lywinged's finding is the reusable one, and it generalises past subject.
A parity test whose inputs fail both validators proves only that the two agree
on rejecting garbage. Agreement on a value neither accepts is not evidence of
agreement. The property a parity matrix has to have is the opposite of the one
this matrix had: for every constraint, at least one value that passes one
validator and fails the other. If no such value can be constructed for a
constraint, the two really do agree there and the test can say so; if it can, the
test either catches the split or it is blind by construction. That reframing
turns "the mutants happened not to cover subject" into a property the matrix
can be checked against, which is what stops the next one.

What I can contribute rather than just propose. Our vectors on #231 are
emitted by a checked-in generator, never hand-written, which is what makes them
reproducible. The same generator can emit a disagreement-seeking set: for each of
the schema's ten patterns, values derived mechanically from the pattern that
satisfy it and violate the model's shape, and the inverse where the model is the
looser one. That is exactly the subject case and, I believe, the 100.0 case
on #206. Say the word and I will send it as a PR against
tests/test_the_schema_and_the_models_agree.py rather than as a suggestion.

On #245, agreed and worth stating in the resolution. Precedence cannot
resolve a surface that says nothing. Promoting the draft would make a document
that disclaims binding into a binding one, which is the wrong move. If it helps,
I can post the exact preimage our producer computes for parent_record_hash,
with the byte layout, as input to the spec text — not as a proposal for what the
spec should say, but so whoever writes it has one real implementation to check
the wording against.

One addition to the "not in scope" line. The release-versus-main gap is not
only #241's problem; it makes a conformance claim ambiguous in the same way this
issue's four surfaces do. A producer saying "conformant to TRACE v0.2" today
cannot be checked unless they also say which tagged artifact they validated
against, because main and the published package are different rules. Whatever
precedence text lands, naming the tagged schema as the thing a claim is made
against would close that with one sentence.

[lywinged]

@chernistry The word is said. Send the generator PR.

Three steers so it lands on the first pass:

1. What it replaces. The current matrix is a hand-written tuple of 28 values (MUTANTS) swept over every path in one base record, with floors of 20 mutants and 25 paths so a sweep that found nothing to mutate cannot pass silently. Your per-constraint property checks the thing those floors cannot, which is discrimination. Keep a floor of the same kind anyway: for each of the ten patterns, either at least one value that splits the two validators, or an explicit assertion that no such value exists for that constraint.

2. The declaration stays two-sided. Today an undeclared disagreement fails one test and a stale declared entry fails the other (test_the_two_validators_agree_except_where_declared, test_no_declared_divergence_has_quietly_been_resolved). New splits your generator finds should enter DECLARED_DIVERGENCES with a reason string, not a new exemption list. One thing you will meet when you open the file: it already carries one environment-conditional divergence, format: uri on transparency and appraisal.verifier, handled by a computed exemption rather than a declared entry, because jsonschema enforces format only when a checker for it is installed. That pair sits outside your ten patterns, so your set does not need to cover it, but you will want to know why the exemption is computed before you touch the assertions it feeds.

3. On the tagged artifact sentence: agreed, and it is the version of the precedence rule that is actually checkable. "Conformant to TRACE v0.2" is not a claim anyone can verify while main and the published package disagree. "Conformant to the schema at tag X" is. That sentence belongs in the precedence text.

For the preimage bytes: yes, post them on #245. One real implementation to check the wording against is exactly what that issue is missing.

[chernistry]

Sent: #270.

Two things I did not expect to find, both worth reading before the diff:

• Five of the ten pattern constraints were unreachable by the existing matrix. They sit on optional members BASE omits — model.weights_digest, delegation.parent_record_hash, references[].retention, references[].digest, signature — so nothing ever mutated them. That is the same failure shape as schema: the subject pattern accepts identities the reference model refuses #244 one level up: not a disagreement nobody resolved, a constraint nobody asked about.
• models.py names this file as the thing that holds its pattern constants mirrored, and this file did not hold them. Two of the ten were pinned in test_references_block.py; the other eight were hand-maintained under that comment.

On steer 1, I made the floor stronger than "at least one splitting value per pattern", because for a mirrored constraint no such value can exist — demanding one would be demanding something unconstructible. The test asks for the fact that makes it unconstructible (the schema's pattern string is one the model carries) and falls through to requiring a declared splitter wherever that is not true. Three outcomes per constraint, none of them silent.

On steer 2: DECLARED_DIVERGENCES is reused, no new entries, and the third case gets its own two-sided set rather than an exemption list. I read the computed _URI_FORMAT_ENFORCED exemption before touching the assertions it feeds; format is not pattern, so it stays outside these ten.

On steer 3: agreed on the tagged-artifact sentence. One thing already true that the issue lists as open — tests/test_validate.py:142 pins the two schema copies against each other, so the by-hand part of that bullet is the published package versus main, not the two files in the repository.

Preimage bytes for #245 are on that issue: sha256: + hex SHA-256 of the JCS canonical form of the complete signed parent record, signature included, verified across two hops of a committed delegation chain.

[chernistry]

A fifth instance, and it is the first one I have that sits downstream of the spec rather than inside it — which I think bears on the precedence sentence more than the four I filed here.

This week our own exporter documentation told operators to verify records with --level 1 and called that the level to use in production. Our conformance test had always invoked --level 0. Both had been true-looking for weeks; nobody compared them, because they are different surfaces and nothing said which one a reader should believe. Measured against the four committed vectors, --level 1 fails TR-RTE-001 and TR-RTE-004 on every one — a software-only deployment cannot reach it, since both are properties of the deployment and of the verifier's call rather than of the record. Level 0 passes all four. The prose was the only thing that disagreed, and the prose is what a producer copies.

The reason I am putting it here rather than filing it: it is the same shape as #232 and #240, but the disagreement had propagated out of the spec and into a consumer's own documentation. A producer reading our page would have published a Level 1 conformance claim in good faith, from a page that was wrong in the same way and for the same reason. That is the cost of the missing precedence sentence measured one hop further out than this issue currently measures it — and it argues that whatever ruling lands should be stated somewhere a downstream project can cite when writing its own docs, not only somewhere the spec's own four surfaces can be ranked against each other.

Concretely, the two things that made it detectable on our side were the conformance suite pinning a level and the vectors being committed. Both are things TRACE already gives producers. A one-line normative statement of which surface wins would let a producer's own test assert the claim its docs make, which is the loop that closed this for us.

Three of mine are still open and waiting on you, no rush intended: #234 (schema description vs §3.1.1), #242 (docs/trust-levels.md on the all-zero convention), and #270 (the pattern-constraint generator, approved). #234 and #242 are why #232 and #240 are still open — happy to rebase either if they have gone stale.