Skip to content

🧪 test(pypi): close the pypi and webhook mutation survivors - #2255

Merged
gaborbernat merged 37 commits into
mainfrom
test/pypi-mutants-1893
Sep 9, 2026
Merged

🧪 test(pypi): close the pypi and webhook mutation survivors#2255
gaborbernat merged 37 commits into
mainfrom
test/pypi-mutants-1893

Conversation

@gaborbernat

Copy link
Copy Markdown
Member

The nightly mutation run behind #1893 reported survivors across peryx-ecosystem-pypi: the search text budgets and their separator, the streaming JSON transformer and validator, the shadow query bounds, the page streaming fallbacks, the ranged wheel and provenance size limits, the mutation handlers, and the gate that wakes the webhook scheduler after a yank, restore or delete. 🧬 Each survivor is a decision the code makes that nothing observed, so a wrong operator or a dropped guard would ship unnoticed.

Most rows close by pinning the behaviour at the exact edge the mutation moves: a budget at its last admitted byte, a brace or bracket inside a captured string, a member or bundle of exactly the limit, a token spent on precisely the project the path names. Where a mutation was unobservable the code changed so the choice matters. Hex accumulation in the validator and transformer is spelled as value * 16 + digit, since with a zeroed low nibble |, ^ and + agree on every input. The catalog text cap is applied once as INDEXED_TEXT_BYTES instead of a share arithmetic that computed the same bound a second time, and a whitespace guard plus a stream size guard that no input could reach are gone. The changed-count gate moves out of serving/mutate.rs into peryx_events::webhook::notify_changed, next to the wait it governs, because from the HTTP surface a skipped wake only shows as the delivery arriving one scheduler tick later.

🔎 No externally visible behaviour changes. peryx-events gains the public notify_changed, and the PyPI mutation handlers call it with their result instead of deciding locally.

Refs #1893

Nothing drove the streaming transformer's \uXXXX key decoder, so its hex
arithmetic could be changed in either letter range without a test noticing. A
member name spelled with escapes is the same name, so the page has to dispatch
the same way as one spelled literally.

The two letter ranges are separate arms, so each spelling is its own case, and a
third mixes an escape with literal bytes to cross back out of the escape state.
Nothing read the three indexed-text budgets or drove push_text directly, so the
arithmetic dividing one document allowance between identity, core metadata and
catalog text could be changed without a test noticing, and so could the rule
that separates appended values without opening the document with a gap.
Asserting only that the three shares add up could not fail: the catalog share is
defined as what the other two leave, so the sum balances whatever the other two
become. Turning a division into a remainder left identity holding three bytes
and the total still correct.

What the shares are for is holding text, so each has to be big enough to hold
some. That is the property the arithmetic exists to produce, and it is what a
remainder breaks.
The retention estimate's only check compared it against a second copy of the
same formula in the fixture, which changes with any operator changed inside it,
so none of its eight arithmetic mutants could fail.

What the estimate promises is that every owned field contributes its bytes.
Lengthening one field by six has to move the total by six, and a candidate
holding no text has to cost the struct alone.
The page file cap sat behind a constant of five hundred thousand, so moving its
comparison changed nothing any test could see. The counter it reads is a field,
so seeding it puts the boundary one call away and needs no page at all.

The file that reaches the cap is within it and fails on its own contents; the
first one past it is refused as too large.
Shifting and or-ing over disjoint bit ranges is an addition in other clothes,
which is why swapping the operator changed nothing: with the low nibble always
zero, |, ^ and + agree on every input. An operator whose choice cannot be
observed is an operator whose choice was arbitrary.

Written as the multiply and add it performs, a wrong operator shows. Four hex
digits is the most \uXXXX carries, so the accumulation tops out at 0xFFF * 16 +
15, which is exactly u16::MAX, and the arm sets Literal on the fourth digit so
no fifth can arrive.
Nothing seeded a status without a reason or a reason without a status, so the
condition that skips only when both are missing could be loosened to skip when
either is, and the page would silently lose a status peryx held.
The arm skipped whitespace only while the capture buffer was empty, so it
dropped leading whitespace and let interior whitespace accumulate. Nothing can
tell the difference: capture.is_empty() appears nowhere else, the buffer carries
no length or index check, and its only other reader is serde_json, which skips
leading whitespace the way the grammar says to.

That leaves no memory property to credit it with either, since interior
whitespace already goes where the guard was refusing to let leading whitespace
go. A branch no observer can detect was carrying no decision.

The test asserts what the argument rests on: a page that indents its file
elements transforms exactly as one that packs them.
The budget is written as a product of three numbers and compared against
nothing, so an operator changed inside it would move the size every project is
planned against while every test that merely passes it along kept passing.
Both key parsers require two things at once, and nothing checked that either is
load-bearing: a key naming an empty project, a further path segment, or no
filename has to be refused rather than indexed under a name it does not have.
Nothing checked either negation. A sidecar is servable only when it names its
own digest, so a claim that one merely exists has to be dropped rather than
kept; and an upstream URL is rewritten to this node's route while one already
pointing here is left alone rather than routed twice.
Catalog text is allowed whatever identity and core metadata leave unspent, and
nothing checked that. A project with a short name and no metadata can list far
more versions than the catalog's own share would hold; subtracting that leftover
instead of adding it cuts the list short while every other section looks the
same.
The listing was three orders of magnitude too small to tell anything apart. The
whole allowance is MAX_TOKEN_LEN, just under 64 KiB, so the catalog's own share
is about 16 KiB and the leftover under test is about 32 KiB; a thirty-version
listing fits inside every arrangement of those numbers and discriminates
nothing.

Five thousand versions crosses the boundary and runs in 25 milliseconds, so the
subtraction now fails the test that the addition passes.
Working out what identity and core metadata left unspent computed the same
bound the assembly already applies. The three sections are joined under
INDEXED_TEXT_BYTES, and the separators between them make that join bind a byte
or two sooner than any share arithmetic could, so the inner limit could never
be the one that stopped anything.

Checked rather than argued: across twenty-four shapes, from an empty label to
one past the identity share, and from one version to twelve thousand, the text
comes back byte for byte the same. The share the catalog was computing existed
only to feed that expression, so it goes with it.
An assertion written directly over two constants folds at compile time, which
clippy rejects. Binding the sum first keeps the check that neither share may
collapse and that the two together leave room for catalog text.
A versions array lists strings, so an object inside one is refused. Which
refusal arrives depends on finding where that array ends: following the nesting
reports the object as ill-typed, while ignoring it takes the object's own brace
for the array's close and reports the document truncated.

Nothing drove a nested object through that scanner, so the depth it keeps could
be dropped without a test noticing.
A brace does not close an array, so a stray one inside versions leaves the array
open and the page runs out before it ends. Nothing drove that, so the test for
which byte closes the array could be loosened to accept either bracket or depth
alone, turning a document that was really cut short into a parse error about the
wrong thing.
A policy denial reaches the client in the vocabulary it speaks: the neutral
engine names a resource, an artifact and a group, and a PyPI reader knows those
as a project, a filename and a version. Nothing checked the translation, so an
arm dropped from it would send engine jargon to a client with no way to act on
it.

The replay window and the JSON preflight are each written as a product and read
nowhere else, so an operator changed inside either would resize it while every
test that stays within the old size kept passing.
Nothing exercised ShadowQuery::validate, so its two size bounds could admit or
refuse one item either side of themselves unnoticed. A filter or cursor of
exactly the permitted size is still a request peryx answers, and only what
passes the bound is refused.

A cursor is a position rather than a value, so an empty one names nothing and is
refused whatever its length.
A shift-and-or over disjoint bit ranges is an addition wearing other clothes:
the low nibble is always zero, so |, ^ and + agree on every one of the 65536
prefix-and-digit pairs and the operator's choice cannot be observed. This is the
same shape #2209 rewrote in the page transformer.

Written as the multiply and add it performs, a wrong operator shows: dividing
instead of multiplying changes the result on 65520 of those pairs and
subtracting instead of adding on 61440. Four hex digits is all \uXXXX carries,
so the accumulation tops out at 0xFFF * 16 + 15, exactly u16::MAX.
The limit is checked twice, from the length upstream declares and from the bytes
that actually arrive, because a declared length is a claim rather than a
measurement. Neither edge was exercised, so both comparisons could admit or
refuse one byte either side of themselves unnoticed.

Building the response in process rather than over a socket keeps a two mebibyte
body to a Vec allocation: each case runs in fifteen milliseconds.
Reading a wheel's metadata over ranges is arithmetic on offsets and lengths:
where the central directory starts and ends, where the member's data begins, and
how far it runs. Every one of those ends is inclusive, so each is a length short
of the next offset, and getting any of them wrong asks the server for the wrong
bytes. The answer is then a refused range or a member that will not decode,
never the metadata.

Nothing drove that arithmetic. The only ranged-read test rejected bad filenames
without fetching anything, which left six lines of offset and length work
unexercised.

The server answers the way a real one does, a HEAD carrying the length and a
strong validator and then each GET returning exactly the bytes its Range asked
for, so the test pins the member's contents rather than the shape of the reply.
The budget is a comparison, so only a member at it separates admitting from
refusing one: an over-limit case and an under-limit case agree whichever way the
operator points. A member of exactly sixteen mebibytes is read, one byte more is
declined.

Both sizes a ZIP records are asked the same question, and either one over is
enough to decline, so a member that compresses small is still judged by what it
decodes to rather than by what it occupies.

Storing the member uncompressed is what makes the archived size meet the budget
too; the compressing case is what tells the two questions apart.
A finalized upload's result is replayable for a day, so an identical resend
inside that window replays and never reaches the store. Once the reaper takes
the record, the same bytes are a fresh operation that reaches the store and finds
them already there, and the audit trail says so: a success carrying no reason,
then a noop naming the duplicate.

That path had no test because no test could reach it. Pruning is what opens it,
and pinning the window from both sides fixes where the day ends: a reaper a
second early prunes nothing, one on time prunes exactly this record.
An upstream with nothing to say leaves peryx serving whatever it holds of its
own, so a page is absent only when there is nothing local at all. Holding either
files or versions is enough to answer from the local side.

Nothing exercised a project holding versions but no files, so requiring both to
be missing could have been loosened to either without a test noticing, and a
project peryx could still describe would have answered not-found.
A page transformed from a cached record stays servable for exactly the freshness
that record was granted, counted from when it was fetched. Nothing read the
expiry back, so the two could have been combined any other way and every test
that serves within the window kept passing.

The entry is fresh a second before the span runs out and stale on it, which is
what fixes the expiry at their sum. The span comes from the index rather than a
constant, since a record granting none of its own takes it from there.
The streaming path cannot total a project's bytes as it goes, so an index
carrying a resource size limit falls back to the buffered one. Every test served
a page whose index had no such limit, so the guard could have been dropped and
the limited page would have quietly streamed anyway.
A resource size limit makes a policy active, and an active policy already sends
the page down the buffered path before `streaming_parts` runs, so the guard
arm asking the same question again could never match. It read as the thing
keeping a limited index off the stream while the check above it did that work.

Line coverage could not see it: the guard and its body share a line, so
evaluating the guard marked the line covered even though the arm never took.

The check above now says why it also covers the size limit, and the test added
alongside pins that path, so a limit that stops implying an active policy fails
rather than quietly streaming a page whose bytes nothing totals.
Two properties of a delete had nothing reading them back.

A token scoped to one project must not delete another. Every fixture token held
a wildcard grant, and a wildcard passes whether or not the request says which
project it addresses, so the segment scan that finds the project could have
returned nothing and no test would have noticed. The narrow token added
alongside distinguishes them: without a project the check widens to the whole
index, which only a wildcard grant satisfies.

A delete records a reason only from a reason parameter. The trash record read
back its reason from a request that supplied one, so an unrelated parameter
could have supplied it instead.
The PyPI mutation handlers only wake the webhook scheduler when a yank,
restore or delete touched at least one file. From the HTTP surface that gate
is invisible: a skipped wake still delivers, one scheduler tick later, so the
only difference is latency and no honest test can assert on it.

Moving the gate into peryx-events puts it next to the wait it governs, where
the crate's own tests can park the scheduler on a far deadline and observe
whether a given result resumes it. Each comparison and the notify call itself
now flips a named test.
Earlier commits on this branch were formatted at rustfmt's default 100
columns; the repo pins 120, and the fmt check would reject the branch.
The coverage gate measures test files and rejects lines that only run when
an assertion fails: a format argument in an assert message, the panic arm of
a let-else that the setup never takes, and the false arm matches! leaves on
its own line. Asserting the whole value in one expression, and binding what
the message reports before asserting, removes those paths without weakening
what the tests check.
A token scoped to another project was refused, but so was any spelling of
the project name the path did not carry, so nothing showed the delete was
authorised against the segment itself. Deleting with a token granted exactly
that project is the request only the real name satisfies.

An html upstream page served as json passes through the buffered path in
one fetch. Forcing it down the json stream still answers correctly, because
the stream fails and the buffered path fetches again, so the only sign is
the second upstream request. Counting the fetches makes that visible.
@gaborbernat gaborbernat added type:test Testing, conformance, metrics, or validation coverage area:tests Conformance, perf tests, and observability labels Sep 9, 2026
@gaborbernat
gaborbernat force-pushed the test/pypi-mutants-1893 branch from b95a018 to b5baa08 Compare September 9, 2026 10:27
@codspeed-hq

codspeed-hq Bot commented Sep 9, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 29 untouched benchmarks
⏩ 133 skipped benchmarks1


Comparing test/pypi-mutants-1893 (b5baa08) with main (a7dab14)

Open in CodSpeed

Footnotes

  1. 133 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@gaborbernat
gaborbernat merged commit d29e3ae into main Sep 9, 2026
22 checks passed
@gaborbernat
gaborbernat deleted the test/pypi-mutants-1893 branch September 9, 2026 11:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:tests Conformance, perf tests, and observability type:test Testing, conformance, metrics, or validation coverage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant