Skip to content

Mutation job reports survivors its own build excludes #2208

Description

@gaborbernat

Problem

The nightly mutation job reports survivors that no test can ever kill.

cargo mutants --workspace --all-features parses with syn and does not evaluate cfg. It therefore finds mutants inside code the job's own feature selection excludes from the build, applies them to text that never compiles, observes a green test run, and reports them as survived.

peryx-web shows the shape clearly. Its pagination helpers are gated:

#[cfg(all(target_arch = "wasm32", not(feature = "ssr"), feature = "hydrate"))]
fn previous_disabled(state: AnalyticsState) -> bool { ... }

That gate fails twice over in the job: the runner is not wasm32, and --all-features turns ssr on, so not(feature = "ssr") is false. Confirmed by counterfactual in both directions — a compile_error! inside those functions compiles clean under --all-features --all-targets, while the same injection into the ssr variant of reactive_value fails with E0433.

Of that crate's 159 reported survivors:

count group status
71 not(ssr) gated, no arch gate unbuildable in this job, killable under a hydrate-configured pass
57 wasm32 gated unbuildable on any host runner
22 compiled today genuinely untested
9 stale entries

So 137 of 159 are noise. --all-features is the direct cause: every not(feature = ...) gate in the workspace is falsified by it.

Why it matters now

The standing goal is that all mutations pass. That is unreachable while the job reports mutants which are unbuildable by construction, and the noise hides the real ones — here, 22 real gaps behind 137 that cannot be acted on.

It also wastes the run. Those mutants are built and tested on every shard.

Required change

Two parts, and the second needs a decision.

Add a second mutants pass for the hydrate build. Configured --no-default-features --features hydrate, it compiles the 71 not(ssr) functions on the host and mutates them for real. This is a job configuration change, not test writing, and it converts the largest group from noise into actionable findings.

Decide the disposition of the wasm32-gated 57. There is no wasm test suite in the repository — no wasm-bindgen-test anywhere in peryx-web, despite a wasm-coverage feature wired for minicov. Either add a wasm test target so they can be reached, or exclude them in .cargo/mutants.toml with the reason recorded. An undocumented exclusion is the wrong answer; a documented one is defensible until a wasm suite exists.

Whichever is chosen, the exclusion or the extra pass must be stated in the config rather than left for each reader to rediscover.

Acceptance criteria

  • The nightly mutation job reports no mutant that its own build excludes.
  • The 71 not(ssr) mutants are either mutated under a hydrate-configured pass or excluded with a stated reason.
  • The wasm32 group has a decided disposition recorded in .cargo/mutants.toml or a wasm target.
  • A reader can tell from the config which mutants are skipped and why.

Boundary

Job configuration and .cargo/mutants.toml only. Writing tests for the mutants this unblocks is #1893, not this issue. Do not change any cfg gate to make a mutant buildable — the gates are correct.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:ciContinuous integration and repository checksarea:testsConformance, perf tests, and observabilitypriority:P1Protocol parity or important product gaptype:bugIncorrect behavior or missing validation

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions