You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hephaestus already has a PIT mutation profile, introduced in #1317, for six security-sensitive server classes. The immediate task is to make that existing suite trustworthy and decide whether its value justifies ongoing ownership—not to add Stryker or a second mutation framework.
A local run on 2026-08-26 produced:
Status
Count
Generated mutants
203
Killed
137
Survived
18
No coverage
40
Run error
8
PIT analysis took about 1m29s and the complete Maven command about 3m01s. All eight run errors were in EncryptionContext, yet Maven returned BUILD SUCCESS. Survivors affected behavior in ServerUrlValidator, ReturnToValidator, AdminBootstrapPolicy and IssuerDiscoveryProbe.
That silent-success mode is more urgent than a mutation-score threshold.
Eliminate RUN_ERROR, TIMED_OUT and MEMORY_ERROR, and fail the wrapper/workflow when PIT reports a technical analysis failure even if Maven exits zero.
Triage the current 18 survivors and 40 no-coverage mutations.
Add tests where a mutant exposes genuinely unverified behavior.
Document narrowly justified equivalent or unproductive mutants instead of adding implementation-coupled tests merely to improve a percentage.
Expose a reproducible manual, non-required workflow with an actionable summary and HTML/XML artifacts.
Do not use PIT's experimental incremental analysis for this initial 203-mutant scope. Do not promote mutation testing to required PR CI in this issue.
Evaluation contract
Name an owner for mutant triage and agree a runner-time budget before starting. Record:
setup/compilation time;
mutation-analysis time;
human triage time;
actionable survivors;
equivalent/unproductive mutants;
tests improved.
Done when
Technical analysis errors cannot report a successful mutation run.
Existing survivors and uncovered mutations are triaged.
Reports and summaries are reproducible from a manual non-required workflow.
A decision is recorded: drop, keep manual/advisory, run periodically, or open a separate required-gate proposal.
This evaluation issue is closed after that decision.
Mutation testing cannot detect omitted implementations, prevent test/config tampering, or replace integration, E2E and independent review. A score is not the goal; useful fault detection at acceptable compute and attention cost is.
Hephaestus already has a PIT mutation profile, introduced in #1317, for six security-sensitive server classes. The immediate task is to make that existing suite trustworthy and decide whether its value justifies ongoing ownership—not to add Stryker or a second mutation framework.
A local run on 2026-08-26 produced:
PIT analysis took about 1m29s and the complete Maven command about 3m01s. All eight run errors were in
EncryptionContext, yet Maven returnedBUILD SUCCESS. Survivors affected behavior inServerUrlValidator,ReturnToValidator,AdminBootstrapPolicyandIssuerDiscoveryProbe.That silent-success mode is more urgent than a mutation-score threshold.
Scope
RUN_ERROR,TIMED_OUTandMEMORY_ERROR, and fail the wrapper/workflow when PIT reports a technical analysis failure even if Maven exits zero.Do not use PIT's experimental incremental analysis for this initial 203-mutant scope. Do not promote mutation testing to required PR CI in this issue.
Evaluation contract
Name an owner for mutant triage and agree a runner-time budget before starting. Record:
Done when
Mutation testing cannot detect omitted implementations, prevent test/config tampering, or replace integration, E2E and independent review. A score is not the goal; useful fault detection at acceptable compute and attention cost is.
References: