Summary
npm run test:frontend:coverage gates on --test-coverage-lines=78. That line number is not reproducible on CI: two runs over byte-identical sources produced 81.82% and 76.47%, and the second is below the floor. Branch and function coverage were identical in both.
This means the line floor can redden a PR that changed nothing relevant. It already did: PR #1887 (a release version bump plus docs, zero source files touched) failed with
Error: 76.47% line coverage does not meet threshold of 78%.
while main's own run of the same sources reported 81.82% and did not.
Evidence
Two CI runs, same TypeScript sources (the PR differed only in package.json versions, docs, and lockfiles):
Both executed identically: 5935 tests, 1297 suites, 5934 pass, 0 fail, 1 skipped, 0 todo.
Comparing the two per-file tables by full path:
- 114 of 444 files report different line coverage
- 0 files report different function coverage
- 0 files report different branch coverage
- the counted file set is identical (no file entered or left the report)
The swings are large, not rounding noise:
| file |
run A |
run B |
packages/core/src/storymap-sample.ts |
99.21% |
34.65% |
packages/map/src/qml-import.ts |
97.53% |
48.48% |
packages/map/src/qml-export.ts |
94.38% |
44.52% |
packages/core/src/visibility.ts |
93.42% |
46.05% |
packages/map/src/sld-export.ts |
97.70% |
54.75% |
packages/core/src/layer-ref-scrub.ts |
97.96% |
57.14% |
Affected files cluster in packages/plugins (46), packages/core (37), packages/map (16), apps/geolibre-desktop (9). Some are referenced by many test files (visibility.ts by 33); others (storymap-sample.ts, layer-ref-scrub.ts) by none directly, reaching the report only transitively through a package index.
Not reproducible locally
Two consecutive local runs on one tree differed in 1 file (packages/core/src/store.ts, 75.35% vs 76.36%), totals 82.84% vs 82.86%. So this looks specific to the CI runner, plausibly node --test's worker-process count and the merging of per-process V8 coverage under a 4-core, memory-constrained runner. That functions and branches are perfectly stable while lines are not suggests the line ranges, rather than the coverage data as a whole, are what get lost or partially merged.
Why it matters
Per CLAUDE.md the floors are a ratchet meant to catch real regressions. A gate that swings 5+ points run to run on identical code cannot do that: it produces failures nobody can act on, and it trains people to re-run CI until it passes, which is exactly how a genuine regression slips through.
Options
- Lower the line floor to sit under the observed trough (the swing bottomed at 76.47%, so ~74) and keep functions and branches where they are, since those are stable.
- Drop the line gate and ratchet on functions and branches only, which are reproducible run to run.
- Pin
node --test concurrency in CI (--test-concurrency=1 or a small fixed number) and re-measure. Slower, but if it makes lines deterministic the floor can stay meaningful.
- Report upstream to Node if a minimal reproduction can be extracted.
I lean toward 3 for a measurement, then 1 or 2 depending on what it shows.
Context
Found while diagnosing the separate, real coverage regression fixed in #1888. That one was deterministic (function coverage, 72.90% to 60.36%) and is unrelated to this flake.
Summary
npm run test:frontend:coveragegates on--test-coverage-lines=78. That line number is not reproducible on CI: two runs over byte-identical sources produced 81.82% and 76.47%, and the second is below the floor. Branch and function coverage were identical in both.This means the line floor can redden a PR that changed nothing relevant. It already did: PR #1887 (a release version bump plus docs, zero source files touched) failed with
while
main's own run of the same sources reported 81.82% and did not.Evidence
Two CI runs, same TypeScript sources (the PR differed only in
package.jsonversions, docs, and lockfiles):mainpush (run 31764040051)Both executed identically: 5935 tests, 1297 suites, 5934 pass, 0 fail, 1 skipped, 0 todo.
Comparing the two per-file tables by full path:
The swings are large, not rounding noise:
packages/core/src/storymap-sample.tspackages/map/src/qml-import.tspackages/map/src/qml-export.tspackages/core/src/visibility.tspackages/map/src/sld-export.tspackages/core/src/layer-ref-scrub.tsAffected files cluster in
packages/plugins(46),packages/core(37),packages/map(16),apps/geolibre-desktop(9). Some are referenced by many test files (visibility.tsby 33); others (storymap-sample.ts,layer-ref-scrub.ts) by none directly, reaching the report only transitively through a package index.Not reproducible locally
Two consecutive local runs on one tree differed in 1 file (
packages/core/src/store.ts, 75.35% vs 76.36%), totals 82.84% vs 82.86%. So this looks specific to the CI runner, plausiblynode --test's worker-process count and the merging of per-process V8 coverage under a 4-core, memory-constrained runner. That functions and branches are perfectly stable while lines are not suggests the line ranges, rather than the coverage data as a whole, are what get lost or partially merged.Why it matters
Per
CLAUDE.mdthe floors are a ratchet meant to catch real regressions. A gate that swings 5+ points run to run on identical code cannot do that: it produces failures nobody can act on, and it trains people to re-run CI until it passes, which is exactly how a genuine regression slips through.Options
node --testconcurrency in CI (--test-concurrency=1or a small fixed number) and re-measure. Slower, but if it makes lines deterministic the floor can stay meaningful.I lean toward 3 for a measurement, then 1 or 2 depending on what it shows.
Context
Found while diagnosing the separate, real coverage regression fixed in #1888. That one was deterministic (function coverage, 72.90% to 60.36%) and is unrelated to this flake.