Stale expected VCFs: generic-variant-calling-wgs-pe and sars-cov-2-pe-illumina-wgs failing all weeklies on cosmetic lofreq drift

[jmchilton]

Note

Posted by Claude (AI assistant) on behalf of @jmchilton — not authored by them personally.

Summary

Two variant-calling workflows have been failing in every weekly run because their checked-in expected VCFs have gone stale. The drift is entirely cosmetic — lofreq changed how it formats allele-frequency floats, and newer lofreq/bcftools emit extra VCF header lines. No variant calls have changed: zero gained, zero lost.

Affected:

• workflows/variant-calling/generic-variant-calling-wgs-pe (Generic-variation-analysis-on-WGS-PE-data) — 3 failing outputs
• workflows/sars-cov-2-variant-calling/sars-cov-2-pe-illumina-wgs-variant-calling (pe-wgs-variation) — 1 failing output

Consistently failing in the weeklies

weekly run	Generic-variation-analysis-on-WGS-PE-data	pe-wgs-variation
2026-06-29	FAILURE (3 outputs)	FAILURE (1 output)
2026-07-06	FAILURE (3 outputs)	FAILURE (1 output)
2026-07-13	FAILURE (3 outputs)	FAILURE (1 output)

Also fails identically in PR #1286. Deterministic, not flaky.

Root cause 1: lofreq AF float formatting

Pairing every changed data line by position, the variants are identical — same position, same REF/ALT, same QUAL, same DP, DP4, SB, INDEL, HRUN. Only the AF= string differs, with trailing zeros now stripped:

expected	current output
AF=1.000000	AF=1
AF=0.750000	AF=0.75
AF=0.800000	AF=0.8
AF=0.500000	AF=0.5
AF=0.093750	AF=0.09375
AF=0.118110	AF=0.11811

In Generic-variation-analysis-on-WGS-PE-data, the diff superficially looks like 12 variants vanishing and 12 appearing — but they are at the same 12 positions (189951–191034), differing only in AF formatting. Nothing is lost.

Root cause 2: new VCF header lines

Newer lofreq/bcftools emit header lines that did not exist when these files were baselined:

##contig=<ID=NC_045512.2>
##FILTER=<ID=PASS,Description="All filters passed">
##bcftools_concatVersion=1.21+htslib-1.21
##bcftools_concatCommand=concat -a -O z -o /tmp/.../concat.vcf.gz ... ; Date=Mon Jul 13 01:10:45 2026

plus additional dynamic min_snvqual_* / min_indelqual_* FILTER thresholds.

Why the tests fail

Both use compare: diff with lines_diff: 6. That budget was presumably sized to absorb the two inherently volatile header lines — ##fileDate (changes daily) and ##source (embeds a job tmp path). The new header lines plus the reformatted AF values push the diff well past 6.

Proposed fix

Re-baseline the expected VCFs. This is safe: the outputs are scientifically identical, so no real regression is being masked.

Two wrinkles worth deciding on:

1. ##bcftools_concatCommand embeds both a tmp path and a timestamp, so it is volatile on every run — same as ##fileDate and ##source. After a re-baseline there will still be ~3 permanently-differing header lines, right at the lines_diff: 6 ceiling.
2. Given that, it may be better to stop comparing VCF headers at all — switch these outputs to header-insensitive assertions (or strip the header before comparison) so this stops recurring every time lofreq or bcftools changes its provenance block. Otherwise this will silently rot again on the next tool bump.

Note on scope

The class: Collection / collection_type test-schema regularization for these two workflows (split out of #1286) is held back pending this fix — see #1290, #1291, #1292 for the equivalent change on the workflows that are green.

Caveat

Planemo truncates long diffs (*SNIP*), so the above characterizes every line the report showed, but not literally every line in the files. Regenerating and diffing the full outputs locally would confirm there is nothing else hiding in the truncated region.