bump viral-ngs 3.0.10, remove GATK IndelRealigner deps, update lofreq pipeline#650
Conversation
…ual/call-parallel GATK IndelRealigner has been removed upstream, so: - Remove --skipRealign flag from align_and_fix (no longer supported) - Update lofreq task to run viterbi realignment and indelqual before variant calling, replacing the GATK-based indel realignment - Switch lofreq call to call-parallel for better performance - Add --call-indels flag (toggled by new call_indels input, default true) - Scale lofreq disk size with input BAM size
…warnings - actions/checkout v4 -> v6 - actions/setup-python v5 -> v6 - actions/setup-java v4 -> v5 - docker/setup-buildx-action v3 -> v4
There was a problem hiding this comment.
Pull request overview
Updates the pipeline to use newer viral-ngs/py3-bio container images, removes reliance on deprecated GATK IndelRealigner behavior, and refreshes the intrahost LoFreq calling steps accordingly.
Changes:
- Bump
viral-ngsfrom 3.0.9 → 3.0.10 andpy3-biofrom 0.1.4 → 0.1.5 across WDL tasks and module requirements. - Remove deprecated
--skipRealignwiring fromalign_readsnow that IndelRealigner is gone upstream. - Update the
lofreqWDL task to realign/normalize indels internally and uselofreq call-parallel, with a newcall_indelsinput.
Reviewed changes
Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| requirements-modules.txt | Bumps referenced module image versions (viral-ngs, py3-bio). |
| pipes/WDL/tasks/tasks_utils.wdl | Updates docker tags for utility tasks to viral-ngs:3.0.10-* and py3-bio:0.1.5. |
| pipes/WDL/tasks/tasks_terra.wdl | Updates Terra helper task docker tags to viral-ngs:3.0.10-*. |
| pipes/WDL/tasks/tasks_taxon_filter.wdl | Updates taxon filter task docker tags to viral-ngs:3.0.10-*. |
| pipes/WDL/tasks/tasks_sarscov2.wdl | Updates SARS-CoV-2 metadata task docker tags to py3-bio:0.1.5. |
| pipes/WDL/tasks/tasks_reports.wdl | Updates reporting task docker tags to viral-ngs:3.0.10-* / py3-bio:0.1.5. |
| pipes/WDL/tasks/tasks_read_utils.wdl | Updates read-utils task docker tags to viral-ngs:3.0.10-core. |
| pipes/WDL/tasks/tasks_nextstrain.wdl | Updates Nextstrain-related task docker tags to viral-ngs:3.0.10-*. |
| pipes/WDL/tasks/tasks_ncbi.wdl | Updates NCBI-related task docker tags to viral-ngs:3.0.10-* / py3-bio:0.1.5. |
| pipes/WDL/tasks/tasks_metagenomics.wdl | Updates metagenomics task docker tags to viral-ngs:3.0.10-classify. |
| pipes/WDL/tasks/tasks_megablast.wdl | Updates megablast task docker tags to viral-ngs:3.0.10-*. |
| pipes/WDL/tasks/tasks_intrahost.wdl | Updates LoFreq task to include viterbi realignment + indel quality insertion and switch to call-parallel; bumps docker tag. |
| pipes/WDL/tasks/tasks_interhost.wdl | Updates interhost task docker tags to viral-ngs:3.0.10-*. |
| pipes/WDL/tasks/tasks_demux.wdl | Updates demux task docker tags to viral-ngs:3.0.10-* / py3-bio:0.1.5. |
| pipes/WDL/tasks/tasks_assembly.wdl | Removes deprecated --skipRealign plumbing from align_reads; bumps docker tags to viral-ngs:3.0.10-*. |
| .github/workflows/build.yml | Bumps GitHub Action versions used by CI workflows. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| lofreq indelqual --dindel \ | ||
| -f reference.fasta \ | ||
| -o indelqual.bam \ | ||
| realigned.bam | ||
|
|
||
| # re-sort and re-index after viterbi+indelqual | ||
| samtools sort -o sorted.bam indelqual.bam |
There was a problem hiding this comment.
call_indels can be set to false, but the workflow still runs lofreq indelqual --dindel and the subsequent resort/reindex steps unconditionally. If indel calling is disabled, these steps are likely unnecessary work and extra I/O; consider guarding indelqual + re-sort/re-index behind call_indels (and otherwise call directly on the viterbi output or the original BAM).
| lofreq indelqual --dindel \ | |
| -f reference.fasta \ | |
| -o indelqual.bam \ | |
| realigned.bam | |
| # re-sort and re-index after viterbi+indelqual | |
| samtools sort -o sorted.bam indelqual.bam | |
| if ~{call_indels}; then | |
| lofreq indelqual --dindel \ | |
| -f reference.fasta \ | |
| -o indelqual.bam \ | |
| realigned.bam | |
| # re-sort after viterbi+indelqual | |
| samtools sort -o sorted.bam indelqual.bam | |
| else | |
| # when not calling indels, skip indelqual and sort viterbi output directly | |
| samtools sort -o sorted.bam realigned.bam | |
| fi | |
| # index sorted BAM for variant calling |
| realigned.bam | ||
|
|
||
| # re-sort and re-index after viterbi+indelqual | ||
| samtools sort -o sorted.bam indelqual.bam |
There was a problem hiding this comment.
The task reserves cpu cores (default 4) but samtools sort is invoked without -@ to use multiple threads. Consider passing -@ ~{cpu} (or a capped value) so the sort step benefits from the allocated CPUs and reduces wall time.
| samtools sort -o sorted.bam indelqual.bam | |
| samtools sort -@ ~{cpu} -o sorted.bam indelqual.bam |
Summary
--skipRealign: GATK IndelRealigner has been removed upstream from viral-ngs, so the--skipRealignflag onread_utils align_and_fixis no longer needed. Removed the flag and associatedskip_realignvariable from thealign_readstask.lofreqtask: Since GATK IndelRealigner is no longer available, the lofreq task now handles indel normalization internally:lofreq viterbifor Viterbi-based read realignment around indelslofreq indelqual --dindelto insert indel qualities (required for indel calling)lofreq calltolofreq call-parallelfor parallelized variant calling--call-indelsflag (controlled by newcall_indelsBoolean input, defaulttrue)Test plan
miniwdl checkpasses on all workflowsvalidate_wdl_miniwdlpassesvalidate_wdl_womtoolpassestest_miniwdlpasses (exercises bothalign_readsandlofreqviaassemble_refbased)test_cromwellpassestest_docspasses