Skip to content

bump viral-ngs 3.0.10, remove GATK IndelRealigner deps, update lofreq pipeline#650

Merged
dpark01 merged 4 commits into
masterfrom
bump-viral-ngs
Mar 21, 2026
Merged

bump viral-ngs 3.0.10, remove GATK IndelRealigner deps, update lofreq pipeline#650
dpark01 merged 4 commits into
masterfrom
bump-viral-ngs

Conversation

@dpark01

@dpark01 dpark01 commented Mar 21, 2026

Copy link
Copy Markdown
Member

Summary

  • Bump docker images: viral-ngs 3.0.9 → 3.0.10, py3-bio 0.1.4 → 0.1.5 across all WDL task files
  • Remove deprecated --skipRealign: GATK IndelRealigner has been removed upstream from viral-ngs, so the --skipRealign flag on read_utils align_and_fix is no longer needed. Removed the flag and associated skip_realign variable from the align_reads task.
  • Update lofreq task: Since GATK IndelRealigner is no longer available, the lofreq task now handles indel normalization internally:
    • Added lofreq viterbi for Viterbi-based read realignment around indels
    • Added lofreq indelqual --dindel to insert indel qualities (required for indel calling)
    • Switched from lofreq call to lofreq call-parallel for parallelized variant calling
    • Added --call-indels flag (controlled by new call_indels Boolean input, default true)
    • Scaled disk size with input BAM size instead of flat 200 GB
    • Increased CPU (2 → 4) and memory (3 GB → 7 GB) for parallel calling
  • Bump GitHub Actions: checkout v4→v6, setup-python v5→v6, setup-java v4→v5, setup-buildx-action v3→v4 to resolve Node.js 20 deprecation warnings

Test plan

  • miniwdl check passes on all workflows
  • CI: validate_wdl_miniwdl passes
  • CI: validate_wdl_womtool passes
  • CI: test_miniwdl passes (exercises both align_reads and lofreq via assemble_refbased)
  • CI: test_cromwell passes
  • CI: test_docs passes
  • Verify no test expected output updates needed (or update if necessary)

dpark01 added 4 commits March 20, 2026 15:01
…ual/call-parallel

GATK IndelRealigner has been removed upstream, so:
- Remove --skipRealign flag from align_and_fix (no longer supported)
- Update lofreq task to run viterbi realignment and indelqual before
  variant calling, replacing the GATK-based indel realignment
- Switch lofreq call to call-parallel for better performance
- Add --call-indels flag (toggled by new call_indels input, default true)
- Scale lofreq disk size with input BAM size
…warnings

- actions/checkout v4 -> v6
- actions/setup-python v5 -> v6
- actions/setup-java v4 -> v5
- docker/setup-buildx-action v3 -> v4
@dpark01
dpark01 marked this pull request as ready for review March 21, 2026 17:21
Copilot AI review requested due to automatic review settings March 21, 2026 17:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates the pipeline to use newer viral-ngs/py3-bio container images, removes reliance on deprecated GATK IndelRealigner behavior, and refreshes the intrahost LoFreq calling steps accordingly.

Changes:

  • Bump viral-ngs from 3.0.9 → 3.0.10 and py3-bio from 0.1.4 → 0.1.5 across WDL tasks and module requirements.
  • Remove deprecated --skipRealign wiring from align_reads now that IndelRealigner is gone upstream.
  • Update the lofreq WDL task to realign/normalize indels internally and use lofreq call-parallel, with a new call_indels input.

Reviewed changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
requirements-modules.txt Bumps referenced module image versions (viral-ngs, py3-bio).
pipes/WDL/tasks/tasks_utils.wdl Updates docker tags for utility tasks to viral-ngs:3.0.10-* and py3-bio:0.1.5.
pipes/WDL/tasks/tasks_terra.wdl Updates Terra helper task docker tags to viral-ngs:3.0.10-*.
pipes/WDL/tasks/tasks_taxon_filter.wdl Updates taxon filter task docker tags to viral-ngs:3.0.10-*.
pipes/WDL/tasks/tasks_sarscov2.wdl Updates SARS-CoV-2 metadata task docker tags to py3-bio:0.1.5.
pipes/WDL/tasks/tasks_reports.wdl Updates reporting task docker tags to viral-ngs:3.0.10-* / py3-bio:0.1.5.
pipes/WDL/tasks/tasks_read_utils.wdl Updates read-utils task docker tags to viral-ngs:3.0.10-core.
pipes/WDL/tasks/tasks_nextstrain.wdl Updates Nextstrain-related task docker tags to viral-ngs:3.0.10-*.
pipes/WDL/tasks/tasks_ncbi.wdl Updates NCBI-related task docker tags to viral-ngs:3.0.10-* / py3-bio:0.1.5.
pipes/WDL/tasks/tasks_metagenomics.wdl Updates metagenomics task docker tags to viral-ngs:3.0.10-classify.
pipes/WDL/tasks/tasks_megablast.wdl Updates megablast task docker tags to viral-ngs:3.0.10-*.
pipes/WDL/tasks/tasks_intrahost.wdl Updates LoFreq task to include viterbi realignment + indel quality insertion and switch to call-parallel; bumps docker tag.
pipes/WDL/tasks/tasks_interhost.wdl Updates interhost task docker tags to viral-ngs:3.0.10-*.
pipes/WDL/tasks/tasks_demux.wdl Updates demux task docker tags to viral-ngs:3.0.10-* / py3-bio:0.1.5.
pipes/WDL/tasks/tasks_assembly.wdl Removes deprecated --skipRealign plumbing from align_reads; bumps docker tags to viral-ngs:3.0.10-*.
.github/workflows/build.yml Bumps GitHub Action versions used by CI workflows.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +175 to +181
lofreq indelqual --dindel \
-f reference.fasta \
-o indelqual.bam \
realigned.bam

# re-sort and re-index after viterbi+indelqual
samtools sort -o sorted.bam indelqual.bam

Copilot AI Mar 21, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

call_indels can be set to false, but the workflow still runs lofreq indelqual --dindel and the subsequent resort/reindex steps unconditionally. If indel calling is disabled, these steps are likely unnecessary work and extra I/O; consider guarding indelqual + re-sort/re-index behind call_indels (and otherwise call directly on the viterbi output or the original BAM).

Suggested change
lofreq indelqual --dindel \
-f reference.fasta \
-o indelqual.bam \
realigned.bam
# re-sort and re-index after viterbi+indelqual
samtools sort -o sorted.bam indelqual.bam
if ~{call_indels}; then
lofreq indelqual --dindel \
-f reference.fasta \
-o indelqual.bam \
realigned.bam
# re-sort after viterbi+indelqual
samtools sort -o sorted.bam indelqual.bam
else
# when not calling indels, skip indelqual and sort viterbi output directly
samtools sort -o sorted.bam realigned.bam
fi
# index sorted BAM for variant calling

Copilot uses AI. Check for mistakes.
realigned.bam

# re-sort and re-index after viterbi+indelqual
samtools sort -o sorted.bam indelqual.bam

Copilot AI Mar 21, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The task reserves cpu cores (default 4) but samtools sort is invoked without -@ to use multiple threads. Consider passing -@ ~{cpu} (or a capped value) so the sort step benefits from the allocated CPUs and reduces wall time.

Suggested change
samtools sort -o sorted.bam indelqual.bam
samtools sort -@ ~{cpu} -o sorted.bam indelqual.bam

Copilot uses AI. Check for mistakes.
@dpark01
dpark01 merged commit f251675 into master Mar 21, 2026
19 checks passed
@dpark01
dpark01 deleted the bump-viral-ngs branch March 21, 2026 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants