Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,14 @@
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [1.4.3] - Ancient Hippaforalkus (H3) - [2025-10-15]

Our 10th release for sanger-tol/treeval, containing bug fixes.

### Enhancements & Fixes

- Bug fix fixing mode selecting subworkflows.

## [1.4.2] - Ancient Hippaforalkus (H2) - [2025-10-08]

Our 9th release for sanger-tol/treeval, containing bug fixes.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@

## Introduction

**sanger-tol/treeval [1.4.2 - Ancient Hippaforalkus-]** is a bioinformatics best-practice analysis pipeline for the generation of data supplemental to the curation of reference quality genomes. This pipeline has been written to generate flat files compatible with [JBrowse2](https://jbrowse.org/jb2/) as well as HiC maps for use in Juicebox, PretextView and HiGlass.
**sanger-tol/treeval [1.4.3 - Ancient Hippaforalkus-]** is a bioinformatics best-practice analysis pipeline for the generation of data supplemental to the curation of reference quality genomes. This pipeline has been written to generate flat files compatible with [JBrowse2](https://jbrowse.org/jb2/) as well as HiC maps for use in Juicebox, PretextView and HiGlass.

The pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It uses Docker/Singularity containers making installation trivial and results highly reproducible. The [Nextflow DSL2](https://www.nextflow.io/docs/latest/dsl2.html) implementation of this pipeline uses one container per process which makes it much easier to maintain and update software dependencies. Where possible, these processes have been submitted to and installed from [nf-core/modules](https://github.com/nf-core/modules) in order to make them available to all nf-core pipelines, and to everyone within the Nextflow community!

Expand Down
2 changes: 1 addition & 1 deletion nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -313,7 +313,7 @@ manifest {
description = """A pipeline to generate supplemental data for genome curation."""
mainScript = 'main.nf'
nextflowVersion = '!>=24.04.4'
version = '1.4.2'
version = '1.4.3'
doi = '10.5281/zenodo.10047653'
}

Expand Down
6 changes: 3 additions & 3 deletions ro-crate-metadata.json
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
"@type": "Dataset",
"creativeWorkStatus": "Stable",
"datePublished": "2025-06-20T14:01:56+00:00",
"description": "# sanger-tol/treeval\n\n[![GitHub Actions CI Status](https://github.com/sanger-tol/treeval/actions/workflows/ci.yml/badge.svg)](https://github.com/sanger-tol/treeval/actions/workflows/ci.yml)\n[![GitHub Actions Linting Status](https://github.com/sanger-tol/treeval/actions/workflows/linting.yml/badge.svg)](https://github.com/sanger-tol/treeval/actions/workflows/linting.yml)[![Cite with Zenodo](http://img.shields.io/badge/DOI-10.5281/zenodo.XXXXXXX-1073c8?labelColor=000000)](https://doi.org/10.5281/zenodo.XXXXXXX)\n[![nf-test](https://img.shields.io/badge/unit_tests-nf--test-337ab7.svg)](https://www.nf-test.com)\n\n[![Nextflow](https://img.shields.io/badge/version-%E2%89%A524.04.4-green?style=flat&logo=nextflow&logoColor=white&color=%230DC09D&link=https%3A%2F%2Fnextflow.io)](https://www.nextflow.io/)\n[![nf-core template version](https://img.shields.io/badge/nf--core_template-3.3.1-green?style=flat&logo=nfcore&logoColor=white&color=%2324B064&link=https%3A%2F%2Fnf-co.re)](https://github.com/nf-core/tools/releases/tag/3.3.1)\n[![run with conda](http://img.shields.io/badge/run%20with-conda-3EB049?labelColor=000000&logo=anaconda)](https://docs.conda.io/en/latest/)\n[![run with docker](https://img.shields.io/badge/run%20with-docker-0db7ed?labelColor=000000&logo=docker)](https://www.docker.com/)\n[![run with singularity](https://img.shields.io/badge/run%20with-singularity-1d355c.svg?labelColor=000000)](https://sylabs.io/docs/)\n[![Launch on Seqera Platform](https://img.shields.io/badge/Launch%20%F0%9F%9A%80-Seqera%20Platform-%234256e7)](https://cloud.seqera.io/launch?pipeline=https://github.com/sanger-tol/treeval)\n\n## Introduction\n\n**sanger-tol/treeval [1.4.2 - Ancient Hippaforalkus-]** is a bioinformatics best-practice analysis pipeline for the generation of data supplemental to the curation of reference quality genomes. This pipeline has been written to generate flat files compatible with [JBrowse2](https://jbrowse.org/jb2/) as well as HiC maps for use in Juicebox, PretextView and HiGlass.\n\nThe pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It uses Docker/Singularity containers making installation trivial and results highly reproducible. The [Nextflow DSL2](https://www.nextflow.io/docs/latest/dsl2.html) implementation of this pipeline uses one container per process which makes it much easier to maintain and update software dependencies. Where possible, these processes have been submitted to and installed from [nf-core/modules](https://github.com/nf-core/modules) in order to make them available to all nf-core pipelines, and to everyone within the Nextflow community!\n\nYou can also set up and attempt to run the pipeline here: https://gitpod.io/#https://github.com/BGAcademy23/treeval-curation\nThis is a gitpod set up for BGA23 with a version of TreeVal, although for now gitpod will not run a nextflow pipeline die to issues with using singularity. We will be replacing this with an AWS instance soon.\n\nThe treeval pipeline has a sister pipeline currently named [curationpretext](https://github.com/sanger-tol/curationpretext) which acts to regenerate the pretext maps and accessory files during genomic curation in order to confirm interventions. This pipeline is sufficiently different to the treeval implementation that it is written as it's own pipeline.\n\n1. Parse input yaml ( YAML_INPUT )\n2. Generate my.genome file ( GENERATE_GENOME )\n3. Generate insilico digests of the input assembly ( INSILICO_DIGEST )\n4. Generate gene alignments with high quality data against the input assembly ( GENE_ALIGNMENT )\n5. Generate a repeat density graph ( REPEAT_DENSITY )\n6. Generate a gap track ( GAP_FINDER )\n7. Generate a map of self complementary sequence ( SELFCOMP )\n8. Generate syntenic alignments with a closely related high quality assembly ( SYNTENY )\n9. Generate a coverage track using PacBio data ( LONGREAD_COVERAGE )\n10. Generate HiC maps, pretext and higlass using HiC cram files ( HIC_MAPPING )\n11. Generate a telomere track based on input motif ( TELO_FINDER )\n12. Run Busco and convert results into bed format ( BUSCO_ANNOTATION )\n13. Ancestral Busco linkage if available for clade ( BUSCO_ANNOTATION:ANCESTRAL_GENE )\n14. Count KMERs with FastK and plot the spectra using MerquryFK ( KMER )\n15. Generate a coverge track using KMER data ( KMER_READ_COVERAGE )\n\n## Usage\n\n> [!NOTE]\n> If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how to set-up Nextflow. Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline) with `-profile test` before running the workflow on actual data.\n\nCurrently, it is advised to run the pipeline with docker or singularity as a small number of major modules do not currently have a conda env associated with them.\n\nNow, you can run the pipeline using:\n\n```bash\n# For the FULL pipeline\nnextflow run main.nf -profile singularity --input treeval.yaml --outdir {OUTDIR}\n\n# For the RAPID subset\nnextflow run main.nf -profile singularity --input treeval.yaml -entry RAPID --outdir {OUTDIR}\n```\n\nAn example treeval.yaml can be found [here](assets/local_testing/nxOscDF5033.yaml).\n\nFurther documentation about the pipeline can be found in the following files: [usage](https://pipelines.tol.sanger.ac.uk/treeval/dev/usage), [parameters](https://pipelines.tol.sanger.ac.uk/treeval/dev/parameters) and [output](https://pipelines.tol.sanger.ac.uk/treeval/dev/output).\n\n> **Warning:**\n> Please provide pipeline parameters via the CLI or Nextflow `-params-file` option. Custom config files including those\n> provided by the `-c` Nextflow option can be used to provide any configuration _**except for parameters**_;\n> see [docs](https://nf-co.re/usage/configuration#custom-configuration-files).\n\n## Credits\n\nsanger-tol/treeval has been written by Damon-Lee Pointon (@DLBPointon), Yumi Sims (@yumisims) and William Eagles (@weaglesBio).\n\nWe thank the following people for their extensive assistance in the development of this pipeline:\n\n<ul>\n <li>@gq1 - For building the infrastructure around TreeVal and helping with code review</li>\n <li>@ksenia-krasheninnikova - For help with C code implementation and YAML parsing</li>\n <li>@mcshane - For guidance on algorithms </li>\n <li>@muffato - For code reviews and code support</li>\n <li>@priyanka-surana - For help with the majority of code reviews and code support</li>\n</ul>\n\n## Contributions and Support\n\nIf you would like to contribute to this pipeline, please see the [contributing guidelines](.github/CONTRIBUTING.md).\n\n## Citations\n\nIf you use sanger-tol/treeval for your analysis, please cite it using the following doi: [10.5281/zenodo.10047653](https://doi.org/10.5281/zenodo.10047653).\n\n### Tools\n\nAn extensive list of references for the tools used by the pipeline can be found in the [`CITATIONS.md`](CITATIONS.md) file.\n\nYou can cite the `nf-core` publication as follows:\nThis pipeline uses code and infrastructure developed and maintained by the [nf-core](https://nf-co.re) community, reused here under the [MIT license](https://github.com/nf-core/tools/blob/main/LICENSE).\n\n> **The nf-core framework for community-curated bioinformatics pipelines.**\n>\n> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.\n>\n> _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).\n",
"description": "# sanger-tol/treeval\n\n[![GitHub Actions CI Status](https://github.com/sanger-tol/treeval/actions/workflows/ci.yml/badge.svg)](https://github.com/sanger-tol/treeval/actions/workflows/ci.yml)\n[![GitHub Actions Linting Status](https://github.com/sanger-tol/treeval/actions/workflows/linting.yml/badge.svg)](https://github.com/sanger-tol/treeval/actions/workflows/linting.yml)[![Cite with Zenodo](http://img.shields.io/badge/DOI-10.5281/zenodo.XXXXXXX-1073c8?labelColor=000000)](https://doi.org/10.5281/zenodo.XXXXXXX)\n[![nf-test](https://img.shields.io/badge/unit_tests-nf--test-337ab7.svg)](https://www.nf-test.com)\n\n[![Nextflow](https://img.shields.io/badge/version-%E2%89%A524.04.4-green?style=flat&logo=nextflow&logoColor=white&color=%230DC09D&link=https%3A%2F%2Fnextflow.io)](https://www.nextflow.io/)\n[![nf-core template version](https://img.shields.io/badge/nf--core_template-3.3.1-green?style=flat&logo=nfcore&logoColor=white&color=%2324B064&link=https%3A%2F%2Fnf-co.re)](https://github.com/nf-core/tools/releases/tag/3.3.1)\n[![run with conda](http://img.shields.io/badge/run%20with-conda-3EB049?labelColor=000000&logo=anaconda)](https://docs.conda.io/en/latest/)\n[![run with docker](https://img.shields.io/badge/run%20with-docker-0db7ed?labelColor=000000&logo=docker)](https://www.docker.com/)\n[![run with singularity](https://img.shields.io/badge/run%20with-singularity-1d355c.svg?labelColor=000000)](https://sylabs.io/docs/)\n[![Launch on Seqera Platform](https://img.shields.io/badge/Launch%20%F0%9F%9A%80-Seqera%20Platform-%234256e7)](https://cloud.seqera.io/launch?pipeline=https://github.com/sanger-tol/treeval)\n\n## Introduction\n\n**sanger-tol/treeval [1.4.3 - Ancient Hippaforalkus-]** is a bioinformatics best-practice analysis pipeline for the generation of data supplemental to the curation of reference quality genomes. This pipeline has been written to generate flat files compatible with [JBrowse2](https://jbrowse.org/jb2/) as well as HiC maps for use in Juicebox, PretextView and HiGlass.\n\nThe pipeline is built using [Nextflow](https://www.nextflow.io), a workflow tool to run tasks across multiple compute infrastructures in a very portable manner. It uses Docker/Singularity containers making installation trivial and results highly reproducible. The [Nextflow DSL2](https://www.nextflow.io/docs/latest/dsl2.html) implementation of this pipeline uses one container per process which makes it much easier to maintain and update software dependencies. Where possible, these processes have been submitted to and installed from [nf-core/modules](https://github.com/nf-core/modules) in order to make them available to all nf-core pipelines, and to everyone within the Nextflow community!\n\nYou can also set up and attempt to run the pipeline here: https://gitpod.io/#https://github.com/BGAcademy23/treeval-curation\nThis is a gitpod set up for BGA23 with a version of TreeVal, although for now gitpod will not run a nextflow pipeline die to issues with using singularity. We will be replacing this with an AWS instance soon.\n\nThe treeval pipeline has a sister pipeline currently named [curationpretext](https://github.com/sanger-tol/curationpretext) which acts to regenerate the pretext maps and accessory files during genomic curation in order to confirm interventions. This pipeline is sufficiently different to the treeval implementation that it is written as it's own pipeline.\n\n1. Parse input yaml ( YAML_INPUT )\n2. Generate my.genome file ( GENERATE_GENOME )\n3. Generate insilico digests of the input assembly ( INSILICO_DIGEST )\n4. Generate gene alignments with high quality data against the input assembly ( GENE_ALIGNMENT )\n5. Generate a repeat density graph ( REPEAT_DENSITY )\n6. Generate a gap track ( GAP_FINDER )\n7. Generate a map of self complementary sequence ( SELFCOMP )\n8. Generate syntenic alignments with a closely related high quality assembly ( SYNTENY )\n9. Generate a coverage track using PacBio data ( LONGREAD_COVERAGE )\n10. Generate HiC maps, pretext and higlass using HiC cram files ( HIC_MAPPING )\n11. Generate a telomere track based on input motif ( TELO_FINDER )\n12. Run Busco and convert results into bed format ( BUSCO_ANNOTATION )\n13. Ancestral Busco linkage if available for clade ( BUSCO_ANNOTATION:ANCESTRAL_GENE )\n14. Count KMERs with FastK and plot the spectra using MerquryFK ( KMER )\n15. Generate a coverge track using KMER data ( KMER_READ_COVERAGE )\n\n## Usage\n\n> [!NOTE]\n> If you are new to Nextflow and nf-core, please refer to [this page](https://nf-co.re/docs/usage/installation) on how to set-up Nextflow. Make sure to [test your setup](https://nf-co.re/docs/usage/introduction#how-to-run-a-pipeline) with `-profile test` before running the workflow on actual data.\n\nCurrently, it is advised to run the pipeline with docker or singularity as a small number of major modules do not currently have a conda env associated with them.\n\nNow, you can run the pipeline using:\n\n```bash\n# For the FULL pipeline\nnextflow run main.nf -profile singularity --input treeval.yaml --outdir {OUTDIR}\n\n# For the RAPID subset\nnextflow run main.nf -profile singularity --input treeval.yaml -entry RAPID --outdir {OUTDIR}\n```\n\nAn example treeval.yaml can be found [here](assets/local_testing/nxOscDF5033.yaml).\n\nFurther documentation about the pipeline can be found in the following files: [usage](https://pipelines.tol.sanger.ac.uk/treeval/dev/usage), [parameters](https://pipelines.tol.sanger.ac.uk/treeval/dev/parameters) and [output](https://pipelines.tol.sanger.ac.uk/treeval/dev/output).\n\n> **Warning:**\n> Please provide pipeline parameters via the CLI or Nextflow `-params-file` option. Custom config files including those\n> provided by the `-c` Nextflow option can be used to provide any configuration _**except for parameters**_;\n> see [docs](https://nf-co.re/usage/configuration#custom-configuration-files).\n\n## Credits\n\nsanger-tol/treeval has been written by Damon-Lee Pointon (@DLBPointon), Yumi Sims (@yumisims) and William Eagles (@weaglesBio).\n\nWe thank the following people for their extensive assistance in the development of this pipeline:\n\n<ul>\n <li>@gq1 - For building the infrastructure around TreeVal and helping with code review</li>\n <li>@ksenia-krasheninnikova - For help with C code implementation and YAML parsing</li>\n <li>@mcshane - For guidance on algorithms </li>\n <li>@muffato - For code reviews and code support</li>\n <li>@priyanka-surana - For help with the majority of code reviews and code support</li>\n</ul>\n\n## Contributions and Support\n\nIf you would like to contribute to this pipeline, please see the [contributing guidelines](.github/CONTRIBUTING.md).\n\n## Citations\n\nIf you use sanger-tol/treeval for your analysis, please cite it using the following doi: [10.5281/zenodo.10047653](https://doi.org/10.5281/zenodo.10047653).\n\n### Tools\n\nAn extensive list of references for the tools used by the pipeline can be found in the [`CITATIONS.md`](CITATIONS.md) file.\n\nYou can cite the `nf-core` publication as follows:\nThis pipeline uses code and infrastructure developed and maintained by the [nf-core](https://nf-co.re) community, reused here under the [MIT license](https://github.com/nf-core/tools/blob/main/LICENSE).\n\n> **The nf-core framework for community-curated bioinformatics pipelines.**\n>\n> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.\n>\n> _Nat Biotechnol._ 2020 Feb 13. doi: [10.1038/s41587-020-0439-x](https://dx.doi.org/10.1038/s41587-020-0439-x).\n",
"hasPart": [
{
"@id": "main.nf"
Expand Down Expand Up @@ -143,10 +143,10 @@
},
"url": [
"https://github.com/sanger-tol/treeval",
"https://nf-co.re/sanger-tol/treeval/1.4.2/"
"https://nf-co.re/sanger-tol/treeval/1.4.3/"
],
"version": [
"1.4.2"
"1.4.3"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion tests/default.nf.test.snap
Original file line number Diff line number Diff line change
Expand Up @@ -236,7 +236,7 @@
"windowmasker": "1.0.0"
},
"Workflow": {
"sanger-tol/treeval": "v1.4.2"
"sanger-tol/treeval": "v1.4.3"
}
},
[
Expand Down
17 changes: 14 additions & 3 deletions workflows/treeval.nf
Original file line number Diff line number Diff line change
Expand Up @@ -92,9 +92,20 @@ workflow TREEVAL {
// Determine workflow steps based on run mode
// Take processes from the mode's include list, remove any CLI excluded steps
//
include_workflow_steps = mode_include_map.containsKey(mode) ?
(mode_include_map[mode] - exclude_steps_list).unique() :
(all_steps_list - exclude_steps_list).unique()

// Extract the actual value from DataflowVariable if needed
def actual_mode = mode.toString()
if (actual_mode.contains('DataflowVariable')) {
// Extract value from DataflowVariable(value=RAPID_TOL) format
actual_mode = actual_mode.replaceAll('.*DataflowVariable\\(value=', '').replaceAll('\\).*', '')
}

// Determine which workflow steps to run based on mode
if (mode_include_map.containsKey(actual_mode)) {
include_workflow_steps = (mode_include_map[actual_mode] - exclude_steps_list).unique()
} else {
include_workflow_steps = (all_steps_list - exclude_steps_list).unique()
}

// This acts as a "double check" for the user
log.info "[Treeval: Info] PROCESSES TO RUN INCLUDE: $include_workflow_steps"
Expand Down