NFCORE_AMPLISEQ:AMPLISEQ:DADA2_ERR fail

[stephschoeman]

I have both COI and 16s data. I ran Nextflow on COI and got some questionable results, but the pipeline worked. When running it on 16S, it seems to get stuck on the NFCORE_AMPLISEQ:AMPLISEQ:DADA2_ERR step for 8 hours, then it terminates.

Why would this happen? After filtering, I have roughly 14 million reads across 10 samples.

Code:
module load software/nextflow-25.10.2
nextflow run nf-core/ampliseq
-profile singularity
--input_folder "/16s1/"
--extension "/*_R{1,2}_001.fastq.gz"
--FW_primer CCTACGGGNGGCWGCAG
--RV_primer GACTACHVGGGTATCTAATCC
--trunclenf 280
--trunclenr 220
--dada_ref_taxonomy silva=138.2
--outdir "/results_1"

Error:

Timeline:

Thanks in advance!

[d4straub]

Hi there,
14 million reads are a lot. Potentially the time is indeed needed. 1e8 bases are used by default to produce an error profile, see

ampliseq/conf/modules.config

Lines 188 to 194 in 2723d4c

	ext.args = [
	'nbases = 1e8, nreads = NULL, randomize = TRUE, MAX_CONSIST = 10, OMEGA_C = 0',
	"qualityType = \"${params.quality_type}\"",
	params.binned_quality ? "errorEstimationFunction = binnedQualErrfun" :
	params.pacbio ? "errorEstimationFunction = PacBioErrfun" :
	"errorEstimationFunction = loessErrfun"
	].join(',').replaceAll('(,)*$', "")

this could be reduced with minimal expected impact on results but potentially large impact on runtime.
Example in a config file used with "-c dada2_err.config" that contains (nbases reduced to 1e5):

process {
    withName: DADA2_ERR {
        ext.seed = "${params.seed}"
        ext.prefix = { meta.region ? "region-${meta.region}_run-${meta.run}" : "${meta.run}" }
        ext.cmd_errfun = { params.binned_quality ? "binnedQualErrfun <- makeBinnedQualErrfun(c(${params.binned_quality}))" : "" }
        ext.args = [
            'nbases = 1e5, nreads = NULL, randomize = TRUE, MAX_CONSIST = 10, OMEGA_C = 0',
            "qualityType = \"${params.quality_type}\"",
            params.binned_quality ? "errorEstimationFunction = binnedQualErrfun" :
                params.pacbio ? "errorEstimationFunction = PacBioErrfun" :
                    "errorEstimationFunction = loessErrfun"
        ].join(',').replaceAll('(,)*$', "")
}

The alternative is to allow DADA2_ERR more time via a config, e.g.

process {
    withName: DADA2_ERR {
        time   = 24.h
}