Skip to content

Spades finished abnormally, OS return value: -9 #1588

Description

@jokopebri

Description of bug

Hi,

I encountered a problem while running SPAdes. I've attached the log and param file below.

params.txt
spades.log

Environment:

SPAdes version: 4.0.0
Python version: 3.10.14
OS: Linux-4.18.0-372.9.1.el8.x86_64-x86_64-with-glibc2.28

I'm running SPAdes on an HPC cluster using the following command:

spades.py -1 READ_QC/N/final_pure_reads_1.fastq -2 READ_QC/N/final_pure_reads_2.fastq -m 1024 -t 32 --meta -o metaspades_N

Error:

== Error == system call for: "['/mgpfs/home/joko047/.conda/envs/wgs/bin/spades-hammer', '/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N/corrected/configs/config.info']" finished abnormally, OS return value: -9

Could you help me understand what might be causing the issue?

Thank you.

spades.log

Command line: /mgpfs/home/joko047/.conda/envs/wgs/bin/spades.py -1 /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_1.fastq -2 /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_2.fastq -m 1024 -t 32 --meta -o /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N

System information:
SPAdes version: 4.0.0
Python version: 3.10.14
OS: Linux-4.18.0-372.9.1.el8.x86_64-x86_64-with-glibc2.28

Output dir: /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N
Mode: read error correction and assembling
Debug mode is turned OFF

Dataset parameters:
Metagenomic mode
Reads:
Library number: 1, library type: paired-end
orientation: fr
left reads: ['/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_1.fastq']
right reads: ['/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_2.fastq']
interlaced reads: not specified
single reads: not specified
merged reads: not specified
Read error correction parameters:
Iterations: 1
PHRED offset will be auto-detected
Corrected reads will be compressed
Assembly parameters:
k: [21, 33, 55]
Repeat resolution is enabled
Mismatch careful mode is turned OFF
MismatchCorrector will be SKIPPED
Coverage cutoff is turned OFF
Assembly graph output will use GFA v1.2 format
Other parameters:
Dir for temp files: /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N/tmp
Threads: 32
Memory limit (in Gb): 1024

======= SPAdes pipeline started. Log can be found here: /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N/spades.log

/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_1.fastq: max reads length: 151
/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_2.fastq: max reads length: 151

Reads length: 151

===== Before start started.

===== Read error correction started.

===== Read error correction started.

== Running: /mgpfs/home/joko047/.conda/envs/wgs/bin/spades-hammer /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N/corrected/configs/config.info

0:00:00.000 1M / 48M INFO General (main.cpp : 76) Starting BayesHammer, built from N/A, git revision N/A
0:00:00.019 1M / 48M INFO General (main.cpp : 77) Loading config from "/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N/corrected/configs/config.info"
0:00:00.034 1M / 48M INFO General (main.cpp : 79) Maximum # of threads to use (adjusted due to OMP capabilities): 32
0:00:00.043 1M / 48M INFO General (memory_limit.cpp : 55) Memory limit set to 1024 Gb
0:00:00.043 1M / 48M INFO General (main.cpp : 87) Trying to determine PHRED offset
0:00:00.048 1M / 48M INFO General (main.cpp : 93) Determined value is 33
0:00:00.048 1M / 48M INFO General (hammer_tools.cpp : 40) Hamming graph threshold tau=1, k=21, subkmer positions = [ 0 10 ]
0:00:00.048 1M / 48M INFO General (main.cpp : 114) Size of aux. kmer data 24 bytes
=== ITERATION 0 begins ===
0:00:00.048 1M / 48M INFO K-mer Counting (kmer_data.cpp : 284) Estimating k-mer count
0:00:00.135 513M / 542M INFO K-mer Counting (kmer_data.cpp : 289) Processing "/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_1.fastq"
0:03:46.009 513M / 546M INFO K-mer Counting (kmer_data.cpp : 298) Processed 112734664 reads
0:03:46.009 513M / 546M INFO K-mer Counting (kmer_data.cpp : 289) Processing "/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_2.fastq"
0:07:30.271 513M / 547M INFO K-mer Counting (kmer_data.cpp : 298) Processed 225469328 reads
0:07:30.271 513M / 547M INFO K-mer Counting (kmer_data.cpp : 303) Total 225469328 reads processed
0:07:31.111 513M / 547M INFO K-mer Counting (kmer_data.cpp : 306) Estimated 22224198115 distinct kmers
0:07:31.111 1M / 547M INFO K-mer Counting (kmer_data.cpp : 310) Filtering singleton k-mers
mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (86981476352 bytes, address: 0x1531066f0000, alignment: 67108864, commit: 1)
0:07:49.251 82G / 82G INFO K-mer Counting (kmer_data.cpp : 316) Processing "/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_1.fastq"
1:56:50.356 82G / 82G INFO K-mer Counting (kmer_data.cpp : 325) Processed 112734664 reads
1:56:50.356 82G / 82G INFO K-mer Counting (kmer_data.cpp : 316) Processing "/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_2.fastq"
3:44:24.786 82G / 82G INFO K-mer Counting (kmer_data.cpp : 325) Processed 225469328 reads
3:44:24.786 82G / 82G INFO K-mer Counting (kmer_data.cpp : 330) Total 225469328 reads processed
3:44:24.817 82G / 82G INFO General (kmer_index_builder.hpp : 258) Splitting kmer instances into 16 files using 32 threads. This might take a while.
3:44:24.817 82G / 82G INFO General (file_limit.hpp : 43) Open file limit set to 1024
3:44:24.817 82G / 82G INFO General (kmer_splitter.hpp : 94) Memory available for splitting buffers: 9.82218 Gb
3:44:24.817 82G / 82G INFO General (kmer_splitter.hpp : 102) Using cell size of 4194304
3:44:24.980 100G / 100G INFO K-mer Splitting (kmer_data.cpp : 98) Processing "/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_1.fastq"
3:44:57.955 100G / 112G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 9563722 reads
3:45:30.324 100G / 112G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 18880515 reads
3:46:02.692 100G / 112G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 28231959 reads
3:46:36.323 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 37807904 reads
3:47:09.465 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 47263244 reads
3:47:42.336 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 56662752 reads
3:48:15.865 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 66467865 reads
3:48:49.434 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 76260078 reads
3:49:22.839 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 86158082 reads
3:49:56.476 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 95977771 reads
3:50:30.227 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 105835816 reads
3:50:54.319 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 112734664 reads
3:50:54.319 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 98) Processing "/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_2.fastq"
3:52:33.472 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 108) Processed 141174973 reads
3:57:22.910 100G / 113G INFO K-mer Splitting (kmer_data.cpp : 113) Total 225469328 reads processed
3:57:22.960 82G / 113G INFO General (kmer_index_builder.hpp : 264) Starting k-mer counting.
4:00:28.041 82G / 223G INFO General (kmer_index_builder.hpp : 275) K-mer counting done. There are 8955612502 kmers in total.
4:00:30.354 69M / 223G INFO K-mer Index Building (kmer_index_builder.hpp : 410) Building perfect hash indices
4:07:28.289 6334M / 223G INFO K-mer Index Building (kmer_index_builder.hpp : 446) Index built. Total 8955612502 kmers, 6468330656 bytes occupied (5.77812 bits per kmer).
4:07:28.289 6334M / 223G INFO K-mer Counting (kmer_data.cpp : 355) Arranging kmers in hash map order
mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (71651295232 bytes, address: 0x1534982f0000, alignment: 67108864, commit: 1)
mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (71651295232 bytes, address: 0x1523e9400000, alignment: 67108864, commit: 1)
4:10:46.843 140G / 223G INFO General (main.cpp : 149) Clustering Hamming graph.
5:05:29.640 140G / 223G INFO General (main.cpp : 156) Extracting clusters:
5:05:29.640 140G / 223G INFO General (concurrent_dsu.cpp : 19) Connecting to root
5:05:40.887 140G / 223G INFO General (concurrent_dsu.cpp : 35) Calculating counts
mimalloc: warning: unable to allocate aligned OS memory directly, fall back to over-allocation (146033082368 bytes, address: 0x1501e7c00000, alignment: 67108864, commit: 1)

== Error == system call for: "['/mgpfs/home/joko047/.conda/envs/wgs/bin/spades-hammer', '/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N/corrected/configs/config.info']" finished abnormally, OS return value: -9
None

In case you have troubles running SPAdes, you can report an issue on our GitHub repository github.com/ablab/spades
Please provide us with params.txt and spades.log files from the output directory.

SPAdes log can be found here: /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N/spades.log

Thank you for using metaSPAdes! If you use it in your research, please cite:

Nurk, S., Meleshko, D., Korobeynikov, A. and Pevzner, P.A., 2017. metaSPAdes: a new versatile metagenomic assembler. Genome research, 27(5), pp.824-834.
doi.org/10.1101/gr.213959.116

params.txt

Command line: /mgpfs/home/joko047/.conda/envs/wgs/bin/spades.py -1 /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_1.fastq -2 /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_2.fastq -m 1024 -t 32 --meta -o /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N

System information:
SPAdes version: 4.0.0
Python version: 3.10.14
OS: Linux-4.18.0-372.9.1.el8.x86_64-x86_64-with-glibc2.28

Output dir: /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N
Mode: read error correction and assembling
Debug mode is turned OFF

Dataset parameters:
Metagenomic mode
Reads:
Library number: 1, library type: paired-end
orientation: fr
left reads: ['/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_1.fastq']
right reads: ['/mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/READ_QC/N/final_pure_reads_2.fastq']
interlaced reads: not specified
single reads: not specified
merged reads: not specified
Read error correction parameters:
Iterations: 1
PHRED offset will be auto-detected
Corrected reads will be compressed
Assembly parameters:
k: [21, 33, 55]
Repeat resolution is enabled
Mismatch careful mode is turned OFF
MismatchCorrector will be SKIPPED
Coverage cutoff is turned OFF
Assembly graph output will use GFA v1.2 format
Other parameters:
Dir for temp files: /mgpfs/home/joko047/sentarum-TNDS_shotgun_metagenome/metaspades_N/tmp
Threads: 32
Memory limit (in Gb): 1024

SPAdes version

4.0.0

Operating System

Linux-4.18.0-372.9.1.el8.x86_64-x86_64-with-glibc2.28

Python Version

3.10.14

Method of SPAdes installation

Conda

No errors reported in spades.log

  • Yes

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions