No : characters allowed in headers

[claumer]

Description of the bug

Hallo @muffato et al,

I've now gotten this nextflow blobtoolkit process to work repeatedly for me, but it took a bit of debugging, and I thought it would be valuable to share one observation: if your assembly includes any : characters in the header, it will spit up with an error message like the below. This was relevant for me as I'm using a tool called samba (part of the Masurca suite) to do scaffolding and gap filling, and it records the joins between contigs using : delimiters. I don't know if it's a bug that needs fixing (how common can this be?) but it seemed worth reporting/documenting.

Regards,
Chris

PS - I was also finding some confusion in certain runs around the --align flag, which seems to be basically ignored if passed as a runtime parameter. I ended up editing a config file.

Execution cancelled -- Finishing pending tasks before exit
ERROR ~ Error executing process > 'SANGERTOL_BLOBTOOLKIT:BLOBTOOLKIT:BLOBTOOLS:BLOBTOOLKIT_CREATEBLOBDIR (Celg_test_preblob)'

Caused by:
  Process `SANGERTOL_BLOBTOOLKIT:BLOBTOOLKIT:BLOBTOOLS:BLOBTOOLKIT_CREATEBLOBDIR (Celg_test_preblob)` terminated with an error exit status (1)


Command executed:

  blobtools replace \
      --bedtsvdir windowstats \
      --meta Celg_test_preblob.yaml \
      --taxdump $(dirname taxdump/taxdump.json) \
      --taxrule buscogenes \
      --busco lineage01/full_table.tsv --busco lineage02/full_table.tsv --busco lineage03/full_table.tsv --busco lineage04/full_table.tsv \
      --hits Celg_test_preblob.txt \
      --threads 6 \
      --evalue 1.0e-25 --hit-count 10 \
      Celg_test_preblob
  
  cat <<-END_VERSIONS > versions.yml
  "SANGERTOL_BLOBTOOLKIT:BLOBTOOLKIT:BLOBTOOLS:BLOBTOOLKIT_CREATEBLOBDIR":
      blobtoolkit: $(btk --version | cut -d' ' -f2 | sed 's/v//')
  END_VERSIONS

Command exit status:
  1

Command output:
  Reading all TSV files in windowstats
  Loading parsed taxdump

Command error:
  INFO:    Environment variable SINGULARITYENV_TMPDIR is set, but APPTAINERENV_TMPDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_TASK_WORKDIR is set, but APPTAINERENV_NXF_TASK_WORKDIR is preferred
  INFO:    Environment variable SINGULARITYENV_NXF_DEBUG is set, but APPTAINERENV_NXF_DEBUG is preferred
  Reading all TSV files in windowstats
  Loading parsed taxdump
  Traceback (most recent call last):
    File "/opt/conda/envs/btk_env/bin/blobtools", line 8, in <module>
      sys.exit(cli())
    File "/opt/conda/envs/btk_env/lib/python3.9/site-packages/blobtools/blobtools.py", line 105, in cli
      sys.exit(subcommand())
    File "/opt/conda/envs/btk_env/lib/python3.9/site-packages/blobtools/lib/add.py", line 203, in cli
      main(args)
    File "/opt/conda/envs/btk_env/lib/python3.9/site-packages/blobtools/lib/add.py", line 149, in main
      parsed = field["module"].parse(
    File "/opt/conda/envs/btk_env/lib/python3.9/site-packages/blobtools/lib/hits.py", line 545, in parse
      blast = parse_blast(
    File "/opt/conda/envs/btk_env/lib/python3.9/site-packages/blobtools/lib/hits.py", line 103, in parse_blast
      seq_id, rest = query.split(":")
  ValueError: too many values to unpack (expected 2)

Work dir:
  /mnt/shared/projects/nhm/claumer/PiMmS_paper_assemblies/Celg_assemblies/work/77/dceb8370d6dc030b6852ff14b17182

Container:
  /mnt/shared/projects/nhm/claumer/PiMmS_paper_assemblies/Celg_assemblies/work/singularity/docker.io-genomehubs-blobtoolkit-4.4.6.img

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`

 -- Check '.nextflow.log' file for details
ERROR ~ Pipeline failed. Please refer to troubleshooting docs: https://nf-co.re/docs/usage/troubleshooting

 -- Check '.nextflow.log' file for details

Command used and terminal output

nextflow run sanger-tol/blobtoolkit \
   -profile singularity \
   --input samplesheet.csv \
   --outdir Celg_PiMmS_blob \
   --fasta Celg_test_preblob.fasta \
   --taxon "Caenorhabditis elegans" \
   --busco_lineages nematoda_odb12 \
   --taxdump /home/claumer/scratch/private/taxdump_2024_10/ \
   --blastp /home/claumer/scratch/private/uniprot_2024_10/reference_proteomes.dmnd \
   --blastn /mnt/shared/datasets/databases/ncbi/nt.nal \
   --blastx /home/claumer/scratch/private/uniprot_2024_10/reference_proteomes.dmnd

Relevant files

No response

System information

No response

[muffato]

Hi Chris.

I'm not surprised. The internal blobtools commands are very sensitive to sequence names. Rich has had to modify the Busco and blast parsers multiple times over the years because the software themselves were changing their output.

@rjchallis : do you think this error could be addressed in blobtools ?