MultiQC error: cannot import name 'TypedDict' from 'typing'

[VerenaPasserini]

Hi,

I am using bcbio since a few years and it always worked nicely but last week after upgrading the tool I run a variant calling pipeline and got an error when running multiqc. I don't think it's the upgrade the problem but it might be that now multiqc requires python >= 3.8?!?

I paste below the error I got, but in the meantime I was wondering if I can run an alignment/variant calling pipeline skipping the multiqc step? I tried with tools_off: multiqc but it did not work.

Thanks,

Verena

Version info

• bcbio version (bcbio_nextgen.py --version): 1.2.9
• OS name and version (lsb_release -ds): Scientific Linux 7.3 (Nitrogen)

To Reproduce
Exact bcbio command you have used:

sbatch bcbio_nextgen.py ../config/alignment_150124.yaml -n 16

Your yaml configuration file:

fc_name: bcbio_variants
upload:
  dir: final
details:
 - analysis: variant2
   description: DLBCL_843
   files: [/home/fastq/843_R1.fastq.gz, /home/fastq/843_R2.fastq.gz]
   genome_build: hg19
   algorithm:
     aligner: bwa
     mark_duplicates: true
     remove_lcr: true
     variantcaller: false
     trim_reads: atropos
     tools_off: multiqc

Log files (could be found in work/log)
Log files are attached, here's the error:

[2024-02-01T14:40Z] multiprocessing: qsignature_summary
[2024-02-01T14:40Z] summarize metadata
[2024-02-01T14:40Z] multiprocessing: multiqc_summary
[2024-02-01T14:40Z] summarize target information
[2024-02-01T14:40Z] summarize fastqc
[2024-02-01T14:40Z] summarize metrics
[2024-02-01T14:40Z] Run multiqc
[2024-02-01T14:40Z] Traceback (most recent call last):
[2024-02-01T14:40Z]   File "/home/passerini/resources/bcbio-1.2.9/anaconda/bin/multiqc", line 6, in <module>
[2024-02-01T14:40Z]     from multiqc.__main__ import run_multiqc
[2024-02-01T14:40Z]   File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/multiqc/__init__.py", line 16, in <module>
[2024-02-01T14:40Z]     from .multiqc import run
[2024-02-01T14:40Z]   File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/multiqc/multiqc.py", line 29, in <module>
[2024-02-01T14:40Z]     from .modules.base_module import ModuleNoSamplesFound
[2024-02-01T14:40Z]   File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/multiqc/modules/base_module.py", line 14, in <module>
[2024-02-01T14:40Z]     import markdown
[2024-02-01T14:40Z]   File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/markdown/__init__.py", line 42, in <module>
[2024-02-01T14:40Z]     from .core import Markdown, markdown, markdownFromFile
[2024-02-01T14:40Z]   File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/markdown/core.py", line 27, in <module>
[2024-02-01T14:40Z]     from . import util
[2024-02-01T14:40Z]   File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/markdown/util.py", line 32, in <module>
[2024-02-01T14:40Z]     from typing import TYPE_CHECKING, Generic, Iterator, NamedTuple, TypeVar, TypedDict, overload
[2024-02-01T14:40Z] ImportError: cannot import name 'TypedDict' from 'typing' (/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/typing.py)
[2024-02-01T14:40Z] Uncaught exception occurred
Traceback (most recent call last):
  File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/bcbio/provenance/do.py", line 26, in run
    _do_run(cmd, checks, log_stdout, env=env)
  File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/bcbio/provenance/do.py", line 106, in _do_run
    raise subprocess.CalledProcessError(exitcode, error_msg)
subprocess.CalledProcessError: Command 'set -o pipefail; export PATH=/home/passerini/resources/bcbio-1.2.9/anaconda/bin:"$PATH" && export LC_ALL=en_US.utf8 && export LANG=en_US.utf8 &&  /home/passerini/resources/bcbio-1.2.9/anaconda/bin/multiqc -c /home/passerini/inMIND/alignment/alignment_150124/work/qc/multiqc/multiqc_config.yaml -f -l /home/passerini/inMIND/alignment/alignment_150124/work/qc/multiqc/list_files.txt  -o /home/passerini/inMIND/alignment/alignment_150124/work/bcbiotx/tmpfc4e7m16
Traceback (most recent call last):
  File "/home/passerini/resources/bcbio-1.2.9/anaconda/bin/multiqc", line 6, in <module>
    from multiqc.__main__ import run_multiqc
  File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/multiqc/__init__.py", line 16, in <module>
    from .multiqc import run
  File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/multiqc/multiqc.py", line 29, in <module>
    from .modules.base_module import ModuleNoSamplesFound
  File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/multiqc/modules/base_module.py", line 14, in <module>
    import markdown
  File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/markdown/__init__.py", line 42, in <module>
    from .core import Markdown, markdown, markdownFromFile
  File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/markdown/core.py", line 27, in <module>
    from . import util
  File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/markdown/util.py", line 32, in <module>
    from typing import TYPE_CHECKING, Generic, Iterator, NamedTuple, TypeVar, TypedDict, overload
ImportError: cannot import name 'TypedDict' from 'typing' (/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/typing.py)
' returned non-zero exit status 1.

bcbio-nextgen-commands.log
bcbio-nextgen-debug.log

[lpantano]

sorry, I come late to this. I am starting to help bcbio community after joining the Harvard Chan Bioinformatics Core.

Can you let me know what MultiQC version you have installed now? I was wondering that you could try to downgrade the MultiQC version to the previous one.
Did you find out how to skip multiqc?

[VerenaPasserini]

Dear Laura,
thanks for following up on this.
I did not manage to skip multiqc unfortunately, I tried to turn it off but I still get the same error.
MultiQC got installed together with bcbio 1.2.9 installation, but I cannot even check which version is installed because when I run multiqc --version I get:

Traceback (most recent call last): File "/home/passerini/resources/bcbio-1.2.9/anaconda/bin/multiqc", line 6, in <module> from multiqc.__main__ import run_multiqc File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/multiqc/__init__.py", line 16, in <module> from .multiqc import run File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/multiqc/multiqc.py", line 29, in <module> from .modules.base_module import ModuleNoSamplesFound File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/multiqc/modules/base_module.py", line 14, in <module> import markdown File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/markdown/__init__.py", line 42, in <module> from .core import Markdown, markdown, markdownFromFile File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/markdown/core.py", line 27, in <module> from . import util File "/home/passerini/resources/bcbio-1.2.9/anaconda/lib/python3.7/site-packages/markdown/util.py", line 32, in <module> from typing import TYPE_CHECKING, Generic, Iterator, NamedTuple, TypeVar, TypedDict, overload

Thanks!
Verena