Skip to content

ont-artic: repoint consistently-broken NCBI SRA read URL at ENA + test schema regularization#1298

Merged
mvdbeek merged 1 commit into
galaxyproject:mainfrom
jmchilton:schema/ont-artic-sra-url-and-tests
Jul 15, 2026
Merged

ont-artic: repoint consistently-broken NCBI SRA read URL at ENA + test schema regularization#1298
mvdbeek merged 1 commit into
galaxyproject:mainfrom
jmchilton:schema/ont-artic-sra-url-and-tests

Conversation

@jmchilton

Copy link
Copy Markdown
Member

Unblocks ont-artic-variation and carries its share of the test-schema regularization split out of #1286.

The read URL is consistently broken, not flaky

The test fetched its ONT reads from:

https://www.be-md.ncbi.nlm.nih.gov/Traces/sra-reads-be/fastq?acc=SRR12447380

be-md.ncbi.nlm.nih.gov is an NCBI backend host, not a public data endpoint. It no longer serves this accession, and it fails in the nastiest possible way: it returns HTTP 200 with an HTML page titled "NCBI - WWW Error Diagnostic" (3713 bytes) instead of a FASTQ.

Because the status code is 200, Galaxy downloads it happily, then the content fails to decompress as fastqsanger.gz and the declared SHA-1 does not match — so the upload job errors and the workflow is never invoked:

Upload job produced output [3: fastq?acc=SRR12447380] in state [error]

This has failed in every run I can find — identical accession, identical error:

run result
weekly 2026-06-29 ERROR
weekly 2026-07-06 ERROR
weekly 2026-07-13 ERROR
PR #1286 ERROR

It is not a transient SRA outage and no amount of retrying will fix it. It reads like a network flake, which is probably why it has survived this long.

The fix

Repointed at ENA, which serves the same accession from a stable, public, versioned path:

https://ftp.sra.ebi.ac.uk/vol1/fastq/SRR124/080/SRR12447380/SRR12447380_1.fastq.gz

The declared SHA-1 is re-derived from the actual file, since the old hash described whatever the dead endpoint used to serve. The download was verified before hashing:

  • md5 c68b7c19b4ee19d668a927e55c9e4d78 — matches ENA's independently published md5 for this run exactly
  • gzip integrity check passes
  • read headers confirm the accession (@SRR12447380.1 1/1)
  • new SHA-1: cd93c6dcbcf46b795d7cc2f41a81c9d87f8601d6

Test schema

Also adds the class: Collection / collection_type annotations for this workflow — same mechanical change as #1290/#1291/#1292, held back from those because this workflow could not run at all.

Expectations

planemo workflow_lint --iwc passes locally. The workflow test has never successfully executed on CI — it has been dying at upload — so this run is the first real signal. If something downstream fails (e.g. the variant-call comparison, which allows lines_diff: 6), that is a pre-existing problem this PR merely reveals, and can be handled separately.

…gularization

The be-md.ncbi.nlm.nih.gov backend host returns HTTP 200 with an HTML error
page instead of the FASTQ, so the upload job has errored in every run.
Repoint at ENA and re-derive the SHA-1 (verified against ENA's published md5).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@jmchilton
jmchilton requested a review from wm75 as a code owner July 14, 2026 22:03
@mvdbeek
mvdbeek merged commit a9f97ad into galaxyproject:main Jul 15, 2026
8 checks passed
@mvdbeek

mvdbeek commented Jul 15, 2026

Copy link
Copy Markdown
Member

Attention: deployment skipped!

https://github.com/galaxyproject/iwc/actions/runs/29392483056

@wm75

wm75 commented Jul 15, 2026

Copy link
Copy Markdown
Member

Thanks so much @jmchilton for fixing this and all your other cleanup work!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants