Schema regularization/validation fixes for tests around workflow outputs

[jmchilton]

I think a bunch of this will be red based on other structural PRs uncovering pre-existing red workflows - but I'll extract the green stuff and try to work on isolating and green-ifying the red workflows.

FOR CONTRIBUTOR:

• I have read the Adding workflows guidelines
• I have added myself to the CODEOWNERS File for the workflow
• License permits unrestricted use (educational + commercial)
• Please also take note of the reviewer guidelines below to facilitate a smooth review process.

FOR REVIEWERS:

• .dockstore.yml: file is present and aligned with creator metadata in workflow. ORCID identifiers are strongly encouraged in creator metadata. The .dockstore.yml file is required to run tests
• Workflow is sufficiently generic to be used with lab data and does not hardcode sample names, reference data and can be run without reading an accompanying tutorial.
• In workflow: annotation field contains short description of what the workflow does. Should start with This workflow does/runs/performs … xyz … to generate/analyze/etc …
• In workflow: workflow inputs and outputs have human readable names (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless it is generally understood. Altering input or output labels requires adjusting these labels in the the workflow-tests.yml file as well
• In workflow: name field should be human readable (spaces are fine, no underscore, dash only where spelling dictates it), no abbreviation unless generally understood
• Workflow folder: prefer dash (-) over underscore (_), prefer all lowercase. Folder becomes repository in iwc-workflows organization and is included in TRS id
• Readme explains what workflow does, what are valid inputs and what outputs users can expect. If a tutorial or other resources exist they can be linked. If a similar workflow exists in IWC readme should explain differences with existing workflow and when one might prefer one workflow over another
• Changelog contains appropriate entries
• Large files (> 100 KB) are uploaded to zenodo and location urls are used in test file

[mvdbeek]

Looks great, i wonder if we'll hit the 6h timeout but in principle this is fine of course

[github-actions]

Test Results (powered by Planemo)

Test Summary

Test State	Count
Total	32
Passed	26
Error	3
Failure	3
Skipped	0

Errored Tests

• ❌ Pathogen-Detection-PathoGFAIR-Samples-Aggregation-and-Visualisation.ga_0
• ❌ ont-artic-variation.ga_0
• ❌ rnaseq-sr.ga_0

Failed Tests

• ❌ Generic-variation-analysis-on-WGS-PE-data.ga_0
• ❌ baredSC-1d-logNorm.ga_0
• ❌ pe-wgs-variation.ga_0

Passed Tests

• ✅ Generic-variation-analysis-reporting.ga_0
• ✅ MAGs-binning-evaluation.ga_0
• ✅ MAGs-generation.ga_0
• ✅ MAGs-generation.ga_1
• ✅ MAGs-taxonomy-annotation.ga_0
• ✅ Nanopore-Pre-Processing.ga_0
• ✅ Taxonomy-Profiling-and-Visualization-with-Krona.ga_0
• ✅ Velocyto-on10X-filtered-barcodes.ga_0
• ✅ Velocyto-on10X-from-bundled.ga_0
• ✅ WGS-PE-variant-calling-in-haploid-system.ga_0
• ✅ baredSC-2d-logNorm.ga_0
• ✅ genotype-variant-calling-wgs-pe.ga_0
• ✅ host-or-contamination-removal-on-long-reads.ga_0
• ✅ host-or-contamination-removal-on-short-reads.ga_0
• ✅ host-or-contamination-removal-on-short-reads.ga_1
• ✅ metagenomic-raw-reads-amr-analysis.ga_0
• ✅ metaprosip.ga_0
• ✅ mfassignr.ga_0
• ✅ pe-artic-variation.ga_0
• ✅ pe-wgs-ivar-analysis.ga_0
• ✅ pox-virus-half-genome.ga_0
• ✅ rnaseq-pe.ga_0
• ✅ scrna-seq-fastq-to-matrix-10x-cellplex.ga_0
• ✅ scrna-seq-fastq-to-matrix-10x-v3.ga_0
• ✅ short-read-quality-control-and-trimming.ga_0
• ✅ tissue-micro-array-analysis.ga_0

[jmchilton]

Note

Posted by Claude (AI assistant) on behalf of @jmchilton — not authored by them personally.

Closing this in favor of smaller PRs. This one bundled 55 workflow directories, which CI structurally cannot complete: the PR test job caps at max-chunks: 4 (.github/workflows/workflow_test.yml), so ~14 full Galaxy test runs landed in each chunk. Chunk 0 was cancelled at GitHub's 6h job limit part way through hyphy, and chunk 1's job died so hard GitHub dropped its log entirely, part way through cutandrun. Neither uploaded a results artifact, so 6 workflows never got a verdict at all.

What CI did tell us

From the combined artifact plus the raw chunk logs of run 29268785765: 38 workflow directories passed, 11 failed, 6 got no result.

Every failure has a pre-existing cause unrelated to the schema change in this PR. The diff here is two mechanical patterns only — class: Collection added to outputs that already had element_tests:/elements:, and type: → collection_type: on nested collection elements in inputs. No .ga files or expected data touched. The 18-minute lint job produced exactly one ERROR across the whole repo, and it is a missing Toolshed revision, not anything in this diff.

Replacement PRs (expected green)

All workflows in these passed on this branch:

• tests: declare class:Collection / collection_type (epigenetics, data-fetching, chem, read-prep) #1290 — epigenetics, data-fetching, computational chemistry, read preprocessing (10 dirs)
• tests: declare class:Collection / collection_type (amplicon, bacterial, microbiome) #1291 — amplicon, bacterial genomics, microbiome (15 dirs)
• tests: declare class:Collection / collection_type (variant calling, omics, imaging) #1292 — variant calling, transcriptomics, scRNA-seq, imaging, proteomics, metabolomics (13 dirs)

Held back — pre-existing breakage, needs work first

Root cause	Workflows
compleasm/0.2.6+galaxy3 crashes parsing busco-data.ezlab.org's file_versions.tsv (ValueError: too many values to unpack (expected 3) in download_placement) — upstream tool bug	Assembly-Hifi-HiC-phasing-VGP4, Purge-duplicate-contigs-VGP6, Purge-duplicates-one-haplotype-VGP6b, Scaffolding-HiC-VGP8, hi-c-contact-map
Tool revision not installable: iuc/fasttree/2.1.10+galaxy1 is not in the Toolshed (this is also the sole lint error); iuc/featurecounts/2.1.1+galaxy1 fails to install	PathoGFAIR samples aggregation, rnaseq-sr
Stale expected VCFs — lofreq/SnpEff output has drifted from the checked-in files	generic-variant-calling-wgs-pe, sars-cov-2-pe-illumina-wgs-variant-calling
Flaky/too-tight asserts — SRA upload of SRR12447380 errored; baredSC PNG size assert 108407 ± 10000 got 119622	sars-cov-2-ont-artic, baredSC-1d-logNorm

Never ran

hyphy, cutandrun, hic-hicup-cooler, functional-annotation-of-sequences, histological-staining-area-quantification, multiplex-tissue-microarray-analysis. These need their own small PR to get an actual verdict; cutandrun is the prime suspect for killing chunk 1 and is worth isolating.

Suggested follow-up

Raising max-chunks above 4 for PR CI (the weekly ci.yaml already uses 40) would stop repo-wide mechanical changes from hitting this wall again.