Skip to content

Commit 4bb3efb

Browse files
authored
Merge pull request #346 from galaxyproject/nextflow-eager-scenario
Add nf-core/eager fixture + compare-and-contrast scenario against iwc#1234
2 parents 194338f + b746411 commit 4bb3efb

2 files changed

Lines changed: 125 additions & 0 deletions

File tree

content/pipelines/nextflow-to-galaxy/scenarios.md

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,115 @@ fixtures with `make fixtures-nextflow`.
2525
per step (discover or author); the partial scope is stated explicitly rather
2626
than silently dropping the remaining pipeline.
2727

28+
## Case: nf-core/eager vs. a human-authored aDNA workflow (compare and contrast)
29+
30+
A **reference case**, not a matching test. galaxyproject/iwc#1234 ("Add
31+
nf-core/eager style ancient DNA (aDNA) analysis workflow", human-authored by Mert
32+
Aydın) targets the same source pipeline this journey converts, so it gives us a
33+
hand-built answer to hold the machine's answer against. The journey will never
34+
reproduce it, and shouldn't try — the value is in *where* and *why* the two
35+
diverge. Nothing here is a pass/fail gate; the pipeline oracle in `eval.md` still
36+
supplies the actual gates.
37+
38+
Three caveats to keep the comparison honest:
39+
40+
- **Version boundary.** The PR's README cites `nf-co.re/eager/2.5.3/` — eager
41+
2.x, which is DSL1 and therefore out of scope for [[summarize-nextflow]]. The
42+
fixture pins the DSL2 rewrite on `dev` (3.x). The aDNA tool vocabulary is
43+
substantially the same across the two, so the contrast is meaningful, but this
44+
is not a controlled A/B and per-step differences may be version artifacts
45+
rather than authoring differences.
46+
- **Not in the corpus.** The PR is unmerged, so `paleogenomics/adna-analysis`
47+
does not exist at the pinned IWC SHA. [[compare-against-iwc-exemplar]] cannot
48+
reach it and will settle on some other exemplar; that phase's output is not the
49+
comparison. Supply the PR out-of-band by URL when doing the write-up, per the
50+
URL-not-mirror principle — do not mirror the `.ga` into the repo.
51+
- **Human ≠ correct.** The PR is open and unreviewed. Where the journey diverges,
52+
either side may be the better answer.
53+
54+
- fixture: `workflow-fixtures/pipelines/nf-core__eager` (DSL2 `dev` pin; large —
55+
~110 `.nf` files, ~45 nf-core modules, ~23 local subworkflows).
56+
- reference: <https://github.com/galaxyproject/iwc/pull/1234>, workflow
57+
`paleogenomics/adna-analysis`.
58+
- expect (gated by `eval.md`): eager is a toolkit, not a linear pipeline — its
59+
schema exposes ~15 independent `run_*`/`skip_*` branch groups (preprocessing,
60+
mapping, bam_filtering, deduplication, damage_manipulation,
61+
adna_damage_analysis, genotyping, metagenomics, contamination_estimation,
62+
human_sex_determination, consensus_sequence, host_removal, …). The journey must
63+
reach a **stated scope decision** about which branches it carries, and say so
64+
explicitly rather than silently flattening the toolkit into whichever path the
65+
template happened to take.
66+
67+
A scope decision that freezes *every* branch into one hardcoded path satisfies
68+
the letter of that and misses the point: eager's optionality is much of what
69+
eager is, and a conversion that discards all of it has answered an easier
70+
question than the one asked. So the bar is that some meaningful part of the
71+
source's branch surface reaches the user as **run-time choice** — a workflow an
72+
analyst can steer without editing it. Which choices, how many, and by what
73+
mechanism are the journey's call, and a defensible small set beats a large one.
74+
The point is that the collapse to a single frozen path should have to be argued
75+
for, not fallen into.
76+
77+
Deliberately not prescribed: the reference keeps four user-facing controls and
78+
spends 12 `when` steps doing it, but that is **calibration, not a quota**
79+
evidence that this much optionality is reachable in gxformat2 at all. A journey
80+
that keeps three different controls, or reaches the same coverage through
81+
optional inputs rather than `when` gating, passes. A journey that pads its count
82+
with toggles eager's schema doesn't motivate fails on intent-fidelity — the
83+
optionality has to be earned by the source, not manufactured to hit a number.
84+
- contrast (advisory; record, don't gate):
85+
- **Which branches survive as branches.** The PR is conditional-rich by IWC
86+
standards: 52 steps, of which 12 carry a `when`. It keeps four user-facing
87+
branch controls — `Choose Read Type` (SE/PE), `Choose Mapper` (BWA/Bowtie2),
88+
and two optional-input conditionals (HapMap present → ANGSD X-contamination;
89+
BED present → Sex.DetERRmine variant) — while hardcoding one pick for
90+
preprocessing (AdapterRemoval), dedup (Picard MarkDuplicates), genotyping
91+
(FreeBayes), profiling (Kraken2), and damage (mapDamage). Note the line drawn
92+
is *not* "data-flow shape stays, tool-swap goes": the mapper switch is a tool
93+
swap and survives. The kept set reads closer to "choices an aDNA analyst
94+
actually makes at runtime". Does the journey's scope decision land on a
95+
comparable set, and on what principle?
96+
- **The conditional plumbing tax.** Galaxy has no native if/else, so eager's
97+
toggles are paid for structurally: 7 `map_param_value` steps turn selections
98+
and optional-input presence into `when` booleans, and 6 `pick_value` steps
99+
rejoin the branches — 13 of 41 tool steps (~⅓) are param plumbing, not
100+
science. The pattern is duplicate-per-branch + gate + merge (`AdapterRemoval
101+
(Single-End)`/`(Paired-End)`, `Map with BWA`/`Bowtie2`). Record what the
102+
journey does here rather than scoring it against this idiom — duplicate-gate-
103+
merge is one known-workable answer, not the sanctioned one, and a cheaper
104+
route to the same run-time coverage would be a finding worth having. The tax
105+
is also the honest argument *for* a narrower scope: ~⅓ of the step budget
106+
going to plumbing is a real cost, and a journey that cites it while keeping
107+
fewer controls has reasoned, not given up. A worked human example of that
108+
trade priced out in full is the main reason this case earns its keep.
109+
- **Where the human answer looks arguable.** Steps 39/40 (Sex.DetERRmine
110+
with/without BED) are *both* registered workflow outputs, as are both ANGSD
111+
paths — so any given run yields outputs that are necessarily empty. If the
112+
journey avoids that, it is not automatically wrong for differing from the
113+
reference.
114+
- **Which branches were dropped entirely.** No circularmapper, malt/megan,
115+
metaphlan, krakenuniq, pmdtools, multivcfanalyzer/consensus, pileupcaller, or
116+
mapad in the PR. Does the journey's scope decision land on a comparable
117+
subset, and does it justify the cut?
118+
- **Tool resolution.** Per `eval.md`, every tool is discovered or authored. The
119+
human found Shed wrappers for the whole aDNA set (EndorSpy, Sex.DetERRmine,
120+
MtNucRatioCalculator, mapDamage included). Where the journey chose to *author*
121+
a wrapper for something that already exists in the Shed, that's a
122+
discover-shed-tool recall gap worth filing.
123+
- **Interface shape.** The human surfaced ~10 named inputs with prose-legible
124+
labels and pushed eager's flag sprawl into defaults. Compare against what the
125+
interface brief produced — count, naming, and how much of the ~200-param
126+
surface leaked through.
127+
- **Test strategy.** The PR ships a Planemo suite on sub-sampled `JK2067` /
128+
`hs37d5` / HapMap data. Compare against what
129+
[[nextflow-test-to-galaxy-test-plan]][[implement-galaxy-workflow-test]]
130+
produce — did the journey reach real aDNA fixtures, or synthesize? This is
131+
the case most likely to expose the pipeline's known test-data gap: the spine
132+
has no [[find-test-data]] phase, so `implement-galaxy-workflow-test` takes
133+
`test-data-refs` with nothing upstream producing it (the validator warns on
134+
exactly this). aDNA reference data is heavy and non-obvious; if the gap bites
135+
anywhere, it bites here.
136+
28137
## Case: branch-control fidelity
29138

30139
- fixture: a Nextflow summary with a skip-style branch control (e.g.

workflow-fixtures/fixtures.yaml

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,22 @@ pipelines:
9191
branching, different domain from rnaseq/sarek — stresses the
9292
incompatibility report on parallel-tool patterns.
9393
94+
- name: nf-core/eager
95+
flavor: nf-core
96+
tier: large
97+
repo: https://github.com/nf-core/eager.git
98+
sha: f77d2464a7c6fe06b67ce1832f2fe72ba21c8fde
99+
notes: >
100+
Ancient DNA (aDNA) analysis. Pinned to the DSL2 rewrite on `dev`
101+
(eager 3.x), NOT to a release: the latest tag (2.5.3) is DSL1
102+
(`nextflow.enable.dsl=1`, monolithic main.nf), which
103+
summarize-nextflow short-circuits by design. Bump the SHA rather
104+
than adding a `tag` until 3.0 ships. Large: ~110 .nf files, ~45
105+
nf-core modules, ~23 local subworkflows; damage-profiling and
106+
metagenomics branches on top of a preprocess/map/dedup spine.
107+
Backs the compare-and-contrast case in the NEXTFLOW → GALAXY
108+
pipeline scenarios against galaxyproject/iwc#1234.
109+
94110
# ----- ref-data complexity bridge (nf-core) -----
95111
# Small/medium pipelines staged on rungs of a reference-data
96112
# complexity ladder. Used to scope which ref-data patterns the

0 commit comments

Comments
 (0)