Skip to content

Make plugin metadata prefetch failure non-fatal#7019

Open
jorgee wants to merge 2 commits intomasterfrom
7013-warn-plugin-metadata-prefech-errors
Open

Make plugin metadata prefetch failure non-fatal#7019
jorgee wants to merge 2 commits intomasterfrom
7013-warn-plugin-metadata-prefech-errors

Conversation

@jorgee
Copy link
Copy Markdown
Contributor

@jorgee jorgee commented Apr 9, 2026

Summary

Fixes #7013

  • Make HttpPluginRepository.prefetch() catch exceptions and log a warning instead of aborting startup
  • Initialize the plugins map to an empty HashMap on failure so that getPlugin() can still attempt on-demand lazy fetches later
  • Refactor tests to verify prefetch error tolerance while still testing fetchMetadata() error behavior directly

Context

When all plugins are pinned to exact versions and pre-installed locally, Nextflow unconditionally calls the plugin registry API at startup. If the registry is unreachable, the run aborts even though the fetched metadata would never be used.

This affects users in airlocked/firewalled environments who pre-install plugins into container images. NXF_OFFLINE=true works around it but also blocks remote pipeline cloning, making it unusable for environments that need to fetch pipelines from a Git server.

With this fix, the prefetch error is deferred — it only surfaces later if something genuinely needs the metadata (version resolution for unpinned plugins, download URLs for missing plugins).

Test plan

  • HttpPluginRepositoryTest — all 9 tests pass
  • All nextflow.plugin.* tests pass
  • Manual: pre-install a plugin, block registry access, verify nextflow run succeeds

🤖 Generated with Claude Code

Signed-off-by: jorgee <jorge.ejarque@seqera.io>
@netlify
Copy link
Copy Markdown

netlify bot commented Apr 9, 2026

Deploy Preview for nextflow-docs-staging canceled.

Name Link
🔨 Latest commit f06e2ce
🔍 Latest deploy log https://app.netlify.com/projects/nextflow-docs-staging/deploys/69d8d8acfbb6da0008bd610f

@jorgee
Copy link
Copy Markdown
Contributor Author

jorgee commented Apr 9, 2026

Tested without connectivity to registry.

  • When no plugin available: Print warning and fail because it can't download the plugin
 N E X T F L O W   ~  version 26.03.2-edge

WARN: Failed to prefetch plugin metadata - Unable to connect to https://registry.nextflow.io/api/v1/plugins/dependencies?plugins=nf-schema%402.4.2&nextflowVersion=26.03.2-edge - cause: null
Downloading plugin nf-schema@2.4.2
ERROR ~ Plugin with id nf-schema not found in any repository
  • When plugin version is pinned and exists: Print the warning message at prefetch and succesfuly run the pipeline
 N E X T F L O W   ~  version 26.03.2-edge

WARN: Failed to prefetch plugin metadata - Unable to connect to https://registry.nextflow.io/api/v1/plugins/dependencies?plugins=nf-schema%402.4.2&nextflowVersion=26.03.2-edge - cause: null
Launching `main.nf` [nostalgic_almeida] revision: 6b4f49b787

Plugin loaded successfully
  • When no pinned version: Print the warning and fail because can't check latest version.
 N E X T F L O W   ~  version 26.03.2-edge

WARN: Failed to prefetch plugin metadata - Unable to connect to https://registry.nextflow.io/api/v1/plugins/dependencies?plugins=nf-schema&nextflowVersion=26.03.2-edge - cause: null
ERROR ~ Cannot find latest version of nf-schema plugin

 -- Check '.nextflow.log' file for details

Comment on lines +90 to +91
log.warn "Failed to prefetch plugin metadata - ${e.message}"
log.debug "Plugin metadata prefetch error details", e
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's keep only warn

@MatthiasZepper
Copy link
Copy Markdown

Thanks for addressing this problem, it was indeed causing an issue for us at NGI (me) when setting up a recent Nextflow version on our HPC.

Already in its current form, this will likely work perfectly for us. However, since there is already an environment variable NXF_DISABLE_CHECK_LATEST, you could optionally extent its reach to not only disable checking for the latest Nextflow version, but also for the latest plugin versions?

That would introduce a 4th case: When no pinned version, but NXF_DISABLE_CHECK_LATEST=true : Print a warning and try using the most recent locally available version of the plugin.

I see why allowing this may cause hard to debug issues (incompatible Nextflow/Plugin combinations?), but also reckon that few developers and almost no users will actually track the minimally required plugin version. If somebody is already keeping their NXF_HOME around for a while and adding to it, it will likely contain several older versions of a plugin and the most recent of them may actually work. Giving the user the power to launch a "Try with what is locally available" run seems helpful in certain edge cases such as a read-only shared plugin directory on an HPC?

Signed-off-by: jorgee <jorge.ejarque@seqera.io>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Plugin metadata prefetch fails hard even when result is unused (pre-installed plugins, exact versions)

4 participants