Skip to content

Plugins: Use any available cached version if NXF_OFFLINE is true #4329

Closed
@ewels

Description

@ewels

Currently, running offline with Nextflow plugins is possible, but it is a little difficult. I think that with some fairly minor changes we could streamline the process a little.

'cc @remiolsen and @MatthiasZepper to correct / add more detail if needed.

Usage scenario

Currently, working offline with plugins involves the following steps:

  • Fetch required plugins on an online system
  • Copy the $NXF_HOME dir / Nextflow plugins cache to the offline system
  • Use a custom config file to overwrite the plugin include in the pipeline to specifically pin the version of the plugin that was downloaded
    • This step is difficult, especially with more complex deployments such as running on Tower / with the Tower CLI

Pinning the plugin version within the workflow itself is undesirable, as that means that any bugfixes in the plugin do not make it out to pipelines for a long time. This is one the main benefits of moving code into a plugin.

My suggestion is that if NXF_OFFLINE is set for the workflow run (and potentially if the request to the plugins JSON times out / fails), and the plugin version in the workflow config is unpinned, then Nextflow should check the local plugins cache and use the latest available version.

If the plugin version is pinned and not available in the cache, then fine - the workflow must fail. No question there.

Metadata

Metadata

Assignees

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions