Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .devcontainer/codespaces-dev/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@
// Nextflow installation version
"NXF_HOME": "/workspaces/.nextflow",
"NXF_EDGE": "0",
"NXF_VER": "25.04.3",
"NXF_VER": "25.10.0",
// Other env vars
"HOST_PROJECT_PATH": "/workspaces/training",
"SHELL": "/bin/bash" // Ush bash
Expand Down
2 changes: 1 addition & 1 deletion .devcontainer/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@
// Nextflow installation version
"NXF_HOME": "/workspaces/.nextflow",
"NXF_EDGE": "0",
"NXF_VER": "25.04.3",
"NXF_VER": "25.10.0",
// Other env vars
"HOST_PROJECT_PATH": "/workspaces/training",
"SHELL": "/bin/bash" // Ush bash
Expand Down
2 changes: 1 addition & 1 deletion .devcontainer/local-dev/devcontainer.json
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@
// Nextflow installation version
"NXF_HOME": "/workspaces/training/.nextflow",
"NXF_EDGE": "0",
"NXF_VER": "25.04.3",
"NXF_VER": "25.10.0",
// Other env vars
"HOST_PROJECT_PATH": "/workspaces/training",
"SHELL": "/bin/bash" // Ush bash
Expand Down
135 changes: 105 additions & 30 deletions docs/hello_nextflow/01_hello_world.md
Original file line number Diff line number Diff line change
Expand Up @@ -212,7 +212,7 @@ nextflow run hello-world.nf
You console output should look something like this:

```console title="Output" linenums="1"
N E X T F L O W ~ version 25.04.3
N E X T F L O W ~ version 25.10.0

Launching `hello-world.nf` [goofy_torvalds] DSL2 - revision: c33d41f479

Expand Down Expand Up @@ -305,7 +305,7 @@ Learn how to manage your workflow executions conveniently.

Knowing how to launch workflows and retrieve outputs is great, but you'll quickly find there are a few other aspects of workflow management that will make your life easier, especially if you're developing your own workflows.

Here we show you how to use the `publishDir` directive to store in an output folder all the main results from your pipeline run, the `resume` feature for when you need to re-launch the same workflow, and how to delete older work directories with `nextflow clean`.
Here we show you how to use workflow outputs to store in an output folder all the main results from your pipeline run, the `resume` feature for when you need to re-launch the same workflow, and how to delete older work directories with `nextflow clean`.

### 3.1. Publish outputs

Expand All @@ -314,34 +314,76 @@ This is done on purpose; Nextflow is in control of this directory and we are not

However, that makes it inconvenient to retrieve outputs that we care about.

Fortunately, Nextflow provides a way to manage this more conveniently, called the `publishDir` directive, which acts at the process level.
This directive tells Nextflow to publish the output(s) of the process to a designated output directory. By default, the outputs are published as symbolic links from the `work` directory.
It allows us to retrieve the desired output file without having to dig down into the work directory.
Fortunately, Nextflow provides a way to manage this more conveniently using **workflow outputs**.
Workflow outputs allow you to declare which outputs from your workflow should be published to a designated output directory.
This approach centralizes output publishing at the workflow level rather than at individual processes.

#### 3.1.1. Add a `publishDir` directive to the `sayHello` process
#### 3.1.1. Add workflow outputs to publish results

In the workflow script file `hello-world.nf`, make the following code modification:
In the workflow script file `hello-world.nf`, make the following code modification to add a `publish:` section to the workflow and an `output` block:

=== "After"

```groovy title="hello-world.nf" linenums="6" hl_lines="3"
process sayHello {
```groovy title="hello-world.nf" linenums="1" hl_lines="16-28"
#!/usr/bin/env nextflow

publishDir 'results', mode: 'copy'
process sayHello {

output:
path 'output.txt'

script:
"""
echo 'Hello World!' > output.txt
"""
}

workflow {

main:
// emit a greeting
sayHello()

publish:
greetings = sayHello.out
}

output {
greetings {
path '.'
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Question -- My understanding is that you can just leave the curly braces empty (ie greetings { } ) unless you wanted to override or customize the output location, is that correct? If so, are you specifying path '.' to show explicitly how one would write the path to modify the location?

Asking because I'm trying to decide what to do for the nf4science template; and unless there's a reason to do otherwise I'd like that to be consistent with how we teach it in Hello.

}
}
```

=== "Before"

```groovy title="hello-world.nf" linenums="6"
```groovy title="hello-world.nf" linenums="1" hl_lines="17-18"
#!/usr/bin/env nextflow

process sayHello {

output:
path 'output.txt'

script:
"""
echo 'Hello World!' > output.txt
"""
}

workflow {

// emit a greeting
sayHello()
}
```

Let's break down what we added:

1. **`main:` section in the workflow**: When using `publish:`, we need to explicitly mark the main workflow logic with `main:`
2. **`publish:` section in the workflow**: This assigns the output channel from `sayHello()` to a named output called `greetings`
3. **`output` block**: This declares how the `greetings` output should be published. The `path '.'` means files will be published to the root of the output directory (which defaults to `results`)

#### 3.1.2. Run the workflow again

Now run the modified workflow script:
Expand All @@ -353,7 +395,7 @@ nextflow run hello-world.nf
The log output should look very familiar:

```console title="Output" linenums="1"
N E X T F L O W ~ version 25.04.3
N E X T F L O W ~ version 25.10.0

Launching `hello-world.nf` [jovial_mayer] DSL2 - revision: 35bd3425e5

Expand All @@ -366,14 +408,50 @@ Our `output.txt` file is in this directory.
If you check the contents it should match the output in the work subdirectory.
This is how we publish results files outside of the working directories conveniently.

When you're dealing with very large files that you don't need to retain for long, you may prefer to set the `publishDir` directive to make a symbolic link to the file instead of copying it.
However, if you delete the work directory as part of a cleanup operation, you will lose access to the file, so always make sure you have actual copies of everything you care about before deleting anything.
By default, Nextflow creates symbolic links from the output directory to files in the work directory.
This is efficient because it doesn't duplicate files, but it means if you delete the work directory, you'll lose access to the outputs.

!!! note
In this training environment, we've configured Nextflow to copy files instead of creating symlinks, which is safer for learning purposes.
You can configure this behavior in your own pipelines using the `workflow.output.mode` setting (covered later in this training).

!!! tip

You can change the output directory name by using the `-output-dir` command-line option:
```bash
nextflow run hello-world.nf -output-dir my-results
```

!!! note "Understanding workflow outputs vs. publishDir (legacy syntax)"

Before workflow outputs were introduced, Nextflow used a different approach to publishing results called the `publishDir` directive.
This directive was applied at the process level and looked like this:

```groovy
process sayHello {

A newer syntax option documented [here](https://www.nextflow.io/docs/latest/workflow.html#publishing-outputs) has been proposed to make it possible to declare and publish workflow-level outputs.
This will eventually make using `publishDir` at the process level redundant for completed pipelines.
However, we expect that `publishDir` will still remain very useful during pipeline development.
publishDir 'results', mode: 'copy'

output:
path 'output.txt'

script:
"""
echo 'Hello World!' > output.txt
"""
}
```

The `publishDir` directive tells Nextflow to copy (or symlink) the process outputs to the specified directory.
You can set options like `mode: 'copy'` to control how files are published.

The modern **workflow outputs** approach you just learned provides several advantages over `publishDir`:

- **Centralized**: All publishing logic is in one place at the workflow level, rather than scattered across individual process definitions
- **Flexible**: You can easily control what gets published from the workflow level without modifying process code
- **Cleaner modules**: Processes don't need to know where their outputs should be published, making them more reusable

The `publishDir` directive is still supported but is being phased out in favor of workflow outputs.
You will encounter `publishDir` frequently when reading existing Nextflow pipelines and working with nf-core modules, so it's important to understand both approaches.

### 3.2. Re-launch a workflow with `-resume`

Expand All @@ -397,7 +475,7 @@ nextflow run hello-world.nf -resume
The console output should look similar.

```console title="Output" linenums="1"
N E X T F L O W ~ version 25.04.3
N E X T F L O W ~ version 25.10.0

Launching `hello-world.nf` [golden_cantor] DSL2 - revision: 35bd3425e5

Expand All @@ -411,7 +489,7 @@ Nextflow is literally pointing you to the previous execution and saying "I alrea

!!! note

When your re-run a pipeline with `resume`, Nextflow does not overwrite any files written to a `publishDir` directory by any process call that was previously run successfully.
When you re-run a pipeline with `resume`, Nextflow does not overwrite any files written to the output directory by any process call that was previously run successfully.

### 3.3. Delete older work directories

Expand Down Expand Up @@ -454,7 +532,8 @@ Removed /workspaces/training/hello-nextflow/work/a3/7be2fad5e71e5f49998f795677fd
Deleting work subdirectories from past runs removes them from Nextflow's cache and deletes any outputs that were stored in those directories.
That means it breaks Nextflow's ability to resume execution without re-running the corresponding processes.

You are responsible for saving any outputs that you care about or plan to rely on! If you're using the `publishDir` directive for that purpose, make sure to use the `copy` mode, not the `symlink` mode.
You are responsible for saving any outputs that you care about or plan to rely on by publishing them to the output directory!
Make sure your workflow outputs use the `copy` mode (the default), not the `symlink` mode.

### Takeaway

Expand Down Expand Up @@ -489,11 +568,9 @@ In the process block, make the following code change:

=== "After"

```groovy title="hello-world.nf" linenums="6" hl_lines="5 6"
```groovy title="hello-world.nf" linenums="3" hl_lines="3 4"
process sayHello {

publishDir 'results', mode: 'copy'

input:
val greeting

Expand All @@ -503,11 +580,9 @@ In the process block, make the following code change:

=== "Before"

```groovy title="hello-world.nf" linenums="6"
```groovy title="hello-world.nf" linenums="3"
process sayHello {

publishDir 'results', mode: 'copy'

output:
path 'output.txt'
```
Expand Down Expand Up @@ -586,7 +661,7 @@ nextflow run hello-world.nf --greeting 'Bonjour le monde!'
If you made all three edits correctly, you should get another successful execution:

```console title="Output" linenums="1"
N E X T F L O W ~ version 25.04.3
N E X T F L O W ~ version 25.10.0

Launching `hello-world.nf` [elated_lavoisier] DSL2 - revision: 7c031b42ea

Expand Down Expand Up @@ -639,7 +714,7 @@ nextflow run hello-world.nf
The console output should look the same.

```console title="Output" linenums="1"
N E X T F L O W ~ version 25.04.3
N E X T F L O W ~ version 25.10.0

Launching `hello-world.nf` [determined_edison] DSL2 - revision: 3539118582

Expand Down Expand Up @@ -668,7 +743,7 @@ nextflow run hello-world.nf --greeting 'Konnichiwa!'
The console output should look the same.

```console title="Output" linenums="1"
N E X T F L O W ~ version 25.04.3
N E X T F L O W ~ version 25.10.0

Launching `hello-world.nf` [elegant_faraday] DSL2 - revision: 3539118582

Expand Down
Loading
Loading