Multi-channel workflow outputs

[dvmcconnell]

New feature

I'm wondering if there is any interest in being able to use a multi-channel in the publish: block of a workflow. This would provide (in my eyes) a cleaner interface than some of the other options I've found, but I am not sure what it would take to make it happen.

Use case

It would be nice to use the multi-channel just like a map in the output block when specifying the path for each output. Here's sort of what I'm envisioning, with a process/workflow/output where a process writes multiple files and we want each to go to a different location:

process my_process {
    input:
    ...

    output:
    path("file_1.txt", emit: file_1)
    path("file_2.txt", emit: file_2)
    path("file_3.txt", emit: file_3)
}

workflow {
    main:
    my_process(...)

    publish:
    my_process_outputs = my_process.out
}

output {
    my_process_outputs {
        path { ch -> 
            ch.file_1 >> "directory_1"
            ch.file_2 >> "directory_2"
            ch.file_3 >> "directory_3"
        }
    }
}

If my_process had a new output added, all you would need to do is update the output block, rather than having to update some map in the workflow as well the output block. It feels sort of "right" since multi-channels already have attribute access.

Suggested implementation

Can't comment on this

[bentsherman]

I think the best way to improve this is with record types. The problem right now is that people typically emit a separate channel for each output file type , when really a process should have only one output channel of records, where each record contains all of the file types with their emit name.

This gives you almost exactly the same syntax as your example:

process my_process {
    input:
    ...

    output:
    file_1 = path("file_1.txt")
    file_2 = path("file_2.txt")
    file_3 = path("file_3.txt")
}

workflow {
    main:
    my_process(...)

    publish:
    my_process_outputs = my_process.out
}

output {
    my_process_outputs {
        path { sample -> 
            sample.file_1 >> "directory_1"
            sample.file_2 >> "directory_2"
            sample.file_3 >> "directory_3"
        }
    }
}

In a way it's a transpose of the current approach. Right now a process returns a record of channels, but in the future it should return a channel of records