Skip to content

Use methods.setup_process_afterscript() for process logs #172

@nwiltsie

Description

@nwiltsie

We should incorporate the new methods.setup_process_afterscript() from uclahs-cds/pipeline-Nextflow-config#64 into this pipeline.

Under the new method, log files are automatically saved to ${params.log_output_dir}/process-log/${task.ext.log_dir}${task.ext.log_dir_suffix}/ (no need to specify a publishDir directive).

${task.ext.log_dir} defaults to ${task.process.replace(':', '/')}.
${task.ext.log_dir_suffix} defaults to a blank string.

The key steps are:

  1. Add methods.setup_process_afterscript() to methods.setup().
  2. For each Nextflow process:
    1. Remove the file ".command.*" output directive.
    2. Add ext log_dir { ... } and ext log_sur_suffix { ... } directives, if appropriate.
    3. Remove the publishDir directive with pattern: ".command.*".

As an example, this publishDir directive...

publishDir path: "${params.log_output_dir}/process-log",
pattern: ".command.*",
mode: "copy",
saveAs: { "${task.process.replace(':', '/')}/${task.process.split(':')[-1]}-${sample_id}-${interval_id}/log${file(it).getName()}" }

... should be replaced with this directive:

ext log_dir_suffix { "/${task.process.split(':')[-1]}-${sample_id}-${interval_id}" }

See https://github.com/uclahs-cds/pipeline-recalibrate-BAM/pull/76/files for a full example.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions