DSL2 - Genotyping Input Updates

[ilight1542]

Main bulk of this adjustment relates to #1128.

Otherwise, this is a QOL improvement/update for post mapping inputs into genotyping and its interactions with damage manipulation. Ascurrently implemented, this is a bit inflexible and requires the user to have specific parameter combinations for saving damage_manipulation outputs.

goal for improvement:

• Allow for library by library genotyping
• Uncouple damage manipulation saving from genotyping input

• Allow for multiple damage manipulation tool calls (currently allowed, but previously only one would be saved if used for genotyping input)
• Removed overwriting of final_bams/raw by multiple calls of samtools merge
• Expand parameter checks/warnings to guide end user through best practices

PR checklist

• This comment contains a description of changes (with reason).
• If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - add to the software_versions process and a regex to scrape_software_versions.py
  • If you've added a new tool - have you followed the pipeline conventions in the [contribution docs](https://github.com/nf-core/eager/tree/master/.github/CONTRIBUTING.md)
  • If necessary, also make a PR on the nf-core/eager branch on the nf-core/test-datasets repository.
• Make sure your code lints (nf-core lint .).
• Ensure the test suite passes (nextflow run . -profile test,docker).
• Usage Documentation in docs/usage.md is updated.
• Output Documentation in docs/output.md is updated.
• CHANGELOG.md is updated.
• README.md is updated (including new tool citations and authors/contributors).

[ilight1542]

Current issue, Read Group assignment is done at least in some cases to be consistent across all libraries of a given sample -- but this breaks mpiluep calling which expects each input file to have a unique Read Group Sample ID.

will check if it breaks everything to swap to libID read groups (possibly only in the case we are running genotyping on unmerged libs?) 🗡️

RG assigned:

• bowtie2
• bwa

RG not assigned (?):

• circular mapper

[github-actions]

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 6cb1598

+| ✅ 408 tests passed       |+
#| ❔   1 tests were ignored |#
!| ❗  46 tests had warnings |!

❗ Test warnings:

• readme - README contains the placeholder zenodo.XXXXXXX. This should be replaced with the zenodo doi (after the first release).
• pipeline_todos - TODO string in nextflow.config: Specify your pipeline's command line flags
• pipeline_todos - TODO string in main.nf: Remove this line if you don't need a FASTA file
• pipeline_todos - TODO string in ro-crate-metadata.json: "description": "

  \n \n <source media="(prefers-color-scheme: dark)" srcset="docs/images/nf-core-eager_logo_dark.png">\n <img alt="nf-core/eager" src="docs/images/nf-core-eager_logo_light.png">\n \n

  \n\n\n\n\n\n\n\n\n\n\n\n\n\n## Introduction\n\nnf-core/eager is a bioinformatics pipeline that ...\n\n TODO nf-core:\n Complete this sentence with a 2-3 sentence summary of what types of data the pipeline ingests, a brief overview of the\n major pipeline sections and the types of output it produces. You're giving an overview to someone new\n to nf-core here, in 15-20 seconds. For an example, see https://github.com/nf-core/rnaseq/blob/master/README.md#introduction\n\n\n Include a figure that guides the user through the major workflow steps. Many nf-core\n workflows use the "tube map" design for that. See https://nf-co.re/docs/contributing/design_guidelines#examples for examples. \n Fill in short bullet-pointed list of the default steps in the pipeline 1. Read QC (FastQC)2. Present QC for raw reads (MultiQC)\n\n## Usage\n\n> [!NOTE]\n> If you are new to Nextflow and nf-core, please refer to this page on how to set-up Nextflow. Make sure to test your setup with -profile test before running the workflow on actual data.\n\n Describe the minimum required steps to execute the pipeline, e.g. how to prepare samplesheets.\n Explain what rows and columns represent. For instance (please edit as appropriate):\n\nFirst, prepare a samplesheet with your input data that looks as follows:\n\nsamplesheet.csv:\n\ncsv\nsample,fastq_1,fastq_2\nCONTROL_REP1,AEG588A1_S1_L002_R1_001.fastq.gz,AEG588A1_S1_L002_R2_001.fastq.gz\n\n\nEach row represents a fastq file (single-end) or a pair of fastq files (paired end).\n\n\n\nNow, you can run the pipeline using:\n\n update the following command to include all required parameters for a minimal example \n\nbash\nnextflow run nf-core/eager \\\n -profile <docker/singularity/.../institute> \\\n --input samplesheet.csv \\\n --outdir <OUTDIR>\n\n\n> [!WARNING]\n> Please provide pipeline parameters via the CLI or Nextflow -params-file option. Custom config files including those provided by the -c Nextflow option can be used to provide any configuration except for parameters; see docs.\n\nFor more details and further functionality, please refer to the usage documentation and the parameter documentation.\n\n## Pipeline output\n\nTo see the results of an example test run with a full size dataset refer to the results tab on the nf-core website pipeline page.\nFor more details about the output files and reports, please refer to the\noutput documentation.\n\n## Credits\n\nnf-core/eager was originally written by The nf-core/eager community.\n\nWe thank the following people for their extensive assistance in the development of this pipeline:\n\n If applicable, make list of people who have also contributed \n\n## Contributions and Support\n\nIf you would like to contribute to this pipeline, please see the contributing guidelines.\n\nFor further information or help, don't hesitate to get in touch on the Slack #eager channel (you can join with this invite).\n\n## Citations\n\n Add citation for pipeline after first release. Uncomment lines below and update Zenodo doi and badge at the top of this file. \n If you use nf-core/eager for your analysis, please cite it using the following doi: 10.5281/zenodo.XXXXXX \n\n Add bibliography of tools and data used in your pipeline \n\nAn extensive list of references for the tools used by the pipeline can be found in the CITATIONS.md file.\n\nYou can cite the nf-core publication as follows:\n\n> The nf-core framework for community-curated bioinformatics pipelines.\n>\n> Philip Ewels, Alexander Peltzer, Sven Fillinger, Harshil Patel, Johannes Alneberg, Andreas Wilm, Maxime Ulysse Garcia, Paolo Di Tommaso & Sven Nahnsen.\n>\n> Nat Biotechnol. 2020 Feb 13. doi: 10.1038/s41587-020-0439-x.\n",
• pipeline_todos - TODO string in README.md: Include a figure that guides the user through the major workflow steps. Many nf-core
• pipeline_todos - TODO string in README.md: Fill in short bullet-pointed list of the default steps in the pipeline
• pipeline_todos - TODO string in awsfulltest.yml: You can customise AWS full pipeline tests as required
• pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
• pipeline_todos - TODO string in base.config: Check the defaults for all processes
• pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
• pipeline_todos - TODO string in test_minimal.config: Specify the paths to your test data on nf-core/test-datasets
• pipeline_todos - TODO string in test_minimal.config: Give any required params for the test so that command line flags are not needed
• pipeline_todos - TODO string in test_full.config: Specify the paths to your full test data ( on nf-core/test-datasets or directly in repositories, e.g. SRA)
• pipeline_todos - TODO string in test_full.config: Give any required params for the test so that command line flags are not needed
• pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
• pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
• pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
• pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
• local_component_structure - build_intervals.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
• local_component_structure - collect_genotypes.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
• local_component_structure - print_contamination_angsd.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
• local_component_structure - host_removal.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
• local_component_structure - samtools_view_genome.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
• local_component_structure - filter_bam_fragment_length.nf in modules/local should be moved to a TOOL/SUBTOOL/main.nf structure
• local_component_structure - preprocessing.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - reference_indexing_single.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - metagenomics_postprocessing.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - run_sex_determination.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - metagenomics.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - genotype.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - metagenomics_complexityfilter.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - calculate_damage.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - reference_indexing.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - elongate_reference.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - preprocessing_fastp.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - bamfiltering.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - reference_indexing_multi.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - merge_libraries.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - manipulate_damage.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - circularmapper.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - preprocessing_adapterremoval.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - deduplicate.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - merge_lanes_inputbam.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - estimate_contamination.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - map.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
• local_component_structure - metagenomics_profiling.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

• nextflow_config - Config default ignored: params.contamination_estimation_angsd_hapmap

✅ Tests passed:

• files_exist - File found: .gitattributes
• files_exist - File found: .gitignore
• files_exist - File found: .nf-core.yml
• files_exist - File found: .editorconfig
• files_exist - File found: .prettierignore
• files_exist - File found: .prettierrc.yml
• files_exist - File found: CHANGELOG.md
• files_exist - File found: CITATIONS.md
• files_exist - File found: CODE_OF_CONDUCT.md
• files_exist - File found: LICENSE or LICENSE.md or LICENCE or LICENCE.md
• files_exist - File found: nextflow_schema.json
• files_exist - File found: nextflow.config
• files_exist - File found: README.md
• files_exist - File found: .github/.dockstore.yml
• files_exist - File found: .github/CONTRIBUTING.md
• files_exist - File found: .github/ISSUE_TEMPLATE/bug_report.yml
• files_exist - File found: .github/ISSUE_TEMPLATE/config.yml
• files_exist - File found: .github/ISSUE_TEMPLATE/feature_request.yml
• files_exist - File found: .github/PULL_REQUEST_TEMPLATE.md
• files_exist - File found: .github/workflows/branch.yml
• files_exist - File found: .github/workflows/ci.yml
• files_exist - File found: .github/workflows/linting_comment.yml
• files_exist - File found: .github/workflows/linting.yml
• files_exist - File found: assets/email_template.html
• files_exist - File found: assets/email_template.txt
• files_exist - File found: assets/sendmail_template.txt
• files_exist - File found: assets/nf-core-eager_logo_light.png
• files_exist - File found: conf/modules.config
• files_exist - File found: conf/test.config
• files_exist - File found: conf/test_full.config
• files_exist - File found: docs/images/nf-core-eager_logo_light.png
• files_exist - File found: docs/images/nf-core-eager_logo_dark.png
• files_exist - File found: docs/output.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/README.md
• files_exist - File found: docs/usage.md
• files_exist - File found: main.nf
• files_exist - File found: assets/multiqc_config.yml
• files_exist - File found: conf/base.config
• files_exist - File found: conf/igenomes.config
• files_exist - File found: conf/igenomes_ignored.config
• files_exist - File found: .github/workflows/awstest.yml
• files_exist - File found: .github/workflows/awsfulltest.yml
• files_exist - File found: modules.json
• files_exist - File found: ro-crate-metadata.json
• files_exist - File not found check: .github/ISSUE_TEMPLATE/bug_report.md
• files_exist - File not found check: .github/ISSUE_TEMPLATE/feature_request.md
• files_exist - File not found check: .github/workflows/push_dockerhub.yml
• files_exist - File not found check: .markdownlint.yml
• files_exist - File not found check: .nf-core.yaml
• files_exist - File not found check: .yamllint.yml
• files_exist - File not found check: bin/markdown_to_html.r
• files_exist - File not found check: conf/aws.config
• files_exist - File not found check: docs/images/nf-core-eager_logo.png
• files_exist - File not found check: lib/Checks.groovy
• files_exist - File not found check: lib/Completion.groovy
• files_exist - File not found check: lib/NfcoreTemplate.groovy
• files_exist - File not found check: lib/Utils.groovy
• files_exist - File not found check: lib/Workflow.groovy
• files_exist - File not found check: lib/WorkflowMain.groovy
• files_exist - File not found check: lib/WorkflowEager.groovy
• files_exist - File not found check: parameters.settings.json
• files_exist - File not found check: pipeline_template.yml
• files_exist - File not found check: Singularity
• files_exist - File not found check: lib/nfcore_external_java_deps.jar
• files_exist - File not found check: .travis.yml
• nextflow_config - Found nf-schema plugin
• nextflow_config - Config variable found: manifest.name
• nextflow_config - Config variable found: manifest.nextflowVersion
• nextflow_config - Config variable found: manifest.description
• nextflow_config - Config variable found: manifest.version
• nextflow_config - Config variable found: manifest.homePage
• nextflow_config - Config variable found: timeline.enabled
• nextflow_config - Config variable found: trace.enabled
• nextflow_config - Config variable found: report.enabled
• nextflow_config - Config variable found: dag.enabled
• nextflow_config - Config variable found: process.cpus
• nextflow_config - Config variable found: process.memory
• nextflow_config - Config variable found: process.time
• nextflow_config - Config variable found: params.outdir
• nextflow_config - Config variable found: params.input
• nextflow_config - Config variable found: validation.help.enabled
• nextflow_config - Config variable found: manifest.mainScript
• nextflow_config - Config variable found: timeline.file
• nextflow_config - Config variable found: trace.file
• nextflow_config - Config variable found: report.file
• nextflow_config - Config variable found: dag.file
• nextflow_config - Config variable found: validation.help.beforeText
• nextflow_config - Config variable found: validation.help.afterText
• nextflow_config - Config variable found: validation.help.command
• nextflow_config - Config variable found: validation.summary.beforeText
• nextflow_config - Config variable found: validation.summary.afterText
• nextflow_config - Config variable (correctly) not found: params.nf_required_version
• nextflow_config - Config variable (correctly) not found: params.container
• nextflow_config - Config variable (correctly) not found: params.singleEnd
• nextflow_config - Config variable (correctly) not found: params.igenomesIgnore
• nextflow_config - Config variable (correctly) not found: params.name
• nextflow_config - Config variable (correctly) not found: params.enable_conda
• nextflow_config - Config variable (correctly) not found: params.max_cpus
• nextflow_config - Config variable (correctly) not found: params.max_memory
• nextflow_config - Config variable (correctly) not found: params.max_time
• nextflow_config - Config variable (correctly) not found: params.validationFailUnrecognisedParams
• nextflow_config - Config variable (correctly) not found: params.validationLenientMode
• nextflow_config - Config variable (correctly) not found: params.validationSchemaIgnoreParams
• nextflow_config - Config variable (correctly) not found: params.validationShowHiddenParams
• nextflow_config - Config timeline.enabled had correct value: true
• nextflow_config - Config report.enabled had correct value: true
• nextflow_config - Config trace.enabled had correct value: true
• nextflow_config - Config dag.enabled had correct value: true
• nextflow_config - Config manifest.name began with nf-core/
• nextflow_config - Config variable manifest.homePage began with https://github.com/nf-core/
• nextflow_config - Config dag.file ended with .html
• nextflow_config - Config variable manifest.nextflowVersion started with >= or !>=
• nextflow_config - Config manifest.version ends in dev: 3.0.0dev
• nextflow_config - Config params.custom_config_version is set to master
• nextflow_config - Config params.custom_config_base is set to https://raw.githubusercontent.com/nf-core/configs/master
• nextflow_config - Lines for loading custom profiles found
• nextflow_config - nextflow.config contains configuration profile test
• nextflow_config - Config default value correct: params.igenomes_base= s3://ngi-igenomes/igenomes/
• nextflow_config - Config default value correct: params.fasta_circularmapper_elongationfactor= 500
• nextflow_config - Config default value correct: params.custom_config_version= master
• nextflow_config - Config default value correct: params.custom_config_base= https://raw.githubusercontent.com/nf-core/configs/master
• nextflow_config - Config default value correct: params.publish_dir_mode= copy
• nextflow_config - Config default value correct: params.max_multiqc_email_size= 25.MB
• nextflow_config - Config default value correct: params.validate_params= true
• nextflow_config - Config default value correct: params.pipelines_testdata_base_path= https://raw.githubusercontent.com/nf-core/test-datasets/
• nextflow_config - Config default value correct: params.sequencing_qc_tool= fastqc
• nextflow_config - Config default value correct: params.preprocessing_tool= fastp
• nextflow_config - Config default value correct: params.preprocessing_minlength= 25
• nextflow_config - Config default value correct: params.preprocessing_trim5p= 0
• nextflow_config - Config default value correct: params.preprocessing_trim3p= 0
• nextflow_config - Config default value correct: params.preprocessing_fastp_complexityfilter_threshold= 10
• nextflow_config - Config default value correct: params.preprocessing_adapterremoval_trimbasequalitymin= 20
• nextflow_config - Config default value correct: params.preprocessing_adapterremoval_adapteroverlap= 1
• nextflow_config - Config default value correct: params.preprocessing_adapterremoval_qualitymax= 41
• nextflow_config - Config default value correct: params.fastq_shard_size= 1000000
• nextflow_config - Config default value correct: params.mapping_tool= bwaaln
• nextflow_config - Config default value correct: params.mapping_bwaaln_n= 0.01
• nextflow_config - Config default value correct: params.mapping_bwaaln_k= 2
• nextflow_config - Config default value correct: params.mapping_bwaaln_l= 1024
• nextflow_config - Config default value correct: params.mapping_bwaaln_o= 2
• nextflow_config - Config default value correct: params.mapping_bwamem_k= 19
• nextflow_config - Config default value correct: params.mapping_bwamem_r= 1.5
• nextflow_config - Config default value correct: params.mapping_bowtie2_alignmode= local
• nextflow_config - Config default value correct: params.mapping_bowtie2_sensitivity= sensitive
• nextflow_config - Config default value correct: params.mapping_bowtie2_n= 0
• nextflow_config - Config default value correct: params.mapping_bowtie2_l= 20
• nextflow_config - Config default value correct: params.mapping_bowtie2_trim5= 0
• nextflow_config - Config default value correct: params.mapping_bowtie2_trim3= 0
• nextflow_config - Config default value correct: params.mapping_bowtie2_maxins= 500
• nextflow_config - Config default value correct: params.mapping_mapad_p= 0.03
• nextflow_config - Config default value correct: params.mapping_mapad_f= 0.5
• nextflow_config - Config default value correct: params.mapping_mapad_t= 0.5
• nextflow_config - Config default value correct: params.mapping_mapad_d= 0.02
• nextflow_config - Config default value correct: params.mapping_mapad_s= 1.0
• nextflow_config - Config default value correct: params.mapping_mapad_i= 0.001
• nextflow_config - Config default value correct: params.mapping_mapad_x= 0.5
• nextflow_config - Config default value correct: params.mapping_mapad_gapreadenddistance= 5.0
• nextflow_config - Config default value correct: params.mapping_mapad_maxnumopengaps= 2.0
• nextflow_config - Config default value correct: params.mapping_mapad_baseerrorrate= 0.02
• nextflow_config - Config default value correct: params.bamfiltering_minreadlength= 0
• nextflow_config - Config default value correct: params.bamfiltering_mappingquality= 0
• nextflow_config - Config default value correct: params.bamfiltering_genomicbamfilterflag= 4
• nextflow_config - Config default value correct: params.metagenomics_input= unmapped
• nextflow_config - Config default value correct: params.metagenomics_complexity_tool= bbduk
• nextflow_config - Config default value correct: params.metagenomics_complexity_entropy= 0.3
• nextflow_config - Config default value correct: params.metagenomics_prinseq_mode= entropy
• nextflow_config - Config default value correct: params.metagenomics_prinseq_dustscore= 0.5
• nextflow_config - Config default value correct: params.metagenomics_krakenuniq_ramchunksize= 16G
• nextflow_config - Config default value correct: params.metagenomics_malt_mode= BlastN
• nextflow_config - Config default value correct: params.metagenomics_malt_alignmentmode= SemiGlobal
• nextflow_config - Config default value correct: params.metagenomics_malt_minpercentidentity= 85
• nextflow_config - Config default value correct: params.metagenomics_malt_toppercent= 1
• nextflow_config - Config default value correct: params.metagenomics_malt_minsupportmode= percent
• nextflow_config - Config default value correct: params.metagenomics_malt_minsupportpercent= 0.01
• nextflow_config - Config default value correct: params.metagenomics_malt_minsupportreads= 1
• nextflow_config - Config default value correct: params.metagenomics_malt_maxqueries= 100
• nextflow_config - Config default value correct: params.metagenomics_malt_memorymode= load
• nextflow_config - Config default value correct: params.metagenomics_malt_group_size= 0
• nextflow_config - Config default value correct: params.metagenomics_maltextract_filter= def_anc
• nextflow_config - Config default value correct: params.metagenomics_maltextract_toppercent= 0.01
• nextflow_config - Config default value correct: params.metagenomics_maltextract_minpercentidentity= 85.0
• nextflow_config - Config default value correct: params.deduplication_tool= markduplicates
• nextflow_config - Config default value correct: params.damage_manipulation_rescale_seqlength= 12
• nextflow_config - Config default value correct: params.damage_manipulation_rescale_length_5p= 0
• nextflow_config - Config default value correct: params.damage_manipulation_rescale_length_3p= 0
• nextflow_config - Config default value correct: params.damage_manipulation_pmdtools_threshold= 3
• nextflow_config - Config default value correct: params.damage_manipulation_bamutils_trim_double_stranded_none_udg_left= 0
• nextflow_config - Config default value correct: params.damage_manipulation_bamutils_trim_double_stranded_none_udg_right= 0
• nextflow_config - Config default value correct: params.damage_manipulation_bamutils_trim_double_stranded_half_udg_left= 0
• nextflow_config - Config default value correct: params.damage_manipulation_bamutils_trim_double_stranded_half_udg_right= 0
• nextflow_config - Config default value correct: params.damage_manipulation_bamutils_trim_single_stranded_none_udg_left= 0
• nextflow_config - Config default value correct: params.damage_manipulation_bamutils_trim_single_stranded_none_udg_right= 0
• nextflow_config - Config default value correct: params.damage_manipulation_bamutils_trim_single_stranded_half_udg_left= 0
• nextflow_config - Config default value correct: params.damage_manipulation_bamutils_trim_single_stranded_half_udg_right= 0
• nextflow_config - Config default value correct: params.genotyping_source= raw
• nextflow_config - Config default value correct: params.genotyping_use_unmerged_libraries= false
• nextflow_config - Config default value correct: params.genotyping_reference_ploidy= 2
• nextflow_config - Config default value correct: params.genotyping_pileupcaller_min_base_quality= 30
• nextflow_config - Config default value correct: params.genotyping_pileupcaller_min_map_quality= 30
• nextflow_config - Config default value correct: params.genotyping_pileupcaller_method= randomHaploid
• nextflow_config - Config default value correct: params.genotyping_pileupcaller_transitions_mode= AllSites
• nextflow_config - Config default value correct: params.genotyping_gatk_call_conf= 30
• nextflow_config - Config default value correct: params.genotyping_gatk_ug_downsample= 250
• nextflow_config - Config default value correct: params.genotyping_gatk_ug_out_mode= EMIT_VARIANTS_ONLY
• nextflow_config - Config default value correct: params.genotyping_gatk_ug_genotype_mode= SNP
• nextflow_config - Config default value correct: params.genotyping_gatk_ug_defaultbasequalities= -1
• nextflow_config - Config default value correct: params.genotyping_gatk_hc_out_mode= EMIT_VARIANTS_ONLY
• nextflow_config - Config default value correct: params.genotyping_gatk_hc_emitrefconf= GVCF
• nextflow_config - Config default value correct: params.genotyping_freebayes_min_alternate_count= 1
• nextflow_config - Config default value correct: params.genotyping_freebayes_skip_coverage= 0
• nextflow_config - Config default value correct: params.genotyping_angsd_glmodel= samtools
• nextflow_config - Config default value correct: params.genotyping_angsd_glformat= binary
• nextflow_config - Config default value correct: params.mitochondrion_header= MT
• nextflow_config - Config default value correct: params.mapstats_preseq_mode= c_curve
• nextflow_config - Config default value correct: params.mapstats_preseq_stepsize= 1000
• nextflow_config - Config default value correct: params.mapstats_preseq_terms= 100
• nextflow_config - Config default value correct: params.mapstats_preseq_maxextrap= 10000000000
• nextflow_config - Config default value correct: params.mapstats_preseq_bootstrap= 100
• nextflow_config - Config default value correct: params.mapstats_preseq_cval= 0.95
• nextflow_config - Config default value correct: params.damagecalculation_tool= damageprofiler
• nextflow_config - Config default value correct: params.damagecalculation_yaxis= 0.3
• nextflow_config - Config default value correct: params.damagecalculation_xaxis= 25
• nextflow_config - Config default value correct: params.damagecalculation_damageprofiler_length= 100
• nextflow_config - Config default value correct: params.damagecalculation_mapdamage_downsample= 0
• nextflow_config - Config default value correct: params.host_removal_mode= remove
• nextflow_config - Config default value correct: params.contamination_estimation_angsd_chrom_name= X
• nextflow_config - Config default value correct: params.contamination_estimation_angsd_range_from= 5000000
• nextflow_config - Config default value correct: params.contamination_estimation_angsd_range_to= 154900000
• nextflow_config - Config default value correct: params.contamination_estimation_angsd_mapq= 30
• nextflow_config - Config default value correct: params.contamination_estimation_angsd_minq= 30
• files_unchanged - .gitattributes matches the template
• files_unchanged - .prettierrc.yml matches the template
• files_unchanged - CODE_OF_CONDUCT.md matches the template
• files_unchanged - LICENSE matches the template
• files_unchanged - .github/.dockstore.yml matches the template
• files_unchanged - .github/CONTRIBUTING.md matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/bug_report.yml matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/config.yml matches the template
• files_unchanged - .github/ISSUE_TEMPLATE/feature_request.yml matches the template
• files_unchanged - .github/PULL_REQUEST_TEMPLATE.md matches the template
• files_unchanged - .github/workflows/branch.yml matches the template
• files_unchanged - .github/workflows/linting_comment.yml matches the template
• files_unchanged - .github/workflows/linting.yml matches the template
• files_unchanged - assets/email_template.html matches the template
• files_unchanged - assets/email_template.txt matches the template
• files_unchanged - assets/sendmail_template.txt matches the template
• files_unchanged - assets/nf-core-eager_logo_light.png matches the template
• files_unchanged - docs/images/nf-core-eager_logo_light.png matches the template
• files_unchanged - docs/images/nf-core-eager_logo_dark.png matches the template
• files_unchanged - docs/README.md matches the template
• files_unchanged - .gitignore matches the template
• files_unchanged - .prettierignore matches the template
• actions_ci - '.github/workflows/ci.yml' is triggered on expected events
• actions_ci - '.github/workflows/ci.yml' checks minimum NF version
• actions_awstest - '.github/workflows/awstest.yml' is triggered correctly
• actions_awsfulltest - .github/workflows/awsfulltest.yml is triggered correctly
• actions_awsfulltest - .github/workflows/awsfulltest.yml does not use -profile test
• readme - README Nextflow minimum version badge matched config. Badge: 24.04.2, Config: 24.04.2
• plugin_includes - No wrong validation plugin imports have been found
• pipeline_name_conventions - Name adheres to nf-core convention
• template_strings - Did not find any Jinja template strings (0 files)
• schema_lint - Schema lint passed
• schema_lint - Schema title + description lint passed
• schema_lint - Input mimetype lint passed: 'text/csv'
• schema_params - Schema matched params returned from nextflow config
• system_exit - No System.exit calls found
• actions_schema_validation - Workflow validation passed: awsfulltest.yml
• actions_schema_validation - Workflow validation passed: download_pipeline.yml
• actions_schema_validation - Workflow validation passed: linting.yml
• actions_schema_validation - Workflow validation passed: release-announcements.yml
• actions_schema_validation - Workflow validation passed: ci.yml
• actions_schema_validation - Workflow validation passed: linting_comment.yml
• actions_schema_validation - Workflow validation passed: fix-linting.yml
• actions_schema_validation - Workflow validation passed: template_version_comment.yml
• actions_schema_validation - Workflow validation passed: branch.yml
• actions_schema_validation - Workflow validation passed: clean-up.yml
• actions_schema_validation - Workflow validation passed: awstest.yml
• merge_markers - No merge markers found in pipeline files
• modules_json - Only installed modules found in modules.json
• multiqc_config - assets/multiqc_config.yml found and not ignored.
• multiqc_config - assets/multiqc_config.yml contains report_section_order
• multiqc_config - assets/multiqc_config.yml contains export_plots
• multiqc_config - assets/multiqc_config.yml contains report_comment
• multiqc_config - assets/multiqc_config.yml follows the ordering scheme of the minimally required plugins.
• multiqc_config - assets/multiqc_config.yml contains a matching 'report_comment'.
• multiqc_config - assets/multiqc_config.yml contains 'export_plots: true'.
• modules_structure - modules directory structure is correct 'modules/nf-core/TOOL/SUBTOOL'
• base_config - conf/base.config found and not ignored.
• modules_config - conf/modules.config found and not ignored.
• modules_config - MULTIQC found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_CONVERT_BAM_INPUT found in conf/modules.config and Nextflow scripts.
• modules_config - CAT_FASTQ_CONVERTED_BAM found in conf/modules.config and Nextflow scripts.
• modules_config - FASTQC found in conf/modules.config and Nextflow scripts.
• modules_config - FASTQC_PROCESSED found in conf/modules.config and Nextflow scripts.
• modules_config - FALCO found in conf/modules.config and Nextflow scripts.
• modules_config - FALCO_PROCESSED found in conf/modules.config and Nextflow scripts.
• modules_config - FASTP_SINGLE found in conf/modules.config and Nextflow scripts.
• modules_config - FASTP_PAIRED found in conf/modules.config and Nextflow scripts.
• modules_config - ADAPTERREMOVAL_SINGLE found in conf/modules.config and Nextflow scripts.
• modules_config - ADAPTERREMOVAL_PAIRED found in conf/modules.config and Nextflow scripts.
• modules_config - CAT_FASTQ_ADAPTERREMOVAL found in conf/modules.config and Nextflow scripts.
• modules_config - GUNZIP_FASTA found in conf/modules.config and Nextflow scripts.
• modules_config - GUNZIP_PMDFASTA found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_FAIDX found in conf/modules.config and Nextflow scripts.
• modules_config - PICARD_CREATESEQUENCEDICTIONARY found in conf/modules.config and Nextflow scripts.
• modules_config - BOWTIE2_BUILD found in conf/modules.config and Nextflow scripts.
• modules_config - BWA_INDEX found in conf/modules.config and Nextflow scripts.
• modules_config - GUNZIP_ELONGATED_FASTA found in conf/modules.config and Nextflow scripts.
• modules_config - CIRCULARMAPPER_CIRCULARGENERATOR found in conf/modules.config and Nextflow scripts.
• modules_config - BWA_INDEX_CIRCULARISED found in conf/modules.config and Nextflow scripts.
• modules_config - MAPAD_INDEX found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_INDEX_BAM_INPUT found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_MERGE_LANES_BAMINPUT found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_SORT_MERGED_LANES_BAMINPUT found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_INDEX_MERGED_LANES_BAMINPUT found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_FLAGSTAT_MERGED_LANES_BAMINPUT found in conf/modules.config and Nextflow scripts.
• modules_config - FILTER_BAM_FRAGMENT_LENGTH found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_VIEW_BAM_FILTERING found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_LENGTH_FILTER_INDEX found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_FLAGSTAT_FILTERED found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_FASTQ_SAVEBAMFILTERINGREADS found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_FASTQ_METAGENOMICS found in conf/modules.config and Nextflow scripts.
• modules_config - CAT_FASTQ_METAGENOMICS found in conf/modules.config and Nextflow scripts.
• modules_config - SEQKIT_SPLIT2 found in conf/modules.config and Nextflow scripts.
• modules_config - BWA_ALN found in conf/modules.config and Nextflow scripts.
• modules_config - BWA_SAMSE found in conf/modules.config and Nextflow scripts.
• modules_config - BWA_MEM found in conf/modules.config and Nextflow scripts.
• modules_config - BOWTIE2_ALIGN found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_INDEX_MEM found in conf/modules.config and Nextflow scripts.
• modules_config - MAPAD_MAP found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_SORT_MAPAD found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_INDEX_MEM found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_MERGE_LANES found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_SORT_MERGED_LANES found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_INDEX_MERGED_LANES found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_FLAGSTAT_MERGED_LANES found in conf/modules.config and Nextflow scripts.
• modules_config - CIRCULARMAPPER_REALIGNSAMFILE found in conf/modules.config and Nextflow scripts.
• modules_config - PICARD_MARKDUPLICATES found in conf/modules.config and Nextflow scripts.
• modules_config - DEDUP found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_MERGE_DEDUPPED found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_SORT_DEDUPPED found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_INDEX_DEDUPPED found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_FLAGSTAT_DEDUPPED found in conf/modules.config and Nextflow scripts.
• modules_config - HOST_REMOVAL found in conf/modules.config and Nextflow scripts.
• modules_config - ENDORSPY found in conf/modules.config and Nextflow scripts.
• modules_config - PRESEQ_CCURVE found in conf/modules.config and Nextflow scripts.
• modules_config - PRESEQ_LCEXTRAP found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_VIEW_GENOME found in conf/modules.config and Nextflow scripts.
• modules_config - BEDTOOLS_COVERAGE_DEPTH found in conf/modules.config and Nextflow scripts.
• modules_config - BEDTOOLS_COVERAGE_BREADTH found in conf/modules.config and Nextflow scripts.
• modules_config - ANGSD_DOCOUNTS found in conf/modules.config and Nextflow scripts.
• modules_config - ANGSD_CONTAMINATION found in conf/modules.config and Nextflow scripts.
• modules_config - PRINT_CONTAMINATION_ANGSD found in conf/modules.config and Nextflow scripts.
• modules_config - MTNUCRATIO found in conf/modules.config and Nextflow scripts.
• modules_config - QUALIMAP_BAMQC_WITHBED found in conf/modules.config and Nextflow scripts.
• modules_config - BBMAP_BBDUK found in conf/modules.config and Nextflow scripts.
• modules_config - MALT_RUN found in conf/modules.config and Nextflow scripts.
• modules_config - CAT_CAT_MALT found in conf/modules.config and Nextflow scripts.
• modules_config - KRAKEN2_KRAKEN2 found in conf/modules.config and Nextflow scripts.
• modules_config - KRAKENUNIQ_PRELOADEDKRAKENUNIQ found in conf/modules.config and Nextflow scripts.
• modules_config - METAPHLAN_METAPHLAN found in conf/modules.config and Nextflow scripts.
• modules_config - MALTEXTRACT found in conf/modules.config and Nextflow scripts.
• modules_config - MEGAN_RMA2INFO found in conf/modules.config and Nextflow scripts.
• modules_config - AMPS found in conf/modules.config and Nextflow scripts.
• modules_config - TAXPASTA_MERGE found in conf/modules.config and Nextflow scripts.
• modules_config - TAXPASTA_STANDARDISE found in conf/modules.config and Nextflow scripts.
• modules_config - DAMAGEPROFILER found in conf/modules.config and Nextflow scripts.
• modules_config - CALCULATE_MAPDAMAGE2 found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_DEPTH_SEXDETERRMINE found in conf/modules.config and Nextflow scripts.
• modules_config - SEXDETERRMINE found in conf/modules.config and Nextflow scripts.
• modules_config - BEDTOOLS_MASKFASTA found in conf/modules.config and Nextflow scripts.
• modules_config - MAPDAMAGE2 found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_INDEX_DAMAGE_RESCALED found in conf/modules.config and Nextflow scripts.
• modules_config - PMDTOOLS_FILTER found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_INDEX_DAMAGE_FILTERED found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_FLAGSTAT_DAMAGE_FILTERED found in conf/modules.config and Nextflow scripts.
• modules_config - BAMUTIL_TRIMBAM found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_INDEX_DAMAGE_TRIMMED found in conf/modules.config and Nextflow scripts.
• modules_config - PRINSEQPLUSPLUS found in conf/modules.config and Nextflow scripts.
• modules_config - BBMAP_BBDUK found in conf/modules.config and Nextflow scripts.
• modules_config - MALT_RUN found in conf/modules.config and Nextflow scripts.
• modules_config - CAT_CAT_MALT found in conf/modules.config and Nextflow scripts.
• modules_config - UNTAR_METAGENOMICS found in conf/modules.config and Nextflow scripts.
• modules_config - KRAKEN2_KRAKEN2 found in conf/modules.config and Nextflow scripts.
• modules_config - KRAKENUNIQ_PRELOADEDKRAKENUNIQ found in conf/modules.config and Nextflow scripts.
• modules_config - METAPHLAN_METAPHLAN found in conf/modules.config and Nextflow scripts.
• modules_config - MALTEXTRACT found in conf/modules.config and Nextflow scripts.
• modules_config - MEGAN_RMA2INFO found in conf/modules.config and Nextflow scripts.
• modules_config - AMPS found in conf/modules.config and Nextflow scripts.
• modules_config - TAXPASTA_MERGE found in conf/modules.config and Nextflow scripts.
• modules_config - TAXPASTA_STANDARDISE found in conf/modules.config and Nextflow scripts.
• modules_config - PICARD_ADDORREPLACEREADGROUPS found in conf/modules.config and Nextflow scripts.
• modules_config - SAMTOOLS_MPILEUP_PILEUPCALLER found in conf/modules.config and Nextflow scripts.
• modules_config - SEQUENCETOOLS_PILEUPCALLER found in conf/modules.config and Nextflow scripts.
• modules_config - COLLECT_GENOTYPES found in conf/modules.config and Nextflow scripts.
• modules_config - EIGENSTRATDATABASETOOLS_EIGENSTRATSNPCOVERAGE found in conf/modules.config and Nextflow scripts.
• modules_config - GATK_REALIGNERTARGETCREATOR found in conf/modules.config and Nextflow scripts.
• modules_config - GATK_INDELREALIGNER found in conf/modules.config and Nextflow scripts.
• modules_config - GATK_UNIFIEDGENOTYPER found in conf/modules.config and Nextflow scripts.
• modules_config - BCFTOOLS_INDEX_UG found in conf/modules.config and Nextflow scripts.
• modules_config - GATK4_HAPLOTYPECALLER found in conf/modules.config and Nextflow scripts.
• modules_config - FREEBAYES found in conf/modules.config and Nextflow scripts.
• modules_config - BCFTOOLS_INDEX_FREEBAYES found in conf/modules.config and Nextflow scripts.
• modules_config - BCFTOOLS_STATS_GENOTYPING found in conf/modules.config and Nextflow scripts.
• modules_config - ANGSD_GL found in conf/modules.config and Nextflow scripts.
• nfcore_yml - Repository type in .nf-core.yml is valid: pipeline
• nfcore_yml - nf-core version in .nf-core.yml is set to the latest version: 3.2.1

Run details

• nf-core/tools version 3.2.1
• Run at 2025-05-16 09:34:27

[ilight1542]

Got a working implementation for redeclaring the Read Group ID's. Needs test with multiple ref genomes/multiple libs from same sample.

Then on to ANGSD

[TCLamnidis]

Various small-ish changes.

The PR seems contains various changes from the nf-test conversion branch. Merging this in its current state would likely break testing for everything. Please either rebase/revert commits causing merging that history, OR we wait till #1063 is merged.

[ilight1542]

@TCLamnidis Please take another look -- I have merged in all the changes from dev. There were a few merge conflicts in the modules.config file which were due to naming changes when I generalized the sample merging post mapping to be able to be run separately for damage manipulation.

The CI nf-core test failed on MALT due to insufficient memory. Not sure how this has behaved for others.