big commit: add sage #346
Draft
big commit: add sage #346
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Wiz Scan Summary
To detect these findings earlier in the dev lifecycle, try using Wiz Code VS Code Extension. |
|
Bleep bloop, I am a robot. Alas, some of the Nextflow configuration tests failed! test/configtest-F16.json@ ["params","algorithm",["set"],{}]
+ "sage"
@ ["params","base_allocations","call_sSNV_SAGE"]
+ {"cpus":"8","memory":"16 GB"}
@ ["params","base_allocations","run_REDUX_SAGE_normal"]
+ {"cpus":"4","memory":"16 GB"}
@ ["params","base_allocations","run_REDUX_SAGE_tumor"]
+ {"cpus":"4","memory":"16 GB"}
@ ["params","ncbi_build"]
- "GRCh38"
@ ["params","redux_jitter_msi_only"]
- false
+ true
@ ["params","retry_information","call_sSNV_SAGE"]
+ {"memory":{"operand":"8 GB","strategy":"add"}}
@ ["params","retry_information","run_REDUX_SAGE_normal"]
+ {"memory":{"operand":"8 GB","strategy":"add"}}
@ ["params","retry_information","run_REDUX_SAGE_tumor"]
+ {"memory":{"operand":"8 GB","strategy":"add"}}
@ ["params","save_intermediate_files"]
- false
+ true
@ ["params","docker_image_redux"]
+ "ghcr.io/uclahs-cds/redux:branch-sfitz-initial-files"
@ ["params","docker_image_sage"]
+ "ghcr.io/uclahs-cds/sage:4.0"
@ ["params","redux_form_consensus"]
+ false
@ ["params","redux_ref_genome_msi_file"]
+ "/hot/user/sfitzgibbon/hmf_pipeline_resources.38_v2.0--3/dna/variants/msi_jitter_sites.38.tsv.gz"
@ ["params","redux_unmap_regions"]
+ ""
@ ["params","redux_version"]
+ "branch-sfitz-initial-files"
@ ["params","reference_version"]
+ "38"
@ ["params","sage_command_mem_diff"]
+ "3 GB"
@ ["params","sage_ensembl_data_dir"]
+ "/hot/user/sfitzgibbon/hmf_pipeline_resources.38_v2.0--3/common/ensembl_data"
@ ["params","sage_high_confidence_bed"]
+ "/hot/user/sfitzgibbon/hmf_pipeline_resources.38_v2.0--3/dna/variants/HG001_GRCh38_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-10X-SOLID_CHROM1-X_v.3.3.2_highconf_nosomaticdel_noCENorHET7.bed.gz"
@ ["params","sage_hotspots"]
+ "/hot/user/sfitzgibbon/hmf_pipeline_resources.38_v2.0--3/dna/variants/KnownHotspots.somatic.38.vcf.gz"
@ ["params","sage_panel_bed"]
+ "/hot/user/sfitzgibbon/hmf_pipeline_resources.38_v2.0--3/dna/variants/ActionableCodingPanel.38.bed.gz"
@ ["params","sage_version"]
+ "4.0"
@ ["process","withName:call_sSNV_SAGE"]
+ {"cpus":"8","ext":{"retry_codes":[]},"memory":{"1":"16 GB","2":"24 GB","3":"31 GB","closure":"retry_updater(16 GB, add, 8 GB, $task.attempt, memory)"}}
@ ["process","withName:run_REDUX_SAGE"]
+ {"ext":{"retry_codes":[]}}
@ ["process","withName:run_REDUX_SAGE_normal"]
+ {"cpus":"4","memory":{"1":"16 GB","2":"24 GB","3":"31 GB","closure":"retry_updater(16 GB, add, 8 GB, $task.attempt, memory)"}}
@ ["process","withName:run_REDUX_SAGE_tumor"]
+ {"cpus":"4","memory":{"1":"16 GB","2":"24 GB","3":"31 GB","closure":"retry_updater(16 GB, add, 8 GB, $task.attempt, memory)"}}
@ ["valid_algorithms",["set"],{}]
+ "sage"test/configtest-F32.json@ ["params","algorithm",["set"],{}]
+ "sage"
@ ["params","base_allocations","call_sSNV_SAGE"]
+ {"cpus":"2","memory":"24 GB"}
@ ["params","base_allocations","run_REDUX_SAGE_normal"]
+ {"cpus":"8","memory":"32 GB"}
@ ["params","base_allocations","run_REDUX_SAGE_tumor"]
+ {"cpus":"8","memory":"32 GB"}
@ ["params","ncbi_build"]
- "GRCh38"
@ ["params","redux_jitter_msi_only"]
- false
+ true
@ ["params","retry_information","call_sSNV_SAGE"]
+ {"memory":{"operand":"16 GB","strategy":"add"}}
@ ["params","retry_information","run_REDUX_SAGE_normal"]
+ {"memory":{"operand":"16 GB","strategy":"add"}}
@ ["params","retry_information","run_REDUX_SAGE_tumor"]
+ {"memory":{"operand":"16 GB","strategy":"add"}}
@ ["params","save_intermediate_files"]
- false
+ true
@ ["params","docker_image_redux"]
+ "ghcr.io/uclahs-cds/redux:branch-sfitz-initial-files"
@ ["params","docker_image_sage"]
+ "ghcr.io/uclahs-cds/sage:4.0"
@ ["params","redux_form_consensus"]
+ false
@ ["params","redux_ref_genome_msi_file"]
+ "/hot/user/sfitzgibbon/hmf_pipeline_resources.38_v2.0--3/dna/variants/msi_jitter_sites.38.tsv.gz"
@ ["params","redux_unmap_regions"]
+ ""
@ ["params","redux_version"]
+ "branch-sfitz-initial-files"
@ ["params","reference_version"]
+ "38"
@ ["params","sage_command_mem_diff"]
+ "3 GB"
@ ["params","sage_ensembl_data_dir"]
+ "/hot/user/sfitzgibbon/hmf_pipeline_resources.38_v2.0--3/common/ensembl_data"
@ ["params","sage_high_confidence_bed"]
+ "/hot/user/sfitzgibbon/hmf_pipeline_resources.38_v2.0--3/dna/variants/HG001_GRCh38_GIAB_highconf_CG-IllFB-IllGATKHC-Ion-10X-SOLID_CHROM1-X_v.3.3.2_highconf_nosomaticdel_noCENorHET7.bed.gz"
@ ["params","sage_hotspots"]
+ "/hot/user/sfitzgibbon/hmf_pipeline_resources.38_v2.0--3/dna/variants/KnownHotspots.somatic.38.vcf.gz"
@ ["params","sage_panel_bed"]
+ "/hot/user/sfitzgibbon/hmf_pipeline_resources.38_v2.0--3/dna/variants/ActionableCodingPanel.38.bed.gz"
@ ["params","sage_version"]
+ "4.0"
@ ["process","withName:call_sSNV_SAGE"]
+ {"cpus":"2","ext":{"retry_codes":[]},"memory":{"1":"24 GB","2":"40 GB","3":"56 GB","closure":"retry_updater(24 GB, add, 16 GB, $task.attempt, memory)"}}
@ ["process","withName:run_REDUX_SAGE"]
+ {"ext":{"retry_codes":[]}}
@ ["process","withName:run_REDUX_SAGE_normal"]
+ {"cpus":"8","memory":{"1":"32 GB","2":"48 GB","3":"64 GB","closure":"retry_updater(32 GB, add, 16 GB, $task.attempt, memory)"}}
@ ["process","withName:run_REDUX_SAGE_tumor"]
+ {"cpus":"8","memory":{"1":"32 GB","2":"48 GB","3":"64 GB","closure":"retry_updater(32 GB, add, 16 GB, $task.attempt, memory)"}}
@ ["valid_algorithms",["set"],{}]
+ "sage"If the above changes are surprising, stop and determine what happened. If the above changes are expected, there are two ways to fix this:
|
Contributor
Author
|
/fix-tests |
|
Bleep bloop, I am a robot. I have updated all of the failing tests for you with 97d1724. You must review my work before merging this pull request! |
|
Bleep bloop, I am a robot. Alas, some of the Nextflow configuration tests failed! test/configtest-F16.json@ ["params","sage_skip_msi_jitter"]
- false
@ ["params","redux_skip"]
+ falsetest/configtest-F32.json@ ["params","sage_skip_msi_jitter"]
- false
@ ["params","redux_skip"]
+ falseIf the above changes are surprising, stop and determine what happened. If the above changes are expected, there are two ways to fix this:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
PR Description: Add SAGE and REDUX Support to pipeline-call-sSNV
Summary
This PR adds comprehensive support for SAGE (Somatic and germline Aberration GEnotyper) and REDUX (Duplicate marking, consensus reads, UMIs and read unmapping) to the pipeline-call-sSNV Nextflow pipeline, expanding the available somatic SNV calling tools from four to five and introducing advanced read preprocessing capabilities.
What's Added
🧬 SAGE Somatic Variant Caller
🔧 REDUX Read Preprocessing
Implementation Details
New Configuration Parameters
New Modules and Processes
module/sage.nf: Main SAGE workflow orchestrationmodule/sage-processes.nf: Individual SAGE and REDUX process definitionsrun_REDUX_SAGE: Process for running REDUX preprocessing on tumor/normal BAMscall_sSNV_SAGE: Process for SAGE variant calling using preprocessed dataWorkflow Integration
algorithm = ['somaticsniper', 'strelka2', 'mutect2', 'muse', 'sage']Advanced Features
Testing and Validation
Benefits
Breaking Changes
None. This is a purely additive enhancement that maintains full backward compatibility.
Documentation Updates
Tool Versions:
Resource Files Required:
This enhancement significantly expands the pipeline's capabilities while maintaining the robust, scalable architecture that users expect from pipeline-call-sSNV.
Closes #331
Testing Results
Checklist
I have read the code review guidelines and the code review best practice on GitHub check-list.
I have reviewed the Nextflow pipeline standards.
The name of the branch is meaningful and well formatted following the standards, using [AD_username (or 5 letters of AD if AD is too long)]-[brief_description_of_branch].
I have set up or verified the branch protection rule following the github standards before opening this pull request.
I have added my name to the contributors listings in the
manifestblock in thenextflow.configas part of this pull request, am listed already, or do not wish to be listed. (This acknowledgement is optional.)I have added the changes included in this pull request to the
CHANGELOG.mdunder the next release version or unreleased, and updated the date.I have updated the version number in the
metadata.yamlandmanifestblock of thenextflow.configfile following semver, or the version number has already been updated. (Leave it unchecked if you are unsure about new version number and discuss it with the infrastructure team in this PR.)I have tested the pipeline on at least one A-mini sample.