Skip to content

Dsl2 new test configs #1121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 67 commits into from
May 2, 2025
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
67 commits
Select commit Hold shift + click to select a range
f8b0d2b
adding modern test
jbv2 Feb 28, 2025
e44842f
adding microbial genome test, but missing samplesheet dsl2 to test
jbv2 Feb 28, 2025
24cc109
adding short dna, still need the ncbi dir to be set
jbv2 Mar 7, 2025
ead8ada
adding test_humanpopgen
jbv2 Mar 7, 2025
66aafdf
adding test default
jbv2 Mar 7, 2025
2689873
minor updates
jbv2 Mar 7, 2025
981cd92
finnished v1 test configs
jbv2 Mar 14, 2025
3597d6f
finnished but error with mapDamage --rescale. Mixing different refere…
jbv2 Mar 14, 2025
67bde36
removing old profiles
jbv2 Mar 24, 2025
56a3ab5
replacing test.config
jbv2 Mar 24, 2025
ff452d9
replacing nothing to minimal
jbv2 Mar 24, 2025
dd12a5b
adding genotyping source
jbv2 Mar 24, 2025
2c0bb13
update nextflow.config
jbv2 Mar 24, 2025
e10099b
Merge branch 'dev' into dsl2-new_test_configs
jfy133 Mar 25, 2025
57809e5
Update conf/test.config
jfy133 Mar 25, 2025
636e5d5
Update conf/test.config
jfy133 Mar 25, 2025
b08e6ac
Move kraken2 to humanpopgen
jfy133 Mar 25, 2025
ccf180c
removing fastp
jbv2 Mar 25, 2025
b1bba76
adding fastp and changing input and fasta
jbv2 Mar 25, 2025
52df92b
excluding fasta
jbv2 Mar 25, 2025
22433a9
Deactivate rescaling, fix freebayes problem
jfy133 Mar 25, 2025
030e28d
Merge branch 'dsl2-new_test_configs' of https://github.com/jbv2/eager…
jfy133 Mar 25, 2025
040ceea
Merge branch 'dev' into dsl2-new_test_configs
jfy133 Mar 26, 2025
89262a2
Update conf/modules.config
jbv2 Mar 28, 2025
04c75a1
Update conf/modules.config
jbv2 Mar 28, 2025
23c110f
Merge branch 'dev' into dsl2-new_test_configs
jbv2 Mar 28, 2025
13bc21a
Update conf/modules.config
jbv2 Mar 28, 2025
396fd25
And for haplotypecaller
jfy133 Mar 28, 2025
d73f2e9
Merge branch 'dsl2-new_test_configs' of https://github.com/jbv2/eager…
jfy133 Mar 28, 2025
2051309
Use correct GATK UG ploidy flag and remove duplicate
jfy133 Apr 4, 2025
8a471e3
Use new test configs in ci test
jfy133 Apr 4, 2025
f8a388f
Fix parameter for GATK HC too
jfy133 Apr 4, 2025
9f1c719
Replace minimal test data with mammoth mt data to run faster
jfy133 Apr 4, 2025
f48cfeb
Fix HC caller command as they renamed the parameter
jfy133 Apr 4, 2025
da8718c
Limit MALT memory
jfy133 Apr 4, 2025
f42015f
removing double test profiles
jbv2 Apr 11, 2025
f5e2a54
adding missing rescale step
jbv2 Apr 11, 2025
9184a90
not rescaling
jbv2 Apr 11, 2025
7c60084
add test_long.config
jbv2 Apr 11, 2025
dfc7ec8
final rev
jbv2 Apr 11, 2025
5f5c10a
changing mapping tool
jbv2 Apr 11, 2025
59228bb
getting manipulate damage from eager/dev
jbv2 Apr 11, 2025
fb3be94
getting manipulate damage from eager/dev
jbv2 Apr 11, 2025
834c55b
increasing time for test long
jbv2 Apr 11, 2025
b88568f
Merge branch 'dev' into dsl2-new_test_configs
jbv2 Apr 11, 2025
a04858f
fixing problem with merging mapad
jbv2 Apr 25, 2025
d5387aa
updating short_dna to run map AD
jbv2 Apr 25, 2025
b134e95
removing params from CI
jbv2 Apr 25, 2025
24bfdaa
latest_everything
jbv2 Apr 25, 2025
4ca1cfe
changing input to run mapad
jbv2 Apr 25, 2025
4258406
adding rescaled as source for genotyping
jbv2 Apr 25, 2025
ed59eb4
swap to fastp
TCLamnidis Apr 25, 2025
2e96488
add adapterlist
TCLamnidis Apr 25, 2025
a243300
specifying elongation factor
jbv2 Apr 25, 2025
4a30622
fix typo
TCLamnidis Apr 25, 2025
f124ab8
add mapdamage downsampling
TCLamnidis Apr 25, 2025
ddc97b7
Merge branch 'dsl2-new_test_configs' of https://github.com/jbv2/eager…
jbv2 Apr 25, 2025
187ee0a
Merge branch 'dsl2-new_test_configs' of https://github.com/jbv2/eager…
jbv2 Apr 25, 2025
52a940f
fix profile names in commented commands
TCLamnidis Apr 25, 2025
9810e80
Update .github/workflows/ci.yml
jfy133 Apr 25, 2025
fdd005d
Revert "swap to fastp"
TCLamnidis Apr 25, 2025
85e4a94
swap to adapterremoval
TCLamnidis Apr 25, 2025
eb2b330
add snpcapturebed
TCLamnidis Apr 25, 2025
b72c6bd
move java opts to beforescript
TCLamnidis May 2, 2025
0318736
Merge branch 'dev' into dsl2-new_test_configs
TCLamnidis May 2, 2025
737b4ed
re-add latest-everything
TCLamnidis May 2, 2025
3f9145b
Update subworkflows/local/genotype.nf
TCLamnidis May 2, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
825 changes: 369 additions & 456 deletions conf/modules.config

Large diffs are not rendered by default.

47 changes: 32 additions & 15 deletions conf/test.config
Original file line number Diff line number Diff line change
Expand Up @@ -19,30 +19,47 @@ process {
}

params {
config_profile_name = 'Test profile'
config_profile_description = 'Minimal test dataset to check pipeline function'
config_profile_name = 'Test profile'
config_profile_description = 'Minimal test dataset to check pipeline function'

// Input data
// TODO nf-core: Specify the paths to your test data on nf-core/test-datasets
// TODO nf-core: Give any required params for the test so that command line flags are not needed
input = params.pipelines_testdata_base_path + 'eager/testdata/Mammoth/samplesheet_v3.tsv'
input = params.pipelines_testdata_base_path + 'eager/testdata/Mammoth/samplesheet_v3.tsv'

// Genome references
fasta = params.pipelines_testdata_base_path + 'eager/reference/Mammoth/Mammoth_MT_Krause.fasta'
fasta = params.pipelines_testdata_base_path + 'eager/reference/Mammoth/Mammoth_MT_Krause.fasta'

// Preprocessing
preprocessing_tool = 'adapterremoval'

// Sharding FASTQ
run_fastq_sharding = true
fastq_shard_size = 5000
run_fastq_sharding = true
fastq_shard_size = 5000

// Mapping
mapping_tool = 'bwaaln'
skip_qualimap = false

// BAM filtering
run_bamfiltering = true
bamfiltering_minreadlength = 30
bamfiltering_mappingquality = 37
run_bamfiltering = true
bamfiltering_minreadlength = 30
bamfiltering_mappingquality = 37
deduplication_tool = 'markduplicates'

// PreSeq
mapstats_preseq_mode = 'c_curve'

// Damage calculation
damagecalculation_tool = 'damageprofiler'
skip_qualimap = false

// Map Stats
run_bedtools_coverage = true
mapstats_bedtools_featurefile = params.pipelines_testdata_base_path + 'eager/reference/Mammoth/Mammoth_MT_Krause.gff3'
// Genotyping
run_genotyping = true
genotyping_source = 'raw'
genotyping_tool = 'ug'

// Metagenomic screening
run_metagenomics = false
run_metagenomics = true
metagenomics_profiling_tool = 'metaphlan'
metagenomics_profiling_database = params.pipelines_testdata_base_path + 'eager/databases/metaphlan/metaphlan4_database.tar.gz'
metagenomics_run_postprocessing = true
}
55 changes: 0 additions & 55 deletions conf/test_humanbam.config

This file was deleted.

70 changes: 70 additions & 0 deletions conf/test_humanpopgen.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Nextflow config file for running minimal tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Defines input files and everything required to run a fast and simple pipeline test.

Use as follows:
nextflow run nf-core/eager -profile test_humanpopgen,<docker/singularity> --outdir <OUTDIR>

----------------------------------------------------------------------------------------
*/

process {
resourceLimits = [
cpus: 4,
memory: '15.GB',
time: '1.h',
]

// To avoid pipeline failure due to not having X reads and to not have overcrowded datasets in the test
withName: ANGSD_CONTAMINATION {
errorStrategy = { task.exitStatus in [134] ? 'ignore' : 'finish' }
}
}

params {
config_profile_name = 'Test human popgen profile'
config_profile_description = 'Minimal test dataset to check pipeline function'

// Input data
input = params.pipelines_testdata_base_path + 'eager/testdata/Human/human_design_bam_eager3.tsv'

// Genome references
fasta = params.pipelines_testdata_base_path + 'eager/reference/Human/hs37d5_chr21-MT.fa.gz'

// Mapping
mapping_tool = 'bowtie2'
convert_inputbam = true

// BAM filtering
run_bamfiltering = true
bamfiltering_minreadlength = 30
bamfiltering_mappingquality = 37

// Damage
damagecalculation_tool = 'mapdamage'
run_trim_bam = true

// Contamination
run_mtnucratio = true
run_contamination_estimation_angsd = true

// Genotyping
genotyping_tool = 'pileupcaller'
run_genotyping = true
genotyping_source = 'trimmed'
genotyping_pileupcaller_bedfile = params.pipelines_testdata_base_path + 'eager/reference/Human/1240K.pos.list_hs37d5.0based.bed.gz'
genotyping_pileupcaller_snpfile = params.pipelines_testdata_base_path + 'eager/reference/Human/1240K_covered_in_JK2067_downsampled_s0.1.numeric_chromosomes.snp'

//Sex Determination
run_sexdeterrmine = true
sexdeterrmine_bedfile = params.pipelines_testdata_base_path + 'eager/reference/Human/1240K.pos.list_hs37d5.0based.bed.gz'

// Metagenomics
run_metagenomics = true
metagenomics_complexity_tool = 'prinseq'
metagenomics_profiling_tool = 'kraken2'
metagenomics_profiling_database = params.pipelines_testdata_base_path + 'eager/databases/kraken/eager_test.tar.gz'
metagenomics_run_postprocessing = true
}
35 changes: 0 additions & 35 deletions conf/test_kraken2.config

This file was deleted.

36 changes: 0 additions & 36 deletions conf/test_malt.config

This file was deleted.

37 changes: 0 additions & 37 deletions conf/test_metaphlan.config

This file was deleted.

61 changes: 61 additions & 0 deletions conf/test_microbial.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Nextflow config file for running minimal tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Defines input files and everything required to run a fast and simple pipeline test.

Use as follows:
nextflow run nf-core/eager -profile test_microbial,<docker/singularity> --outdir <OUTDIR>

----------------------------------------------------------------------------------------
*/

process {
resourceLimits = [
cpus: 4,
memory: '15.GB',
time: '1.h',
]
}

params {
config_profile_name = 'Test microbial profile'
config_profile_description = 'Minimal test dataset to check pipeline function'

// Input data
input = params.pipelines_testdata_base_path + 'eager/testdata/Mammoth/samplesheet_PE_only_v3.tsv'

// Genome references
fasta_sheet = params.pipelines_testdata_base_path + 'eager/reference/reference_sheet_multiref.csv'

// Host Removal
run_host_removal = true

// Preprocessing
sequencing_qc_tool = 'falco'
preprocessing_tool = 'fastp'
preprocessing_excludeunmerged = true

// Mapping
mapping_tool = 'circularmapper'

// BAM filtering
deduplication_tool = "dedup"
run_bamfiltering = true
bamfiltering_minreadlength = 30
bamfiltering_mappingquality = 37

// Metagenomics
run_metagenomics = true
metagenomics_profiling_tool = 'krakenuniq'
metagenomics_profiling_database = params.pipelines_testdata_base_path + 'eager/databases/krakenuniq/testdb-krakenuniq.tar.gz'
run_host_removal = true

// Bedtools coverage
run_bedtools_coverage = true

// Genotyping
run_genotyping = true
genotyping_source = 'raw'
genotyping_tool = 'freebayes'
}
File renamed without changes.
Loading