Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dsl2 new test configs #1121

Open
wants to merge 45 commits into
base: dev
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
f8b0d2b
adding modern test
jbv2 Feb 28, 2025
e44842f
adding microbial genome test, but missing samplesheet dsl2 to test
jbv2 Feb 28, 2025
24cc109
adding short dna, still need the ncbi dir to be set
jbv2 Mar 7, 2025
ead8ada
adding test_humanpopgen
jbv2 Mar 7, 2025
66aafdf
adding test default
jbv2 Mar 7, 2025
2689873
minor updates
jbv2 Mar 7, 2025
981cd92
finnished v1 test configs
jbv2 Mar 14, 2025
3597d6f
finnished but error with mapDamage --rescale. Mixing different refere…
jbv2 Mar 14, 2025
67bde36
removing old profiles
jbv2 Mar 24, 2025
56a3ab5
replacing test.config
jbv2 Mar 24, 2025
ff452d9
replacing nothing to minimal
jbv2 Mar 24, 2025
dd12a5b
adding genotyping source
jbv2 Mar 24, 2025
2c0bb13
update nextflow.config
jbv2 Mar 24, 2025
e10099b
Merge branch 'dev' into dsl2-new_test_configs
jfy133 Mar 25, 2025
57809e5
Update conf/test.config
jfy133 Mar 25, 2025
636e5d5
Update conf/test.config
jfy133 Mar 25, 2025
b08e6ac
Move kraken2 to humanpopgen
jfy133 Mar 25, 2025
ccf180c
removing fastp
jbv2 Mar 25, 2025
b1bba76
adding fastp and changing input and fasta
jbv2 Mar 25, 2025
52df92b
excluding fasta
jbv2 Mar 25, 2025
22433a9
Deactivate rescaling, fix freebayes problem
jfy133 Mar 25, 2025
030e28d
Merge branch 'dsl2-new_test_configs' of https://github.com/jbv2/eager…
jfy133 Mar 25, 2025
040ceea
Merge branch 'dev' into dsl2-new_test_configs
jfy133 Mar 26, 2025
89262a2
Update conf/modules.config
jbv2 Mar 28, 2025
04c75a1
Update conf/modules.config
jbv2 Mar 28, 2025
23c110f
Merge branch 'dev' into dsl2-new_test_configs
jbv2 Mar 28, 2025
13bc21a
Update conf/modules.config
jbv2 Mar 28, 2025
396fd25
And for haplotypecaller
jfy133 Mar 28, 2025
d73f2e9
Merge branch 'dsl2-new_test_configs' of https://github.com/jbv2/eager…
jfy133 Mar 28, 2025
2051309
Use correct GATK UG ploidy flag and remove duplicate
jfy133 Apr 4, 2025
8a471e3
Use new test configs in ci test
jfy133 Apr 4, 2025
f8a388f
Fix parameter for GATK HC too
jfy133 Apr 4, 2025
9f1c719
Replace minimal test data with mammoth mt data to run faster
jfy133 Apr 4, 2025
f48cfeb
Fix HC caller command as they renamed the parameter
jfy133 Apr 4, 2025
da8718c
Limit MALT memory
jfy133 Apr 4, 2025
f42015f
removing double test profiles
jbv2 Apr 11, 2025
f5e2a54
adding missing rescale step
jbv2 Apr 11, 2025
9184a90
not rescaling
jbv2 Apr 11, 2025
7c60084
add test_long.config
jbv2 Apr 11, 2025
dfc7ec8
final rev
jbv2 Apr 11, 2025
5f5c10a
changing mapping tool
jbv2 Apr 11, 2025
59228bb
getting manipulate damage from eager/dev
jbv2 Apr 11, 2025
fb3be94
getting manipulate damage from eager/dev
jbv2 Apr 11, 2025
834c55b
increasing time for test long
jbv2 Apr 11, 2025
b88568f
Merge branch 'dev' into dsl2-new_test_configs
jbv2 Apr 11, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 33 additions & 15 deletions conf/test.config
Original file line number Diff line number Diff line change
Expand Up @@ -18,31 +18,49 @@ process {
]
}

// TO DO: Change name to test.config once migration is complete.
params {
config_profile_name = 'Test profile'
config_profile_description = 'Minimal test dataset to check pipeline function'
config_profile_name = 'Test profile'
config_profile_description = 'Minimal test dataset to check pipeline function'

// Input data
// TODO nf-core: Specify the paths to your test data on nf-core/test-datasets
// TODO nf-core: Give any required params for the test so that command line flags are not needed
input = params.pipelines_testdata_base_path + 'eager/testdata/Mammoth/samplesheet_v3.tsv'
input = params.pipelines_testdata_base_path + 'eager/testdata/Mammoth/samplesheet_v3.tsv'

// Genome references
fasta = params.pipelines_testdata_base_path + 'eager/reference/Mammoth/Mammoth_MT_Krause.fasta'
fasta = params.pipelines_testdata_base_path + 'eager/reference/Mammoth/Mammoth_MT_Krause.fasta'

// Preprocessing
preprocessing_tool = 'adapterremoval'

// Sharding FASTQ
run_fastq_sharding = true
fastq_shard_size = 5000
run_fastq_sharding = true
fastq_shard_size = 5000

// Mapping
mapping_tool = 'bwaaln'
skip_qualimap = false

// BAM filtering
run_bamfiltering = true
bamfiltering_minreadlength = 30
bamfiltering_mappingquality = 37
run_bamfiltering = true
bamfiltering_minreadlength = 30
bamfiltering_mappingquality = 37
deduplication_tool = 'markduplicates'

// PreSeq
mapstats_preseq_mode = 'c_curve'

// Damage calculation
damagecalculation_tool = 'damageprofiler'
skip_qualimap = false

// Map Stats
run_bedtools_coverage = true
mapstats_bedtools_featurefile = params.pipelines_testdata_base_path + 'eager/reference/Mammoth/Mammoth_MT_Krause.gff3'
// Genotyping
run_genotyping = true
genotyping_source = 'raw'
genotyping_tool = 'ug'

// Metagenomic screening
run_metagenomics = false
run_metagenomics = true
metagenomics_profiling_tool = 'metaphlan'
metagenomics_profiling_database = params.pipelines_testdata_base_path + 'eager/databases/metaphlan/metaphlan4_database.tar.gz'
metagenomics_run_postprocessing = true
}
55 changes: 0 additions & 55 deletions conf/test_humanbam.config

This file was deleted.

64 changes: 64 additions & 0 deletions conf/test_humanpopgen.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Nextflow config file for running minimal tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Defines input files and everything required to run a fast and simple pipeline test.

Use as follows:
nextflow run nf-core/eager -profile test_humanpopgen,<docker/singularity> --outdir <OUTDIR>

----------------------------------------------------------------------------------------
*/

process {
resourceLimits = [
cpus: 4,
memory: '15.GB',
time: '1.h'
]

// To avoid pipeline failure due to not having X reads and to not have overcrowded datasets in the test
withName: ANGSD_CONTAMINATION {
errorStrategy = { task.exitStatus in [134] ? 'ignore' : 'finish' }
}
}

params {
config_profile_name = 'Test human popgen profile'
config_profile_description = 'Minimal test dataset to check pipeline function'

// Input data
input = params.pipelines_testdata_base_path + 'eager/testdata/Human/human_design_bam_eager3.tsv'

// Genome references
fasta = params.pipelines_testdata_base_path + 'eager/reference/Human/hs37d5_chr21-MT.fa.gz'

// Mapping
mapping_tool = 'bowtie2'
convert_inputbam = true

// BAM filtering
run_bamfiltering = true
bamfiltering_minreadlength = 30
bamfiltering_mappingquality = 37

// Damage
damagecalculation_tool = 'mapdamage'
run_trim_bam = true

// Contamination
run_mtnucratio = true
run_contamination_estimation_angsd = true

// Genotyping
genotyping_tool = 'pileupcaller'
run_genotyping = true
genotyping_source = 'trimmed'
genotyping_pileupcaller_bedfile = params.pipelines_testdata_base_path + 'eager/reference/Human/1240K.pos.list_hs37d5.0based.bed.gz'
genotyping_pileupcaller_snpfile = params.pipelines_testdata_base_path + 'eager/reference/Human/1240K_covered_in_JK2067_downsampled_s0.1.numeric_chromosomes.snp'

//Sex Determination
run_sexdeterrmine = true
sexdeterrmine_bedfile = params.pipelines_testdata_base_path + 'eager/reference/Human/1240K.pos.list_hs37d5.0based.bed.gz'

}
36 changes: 0 additions & 36 deletions conf/test_krakenuniq.config

This file was deleted.

36 changes: 0 additions & 36 deletions conf/test_malt.config

This file was deleted.

37 changes: 0 additions & 37 deletions conf/test_metaphlan.config

This file was deleted.

65 changes: 65 additions & 0 deletions conf/test_microbial.config
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Nextflow config file for running minimal tests
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Defines input files and everything required to run a fast and simple pipeline test.

Use as follows:
nextflow run nf-core/eager -profile test_microbial,<docker/singularity> --outdir <OUTDIR>

----------------------------------------------------------------------------------------
*/

process {
resourceLimits = [
cpus: 4,
memory: '15.GB',
time: '1.h'
]
}

params {
config_profile_name = 'Test microbial profile'
config_profile_description = 'Minimal test dataset to check pipeline function'

// Input data
input = params.pipelines_testdata_base_path + 'eager/testdata/Mammoth/samplesheet_PE_only_v3.tsv'

// Genome references
fasta_sheet = params.pipelines_testdata_base_path + 'eager/reference/reference_sheet_multiref.csv'

// Host Removal
run_host_removal = true

// Preprocessing
sequencing_qc_tool = 'falco'
preprocessing_excludeunmerged = true

// Mapping
mapping_tool = 'circularmapper'

// BAM filtering
deduplication_tool = "dedup"
run_bamfiltering = true
bamfiltering_minreadlength = 30
bamfiltering_mappingquality = 37

// Metagenomics
run_metagenomics = true
metagenomics_profiling_tool = 'krakenuniq'
metagenomics_profiling_database = params.pipelines_testdata_base_path + 'eager/databases/krakenuniq/testdb-krakenuniq.tar.gz'
run_host_removal = true

// Manipulate Damage
run_mapdamage_rescaling = true

// Bedtools coverage
run_bedtools_coverage = true

// Genotyping
run_genotyping = true
genotyping_source = 'rescaled'
genotyping_tool = 'freebayes'


}
File renamed without changes.
Loading
Loading