Skip to content

Resource allocation tweaks #194

@yashpatel6

Description

@yashpatel6
          Looks good to me, although there are some tweaks I think could be made to improve resource usage.
  • The GATK tools are generally single processor. I'm pretty sure the cpus for all of them here can be set to min and max of 1. call-sSNV Mutect2 uses 1 for all partitions (even M64) and has been fine as far as I know. This will make the biggest impact for HaplotypeCaller which runs for ~45 minutes x 50 scatters and is currently set at 3 (I should have caught this earlier). Some of the others are set at 10 cpus but only run for a minute or so.
  • The memory allocations in the first line below (same for all partitions) could be reduced for the first try. Again the ones that run for only a couple minutes don't matter but two of them run typically for ~ 20m and 40m.
measure run_MergeVcfs_Picard_VCF run_MergeVcfs_Picard_GVCF run_VariantRecalibratorSNP_GATK run_VariantRecalibratorINDEL_GATK run_ApplyVQSR_GATK_SNP run_ApplyVQSR_GATK_INDEL
current mem 15 15 10 10 10 10
PCAWG min mem 5 5.3 2.2 1.2 0.6 0.5
PCAWG max mem 5.1 5.5 3.2 1.8 1.2 1.1
runtime 2m 40m 23m 1m 2m 1m
  • This new method of viewing resource allocations really helps with this! I created a tsv for call-gSNPs current settings,

Originally posted by @sorelfitzgibbon in #189 (review)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions