v2.4.1.1

What's Changed

• Fix GH Actions builds by @dpark01 in #595
• metagenomic denovo optimizations and fixes by @dpark01 in #594

Full Changelog: v2.4.1.0...v2.4.1.1

v2.4.1.0

What's Changed

New workflows or functionality

• add workflow "unpack_archive_to_bucket"; add task "unpack_archive_to_bucket_path" by @tomkinsc in #573
• assembly and genbank updates by @dpark01 in #579
• NCBI submission workflow updates, new workflow assemble_denovo_metagenomic, submit_biosample, submit_sra, submit_genbank. Prune vestigal workflows demux_metag, taxid_to_nextclade, contigs. by @dpark01 in #583

Workflow edits and fixes

• s/Array[File]+/Array[File?]+/g for tsv_files input in terra_tsv_to_table.wdl by @tomkinsc in #563
• bugfix: demux_deplete with biosample attributes by @dpark01 in #572
• scaffold_and_refine_multitaxa: add more output columns by @dpark01 in #574
• update workflows for genbank submission by @dpark01 in #576
• genbank bug fixes by @dpark01 in #581
• demux and nextclade updates by @dpark01 in #582
• genbank_single: made mapped bam input optional by @tomkinsc in #584
• Optimizations for Genbank submission of coinfections by @dpark01 in #587
• hotfix by @dpark01 in #590

New options, inputs, or outputs

• demux_deplete: allow multiple biosample attribute files by @tomkinsc in #561
• add contextual fasta input for augur_from_assemblies by @tomkinsc in #558
• add deduplication of sequences by ID in augur_from_assemblies by @tomkinsc in #564
• Expose --timetree parameter in refine_augur_tree by @lakras in #570
• Exposed --root-sequence parameter of augur ancestral by @lakras in #585
• report pct_top_hit_of_total_reads as one of the outputs of align_and_… by @tomkinsc in #577

Infrastructural updates

• update terra introspection to support GCP Batch backend by @tomkinsc in #575
• fix check_terra_env by @dpark01 in #589
• add task: download_from_url() to download a URL to a file by @tomkinsc in #562

Dependency changes

• update viral-core 2.3.6 -> 2.4.0 by @tomkinsc in #566
• bump viral core 2.4.0 -> 2.4.1 by @tomkinsc in #569
• bump pangolin and nextclade by @tomkinsc in #565
• bump viral-classify 2.2.4.2 -> 2.2.5 by @tomkinsc in #568
• remove exemption from global version pin for viral-core image used in check_terra_env; update viral-core spec of the task to 2.4.0 by @tomkinsc in #571

Build changes

• limit GH build concurrency by @tomkinsc in #549
• cleanup .dockstore.yml by @dpark01 in #578

Full Changelog: v2.3.6.0...v2.4.1.0

v2.3.6.0

What's Changed

enterics related

• update enterics visualization docker image by @jpramosbarroso in #516
• Make create_enterics_qc_viz whitespace tolerant by @dpark01 in #517
• update personal branch by @jpramosbarroso in #520
• create_enterics_qc_viz simplify and improve portability by @jpramosbarroso in #521
• update docker for try/except block with new error message by @jpramosbarroso in #522

assembly scaffolding & reference selection improvements

• scaffold_and_refine_multitaxa -- more thorough assembly outputs by @dpark01 in #524
• scaffolding and reference selection based on ANI by @dpark01 in #528
• Reference selection improvements and nextclade generalization by @dpark01 in #529
• scaffolding regression fixes plus docker updates by @dpark01 in #547
• scaffold multitaxa: drop empty assembly output rows by @dpark01 in #554
• report reference genome length in align_reads output; use this value for assemble_refbased. reference_genome_length by @tomkinsc in #538

demux_deplete and terra table data model updates

• demux_deplete: make depletion optional, load biosample metadata to tables by @dpark01 in #551
• demux_deplete small updates by @dpark01 in #552
• demux_deplete fixes/updates by @dpark01 in #555

miscellaneous fixes and edge cases

• allow additional options for picard in FastqToUBAM by @tomkinsc in #518
• WDL code/style cleanups by @dpark01 in #523
• expanded parameterization of align_and_count and additional output metrics by @tomkinsc in #525
• guard against shell pipefails in align_and_count in instances where empty inputs would result in div-by-zero conditions by @tomkinsc in #535
• classify_single: parameterize taxa to deplete by @dpark01 in #536
• bugfix fetch_sra_to_bam by @tomkinsc in #542
• Update polyphonia input parameters and defaults by @lakras in #545

docker container updates

• build and container updates by @dpark01 in #526
• update viral-core 2.3.0 -> 2.3.1 by @tomkinsc in #527
• increase default instance memory for kraken2 task 72 -> 90GB by @tomkinsc in #540
• nextclade updates by @dpark01 in #557
• bump upstream docker images by @dpark01 in #556

build & CI related

• add build environment spec to .readthedocs.yml config by @tomkinsc in #541
• documentation build fixes by @tomkinsc in #546
• bugfix dx deploy failures by @tomkinsc in #548

Full Changelog: v2.2.4.0...v2.3.6.0

v2.2.4.0

What's Changed

New workflows

• Enterics QC visualizations
  • WDL for creating Enterics Visualizations by @schaluva in #500
  • create generalized version of enterics visualization WDL by @schaluva in #501
  • Visualization HTML PR by @jpramosbarroso in #513
  • Fixing duplicate inputs bug by @jpramosbarroso in #514
  • integrating threshold file by @jpramosbarroso in #515
• Terra-on-GCP-specific features
  • add check_terra_env task by @tomkinsc in #497
  • add workflows to add to or create library and sample tables based on demultiplexing output by @tomkinsc in #507
  • mark check_terra_env task as volatile by @tomkinsc in #510
  • include user email among outputs of check_terra_env task by @tomkinsc in #508
• transmission reconstruction from @ispecht
  • add reconstructR workflow by @dpark01 in #504
• mass-scale metagenomic denovo assembly
  • add new workflow scaffold_and_refine_multitaxa by @dpark01 in #506
  • more scaffolding updates by @dpark01 in #511

Updates to existing workflows

• pass through sbt template if optional defaults_yaml is not provided and author_list string is empty by @tomkinsc in #492
• mark downsample_bams task as volatile (nondeterministic) by @tomkinsc in #494
• expose numberOfNegativeControls in illumina_demux by @tomkinsc in #498
• add masked_positions and masked_positions_file arguments to task polyphonia_detect_cross_contamination by @lakras in #499
• fix mem high watermark reporting by @tomkinsc in #502
• mass update of upstream docker tags by @dpark01 in #505
  • broadinstitute/viral-core: 2.1.33 -> 2.2.4
  • broadinstitute/viral-classify: 2.1.33.0 -> 2.2.3.0 (later bumped to 2.2.4.0 in #511)
  • broadinstitute/viral-baseimage: 0.1.20 -> 0.2.0
  • staphb/vadr: 1.5.1 -> 1.6.3
  • staphb/pangolin: 4.3.1-pdata-1.22 -> 4.3.1-pdata-1.23.1
  • nextstrain/nextclade: 2.12.0 -> 2.14.0

Build and CI

• enable merge_group builds in github actions CI by @dpark01 in #503
• allow docker image pinning to be skipped with #skip-global-version-pin annotation by @tomkinsc in #493
• dockstore updates by @dpark01 in #512
• clean up disk space in Github Actions VMs #505

New Contributors

• @jpramosbarroso made their first contribution in #513
• @ispecht 's code is utilized in a new workflow for #504

Full Changelog: v2.1.33.17...v2.2.4.0

v2.1.33.17

What's Changed

new features or functionality

• extend beast task and workflow to glob and output *.ops, *.rates, and *.root files by @tomkinsc in #478
• Subsampler and augur workflow by @dpark01 in #483

bug fixes and small changes

• syntax bugfixes by @dpark01 in #462
• tweaks to demux_only and demux_metag by @dpark01 in #464
• omit --dirs from the MultiQC invocation to only show sample names in report by @tomkinsc in #467
• hotfix for bug in output path by @dpark01 in #480
• sc2 nextstrain: reduce temp file sizes by @dpark01 in #484
• Fetch_SRA_to_BAM - tolerate no isolate defined by @dpark01 in #489
• Update genbank.WDL so author_list input is optional by @tomkinsc in #473

documentation updates

• Enhancing documentation for viral-pipelines by @golu099 in #461
• move transposed parameter_meta block outside command block scope to where it belongs by @tomkinsc in #477
• Update LICENSE copyright year by @tomkinsc in #476

parameterization and default VM shapes

• de novo assembly & scaffold mem default 31 -> 63 GB by @tomkinsc in #465
• parameterize disk size for illumina_demux task by @tomkinsc in #470
• max_records_in_ram param 2000000 -> 10000000 for Nova S2 by @tomkinsc in #468

upstream docker tags / version dependencies

• bump pangolin by @dpark01 in #482

build updates and fixes

• delete the .travis directory by @tomkinsc in #448
• dnanexus CI build fixes by @dpark01 in #485 #486 #487 #488
• GitHub actions update by @tomkinsc in #474

Full Changelog: v2.1.33.16...v2.1.33.17

v2.1.33.16

What's Changed

functionality changes and bug fixes

• Convert amplicon16S_analysis workflow (QIIME) from single-sample to multi-sample by @golu099 in #452
• samtools ampliconstats needs bed in certain order by @dpark01 in #454

parameterization and defaults

• parameterize and update defaults for VM shapes in nextstrain tasks by @dpark01 in #455
• parameterize more assembly knobs by @dpark01 in #457

update docker tags / version dependencies

• update to pangolin 4.2 / pdata 1.18 by @dpark01 in #451
• Update pangolin, nextclade, and vadr docker images by @dpark01 in #456

build updates and fixes

• documentation build fixes for ReadTheDocs by @tomkinsc in #450
• update comment about DX_API_TOKEN by @tomkinsc in #449

Full Changelog: v2.1.33.15...v2.1.33.16

v2.1.33.15

What's Changed

new and changed workflows

• Add workflow for QIIME2 classification of 16S reads by @golu099 in #439
• simplify augur_from_assemblies workflow by @dpark01 in #440
• denovo assembly SPAdes updates by @dpark01 in #441
• genbank prep and assemble_denovo updates by @dpark01 in #443
• add new workflow nextclade_single by @dpark01 in #444
• update classify_single to accept multiple bams by @dpark01 in #447

smaller docker, VM, and mechanical updates

• more disk for nextstrain; dockstore API fix by @dpark01 in #437
• Make Pipelines Azure compatible Closes #143 by @vsmalladi in #438
• set default GPU to K80->P4 in BEAST task; parameterize beagle_order by @tomkinsc in #442
• pangolin 4.1.3 / pdata 1.17 by @dpark01 in #445

New Contributors

• @vsmalladi made their first contribution in #438
• @golu099 made their first contribution in #439

Full Changelog: v2.1.33.14...v2.1.33.15

v2.1.33.14

What's Changed

Feature changes

• task alignment_metrics: more resilient to large amplicon designs by @dpark01 in #435

Docker image updates

• update nextclade and vadr docker images by @dpark01 in #433
• bump pangolin to 4.1.2-1.14 and nextclade to 2.5.0 by @dpark01 in #434

Full Changelog: v2.1.33.13...v2.1.33.14

v2.1.33.13

What's Changed

Feature changes

• CDC SC2 viral pipeline updates (parameters, reporting, QC) by @dpark01 in #431
• expose exclude/drop list input for nextstrain_build_subsample task by @tomkinsc in #428

Docker image updates

• nextclade 2.0 updates by @dpark01 in #430
• update pangolin docker to 4.1.1-pdata-1.11 by @dpark01 in #432

VM shape tuning

• parameterize machine_mem_gb for ancestral_traits task by @tomkinsc in #427
• in export_auspice_json parameterize task mem, raise default to 64GB; increase AUGUR_RECURSION_LIMIT from 10k to 15k by @tomkinsc in #429

Full Changelog: v2.1.33.12...v2.1.33.13

v2.1.33.12

What's Changed

• lofreq task portability, misc other edits by @dpark01 in #426

Full Changelog: v2.1.33.11...v2.1.33.12