Merge pull request #62 from sbslee/0.16.0-dev

0.16.0 dev

Author: sbslee

CHANGELOG.rst

@@ -1,6 +1,26 @@
 Changelog
 *********
 
+0.16.0 (2022-06-08)
+-------------------
+
+* Add new optional argument ``--comparison-table`` to :command:`train-cnv-caller` and :command:`test-cnv-caller` commands.
+* Update :meth:`sdk.utils.add_cn_samples` and :meth:`sdk.utils.simulate_copy_number` methods to check input files more rigorously.
+* Update :meth:`api.utils.test_cnv_caller` and :meth:`api.utils.train_cnv_caller` methods to accept the latest format of SampleTable[CNVCalls] as input.
+* Update plotting methods to optionally return a list of :class:`matplotlib.figure.Figure` objects for API users (e.g. Jupyter Notebook): :meth:`api.plot.plot_bam_copy_number`, :meth:`api.plot.plot_bam_read_depth`, :meth:`api.plot.plot_cn_af`, :meth:`api.plot.plot_vcf_allele_fraction`, :meth:`api.plot.plot_vcf_read_depth`.
+* :issue:`61`: Fix bug in commands :command:`compute-control-statistics`, :command:`compute-target-depth`, and :command:`prepare-depth-of-coverage` when a BED file is provided by user.
+* Improve CNV caller for CYP2A6, CYP2B6, CYP2D6, CYP2E1, GSTM1, SLC22A2, SULT1A1, UGT1A4, UGT2B15, UGT2B17.
+* Add new CNV call for CYP2A6: ``Unknown1``, ``Hybrid7``, ``Tandem2``.
+* Add new CNV calls for CYP2B6: ``Tandem1``, ``PartialDup1``, ``PartialDup2``, ``ParalogWholeDel1``.
+* Add new CNV call for CYP2D6: ``WholeDel1+Tandem3``. Also, remove ``PseudogeneDownstreamDel``.
+* Add new CNV calls for CYP2E1: ``WholeDel1`` and ``WholeDup1+PartialDup1``.
+* Add new CNV call for SLC22A2: ``NoncodingDel1Hom``.
+* Add new CNV call for SULT1A1: ``Unknown2``, ``Unknown3``, ``Unknown4``.
+* Add new CNV call for UGT1A4: ``NoncodingDel1Hom``.
+* Add new CNV call for UGT2B15: ``PartialDup2``.
+* Add new CNV call for UGT2B17: ``PartialDel2``. Also, define a new star allele ``*S3`` for ``PartialDel3``.
+* :issue:`59`: Update CNV labels.
+
 0.15.0 (2022-05-03)
 -------------------
 
@@ -63,23 +83,28 @@ Changelog
 0.12.0 (2022-01-29)
 -------------------
 
-* Add CNV caller for G6PD (mostly for sex determination since it's located on X chromosome).
-* Improve CNV caller for CYP2A6, CYP2B6, CYP2D6, CYP2E1, GSTM1, SULT1A1, UGT2B15, and UGT2B17.
 * Update :command:`run-ngs-pipeline` command to allow users to provide a custom CNV caller.
 * Update :meth:`api.core.predict_phenotype` method to not raise an error when a given star allele does not exist in the allele table. From now on, the method will output a warning about it but still produce an ``Indeterminate`` call.
 * Fix minor bug with ``--samples`` argument in commands :command:`plot-bam-copy-number`, :command:`plot-bam-read-depth`, :command:`plot-vcf-allele-fraction`, and :command:`plot-vcf-read-depth`.
 * Update :meth:`sdk.utils.add_cn_samples` method to accept a list of samples in addition to a file.
 * Add new argument ``--fontsize`` to :command:`plot-bam-read-depth` command.
 * Fix minor bug in :command:`plot-bam-read-depth` command.
 * Moved 1KGP reference haplotype panels and CNV callers to the ``pypgx-bundle`` `repository <https://github.com/sbslee/pypgx-bundle>`__ (only those files were moved; other files such as ``allele-table.csv`` and ``variant-table.csv`` are intact). From now on, the user needs to clone the ``pypgx-bundle`` repository with matching PyPGx version to their home directory in order for PyPGx to correctly access the moved files. This is undoubtedly annoying, but absolutely necessary for portability reasons because PyPGx has been growing exponentially in file size due to the increasing number of genes supported and their CNV complexity, to the point where it now exceeds upload size limit for PyPI (100 Mb). After removal of those files, the size of PyPGx has reduced from >100 Mb to <1 Mb.
+* Add CNV caller for G6PD (mostly for sex determination since it's located on X chromosome).
+* Improve CNV caller for CYP2A6, CYP2B6, CYP2D6, CYP2E1, GSTM1, SULT1A1, UGT2B15, and UGT2B17.
+* Add new CNV calls for CYP2A6: ``Duplication2``, ``Duplication3``, ``Deletion2Het``, ``Deletion3Het``, ``PseudogeneDuplication``, ``Hybrid2``, ``Hybrid3``. Additionally, some CNV calls have been renamed: ``Hybrid`` → ``Hybrid1``; ``Duplication`` → ``Duplication1``; ``DeletionHet`` → ``Deletion1Het``; ``DeletionHom`` → ``Deletion1Hom``.
+* Add a new CNV call for CYP2B6: ``Duplication``.
+* Add new CNV calls for CYP2D6: ``Unknown1``, ``Tandem1B``, ``Multiplication``. Additionally, some CNV calls have been renamed: ``Tandem1`` → ``Tandem1A``; ``DeletionHet,Tandem1`` → ``DeletionHet,Tandem1A``; ``Duplication,Tandem1`` → ``Duplication,Tandem1A``.
+* Add a new CNV call for CYP2E1: ``Duplication2``. Additionally, a CNV call have been renamed: ``Duplication`` → ``Duplication1``.
+* Add new CNV calls for GSTM1: ``UpstreamDeletionHet`` and ``DeletionHet,UpstreamDeletionHet``.
+* Add a new CNV call for UGT2B15: ``PartialDeletion2``. Additionally, a CNV call have been renamed: ``PartialDeletion`` → ``PartialDeletion1``.
+* Add a new CNV call for UGT2B17: ``PartialDeletionHet``.
 
 0.11.0 (2022-01-01)
 -------------------
 
-* Add CNV caller for CYP4F2 and SULT1A1.
 * Fix minor bug in :command:`compute-copy-number` command.
 * Update :command:`plot-cn-af` command to check input files more rigorously.
-* Improve CNV caller for CYP2A6, CYP2D6, and SLC22A2.
 * Add new method :meth:`sdk.utils.add_cn_samples`.
 * Update :command:`compare-genotypes` command to output CNV comparisonw results as well.
 * Update :command:`estimate-phase-beagle` command. From now on, the 'chr' prefix in contig names (e.g. 'chr1' vs. '1') will be automatically added or removed as necessary to match the reference VCF’s contig names.
@@ -89,6 +114,9 @@ Changelog
 * Change 1KGP reference haplotype panels for GRCh38. Previously, PyPGx was using the panels from `Lowy-Gallego et al., 2019 <https://wellcomeopenresearch.org/articles/4-50>`__ where the authors had aligned sequence reads against the full GRCh38 reference, including ALT contigs, decoy, and EBV/IMGT/HLA sequences. This resulted in poor phasing/imputation performance for highly polymorphic PGx genes (e.g. CYP2D6) presumably because the panels were missing haplotype information for lots of SNVs/indels as sequence reads with those variants were mapped to ALT contigs; however, the panels were still the best option at the time (definitely better than lifting over GRCh37 panels). Fortunately, `Byrska-Bishop et al., 2021 <https://www.biorxiv.org/content/10.1101/2021.02.06.430068v2>`__ from New York Genome Center has recently published a new set of GRCh38 panels which apparently has less of this problem despite still having sequence reads aligned in the presence of ALT contigs, etc. When empirically tested, these panels showed a significant increase in phasing/imputation performance. Therefore, from now on, PyPGx will use these panels for GRCh38 data.
 * Update GRCh38 variant information for following alleles: CYP2D6\*35, CYP2D6\*45, CYP2D6\*46.
 * Update gene region for SLC22A2 to match GRCh37 and GRCh38.
+* Add CNV caller for CYP4F2 and SULT1A1.
+* Improve CNV caller for CYP2A6, CYP2D6, and SLC22A2.
+* Add a new CNV call for CYP2D6: ``Tandem3``.
 
 0.10.1 (2021-12-20)
 -------------------
@@ -112,8 +140,10 @@ Changelog
 * Rename ``--samples`` argument from :command:`run-ngs-pipeline` command to ``--samples-without-sv``.
 * Update :command:`run-ngs-pipeline` and :command:`run-chip-pipeline` commands to be able to subset/exclude specified samples.
 * Remove ``--fn`` argument from :command:`filter-samples` command.
-* Update CNV caller for CYP2D6, GSTM1, and UGT1A4.
 * Update :meth:`api.plot.plot_cn_af` method to accept both VcfFrame[Imported] and VcfFrame[Consolidated].
+* Improve CNV caller for CYP2D6, GSTM1, and UGT1A4.
+* Add a new CNV call for CYP2D6: ``Tandem2C``, ``DeletionHom``.
+* Add a new CNV call for UGT1A4: ``Intron1DeletionB``. Additionally, a CNV call have been renamed: ``Intron1Deletion`` → ``Intron1DeletionA``.
 
 0.9.0 (2021-12-05)
 ------------------

README.rst

@@ -26,18 +26,34 @@ README
 Introduction
 ============
 
-The main purpose of the PyPGx package, which is completely free and open
-source, is to provide a unified platform for pharmacogenomics (PGx) research.
+The main purpose of the PyPGx package is to provide a unified platform for
+pharmacogenomics (PGx) research. PyPGx is and always will be completely free
+and open source.
 
 The package is written in Python, and supports both command line interface
 (CLI) and application programming interface (API) whose documentations are
 available at the `Read the Docs <https://pypgx.readthedocs.io/en/latest/>`_.
 
-PyPGx can be used to predict PGx genotypes and phenotypes using various
-genomic data, including data from next-generation sequencing (NGS), single
-nucleotide polymorphism (SNP) array, and long-read sequencing. Importantly,
-PyPGx is compatible with both of the Genome Reference Consortium Human (GRCh)
-builds, GRCh37 (hg19) and GRCh38 (hg38).
+Quick links:
+
+- `README <https://pypgx.readthedocs.io/en/latest/readme.html>`__
+- `Genes <https://pypgx.readthedocs.io/en/latest/genes.html>`__
+- `Glossary <https://pypgx.readthedocs.io/en/latest/glossary.html>`__
+- `Tutorials <https://pypgx.readthedocs.io/en/latest/tutorials.html>`__
+- `CLI <https://pypgx.readthedocs.io/en/latest/cli.html>`__
+- `API <https://pypgx.readthedocs.io/en/latest/api.html>`__
+- `SDK <https://pypgx.readthedocs.io/en/latest/sdk.html>`__
+- `FAQ <https://pypgx.readthedocs.io/en/latest/faq.html>`__
+- `Changelog <https://pypgx.readthedocs.io/en/latest/changelog.html>`__
+
+PyPGx can predict PGx genotypes (e.g. ``*4/*5``) and phenotypes (e.g.
+``Poor Metabolizer``) using various genomic data, including data from
+next-generation sequencing (NGS), single nucleotide polymorphism (SNP) array,
+and long-read sequencing. Importantly, for NGS data the package can detect
+`structural variation (SV) <https://pypgx.readthedocs.io/en/latest/
+glossary.html#structural-variation-sv>`__ using a machine learning-based
+approach. Finally, note that PyPGx is compatible with both of the Genome
+Reference Consortium Human (GRCh) builds, GRCh37 (hg19) and GRCh38 (hg38).
 
 There are currently 59 pharmacogenes in PyPGx:
 
@@ -161,6 +177,10 @@ you can access a development branch with the ``git checkout`` command. When
 you do this, please make sure your environment already has all the
 dependencies installed.
 
+.. warning::
+    You're not done yet! Keep scrolling down to obtain the resource bundle
+    for PyPGx, which is essential for running the package.
+
 Resource bundle
 ===============
 
@@ -200,7 +220,10 @@ learn.org/stable/modules/generated/sklearn.svm.SVC.html>`__-based multiclass
 classifier using the `one-vs-rest strategy <https://scikit-learn.org/stable
 /modules/generated/sklearn.multiclass.OneVsRestClassifier.html>`__ for each
 gene for each GRCh build. Each classifier is trained using copy number
-profiles of real NGS samples as well as simulated ones.
+profiles of real NGS samples as well as simulated ones, including those from
+`1KGP <https://pypgx.readthedocs.io/en/latest/glossary.html#genomes-project-
+1kgp>`__ and `GeT-RM <https://pypgx.readthedocs.io/en/latest/
+glossary.html#genetic-testing-reference-materials-coordination-program-get-rm>`__.
 
 You can plot copy number profile and allele fraction profile with PyPGx to
 visually inspect SV calls. Below are CYP2D6 examples:

docs/cli.rst

@@ -527,7 +527,8 @@ plot-bam-copy-number
    Optional arguments:
      -h, --help            Show this help message and exit.
      --fitted              Show the fitted line as well.
-     --path PATH           Create plots in this directory.
+     --path PATH           Create plots in this directory (default: current
+                           directory).
      --samples TEXT [TEXT ...]
                            Specify which samples should be included for analysis
                            by providing a text file (.txt, .tsv, .csv, or .list)
@@ -556,7 +557,8 @@ plot-bam-read-depth
    
    Optional arguments:
      -h, --help            Show this help message and exit.
-     --path PATH           Create plots in this directory.
+     --path PATH           Create plots in this directory (default: current
+                           directory).
      --samples TEXT [TEXT ...]
                            Specify which samples should be included for analysis
                            by providing a text file (.txt, .tsv, .csv, or .list)
@@ -586,7 +588,8 @@ plot-cn-af
    
    Optional arguments:
      -h, --help            Show this help message and exit.
-     --path PATH           Create plots in this directory.
+     --path PATH           Create plots in this directory (default: current
+                           directory).
      --samples TEXT [TEXT ...]
                            Specify which samples should be included for analysis
                            by providing a text file (.txt, .tsv, .csv, or .list)
@@ -615,7 +618,8 @@ plot-vcf-allele-fraction
    
    Optional arguments:
      -h, --help            Show this help message and exit.
-     --path PATH           Create plots in this directory.
+     --path PATH           Create plots in this directory (default: current
+                           directory).
      --samples TEXT [TEXT ...]
                            Specify which samples should be included for analysis
                            by providing a text file (.txt, .tsv, .csv, or .list)
@@ -644,7 +648,8 @@ plot-vcf-read-depth
      -h, --help            Show this help message and exit.
      --assembly TEXT       Reference genome assembly (default: 'GRCh37')
                            (choices: 'GRCh37', 'GRCh38').
-     --path PATH           Create plots in this directory.
+     --path PATH           Create plots in this directory (default: current
+                           directory).
      --samples TEXT [TEXT ...]
                            Specify which samples should be included for analysis
                            by providing a text file (.txt, .tsv, .csv, or .list)
@@ -980,6 +985,7 @@ test-cnv-caller
 
    $ pypgx test-cnv-caller -h
    usage: pypgx test-cnv-caller [-h] [--confusion-matrix PATH]
+                                [--comparison-table PATH]
                                 cnv-caller copy-number cnv-calls
    
    Test CNV caller for target gene.
@@ -997,6 +1003,9 @@ test-cnv-caller
                            Write the confusion matrix as a CSV file where rows
                            indicate actual class and columns indicate prediction
                            class.
+     --comparison-table PATH
+                           Write a CSV file comparing actual vs. predicted CNV
+                           calls for each sample.
 
 train-cnv-caller
 ================
@@ -1005,6 +1014,7 @@ train-cnv-caller
 
    $ pypgx train-cnv-caller -h
    usage: pypgx train-cnv-caller [-h] [--confusion-matrix PATH]
+                                 [--comparison-table PATH]
                                  copy-number cnv-calls cnv-caller
    
    Train CNV caller for target gene.
@@ -1025,4 +1035,7 @@ train-cnv-caller
                            Write the confusion matrix as a CSV file where rows
                            indicate actual class and columns indicate prediction
                            class.
+     --comparison-table PATH
+                           Write a CSV file comparing actual vs. predicted CNV
+                           calls for each sample.
 

docs/create.py

@@ -53,18 +53,34 @@
 Introduction
 ============
 
-The main purpose of the PyPGx package, which is completely free and open
-source, is to provide a unified platform for pharmacogenomics (PGx) research.
+The main purpose of the PyPGx package is to provide a unified platform for
+pharmacogenomics (PGx) research. PyPGx is and always will be completely free
+and open source.
 
 The package is written in Python, and supports both command line interface
 (CLI) and application programming interface (API) whose documentations are
 available at the `Read the Docs <https://pypgx.readthedocs.io/en/latest/>`_.
 
-PyPGx can be used to predict PGx genotypes and phenotypes using various
-genomic data, including data from next-generation sequencing (NGS), single
-nucleotide polymorphism (SNP) array, and long-read sequencing. Importantly,
-PyPGx is compatible with both of the Genome Reference Consortium Human (GRCh)
-builds, GRCh37 (hg19) and GRCh38 (hg38).
+Quick links:
+
+- `README <https://pypgx.readthedocs.io/en/latest/readme.html>`__
+- `Genes <https://pypgx.readthedocs.io/en/latest/genes.html>`__
+- `Glossary <https://pypgx.readthedocs.io/en/latest/glossary.html>`__
+- `Tutorials <https://pypgx.readthedocs.io/en/latest/tutorials.html>`__
+- `CLI <https://pypgx.readthedocs.io/en/latest/cli.html>`__
+- `API <https://pypgx.readthedocs.io/en/latest/api.html>`__
+- `SDK <https://pypgx.readthedocs.io/en/latest/sdk.html>`__
+- `FAQ <https://pypgx.readthedocs.io/en/latest/faq.html>`__
+- `Changelog <https://pypgx.readthedocs.io/en/latest/changelog.html>`__
+
+PyPGx can predict PGx genotypes (e.g. ``*4/*5``) and phenotypes (e.g.
+``Poor Metabolizer``) using various genomic data, including data from
+next-generation sequencing (NGS), single nucleotide polymorphism (SNP) array,
+and long-read sequencing. Importantly, for NGS data the package can detect
+`structural variation (SV) <https://pypgx.readthedocs.io/en/latest/
+glossary.html#structural-variation-sv>`__ using a machine learning-based
+approach. Finally, note that PyPGx is compatible with both of the Genome
+Reference Consortium Human (GRCh) builds, GRCh37 (hg19) and GRCh38 (hg38).
 
 There are currently 59 pharmacogenes in PyPGx:
 
@@ -188,6 +204,10 @@
 you do this, please make sure your environment already has all the
 dependencies installed.
 
+.. warning::
+    You're not done yet! Keep scrolling down to obtain the resource bundle
+    for PyPGx, which is essential for running the package.
+
 Resource bundle
 ===============
 
@@ -227,7 +247,10 @@
 classifier using the `one-vs-rest strategy <https://scikit-learn.org/stable
 /modules/generated/sklearn.multiclass.OneVsRestClassifier.html>`__ for each
 gene for each GRCh build. Each classifier is trained using copy number
-profiles of real NGS samples as well as simulated ones.
+profiles of real NGS samples as well as simulated ones, including those from
+`1KGP <https://pypgx.readthedocs.io/en/latest/glossary.html#genomes-project-
+1kgp>`__ and `GeT-RM <https://pypgx.readthedocs.io/en/latest/
+glossary.html#genetic-testing-reference-materials-coordination-program-get-rm>`__.
 
 You can plot copy number profile and allele fraction profile with PyPGx to
 visually inspect SV calls. Below are CYP2D6 examples: