24_project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data science project in Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For the 4 species gff files store in the `data/` folder (mouse: `GRCm38.gff3`, human: `GRCh38.gff3`, zebrafish: `GRCz11.gff3` and panda: `AilMel.gff3`), load these files into a DataFrame, filter the exons, and calculate their length, their GC content as well as their molecular weigth. Store these calculations into new columns in their respective DataFrame. Plot these three values for the four species onto three graphs using a boxplot.\n",
    "\n",
    "Start by working with the mouse data for exploratory analysis, working through the problem steps by steps. Then, create a re-usable function to apply the analysis onto all datasets. Load all four datasets and apply the newly created function to calculate the new values. After having all DataFrames with the new three columns, visualise the data onto three boxplot graphs.\n",
    "\n",
    "Present the results into a Jupyter notebook, using Pandas, Matplotlib and Biopython. Write reusable and modular code as much as possible using functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas\n",
    "from Bio import SeqIO\n",
    "from Bio import Entrez\n",
    "from Bio.SeqUtils import GC, molecular_weight"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_gc_and_mw_from_gbid(id='NM_177676.6'):\n",
    "    Entrez.email = 'A.N.Other@example.com' # Always tell NCBI who you are\n",
    "    handle = Entrez.efetch(db=\"nucleotide\", id=id, rettype=\"gb\")\n",
    "    seq_record = SeqIO.read(handle, \"gb\")\n",
    "    handle.close()\n",
    "    return GC(seq_record.seq), molecular_weight(seq_record.seq)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(get_gc_and_mw_from_gbid('NM_177676.6'))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_gc_and_mw_from_gbids(df_ids):\n",
    "    gcs = []\n",
    "    mws = []\n",
    "    for i in df_ids:\n",
    "        gc, mw = get_gc_and_mw_from_gbid(i)\n",
    "        gcs.append(gc)\n",
    "        mws.append(mw)\n",
    "    return gcs, mws"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Working with mouse data: exploratory analysis\n",
    "\n",
    "- remove rows with null values\n",
    "- filter all exons\n",
    "- calculate GC contents and molecular weights"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mouse = pandas.read_csv('data/GRCm38.gff3', sep='\\t')\n",
    "mouse.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mouse.dropna(inplace=True)\n",
    "print(mouse.type.unique())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "exon_mouse = mouse[mouse['type']=='exon']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "exon_mouse.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "small_exon_mouse = exon_mouse.iloc[:9,]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gcs, mws = get_gc_and_mw_from_gbids(small_exon_mouse['gbid'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "small_exon_mouse.insert(len(small_exon_mouse.columns), 'len', small_exon_mouse['end'] - small_exon_mouse['start'] + 1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "small_exon_mouse.insert(len(small_exon_mouse.columns), 'gc', gcs)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "small_exon_mouse.insert(len(small_exon_mouse.columns), 'mw', mws)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "small_exon_mouse.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.boxplot([small_exon_mouse['gc']], labels=['mouse'])\n",
    "plt.ylabel('GC content (%)')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Creating a re-usable function to apply the analysis onto other datasets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_exons(data):\n",
    "    data.dropna(inplace=True)\n",
    "    exons = data[data['type']=='exon']\n",
    "    exons = exons[:9]\n",
    "    gcs, mws = get_gc_and_mw_from_gbids(exons['gbid'])\n",
    "    exons.insert(len(exons.columns), 'len', exons['end'] - exons['start'] + 1)\n",
    "    exons.insert(len(exons.columns), 'gc', gcs)\n",
    "    exons.insert(len(exons.columns), 'mw', mws)\n",
    "    return exons"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading all four datasets and calculating new values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mouse = pandas.read_csv('data/GRCm38.gff3', sep='\\t')\n",
    "mouse_small_exons = get_exons(mouse)\n",
    "mouse_small_exons.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "human = pandas.read_csv('data/GRCh38.gff3', sep='\\t')\n",
    "human_small_exons = get_exons(human)\n",
    "human_small_exons.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "zebrafish = pandas.read_csv('data/GRCz11.gff3', sep='\\t')\n",
    "zebrafish_small_exons = get_exons(zebrafish)\n",
    "zebrafish_small_exons.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "panda = pandas.read_csv('data/AilMel.gff3', sep='\\t')\n",
    "panda_small_exons = get_exons(panda)\n",
    "panda_small_exons.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualising data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Comparing exon's length, GC content and molecular weight across four species"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.boxplot([mouse_small_exons['len'], \n",
    "             human_small_exons['len'], \n",
    "             zebrafish_small_exons['len'],\n",
    "             panda_small_exons['len']], \n",
    "            labels=['mouse', 'human', 'zebrafish', 'panda']\n",
    "            )\n",
    "plt.ylabel('Feature length (bp)')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.boxplot([mouse_small_exons['gc'], \n",
    "             human_small_exons['gc'], \n",
    "             zebrafish_small_exons['gc'],\n",
    "             panda_small_exons['gc']], \n",
    "            labels=['mouse', 'human', 'zebrafish', 'panda']\n",
    "            )\n",
    "plt.ylabel('GC content (%)')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as plt\n",
    "plt.boxplot([mouse_small_exons['mw'], \n",
    "             human_small_exons['mw'], \n",
    "             zebrafish_small_exons['mw'],\n",
    "             panda_small_exons['mw']], \n",
    "            labels=['mouse', 'human', 'zebrafish', 'panda']\n",
    "            )\n",
    "plt.ylabel('Molecular weight')\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}