my first commit

Author: Jai Shah

Canny Edge Detection/CannyEdgeDetection.ipynb

@@ -0,0 +1,357 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Dynamic Time Warping (DTW)\n",
+    "This notebook describes the DTW concepts.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Time Series\n",
+    "* A time series is a collection of observations made sequentially.\n",
+    "* Time series occur in different fields such as  medical, scientific and businesses domains.\n",
+    "* Finding the similarity between two time series is useful for clustering and classification.\n",
+    "\n",
+    "\n",
+    "\n",
+    "## Dynamic Time Warping (DTW)\n",
+    "Dynamic Time Warping (DTW) is an algorithm for measuring similarity between two sequences which may not have the same length.\n",
+    "\n",
+    "<img  src=\"BWQ6YDNCD5RH21TRCV6K7URPA05UCM0T.png\"/>\n",
+    "\n",
+    "\n",
+    "In general, the DTW maps each element in the first sequence to an element in the second series. Assuming that a distance function is defined for each pair of points, the goal is to find a mapping that minimizes the total distance between all the points.\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {
+    "scrolled": false
+   },
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "Automatic pdb calling has been turned OFF\n"
+     ]
+    },
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "7f357ddcb04c4bcbb1fa5fec2667d13c"
+      }
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "try:\n",
+    "    if __IPYTHON__:\n",
+    "        from IPython import get_ipython\n",
+    "\n",
+    "        get_ipython().magic('matplotlib inline')\n",
+    "        from ipython_utilities import *\n",
+    "        from ipywidgets import interact, fixed, FloatSlider, IntSlider, Label, Checkbox, FloatRangeSlider\n",
+    "        from IPython.display import display\n",
+    "\n",
+    "        in_ipython_flag = True\n",
+    "except:\n",
+    "    in_ipython_flag = False\n",
+    "import cv2 as cv\n",
+    "from matplotlib import pyplot as plt\n",
+    "import numpy as np\n",
+    "from PIL import Image\n",
+    "from ipywidgets import interact, fixed, FloatSlider, IntSlider, Label, Checkbox, FloatRangeSlider\n",
+    "\n",
+    "%matplotlib inline\n",
+    "%pdb\n",
+    "\n",
+    "def display_two_sequences(n1,n2):\n",
+    "    index1 = np.linspace(0,15,n1)\n",
+    "    index2 = np.linspace(0,15,n2)\n",
+    "    A = 5*np.sin(index1)\n",
+    "    B = 3*np.sin(index2 + 1)\n",
+    "    # s1 = [1, 2, 3, 4]\n",
+    "    # s2 = [2, 3, 5, 6, 8]\n",
+    "    # ob_dtw = cl_dtw()\n",
+    "    # distance,_ = ob_dtw.calculate_dtw_distance(s1, s2)\n",
+    "\n",
+    "    fig = plt.figure(figsize=(12,4))\n",
+    "    plt.plot(index1, A, '-ro', label='A')\n",
+    "    plt.plot(index2, B, '-bo' ,label='B')\n",
+    "    plt.ylabel('value')\n",
+    "    plt.xlabel('index')\n",
+    "    plt.legend()\n",
+    "\n",
+    "    plt.show()\n",
+    "    plt.pause(0.001)\n",
+    "    \n",
+    "interact(display_two_sequences,\n",
+    "            n1=IntSlider(min=0, \n",
+    "              max=100, step=1,value=5,\n",
+    "              description='# of points in sequence 1',\n",
+    "              continuous_update=True),\n",
+    "            n2=IntSlider(min=0, \n",
+    "              max=100, step=1,value=7,\n",
+    "              description='# of points in sequence 2',\n",
+    "              continuous_update=True));\n",
+    "\n",
+    "# arrange_widgets_in_grid(controls)\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Consider the two sequences $A=\\{a_1, a_2, ... , a_M\\}$ and $B=\\{b_1, b_2, ... , b_N\\}$ as shown above. Where $a_k$ is the $k_{th}$ sample point in the sequence $A$ and $b_k$ is the $k_{th}$ sample point in the sequence $B$. To find the similarity of these two sequences we need to calculate the total distance between the two sequences. To calculate the total distance we can connects each point in sequence $A$ to a point in sequence $B$ and accumulate the distances between the corresponding points. Since there are so many different possibilities for connecting different points, then the question is what is the best possible arrangement which results in the minimum total distance? \n",
+    "In order for the similarity measure to be meaningful we need to impose some constraints on how the points on the two sequence should be connected:\n",
+    "* **Boundary conditions:** the first points should be connected to each other and the last points should be connected to each point.\n",
+    "* **Monotonicity:** The alignment can not go backward. \n",
+    "* **Continuity:** The alignment can not skip an element.\n",
+    "* **Warping window:** The same point (feature) should not be repeated too many times.\n",
+    "\n",
+    "### Path\n",
+    "A path $P$ is defined as an ordered set of 2-tuples:\n",
+    "\n",
+    "$$\\large P=p_1,p_2, .....p_q$$\n",
+    "$$\\large p_k=(i_k,j_k)$$\n",
+    "\n",
+    "where $q$ is the number of connections (correspondences) and  $i_k$ and $j_k$ are the indexes of the connecting elements. $P_s$ is called a \"Warping\" function.\n",
+    "\n",
+    "For example $P=(1,1), (1,2), (2,3), ... $ means:\n",
+    "* Point 1 in sequence A is connected to point 1 in sequence B\n",
+    "* Point 1 in sequence A is connected to point 2 in sequence B\n",
+    "* Point 2 in sequence A is connected to point 3 in sequence B\n",
+    "* ...\n",
+    "\n",
+    "\n",
+    "\n",
+    "\n",
+    "A path can be shown on a grid of M rows by N columns. The image below shows two possible paths. Notice that there are many possible paths for connecting elements of the sequence A to elements of the sequence B without violating any the constraints. The cost of each path is the accumulated distance between the corresponding points. **The goal of the DTW algorithm is to find the best path which minimizes the total cost.**\n",
+    "<img  src=\"V4VW3VNNRET6TKXJW2MR0A024BD0EMUM.png\"/>\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Finding the Best Path\n",
+    "To find the best path:\n",
+    "* Set an M by N matrix G\n",
+    "* Set $G\\left[1,1\\right]=d(1,1)$    where $d(i,j)$ is the distance between elements i and j in the sequences respectively.\n",
+    "* Calculate each element of the the matrix G as:\n",
+    "\n",
+    "$$\\large G\\left[i,j\\right]=d(i,j)+min(G\\left[i,j-1\\right],G\\left[i-1,j\\right],G\\left[i-1,j-1\\right])$$\n",
+    "\n",
+    "* Total distance will be $D(A,B)=G\\left[M,N\\right]$\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "application/vnd.jupyter.widget-view+json": {
+       "model_id": "e160798a8b184acf9fdc255e3765d910"
+      }
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    }
+   ],
+   "source": [
+    "def dtw(s1, s2, window=3):\n",
+    "    grid = np.inf*np.ones((len(s1), len(s2)))\n",
+    "    # grid[0, :] = abs(s1[0] - s2)\n",
+    "    for i in range(window+1):\n",
+    "        grid[0, i] = abs(s1[0] - s2[i])\n",
+    "    for j in range(window+1):\n",
+    "        grid[j, 0] = abs(s2[0] - s1[j])\n",
+    "    \n",
+    "    for i in range(1, len(s1)):\n",
+    "        for j in range(1, len(s2)):\n",
+    "            if abs(i-j) > window:\n",
+    "                continue\n",
+    "            grid[i, j] = abs(s1[i] - s2[j]) + min(grid[i - 1, j], grid[i, j-1], grid[i-1, j-1])\n",
+    "            \n",
+    "    \n",
+    "    print(grid)\n",
+    "    print(grid[-1, -1])\n",
+    "    \n",
+    "def display_two_sequences(n1,n2):\n",
+    "    index1 = np.linspace(0,15,n1)\n",
+    "    index2 = np.linspace(0,15,n2)\n",
+    "    A = [5, 6, 9, 2, 6]*2\n",
+    "    B = [5, 7, 2, 6, 9 , 2]*2\n",
+    "    #A = 5*np.sin(index1)\n",
+    "    #B = 3*np.sin(index2 + 1)\n",
+    "    # s1 = [1, 2, 3, 4]\n",
+    "    # s2 = [2, 3, 5, 6, 8]\n",
+    "    # ob_dtw = cl_dtw()\n",
+    "    # distance,_ = ob_dtw.calculate_dtw_distance(s1, s2)\n",
+    "    print(A)\n",
+    "    print(B)\n",
+    "    \n",
+    "    dtw(A, B)\n",
+    "#     fig = plt.figure(figsize=(12,4))\n",
+    "#     plt.plot(index1, A, '-ro', label='A')\n",
+    "#     plt.plot(index2, B, '-bo' ,label='B')\n",
+    "#     plt.ylabel('value')\n",
+    "#     plt.xlabel('index')\n",
+    "#     plt.legend()\n",
+    "\n",
+    "#     plt.show()\n",
+    "#     plt.pause(0.001)\n",
+    "controls = interact(display_two_sequences,\n",
+    "             n1=IntSlider(min=0, \n",
+    "            max=100, step=1,value=5,\n",
+    "            description='# of points in sequence 1',\n",
+    "            continuous_update=True),\n",
+    "                      n2=IntSlider(min=0, \n",
+    "            max=100, step=1,value=7,\n",
+    "            description='# of points in sequence 2',\n",
+    "            continuous_update=True));\n",
+    "\n",
+    "# arrange_widgets_in_grid(controls)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {
+    "collapsed": true
+   },
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Warping Window\n",
+    "In order to gurantee that the alignment does not get stuck in one element a warping window is defined. The warping window limits the difference between the indexes of the two sequence. In cases where the number of elements in both sequences are equal, the warping window forces the path not to wander too far from the diagnoal.\n",
+    "<img  src=\"PMCFBXFYO2WUX5EPRCQM0K8DALBYR0KB.png\"/>\n",
+    "\n",
+    "In a given path the warping window constraints can be imposed by limiting $\\left| {i_k - j_k} \\right| \\le r,r > 0$\n",
+    "\n",
+    "\n",
+    "\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Slope Constraints\n",
+    "In order to prevent that a very short section of one sequence match a very long section of another, a slope constraint is imposed on the path.\n",
+    "<img  src=\"A3VPKJJ0GMVJSNXICNJ5L5Y9JT9K2YKT.png\"/>\n",
+    "\n",
+    "$$\\large {{\\left( {{j_{k + s}} - {j_k}} \\right)} \\over {\\left( {{i_{k + s}} - {i_k}} \\right)}} \\le {s_h}$$\n",
+    "\n",
+    "and \n",
+    "\n",
+    "$$\\large {{\\left( {{i_{k + s}} - {i_k}} \\right)} \\over {\\left( {{j_{k + s}} - {j_k}} \\right)}} \\le {s_v}$$\n",
+    "\n",
+    "where $S_h>0$ and $S_v>0$ are the limiting slope constants.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Time Normalized Distance Measure\n",
+    "The time normalized distance between two sequences $A$ and $B$ for a particular path $P$ is defined as:\n",
+    "\n",
+    "$$\\large D(A,B)= {{{\\sum\\limits_{k = 1}^q {d({p_k}) \\cdot {w_k}} } \\over {\\sum\\limits_{k = 1}^q {{w_k}} }}} $$\n",
+    "\n",
+    "$$\\large p_k=(i_k,j_k)$$\n",
+    "\n",
+    "where $d(p_k)$ is the distance between element $i_k$ in the sequence A and element $j_k$ in the sequence B.\n",
+    "\n",
+    "$w_k$ is the weighting coefficient for the connection k. \n",
+    "\n",
+    "The best path $P^*$ is found by minimizing the $D(A,B)$\n",
+    "\n",
+    "$$\\large {P^*} = \\mathop {argmin }\\limits_P (D(A,B))$$\n",
+    "\n",
+    "\n",
+    "The term $\\sum\\limits_{k = 1}^q {{w_k}}  $ in the denominator of the $D(A,B)$ complicates the optimization of the best path because it dependes on the length of the path. It is desirable to find some weighting coefficients that are independent of the path $P$. For example if we define \n",
+    "$$\\large w_k= (i_k-i_{k-1})+(j_k-j_{k-1})$$\n",
+    "\n",
+    "then \n",
+    "$$\\large \\sum\\limits_{k = 1}^q {{w_k}}=M+N=C$$\n",
+    "\n",
+    "This means that the denominator of the $D(A,B)$ is a constant\n",
+    "\n",
+    "$$\\large D(A,B)={1 \\over C}\\mathop {\\mathop {argmin }\\limits_P \\left[ {\\sum\\limits_{s = 1}^k {d({p_s}) \\cdot {w_s}} } \\right]}\\limits_{} $$ \n",
+    "\n",
+    "A an alternative we can define \n",
+    "$\\large w_k= (i_k-i_{k-1})$ which implies $\\large \\sum\\limits_{k = 1}^q {{w_k}}=M=C$\n",
+    "\n",
+    "or $\\large w_k= (j_k-j_{k-1})$ which implies $\\large \\sum\\limits_{k = 1}^q {{w_k}}=N=C$\n",
+    "\n",
+    "The algorithm for finding the best path with time normalization $w_k=(ik−ik−1)+(jk−jk−1)$ will be:\n",
+    "\n",
+    "* Set an M by N matrix G\n",
+    "* Set $G\\left[1,1\\right]=2d(1,1)$    where $d(i,j)$ is the distance between elements i and j in the sequences respectively.\n",
+    "* Calculate each element of the the matrix G as:\n",
+    "\n",
+    "$$\\large G\\left[i,j\\right]=min(G\\left[i,j-1\\right],G\\left[i-1,j\\right],2d(i,j)G\\left[i-1,j-1\\right])$$\n",
+    "* Total distance will be:\n",
+    "$$\\large D(A,B)={{G\\left[M,N\\right]}\\over {(N+M)}}$$\n"
+   ]
+  }
+ ],
+ "metadata": {
+  "anaconda-cloud": {},
+  "kernelspec": {
+   "display_name": "Python [Root]",
+   "language": "python",
+   "name": "Python [Root]"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.5.2"
+  },
+  "widgets": {
+   "state": {
+    "0555dd1618ac47b3879d19b0892a25a2": {
+     "views": [
+      {
+       "cell_index": 2
+      }
+     ]
+    },
+    "47a1a1eb88fb4c3d914c19a8c7d83596": {
+     "views": [
+      {
+       "cell_index": 5
+      }
+     ]
+    }
+   },
+   "version": "1.2.0"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 1
+}