em.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Expectation Maximization\n",
    "\n",
    "The EM algorithm is a maximum likelihood estimation approach used in the presence of latent variables. This is done by first estimating the values for the latent variables and then optimizing the model. This is repeated until convergence. This two steps can be summarized as:\n",
    "* E-Step: Estimate missing variables in the dataset.\n",
    "* M-Step: Update the parameters of the model to maximize likelihood.\n",
    "\n",
    "This notebook covers a practical implementation in Python of the EM algorithnm along with some examples."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Unidimensional case with Gaussians\n",
    "Suppose we have some data that we want to group into different clusters. \n",
    "We could consider a cluster to be a gaussian bell, following the normal distribution *N*($\\mu$, $\\sigma^{2}$). \n",
    "\n",
    "Each cluster would then be determined by two parameters: its mean $\\mu$, and its variance $\\sigma^{2}$. \n",
    "For each sample $s$ in the data, a probability  $\\pi_{{s}_{i}}$ of belonging to the $i^{th}$ cluster could be considered. \n",
    "\n",
    "\n",
    "The goal is to find the means and variance of the different gaussians that could de used to cluster the data, while maximizing the likelihood."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Data generation\n",
    "First, let's define a function to generate some data points to use as our dataset. This function receives the total number of samples to generate, the centers of mass of each of the different groups we want in our data and how spread we want samples in that group to be. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "def generate_data(N, cm, spread):\n",
    "    groups = min(len(cm), len(spread))\n",
    "    data = []\n",
    "    for i in range(groups):\n",
    "        x = np.linspace(-spread[i] / 2, spread[i] / 2, num=int(N / groups))\n",
    "        data.append(x * np.random.rand(len(x)) + cm[i])\n",
    "\n",
    "    return np.concatenate(np.array(data)).flatten()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now let's generate a dataset of a 90 samples, composed of 3 different groups. Each group of samples will have a different center of mass and a different spread. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "num_samples = 90\n",
    "centers = (-5,2,13)\n",
    "spreads = (5,1,8)\n",
    "\n",
    "data = generate_data(num_samples,centers,spreads)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's visualize the data points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAQCUlEQVR4nO3cbYxc5XnG8evaN8PaiJddx3FtvEODpWpRJVJGINS0osUYEykxbVEF/VBLpdpqHT6kUaUaIRVKYolETYmqolRugmKxbgBFQlhNU9c4RZGqCjGmtMFqqR2XYLsGDEZIKGqR6d0Pc9Yej2e9M57jPbu+/z/paM95zjMz9z7n5fI5Z9aOCAEA8hqougAAQLUIAgBIjiAAgOQIAgBIjiAAgOSGqi7gQoyPj0etVqu6DABYUvbv3/9uRKxsb1+SQVCr1dRoNKouAwCWFNs/7dTOrSEASI4gAIDkCAIASI4gAIDkCAIASI4gAIDkCAIASI4gAIDkCAIASI4gAIDkCAIASI4gAIDkCAIASI4gAIDkCAIASI4gAIDkCAIASI4gAIDkCAIASI4gAIDkCAIASI4gAIDkCAIASI4gAIDkCAIASK6UILC9yfbrtg/Z3tZh/TLbzxTrX7Jda1u/zvaHtv+ojHoAAN3rOwhsD0p6QtJdkiYl3Wd7sq3b/ZLej4jrJT0u6att6/9c0g/6rQUA0LsyrghulnQoIg5HxEeSnpa0ua3PZkk7i/nvSbrdtiXJ9t2S/kvSgRJqAQD0qIwgWCPpSMvy0aKtY5+IOCXpA0ljtldI+mNJfzrfh9iest2w3Thx4kQJZQMApOofFj8i6fGI+HC+jhGxIyLqEVFfuXLlxa8MAJIYKuE9jkm6tmV5bdHWqc9R20OSrpT0nqRbJN1j+2uSrpL0f7b/JyL+soS6AABdKCMIXpa03vZ1ap7w75X0O219dkvaIumfJd0j6YcREZJ+ZbaD7UckfUgIAMDC6jsIIuKU7Qck7ZE0KOnJiDhg+1FJjYjYLenbkp6yfUjSSTXDAgCwCLj5D/OlpV6vR6PRqLoMAFhSbO+PiHp7e9UPiwEAFSMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACA5ggAAkiMIACC5UoLA9ibbr9s+ZHtbh/XLbD9TrH/Jdq1ov8P2fts/Ln7+ehn1AAC613cQ2B6U9ISkuyRNSrrP9mRbt/slvR8R10t6XNJXi/Z3JX0uIn5R0hZJT/VbDwCgN2VcEdws6VBEHI6IjyQ9LWlzW5/NknYW89+TdLttR8S/RMR/F+0HJF1ue1kJNQEAulRGEKyRdKRl+WjR1rFPRJyS9IGksbY+vyXplYj43xJqAgB0aajqAiTJ9g1q3i7aeJ4+U5KmJGndunULVBkAXPrKuCI4JunaluW1RVvHPraHJF0p6b1iea2k5yT9bkT8ZK4PiYgdEVGPiPrKlStLKBsAIJUTBC9LWm/7Otsjku6VtLutz241HwZL0j2SfhgRYfsqSd+XtC0i/qmEWgAAPeo7CIp7/g9I2iPp3yU9GxEHbD9q+/NFt29LGrN9SNKXJM1+xfQBSddL+hPbrxbTJ/qtCQDQPUdE1TX0rF6vR6PRqLoMAFhSbO+PiHp7O39ZDADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJEQQAkBxBAADJlRIEtjfZft32IdvbOqxfZvuZYv1Ltmst6x4s2l+3fWcZ9XSya9cu1Wo1DQwMqFaraevWrarVarKtwcFB2T5rGh8f19atW3X55Zefs67badmyZRoYGND4+LjGx8c7fvbQ0JBsq1aradeuXefUPD4+fvr9Zuvs1PdSMbudWsemm2loaEhbt26tuvxLWvsx1Gl/7XX9XK/ZtWuXVqxYcda+P9f2bX2P2WNtruO6dX9pb9uwYcPp92s97lasWKErrrji9PLAwMDpc0RrjbPnjG6O7bLHvm8R0dckaVDSTyT9vKQRSf8qabKtz1ZJf1XM3yvpmWJ+sui/TNJ1xfsMzveZN910U/RiZmYmRkdHQ9KinkZHR2NmZuZ0zcPDw131vVSUsZ2mp6er/jUuSZ22Tfv+2uv64eHhGBkZOec109PTMTAw0NX2LfvYnpycPKemso/tsse+F5Ia0ek83qmxl0nSrZL2tCw/KOnBtj57JN1azA9JeleS2/u29jvf1GsQTExMVH6S73aamJjouubZvpeKMrbT4OBg1b/GJWmubTPf/trL/ty6Dbvdvkvx2C577HuhOYKgjFtDayQdaVk+WrR17BMRpyR9IGmsy9dKkmxP2W7Ybpw4caKnAt98882e+ldpttZual5Kv1c3yvh9Pv744xIqQbu5ts18+2sv+/Os823D9nVL6Ri40FrnG9syLJmHxRGxIyLqEVFfuXJlT69dt27dRaqqfLO1dlPzUvq9ulHG7zM4OFhCJWg317aZb3/tZX+edb5t2L5uKR0DF1rrfGNbhjKC4Jika1uW1xZtHfvYHpJ0paT3unxt37Zv367R0dGy37Z0o6Oj2r59u6RmzcPDw131vVSUsZ2mpqZKqgatOm2b9v211/XDw8MaGRk55zVTU1MaGOh8amrfvmUf25OTk+fUVIZ+jtf5xrYUne4X9TKpec//sJoPe2cfFt/Q1ucLOvth8bPF/A06+2HxYV2Eh8URzQcuExMTYTsmJiZienr69L23Tg+mxsbGYnp6Oi677LILvic4MjIStmNsbCzGxsY6fvbs/dCJiYlzHv7MzMzE2NjY6febrbNT30vF7HZqHZtupsHBQR4UX2Ttx1Cn/bXX9XO9ZmZmJpYvX37Wvj/X9m19j9ljba7junV/aW+7/fbbT79f63G3fPnyWLFixell26fPEa01zp4zujm2yx77bmmOZwRuruuP7c9K+oaa3yB6MiK22360+NDdti+T9JSkT0s6KeneiDhcvPYhSb8n6ZSkL0bED+b7vHq9Ho1Go++6ASAT2/sjon5OexlBsNAIAgDo3VxBsGQeFgMALg6CAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACS6ysIbF9je6/tg8XPq+fot6Xoc9D2lqJt1Pb3bf+H7QO2H+unFgDAhen3imCbpH0RsV7SvmL5LLavkfSwpFsk3Szp4ZbA+LOI+AVJn5b0y7bv6rMeAECP+g2CzZJ2FvM7Jd3doc+dkvZGxMmIeF/SXkmbIuJnEfGPkhQRH0l6RdLaPusBAPSo3yBYFRHHi/m3JK3q0GeNpCMty0eLttNsXyXpc2peVQAAFtDQfB1svyDpkx1WPdS6EBFhO3otwPaQpO9K+ouIOHyeflOSpiRp3bp1vX4MAGAO8wZBRGyYa53tt22vjojjtldLeqdDt2OSbmtZXivpxZblHZIORsQ35qljR9FX9Xq958ABAHTW762h3ZK2FPNbJD3foc8eSRttX108JN5YtMn2VyRdKemLfdYBALhA/QbBY5LusH1Q0oZiWbbrtr8lSRFxUtKXJb1cTI9GxEnba9W8vTQp6RXbr9r+/T7rAQD0yBFL7y5LvV6PRqNRdRkAsKTY3h8R9fZ2/rIYAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJIjCAAgOYIAAJLrKwhsX2N7r+2Dxc+r5+i3pehz0PaWDut3236tn1oAABem3yuCbZL2RcR6SfuK5bPYvkbSw5JukXSzpIdbA8P2b0r6sM86AAAXqN8g2CxpZzG/U9LdHfrcKWlvRJyMiPcl7ZW0SZJsr5D0JUlf6bMOAMAF6jcIVkXE8WL+LUmrOvRZI+lIy/LRok2Svizp65J+Nt8H2Z6y3bDdOHHiRB8lAwBaDc3XwfYLkj7ZYdVDrQsREbaj2w+2faOkT0XEH9quzdc/InZI2iFJ9Xq9688BAJzfvEEQERvmWmf7bdurI+K47dWS3unQ7Zik21qW10p6UdKtkuq23yjq+ITtFyPiNgEAFky/t4Z2S5r9FtAWSc936LNH0kbbVxcPiTdK2hMR34yIn4uImqTPSPpPQgAAFl6/QfCYpDtsH5S0oViW7brtb0lSRJxU81nAy8X0aNEGAFgEHLH0brfX6/VoNBpVlwEAS4rt/RFRb2/nL4sBIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSIwgAIDmCAACSc0RUXUPPbJ+Q9NOq6yiMS3q36iIWAcbhDMaiiXE4Y7GMxURErGxvXJJBsJjYbkREveo6qsY4nMFYNDEOZyz2seDWEAAkRxAAQHIEQf92VF3AIsE4nMFYNDEOZyzqseAZAQAkxxUBACRHEABAcgRBCWw/YvuY7VeL6bNV17SQbG+y/brtQ7a3VV1PVWy/YfvHxT7QqLqehWT7Sdvv2H6tpe0a23ttHyx+Xl1ljQtljrFY1OcIgqA8j0fEjcX0d1UXs1BsD0p6QtJdkiYl3Wd7stqqKvVrxT6waL8zfpF8R9KmtrZtkvZFxHpJ+4rlDL6jc8dCWsTnCIIA/bpZ0qGIOBwRH0l6WtLmimvCAouIH0k62da8WdLOYn6npLsXtKiKzDEWixpBUJ4HbP9bcVmY4hK4sEbSkZblo0VbRiHpH2zvtz1VdTGLwKqIOF7MvyVpVZXFLAKL9hxBEHTJ9gu2X+swbZb0TUmfknSjpOOSvl5psajKZyLil9S8TfYF279adUGLRTS/p575u+qL+hwxVHUBS0VEbOimn+2/lvS3F7mcxeSYpGtbltcWbelExLHi5zu2n1PzttmPqq2qUm/bXh0Rx22vlvRO1QVVJSLenp1fjOcIrghKUOzks35D0mtz9b0EvSxpve3rbI9IulfS7oprWnC2l9u+YnZe0kbl2g862S1pSzG/RdLzFdZSqcV+juCKoBxfs32jmpe+b0j6g2rLWTgRccr2A5L2SBqU9GREHKi4rCqskvScbal5XP1NRPx9tSUtHNvflXSbpHHbRyU9LOkxSc/avl/N/zb+t6urcOHMMRa3LeZzBP/FBAAkx60hAEiOIACA5AgCAEiOIACA5AgCAEiOIACA5AgCAEju/wGQlqDg3oRoEwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "from matplotlib import pyplot as plt\n",
    "plt.scatter(data, np.zeros(len(data)), c='k')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Probabilities visualized\n",
    "Next, let's define how to evaluate the probability of a set of points belonging to a gaussian distribution with mean $\\mu$ and std $\\sigma$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def eval(x, mu, sigma):\n",
    "    _num = np.exp(-((x - mu) ** 2) / (2 * sigma ** 2))\n",
    "    _den = sigma * ((2 * (np.pi) ** 2) ** 0.5)\n",
    "\n",
    "    return _num / _den"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's also define a function that plots these probabilities."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot(x, mus, sigmas, pis):\n",
    "    fig = plt.figure()\n",
    "    ax = fig.add_subplot(1, 1, 1)\n",
    "    ax.scatter(x, np.zeros(len(x)), c='k')\n",
    "\n",
    "    for i in range(len(mus)):\n",
    "        x_axis = np.linspace(mus[i] - 3 * sigmas[i], mus[i] + 3 * sigmas[i], len(x))\n",
    "        ax.plot(x_axis, pis[i] * eval(x_axis, mus[i], sigmas[i]))\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We know the center of mass (mean) and spread (variance) of the dataset since we generated it. Ideally, we would like to get something like the following plot, which ilustrates 3 gaussians each centered at the mean of the groups and with the corresponding spread. The probability assigned to each cluster is 1/3, since our data generator creates the same number of samples per group."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Means:(-5, 2, 13). \n",
      "Stds :(1.118033988749895, 0.5, 1.4142135623730951)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3dd3hc5ZX48e9Rr26S3G3JHReMAWHTSwzYJoB3waYkIbCQNSHxs9lNsoH82IewZJ22JOyThABOIBQvLWZJDDGYltgUd4oLtmxZlhsukiWrd53fHzNjyeORNJJm5s4dnc/z6JmZe+/ce0aaOfPq3Pe+r6gqxhhjYlec0wEYY4wJL0v0xhgT4yzRG2NMjLNEb4wxMc4SvTHGxLgEpwPwl52drXl5eU6HYYwxrrJ58+ZSVc0JtC7qEn1eXh6bNm1yOgxjjHEVEdnX0Tor3RhjTIyzRG+MMTHOEr0xxsQ4S/TGGBPjLNEbY0yMs0RvjDExzhK9McbEOEv0xn1amuGj30Lxh05HYowrRN0FU8Z0afur8Nb9kJgO9xZDQpLTERkT1axFb9xn52ue26YaOLTZ2ViMcQFL9MZ9DmyEsVd47693NhZjXMASvXGXunKo+gLGfQn6jYBjnzsdkTFRzxK9cZfSQs9t9gTIGg+lu52NxxgXsERv3KVsj+c2a7wn2fseG2M6ZIneuMuJA57b/qM8P/UVUF/pbEzGRDlL9MZdKg9Ceg4kpkD/kd5lh5yNyZgoZ4neuEvFQc9JWGhL9BUHnYvHGBewRG/cpfIw9Bvuue+7rTrsXDzGuEBQiV5E5opIgYgUish9AdZfKiIfi0iziCwIsL6fiBwUkd+GImjTh1UfhYwhnvvpg9uWGWM61GWiF5F44FFgHjAFuFVEpvhtth+4A3i+g938GFjT8zCNAVqaoPZ4W6JPTIGU/lB9zNm4jIlywbToZwKFqlqkqo3Ai8D89huoarGqbgFa/Z8sIucCQ4C3QhCv6ctqSgGFjMFtyzKGWIvemC4Ek+hHAAfaPT7oXdYlEYkDfgl8v4vtFonIJhHZVFJSEsyuTV9U4225t0/06YOtRW9MF8J9MvZbwEpV7bRbhKouVdV8Vc3PyckJc0jGtWpKPbdp2W3L0rPalhtjAgpmmOJDwKh2j0d6lwXjAuASEfkWkAEkiUi1qp52QteYLtWWeW7T2yX6tGxP3d4Y06FgEv1GYIKIjMGT4G8BvhLMzlX1q777InIHkG9J3vRYra9Fn9W2LD3bM9BZawvExTsTlzFRrsvSjao2A4uBVcAO4GVV3S4iD4nI9QAicp6IHAQWAk+IyPZwBm36qJpSkDhIGdC2LC0b0LbWvjHmNEHNMKWqK4GVfsseaHd/I56STmf7eBp4utsRGuNTVwapAyGuXfskbZDntvY4ZNj5HWMCsStjjXvUlkHqoFOXpQ703NafiHw8xriEJXrjHnXlbS14H1+it9KNMR2yRG/co668LbH7+BJ/XXnk4zHGJSzRG/cIlOh9jy3RG9MhS/TGPQIl+uR+IPGeE7XGmIAs0Rt3aGmCxurTE72IZ2Cz+gpn4jLGBSzRG3fwJfKU/qevSx0AddbrxpiOWKI37nAy0Q84fZ216I3plCV64w6+FnugFn3KAOtHb0wnLNEbd/Al8lRr0RvTXZbojTvUd9Kitxq9MZ2yRG/cobOTsdaiN6ZTluiNO9RXem6T+52+LrkftDRAU31kYzLGJSzRG3doqPRcGJWUfvo6Xyu/oTKyMRnjEpbojTvUV0JKP88FUv58rfx6S/TGBGKJ3rhDfUXgsg14vgAAGqxOb0wgluiNOzRUBj4RC23L7YSsMQFZojfuUN9JorfSjTGdCirRi8hcESkQkUIROW1ybxG5VEQ+FpFmEVnQbvkMEVkrIttFZIuI3BzK4E0f0lAVROmmKnLxGOMiXSZ6EYkHHgXmAVOAW0Vkit9m+4E7gOf9ltcCX1fVqcBc4H9EJMCljcZ0oaECkjMDr/Mtt0RvTEDBTA4+EyhU1SIAEXkRmA987ttAVYu961rbP1FVd7W7/4WIHANyALuM0XRPQ1XHiT7Jl+itdGNMIMGUbkYAB9o9Puhd1i0iMhNIAvYEWLdIRDaJyKaSkpLu7trEOtXOE318AiSmW4vemA5E5GSsiAwDngP+SVVb/der6lJVzVfV/JycnEiEZNykqQ5am9tq8YEkZ1qL3pgOBJPoDwGj2j0e6V0WFBHpB/wVuF9V13UvPGNoa6l31KIHz5eAteiNCSiYRL8RmCAiY0QkCbgFWBHMzr3bvwo8q6rLex6m6dMaOhnnxic507pXGtOBLhO9qjYDi4FVwA7gZVXdLiIPicj1ACJynogcBBYCT4jIdu/TbwIuBe4QkU+9PzPC8kpM7PIl+qSMjrdJzrQWvTEdCKbXDaq6Eljpt+yBdvc34inp+D9vGbCslzGavq6h2nPbVY2++lhk4jHGZezKWBP9gqnRJ1mL3piOWKI30c+XwLss3ViN3phALNGb6NfoLd10ejI2w1PiUY1MTMa4iCV6E/1O9rrppHSTnAna4ulzb4w5hSV6E/0aqiAuARKSO97GxrsxpkOW6E30a6jy1OcDzS7lk2SJ3piOWKI30a+huvP6PHhq9ACNluiN8WeJ3kS/xuq2RN4RX48cX597Y8xJluhN9Ots5Eofq9Eb0yFL9Cb6NVZ33oce2hJ9o7XojfFnid5Ev4bulG6sRW+MP0v0Jvp1p3RjLXpjTmOJ3kS/xuq27pMdSUoHxFr0xgRgid5EN9Xget2IeMo31uvGmNNYojfRrakWtLXrk7Hg+TKw0o0xp7FEb6Kbr4XeVYsePF8GluiNOY0lehPdfIk72Ba9lW6MOU1QiV5E5opIgYgUish9AdZfKiIfi0iziCzwW3e7iOz2/tweqsBNHxHMWPQ+1qI3JqAuE72IxAOPAvOAKcCtIjLFb7P9wB3A837PHQT8CJgFzAR+JCIDex+26TMaazy3wZRukjOtRW9MAMG06GcChapapKqNwIvA/PYbqGqxqm4BWv2eOwd4W1XLVLUceBuYG4K4TV9xsnTTRfdK8LborXulMf6CSfQjgAPtHh/0LgtGUM8VkUUisklENpWUlAS5a9MnnJwvNpjSTXrbfwDGmJOi4mSsqi5V1XxVzc/JyXE6HBNNTrbo07ve1k7GGhNQMIn+EDCq3eOR3mXB6M1zjWlL3EGdjM2E5jpoaQ5vTMa4TDCJfiMwQUTGiEgScAuwIsj9rwKuFpGB3pOwV3uXGRMcXykm2O6VYD1vjPHTZaJX1WZgMZ4EvQN4WVW3i8hDInI9gIicJyIHgYXAEyKy3fvcMuDHeL4sNgIPeZcZE5zGKkhIhfiErrf1lXcs0RtziiA+PaCqK4GVfsseaHd/I56yTKDnPgU81YsYTV8WzBDFPr5Wv52QNeYUUXEy1pgONdYEdyIW2s0yZS16Y9qzRG+iWzBDFPucLN1YX3pj2rNEb6JbQ1X3SzfWojfmFJboTXTrSenGTsYacwpL9Ca6BTMxuI/1ujEmIEv0Jrr1pNeNlW6MOYUlehPdun0yVqx7pTF+LNGb6OWbLzbYGr1v3lgr3RhzCkv0Jno11Xnmiw22dAPeESwt0RvTniV6E726M42gj41gacxpLNGb6NWdaQR9rHRjzGks0ZvodXLkyiBr9GDTCRoTgCV6E718LXOr0RvTK5boTfRq6MZ8sT5WujHmNJboTfRq7MZ8sT52MtaY01iiN9GrO9MI+liL3pjTWKI30atHNfoMaKqF1pbwxGSMCwWV6EVkrogUiEihiNwXYH2yiLzkXb9eRPK8yxNF5BkR2SoiO0Tkh6EN38S07swX65Nss0wZ46/LRC8i8cCjwDxgCnCriEzx2+wuoFxVxwOPAD/3Ll8IJKvqmcC5wN2+LwFjutRQBfHJEJ8Y/HOSbIJwY/wF06KfCRSqapGqNgIvAvP9tpkPPOO9vxyYLSICKJAuIglAKtAIVIYkchP7GrsxcqWPjWBpzGmCSfQjgAPtHh/0Lgu4jao2AxVAFp6kXwMcBvYDD6tqmf8BRGSRiGwSkU0lJSXdfhEmRjV0Yyx6n5OlG5tO0BifcJ+MnQm0AMOBMcD3RGSs/0aqulRV81U1PycnJ8whGddorG6bNSpY1qI35jTBJPpDwKh2j0d6lwXcxlum6Q8cB74CvKmqTap6DPgQyO9t0KaPaKjqRYveEr0xPsEk+o3ABBEZIyJJwC3ACr9tVgC3e+8vAN5TVcVTrvkSgIikA+cDO0MRuOkDelSj9/4HYC16Y07qMtF7a+6LgVXADuBlVd0uIg+JyPXezZ4EskSkEPgu4OuC+SiQISLb8Xxh/FFVt4T6RZgY1VhjNXpjQiAhmI1UdSWw0m/ZA+3u1+PpSun/vOpAy40JSnfmi/WxGr0xp7ErY0306s58sT4n5421RG+MjyV6E51888V2t0XvmzfWWvTGnGSJ3kSnplrPfLHdrdGDp0um1eiNOckSvYlOvmkEu9uPHmyoYmP8WKI30cmXqHuS6JMy2r4ojDGW6E2UauzBxOA+yTYmvTHtWaI30amhB7NL+ST3s9KNMe1YojfRqSezS/kkZdjJWGPasURvolNjL2r0djLWmFNYojfRqaE3NfpMOxlrTDuW6COkvKaRjcVlHKusdzoUd/Al6pR+3X9uUga0NkFzQ2hjMsalghrrxvScqvL794v45Vu7aGhuJSFO+MYlY/nBnEnExYnT4UWvhiqQOEhM6/5zk/u17SMhObRxGeNClujD7LHVe/jFmwVcNWUIX5k5mr9uPczjq/dQ39TCg9dPdTq86OUb50Z68GXoq+s3VEJ6dmjjMsaFLNGH0eZ95Ty8qoBrpw/j17ecTVyccPmkHDJTEvjjh8VcOC6Lq6cOdTrM6NRQ1bMTsdDWJdNOyBoDWI0+bFpalR+t2MaQfin87MbpJ8s0IsL/u2YyZwzN5D9f+5z6phaHI41SvUr0vha9nZA1BizRh81ftx5m26FK7p17BhnJp/7jlBgfxwPXTeHQiTqWrdvnUIRRrqGqZxdLQbtZpizRGwOW6MNCVXns73sYl5PO9WcND7jNheOymTVmEH94fy8NzdaqP01jdc+6VoK16I3xY4k+DNYWHWfH4Uruvmxcpz1r7rl8HEcq63lj65EIRucS9ZU961oJbYnero41Bggy0YvIXBEpEJFCEbkvwPpkEXnJu369iOS1WzddRNaKyHYR2SoiKaELPzr97/r99E9N7LA173PphBxys9J4fv3+CEXmIlajNyZkukz0IhKPZ5LvecAU4FYRmeK32V1AuaqOBx4Bfu59bgKwDPimqk4FLgeaQhZ9FCqvaeSt7Ue48ZyRpCTGd7ptXJxw68zRbCguY0+J9RA5RUMVJPfv2XOT0j198C3RGwME16KfCRSqapGqNgIvAvP9tpkPPOO9vxyYLSICXA1sUdXPAFT1uKrGdEH69a2HaWpRbjx3RFDb/+PZIxCBv3xyKMyRuUhrq6fs0tMWvYjnufWVoY3LGJcKJtGPAA60e3zQuyzgNqraDFQAWcBEQEVklYh8LCI/CHQAEVkkIptEZFNJSUl3X0NU+csnh5gwOIMpw4KrLw/pl8KF47L486dfoKphjs4lejOgmU9yP2vRG+MV7pOxCcDFwFe9t/8oIrP9N1LVpaqar6r5OTk5YQ4pfA5X1LFpXznXnzUc6cYVnfPPGsH+slq2HbIWKOC5ohV6megz2/ZjTB8XTKI/BIxq93ikd1nAbbx1+f7AcTyt/zWqWqqqtcBK4JzeBh2t3tp+FIB5Zw7r1vOunDKE+Djhze2HwxGW+/RmvlgfS/TGnBRMot8ITBCRMSKSBNwCrPDbZgVwu/f+AuA99dQhVgFnikia9wvgMuDz0IQefd7cdoTxgzMYP7h7/b8HpScxa8wg3thm3SyBttp6T7tXgpVujGmny0TvrbkvxpO0dwAvq+p2EXlIRK73bvYkkCUihcB3gfu8zy0HfoXny+JT4GNV/WvoX4bzKuqa2FBcxtVThvTo+VdPGUJRSQ17S2tCHJkLnSzd9CLRp/Szk7HGeAU1qJmqrsRTdmm/7IF29+uBhR08dxmeLpYxbfWuElpaldmTe5boZ08ewoOvfc67O47yjUvGhjg6l6mv8Nz2JtFb6caYk+zK2BB5b8dRBqUnMWPUgB49f9SgNCYOyeC9ncdCHJkL9WbSER8r3RhzkiX6EGhtVdbsLuXSCdnE92IykSsmDWZjcRk1Dc0hjM6FQlW6aa6H5sbQxGSMi9l49CGw7YsKymoauXzS4F7t57KJOTyxpoi1e45zZQ9r/TGhvtJzZWtSes/34buqtqESEmzykVijqnz0xUe8ve9tDtccZkDyAC4beRlz8uYQH9f5Fel9kbXoQ2DNLs9FXhdP6F1COTdvIKmJ8aze5e6LxnqtodJTY+/J7FI+vrKPr95vYkZpXSmL3l7EN9/5Jm8Vv0V1YzUbjmzg3vfvZcFrC9hzYo/TIUYda9GHwJrdpUwb0Y/sjN7NT5qcEM8F47L4oLA0RJG5VH0lpPRwnBufk/PG2gnZWHKg6gB3rrqTE/Un+OHMH7Jw0kIS4xJRVd7Z/w5L1i3hqyu/yu9m/45zhsTsJTvdZi36XqppaOaT/eVcPD40V/RePD6bvaU1HCyvDcn+XKm+oucDmvn4viisRR8zjtcdZ9Fbi6hrruPpeU/zlclfITEuEfDM3HZV7lW8eO2L5KTmsPjdxewq3+VwxNHDEn0vbSguo6lFuXh8aOrAvvLPR4XHQ7I/V2oIQYv+ZOnGWvSxoKW1hfvev4+SuhJ+N/t3TM2aGnC7oelDWXrVUlISUvju379LdaONCguW6Hvtw92lJCXEkZ83MCT7mzA4g8GZybzfl8s39RUhSPTWoo8lz37+LOsOr+OHM3/I9JzpnW47LGMYv7j0FxyoOsBPN/w0QhFGN0v0vfTRnuPk5w7scuz5YIkIF47LYu2e4313NMvezC7lYzX6mLGvch+PfvooXxr1JW6YcENQz8kfms9d0+5ixZ4VfHToozBHGP0s0fdCeU0jnx+u5MJxWSHd74XjsimtbqDwWB/9tzMULfrkfoBYiz4G/GzDz0iMS+T+8+/v1qiwd591N3n98liyfglNLTE931GXLNH3wroiTx39gnGh7ad9gfeL48O+WL5pbfV2r+xliz4uzrMPS/Su9v7B9/ng0Ad886xvMjite9epJMcnc+/Me9lftZ/ndz4fpgjdwRJ9L6wtOk5aUjzTR/ay9eln1KA0Rg5MZV1RWUj36woNlYBCas+GkjhFSn9L9C7Wqq088vEjjMocxVfO+EqP9nHxiIu5aMRFPLHlCSob+24ZzxJ9L6zdc5z8vEEkxof+13j+2CzW7T1Oa2sfq9PXn/DcpoQg0af2h7oTvd+PccSq4lXsLt/N4hmLSYxP7PF+vnP2d6hqrOLZ7c+GMDp3sUTfQyVVDew+Vs0FY0Nbn/e5YGwWJ2qb2Hmkjw3M5WuB97ZGD54vC2vRu1KrtvLYZ48xfsB45o6Z26t9Tc6azFW5V/Hc589R0dA33w+W6Hto/V5Pff78sYPCsv/zvXV633mAPsPXAg9Z6cZa9G707v532Vuxl7un302c9D5N3T39bmqba3lh5wshiM59LNH30PqiMtKT4pk2IrT1eZ8RA1IZNSj15BdKn3GydBOiFr2VblxHVfn9lt+T2y+Xq3KvCsk+Jw2axOUjL2fZjmXUNvW9q84t0ffQuqLw1ed9zh+Txfq9ZX2rTn+ydBOKGv0Aa9G70IYjG9hRtoM7pt4R0pEo7zzzTioaKvhz4Z9Dtk+3CCpLichcESkQkUIRuS/A+mQRecm7fr2I5PmtHy0i1SLy/dCE7azj1Z76/KwwlW18zvfW6QuO9qE6fV255zYUpZvUAdBUa2PSu8yznz/LoJRBXDfuupDud0bODKZnT2fZjmW0tLaEdN/RrstELyLxwKPAPGAKcKuITPHb7C6gXFXHA48AP/db/yvgjd6HGx3W7/V0e5w1JjwnYn18XyTr+1Kdvu4ExCVAUvcmWA/I91+BtepdY2/FXtYcXMMtk24hOb53o8H6ExG+PvXrHKg6wN8P/j2k+452wbToZwKFqlqkqo3Ai8B8v23mA8947y8HZov3EjYR+QdgL7A9NCE7b12Y+s/7GzmwD/anryv3JOjejEXvkzqwbZ/GFV7Y+QKJcYksnBRwCupemz16NkPTh/LCjr51UjaYRD8CONDu8UHvsoDbqGozUAFkiUgGcC/wn50dQEQWicgmEdlUUhL9k26sLyrj3NyBYa3P+8wak8WG4j5Up68/0Zage8sSvatUN1bzl8K/MDdvLtmp4ZkVLCEugZsn3cz6I+spLC8MyzGiUbgz1YPAI6ra6aAtqrpUVfNVNT8nJzTjuodLWU0jBUerOD9M/ef9nT92EGU1jew61kfq9HXloanPQ9t+rOeNK7xe9Dq1zbXcesatYT3OjRNuJCkuiZcKXgrrcaJJMIn+EDCq3eOR3mUBtxGRBKA/cByYBfxCRIqBfwX+n4gs7mXMjvLVy8PVf96f7wtlfV8p39SVh6FF30d+dy6mqrxU8BJTsqZwZs6ZYT3WwJSBzMmbw2tFr1HTVBPWY0WLYBL9RmCCiIwRkSTgFmCF3zYrgNu99xcA76nHJaqap6p5wP8AP1HV34Yodkes31tGamI8Z44IUauzC6MGpTFiQB/qT19bDqkh+hL17cdKN1Fv89HNFJ4o5OZJN0fkeDdNuomaphr+WvTXiBzPaV0mem/NfTGwCtgBvKyq20XkIRG53rvZk3hq8oXAd4HTumDGCk//+YEkJUTuEoTzx2axrqisb4xPX1cGaSFK9Mn9QOKg1lr00e5Pu/5EZmImc/N6N9xBsM7KOYtJAyfxcsHLfeJzFVS2UtWVqjpRVcep6hLvsgdUdYX3fr2qLlTV8ao6U1WLAuzjQVV9OLThR1ZZTSM7j1Qxa0xkyjY+J+v0R2N8fPrmRmisDl2LPi7OU76x0k1UK68v5+19b3PtuGtJS0yLyDFFhIUTF1JQXsDW0q0ROaaT7MrYblh/cvz5yJyI9fHV6WN+3BtfQk4LUY0ePF8a1qKPaiv2rKCptYmFE8PTpbIjXx77ZVITUvnTrj9F9LhOsETfDW3jz0emPu/jq9Ov3RPjid6XkEN1MhY8ZSBr0UctVWX5ruXMyJnBhIETInrsjKQMrhlzDauKV1HVGNu92izRd0M4x5/vygXj+sD49Cdb9CHsQ52WZS36KLbp6CaKK4tZMHGBI8dfMHEBdc11rCxa6cjxI8USfZDCPf58Vy4c1wfGp6/xTp2YFsLfcdogqI3x/4RcbPmu5WQmZnJ13tWOHH9q1lTOGHQGy3cvj+mTspbog/TRHk8Sumi8M4ned17AF0dM8iXk9FC26LM9XyAx/CF2qxP1J3hn3ztcO+5aUhNSHYlBRFgwYQE7y3by+fHPHYkhEizRB2ntnuNkpiQwdXh4x7fpyLD+qYzNTuejWK7Tn6zRh7BXU1oWtDZBQwz/J+RSrxW9RmNro2NlG59rxl4T8ydlLdEH6cM9pZw/Nov4uBAMttVDF47PYn3RcZpaWh2LIaxqSz193xOSQrdP338HtTH8n5ALqSqv7HqF6dnTmThwoqOxZCZlMidvDiv3rozZK2Ut0Qdh//FaDpTVccmE8Ay0FKyLx2dT09jCZwdidOyWmtLQlm2g7cRuTQz/J+RCnxz7hD0VexxvzfucPCm7NzZPylqiD8IHhb76vLOJ/oKx2Yi0xRNzao5BeogHtfN9cdRE/6iofcnyXcvJSMxgTt4cp0MBYHr2dCYMnMCfCmKzfGOJPggfFJYwrH8KY7PTHY2jf1oi00f05/3dsZroS0Of6DMGe/dtiT5aVDRUsKp4FV8e++WIXQnbFd+VsjvKdrD9eMxMnXGSJfouNLe08sHuUi6ZkI2EYjKMXrpkQg6fHjhBZX2T06GEXk1JGEs3luijxYo9K2hsbYz4lbBduXasp/dPLLbqLdF34bODFVTWN3PpxOgYJ//SiTm0tCofxVr5prXF070y1C36xBTPCV5L9FFBVXm54GXPoGKDJjkdzikykzKZN2YeK/eujLkrZS3Rd2HNrhLixHMiNBqcPXoAGckJrN4VY4m+phS0FTKGhH7fGYOh+mjo92u6beORjRRXFkdda97npok3Uddcx+tFrzsdSkhZou/C33eVcNaoAQxIC2GXv15IjI/jovFZrC44FltX8vkScVgS/RCoPhb6/Zpue7HgRfon92fumMgMR9xdU7OnMjVrKi/tfCmmPl+W6DtRWt3AloMnuGLSYKdDOcUVkwbzRUV9bA1b7EvE4WrRVx0J/X5NtxytOcp7+9/jhvE3kByf7HQ4Hbp50s3sqdjDpqObnA4lZCzRd2LNrhJUibpEf7k3nvd2xlAr9WSLPgznQjKGWOkmCry862VatZWFk6KzbOMzb8w8+if354WdLzgdSshYou/EuzuOkZOZzNTh/ZwO5RRD+6cwZVg/3tsZQ8mr6gvPbeaw0O87c6hnQhMbBsExjS2NLN+1nMtGXsaozFFdP8FBKQkp3DjhRt7d/y6Hqw87HU5IBJXoRWSuiBSISKGInDZNoIgki8hL3vXrRSTPu/wqEdksIlu9t18Kbfjh09DcwupdJVw5eTBxDg570JErpwxh875yjlc3OB1KaFQd8YxDnxiGwa0yh7cdwzjijb1vUFZfxq2Tb3U6lKDcMukWAF4oiI1WfZeJXkTigUeBecAU4FYRmeK32V1AuaqOBx4Bfu5dXgpcp6pn4pk8/LlQBR5u64rKqG5o5srJYagZh8BVk4fQqjFUvqk8HJ7WPHha9ABVsdE6cxtV5bnPn2P8gPFcMOwCp8MJyrCMYcwePZvlu5ZT21TrdDi9FkyLfiZQqKpFqtoIvAjM99tmPvCM9/5yYLaIiKp+oqre/8nZDqSKSPSehWln1fYjpCXFOz7sQUemjejHsP4prNoeI+Wbqi/Cl+j7eVv0lV90vp0Ji41HNlJQXsBtU26LiosOg/X1KV+nqrGKPxf+2elQei2YRD8CONDu8UHvsoDbqGozUAH4D9x+I/CxqkZ9rXezUtcAABFUSURBVKGlVXlr+1GumDSYlMR4p8MJSESYM3Uoa3aXUN3Q7HQ4vVdxCPr7v61CpJ93v5WHwrN/06mntj/FoJRBfHnsl50OpVtmDJ7B9JzpPPv5szS3uvszFpGTsSIyFU855+4O1i8SkU0isqmkxPkrGDcVl1Fa3cDcaUOdDqVT86YNpbG5lb+5vXzT3OAZ0KzfyPDsPynNU/+vOBie/ZsOFZQV8OGhD/na5K9FdZfKjtw57U4OVR/inX3vOB1KrwST6A8B7U+Tj/QuC7iNiCQA/YHj3scjgVeBr6vqnkAHUNWlqpqvqvk5Oc4PNfD6lsOkJMbxpTOiq1ulv/y8QQzOTOb1LS4vSfha2v3DlOh9+66wFn2kPbntSdIS0rhp0k1Oh9IjV4y6grx+efxh6x9cfQFVMIl+IzBBRMaISBJwC7DCb5sVeE62AiwA3lNVFZEBwF+B+1T1w1AFHU7NLa2s3HqY2WcMIT05welwOhUfJ1xz5jD+VlDi7kHOTngrg2FN9KOh4kDX25mQKa4oZlXxKm4+42b6JzszM1tvxUkc3zjzGxSUF7D64Gqnw+mxLhO9t+a+GFgF7ABeVtXtIvKQiFzv3exJIEtECoHvAr4umIuB8cADIvKp9yeqm8kfFJZyvKaR684K04nBELt+xnAam1t5c5uLuw6e2Oe5HZgbvmMMGA3l+2zu2Aj6/dbfkxSXxO1Tbu964yh2zdhrGJExgic+e8K1rfqgavSqulJVJ6rqOFVd4l32gKqu8N6vV9WFqjpeVWeqapF3+X+parqqzmj3E9UF5Vc+PkT/1ESuiPKyjc/ZowYwJjudVza7uP58Yj9IfPhq9OD5EmmqaZuA3ITV3oq9vF70OjdNuomsVP9+Ge6SGJfI3dPvZtvxba5t1duVse1U1DXx1vYjXH/WcJITorO3jT8R4YazR7B+bxkHylza37e82NPjJj6MpbKBeZ7bsr3hO4Y56Xef/o7k+GTuOvMup0MJievGXcfozNH85pPf0Krum7PZEn07f/n0EA3NrSzMD2PLMgxuPHckcQIvbXRpDfr4Hhg0LrzHGDTWc1tWFN7jGLaVbuPN4je5bcptDEoZ5HQ4IZEQl8Disxezq3yXK4cwtkTvpao8v34/00b0Y/rIAU6H0y3DB6RyxaTBvLzpAE0tLmttqHqSb1aYE/3APECgLGDHLxMiqsrDmx5mUMog7px2p9PhhNScvDlMzZrKrz/+NXXNdU6H0y2W6L027C1j55EqvjorjCcEw+ir54/mWFWD+07K1pRC/QnIGh/e4yQke07Ilu4O73H6uFX7VrH56Ga+PePbpCc6O8dyqMVJHN/P/z5Ha4/yx21/dDqcbrFE7/XHD4sZkJbIP8wI09WZYXb5xMHkZaXxxw9dVoMuLfDcZk8M/7FyJkHprvAfp4+qbarl4Y0Pc8agM7hxwo1OhxMW+UPzmZs3l6e2PcWBKveUSi3RA0Ul1az6/AhfnTWa1CR3nIT1Fxcn3HnxGD7ef4INe8ucDid4JTs9tzkRmD80Z5KnRd/i7svZo9VvPvkNR2uPcv+s+4mPc+fnKBjfy/8e8RLPj9f+2DXdLS3RA4+v3kNSfBz/dNEYp0PplYXnjiIrPYlH/1bodCjBO7odUvq3jUcTToOnQkuD1enD4NNjn/K/O/6XmyfdzIzBM5wOJ6yGpg/lO+d8h7WH1/KXPX9xOpyg9PlEv7e0hlc+PsStM0eTneG+sTjaS02K5xuXjGX1rhI27yt3OpzgHNkGQ6ZBJEY1HDrNe8yt4T9WH1LbVMv9H9zPsPRh/Os5/+p0OBFx86SbOWfwOfxsw884VB39Q2v0+UT/y7cKSIqP49tXhPlkYITcfmEu2RnJ/PyNndH/b2VLExzZAsMi1ALMOQPik+GLTyJzvD5AVVmyfgkHqg7wXxf/FxlJGU6HFBHxcfH85JKfAHDvmntpao3uIUj6dKLfWFzG61sO88+XjiUn092teZ+0pAT+7aoJbCgu441o74FzbAc018OIcyJzvPhEGHomHPo4MsfrA17Z/Qor9qzgnrPu4byh5zkdTkSNyBjBgxc+yGcln/GrTb9yOpxO9dlE39jcyn+8uo3h/VO457Iw9+GOsFvOG83kYf146LXPqYrmwc4OrPfcjpoZuWOOmgVffOwZGtn0ysYjG1mybgkXDb+IRdMXOR2OI+bmzeVrk7/Gsh3LeGXXK06H06E+m+h/+95uCo5W8eN/mObanjYdiY8TfnrDmRytqmfJX3c4HU7Hit/3jG/TP4KTRede4Pkv4tDmyB0zBhWUFfAv7/0Lo/uN5r8v+++Y7mXTle/lf4+LRlzEj9f9mL/t/5vT4QTUJxP9uqLj/PZvhdx4zkhmR+mcsL01Y9QAvnnZOF7ceCA6x6tvaYai1TD28siciPXJuwQkDva8F7ljxpiCsgL++a1/Jj0xncevfJzMpEynQ3JUQlwCv7rsV0weNJnvrf4eaw6ucTqk0/S5RH+grJZv/+/H5GWn85/zpzodTlh996qJnDN6AD9YvoXtX1Q4Hc6pDqzzXBE74crIHjd1AIw6HwreiOxxY8TGIxv5p1X/RFJ8Ek/NeYphGe4Yzjvc0hLTePyqx5kwcALfee87rNjjP2WHs/pUoj9WWc/Xn9pAU0srS2/LJyPKJxbprcT4OB772rn0T03kjj9uZG9pjdMhtdn2CiSkwvgIJ3qAydfB0W1QUhD5Y7uUqvLizhdZ9PYislOzeWbeM4zuN9rpsKJK/+T+/OHqP3DukHO5/4P7+eWmX0ZNb5w+k+j3ltZw0xNrOVpZz1N3nMf4wX2jG9iQfik8e+dMWluVm55Yy7ZDUdCyb6iGrcs9CTfZgX/7p90IcQmw+ZnIH9uFjtYc5V/+9i8sWb+EC4ZdwHPznmNEhjuHCgm3zKRMHrvyMW6edDNPb3+a21bexu5y58dX6hOJfuXWw8z/7QdU1jez7BuzyM+LjaFTgzVhSCYv3X0+SfFxLHj8I55fv9/ZPvYbfw8NlTAr4Fzx4Zc5BKb+I2x+Gqqdn4w+WtU21bJ0y1Ku//P1rP1iLT847wf8dvZvXTstYKQkxifyH+f/B7+87Jccqj7EwtcW8tP1P6W0rtSxmCTaLqrJz8/XTZs2hWRfu45W8Ys3C3hnx1Gmj+zPo185h1GD0kKybzcqqWrg3176lA8KS5k1ZhA/vGYyM0ZFeEjmE/vhdxd6er989U+RPXZ7JbvgsQtg2gK44Qnn4ohC+yv388ruV1i+azmVjZVcMeoK/v28f2dUZgR7R8WI8vpyfvPJb/i/3f9HQlwC1469loUTFzIlawoS4k4IIrJZVfMDrlTVLn+AuUABUIhnom//9cnAS97164G8dut+6F1eAMzp6ljnnnuu9saJmkb98ycH9bYn12vuva/r1Afe1Mf+XqiNzS092t+yZcs0NzdXRURzc3P1nnvu0aysLAVO+8nKytLhw4cHXBfMT1ZWlmZlZZ1yrNzcXAU0Pj5eAc3NzdVly5adEl/7eOLi4gJu59PS0qrL1hXr2Q+9pbn3vq4LH/tIX9ywT0uq6nv8Ow9aS7Pq45eo/mSkatneLjf3f23B/g4Dve6A3lui+qN+qpuf7d3rcrmaxhrdcHiD3v3M3Tr5J5N12tPTdOqTU/WGZ27Qz459pqqnfw7834P+n5FA2y5btkzT09NPea/ec889p8Xj21/7973vfR3Mj2+//u8fETntvm//vtusrKxTYuzW+ykA32tJHpqsExdP1BlPz9BpT0/Ta165Rn+x4Re6+sBqPVF/osf7bw/YpB3l8I5WaFuijgf2AGOBJOAzYIrfNt8CHvfevwV4yXt/inf7ZGCMdz/xnR2vp4n+ixO1+uVfr9Ex972uufe+rhf85B39n7d3aVl1Q4/2p+r5I6WlpfU4cYfrJy0tTZctW6bLli3TxMTELrcLpKq+SZeu3qOX//ffNPdez+/s6l+t1s+/qOjx7yso215V3fVWl5t19do6+0lMTAzuw9nSrPrOQ6pVx0Lwwtzl1d2v6nfe+45e9+p1Ov2Z6Z7k/tRUHXP/GM2am6UJAxJOeZ/5fw46WxfofXjPPfd0mKzbJ/tQfua68+XQ2U9SUlKPkn2g15KRlaHff/b7uuitRXr2s2frtKen6bSnp+mc5XP0W+98S5d+trTHf9POEn2XpRsRuQB4UFXneB//EEBVf9pum1XebdaKSAJwBMgB7mu/bfvtOjpeT0s3TS2t3PXMJmaMGsBlE7M5e9RA4uJ6969RXl4e+/bt69U+wiU3Nxegy/hyc3MpLi7ucL2qsv2LSlbvKmFd0XF+fcvZDExPCmWoPdLb331Xr7uve3jjw6w5tIa8fnlMHDiRR+59hOIPi2mtPXWGss7eZ8G+BwHi4+NpaWnpcF1zs2fo6Gj9zPXk/dTRa/Htq665jq0lW9lSuoWCsgKKKooYnj6c38z+TY9i7Kx0E0yiXwDMVdVveB/fBsxS1cXtttnm3eag9/EeYBbwILBOVZd5lz8JvKGqy/2OsQhYBDB69Ohzo+UPHRcXF7UDg/nqe0H8/Whtddn0gvT+d+/W1+2Ujn7fnb3Pgn0PBsO3j2j9zPXk/dTZ7zQc783OEn1U9LpR1aWqmq+q+Tk5OU6Hc9Lo0dHbT3j06NFBxRfNr6EzvY3bra/bKR39vjp7nwX7HgRPqz2YddH6d+tJXJ393iItmER/CGh/un2kd1nAbbylm/7A8SCfG7WWLFlCWlr09dJJS0tjyZIlLFmyhMTExC63c6OuXltnEhMTXfu6nRLovd7+fdaddf7S0tJYtGgRcXGB082iRW0DooXyM9fR8borKSmpR++nzn5vEddR8d73AyQARXhOpvpOxk712+bbnHoy9mXv/amcejK2iDCdjA2XWOt14yZh73VjTtGdnjXW66b3v9NQozcnYwFE5Brgf/D0wHlKVZeIyEPeHa8QkRTgOeBsoAy4RVWLvM+9H7gTaAb+VVU7HWQklP3ojTGmr+jVydhIs0RvjDHdF/UnY40xxoSPJXpjjIlxluiNMSbGWaI3xpgYF3UnY0WkBIiOS2PDIxtwbrzSyLLXGrv60ut1y2vNVdWAV5xGXaKPdSKyqaMz47HGXmvs6kuvNxZeq5VujDEmxlmiN8aYGGeJPvKWOh1ABNlrjV196fW6/rVajd4YY2KcteiNMSbGWaI3xpgYZ4k+wkTkQRE5JCKfen+ucTqmcBCRuSJSICKFInKf0/GEk4gUi8hW798z5kbkE5GnROSYdyY537JBIvK2iOz23g50MsZQ6eC1uv4za4neGY+o6gzvz0qngwk1EYkHHgXm4Zkg/lYRmeJsVGF3hffv6er+1h14Gpjrt+w+4F1VnQC8630cC57m9NcKLv/MWqI34TATKFTVIlVtBF4E5jsck+khVV2DZ56J9uYDz3jvPwP8Q0SDCpMOXqvrWaJ3xmIR2eL9NzEm/uX1MwI40O7xQe+yWKXAWyKy2TvRfV8wRFUPe+8fAYY4GUwEuPoza4k+DETkHRHZFuBnPvAYMA6YARwGfulosCYULlbVc/CUqr4tIpc6HVAkeaexi+V+2q7/zCY4HUAsUtUrg9lORH4PvB7mcJzg6knhu0tVD3lvj4nIq3hKV2ucjSrsjorIMFU9LCLDgGNOBxQuqnrUd9+tn1lr0UeY90Ph84/Ato62dbGNwAQRGSMiSXgmjF/hcExhISLpIpLpuw9cTWz+Tf2tAG733r8d+IuDsYRVLHxmrUUfeb8QkRl4/tUtBu52NpzQU9VmEVkMrKJtQvntDocVLkOAV0UEPJ+n51X1TWdDCi0ReQG4HMgWkYPAj4CfAS+LyF14hhW/ybkIQ6eD13q52z+zNgSCMcbEOCvdGGNMjLNEb4wxMc4SvTHGxDhL9MYYE+Ms0RtjTIyzRG+MMTHOEr0xxsS4/w/i08xn/R2aYwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "stds = tuple([(x**0.5)/2 for x in spreads])\n",
    "print(f'Means:{centers}. \\nStds :{stds}')\n",
    "plot(data,centers,stds,(1/3,1/3,1/3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Expectation Maximization \n",
    "Now let's define the EM algorithm. A few points to highlight are:\n",
    "* The function expects the data points (dist), the number of Gaussians wished to use as clusters (K) and the maximum iterations allowed (iterations). \n",
    "* The initial $\\sigma_{i}$ and $\\mu_{i}$ are randomly selected. For example, the means for each cluster is a random integer between the maximum and minimum samples observed.\n",
    "* Initially, the probability of samples belonging to a cluster assigned to each cluster is equally distributed.\n",
    "* A maximum number of iterations is established in case convergence criterion is not met. This rarely happens, but could be so due to the initialization values of the parameters. \n",
    "* On each iteration the $\\sigma$ and $\\mu$ expected are printed.\n",
    "* Upon convergence, a Gaussian with the last parameters is plotted."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "def EM(dist, K, iterations):\n",
    "    M = dist.shape[0]\n",
    "    mus = np.random.randint(min(dist),max(dist),K)\n",
    "    pis = np.ones(K) / K\n",
    "    sigmas = np.random.rand(K) + 1\n",
    "\n",
    "    ll_old = -np.inf\n",
    "\n",
    "    for iter in range(iterations):\n",
    "        print(f'Iteration: {iter}. Means: {mus}. Stds: {sigmas}')\n",
    "\n",
    "        # E-step\n",
    "        w = np.zeros((M, K))\n",
    "        for k in range(K):\n",
    "            w[:, k] = pis[k] * eval(dist, mus[k], sigmas[k])\n",
    "\n",
    "        summed = w.sum(1)\n",
    "        for m in range(M):\n",
    "            w[m, :] /= summed[m]\n",
    "\n",
    "        # M-step\n",
    "        N_k = w.sum(0)\n",
    "        pis = N_k / M\n",
    "\n",
    "        mus = np.zeros(K)\n",
    "        for k in range(K):\n",
    "            for m in range(M):\n",
    "                mus[k] += w[m, k] * dist[m]\n",
    "        mus /= N_k\n",
    "\n",
    "        sigmas = np.zeros(K)\n",
    "        for k in range(K):\n",
    "            for m in range(M):\n",
    "                sigmas[k] += w[m, k] * (dist[m] - mus[k]) * (dist[m] - mus[k]).T\n",
    "        sigmas /= N_k\n",
    "        sigmas = sigmas ** 0.5\n",
    "\n",
    "        # Convergence criterion\n",
    "        ll_new = 0.0\n",
    "        for k in range(K):\n",
    "            ll_new += pis[k] * eval(dist, mus[k], sigmas[k])\n",
    "        ll_new = np.log(ll_new).sum()\n",
    "\n",
    "        if np.abs(ll_new - ll_old) < 0.00000001:\n",
    "            plot(dist, mus, sigmas, pis)\n",
    "            break\n",
    "\n",
    "        ll_old = ll_new\n",
    "\n",
    "    return mus, sigmas, pis"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now it's time to run an experiment. We feed the algorithm with the data, the number of Gaussians we want to use and the maximum iterations. Keep the random seed fixed incase you want to get reproducible resutls.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration: 0. Means: [-1  5  6]. Stds: [1.93255736 1.12812445 1.99904052]\n",
      "Iteration: 1. Means: [-2.23540011  2.04655663 10.55462241]. Stds: [3.4095181  0.39274316 4.59342537]\n",
      "Iteration: 2. Means: [-2.7721669   1.97923505 11.8155787 ]. Stds: [3.26372426 0.15797764 3.48712817]\n",
      "Iteration: 3. Means: [-4.16353486  1.97824017 12.75254271]. Stds: [2.27324942 0.14416836 1.69160446]\n",
      "Iteration: 4. Means: [-4.7693781   1.9710305  12.79884873]. Stds: [1.26960965 0.16447639 1.53844225]\n",
      "Iteration: 5. Means: [-4.89578217  1.96460823 12.79884871]. Stds: [0.90317368 0.17240375 1.53844225]\n",
      "Iteration: 6. Means: [-4.89579073  1.96460771 12.79884871]. Stds: [0.9031436  0.17240429 1.53844225]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de5hT1b3/8fc3mcnAzHCRYUS5DQgjiGhV8FZ7qlVAvCGoVdBfaz22nHo5Pbban7YeqZeHtmrraZ9WPVKr7U+gagUUlaKtpWorVtBaKAoyIMrd4X4Z5pJk/f5IMswMmUkmyWSTzOf1PPMk2XtNspLM/rD47rX3NuccIiKS+3xed0BERDJDgS4ikicU6CIieUKBLiKSJxToIiJ5osCrF+7du7cbNGiQVy8vIpKT3n333W3OufJ46zwL9EGDBrF06VKvXl5EJCeZ2SetrVPJRUQkTyjQRUTyhAJdRCRPKNBFRPKEAl1EJE8o0EVE8oQCXUQkTyjQxVvOwZLHYcfHXvdEJOcp0MVb2z6Cl2+Fl27xuiciOU+BLt7a9WnkdtP73vZDJA8o0MVbNdsjt2be9kMkDyjQxVu1uyO3pj9FkXRpKxJv1e2J3OratiJpU6CLt+r2RW6Ddd72QyQPKNDFWw01B281ShdJiwJdvFUfDXQcBGs97YpIrlOgi7diI3RoEu4ikgoFunir4cDB+8EDrbcTkYQU6OKtpmUW7RgVSYsCXbzVNNAbNEIXSYcCXbylEbpIxiQV6GY23sxWmVmVmd3RRrvLzcyZ2ejMdVHyWkMtdOkRua8aukhaEga6mfmBh4ELgBHAFDMbEaddN+C/gL9nupOSx0J1UBQLdI3QRdKRzAj9NKDKObfWOVcPPA1cGqfdfcD9gCYTS/KC9VDULXI/VO9tX0RyXDKB3g9Y3+TxhuiyRmZ2CjDAOfdyW09kZlPNbKmZLa2urm53ZyUPheoOBrpG6CJpSXunqJn5gIeAWxO1dc7NcM6Nds6NLi8vT/elJR8E66BL94P3RSRlyQT6RmBAk8f9o8tiugEjgb+Y2TrgDGC+doxKUkL1ECiN3legi6QjmUBfAlSa2WAzCwCTgfmxlc653c653s65Qc65QcDbwATn3NIO6bHkD+cio3LV0EUyImGgO+eCwM3AK8CHwLPOuRVmdq+ZTejoDkoeC4cA1yTQGzztjkiuK0imkXNuAbCgxbJprbQ9J/1uSacQK7HESi6qoYukRUeKindiJZaiWA1dI3SRdCjQxTuxAC/oErmmqHaKiqRFgS7eiY3QC4rAH9AIXSRNCnTxTizQ/QEFukgGKNDFO7EA9xdGfjRtUSQtCnTxTtMRuk+BLpIuBbp4RyUXkYxSoIt3VHIRySgFungnFuC+WKBr2qJIOhTo4p1mJZdCCAW97Y9IjlOgi3diAe4viIzSw6qhi6RDgS7eiQV4Y8lFgS6SDgW6eEezXEQySoEu3mksuRSCr0AlF5E0KdDFO40llwJNWxTJAAW6eKex5FIYPVJUs1xE0qFAF+/EAtxXGJnpopKLSFoU6OKdWIDHpi1qp6hIWhTo4p3GQ/+js1zCKrmIpEOBLt5pNg+9QCN0kTQp0MU7zaYt6khRkXQp0MU74QYwP5jpXC4iGaBAF++EGiJBDjqwSCQDFOjinXAoEuQQDXSN0EXSoUAX74QbDga6Ts4lkjYFuninWcmlEHCRUbuIpESBLt4JN0SDnMi0RdAoXSQNCnTxTih4MMhjwa4doyIpU6CLd8LB5jtFY8tEJCUKdPFOs5JL9FZz0UVSpkAX74SCzeehg0ouImlQoIt3mpZcYsGukotIyhTo4p2m89B9muUiki4Funin5aH/oBG6SBoU6OKdpof+q+QikjYFuninWcklNstFJReRVCUV6GY23sxWmVmVmd0RZ/03zWy5mb1vZn81sxGZ76rknXCTWS6NI3Qd+i+SqoSBbmZ+4GHgAmAEMCVOYM92zp3gnDsJeAB4KOM9lfwTanpgkT9yq2mLIilLZoR+GlDlnFvrnKsHngYubdrAObenycMSwGWui5K3VHIRyaiCJNr0A9Y3ebwBOL1lIzO7CfgOEADOzUjvJL+F4x1YpJ2iIqnK2E5R59zDzrkhwO3Af8drY2ZTzWypmS2trq7O1EtLrgo1aJaLSAYlE+gbgQFNHvePLmvN08DEeCucczOcc6Odc6PLy8uT76Xkp3DwYKklVkNXyUUkZckE+hKg0swGm1kAmAzMb9rAzCqbPLwIWJ25LkreCsc7fa5G6CKpSlhDd84Fzexm4BXADzzhnFthZvcCS51z84GbzWwM0ADsBK7tyE5LngjFOfRfs1xEUpbMTlGccwuABS2WTWty/78y3C/pDJqWXHT6XJG06UhR8U44eLB2rhG6SNoU6OIdnZxLJKMU6OIdXYJOJKMU6OKNcAhwqqGLZJACXbwRG4k31tD9zZeLSLsp0MUbseBurKFrHrpIuhTo4o3YEaGahy6SMQp08UbsvOe+lrNcdD50kVQp0MUbsZF4Yw3dB+bTuVxE0qBAF2+0rKFDZLSuGrpIyhTo4o2WNfTYfQW6SMoU6OKNljV0iJx5UYEukjIFunijZQ0dIiN01dBFUqZAF2+ohi6ScQp08YZq6CIZp0AXbzTW0JsGul+BLpIGBbp4I14N3V+oGrpIGhTo4o3Gk3M1raGr5CKSDgW6eKPVnaI69F8kVQp08UbsvOeH1NBVchFJlQJdvBGOE+h+TVsUSYcCXbwRbmXaonaKiqRMgS7eiFtDL1ANXSQNCnTxRtwaeoFq6CJpUKCLN1peUxQ0D10kTQp08UZjDV0lF5FMUaCLN1qtoWuWi0iqFOjijXg1dH+haugiaVCgizc0bVEk4xTo4o14Bxbp0H+RtCjQxRuhODV0v6YtiqRDgS7eiDtC105RkXQo0MUb4QYwP5gdXOYrPDhyF5F2U6CLN0INzcstoJKLSJoU6OKNcKj5QUWgkotImhTo4o1wQ/PD/iFactEIXSRVCnTxRtySSyHgNHVRJEVJBbqZjTezVWZWZWZ3xFn/HTP7wMyWmdlrZlaR+a5KXgk3xCm5REfsGqWLpCRhoJuZH3gYuAAYAUwxsxEtmv0DGO2cOxF4Dngg0x2VPBMKRnaCNhULeO0YFUlJMiP004Aq59xa51w98DRwadMGzrlFzrma6MO3gf6Z7abknXCw+Rx0OFiC0Y5RkZQkE+j9gPVNHm+ILmvN9cAf4q0ws6lmttTMllZXVyffS8k/cUsu0YDXXHSRlGR0p6iZ/R9gNPBgvPXOuRnOudHOudHl5eWZfGnJNaHgoTtFY4GukotISgoSN2EjMKDJ4/7RZc2Y2RjgTuBs51xdZroneSvc0HrJRTtFRVKSzAh9CVBpZoPNLABMBuY3bWBmJwOPAROcc59lvpuSd+JNW/Sphi6SjoSB7pwLAjcDrwAfAs8651aY2b1mNiHa7EGgFPi9mb1vZvNbeTqRiHg7RWPTFhXoIilJpuSCc24BsKDFsmlN7o/JcL8k37U1y0UlF5GU6EhR8UabJRcFukgqFOjijXjTFhtH6Cq5iKRCgS7e0LRFkYxToIs3NG1RJOMU6OIN1dBFMk6BLt4IB+PU0HXov0g6FOjijVC8C1yohi6SDgW6eCPcRslFNXSRlCjQxRuheCUXHfovkg4Fungj3ghds1xE0qJAF29olotIxinQJfucS3CkqAJdJBUKdMm+cChy2+qRoqqhi6RCgS7ZFyup6EhRkYxSoEv2heojt/5A8+Wxx7H1ItIuCnTJvtiRoLpikUhGKdAl+1orufh8YH6VXERSpECX7Gut5AKRUbtKLiIpUaBL9sVG4C1LLhAJeZVcRFKiQJfsC7dSQ4dIGUYlF5GUKNAl+2IllZYHFkFkhK6Si0hKFOiSfY0ll3g1dJVcRFKlQJfsawz0gkPX+QsgWJfd/ojkCQW6ZF+bs1wCOjmXSIoU6JJ9CactKtBFUqFAl+xLNG1RO0VFUqJAl+xLVHJRoIukRIEu2RduY5aLr+DguV5EpF0U6JJ9wdg89HizXAIQ0iwXkVQo0CX7YiWVgqJD16nkIpIyBbpkn2a5iHQIBbpkX1tHihYUaYQukiIFumRfwlkuGqGLpEKBLtnX5rlcdD50kVQp0CX7QnWAgc9/6Dp/QOdyEUmRAl2yL1gXqZWbHbpOs1xEUpZUoJvZeDNbZWZVZnZHnPVfNLP3zCxoZldkvpuSV0IN8cstoEAXSUPCQDczP/AwcAEwAphiZiNaNPsU+BowO9MdlDwUqms90AuKIudDD4ez2yeRPJDMCP00oMo5t9Y5Vw88DVzatIFzbp1zbhmgrVASC9XHP6gIDga9Ruki7ZZMoPcD1jd5vCG6rN3MbKqZLTWzpdXV1ak8heSDYH38My1Ck0DXjlGR9srqTlHn3Azn3Gjn3Ojy8vJsvrQcTkJ14G9lhB4buQc1Qhdpr2QCfSMwoMnj/tFlIqkJ1kNBGztFQSN0kRQkE+hLgEozG2xmAWAyML9juyV5LakRugJdpL0SBrpzLgjcDLwCfAg865xbYWb3mtkEADM71cw2AF8GHjOzFR3ZaclxsXno8cRG6Ap0kXaLc0LqQznnFgALWiyb1uT+EiKlGJHEgnVQ1C3+uoIukVuVXETaTUeKSvaF6qCwa/x12ikqkrKkRuhy0Ieb9/DKii3sORBkRN/uXHzi0XQpjHNOEmldsK0Di6Ij9GBt9vojkicU6EmqbQhxz4sf8Lt3PsVnUFTg50BDiJ++uoqfXXUSpx9T5nUXc0ew9mBwt9QY6Cq5iLSXSi5JqG0I8fXfLuV373zK1C8ew3t3jeWDe8/nd984g66Ffr7y63f488qtXnczdwTbOFI0tlw1dJF2U6An4JzjjjnL+GvVNh684kS+f+Fx9CwOYGacOaSMeTeexbCjunHjrPdYsWm3193NDcHaNgI9OkJvUMlFpL0U6Ak8s2Q9z7+/iVvHHsuXRw84ZH2P4kKe+Nqp9Owa4KZZ77G/LuhBL3NMmyWXooNtRKRdFOht2LjrAPe+9AFnDS3jpi8NbbVdebcifj75JD7ZUcMDC1dmsYc5yLlIWLc6y0U7RUVSpUBvw93zV+Ac3H/5ifh8cS7G0MTpx5Rx7ZmD+H9vf8KyDbuy1MMcFNvZ2doIvTBWcjmQnf6I5BEFeiveXF3NHz/YyrfOq6T/EcVJ/c6t446lrKQo+g+B6+Ae5qhgNKgTznLRCF2kvRTocYTCjukvf8iAXl359y8MSvr3unUp5LvnH8t7n+7iD//a0nEdzGWxnZ2tlVz8heAr0AhdJAUK9Dhe/OcmVm7Zy23jhlFU0L6Dhq4YNYBj+5Tyk1dWEQzpeh+HiI3QWwt0gIKuGqGLpECB3kIwFOZnf/qI447uziUn9m337/t9xq3jhrF2237m/kNnGT5EQ4KSC0Tq6Bqhi7SbAr2Fef/YyLrtNXx7TGXCHaGtGTeiDyP7deeXf66iQaP05hqSGKEXdlWgi6RAgd5EMBTm4UVVHN+3O2NH9En5ecyMW847lk931PC8RunNNdREbgvb2NFcWHywnYgkTYHexIvLNrFuew3/eW4lZqmNzmPOO+5IRhzdnUf+soZQWDNeGjWO0NsKdI3QRVKhQI8Khx0PL1rDsD7dGJfG6DzGzPjPc4fy8bb9vLx8cwZ6mCdiI+9AW4FeohG6SAoU6FGvfrCFqs/2ceOXhqRcO2/p/OOPYuiRpTyyqIqwRukR9bGSS4Iaev3+7PRHJI/o9LlETsD18KI1DCor5uIUZra0xuczbjxnCN959p+8tvKztOryeSMW1IHS1tsESmDXJ9npj3gmGA7y/mfv8/bmt/nXtn+xbs86dtTuoD5UT6GvkLKuZVR0r2Bk75GccfQZnHzkyRT4FFlt0acDvP5RNcs37ub+y0/An6HRecyEz/Xlf/70Eb9cVMWY445Muzaf8xqigd5WDT1QcnAkL3ln/Z71PLPqGV5c+yI7anfgMx+VPSs5sfxEyruWU+Qvoj5UT/WBatbuXsvjyx9nxrIZ9OrSiwsHX8iU4VMY2H2g12/jsNTpA905xy/+XEXfHl2YdHLmL4ta4Pdxw9lD+f685by5ehtfPLY846+RU+r2AdZ2ySVQAvV7s9YlyY61u9by6D8f5ZV1r+D3+Tmn/zlcMPgCPt/385S28T+2/Q37eWvTWyz8eCFPr3qaWR/OYkzFGG783I0MPaL1k+Z1Rp0+0Bev2c67n+zk3kuPJ1DQMbsULh/Vj1/8eTW/+PNq/q2yd+cepdfvj1wguq3PIFAaaedc2+0kJ+yq3cUv/vELnlv9HF38Xbhu5HVcc9w1HFl8ZFK/X1JYwtiKsYytGMu2A9uY/eFsZq+czWufvsakoZP41infoleXXh38LnJDp94p6pzjZ6+t5shuRVwZ51znmVJU4OebZw9hybqdLF6zvcNeJyfU7227fg5QVArhoC5Dl+Occ8xbPY+Ln7+YOavnMHnYZP5w+R/49qhvJx3mLfXu2ptvnfItFl62kKuHX80LVS9w8byLmfPRHJ0Qj04e6IvXbOedj3dw4zlDOvxCz1edOoCjunfhoT9+1Ln/8Or2RkbobSnqfrCt5KRN+zYx9Y9TmfbWNIb0GMLvL/k93zv9exkbSffs0pPbT7ud5yY8x7AjhnH34rv5+qtfZ8PeDRl5/lzVaQPdOcdPXl3F0T26MPm0jt/B0qXQz03nDmXpJzt5/aPqDn+9w1bdXujSve02jYG+p+P7IxnlnOP5que5bP5lLKtexn+f/t88Of5JKo+o7JDXG9JzCL8+/9dMO3MaK7av4PL5lzN39dxOO2jqtIH+xw+28t6nu/jWeZUdPjqPuWr0AAb06soDC1d13nnptXuSGKFH19fqGq25ZHfdbm59/Vbu+ttdDO81nDkT5nDV8KvwWcfGjM98fPnYLzN3wlxG9h7JD976AbcsuoWdtTs79HUPR50y0BtCYe5fuJJjykv48qjMz2xpTaDAx23jhvHB5j288M9Oeo6X2t3QpUfbbWLrNULPGUu2LOGy+ZexaP0ibjnlFn497tf075a9bQugb2lffjXuV9w2+jbe3Pgml8+/nLc3v53VPnitUwb67L9/yprq/XzvguMo8Gf3I7jkxL6c2L8HDyxcRU19J7ygdO2uxIHetWfk9oAu5Xe4awg38PP3fs71r1xPcUExMy+cyfUnXI/fl53/9bbkMx/XHn8tsy+aTWmglKmvTuWhpQ/REGrwpD/Z1ukCffu+On766irOGlrGmONS29OeDp/PuOviEWzeXcsji9Zk/fU95Rwc2Aldj2i7XZdYoHe+/zLnknW71/HVBV/l8eWPM6lyEs9c/AzHlx3vdbcAGN5rOM9c/AxXHHsFT654kmsWXMPa3Wu97laH63SBPv3lDznQEOLuS473bD74qYN6cdnJ/XjsjTWs3tqJZnLU7Y1MR0wU6MXRmRAHdnR8n6TdnHM8u+pZrnzpStbvW89D5zzEPZ+/h+K2jv71QNeCrkw7cxo/+9LP2Lx/M1e9eBWzP5xN2OXvNQo6VaAvWvkZc/+xkW+ePYTKPgl2zHWwOy86jtKiAr773LLOc6m6mm2R25IER8sWdo2ccXF/J5+zfxjasn8LN/zpBu57+z5OKj+JOZfMYWzFWK+71abzBp7H3AlzGX3UaH70zo+Y+sepbNq3yetudYhOE+jb99Xx3eeWMaxPN24+1/vDhctKi7h7wvG8v34XD3eW0kssoIt7J25b0hv2d+LpnYeZsAvz9MqnmfjCRN777D3uPP1OHhv7GH1KcuOEc+XF5Txy3iPcdcZdLK9ezsQXJjLrw1mEwiGvu5ZRnSLQQ2HHLc+8z57aBn4+5aR2X/i5o1x6Uj8mntSXn7/2EW+t2eZ1dzrevi2R29Ik9l2U9oF9Wzu2P5KUFdtX8JUFX2H636dzQu8TmDthLpOHT865U1iYGVcOu5J5l87jlCNP4cfv/JhrFlzD8urlXnctYzpFoE9/+UPeXL2N+y49nuFHJTioJcumTzqBY8pLuXHWe3y8Lc/PAb4neqGP7v0St+1+NOzVhUG8tHX/Vqb9bRpTXprChn0b+OEXfsiMsTOyPh0x0/qW9uXRMY/ywBcfYGvNVq5ecDV3/vVOtuzf4nXX0pb3gf7woiqe+NvHXHfWIK469fA75WZJUQGPf3U0PjO+8uu/s2lXHl96bfen4C+C4rLEbbv3h13rIzNjJKu2HdjGT5b8hIvmXcRLa1/iqyO+ykuTXuKSIZfk3Ki8NWbGBYMv4MWJL3LdyOtY+PFCLpp7Efe/cz/VNblb6svbQHfO8dNXV/HgK6uYeFJf7rpohNddatWg3iU8+bVT2V3TwJWPLWZN9T6vu9QxdnwMPQeCL4k/uyMGQfCAyi5ZtHb3Wu5bfB/j54znqQ+f4vxB5zN/4nxuO/U2ugW8nUTQUUoDpXxn1Hd4cdKLXHjMhcxeOZvxc8Zzz+J7WLMr9/ZtmVfnPBg9erRbunRphzz37poGbp+zjIUrtnDV6AH88LLMX7iiIyzbsIvrnlxCfSjMg1d8jvEjj/K6S5n1y1OhrBKmzE7cds0ieGoifPUFOOacju5Zp7Wvfh+vffoaz1c9z9KtSwn4Alwy5BKuG3kdFd0rvO5e1q3fs54nVjzBi2tepC5UxylHnsLEoRMZUzHmsPlHzczedc6NjrsunwI9FHa88P5GfvSHlezcX88dFwzn+i8Mzqn/Jq7fUcONs95j+cbdXHzi0dw+fjgDeh1e83tTUrcXfjQAzr4dvvS9xO33b4cHj4Ex98AXbun4/nUim/Zt4q1Nb/H6+tdZvHkxdaE6BnQbwGWVlzFp6CTKuiZREstzO2p38HzV88xdPZdP9nxCwBfgjL5ncHb/szmr31n0K01iP1AHaSvQcc4l/AHGA6uAKuCOOOuLgGei6/8ODEr0nKNGjXKZsmlXjfvVG2vcOQ8uchW3v+Qm/OJNt3zDrsb1M2fOdBUVFc7MXEVFhbvhhhtcWVmZAw75KSsrcyNGjIi7LtmfsrIyV1ZW1urrlZWVuZkzZzZ7DzNnzjzYxlfgep41xQ34zhxX8d0X3IQfznVvfPSZqw+GMvaZZdui//2/zv2guzt3sD/pz/GDm0rdijuGed31nDbjqRnumC8e48rGlblh3x7mzvrNWW7kb0a6kb8Z6T7/m8+7Y795rCuuLHYVFRXN/iZbbjMzZ86MuyzWtqSkpPF78/l87oYbbjikL7HfB5zf729sm+zfQ+x5m20r4EpKSlxpaWmztiUlJY1tYq9VVlbWrJ/xtsOWwuGwe/+z992P//5jd/5z5zd+duN+P85d+dsr3dCrhrqS4SVu0HGD3FNPPZXZL68VwFLXSq4mHKGbmR/4CBgLbACWAFOccx80aXMjcKJz7ptmNhmY5Jy7qq3nTXWEvnVPLR9s2sMn2/ezcste3vt0Jx9tjdScTx7Yk2/82zGMP/4ofNESy6xZs5g6dSo1NYfXNSoLCwt58sknueaaa5g1axbXXXcdDQ3NzzfhLy2j+2mTKD1xLL6iEroVFTBq0BGc0K8HQ8pLGdCrKycPOKLxvR6uZs2axQ9v/XemjDDue6OO+iSn/l59QgENIeh99jd45JFHOraTOaI+VM+e+j0cCB6gpqGGvfV72Vu/l111u9hRu4NtB7axtWYrm/dtpmp7FbXUNv5uw84G6tfVM2HUBAbbYL7/H99vtl0UFxczY8YMgEO2mcLCQsyM+vr6Zu2vvfZaHnvsMcLhQw+Ou+GGGxq/t0xuhz6fL+7rpSIQCPDEE09wzTXXJGzrnOPj3R+zePNi5i2Zx4qdKyjoefCib6F9IfqV9uOEihPoU9yH3l1706tLL3oW9aRboBvdAt0oLiyma0FXuge6E/AHUupzWiUXMzsTuNs5d3708feib+5HTdq8Em2z2MwKgC1AuWvjyVMN9Ef/sob7F64EoGdxIZ/r35Mzh5QxdkQfhpQfeiWcQYMG8cknh+cV5CsqKli3bl3CPlpBgIGnnc/Vt03n3XU7qareRyjsKCrwsfK+8Yd9SSnd78Dv9xMMdsITmcXx+49+z72L7211fXFBMUcWH8nRJUfz+vzX2b56O3Wb6qhdX0twd+QzrKiI1MbjfSdtrYvH7/cTCsX/F7rp95YL22F7xN5PQY8CuvTvQlG/IgJHBegxsAeVoyrZun8rNcHW//G68/Q7mTx8ckr9TTfQrwDGO+e+Hn38FeB059zNTdr8K9pmQ/TxmmibbS2eayowFWDgwIGjUvmCN+ysYeueWgb0Kqa8tChhmPl8vsP2ZPdmRjgcTqqPsbYAtQ0h1u+oYdu+es4ccvjXOzPxHRyu32G2rdm1hiVbltC1oCvFhcWUFpbSPdCdHkU96NWlV7PzqbT2uce2mfauS0XseXJhO2yPtj7b2HPVNNSwo3YHu+t2s7dhL/vq97G/YT8HggcY1WdUyhf9aCvQs3qRaOfcDGAGREboqTxH/yOK6X9E8jsJBw4ceNiODAYOHNh4m6iPsbYQufpRZZ9uVObGUddpfwd+/+FxZO/hYEjPIQzpOSSptq197rG/pfauiyfRCD1RXw4HTbet9vxOW58fQHFhMcWFxVk9ECuZeegbgaZXUO4fXRa3TbTk0gM4LM6sNH36dIqLD79ZIoWFhUyfPh2I9LGwsLDVtsXFxY1tc1Gi95fI1KlTM9ibziPe337sb6m96woLCwkEAoe0nzp1Kr5Wjito+r1lcjts7fVSEQgEUtq22vr8PNXa3tLYD5FR/FpgMBAA/gkc36LNTcD/Ru9PBp5N9LyZnOWSSM7Ncmmy97/l7INc1fL9JfPj9/vjzpaQ5LU2MyWVdZ1xlkuqn21HIp1ZLgBmdiHwM8APPOGcm25m90afeL6ZdQGeAk4GdgCTnXNtnk2+Iw8sEhHJV2nX0J1zC4AFLZZNa3K/FvhyOp0UEZH05O25XEREOhsFuohInlCgi4jkCQW6iEie8Oxsi2ZWDRyeRxok1hvoBNeMa0bvOf91tvcLufmeK5xzca+07lmg5zIzW9ratKF8pfec/zrb+4X8e88quYiI5AkFuohInlCgp2aG1x3wgN5z/uts7xfy7D2rhi4ikic0QhcRyRMKdHmYDs0AAAIxSURBVBGRPKFAT5GZ3W1mG83s/ejPhV73qSOY2XgzW2VmVWZ2h9f9yQYzW2dmy6Pfa16eEtTMnjCzz6JXG4st62VmfzSz1dHbI7zsY6a18p7zajtWoKfnf5xzJ0V/FiRunluiFwh/GLgAGAFMMbMR3vYqa74U/V7zZo5yC78BxrdYdgfwmnOuEngt+jif/IZD3zPk0XasQJe2nAZUOefWOufqgaeBSz3uk2SAc+4NItcuaOpS4LfR+78FJma1Ux2slfecVxTo6bnZzJZF/yuXV/89jeoHrG/yeEN0Wb5zwKtm9m70wuadRR/n3Obo/S1Ajly1Nm15sx0r0NtgZn8ys3/F+bkUeBQYApwEbAZ+6mlnJZO+4Jw7hUip6SYz+6LXHcq26KXOOsOc5rzajpO6YlFn5Zwbk0w7M/sV8FIHd8cLyVwgPO845zZGbz8zs3lESk9veNurrNhqZkc75zab2dHAZ153qKM557bG7ufDdqwReoqif/Axk4B/tdY2hy0BKs1ssJkFiFwAfL7HfepQZlZiZt1i94Fx5Od3G8984Nro/WuBFzzsS1bk23asEXrqHjCzk4j8t3Qd8B/edifznHNBM7sZeIWDFwhf4XG3OlofYJ6ZQWT7mO2cW+htlzLPzH4HnAP0NrMNwA+AHwPPmtn1RE5tfaV3Pcy8Vt7zOfm0HevQfxGRPKGSi4hInlCgi4jkCQW6iEieUKCLiOQJBbqISJ5QoIuI5AkFuohInvj/WjRdy60lozQAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "np.random.seed(1)\n",
    "mus, sigmas, pis = EM(data, 3, 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The algorithm was able to cluster the 3 different groups present in the data. Let's try to reduce the number of gaussians to 2 and see what happens."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration: 0. Means: [-1  5]. Stds: [1.72032449 1.00011437]\n",
      "Iteration: 1. Means: [ 0.79286622 10.9172818 ]. Stds: [6.54990243 3.34926968]\n",
      "Iteration: 2. Means: [ 0.19713969 12.61977142]. Stds: [5.6577156  2.23045388]\n",
      "Iteration: 3. Means: [-0.67417782 12.94541772]. Stds: [4.63841982 1.44319091]\n",
      "Iteration: 4. Means: [-1.13963289 12.95000352]. Stds: [3.95405035 1.41026246]\n",
      "Iteration: 5. Means: [-1.32681143 12.87770861]. Stds: [3.68549296 1.46954404]\n",
      "Iteration: 6. Means: [-1.4007573  12.83642442]. Stds: [3.58287254 1.50584829]\n",
      "Iteration: 7. Means: [-1.42232826 12.82395568]. Stds: [3.55264973 1.51676964]\n",
      "Iteration: 8. Means: [-1.42748316 12.82097314]. Stds: [3.54538828 1.5193671 ]\n",
      "Iteration: 9. Means: [-1.42864082 12.82030353]. Stds: [3.5437552  1.51994927]\n",
      "Iteration: 10. Means: [-1.42889694 12.8201554 ]. Stds: [3.54339379 1.520078  ]\n",
      "Iteration: 11. Means: [-1.42895341 12.82012274]. Stds: [3.54331409 1.52010638]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXhb1bnv8e8reYrtxPHs2E7szCEDmZyEMs+FUpIAhQZoCy0FCodzD7c9t6Wnp7Slpaft07m0PVCgUKDMEMJc5pkkzjwnjuPEU+IxdhzHo9b9Q1JiHDuWbUl7b+n9PI8fydKW9JMsvV5aa+21xRiDUkqpyOWyOoBSSqnQ0kKvlFIRTgu9UkpFOC30SikV4bTQK6VUhIuxOkBvGRkZprCw0OoYSinlKGvWrKkzxmT2dZ3tCn1hYSHFxcVWx1BKKUcRkb39XaddN0opFeECKvQicpGI7BCREhG5o4/r40XkSd/1K0Wk0Hd5oYgcEZH1vp//DW58pZRSAxmw60ZE3MCfgQuACmC1iKwwxmztsdkNQKMxZpKILAN+CXzZd91uY8ycIOdWSikVoEBa9AuBEmNMqTGmA3gCWNJrmyXAw77zzwDniYgEL6ZSSqmhCqTQ5wHlPX6v8F3W5zbGmC6gCUj3XTdeRNaJyHsickZfDyAiN4lIsYgU19bWDuoJKKWUOrFQD8ZWA+OMMXOBbwP/FJFRvTcyxtxnjCkyxhRlZvY5O0gppdQQBVLoK4GxPX7P913W5zYiEgOkAPXGmHZjTD2AMWYNsBuYMtzQSimlAhdIoV8NTBaR8SISBywDVvTaZgVwne/8l4C3jTFGRDJ9g7mIyARgMlAanOhKKdVD7Q5453+g+EHwdFudxlYGnHVjjOkSkduA1wE38KAxZouI3AUUG2NWAA8Aj4hICdCA958BwJnAXSLSCXiAbxljGkLxRJRSUWzP+/D41dDR4v195+vw5cfAbbt9Qi0hdjvwSFFRkdE9Y5VSAes4DPcshNgRcN0K2LoCXvseXPQLOOUWq9OFjYisMcYU9XWd7hmrlHK2j/4IzRWw+I8wKhcW3QwTz4N3fg6t2oEAWuiVUk7W1QHFD8CUi6HgVO9lInDhT6G9GdY/Zm0+m9BCr5Ryrh0vw+FaWHDDZy/PngFjF8Gah8Bm3dNW0EKvlHKudY9ByliYeO7x182/HupLoHxl2GPZjRZ6pZQztR+CPe/B9CXgch9//bRLwBUL218Ofzab0UKvlHKmkreguwOmfqHv6xNSoPB02PFqeHPZkBZ6pZQz7XgVRqR5++L7M/ULUL8L6naFL5cNaaFXSjmPMVD6jrdv/kQ7RU2+wHta+m5YYtmVFnqllPPUl0DLARjf54K4x6QWegdryz4ISyy70kKvlHIef+EuHKDQi3j76cs+iuppllrolVLOU/YhJOdA2oSBty04DVrrvIueRSkt9Eop59n3qXdP2EAOZOffY3bfx6HNZGNa6JVSztJcDc2VkL8gsO3TJkBiOlSuCW0uG9NCr5Rylkrf6rb5fS7UeDwRyJsPFVrolVLKGSrXgCsGcmYFfpu8+VC7HdqaQ5fLxrTQK6WcpaIYsmd6158PVF4RYKBqXchi2ZkWeqWUcxgD1Rsgb97gbuffXgu9UkrZXGOZd535nJMHd7vENBiVD/s3hSSW3WmhV0o5h79QD7bQg7dPXwu9UkrZ3P5NIC7Inj7424452bvAWUdr8HPZnBZ6pZRz7N8EGVMGNxDrlzMLjAdqtgU/l81poVdKOcf+TYObVtmT/3b7NwQvj0NooVdKOcORg9Bc4T0e7FCMLoC4kVHZoj/BQs5KhVZHl4eqg0cAcImQlzoCtyuAtUtUdKrd7j3NGkL/PHj3kM2apoVeqVAzxvDWthpe3FjF29trONTWdfS6jOQ4zj8pm6Vz8zhlQrqFKZUt1Wz1nmadNPT7yJwWlYcW1EKvwmZv/WH+e/lmPthVR2piLBfNyGHB+DRi3UJ7p4cPS+p4aWM1T6wu59LZufzwiyeRNTLB6tjKLmq2Q1yy90AiQ5U1HdY9Ai21kJwZvGw2p4VehcVTq8v54QubiXW7uGvJDK5ZOI4Y92eHiJYtHEdbZzf3vV/KPW+X8O6OGv6wbA7nTsu2KLWylZqt3hZ5IEsT9ydrmve0dltUFXodjFUh97/v7ea7z25k4fg03vrOWXztc4XHFXm/hFg3/+e8ybx2+xkUpidx4z/WsHxdZZgTK1uq2Ta8bhs41r8fZf30WuhVSP3qte384tXtXDo7lweuW0D2qMC6YiZkJvP4TaewsDCN259czyOflIU0p7K5w/Xeo0RlThve/SRnQ8LoYwO7UUILvQqZRz7dy1/e3c3VC8fx+y/PIS5mcG+35PgY/v71BZx/UhZ3rtjCm1sPhCipsr26nd7TzKnDux8R7w5XdbuGn8lBtNCrkPhkdz0/WbGFc6dl8bOlM4c8bTIh1s2frp7HzNwU/uOJdew8cCjISZUj+At9xuTh31fGlGP3FyUCKvQicpGI7BCREhG5o4/r40XkSd/1K0WksNf140SkRUT+MzixlZ2VN7Ry62NrKMxI4g/L5gx7bvyIODf3fW0+ifExfPPhYg62dgQpqXKMup0QkzC8GTd+GZOh5YB3B6woMWChFxE38GfgYmA6cLWI9N5j4Qag0RgzCfgd8Mte1/8WiL7Jq1HI4zF856kNdHkM93+tiJEJsUG53zEpI7j3q/OpOniEH63YEpT7VA5StwvSJ4PLPfz78nf/RFH3TSAt+oVAiTGm1BjTATwBLOm1zRLgYd/5Z4DzRLxzoERkKbAH0E9nFHjo4zJWlTXwo0tnUJiRFNT7njculX8/dzIvrK/itc3VQb1vZXN1O4LTbQPerhv/fUaJQAp9HlDe4/cK32V9bmOM6QKagHQRSQa+B/xk+FGV3ZXWtvCr17dz3rQsrpjX+y0SHLeeM5EZuaP4wfObqW9pD8ljKJvpbIPGvccK9HCNLgB3HNRqoQ+WHwO/M8a0nGgjEblJRIpFpLi2tjbEkVQoGGP43rMbiY9x8/PLZyHD2anlBGLdLn5z1Wya2zr5yYtbQ/IYymYa9wAmeC16dwykjoeG0uDcnwMEUugrgZ4jIPm+y/rcRkRigBSgHlgE/EpEyoDbgf8Skdt6P4Ax5j5jTJExpigzM3r2VoskKzZUsbqskR984aSA58oP1bScUdxy9iRWbKhi1Z6GkD6WsoH6Eu9p+sTg3Wf6pGP3GwUCKfSrgckiMl5E4oBlwIpe26wArvOd/xLwtvE6wxhTaIwpBH4P/NwYc0+QsiubaO3o4hevbmdWXgpfmp8flse85ayJjElJ4K6XtuDxmLA8prKIvyCnBbPQT/S26D3dwbtPGxuw0Pv63G8DXge2AU8ZY7aIyF0isti32QN4++RLgG8Dx03BVJHr3vdKqW5q485Lp+MK0zLDI+Lc3HHxNDZXNvPMmoqwPKaySH2Jb4/WUcG7z/RJ0N0BTeUDbxsBAlrUzBjzCvBKr8vu7HG+DbhygPv48RDyKZurOniEe9/fzRdPHsOCwrSwPvbi2bk8/HEZv3p9BxfPygnaVE5lM/W7vYU5mPz3V78bUguDe982pHvGqmH509u78HjgjouHuQbJEIgId146g7qWdh76qCzsj6/CpL4kuP3z8NlCHwW00Ksh21ffytPFFVyzaBz5qYmWZJgzdjTnn5TN3z4opelIpyUZVAi1NcHh2uC36JOzvIcVrI+Onaa00Ksh+9Pbu3C7hFvODnJra5BuP38yzW1dPPjhHktzqBDwt7jTJgT3fkUgfULUTLHUQq+GpKzuMM+tq+TaRQUhn045kJl5KXx+RjYPfriHplZt1UcUfyEO5owbvzQt9Eqd0B/f3kWsW/jW2UFuaQ3R7edP4VB7F/d/GB0f3KjR4PuWFooB07QJcHAfdEd+40ALvRq0isZWXlhfxbWLCmxzTNeTxoziohk5PPxxGS3tXQPfQDlDQymMzIW4EIwBpU0AT5e32Ec4LfRq0B78sAwBbjh9vNVRPuPmsybQ3NbFE6si/4MbNRpKg98/7+e/34bIH9vRQq8Gpam1kydW72Px7FxyR4+wOs5nzB2XysLxaTz44R46uz1Wx1HB0FDqHTQNBX+/fxT002uhV4Py6Mq9tHZ0c+OZ9uib7+3mMydQ1dTGSxurrI6ihqv9EByuCV2LPjkLYpO00CvVU1tnN3//qIwzp2Ry0pgg7o4eROdMzWJSVjL3vleKMboGjqMdHYgNURehCKSNh4bI32lKC70K2IoNVdS1tHOzTVvzAC6XcNMZE9i+/xAf7663Oo4ajkZfoU8L4VhQaiE0loXu/m1CC70KiDGGhz8uY2r2SE6dmG51nBNaPCeXtKQ4Hvq4zOooajhC3aIH7z+Rxr3giewxHS30KiBr9zWypaqZr51aELKDigRLQqybZQvG8ta2A5Q3tFodRw1V4x5ITA/uqpW9pY6H7nY4FNljOlroVUAe+ngvIxNiWDonNIcIDLavnFIAwGMrdaqlYzXsCW1rHo51C0X4FEst9GpAB5rbeHVTNVcVjSUpPqCVrS2XO3oEF07P4YnV+2jrjI6DS0Scxj2h7Z+HY/9IGrXQqyj3z5X76PIYvuprJTvFdacWcrC1kxUbIvtreUTq6oCmitC36FPGgitGW/QqunV1e3h81T7OmpJJYUaS1XEG5ZQJaUzJTubRT/daHUUNVlM5GE/oW/TuGG+xj/CZN1ro1Qm9vb2GmkPtXLtonNVRBk1EuGbhODZWNLG5ssnqOGowGkO4mFlvaeO160ZFt8dX7SNrZDznTsuyOsqQXDY3n/gYF0+s1kFZR/G3sEPddQNRMZdeC73qV+XBI7y7s5YvLxhLjNuZb5WUxFguOXkMy9dV0dqhq1o6RmMZxCR4DwoeaqmFcKQRjhwM/WNZxJmfXhUWT64uB+CqorEWJxmeaxaOo6W9i5c2VFsdRQWqsQxGF4ArDCXK3z10MHLHcrTQqz51dXt4anU5Z07OZGyaNceDDZb5BalMzkrmn7p8sXM0loWnfx56TLEsC8/jWUALverT+7tq2d/cxtULnd2aB++g7NULx7G+/CDbqputjqMGYgw0lIWx0PumDWuhV9HmqdUVpCfFcd5JYegjDYOlc/OIdQtPF1dYHUUNpLUBOg6Fr9AnpMCINC30Kro0HO7gre0HuGxuHrEOHYTtLS0pjvNPymb5+ko6uiJ7ASvHOzrjpjB8jxnhM28i41Osgmr5uko6uw1XOnwQtrcri/JpONzB29trrI6iTuToHPow7omdWhjRe8dqoVfHeXpNBSfnpzA1Z6TVUYLqzMmZZI2M55k15VZHUSdiVYu+qRw8kbkukhZ69RmbK5vYVt3MlfPzrY4SdDFuF5fPy+edHbXUHGqzOo7qz8G9kJQJcWFcciO1ADxd0FwZvscMIy306jOeWVNBXIyLxbOdsRzxYF1ZlE+3x/D82sj8QEeEcE6t9PM/XmNkzqXXQq+O6ujy8ML6Si6cnk1KYqzVcUJiYmYyc8eN5tm1FXpMWbuytNCXhfdxw0QLvTrq3R01NLZ2csW8yOu26enyefnsPNDCliqdU2873Z3QVOndKzacRuWDuLXQq8j3/LpKMpLjOGNyhtVRQurSk8cQ6xaeX6fdN7bTVAGmO/wtencMpORHd6EXkYtEZIeIlIjIHX1cHy8iT/quXykihb7LF4rIet/PBhG5LLjxVbAcbO3grW01LJmT59gFzAI1OjGO86Zl88L6Srq6dU69rfjXmwl3ofc/ZoSudzPgJ1pE3MCfgYuB6cDVIjK912Y3AI3GmEnA74Bf+i7fDBQZY+YAFwH3iogzjkUXZV7aWE1Ht4fL5kbmIGxvl8/Lo66lgw921VkdRfV0dGqlBUczSy2I2Ln0gTTdFgIlxphSY0wH8ASwpNc2S4CHfeefAc4TETHGtBpj/GvDJgA6+mVTz62tYGr2SGbkjrI6SlicPTWL1MRYnl2rSyLYSmOZ99B+oyxocKQWQmsdtLeE/7FDLJBCnwf03MOkwndZn9v4CnsTkA4gIotEZAuwCfhWj8J/lIjcJCLFIlJcW1s7+GehhmVP3WHW7jvIZfPyEBGr44SFdwppLv/aeoDmtk6r4yi/xr2+47i6w//Y/gHgg5G3ymnIO2ONMSuNMTOABcD3RSShj23uM8YUGWOKMjMzQx1J9bJ8XSUisHROdHTb+C2dm0dHl4fXNu23Oorys2JqpV8EL1ccSKGvBHouepLvu6zPbXx98ClAfc8NjDHbgBZg5lDDquAzxrB8fSWfm5BOTspx/4Mj2pyxoylMT2T5ep19YxsH91rTPw/HHjcCB2QDKfSrgckiMl5E4oBlwIpe26wArvOd/xLwtjHG+G4TAyAiBcA0oCwoyVVQrC8/yN76VpZGySBsTyLC0rl5fFJaT3XTEavjqPZD0FpvXYs+MR3ikqOzRe/rU78NeB3YBjxljNkiIneJyGLfZg8A6SJSAnwb8E/BPB3YICLrgeeBW40xOs3BRpavqyQ+xsVFM3OsjmKJpXPyMAZWrK+yOoryLz8Q7p2l/ES8jx2ByyAENNXRGPMK8Eqvy+7scb4NuLKP2z0CPDLMjCpEOrs9vLixmvNPymZUQmQueTCQwowk5owdzfL1Vdx81kSr40Q3K1at7C21EBpKrXv8EInsPWPUCX2wq5aGwx1R2W3T09I5uWyrbmbH/kNWR4luVu4s5Zda4M0RYesgaaGPYsvXVTE6MZazpkT3TKcvzs7F7dIlESzXWAbxo2BEqnUZUguhsxUOR9Y0by30UepwexdvbD3AJbPGEBcT3W+DjOR4zpicwYsbqvB4Iqsl5yiNZd4WtZX7ckToKpbR/QmPYm9sPcCRzm6WRNnc+f4smZNL5cEjFO9ttDpK9LJyDr2fFnoVSZavryRv9AiKCiz8mmwjF07PISHWxQs6p94aHo93j1SrC/3ocd7TCJt5o4U+CtW3tPPBrjounZ2LyxUdSx4MJCk+hgum5/Dypmo6unRFy7BrOQBdbdYX+tgRkJyjLXrlfC9vqqbbY1g6N9fqKLaydE4uB1s7+WBXZA3EOYIdplb6pRZqoVfO98L6KqZmj2RaTnSsVBmoMyZnMjoxluW681T4+adWji60NAZwbIplBNFCH2XKG1pZs7eRxXO0Nd9bXIyLS2aN4Y2t+zncftwiqyqUGssAgdFjB9oy9FILvUe66mq3OknQaKGPMis2eFurS7TQ92nJnDzaOj28sfWA1VGiS8Me7xr0MfFWJ/F1HxlvsY8QWuijiDGGF9ZXsqAwlfzURKvj2FJRQSp5o0foipbh1lgGaeOtTuHlX644go42pYU+imzff4idB1pYrHPn++VyCZfOzuWDXXXUt0TOV3fb8+8sZQdH59JroVcO9ML6KmJcwiWzxlgdxdaWzMml22N4ZVO11VGiQ0crtOy3x4wbgORsiEmIqJk3WuijhMdjeHFDFWdMziAtKc7qOLZ20phRTM0eyQs6+yY8ji5mZpOuG5cr4qZYaqGPEsV7G6k8eESXPAjQ4jm5FO9tpLyh1eookc9Oc+j9Ugsjau9YLfRR4oX1lSTEurhgerbVURxh8WzvrCT/LCUVQv5BT7u06MFX6PdEzHLFWuijQEeXh5c3VXPh9ByS4gM61kzUG5uWSFFBKi+sr8REyIfdthr3QNxISEyzOskxqeOhowUOR8YB8bTQR4H3d9ZysLVTlzwYpCVz89h5oIVt1XpAkpBq2ANphdYuT9ybf6pnhMy80UIfBZavryQtKY4zJkf3AUYG65JZY4hxia5oGWqNe+zVbQMRN5deC32Ea2nv4s1t3gOMxLr1zz0YaUlxnDklkxV6QJLQ8XR7Bz3tNBALvuWKRVv0yhn+tWU/bZ0eXfJgiJbMyaW6qY1VZQ1WR4lMzZXg6bTPXrF+sQkwKldb9MoZlq+vIj91BPP1ACNDcsH0bBLj3Np9Eyp2nHHjlzpeW/TK/moPtfNRSR1L5uQidhrocpDEuBgunJ7Nyxurae/qtjpO5PEXUru16ME7QKwtemV3L26o8h5gRHeSGpalc/Nobuvine16QJKga9gDrhgYlW91kuOljofDNdDeYnWSYdNCH8GWr69kZt4oJmePtDqKo50+KYOM5HiWr9Pum6Br3OMd+HTbcP+OtAne0whYCkELfYQqqWlhY0WTtuaDIMbtYvHsXN7eXkNTa6fVcSJLQymkTbQ6Rd/8hb6h1NocQaCFPkK9sL4SlxzblV8Nz2Vz8+jo9vDKZl3RMmiM8e0sNcHqJH3zjxtooVd2ZIzh+XWVnDYpg6xRCVbHiQgz80YxMTOJ57X7JngO13qXGUi3aYs+IQUSM7TQK3sq3ttIReMRLpur3TbBIiJcNjePVXsadEXLYPEXULu26MGbTQu9sqPn11UyItbNhTNyrI4SUfxLPOuc+iBxTKF3/hRLLfQRpq2zm5c2VHHRzBySdaXKoBqblsjC8Wk8t05XtAyKhlIQN6SMtTpJ/9ImQHMFdB6xOsmwBFToReQiEdkhIiUickcf18eLyJO+61eKSKHv8gtEZI2IbPKdnhvc+Kq3t7bV0NzWxeXztNsmFK6Yl0dp7WE2VDRZHcX56nfD6LEQY+MjnkXIFMsBC72IuIE/AxcD04GrRWR6r81uABqNMZOA3wG/9F1eB1xqjJkFXAc8Eqzgqm/Pra0ge1Q8p07MsDpKRLp41hjiY1w8t7bC6ijO17DbvlMr/dJ9hb5+t7U5himQFv1CoMQYU2qM6QCeAJb02mYJ8LDv/DPAeSIixph1xhj/IXq2ACNEJD4YwdXx6lraeXdnLUvn5uF26ZIHoTAqIZYLZ+SwYkOVLokwHMZAfSmkT7I6yYn5/xE1RH6hzwPKe/xe4busz22MMV1AE5Dea5srgLXGmPbeDyAiN4lIsYgU19bqbuZDtWK9d8mDK+bZcHfyCHLFvDwOtnbqkgjD0VIDHYfsO7XSb8RoSMqE+hKrkwxLWAZjRWQG3u6cm/u63hhznzGmyBhTlJmpB8cYqmfXVjArL4UpuuRBSJ0+KYPMkfE8q903Q+dvIdu96wa8GeudPcUykEJfCfQcFs/3XdbnNiISA6QA9b7f84Hnga8ZY5z9/cfGtlU3s6WqWQdhwyDG7WLpnFze2V5DfctxX1BVIPwtZLu36MHbvRQFLfrVwGQRGS8iccAyYEWvbVbgHWwF+BLwtjHGiMho4GXgDmPMR8EKrY73dHEFsW45OtdbhdaVRWPp8hiWr68aeGN1vPrd4Iq199RKv/QJ0LLf0atYDljofX3utwGvA9uAp4wxW0TkLhFZ7NvsASBdREqAbwP+KZi3AZOAO0Vkve8nK+jPIsp1dHlYvr6S80/KJi3JxlPVIsiU7JHMzk/h6eJynVM/FA27vYcPtOOqlb35B4wdPCAb0KtsjHkFeKXXZXf2ON8GXNnH7X4G/GyYGdUA3t5eQ8PhDq4s0kHYcPpS0Vh+uHwzW6qamZmXYnUcZ6krgYzJVqcIjL/Q15fAmNnWZhki3TM2AjyzppyskfGcOVkHssNp8cm5xMW4eLq4fOCN1TGebm/r2O5TK/3SJgACdbusTjJkWugdruZQG+/sqOWyeXnEuPXPGU4pibF8fkYOy9dX0dapc+oDdnAvdHc4p0UfO8K7B68WemWV5esq6fYYrpyv3TZWuHJ+Pk1HOnlj6wGrozhHnW8GS8YUa3MMRvpkqNdCryxgjOGJ1eXML0hlUpbOnbfCaZMyyBs9gidXa/dNwPwFM90hLXrwfvuo3+3do9eBtNA72Ko9DZTWHmbZAgdMUYtQbpdwVdFYPiyp03XqA1W3C0akQlLvnedtLH2S9yAph5x5hDEt9A72xOpyRsbHcMnJY6yOEtWuLMrHJWirPlD1Jc5qzcOx8YS6ndbmGCIt9A7V1NrJK5uqWTwnl8Q4B8xFjmC5o0dw1pRMnl5TTle3x+o49le7AzId1D8PkDHVe+rQAVkt9A61fH0l7V0erl44zuooCli2cBwHmtt5d4cudHZCrQ1wuOZY4XSKkTkQP8r7T8qBtNA7kDGGx1ftY2beKN1RxybOnZZF5sh4Hl+1z+oo9ubv+sh0WKEX8Wau3W51kiHRQu9Aa/c1sn3/IW3N20is28VVRfm8s6OGyoPOPuxcSPlbxE6aWumXMVX76FX4PPrpPpLjY1iqC5jZytULx2GAx1dqq75fdTshJgFGO7CRkjkFWg7AkYNWJxk0LfQO03C4g5c3VnP5vDyS9ODftpKfmsi5U7N4YnU5HV06KNun2h3eGTcut9VJBu/ogKzzWvVa6B3m6eJyOro9fOWUAqujqD585ZQC6lra+dfW/VZHsafaHc7rn/fz567ZZm2OIdBC7yAej+GxlftYWJimR5GyqTOnZDI2bQSPfLLX6ij2034ImvZB1klWJxma0QUQm+jIAVkt9A7y/q5a9jW08pXPaWvertwu4ZqFBazc08CuA4esjmMv/oHYrOnW5hgql8vbqtcWvQqlhz4uIyM5notm5FgdRZ3AVUX5xMW4eOjjMquj2EvNVu9p1jRrcwxH1nQt9Cp0SmtbeHdHLV85ZRxxMfpns7P05HiWzM7lubWVNLV2Wh3HPmq2QcwIGF1odZKhy5zmPaxga4PVSQZFK4ZDPPxxGXFuF9cu0m4bJ/j6aeM50tnNE6t1quVRNdu8rXmXg8uOv9vJYf30Dn7Fo0dzWyfPrKngi7PHkDky3uo4KgDTc0exaHwa//hkr65/41ezFTIdOhDr5+92OrDF2hyDpIXeAZ5aXc7hjm6+cdp4q6OoQfj6aeOpPHhED0oC0FLr3dkoe4bVSYZnVB4kpGihV8HV1e3h4U/KWFCYquvaOMwF07PJTx3Bgx/tsTqK9Q5s9p7mzLQ2x3CJQPasY8/HIbTQ29xrW/ZT3nCEG06fYHUUNUhul/D108azuqyRtfsarY5jLX8LONvhhR68/6wObAWPc7rktNDbmDGGe98rZUJGEhdMz7Y6jhqCZQvGkjIilvveK7U6irUObIbkHEjKsDrJ8GXPgM7D0Oicb2pa6G3sk9J6NlU28c0zJuB2idVx1BAkxcfwlVPG8frW/eypO2x1HOsc2Oz8/lCle6MAABVJSURBVHk//7cSB/XTa6G3sfveLyUjOY7L5+kqlU523amFxLpc/O2DKG3Vd3V494qNlEKfdRKIG/ZvtDpJwLTQ29T2/c28u6OW608tJCHWgSv9qaOyRiZwxfw8nllTQe2hdqvjhF/tdujugNw5VicJjtgR3qUQqrXQq2H68zu7SYpz6yqVEeLGMybQ2e3hgQ+d068bNNUbvKdjIqTQA4yZfex5OYAWehsqqWnhpY1VfPVzhYxOjLM6jgqCCZnJXHpyLo98Ukbj4Q6r44RX9QaIGwmpEbQfyJjZ3qUQDjljOWot9Db0l3dKiI9x8c0zIuiDobjt3Ekc7uiOvnn11RtgzMnOXvqgtzGzvacO6b6JoFc+MuytP8wLG6q4dlEBGcm63EEkmZI9kotn5vDQR2U0HYmSxc483d4ZNzknW50kuHJmAQLV661OEhAt9Dbzl3d243YJN5+pO0hFotvOncSh9i4e+qjM6ijhUbsdOlshb57VSYIrfiRkTIaqdVYnCUhAhV5ELhKRHSJSIiJ39HF9vIg86bt+pYgU+i5PF5F3RKRFRO4JbvTIs6fuMM+sreCahePIGpVgdRwVAjNyU7hgejb3f1jKwdYo6KuvXOM9zZtvbY5QyJ0HFcVgjNVJBjRgoRcRN/Bn4GJgOnC1iPQ+RMwNQKMxZhLwO+CXvsvbgB8C/xm0xBHsd2/sJM7t4tZzJlodRYXQdy6cQkt7F/e+HwXz6ivXeBcBS4vAb6h58+FwDTRXWp1kQIG06BcCJcaYUmNMB/AEsKTXNkuAh33nnwHOExExxhw2xnyIt+CrE9ha1cyKDVV8/bRCskZqaz6STcsZxZLZufz9oz3UNEf4R6NyjbcgSgTu2e3/llK51tocAQik0OcB5T1+r/Bd1uc2xpguoAlID0bAaPHbN3YwKiGGm8/U1nw0uP38KXR1G+55p8TqKKHT0epd/Cs3wvrn/XJmgisWKoutTjIgWwzGishNIlIsIsW1tbVWxwm74rIG3txWw81nTSQlMdbqOCoMCjOSuGrBWB5ftY+99RG6Bk7VWjDdMHah1UlCIybeO82yfLXVSQYUSKGvBMb2+D3fd1mf24hIDJAC1AcawhhznzGmyBhTlJmZGejNIoLHY/jpy9vIHhXP108rtDqOCqP/OG8ysW4Xv3jVWYelC9i+T72n+QuszRFKYxd5/6F12XtgPZBCvxqYLCLjRSQOWAas6LXNCuA63/kvAW8b44ChaBtYsaGKDeUH+e7np5EYF2N1HBVG2aMS+NZZE3l1835WlgbcLnKO8lWQMRUS06xOEjrjFkFXm+2XQxiw0Pv63G8DXge2AU8ZY7aIyF0isti32QNAuoiUAN8Gjk7BFJEy4LfA9SJS0ceMnah1pKObX762nVl5KVw2V1eojEY3njGBMSkJ/PTlrXg8EdQ28nigfKW3EEaysb7nV77S2hwDCKiP3hjzijFmijFmojHmbt9ldxpjVvjOtxljrjTGTDLGLDTGlPa4baExJs0Yk2yMyTfGbA3NU3Ge+94vpbqpjR9+cTouXW8+Ko2Ic/O9i6axubKZZ9dWWB0neOp2QNvBY4UwUo3MgdRC2PeJ1UlOyBaDsdFoX30rf3m3hC/MymHh+Aj+aqsGtHh2LnPHjeaXr22nqTVClkYo+9B7Wni6tTnCoeB02PuRrQ8tqIXeAsYY7lyxmRiX8MMvak9WtHO5hJ8umUnD4Q5+9XqEDMyWfQgpY2F0FCyzXXg6HGmEGvt2Vmiht8Brm/fz7o5a/u8FUxiTMsLqOMoGZualcP2p4/nnqn2sc/qBxI3xFvrC0yNzR6neCk/znvq/xdiQFvowa2nv4icvbuWkMaO4/tRCq+MoG/n2hVPIGhnPD57fTFe3fbsBBlSzDVrroOA0q5OEx+hx3m8uZR9YnaRfWujD7O6Xt1FzqI2fXzaTGLe+/OqY5PgYfnzpDLZWNzt7HZzSd7ynE862MkV4TTgb9rwP3V1WJ+mTVpowen9nLY+v2seNZ0xg7rhUq+MoG7p41hguOXkMv39zJ9v3N1sdZ2hK3oKMKTB67MDbRoqJ50J7s22XQ9BCHybNbZ1879mNTMxM4v9eMMXqOMrGfrpkJikjYvnOUxvodFoXTmebdwbKxHOtThJeE84CccHut61O0ict9GHykxVbOdDcxm+umkNCrNvqOMrG0pLi+NnSWWypauZPb+2yOs7g7P3Iu6foxPOsThJeI1K9q1mWvGl1kj5poQ+D59dV8OzaCv7tnEnMGTva6jjKAS6amcMV8/L50zslfLy7zuo4gdv5GsSMgPFnWJ0k/KZ83rss86EDVic5jhb6ECutbeEHz29mYWEa/3HeZKvjKAe5a8kMxmckcfsT66lrabc6zsCMgR2vwsRzIDYKpw1Pudh7uvM1a3P0QQt9CLV1dnPbP9cRH+PiD1fP0Vk2alCS4mO45+p5HDzSyXee2mD/tXAObIGmcphykdVJrJE9A1LGwY5XrE5yHK08IWKM4b+e28TW6mZ+feVs3TFKDcn03FHc+cXpvLezlt++sdPqOCe29QXvgOTUi61OYg0RmHaJd0C2zV4zprTQh8j9H+zhuXWVfPuCKZx3UrbVcZSDXbtoHF8uGss975Tw0sYqq+P0zRjY8rx3b9jkLKvTWGfGZdDd4e3CshEt9CHw7o4a/ufVbXxhVg7/fu4kq+MohxMR7lo6g6KCVP7z6Q1srmyyOtLxDmyG+l3eQhfN8hfAqHzY8pzVST5DC32QbSg/yK2PrWVazih+feVsJBrW+lAhFx/j5q9fmU96UjzX/301++pbrY70WRufBFcMnLTE6iTWcrlg5mXeaZaH7TNbSgt9EJXWtvD1h1aTlhTHQ19foEeMUkGVOTKeh7+xgC6Ph68+uJLaQzaZidPdCRue9A7CJqVbncZ6s68BTxdsfMrqJEdpoQ+SisZWvvrAKgR45IZFZI1KsDqSikCTskby4PULONDcxvV/X8XBVhscq7TkTThcA3OutTqJPWRPh9y5sP4x79iFDWihD4LyhlaW3fcpzW2d/P3rCxifkWR1JBXB5o1L5X+/Mp9dB1q49v6VNB62uNivvh+Sc2DyBdbmsJN5X/OOW9jkEINa6IdpX72vyB/p5LFvLuLkfN3zVYXe2VOzuPdr89lV08I196+k3qodqup3e1v0Rd8Ad6w1Gexo1lUQnwKr7rM6CaCFflg2Vhzk8r9+xOGOLv554yla5FVYnTM1i/u/VkRpbQtX/PVjyuoOhz/Ep38FVyzMvz78j21n8ckw76vefQsOlludRgv9UL217QBfvvdTEmLdPPOtzzEzL8XqSCoKnTklk3/euIimI51c/tePWRvOo1MdOgBr/wFzroaRuq/IcU65xXv68R+tzYEW+kHzeAz3vL2Lb/6jmElZyTx366lMyhppdSwVxeYXpPHcracxMiGGZfd9ypOr94XngT/+I3g64bTbw/N4TpOSD7OXef8ZNlu7o5sW+kFoau3kxn8U8+t/7eTSk3N58uZTyBqps2uU9cZnJPH8raexaHwa33t2E999ZgNtnd2he8DGvd7+59nXQPrE0D2O0535/8B44J2fWxpDC32A3ttZy4W/f4/3d9Xyk8Uz+MOyOTpPXtmKd/+Nhfz7uZN4qriCS/74ARvKD4bmwd78sXddm3P+KzT3HylSC2HBN71TLas3WhZDC/0Amlo7+f5zm7juwVWMSojluVtO47pTC3WPV2VLbpfwnQun8ugNi2jt6Obyv37Mr17bzpGOILbuS9707uJ/xncgJS949xupzvouJKbDS7eDJ4Tfsk5AC30/PB7Dk6v3cc5v3uXJ1fu48YzxvPjvpzMrXwddlf2dPjmD124/k8vm5vGXd3dz/m/f47XN+zHD3YGntQFW/B/vMWFP+4/ghI10I1Lh8//jPSjJR3+wJIIM+w8fZEVFRaa42LoD7BpjeGtbDb/+1w627z9EUUEqP1kygxm5WuCVM31aWs+PXtjCjgOHWFiYxn9+fioLx6cN/o48Hnjqq7DzdfjmG969P1VgjIGnr4ftL8H1r8C4RUF/CBFZY4wp6vM6LfRe3R7D61v2c+/7pWwoP0hBeiLfvmAKl56ci8ul3TTK2Tq7PTyxah9/fLuE2kPtnD4pg5vOnMAZkzMC74Z86y744Ddw0S+OTR1UgTvSCH87F9oPwTffgtSCoN69FvoTqDnUxnNrK3ls5V7KG44wLi2RW86eyJfm5xOrR4RSEeZIRzePfFrG/R/soeZQO9NyRnLtonEsnpNHyogT7Nn64e/hzR/BvOvg0j94D7KhBq92BzxwIYwYDde/7J2CGSRa6HtpbuvkrW0HeHljNe/sqKXbY1hYmMY3Ti/kguk5uLUFryJce1c3L26o5sEP97C1upn4GBcXzsjhklk5nD01i4RYt3fD7i5vgf/kHph5BVz+N3C5rQ3vdBVr4JGlEJcM1zwJY04Oyt1GfaH3eAy7alr4YFct7+6oZdWeBjq6PYxJSWDx7FyuLBrLpKzkoD6mUk6xubKJJ1eX8/KmahoOdzAi1s1pk9K5ZGwHF5X8lBFVn8CCG+HiX2qRD5b9m+Cxq+BIA5z/E1h447Bf26gr9HUt7WypamZTxUHWlzdRvLeBg62dAEzOSuasKZlcPGsMc8eO1v53pXy6uj18WtrAh+u3MnbHg1zR+RIdxPCbmG9SO+EyZuePZlZ+CjPGpJCSqAuYDVtLLSy/BUregJxZcM5/w+QLvQcvGYITFfqA7lFELhKRHSJSIiJ39HF9vIg86bt+pYgU9rju+77Ld4jI54f0DAKwfX8zy+77hHk/fYOin73JdQ+u4tf/2klpbQsXTs/m11fO5sPvncMb3z6L//7idOYXpDquyD/22GMUFhbicrnIyMggOTkZETnuJyEhgdjY2D6vG+gnJiYGESEjI4OMjIzPXFZYWMhjjz12NIv/ehHB7XYft42d9c5/oh+Xy8Wtt95qdeSQi3G7OD25iju2f4lrupbTOW0xb537Ik2Tr2BDeRP/8+p2rvnbSmbf9S9m/feLFF7/a2JGph/3vvC/R090+a233trn773fiz1fd/999HxP+t93A/2cf/75x/3NXS7XcX/nnvfd+3Qo7+3+Xg8AkjPh2qfhigegrQke/zI8fd3w/5B9Mcac8AdwA7uBCUAcsAGY3mubW4H/9Z1fBjzpOz/dt308MN53P+4TPd78+fPNUOyrP2yW3POh+e7TG8z9H5Saj3bVmqYjHUO6Lzt69NFHTWJiogEs/UlMTDS33HKLiY2NPeE2jz76qNUvWb8effTRE+bv7+eWW26xOnrodXcZ8+ZPjKkrOe6q+pZ28872A+bm3z1tshf/P5Pztd8Z3LGfeV/0fo/2d/lgX/dgvP9FJCjv/0Df231l7vf2XR3GbHjSmO2vDvlPBxSb/up4f1eYY0X8c8DrPX7/PvD9Xtu8DnzOdz4GqAOk97Y9t+vvZ6iFPtIVFBRYXuT9P263e8BtCgoKrH7J+jXU19Ltdlsd3Rb6e/36e18E8n4Z6HW30/s/0Pd2f5lD9dngBIU+kK6bPKDngsoVvsv63MYY0wU0AekB3hYRuUlEikWkuLa2NoBI0WffvjCtSBiA7u6Bd+O2U97ehpotkOcdDfp7/fp7fYb7unV3d9vq/RRolv62s+K52GKiuDHmPmNMkTGmKDMz0+o4tjRu3DirIxzldg88O8BOeXsbarZAnnc06O/16+/1Ge7r5na7bfV+CjRLf9tZ8VwCKfSVwNgev+f7LutzGxGJAVKA+gBvqwJw9913k5iYaHUMEhMTuemmm4iN7X/WRWJiInfffXcYUw3O3XfffcL8/bnppptCkMZ5+nov+t8XgV4+GDfddFNQ3v/BWIhwMO/t/l4nSz4b/fXpmGP96jFAKd7BVP9g7Ixe2/wbnx2Mfcp3fgafHYwtJUSDsdHg0UcfNQUFBUZETHp6uklKSuqzDzA+Pt7ExMQMq/89PT3dpKenf+aygoKCowNJjz766NHrAeNyuY7bxs565z/Rj4hEx0DsIPR8L/Z+XwRy+S233NLn773fiz1fd/999HxP+t93A/2cd955x/3New/O+n/333fv06G8t/t7PUKBE/TRBzSPXkS+APwe7wycB40xd4vIXb47XiEiCcAjwFygAVhmjCn13fYHwDeALuB2Y8yrJ3osqxc1U0opJ4q6HaaUUiraDHuHKaWUUs6lhV4ppSKcFnqllIpwWuiVUirC2W4wVkRqgb1W5+glA++yDpEikp5PJD0X0OdjZ3Z/LgXGmD73OLVdobcjESnubzTbiSLp+UTScwF9Pnbm5OeiXTdKKRXhtNArpVSE00IfmPusDhBkkfR8Ium5gD4fO3Psc9E+eqWUinDaoldKqQinhV4ppSKcFvp+iMiVIrJFRDwiUtTrurAc8DxUROTHIlIpIut9P1+wOtNQDHTQeqcRkTIR2eT7mzhqZT8ReVBEakRkc4/L0kTkDRHZ5TtNtTLjYPTzfBz7udFC37/NwOXA+z0vFJHpeNfcnwFcBPxFRJx46KHfGWPm+H5esTrMYPle8z8DF+M9CP3Vvr+N053j+5s4bb72Q3g/Dz3dAbxljJkMvOX73Ske4vjnAw793Gih74cxZpsxZkcfVy0BnjDGtBtj9gAlwMLwplN4X/MSY0ypMaYDeALv30ZZwBjzPt5jUfS0BHjYd/5hYGlYQw1DP8/HsbTQD15ABzx3gNtEZKPvK6pjvlL3ECl/h54M8C8RWSMikXDcwmxjTLXv/H4g28owQeLIz01UF3oReVNENvfx4/iW4QDP7a/ARGAOUA38xtKwyu90Y8w8vN1R/yYiZ1odKFh8h7pz+lxux35uYqwOYCVjzPlDuJkjDnge6HMTkb8BL4U4Tig44u8wGMaYSt9pjYg8j7d76v0T38rWDojIGGNMtYiMAWqsDjQcxpgD/vNO+9xEdYt+iFYAy0QkXkTGA5OBVRZnGhTfh87vMrwDz06zGpgsIuNFJA7vAPkKizMNmYgkichI/3ngQpz5d+lpBXCd7/x1wAsWZhk2J39uorpFfyIichnwJyATeFlE1htjPm+M2SIiTwFb8R7w/N+MMd1WZh2CX4nIHLxfpcuAm62NM3jGmC4RuQ14nWMHrd9icazhyAaeFxHwfi7/aYx5zdpIgRORx4GzgQwRqQB+BPwCeEpEbsC79PhV1iUcnH6ez9lO/dzoEghKKRXhtOtGKaUinBZ6pZSKcFrolVIqwmmhV0qpCKeFXimlIpwWeqWUinBa6JVSKsL9f8a8NardsTeXAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "np.random.seed(1)\n",
    "mus, sigmas, pis = EM(data, 2, 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since we clustered the data with 2 gaussians now, the two closest groups got merged and classified into one."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's bring the groups closer to see how the algorithm performs. \n",
    "\n",
    "Do not keep the seed fixed, and re run this cell several times to see how the results change. Sometimes it takes more iterations to converge. Remember the algorithm might not converge due to inizialization, just re run the cell when finished."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration: 0. Means: [-2  7  0]. Stds: [1.37980329 1.76292007 1.771595  ]\n",
      "Iteration: 1. Means: [-1.82080277  6.96152195  1.07309409]. Stds: [1.05803027 1.66523711 1.66203592]\n",
      "Iteration: 2. Means: [-2.04197617  7.1088076   1.42489287]. Stds: [0.59823215 1.50399339 1.47501905]\n",
      "Iteration: 3. Means: [-2.08926719  7.1653378   1.7009021 ]. Stds: [0.49461455 1.42557958 1.13269852]\n",
      "Iteration: 4. Means: [-2.06879971  7.13933465  1.79807834]. Stds: [0.50339359 1.42522766 0.81029457]\n",
      "Iteration: 5. Means: [-2.02461955  7.10525348  1.85719672]. Stds: [0.54050226 1.44206773 0.5243019 ]\n",
      "Iteration: 6. Means: [-1.96643293  7.10193117  1.92309894]. Stds: [0.61565846 1.44376213 0.33984427]\n",
      "Iteration: 7. Means: [-1.89328796  7.1001891   1.97312768]. Stds: [0.72932963 1.44817223 0.1924589 ]\n",
      "Iteration: 8. Means: [-1.88418501  7.10287735  1.97881273]. Stds: [0.74199254 1.44232544 0.16772589]\n",
      "Iteration: 9. Means: [-1.88415379  7.10275689  1.97879341]. Stds: [0.74203766 1.44243848 0.16770333]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXYAAAD4CAYAAAD4k815AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deZhcZZn+8e9TVV29ZYMsQLZOwoQlhNXIogJqQAP4S3TcwqDiGgURlBllcWQEN8ZBFB0Eg4JLMjK4jUGCARdENknYyUZWSIdAVrJ1d239/P44VaHT6e5aUt2nunJ/rquudFWdOvWkuvqut57znnPM3RERkeoRCbsAEREpLwW7iEiVUbCLiFQZBbuISJVRsIuIVJlYWE88bNgwHzduXFhPLyLSLz3xxBOb3X14T8uEFuzjxo1j0aJFYT29iEi/ZGYv5ltGrRgRkSqjYBcRqTIKdhGRKqNgFxGpMgp2EZEqo2AXEakyCnYRkSqjYJeKsTu1mzuev4MtrVvCLkWkX1OwS8W4Z/U93PjEjfxs8c/CLkWkX1OwS8V4YdsLALy4I++OdSLSAwW7VIz1u9YDsGH3hpArEenfFOxSMTa3bgZgU+umkCsR6d8U7FIxtrZtBWBb2zbavT3kakT6LwW7VAR3Z1vbNuKROBnPsDO5M+ySRPotBbtUhLZMG6n2FGMGjgFgR2JHyBWJ9F8KdqkIuRH6yAEjAdiRUrCLlErBLhUhN0IfNWDUXtdFpHgKdqkIu1K7ADhswGFAsBeqiJRGwS4VIRfkIxpG7HVdRIqnYJeKkBuxj6gfsdd1ESmegl0qQkuqBXh9xJ67LiLFU7BLRci1XobUDiEWiakVI7IfFOxSEVrTrQA01DTQEGvYc11Eiqdgl4rQkm4halFqIjXUx+ppSasVI1IqBbtUhNZ0Kw2xBsyM+li9Ruwi+0HBLhWhJdVCfaweIBixa+OpSMkKCnYzm2Zmy81spZld2cNy7zUzN7Mp5StRDgRt6Tbqa4Jgb6hRj11kf+QNdjOLAjcD5wCTgPPNbFIXyw0ELgP+Ue4ipfq1Zlr3jNjrYnW0pdtCrkik/ypkxH4ysNLdV7t7ErgTmNHFcl8D/hPQX6QUrS3dRl20DoD6aD1tGb2NREpVSLCPAtZ1uN6cvW0PMzsJGOPu9/S0IjObZWaLzGzRpk06S468ri3dRl0sCPa6WJ1aMSL7Yb83nppZBLgR+Nd8y7r7bHef4u5Thg8fvr9PLVWkLbN3sKsVI1K6QoJ9PTCmw/XR2dtyBgKTgQfMbC1wKjBPG1ClGG3pNuqj2R57VCN2kf1RSLAvBCaa2XgziwMzgXm5O919u7sPc/dx7j4OeAyY7u6LeqViqUqt6VZqY7VAMN0xkUmEXJFI/5U32N09DVwCLACWAne5+2Izu87Mpvd2gXJgSGQSezae1kZryXiGVHsq5KpE+qdYIQu5+3xgfqfbrulm2bfuf1lyoElkEnv12AES6QQ18ZowyxLpl7TnqYTO3YNWTDRoxeT+1ZRHkdIo2CV0yfYk8PpIPRfs6rOLlEbBLqHLTW3MBXpuD9REWsEuUgoFu4QuNzJXK0akPBTsErrugj2ZSYZWk0h/pmCX0OUCPDePPR6NA+qxi5RKwS6hy7VcaiNBsO+Z7qhgFymJgl1CpxG7SHkp2CV0nWfF7Nl4qgOBiZREwS6h2zNi7xTsGrGLlEbBLqHTrBiR8lKwS+gU7CLlpWCX0OUCPLfRVBtPRfaPgl1C13nEHovEiFpUwS5SIgW7hC533PVcsEMwalcrRqQ0CnYJXW5knmvBQBDyGrGLlEbBLqFLZBJELEIs8vp5X+LRuIJdpEQKdgldMpPcqw0DEI/E9xynXUSKo2CX0CUyib3aMBC0YtRjFymNgl1Cl8wkiUf2DnZtPBUpnYJdQpfMJPcZsavHLlI6BbuELtm+b7CrFSNSOgW7hC6RSezTiqmJ1ijYRUqkYJfQpTKpfWbF1EZqNStGpEQKdgldIpOgJlqz123aeCpSOgW7hK7LeewKdpGSKdgldMn2rqc7alaMSGkU7BK6ZCa5bytGe56KlEzBLqFLZBJdtmJSmVRIFYn0bwp2CV0qk9pnHntNpEYjdpESKdgldMn2JDWRfWfFtHs76fZ0SFWJ9F8KdgldV60YnfdUpHQKdgldV62Y3HUFu0jxFOwSqnZvJ+3pfQ8pkG3NqM8uUjwFu4QqNyLvas/TjveLSOEKCnYzm2Zmy81spZld2cX9nzGz58zsaTN7yMwmlb9UqUa5Efk+Oyhlr2vELlK8vMFuZlHgZuAcYBJwfhfB/T/ufqy7nwB8G7ix7JVKVcqNyNVjFymfQkbsJwMr3X21uyeBO4EZHRdw9x0drjYCXr4SpZrldkLqagclULCLlCKWfxFGAes6XG8GTum8kJl9FrgciANv72pFZjYLmAUwduzYYmuVKpRrtajHLlI+Zdt46u43u/vhwBXAv3ezzGx3n+LuU4YPH16up5Z+bE8rRj12kbIpJNjXA2M6XB+dva07dwLv3p+i5MCxZ+NpNz12HS9GpHiFBPtCYKKZjTezODATmNdxATOb2OHqecCK8pUo1SwX3JrHLlI+eXvs7p42s0uABUAUuN3dF5vZdcAid58HXGJmZwEpYBtwYW8WLdVD89hFyq+Qjae4+3xgfqfbrunw82VlrksOEPlaMQp2keJpz1MJVe4sSd1uPFWwixRNwS6h2tNj727Erh67SNEU7BKq7g4psGfjqUbsIkVTsEuo8m481YhdpGgKdglVd8eKiUViRC2qeewiJVCwS6hS7V3PY4cg7NWKESmegl1C1d2IHYJRu1oxIsVTsEuo9hwErNPJrCEYxWvELlI8BbuEKplJUhOpwcz2uS8eje9p1YhI4RTsEqpkJtllGwaywa6NpyJFU7BLqFLtqS43nELQnlGPXaR4CnYJVTKT3GcOe45mxYiURsEuoUq2J7sdsccjcY3YRUqgYJdQqccuUn4KdglVKpPqNthrojVqxYiUQMEuoUpkEj22YhLtiT6uSKT/U7BLqJLtPW88VStGpHgKdglVKtP9dEfteSpSGgW7hCrZnqQ2WtvlffGoZsWIlELBLqHSPHaR8lOwS6h6nO4Y0bFiREqhYJdQ9biDkkbsIiVRsEuo8s1jz3iGdHu6j6sS6d8U7BKqZHuyy2Oxw+tnVdKoXaQ4CnYJVU899txsGfXZRYqjYJfQuHtw2N4ejhUDGrGLFEvBLqHp6UTW8Prp8jSXXaQ4CnYJTU8nsu54u0bsIsVRsEtoEpngAF/dbjxVsIuURMEuocm1Yro7pEDudgW7SHEU7BKafK0Y9dhFSqNgl9DkWjHqsYuUl4JdQpMbifd02F7QPHaRYinYJTS5k2jkG7HnRvYiUhgFu4RGrRiR3lFQsJvZNDNbbmYrzezKLu6/3MyWmNmzZvZnM2sqf6lSbTSPXaR35A12M4sCNwPnAJOA881sUqfFngKmuPtxwK+Bb5e7UKk++Xrsmu4oUppCRuwnAyvdfbW7J4E7gRkdF3D3v7p7S/bqY8Do8pYp1SgX2N3NY9d0R5HSFBLso4B1Ha43Z2/rzieAe7u6w8xmmdkiM1u0adOmwquUqpSvFaMRu0hpyrrx1Mw+BEwB/qur+919trtPcfcpw4cPL+dTSz+kjacivSNWwDLrgTEdro/O3rYXMzsL+DJwprtrfprkla8VE7EIsUhM0x1FilTIiH0hMNHMxptZHJgJzOu4gJmdCPwImO7uG8tfplSjXO+8u4OAQbBhVT12keLkDXZ3TwOXAAuApcBd7r7YzK4zs+nZxf4LGAD8ysyeNrN53axOZI98rRgIRvNqxYgUp5BWDO4+H5jf6bZrOvx8VpnrkgNAKpMialFike7fhvFoXK0YkSJpz1MJTSKT6HG0Dhqxi5RCwS6hSWQS3W44zYlH4wp2kSIp2CU0yUwy74hdrRiR4inYJTTJ9mTeEbtaMSLFU7BLaJKZZLfHicnRiF2keAp2CU3BG081j12kKAp2CU0hPfbaaC2JtEbsIsVQsEtoEpkEddG6HpeJR7XnqUixFOwSmkJbMeqxixRHwS6hSWbyz4qJR7TxVKRYCnYJTSKToDbWc7DXxeo03VGkSAp2CU2he54mMgncvY+qEun/FOwSmkQmkXcee220lnZvJ+3pPqpKpP9TsEtoCumx5+7XlEeRwinYJRTuTlu6LW+PfU+wawOqSMEU7BKKdHsaxwsesWsDqkjhFOwSirZMG9D9+U5zcvfnlheR/BTsEopcayXfnqdqxYgUT8EuoSjkfKfAnh68gl2kcAp2CUVulktdrMARu2bFiBRMwS6hyPXMCzlWTMflRSQ/BbuEItdaqY/W97iceuwixVOwSyja0tlZMQUcK6bj8iKSn4JdQqFZMSK9R8EuoSh0Hnsu+DViFymcgl1CkZvlkveQAjFtPBUploJdQqFWjEjvUbBLKHKtlXzz2CMWoS5ap3nsIkVQsEsocq2VfMEOQTumNd3a2yWJVA0Fu4SiLd1GzGLURGryLlsXrVOPXaQICnYJRWu6Ne+G05z6WL1aMSJFULBLKNoybXk3nObUxepozagVI1IoBbuEoi3dVlB/HbKtGM1jFymYgl1C0ZZuoz7W83FicupiCnaRYijYJRSt6dbiWjGaFSNSMAW7hKI13Up9TWEj9vpovWbFiBShoGA3s2lmttzMVprZlV3cf4aZPWlmaTN7X/nLlGpTzIi9vqae1pRG7CKFiuVbwMyiwM3A2UAzsNDM5rn7kg6LvQR8FPi33iiyP9rRluLPS19lxau7iEUjHDdqMGccMZx4TF+SIDsrpoiNp2rFiBQub7ADJwMr3X01gJndCcwA9gS7u6/N3tfeCzX2K6lMO7c+sIofPrCK1lSGaMRwd9odDhlUy5fPm8T040eGXWboWtOtNMQaClq2Plav6Y4iRSgk2EcB6zpcbwZOKeXJzGwWMAtg7Nixpayiom3bneTTc57g8TVbOWfyocw6YwLHjR5CKtPOI6s2870/reDSXz7FY6u3cN30Y4hFD9zRe2u6teBZMfWxetLtaVKZFDXR/HuqihzoCgn2snH32cBsgClTpnhfPndve60lyfm3Pcbqzbv53gdP4N0njtpzXzQS5e1HHcKZR4zghvuWc8sDq9jemuL7M08kGrEQqw5PS6qFhprCRuy55VrSLQyODu7NskSqQiHBvh4Y0+H66OxtkpVIZ/jUzxexetNufnzhFM44YniXy0UjxhXTjmJIfQ3funcZwwfU8tXpx/RxteFLt6dJtaeKGrFDMMofXKtgF8mnkGBfCEw0s/EEgT4T+Jderaqfue7uJSxcu43vn39it6He0afPPJyNOxP85KE1HDtqMO99w+g+qLJy5DaEFhvsLemWXqtJpJrkbfK6exq4BFgALAXucvfFZnadmU0HMLM3mlkz8H7gR2a2uDeLriT3PLuBuf94iU+fOaGojaJXnXMUp00Yypf/7zlWbdrVixVWnpZUENAFt2KyG1k1M0akMAVtvXP3+e5+hLsf7u7fyN52jbvPy/680N1Hu3ujuw919wOiv7BxRxtX/+45jh89mH97x5FFPTYWjXDTzBOor4nyhf99mnTmwJlQlBt5FzorprGmMXhcSiN2kUIcuNMyyuCa3y+mNZXhxg+eQE0JM1xGDKrja++ezLPN2/nJQ2t6ocLKVGyw50b2GrGLFEbBXqL7l7zKHxe/wufPmsjhwweUvJ7zjj2Msycdwnf/9ALrth4YI9LcyDs3Es8n9wGwO7W712oSqSYK9hK0pTJce/dijjhkAJ86fcJ+rcvMuHb6MUTMuPbuJfkfUAVywV7oxtPciF3BLlIYBXsJfvS31TRva+W6GZNLasF0NnJIPZdOnciflr7Kgy9sKkOFlS0X0I3xwkbs6rGLFEfBXqQN21u55W8rOe/Ywzh1wtCyrfdjbx5H09AGvvaHJVW/IXVXKpgF1BhTK0akNyjYi3TDghdod7jynKPKut7aWJSrzz2aFRt3cefCdfkf0I/lRt4D4oVtm4hGotTH6hXsIgVSsBdh8cvb+e1TzXzszeMYc3BhMzqK8Y5Jh3Dy+IP57v0vsCuRLvv6K0VuxF5ojx2CdkzucSLSMwV7Ea6/dxmD62u4+K3/1CvrNzOuPvdotuxOMvvB1b3yHJVgd2o3A2oGELHC334DagYo2EUKpGAv0EMrNvP3FZu55G3/xOD63jvC4AljhnDecYdx24Or2bizOs8atDO5s+CpjjkKdpHCKdgL0N7uXP/HpYwaUs+HT2vq9ef74juOJJVp5/t/XtHrzxWG3andDIwPLOoxjfFGdiUV7CKFULAX4J7nNvD8+h1cfvYR1Maivf5844Y1MvPkMdz5+DrWbq6+DYY7UzsZUFPcTl0DawayM7mzlyoSqS4K9jxSmXa+c99yjjp04F7HWO9tl06dSE00wg33Le+z5+wrO5M7ix6xD4wP1IhdpEAK9jzufPwl1m5p4UvTjuzTk2KMGFjHJ08fzx+e3cCzza/12fP2hZ3JnQyqHVTUYwbFB7EzpRG7SCEU7D3YlUhz059XcPL4g3nbkSP6/PlnnTGBgxpquP7eZbhXzwmndiR3MLCm+BF7a7qVVCbVS1WJVI8+PTVef3Pbg6vZvCvJ7I8chVnfn8JuYF0Nl06dyLV3L+FvL2zirSF8uJRbu7eX1IrJjfB3JHcwtL58e/wKsGsjvPIsbFoO29bCjpdh92ZI7IRMMlgmVgd1g6BxOAwaBUMnwPCj4NDjgtuloijYu7FxRxu3/X015x57KCeNPSi0Oi44pYmfPrKWb81fxukTh/f7c6TuSu2i3dsZUjukqMcNjgenxNue3K5g31+7NsLKP8Hqv8GLj8D2l16/r25wENyNw6BxPETjgEM6AW074NXFsOI+2HPcHoPhR8K4t8CEtwaX2uI+tKX8FOzd+M59L5DKtHPFtPIeOqBY8ViEK6YdxcVzn+RXi9Yx8+Sxodazv7YntgMU3WPPnet0R2JH2Ws6IOx4GRb/Dhb/HzQvBBwahgaBfMosOOwEGHF0EOj5uMOO9bBxKbz8FKz7Bzz9S1j44+CDYPwZcMw/w9HvCj4opM8p2Luw+OXt3PXEOj7x5vE0DS1uR5recM7kQ5nSdBA33Lec8447jIF1vbeDVG/LBXtuBF6o3Ah/W9u2stdUtdJJWHY3PDUHVv0V8KB18rar4YhpcMhkiJSwmc0MBo8OLhPPDm7LpIKAX34vLL0bfn8x3HM5HPUuOOnDMO6M0p5LSqJg78TdufbuJRzUEOdzUyeGXQ4QHGrgK++axIybH+a//7KSq849OuySSpYL5oPqimtvDakLgv21RHXNEOoVO14ORs9P/hx2b4LBY+DML8GxH4BhvXM4DKI1weh/3FvgHV+H9U/AM3fCc7+C538NBx8Ob/wknPgh9eT7gIK9k3nPvMzja7byjfdM7tVDBxTr+DFDeP8bRnP7w2v4wBvH7NdZm8KUC+Zig/2g2mD5bQmN2Lu14Vl45Aew+LfQnglG5W/8JBz+9r4dLZvB6CnB5R1fhyW/Dz5oFlwFf/0mnPQROPUiGDKm72o6wOi7UQc72lJ8456lHDd6MDPfWHm97CvOOYq6mijX/P75fjv9cWvbVqD4YK+P1VMXrVMrpjN3WPN3+MV74Eenw/L5cPIsuPQp+Jc7YeJZ4bZAaurg+A/CJ++HT/0VjjwHHv8R3HQ8/PbTsHFZeLVVMQV7BzcsWM7mXQm+NmNyRc4+GTagli+980geXrmF/3t6fdjllGRL2xZqIjVFz2M3Mw6uO5gtrVt6qbJ+xh1W3A+3vxN+9i545XmY+h/whcUw7Vtw8PiwK9zXqJPgvbfBpU/DKZ+GpfPgh6fAnRfAy0+HXV1VUbBnLVy7lV889iIfOW0cx48pbipeX/qXU5o4cewQrrt7CZt3JcIup2hbWrcwtH5oSfsFDK0fyubWzb1QVT/iDsvmw+y3wtz3Bf30c2+Azz8Lp18O9ZX73t1jyJjgw+fzz8MZXwq+ccw+E+Z+AJqfCLu6qqBgB1qSab7062cZNaSeL77zyLDL6VE0Ynz7vcexO5Hh33/X/1oym1o2MaK+tB2thtUPY1Nr9Z8Ttkvt7bBkHtx6Otx5PrS9BtN/AJ97Ek7+FNQUftKSitE4FN7+ZfjCc/C2f4fmx+HHb4df/DOsezzs6vo1BTvwjXuWsmbzbr79vuNorK387ckTDxnIF84+gj8ufoXfPNm/WjKbWjcxrL6AudJdGNEw4sAL9vZMMP/81rfAXR+G1G6Y8UO4ZFGwETIWD7vC/Vc3GM78Inz+uaCdtOFp+MnZ8PMZwQ5UUrQDPtj/+PwG5v7jJT51+njedHhpgROGWWdM4OTxB3PN759n9ab+c9TDV3a/wiGNh5T02BENI9ie2E5rurXMVVWgTDqYLvjDU+FXH4X2FLxnNnx2IZx4QTC9sNrUDgzaSZc9C2dfF+zlesc5cMe5sOovQRtKCnJAB/vqTbv44q+e5fjRg/niO8Pdw7RY0Yhx08wTqI1FuGjOk+zuB+dI3ZXcxa7ULg5tPLSkxx/WeBgQfDhUrVQrPH4b/OBE+N2nIVID77sdLn4smF0SrfxvlPutdgC8+bIg4KddD1vXBLN+bntbMHWyPRN2hRXvgA327S0pPvXzRdTEItx8wUnEY/3vpThscD3fP/9EVmzcyeV3PU17e2WPaNbvCtpGowaUdlz73ONy66kquzfDA/8J3z0G5v8bDDgEzr8TPvMQTH4vRHr/BC8VJ94QzHe/7Gn4f9+H1tfgro/Af0+BhT+BZEv+dRyg+l+alUFbKsOsXyzipa0t/PCCkxh9UEPYJZXs9InD+fJ5k1iw+FW+ds+Sit6Y2ryzGYDRA0eX9Pjc49btXFe2mkL3ynPw+0vgxknwwDdh1BT46D3wifuDOd/aDR9itfCGC+FzT8D7fxr05O+5HL47Cf70VXitit4PZXIAfK/bWyKd4eK5T/KPNVu5aeYJnDqh/x8p8ONvHkfzthbueHgtA2pjXH72EaEcZjifNTvWANA0sLTzxg6vH059rJ4Xd7xYzrL6Xqo1aCksugPWPQaxejjhfDj14uBIidK1SBSOeQ9Meje89Cg8ejM8fFNwOeIcmPKx7F62B+C3m04OqGDflUhz0Zwn+PuKzXzjPZOZcULfnequN5kZXzlvEi2JDD/4y0pakxmuPvdoIhW2k9Wq11YxomEEA+KlHQ7BzJgweAIrX1tZ5sr6gDs0L4Jn/gee+w0ktgfHT3nH14Pjp9SHd2jofscMmt4UXF57CRbdDk/+ApbfExxy+Pjzg0tvHRenHzhggn3d1hY+/YsnWP7qTr79vuP4wJTqOk5FJGJ865+PpbYmwo8fWsP611q54f3HV9T0zeXblnPEQUfs1zqOOOgIHlj3AO5ekd9K9uIOrz4fHCr3+d/AtjXBCSuOnh6E+fgzgpCS0g0ZC2d9Fd56dRDsT82Bh26Ev98AI08MDh88aQYcVNq3xP6qcv7qe4m7c89zG7j6t8/hwI8vnBLKae76QiRiXDv9GEYfVM/19y5j+Ss7+d7MEzhudPh7I+5O7WbVa6uYOnbqfq1n8rDJ/G7l72je2cyYQRX44Zxqg5ceCXb3X3YPvPYiWATGnQ6n/ytMmq5jlPeGWDxo0xzznmBv3Od/ExxZ8v6vBJdDj4UjzwsOMzzyxKpv11R1sK/etItvzl/Kn5Zu5PjRg7lp5omMGxb+8dV7k5kx64zDmTxqMJf/7zO8++aH+chp47hs6kQOagxvZ5YnXn2Cdm/npBEn7dd6co9//JXHKyPY04ngOCcvPQJrHoQXH4V0K0RrYcKZ8JYvBMckHzA87EoPHINGwps+F1y2rg6OD79sPvztP+Fv1wdtr3FvgfFnwtjTghOMVFnQW1izKKZMmeKLFi0q+3rdnWebt/PTR9Yy75mXqYtFuHTqRD7xlvHEogfWDIPtrSm+/cdl/PLxl2iIx7jglLF86NQmxhzc97OArnn4GhasXcCDMx+kNlpb8nrcnWm/mcaEIRO45axbylhhAVKtwXlBX30+OETuy08G/2ayx+wZMSkYmf/T1CA44tU9iOh3WrYGOzqt+kvwIbw9O5smPhBGnQgjTwpG9oceG2z/qNB9BszsCXef0uNC7p73AkwDlgMrgSu7uL8W+N/s/f8AxuVb5xve8AYvl0Qq4wvXbPHvLFjmZ9/4gDdd8Qef9JV7/bq7F/vGHW1Fr2/OnDne2NjoQJeXoUOH9nh/IZdoNOoXXXSRz5kzx4cOHbrXuufMmbOnlosuusij0agDHolEvLGx0c3Mm5qa9lquu/9HU1OTx4c1edPMr/q4K+72cVf+wd9/yyN+24OrfOmG7Z7JtHf5mEKfoxDbWrf5G+e80b/y0Ff2/J9Kec3MzAGf+LGJPvmnk/3F7S/ud217aW93373FfcNz7svudX/sVvf5V7jPeb/7TSe4f3WI+38MCi5fP8z9J9PcF3zZfcnd7rs2uXv+16/z/RdddNFev//GxkYfOnToXvc3NTXtec8A+6y32N9Z5/dU7ufOl55qiUQiPb6vO9bVU+35dP776Px32N268r4m7e3uW9e6P/1L9z9c7n7rGe7XDn3993vdMPebT3W/8wL3+77ij938Gf/waSP9qGFRn3T42LL8XZQKWOR58jXviN3MosALwNlAM7AQON/dl3RY5mLgOHf/jJnNBN7j7h/sab2ljtjXbN7N0+u2sX5bK2u3tPDCqztZ9spOkul2IgZTmg5m+gkjmX7CSAaVcAq5uXPncuGFF5LJ9M3ebWa2z9zzeDzO7bffzsMPP8wtt3Q/Km1oaGD27NlccMEF+9w3d+5cZs2aRUvL6ztxDBgxhg9ccSPNNoLlr+4EYGBtjKNHDmLiiAFseXE5d91xK9tXLKS9dUfe5yhUS6qFu5bfxemjT+c7V3+nx/9TIWKDY4w4ewTXzriWT37ok6/fkdgJiV2QbgtaJOnWYJSdbIHkruCS2BmclLntNWjdFoziWjbDrk2we2Pw2I5qGuCg8TBsIgw7Ag6ZFJxS7uAJ+3x97+o17/j6dXV/qXLrBXp8zs4uvvji/X79CzF16lQeffTRLl1RVk8AAAgJSURBVP+vhb6n5s6dy8c//nGSyWS3y+T+VjquK9/voVvpJGxaBhuXBIcz2LwCtqwgs2UNUfbOg51JxxtHMOjQCdAwDBoOgrohwdE1awcHe8/GBwTf2uKNwUHaYvXBnPxYbbCdpcQDtxUyYi8k2E8Dvuru78xevwrA3b/VYZkF2WUeNbMY8Aow3HtYeanB/qO/reJb9wYH5x8xsJaJhwzgmJGDOWnsEE6dMJQhDfvXRx43bhwvvhj+POmmpiaam5vzfsA0NTWxdu3afW7v7v+RW379a608umoLT6/bxpKXd7Bq0262t6YAeGXuFSSaF+d9jlLEYrGyfWjuU9e8zwWngytEfGDwR9hwMNQfHOzpOWA4DBwJgw4LTic3eAwMGFHwzJV8r3m531tNTcFMj56es7Nyvv77o5D3VKGvV+d15fs9FGvC+CbSW9cxdnCEpiHGyIERRg00JowYwPS3n5odIGwJ9owt9DhG530nOLtVCcoV7O8Dprn7J7PXPwyc4u6XdFjm+ewyzdnrq7LLbO60rlnALICxY8e+oZQ3+aadCXa0pRg5uJ76ePk3eEQikYrYe7OrkXx3y7W3t+9ze3f/j+6WB4jWDyQ6YCjp7a/gqURBjylWOaco7lPXmr/DlhXBlMJYbTBCijcEo+7cyKl2UHDphf5pvte83O+t3GtZzO+5UqaIFvKeKvT16ryuUt77pdTR5frSSUjsyF52QaoFkruDb4Kp1te/TTa9KdhoW4JCgr1Ptw64+2xgNgQj9lLWMXxgLcMHlr7xLZ+xY8dWxIh97NixBY3Yx47t+hR+3f0/ulseYMwhQ4t+TLGi0WjZRoz71DX+9OASknyvebnfW7n1FvM7K+frvz8KeU8V+np1Xlcp7/1S6uhyfbE4xIZBY8hHis3XhAdOAxZ0uH4VcFWnZRYAp2V/jgGbyX4b6O5Szo2n5TRnzpxuNyb1xiW3QbDjJR6P+5w5c/JuZGxoaOhx41FDQ0PBy5f6mGKVuuG0mP97WPK9fl3dv7///2J/Z+V6/fNdpk6d2u3/tdDf3Zw5czwej/f4PLm/lWJ+D+X+vfY1Cth4Wkiwx4DVwHggDjwDHNNpmc8Ct2Z/ngnclW+9lRrs7tU3K6aYGS69MSums/2dFdNbdZWDZsX0k1kxReqLv4tCFRLsBc1jN7Nzge8BUeB2d/+GmV2XfYJ5ZlYH/AI4EdgKzHT31T2ts7fmsYuIVLOy9djdfT4wv9Nt13T4uQ14fylFiohIeR1Yu2KKiBwAFOwiIlVGwS4iUmUU7CIiVSa0ozua2SZgf/fWGEYwZ75SVXJ9qq00qq00lVwbVHZ9nWtrcvcejwMdWrCXg5ktyjftJ0yVXJ9qK41qK00l1waVXV8ptakVIyJSZRTsIiJVpr8H++ywC8ijkutTbaVRbaWp5NqgsusrurZ+3WMXEZF99fcRu4iIdKJgFxGpMlUT7Gb2r2bmZhbyEe5fZ2b/ZWbLzOxZM/udmQ2pgJqmmdlyM1tpZleGXU9HZjbGzP5qZkvMbLGZXRZ2TR2ZWdTMnjKzP4RdS2dmNsTMfp19vy3NntKyIpjZF7K/z+fN7JfZo8GGVcvtZrYxe9a33G0Hm9n9ZrYi++9BFVZf0TlSFcFuZmOAdwAvhV1LJ/cDk939OIITgl8VZjHZE5PfDJwDTALON7NJYdbUSRr4V3efBJwKfLbC6rsMWBp2Ed24Cfijux8FHE+F1Glmo4BLgSnuPpng0N8zQyzpp8C0TrddCfzZ3ScCf85eD8tP2be+onOkKoId+C7wJYID8FcMd7/P3dPZq48Bo8OsBzgZWOnuq909CdwJzAi5pj3cfYO7P5n9eSdBOI0Kt6qAmY0GzgN+HHYtnZnZYOAM4CcA7p5099fCrWovMaA+e6L7BuDlsApx9wcJzhnR0QzgZ9mffwa8u0+L6qCr+krJkX4f7GY2A1jv7s+EXUseHwfuDbmGUcC6DtebqZDg7MzMxhGcuOUf4Vayx/cIBg/lOat3eY0HNgF3ZFtFPzazxrCLAnD39cANBN+mNwDb3f2+cKvaxyHuviH78yvAIWEWk0dBOdIvgt3M/pTtz3W+zACuBq7Jt46Qasst82WCNsPcsOrsT8xsAPAb4PPuvqMC6nkXsNHdnwi7lm7EgJOAW9z9RGA34bYT9sj2q2cQfPiMBBrN7EPhVtW97KnnKuqbf04xOVLQGZTC5u5ndXW7mR1L8IZ5xswg+IrypJmd7O6vhFlbjpl9FHgXMNXD32lgPTCmw/XR2dsqhpnVEIT6XHf/bdj1ZL0ZmJ49RWQdMMjM5rh7pQRUM9Ds7rlvN7+mQoIdOAtY4+6bAMzst8CbgDmhVrW3V83sMHffYGaHARvDLqizYnOkX4zYu+Puz7n7CHcf5+7jCN7gJ/VVqOdjZtMIvr5Pd/eWsOsBFgITzWy8mcUJNmLNC7mmPSz4dP4JsNTdbwy7nhx3v8rdR2ffYzOBv1RQqJN9v68zsyOzN00FloRYUkcvAaeaWUP29zuVCtmw28E84MLszxcCvw+xln2UkiP9Otj7gf8GBgL3m9nTZnZrmMVkN8BcAiwg+OO6y90Xh1lTJ28GPgy8Pft6PZ0dJUt+nwPmmtmzwAnAN0OuB4Dst4hfA08CzxFkTmi775vZL4FHgSPNrNnMPgFcD5xtZisIvmFcX2H1FZ0jOqSAiEiV0YhdRKTKKNhFRKqMgl1EpMoo2EVEqoyCXUSkyijYRUSqjIJdRKTK/H8X5KwNgeRvvAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "num_samples = 90\n",
    "centers = (-2,2,7)\n",
    "spreads = (5,1,8)\n",
    "\n",
    "data = generate_data(num_samples,centers,spreads)\n",
    "mus, sigmas, pis = EM(data, 3, 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Unabalanced groups\n",
    "Let's re-define the data generator to produce unabalanced groups, that is, groups that have more samples than others."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "def generate_data(N, cm, spread):\n",
    "    groups = min(len(cm), len(spread), len(N))\n",
    "    data = []\n",
    "    for i in range(groups):\n",
    "        x = np.linspace(-spread[i] / 2, spread[i] / 2, num=N[i])\n",
    "        data.append(x * np.random.rand(len(x)) + cm[i])\n",
    "\n",
    "    return np.concatenate(np.array(data)).flatten()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now generate an unbalanced dataset and visualize it. Notice how the groups have differnet amount of samples"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAD4CAYAAADhNOGaAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAPGklEQVR4nO3ccahed33H8fcnuSRdklHbJMYuae7ttDAigzofWmRudLPWVHDpNhl1fywwR0Zr/3AyWERYXTVQZa4yJo5Mi8F0WhHEMOeyWCfC2KQ3zs2WrUvM1CaLbWqKUGQrdd/9cU/Kzc1z782T58l98vT3fsHhnvM73+ee7znPPc8n55x7k6pCktSuVeNuQJI0XgaBJDXOIJCkxhkEktQ4g0CSGjc17gYuxaZNm2pmZmbcbUjSRDl69OizVbV54fhEBsHMzAyzs7PjbkOSJkqS7/Ub99aQJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDVuJEGQZGeSJ5McT7K3z/q1SR7p1n8jycyC9duTPJ/kD0fRjyTp4g0dBElWAx8D7gB2AO9IsmNB2TuB56rqNcCDwIcWrP8z4MvD9iJJGtworghuBo5X1YmqegH4LLBrQc0u4EA3/3ngTUkCkORO4L+AJ0bQiyRpQKMIgq3AU/OWT3ZjfWuq6kXgR8DGJBuAPwL+ZLmNJNmTZDbJ7JkzZ0bQtiQJxv+w+P3Ag1X1/HKFVbW/qnpV1du8efPl70ySGjE1gu9xCrh+3vK2bqxfzckkU8DVwA+BW4C3J/kw8Arg/5L8T1X9xQj6kiRdhFEEwWPAjUluYO4D/y7gtxfUHAJ2A/8EvB34alUV8EvnCpK8H3jeEJCklTV0EFTVi0nuBQ4Dq4GHquqJJPcDs1V1CPgk8Okkx4GzzIWFJOkKkLl/mE+WXq9Xs7Oz425DkiZKkqNV1Vs4Pu6HxZKkMTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaN5IgSLIzyZNJjifZ22f92iSPdOu/kWSmG39zkqNJvt19/dVR9CNJunhDB0GS1cDHgDuAHcA7kuxYUPZO4Lmqeg3wIPChbvxZ4G1V9fPAbuDTw/YjSRrMKK4IbgaOV9WJqnoB+Cywa0HNLuBAN/954E1JUlX/UlX/3Y0/AfxUkrUj6EmSdJFGEQRbgafmLZ/sxvrWVNWLwI+AjQtqfhP4ZlX97wh6kiRdpKlxNwCQ5LXM3S66fYmaPcAegO3bt69QZ5L08jeKK4JTwPXzlrd1Y31rkkwBVwM/7Ja3AV8AfqeqvrPYRqpqf1X1qqq3efPmEbQtSYLRBMFjwI1JbkiyBrgLOLSg5hBzD4MB3g58taoqySuALwF7q+ofR9CLJGlAQwdBd8//XuAw8O/A56rqiST3J/m1ruyTwMYkx4H3AOd+xfRe4DXAHyf5Vje9ctieJEkXL1U17h4G1uv1anZ2dtxtSNJESXK0qnoLx/3LYklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGjeSIEiyM8mTSY4n2dtn/dokj3Trv5FkZt6693bjTyZ5yyj66efhhx9mZmaGVatWMTMzw8MPP3zeuk2bNpHkpWnDhg0XjCVh1apVF4wNMm3YsOGlbS/s6Z577llyeX7PLTh3fC7muF511VXNHZ8WLXUeL1WfhKmpKZJM5Lk16H4PrKqGmoDVwHeAnwXWAP8K7FhQcw/wl938XcAj3fyOrn4tcEP3fVYvt83Xv/71NYiDBw/WunXrCnhpWrduXR08eLAOHjxYa9asOW/d5Z5WrVpVd9999wU9LTed67kF/d6z5aYkzRyfFi11Hl9s/SSeW4Pu91KA2er3Od5vcJAJeANweN7ye4H3Lqg5DLyhm58CngWysHZ+3VLToEEwPT3d982fnp5edN3lnlavXn1Jr5uenh74zZ9El/q+tHJ8WrTUeTxI/aT97Ay630tZLAimGN5W4Kl5yyeBWxarqaoXk/wI2NiN//OC127tt5Eke4A9ANu3bx+owe9///sDja+En/zkJ5f0unH2vJIudT9bOT4tGvQ8HvRn4Ur92VmJz6+JeVhcVfurqldVvc2bNw/02sWCY/v27QOHyqisXr36kl43rn5X2qXuZyvHp0VLnceDjA/6/cdtVPu3lFEEwSng+nnL27qxvjVJpoCrgR9e5GuHtm/fPtatW3fe2Lp169i3bx/79u1jzZo1o97kklatWsWePXsu6Gk553puQb/3bDlJmjk+LVrqPL7Y+sVcyefWoPt9SfrdLxpkYu6e/wnmHvaee1j82gU17+L8h8Wf6+Zfy/kPi09wGR4WV809cJmenq4kNT09fd6DloMHD9bGjRvPu/+2fv36C8boHkguHBtkWr9+/UvbXtjT3XffveTylfow63I5d3wu5riuXbu2uePToqXO46XqmfdcbhLPrUH3ezEs8owgc+uGk+StwEeZ+w2ih6pqX5L7u40eSnIV8GngdcBZ4K6qOtG99n3A7wIvAu+uqi8vt71er1ezs7ND9y1JLUlytKp6F4yPIghWmkEgSYNbLAgm5mGxJOnyMAgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkho3VBAkuTbJkSTHuq/XLFK3u6s5lmR3N7YuyZeS/EeSJ5I8MEwvkqRLM+wVwV7g0aq6EXi0Wz5PkmuB+4BbgJuB++YFxp9W1c8BrwN+MckdQ/YjSRrQsEGwCzjQzR8A7uxT8xbgSFWdrarngCPAzqr6cVX9A0BVvQB8E9g2ZD+SpAENGwRbqup0N/8DYEufmq3AU/OWT3ZjL0nyCuBtzF1VSJJW0NRyBUm+Aryqz6r3zV+oqkpSgzaQZAr4DPDnVXViibo9wB6A7du3D7oZSdIilg2CqrptsXVJnk5yXVWdTnId8EyfslPArfOWtwFfm7e8HzhWVR9dpo/9XS29Xm/gwJEk9TfsraFDwO5ufjfwxT41h4Hbk1zTPSS+vRsjyQeBq4F3D9mHJOkSDRsEDwBvTnIMuK1bJkkvyScAquos8AHgsW66v6rOJtnG3O2lHcA3k3wrye8N2Y8kaUCpmry7LL1er2ZnZ8fdhiRNlCRHq6q3cNy/LJakxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIaZxBIUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXFDBUGSa5McSXKs+3rNInW7u5pjSXb3WX8oyePD9CJJujTDXhHsBR6tqhuBR7vl8yS5FrgPuAW4GbhvfmAk+Q3g+SH7kCRdomGDYBdwoJs/ANzZp+YtwJGqOltVzwFHgJ0ASTYA7wE+OGQfkqRLNGwQbKmq0938D4AtfWq2Ak/NWz7ZjQF8APgI8OPlNpRkT5LZJLNnzpwZomVJ0nxTyxUk+Qrwqj6r3jd/oaoqSV3shpPcBLy6qv4gycxy9VW1H9gP0Ov1Lno7kqSlLRsEVXXbYuuSPJ3kuqo6neQ64Jk+ZaeAW+ctbwO+BrwB6CX5btfHK5N8rapuRZK0Yoa9NXQIOPdbQLuBL/apOQzcnuSa7iHx7cDhqvp4Vf1MVc0AbwT+0xCQpJU3bBA8ALw5yTHgtm6ZJL0knwCoqrPMPQt4rJvu78YkSVeAVE3e7fZer1ezs7PjbkOSJkqSo1XVWzjuXxZLUuMMAklqnEEgSY0zCCSpcQaBJDXOIJCkxhkEktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ1ziCQpMYZBJLUOINAkhpnEEhS4wwCSWqcQSBJjTMIJKlxBoEkNc4gkKTGGQSS1DiDQJIal6oadw8DS3IG+N64++hjE/DsuJsYM4+Bx6D1/Ycr9xhMV9XmhYMTGQRXqiSzVdUbdx/j5DHwGLS+/zB5x8BbQ5LUOINAkhpnEIzW/nE3cAXwGHgMWt9/mLBj4DMCSWqcVwSS1DiDQJIaZxCMWJL3JzmV5Fvd9NZx97QSkuxM8mSS40n2jrufcUjy3STf7t732XH3sxKSPJTkmSSPzxu7NsmRJMe6r9eMs8fLaZH9n7jPAIPg8niwqm7qpr8ddzOXW5LVwMeAO4AdwDuS7BhvV2PzK937PjG/Qz6kTwE7F4ztBR6tqhuBR7vll6tPceH+w4R9BhgEGoWbgeNVdaKqXgA+C+wac09aAVX1deDsguFdwIFu/gBw54o2tYIW2f+JYxBcHvcm+bfusvFle1k8z1bgqXnLJ7ux1hTw90mOJtkz7mbGaEtVne7mfwBsGWczYzJRnwEGwSVI8pUkj/eZdgEfB14N3AScBj4y1ma1kt5YVb/A3C2ydyX55XE3NG419/vprf2O+sR9BkyNu4FJVFW3XUxdkr8C/uYyt3MlOAVcP295WzfWlKo61X19JskXmLtl9vXxdjUWTye5rqpOJ7kOeGbcDa2kqnr63PykfAZ4RTBi3Q/+Ob8OPL5Y7cvIY8CNSW5Isga4Czg05p5WVJL1SX763DxwO2289/0cAnZ387uBL46xlxU3iZ8BXhGM3oeT3MTc5fB3gd8fbzuXX1W9mORe4DCwGnioqp4Yc1srbQvwhSQwd179dVX93XhbuvySfAa4FdiU5CRwH/AA8Lkk72Tuv4v/rfF1eHktsv+3TtpngP/FhCQ1zltDktQ4g0CSGmcQSFLjDAJJapxBIEmNMwgkqXEGgSQ17v8Bjvp45c2HRO8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "num_samples = (35,20,5)\n",
    "centers = (-5,2,13)\n",
    "spreads = (5,1,8)\n",
    "\n",
    "data = generate_data(num_samples,centers,spreads)\n",
    "plt.scatter(data, np.zeros(len(data)), c='k')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use EM algorithm to find these clusters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration: 0. Means: [-1  5  6]. Stds: [1.93255736 1.12812445 1.99904052]\n",
      "Iteration: 1. Means: [-3.21004689  2.03328197  7.59526303]. Stds: [3.16191945 0.13723621 5.89273524]\n",
      "Iteration: 2. Means: [-4.02606821  2.01624935  9.58382245]. Stds: [2.57866857 0.12237405 6.41328231]\n",
      "Iteration: 3. Means: [-4.88766009  1.99764022 12.02377628]. Stds: [1.30515426 0.13821751 4.77863982]\n",
      "Iteration: 4. Means: [-5.06920723  1.98349601 13.42196396]. Stds: [0.70934285 0.15187523 2.11774731]\n",
      "Iteration: 5. Means: [-5.0691668   1.98310486 13.56631226]. Stds: [0.70934546 0.15228669 1.66304623]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD4CAYAAAD8Zh1EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3de3Tcd3nn8fczM7pbUmxZ8U2W7ThX59JcnACBUxISaBKWBAptk1I22YXjEsgWtnvOEg7dlE3rA2XZsj17QrcBsoUmNE1b6JrFbCApLGwJIc6FJLZzcRxfJN9ky5Zk3eb27B8zI4/kkTSy5ztjzXxe5+h4Lj/NPGNpPvPV8/v+vj9zd0REZP6LVLoAEREpDQW6iEiVUKCLiFQJBbqISJVQoIuIVIlYpZ548eLFvnr16ko9vYjIvPTss88edvfOQvdVLNBXr17Nli1bKvX0IiLzkpntnu4+tVxERKqEAl1EpEoUFehmdpOZvWpmO8zs3gL332VmfWb2QvbrY6UvVUREZjJrD93MosADwLuBHuAZM9vk7tumbPp37n5PgBpFRKQIxYzQrwF2uPtOd48DjwK3hS1LRETmqphAXwHszbvek71tqg+a2Ytm9g9mtrLQA5nZBjPbYmZb+vr6TqFcERGZTql2in4PWO3ulwE/Ar5ZaCN3f9Dd17v7+s7OgtMoRUTkFBUT6L1A/oi7K3vbBHc/4u7j2atfB64qTXlSFZ77Gzi4tdJViFS9YgL9GeA8M1tjZvXA7cCm/A3MbFne1VuB7aUrUea1kX7YdA/83UcqXYlI1Zt1lou7J83sHuBxIAo85O5bzex+YIu7bwL+wMxuBZJAP3BXwJplPjl+MPNv/xuVrUOkBhR16L+7bwY2T7ntvrzLnwU+W9rSpCqMHKl0BSI1Q0eKSljx4UpXIFIzFOgSlgJdpGwU6BKWAl2kbBToElZi9MTldLpydYjUAAW6hJUcO3E5NT79diJy2hToElZ+oOeP1kWk5BToElZ+oCc1QhcJSYEuYeWHeH64i0jJKdAlLI3QRcpGgS5haYQuUjYKdAkrlThxOZ2sXB0iNUCBLmGl4oUvi0jJKdAlrPwQVw9dJCgFuoQ1aYSemH47ETltCnQJKxWHuuYTl0UkGAW6hJVKQH1L9rICXSQkBbqElRxXoIuUiQJdwkonoC4X6Oqhi4SkQJewUgmoVw9dpBwU6BJWKnFip6gOLBIJSoEuYaXzAl0tF5GgFOgSVioJdU2Zy2kFukhICnQJK53fQ1fLRSQkBbqElYpDrOnEZREJRoEuYaWSEK2DSJ1aLiKBKdAlrHQCIrFMqGunqEhQCnQJK5XIG6Grhy4SkgJdwnEHT2XCPBpToIsEpkCXcHIBHollvtRyEQlKgS7h5AI8Gsu2XFKVrUekyinQJZyJEXqu5aIRukhIRQW6md1kZq+a2Q4zu3eG7T5oZm5m60tXosxbU1su6qGLBDVroJtZFHgAuBlYB9xhZusKbNcKfAp4utRFyjyVC/Bcy0U9dJGgihmhXwPscPed7h4HHgVuK7DdnwB/BoyVsD6Zz3IBPjFCVw9dJKRiAn0FsDfvek/2tglmdiWw0t2/P9MDmdkGM9tiZlv6+vrmXKzMM7meuXroImVx2jtFzSwC/DnwH2bb1t0fdPf17r6+s7PzdJ9aznS5EXlELReRcigm0HuBlXnXu7K35bQClwA/MbNdwFuBTdoxKpN76LHMQUYiEkwxgf4McJ6ZrTGzeuB2YFPuTncfcPfF7r7a3VcDvwBudfctQSqW+WNSDz2q5XNFAps10N09CdwDPA5sBx5z961mdr+Z3Rq6QJnHNG1RpKxixWzk7puBzVNuu2+aba87/bKkKkz00OsyC3Qp0EWC0pGiEs7ECD2qEbpIGSjQJZxJLZeoAl0kMAW6hKMeukhZKdAlHAW6SFkp0CUcBbpIWSnQJZxJO0U1D10kNAW6hDNphF6ntVxEAlOgSzgTh/7XabVFkTJQoEs4kxbnUqCLhKZAl3Cm9tC1U1QkKAW6hKNZLiJlpUCXcBToImWlQJdwpvbQPQXula1JpIop0CWcqYtzgXaMigSkQJdwpi7OlX+biJScAl3CmdpDz79NREpOgS7hKNBFykqBLuHk+uWmHrpIOSjQJZx0EjCIRPJ66FrPRSQUBbqEk05m1nEBjdBFykCBLuGkk5l2C2iWi0gZKNAlnHTqxMhcO0VFglOgSzjp1ImReS7QPV25ekSqnAJdwkkn80boarmIhKZAl3AmBbpaLiKhKdAlHPXQRcpKgS7hpJMn99A1bVEkGAW6hKMeukhZKdAlnPwRuinQRUJToEs42ikqUlYKdAnH03lHiirQRUIrKtDN7CYze9XMdpjZvQXu/7iZvWRmL5jZ/zOzdaUvVeaddBKiU0foOrBIJJRZA93MosADwM3AOuCOAoH9bXe/1N0vB74E/HnJK5X5RztFRcqqmBH6NcAOd9/p7nHgUeC2/A3cfTDvagugMwGLeugiZRYrYpsVwN686z3AW6ZuZGafBP4QqAfeVeiBzGwDsAGgu7t7rrXKfKMDi0TKqmQ7Rd39AXdfC3wG+KNptnnQ3de7+/rOzs5SPbWcqXRgkUhZFRPovcDKvOtd2dum8yjw/tMpSqrEpPXQIyduE5Egign0Z4DzzGyNmdUDtwOb8jcws/Pyrr4XeL10Jcq8Vajl4hqhi4Qyaw/d3ZNmdg/wOBAFHnL3rWZ2P7DF3TcB95jZjUACOArcGbJomSfUQxcpq2J2iuLum4HNU267L+/yp0pcl1SDgj10BbpIKDpSVMIpOG1RLReRUBToEo4OLBIpKwW6hJN/TtHcbJdUonL1iFQ5BbqEM2naoma5iISmQJdwPKXFuUTKSIEu4aiHLlJWCnQJJz/QzTLtFwW6SDAKdAkn/8AiyFxWoIsEo0CXcPIPLAIFukhgCnQJJ7/lAtlA1ywXkVAU6BKGe4FAVw9dJCQFuoTh2emJ6qGLlI0CXcLIBbd66CJlo0CXMHLBbfmBHlUPXSQgBbqEMTFCVw9dpFwU6BJGbiSulotI2SjQJYyJQJ+yU1SLc4kEo0CXMNLZZXKnBnpKI3SRUBToEoZ66CJlp0CXMAoGunroIiEp0CWM6XroCnSRYBToEkYuuKP5gV6neegiASnQJQz10EXKToEuYaiHLlJ2CnQJQz10kbJToEsY0y7OpR66SCgKdAlj2sW5EpWpR6QGKNAljIlZLnUnbtMIXSQoBbqEoZ2iImWnQJcwUuqhi5SbAl3C0Dx0kbIrKtDN7CYze9XMdpjZvQXu/0Mz22ZmL5rZk2a2qvSlyrwyEeh5PfRonXaKigQ0a6CbWRR4ALgZWAfcYWbrpmz2PLDe3S8D/gH4UqkLlXlGPXSRsitmhH4NsMPdd7p7HHgUuC1/A3f/sbuPZK/+AugqbZky72geukjZFRPoK4C9edd7srdN56PAD06nKKkC6qGLlF1s9k2KZ2a/B6wH3jnN/RuADQDd3d2lfGo50xSch16nQBcJqJgRei+wMu96V/a2SczsRuBzwK3uPl7ogdz9QXdf7+7rOzs7T6VemS/UQxcpu2IC/RngPDNbY2b1wO3ApvwNzOwK4K/IhPmh0pcp806q0KH/MfA0pNOVqUmkys0a6O6eBO4BHge2A4+5+1Yzu9/Mbs1u9l+ABcDfm9kLZrZpmoeTWlHwBBfRyfeJSEkV1UN3983A5im33Zd3+cYS11VRqbRz5Pg4na0NmFmly5mfppuHDtm56PVlL0mk2pV0p2g12D8wyl0PPcOrB4e4Zs0ivvaR9bQ3183+jTJZ7gCiqT100AhdJBAd+p/H3fn0oy/Qe2yUT1y3luf3HOU/f29rpcuanwqe4CL7wZhSoIuEoEDP8+T2Qzz9Zj+fuekC/uNNF/LRd5zDd57v5eXegUqXNv+kEmARiOT9iqmHLhKUAj3Pt36xm+XtjdxxTWaO/CeuX0tTXZRHnt5d4crmoXRycv8cpvTQRaTUFOhZvcdG+dnrfXxo/Upi0cx/S1tjHe+9bBmbXtjH8LhGlXOSTk4+qAjUQxcJTIGe9YOX9uMOH7xy8qoGH7yyi+F4ip++1lehyuapdHLyOi5wItDVQxcJQoGe9eT2Q5y/ZAGrOlom3X716oW0N9XxxHYdLzUnhVouGqGLBKVABwZGE/xyVz83XLTkpPti0QjXX9DJP79ykFTaK1DdPJVKTJ7hAnmBrh66SAgKdOAXO4+QSjvXnV94fZnrLjiboyMJtu8fLHNl81g6eXKg53rqKQW6SAgKdDKB3hCLcHn3WQXvf+s5HRPbSZHSycmH/UPeCF1roouEoEAHnnrjCOtXL6QhFi14/9L2RtYsbuGpNxToRUslZuiha4QuEkLNB/rASIJXDgzxljUdM273ljWL+OWuftLqoxcnnTh52qJaLiJB1XygP7/3KABXrVo443ZXrVrI0FiSN/qOl6Os+S9VoIce0YFFIiHVfKA/t+cYEYNfW1m4f55zZTbwn9tztBxlzX8FR+iahy4SUs0H+vN7jnL+klYWNMy88OQ5i1s4q7mO53YfK1Nl81zBaYsaoYuEVNOB7u68sPcYV3TP3G4BMDMuX3kWv+pRoBdlprVc1EMXCaKmA33XkRGGxpL8Wld7Udtf1nUWrx0cYiSulsGsUokC0xZzI3T9/4mEUNOB/mJ2tH1psYG+op20w7Z9OsBoVukC0xYneugaoYuEUNOB/lLPAA2xCOcvaS1q+1zwv9ij9dFnlSq02qJ66CIh1XSgv9g7wEXL2qiLFvffsKStkSVtDbykE17MLl1gp6h66CJB1Wygp9PO9n2DXLKibU7fd/HydrbuU6DPKhWH6JQTQSvQRYKq2UDfe3SEofEklywvrn+ec8nyNnYcOs5oXOuRzEgtF5Gyq9lAf7k3s2Pz4jkG+rrlmR2jrxzQjtEZFTywKDti1whdJIiaDfSt+waIRYzzly6Y0/flWjRbNdNlZqm45qGLlFnNBvq2/YOce/aCaVdYnM6Ks5poa4yxTWujz6xQy8UMLKqWi0ggtRvo+wZZt3xuO0Qhc8TouuVtmos+m1T85ECHTNslFS9/PSI1oCYDvW9onEND46xbNvdAB1i3rJ1XDgzqlHQzKXRgEWRCXi0XkSBqMtBzp5I7lRF67vvGEmnePDxcyrKqRzqdPWNR/cn3KdBFgqnJQM/1v099hJ7bMar56AXleuSxQoHeoJaLSCC1Gej7BllxVhNnNRcInCKce/YC6qLG9v1DJa6sSuQCWyN0kbKqzUDfP8hFy4pbv6WQ+liE885u1UyX6eQCu2Cga6eoSCg1F+hjiRQ7+46fcrslZ93ytolevEwxMULXLBeRcioq0M3sJjN71cx2mNm9Be7/dTN7zsySZvah0pdZOq8eGCLtcNFpBvpFy9qys2XGSlRZFVHLRaQiZg10M4sCDwA3A+uAO8xs3ZTN9gB3Ad8udYGltu00Z7jk5Eb46qMXoJaLSEUUM0K/Btjh7jvdPQ48CtyWv4G773L3F4F0gBpLatu+QVobYqxc2Hxaj5MLdB1gVEAusKcunwsKdJGAign0FcDevOs92dvmzMw2mNkWM9vS19d3Kg9x2jI7RNuIROy0Hqe9uY6uhU3aMVpIcjzzb6zh5PuidQp0kUDKulPU3R909/Xuvr6zs7OcTw1AKu1s339qh/wXsm5Zm+aiFzLRQy8Q6LGGE4EvIiVVTKD3Aivzrndlb5t3dh8ZZiSeOu0ZLjnrlrfx5uFhhsd10uNJJkbo6qGLlFMxgf4McJ6ZrTGzeuB2YFPYssJ4OdvvvmTF3NZAn84ly9txrY1+slQ20DVCFymrWQPd3ZPAPcDjwHbgMXffamb3m9mtAGZ2tZn1AL8F/JWZbQ1Z9Kna2jtAfTTCeUvmtgb6dHIfDLmTZUhWUvPQRSqhwDSEk7n7ZmDzlNvuy7v8DJlWzBnt5X0DXLC0teiTQs9mSVsDixfU87JOGj1ZLrAL7hRVoIuEUjNHiro7L/cOcnGJdohCZm30i5e3T7RyJGvGnaKNarmIBFIzgb63f5SB0QSXdpWmf55z6Yp2Xjs4xFhCJ42ekMwePVtohB6rV6CLBFIzgf5i7zEALltxVkkf99KudlJp13z0fBOzXBpPvi/WmNlp6jo5iEip1U6g92R2iF6w9NRXWSzksuyI/8W9x0r6uPPajCP07G0apYuUXA0F+jEuWtZKfay0L3lpWyOLFzTwYo92jE6YbYQOJ6Y2ikjJ1ESgp9LOSz0DXNZV2nYLZHaMXr6ynRd6NEKfkBwHrPC0RY3QRYKpiUB//dAQw/EUV64qfaADXNG9kJ19wxwb0XQ8AJKjmeC2Auvl5EboidHy1iRSA2oi0J/fkxk9X7FyYZDHv2Jl5oPiBfXRMxJjUNdU+L5coCe1jrxIqdVIoB9lYXMdqzpOb8nc6Vy28izMTnxw1LzkKMSmCfRc0GuELlJyNRHoW3Yd5cruhVihFkAJLGiIceHSNp7dfTTI4887yXGoK7BDFPJG6Oqhi5Ra1Qd639A4Ow8Pc/WaRUGf5+rVC3luz1GSqTP+HB/hJYoYoSc1QhcptaoP9Gd39wNw9erQgb6IkXhKBxhBpj8+2whdLReRkqv6QH/6zX4aYhEuWVG6NVwKyX1gPL2zP+jzzAvxEaibZn9FfUvm38RI+eoRqRFVH+g/33GEq1cvoiEWDfo8S9sbOWdxCz9/43DQ55kXEsPTB3qu5RJXoIuUWlUHet/QOK8eHOLaczvK8nzXntvBL9/sJ1HrffTEKNRPF+gtJ7YRkZKq6kDPjZavXbu4LM/39rWLGY6nNB99xpZL9vbEcPnqEakRVR3o//fVPhY213FpiU45N5trz11MNGL85NVDZXm+M1b8ONRPc1aoWCNYBOIKdJFSq9pAT6edn7zWxzvP7yQaCTP/fKr2pjquWrWQH7/SV5bnOyO5ZwK9YZpAN4P6Vhg/Xt66RGpA1Qb6Cz3H6B+Oc/2FZ5f1ea+/4Gy27R9k37Ea7REnxyGdPDGbpZD6FogPla8mkRpRtYH+g5f2Uxc1rrugvIH+GxcvAeDxrQfK+rxnjHh25F0/w7rzDQs0QhcJoCoD3d3Z/NIB3nHuYtqbCizhGtA5nQu4cGkrm1/aX9bnPWOMZdeFb5xh3n9DG4zrACyRUqvKQH9uz1F6j43y3suWV+T5b7l0GVt2H6XnaA3OtZ4I9Bl2RDe2n9hOREqmKgP977f00Fwf5eZLllbk+T9wxQrc4R+f7a3I81dUbuTdMMMIvbFNgS4SQNUF+tBYgu/9ah+3XLqMloZYRWpYuaiZt5/bwWNb9tbeYl0j2aUPmmZYe75pIYxqZUqRUqu6QH9sSw/D8RR3vm11Rev4yFtX03tslCe2H6xoHWWXC+rmGRZDa1qU2S5dYx92IoFVVaCPJ1N842c7uXr1Qi7tKs/BRNN597olrFzUxF/+5A3cvaK1lNXIkcy/TTMEenMHeBrGavyIWpESq6pA/9un97BvYIw/uOG8SpdCNGL8u+vP41c9A/xoWw2N0o8fhMazIFY//TYLslNJj9f4EbUiJVY1gX74+DhfeeJ1rl3bwTvOLc/aLbP5zStXsLazhT/9/nbGEqlKl1MeQwegdZad0bn7h2p0aqdIIJXZa1hi7s7nvvsSI/Ek9992cbBTzc1VLBrhT95/Cb/7taf54g9e4fO3XlzpksIb6IG2FTNvk7t/oCd8PXJGSHuawfFBBuIDDI4PMpwcZjQxSjwdJ5FOTLQlY5EYdZE6mmJNNNc1s6BuAW31bSxsXEh9dIa/+gSokkB/4Mc7eHzrQf7ovRdx7tkzHKFYAdeuXcy/eftq/ue/7OKSFe186KquSpcUjjv0vwld62ferr0LLApH3yxPXRLcwPgAPUM99BzvYd/xfewf3s/+4f0cGjlE30gf/WP9pPz0/kptrW+ls6mTzuZOljYvZdmCZSxvWc6KBSvoau1iSfMSopGw5z04083rQE+nnf/+zzv4yhOv8YErVvDRd6ypdEkFffbmi3jt4BCf+ccXSaTS3HFNd6VLCmNoP4wPwOILZt4uWgcda+HQ9vLUJact7WkOjx5mz+Ae9g7tnfjaM5S5PjRlbZ7W+laWtixlSfMSLlx0IR2NHSxqXER7QzvtDe00x5ppqmuiIdJALBIjYhEcJ+UpEqkEo8lRRhIjDCYGGRwf5OjYUY6MHeHw6GEOjhzkqf1P0TfSh3NiwkEsEqNrQRddrV10t3azsnUl3W3ddLV20bWgqyZG+EUFupndBPwFEAW+7u5fnHJ/A/At4CrgCPA77r6rtKVOtm3fIH/6/W38/I0j8ObT/MWXv8B3P7OCjRs38uEPf5hHHnmET33qUxw5cmTie1paWkilUoyNjZWkBjPj4x//OF/96ld55JFH+NznPseePXtYtCgzw6O/v5/u7m5uueUW/uWHTzJ65Yf57Hech594lgc2vIfVi2dYwGo+2vtLAN7zb+/lR9t+f8ZNv/WBJj50xX6a3DMrMEpFxVNx+kb7ODh8kIMjBydG2b3He9l3fB+9x3sZT41PbO8pJ344TnQwyjXnX8O71r9rIkyXL1hOa3Ytn9z7Yvfu3USjUVKpFKtWreKWW25h8+bN7Nmzh+7u7on37VwkUgkOjBygZ6iH3uO97B3aS89QD3uG9vD8oecZzltz3zDObj57YjS/rGUZy1qWTXzonN1yNq11rRPt2vz386nWVwk225Q6M4sCrwHvBnqAZ4A73H1b3jafAC5z94+b2e3AB9z9d2Z63PXr1/uWLVvmXPAT2w7y1Z/s4Lk9x2iMpOl74uv0P7Np4v7m5mbuvPNOvvGNbxCPx+f8+Kfihhtu4KmnnmJkZJZD/S1C21s+yFnX3k6krp63rV3Mv3/3+cFPYF0u33voy7z07fv44x+Pkpxlivn1q6NcuDjCOz71dX739/51eQqsQql0ikQ6QTwdJ56KM54aZzw5zmhqlNHEKCPJEYYTwxxPHOd4/DgD4wMMxAcYGB+gf6yf/rF+joweYTB+8to6bfVtrFiwguULlk+E9evPvM6X/9OXGewZhGwHpbm5mQcffPCkwHvkkUfYsGHD7O+LGR7jVLk7/WP9E39J5NpBPUM97Bvex6GRQ6R98i9pQ7SBjsYOfNjZ8dIO4gNxUsdTpIZTRJNR7rrjLm5854201LXQHGumMdZIY7SRhmgDDdEG6qP11EXriFks6H48M3vW3Qv2NYsJ9LcBn3f338he/yyAu38hb5vHs9s8ZWYx4ADQ6TM8+KkG+nee6+FrP3uT91++nC987F+xe8crJ22TGwmcqaItC+m67nbWXPfbfOE3L+Wt55TnFHmhrV69mt27d8/pe1atWsWuXbvCFDSPxVNxbvnOLbg7adKkPU3KU6TTaZKeJJnOfOW3HIoRsxhtDW0sbFhIe0M7HU0ddDR2sLhpMZ3NnSxpXsLSlqUsbVlKS93Jf0FO9zMu9HOc6+9DOX8XEukEfSN9HBg+wMGRgxwaOcTh0cMcHj3MP/3wn0jEEkRbo0Sbo0Sb5t6Xj1mMWCRGNBIlYhFilmkrRSyCmfHpKz/N+9a+75RqP91A/xBwk7t/LHv9I8Bb3P2evG1ezm7Tk73+Rnabw1MeawOwAaC7u/uqub75IfPJm/v0i0Qi8/agHTOb+NA5U2blnK5T+XmYGWkdMXqSZDrJ/U/dT8QyM4ujlg2GbL85FolNzAiJRWLUR+qpj9bTEG2YGDk21TXREmuhpS7z1VrfSlOs6bR+36b7GRf6Oc719+FM+V2YWrdFjUhzhFhzjOe3Zlo5I8kRRpOjjKfGGUuOZf5KSmVm7CTSiYkP3GQ6mfkgzn4guztpT/O+te/j6qVXn1J9MwV6WXeKuvuDwIOQGaGfymPk/zJ2d3cXHAGc6SN0yNReLUGeM93PY7bvkZPFIjHuf/v9lS7jJNP9jAv9HOf6+3Cm/C5MrdtTTmooRdeiLi7quKiClc2umAOLeoGVede7srcV3Cbbcmkns3M0qI0bN9LcPPlkxM3NzWzYsIH6+vLt0b7hhhtOqmMmzc3NbNy4MWBFlbFx40bq6ua2/nw1/j9Us+nec4V+joW2nc6Z9J6Yy2s847j7jF9kRvE7gTVAPfAr4OIp23wS+B/Zy7cDj832uFdddZWXwsMPP+yrVq1yM/NVq1b5ww8/PHF7R0eHAxNfLS0t3tjYOOm20/kyM7/77rtPqqOjo8M7Ojomarr77rsL1liNCv2/F/qKRqNV/f9QzaZ7z820be5nDsyL98RcXmO5AVt8mlydtYcOYGa3AP+NzLTFh9x9o5ndn33gTWbWCPwNcAXQD9zu7jtnesxT3SkqIlLLTruH7u6bgc1Tbrsv7/IY8FunU6SIiJyeqlmcS0Sk1inQRUSqhAJdRKRKKNBFRKpEUbNcgjyxWR8w90NFy2sxcHjWraqDXmv1qqXXWwuvdZW7dxa6o2KBPh+Y2ZbppgdVG73W6lVLr7eWXmsharmIiFQJBbqISJVQoM/swUoXUEZ6rdWrll5vLb3Wk6iHLiJSJTRCFxGpEgp0EZEqoUCfhZl93sx6zeyF7Nctla6p1MzsJjN71cx2mNm9la4nJDPbZWYvZX+WVbXcp5k9ZGaHsmcQy922yMx+ZGavZ/9dWMkaS2Wa11r179XZKNCL8xV3vzz7tXn2zeeP7EnAHwBuBtYBd5jZuspWFdz12Z9ltc1X/mvgpim33Qs86e7nAU9mr1eDv+bk1wpV/F4thgJdrgF2uPtOd48DjwK3VbgmOQXu/lMy5yPIdxvwzezlbwLvL2tRgUzzWmueAr0495jZi9k/86riT9Y8K4C9edd7srdVKwd+aGbPZk9aXu2WuPv+7OUDwJJKFlMG1fxenZUCHTCzJ8zs5QJftwF/CawFLgf2A/+1osXK6XqHu19JpsX0STP79UoXVC7Z05dV8zzlmn+vFnXGomrn7jcWs52ZfQ3434HLKbdiTgJeNdy9N/vvITP7LpmW008rW1VQB81smbvvN7NlwKFKFxSKux/MXa7S9+qsNEKfRfZNkPMB4OXptp2nngHOM7M1ZlZP5iTfmypcUxBm1mJmrbnLwHuovp/nVJuAO7OX7wT+VwVrCaoG3quz0p1flVoAAACKSURBVAh9dl8ys8vJ/Km6C/j9ypZTWu6eNLN7gMc5cRLwrRUuK5QlwHfNDDK/+9929/9T2ZJKx8z+FrgOWGxmPcAfA18EHjOzj5JZrvq3K1dh6UzzWq+r5vdqMXTov4hIlVDLRUSkSijQRUSqhAJdRKRKKNBFRKqEAl1EpEoo0EVEqoQCXUSkSvx/sKwrtTioc8AAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "np.random.seed(1)\n",
    "mus, sigmas, pis = EM(data, 3, 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Cluster using 2 gaussians."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration: 0. Means: [-1  5]. Stds: [1.72032449 1.00011437]\n",
      "Iteration: 1. Means: [-2.11596311  9.1084688 ]. Stds: [4.54227598 5.05099555]\n",
      "Iteration: 2. Means: [-2.49410353 11.56508425]. Stds: [3.5884482  4.86477928]\n",
      "Iteration: 3. Means: [-2.54128407 12.53117768]. Stds: [3.43413946 3.70046708]\n",
      "Iteration: 4. Means: [-2.51032551 13.39073407]. Stds: [3.44029307 2.17527563]\n",
      "Iteration: 5. Means: [-2.50413656 13.56682351]. Stds: [3.44171932 1.66321586]\n",
      "Iteration: 6. Means: [-2.50416281 13.56683945]. Stds: [3.44166033 1.6631331 ]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX8AAAD4CAYAAAAEhuazAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXyU5bn/8c+VnQQIkA1IQgKSAAEEIaBWZBFU0BZcq7ZVa7VWq8e29pzXwV/P0dYjPceeVnta7WJdaqUV3KWKUpEdFQjKDgkBAoQlG5CEJfv1+2MmNsZAJslknlmu9+uVV2aeuWeea2aSbyb3cz/3LaqKMcaY0BLmdAHGGGN8z8LfGGNCkIW/McaEIAt/Y4wJQRb+xhgTgiKcLqC1xMREzczMdLoMY4wJKBs3bixX1SRP2/td+GdmZpKXl+d0GcYYE1BEZH9H2lu3jzHGhCALf2OMCUEW/sYYE4Is/I0xJgRZ+BtjTAjyKPxFZKaI5ItIoYjMbeP2aBFZ6L59nYhktrjtfBH5WES2i8hWEYnxXvnGGGM6o93wF5Fw4GlgFpAD3CIiOa2a3QkcV9WhwJPA4+77RgDzgXtUdSQwFaj3WvXGGGM6xZNx/hOBQlXdCyAiC4A5wI4WbeYAP3Vffg14SkQEuALYoqqbAVS1wkt1Gz+lqizZfpQdh6s+39a7RyQ3TUinV0ykg5UZY1ryJPxTgYMtrhcDF56tjao2iEglkABkAyoiS4AkYIGq/qL1DkTkbuBugEGDBnX0ORg/kX+0moff3sa6fccAEHFtV4U/rtrLT64awZyxA5HmG4wxjunuM3wjgEnABOA08KGIbFTVD1s2UtVngGcAcnNzbXWZAFPf2MQv3t/F82uL6BUTwc+vHc1NE9IJD3OF/OaDJ3j47W38cOEm/rb+AL+8YQyDEmIdrtqY0ObJAd9DQHqL62nubW22cffzxwMVuP5LWKWq5ap6GlgMjOtq0cZ/qCqPLNrOn1bv4+u5aSz/8VS+ceGgz4MfYEx6H978/iX893Wj2XWkilufX8exU3UOVm2M8ST8NwBZIjJYRKKAm4FFrdosAm53X74BWKau9SGXAKNFJNb9R2EKXzxWYALcc2v28bd1B/j+1PP47+vOp29cVJvtwsKEWyYO4oU7JnKksoZ7XtpIbUOjj6s1xjRrN/xVtQG4H1eQ7wReUdXtIvKoiMx2N3sOSBCRQuBBYK77vseBJ3D9AdkEfKqq73r/aRgnfLCjhHmLd3LV6P786xXDPLrP+Iy+/PLGMawvOsZDr2/F1pA2xhke9fmr6mJcXTYttz3c4nINcONZ7jsf13BPE0R2HK7iBws+Y3RqPL+6cSxhYZ4fxJ09ZiD7yk7x5NICzkvuyX3ThnZjpcaYttgZvqbDGpuUf3ttMz2jI3j2tlx6RIV3+DEemD6U2WMG8sQHBRSUVHdDlcaYc7HwNx32at5Bth+u4j++mkNy786dsC0i/Gz2SHpGR/Bf7+yw7h9jfMzC33RI5Zl6/ndJPrkZffna+QO69Fh946L40YwsVu8u54MdJV6q0BjjCQt/0yG//XA3x07X8dPZI71ystY3L8ogK7knj72700b/GONDFv7GY3vKTvLnj4q4KTedUanxXnnMyPAwHvnaSA4cO83za4q88pjGmPZZ+BuPPfbODnpEhvNjD4d1empSViKX56Tw1LLdlFbVePWxjTFts/A3HtlaXMny/DLunXYeSb2ivf74P7lqBGfqG3lu7T6vP7Yx5sss/I1H/rR6Lz2jI/jWRRnd8viZiXHMGj2Av607wMnahm7ZhzHmnyz8TbsOnTjDu1uPcPOEdHp347TM3710CNU1DSzccLD9xsaYLrHwN+16YY2rK+aOSYO7dT9j0/swIbMvz6/ZR0NjU7fuy5hQZ+Fvzqmqpp4FGw5y9egBpPbp0e37++6lQzh04gzvbTva7fsyJpRZ+JtzWrDe1Qf/3UuH+GR/M0akMDgxjmdX77Wzfo3pRhb+5qzqG5t4YW0RFw3px+g074zrb09YmHDnpMFsLq5kQ9Fxn+zTmFBk4W/O6v1tRzlSWeOzT/3Nrh+XRt/YSJ5bs9en+zUmlFj4m7N6Je8gqX16MG1Ysk/32yMqnBvGp/HhzlIqTtb6dN/GhAoLf9OmwyfOsKawnOvHp3Vorn5vuTE3nYYm5a1Nh32+b2NCgYW/adMbnxajCjeMS3Nk/9kpvRiTFs+reQftwK8x3cDC33yJqvLaxmIuHNyPQQmxjtVxw/g0dh2tZvvhKsdqMCZYWfibL8nbf5yiitPcmJvuaB2zx6QSFRHGq3l2xq8x3mbhb77k1byDxEWFc9Xo/o7WER8byRU5Kby9+bDN9W+Ml1n4my84XdfAu1uOcNXoAcRGRThdDjfmpnPidD0f7ix1uhRjgoqFv/mC97Ye5VRdo+NdPs0mDU1kQHyMdf0Y42UW/uYLXv+0mIyEWCZk9nW6FADCw4TrxqWysqCM0mpb6MUYb7HwN58rP1nLJ3srmD1moFfW5/WWOWNTaVJYYpO9GeM1HoW/iMwUkXwRKRSRuW3cHi0iC923rxORTPf2TBE5IyKb3F9/8G75xpve33aUJoWrzx/gdClfkJ3Si6HJPXl36xGnSzEmaLQb/iISDjwNzAJygFtEJKdVszuB46o6FHgSeLzFbXtUdaz76x4v1W26weKtRxiSFMewlF5Ol/IlV40ewLp9x6zrxxgv8eST/0SgUFX3qmodsACY06rNHOBF9+XXgOniT/0Gpl3NXT5Xjx7gV10+zb56/gDUun6M8RpPwj8VaDnUoti9rc02qtoAVAIJ7tsGi8hnIrJSRC5tawcicreI5IlIXllZWYeegPEOf+3yaWZdP8Z4V3cf8D0CDFLVC4AHgb+JSO/WjVT1GVXNVdXcpKSkbi7JtMWfu3yaWdePMd7jSfgfAloO+k5zb2uzjYhEAPFAharWqmoFgKpuBPYA2V0t2niXv3f5NLOuH2O8x5Pw3wBkichgEYkCbgYWtWqzCLjdffkGYJmqqogkuQ8YIyJDgCzAVujwM/7e5dPMun6M8Z52w9/dh38/sATYCbyiqttF5FERme1u9hyQICKFuLp3moeDTga2iMgmXAeC71HVY95+EqZrAqHLp5l1/RjjHR5N3qKqi4HFrbY93OJyDXBjG/d7HXi9izWabtTc5XPftKF+3eXT7KvnD+A3H+5mybaj3HpxptPlGBOw7AzfELdsZylNCleOdHYGT09lp/RiSGIc/9hR4nQpxgQ0C/8Q98HOEgbGxzBy4JcGYfmtGTkpfLK3guqaeqdLMSZgWfiHsJr6RlbvLmNGTkpAdPk0mzEihfpGZVVBudOlGBOwLPxD2NrCcmrqm7g8J8XpUjpkfEZf+sZGsnSndf0Y01kW/iHsgx0l9IyO4MLBCe039iPhYcJlw1NYtquU+sYmp8sxJiBZ+IeopiZl6c5SpgxLIioi8H4MLs9JpvJMPXlFx50uxZiAFHi/9cYrNhefoPxkLZePCKwun2aXZiURFR5mXT/GdJKFf4haurOE8DBh6rDAnEspLjqCrwxNYOnOElTV6XKMCTgW/iFq6Y5SJmb2o09slNOldNrlOSnsrzhNYelJp0sxJuBY+IegAxWnyS+pZkaAjfJpbfpwV/12wpcxHWfhH4I+cPeTzxiR7HAlXdM/Pobz0+L5wMLfmA6z8A9By3eVMjS5JxkJcU6X0mXTh6ewufgEFSdrnS7FmIBi4R9iTtU2sG5fBZcND+xP/c2mDU9CFVbtthXgjOkIC/8Qs7awnPpGDdhRPq2NGhhPYs8oVuRb+BvTERb+IWZ5fhk9oyPIzejndCleERYmTMlOZmVBGY1NNuTTGE9Z+IcQVWVFfimThiYG5Fm9ZzNteBInTtez6eAJp0sxJmAETwKYduWXVHOksiZounyaXTo0iTCBFfmlTpdiTMCw8A8hy3e5+sWnDguOg73N4mMjGZ/Rl+UW/sZ4zMI/hCzPL2XEgN70j49xuhSvmzosmW2HqiitsrV9jfGEhX+IqKqpZ+P+40wLsi6fZtPc/82sKLBRP8Z4wsI/RKzZXU5jkzItSMb3tzZiQC9Sekdbv78xHrLwDxHLd5XSOyaCC9L7OF1KtxARpg1LZnVBuS3wYowHLPxDQFOTsqKgjEuzk4gID963fOqwZKprG9i43xZ4MaY9wZsE5nM7j1ZRVl37eb94sLpkaAIRYcIq6/c3pl0ehb+IzBSRfBEpFJG5bdweLSIL3bevE5HMVrcPEpGTIvKv3inbdMRKdxhOzkp0uJLu1SsmknEZfT9/vsaYs2s3/EUkHHgamAXkALeISE6rZncCx1V1KPAk8Hir258A3ut6uaYzVuaXMWJAb5J7B98Qz9amZCex/XAVpdU25NOYc/Hkk/9EoFBV96pqHbAAmNOqzRzgRffl14DpIiIAInINsA/Y7p2STUdUu4d4TskOziGerTU/z9UF5Q5XYox/8yT8U4GDLa4Xu7e12UZVG4BKIEFEegL/DvzsXDsQkbtFJE9E8srK7F92b/p4TwUNTRoy4Z8zoDeJPaNsimdj2tHdB3x/CjypqudcZFVVn1HVXFXNTUoKjZDylZUFZcRFhTM+o6/TpfhEWJgwOSuJVTbLpzHn5En4HwLSW1xPc29rs42IRADxQAVwIfALESkCfgj8PxG5v4s1Gw+pKisLyrj4vOCaxbM9U4Ylcfx0PdsOVTpdijF+y5NE2ABkichgEYkCbgYWtWqzCLjdffkGYJm6XKqqmaqaCfwa+LmqPuWl2k079pafovj4GaYE6ZQOZzNpaCIi2KgfY86h3fB39+HfDywBdgKvqOp2EXlURGa7mz2Hq4+/EHgQ+NJwUON7zePdp2SFVvgn9IxmdGq8jfc35hwiPGmkqouBxa22Pdzicg1wYzuP8dNO1Ge6YGVBGUMS4xiUEOt0KT43JTuJp5cXUnm6nvjYSKfLMcbvhE5HcIipqW/kk70VTA6RUT6tTclOoklh7R4b8mlMWyz8g9T6fceoqW8KmSGerY1N70OvmAhW2sLuxrTJwj9IrSooIyoijAuHBMdC7R0VER7GpKGJrNpdhqoN+TSmNQv/ILVqdxkTM/sRG+XRYZ2gNDk7iSOVNRSWnvM0E2NCkoV/EDp84gwFJSeZnB3cE7m1p/l4hw35NObLLPyD0Gr31AZTsoN7Cuf2pPbpwdDknhb+xrTBwj8IrSoop3/vGLJTejpdiuMmZyW5D343Ol2KMX7Fwj/INDQ2saawnEuzEnFPrBrSJmcnUtvQxLp9x5wuxRi/YuEfZDYXV1J5pj5kx/e3duHgBKIiwmzIpzGtWPgHmVUFZYSJa34bAz2iwrlwcD+b4tmYViz8g8yq3WWcn9aHvnFRTpfiN6ZkJ1FYepLDJ844XYoxfsPCP4icOF3H5oMnrMunlebXwyZ6M+afLPyDyJrCcpoUpoT4+P7WspJ70r93jA35NKYFC/8gsqqgjN4xEYxJ6+N0KX5FRJiSncSawnIaGpucLscYv2DhHyRUlVUF5UzKSiQi3N7W1iZnJ1Fd08CmgyecLsUYv2ApESTyS6o5WlXD1BA/q/dsJmUlEh4m1vVjjJuFf5BoHsduB3vbFt8jkgvS+1j4G+Nm4R8kVhaUMbx/L/rHxzhdit+akp3EluJKyk/WOl2KMY6z8A8Cp2ob2FB0LGQXbvFU80L2q+2EL2Ms/IPBx3sqqG9UC/92jBoYT0JclE31YAwW/kFhRUEpsVHhjM/s63Qpfi0sTJicncSq3eU0NdnqXia0WfgHOFVlRX4ZXzkvkeiIcKfL8XtTspM4dqqObYcrnS7FGEdZ+Ae4feWnKD5+5vP+bHNurqmuYYV1/ZgQZ+Ef4JqHLk7JsvD3RELPaM5PjbchnybkeRT+IjJTRPJFpFBE5rZxe7SILHTfvk5EMt3bJ4rIJvfXZhG51rvlm5UFZQxJjGNQQqzTpQSMKdlJfHbgOJWn650uxRjHtBv+IhIOPA3MAnKAW0Qkp1WzO4HjqjoUeBJ43L19G5CrqmOBmcAfRSTCW8WHupr6Rj7ZW2EndnXQlGFJNCmsLrRP/yZ0efLJfyJQqKp7VbUOWADMadVmDvCi+/JrwHQREVU9raoN7u0xgA2x8KJP9lZQU9/EVOvv75AxaX2I7xFp/f4mpHkS/qnAwRbXi93b2mzjDvtKIAFARC4Uke3AVuCeFn8MPicid4tInojklZXZL6Snlu8qJSYyjIuGJDhdSkCJCA9jSnYSK/JLbcinCVndfsBXVdep6khgAvCQiHxp/gFVfUZVc1U1NynJPsV6QlVZll/KJeclEhNpQzw7atrwJMpP2pBPE7o8Cf9DQHqL62nubW22cffpxwMVLRuo6k7gJDCqs8Waf9pTdoqDx84wbbjN4tkZk7OSEIFlu0qdLsUYR3gS/huALBEZLCJRwM3AolZtFgG3uy/fACxTVXXfJwJARDKA4UCRVyoPccvdoWXh3zkJPaMZm96H5dbvb0JUu+Hv7qO/H1gC7AReUdXtIvKoiMx2N3sOSBCRQuBBoHk46CRgs4hsAt4Evq+q5d5+EqFoeX4pw1J6kdqnh9OlBKxpw5LZUnzCZvk0IcmjYZequhhY3Grbwy0u1wA3tnG/l4CXulijaaW6pp71+45x56WDnS4loF02PJknPihgZX4Z149Pc7ocY3zKzvANQGt2l9PQpFw2zLp8uiJnQG+SekWzLN/6/U3osfAPQMvzS+kVE8H4DJvFsyvCwoRpw5JYVVBmC7ubkGPhH2CampTl+WVMzk6yhdq9YNqwZKprGti4/7jTpRjjU5YeAWbHkSrKqmuty8dLJmUlEhEmNurHhBwL/wCzbFcpItgUzl7SKyaSCZn9WLarxOlSjPEpC/8As3RnCWPT+5DYM9rpUoLG9BHJFJSc5EDFaadLMcZnLPwDyNHKGrYUVzJjRIrTpQSVy3Ncr+fSnfbp34QOC/8A0hxOV+RY+HtTRkIc2Sk9+WCHhb8JHRb+AWTpzhIyEmIZmtzT6VKCzowRKawvOmYLvJiQYeEfIE7VNvBRYQUzRqQgIk6XE3Rm5KTQ2KSsKLATvkxosPAPEKsKyqhrbLL+/m4yNs11EP0f1vVjQoSFf4D4YGcJ8T0imZBpZ/V2h7AwYcaIZFbml1HXYGf7muBn4R8AGhqbWL6rlMuGJ9tZvd1oxogUTtY2sG5fRfuNjQlwliQB4NMDJzh+ut66fLrZpKxEYiLDWGpdPyYEWPgHgA92HCUyXJicneh0KUEtJjKcS7OS+GBHCaq2tq8Jbhb+fk5V+WBHCRcNSaBXTKTT5QS9y0ekcLiyhu2Hq5wuxZhuZeHv5/JLqimqOM2VI/s7XUpImD4imTCBJduPOl2KMd3Kwt/PLd56FBEs/H0koWc0Fw1J4N2tR6zrxwQ1C38/9/62I0zM7EdSL5vIzVdmjerP3rJT7C496XQpxnQbC38/Vlh6koKSk8waZZ/6fenKkf0Rgfe2WtePCV4W/n7s/W1HAJg5aoDDlYSW5N4x5Gb05T33629MMLLw92OLtx5lfEZf+sfHOF1KyJk1agC7jlazt8y6fkxwsvD3U/srTrHjSJV1+Thkpvt1f2+bdf2Y4GTh76eaQ2emhb8jBvbpwdj0Ptb1Y4KWR+EvIjNFJF9ECkVkbhu3R4vIQvft60Qk0739chHZKCJb3d8v8275weu9bUcZkxZPWt9Yp0sJWVeN7s+2Q1UcPGbLO5rg0274i0g48DQwC8gBbhGRnFbN7gSOq+pQ4Engcff2cuBrqjoauB14yVuFB7NDJ86w+eAJO9DrsFnu198+/Ztg5Mkn/4lAoaruVdU6YAEwp1WbOcCL7suvAdNFRFT1M1U97N6+HeghIjZgvR3vbnG9ZFeNti4fJ6X3i2V0ajzvbLHwN8HHk/BPBQ62uF7s3tZmG1VtACqBhFZtrgc+VdXa1jsQkbtFJE9E8srKyjytPWi99dlhxqb3ISMhzulSQt6csQPZUlxpo35M0PHJAV8RGYmrK+h7bd2uqs+oaq6q5iYlJfmiJL+1u6SaHUeqmDN2oNOlGOBrYwYiAm9tOtx+Y2MCiCfhfwhIb3E9zb2tzTYiEgHEAxXu62nAm8BtqrqnqwUHu7c3HSZM4KvnW/j7g5TeMVw8JIFFmw7ZXD8mqHgS/huALBEZLCJRwM3AolZtFuE6oAtwA7BMVVVE+gDvAnNVda23ig5Wqsrbmw9xydBEm8vHj1wzNpWiitNsLq50uhRjvKbd8Hf34d8PLAF2Aq+o6nYReVREZrubPQckiEgh8CDQPBz0fmAo8LCIbHJ/JXv9WQSJTw+c4OCxM1wztvUhFeOkK0f1Jyo8jLc3tf6H15jAFeFJI1VdDCxute3hFpdrgBvbuN9jwGNdrDFkvL3pENERYVxpJ3b5lfgekVw2PJm/bz7CT64aYesom6BgP8V+or6xiXe2HGFGTgo9oz36m2x86JoLBlJ+spaP9tji7iY4WPj7iTWF5Rw7VWddPn5q6rBkesVE8LaN+jFBwsLfT7z12SHie0QyJTu0h7r6q5jIcGaN6s+S7Uc5U9fodDnGdJmFvx+oPF3P+9uO8rUxA4iKsLfEX10/Lo2TtQ0s3mpn/JrAZ0njB97efIjahiZunjDI6VLMOUwc3I/BiXEs3HCw/cbG+DkLf4epKi+vP8io1N6MSo13uhxzDiLC13PTWV90jD023YMJcBb+Dtt2qIqdR6q4KTe9/cbGcdePTyU8THglzz79m8Bm4e+whXkHiI4IY7aN8gkIyb1imD48mdc3FlPf2OR0OcZ0moW/g87UNfL2Z4e5evQA4ntEOl2O8dBNE9IpP1nHhztLnS7FmE6z8HfQ4q1HqK5t4KYJ1uUTSKZkJ5HSO9q6fkxAs/B30MINBxmcGMfEwf2cLsV0QER4GDeMT2NFfilHKs84XY4xnWLh75DdJdWsLzrG13PTERGnyzEd9PXcdJoUFqy3T/8mMFn4O+T5tUVER4RZl0+AykiIY9qwJP66bj+1DXbGrwk8Fv4OOHG6jjc/K+aasan0i4tyuhzTSXdcMpjyk3W8s9nO+DWBx8LfAS+vP0hNfRN3TMp0uhTTBZdmJTI0uSfPr91nq3yZgGPh72MNjU289HERXzkvgeH9eztdjukCEeGOSzLZfriKDUXHnS7HmA6x8PexJdtLOFxZwx2XDHa6FOMF112QRnyPSF5Yu8/pUozpEAt/H3th7T4G9YvlsuG2mmUw6BEVzs0T01my/SjFx087XY4xHrPw96EtxSfI23+cb38lk/AwG94ZLG67OBMR4S8f73e6FGM8ZuHvQ39cuZde0RHcmJvmdCnGi1L79GDmqP68vO4AlafrnS7HGI9Y+PtIQUk1i7cd4duXZNIrxubxCTb3TR1KdW0DL3xkff8mMFj4+8hTywqJjQznO3agNyjlDOzNFTkpPL9mH1U19unf51T/+WU8EuF0AaGgsPQkf99ymO9NPo++dlJX0Hpgehb/2FHCXz4q4v7LspwuJ/jVVsOnL0H+YijOg4YzEBEDA8fBsJkw/tsQYwsknY198veB3y0vJCYinLsutU/9wWxUajzThyfz7Jp9nKxtcLqc4KUKG56DJ0fCkoegphLG3QZTH4Lxd7j+CHzwMDwxEj75PTTZugtt8Sj8RWSmiOSLSKGIzG3j9mgRWei+fZ2IZLq3J4jIchE5KSJPebf0wFBUfoq3Nh3iWxcNIrFntNPlmG72L9OzOHG6nvmf2MifbnHmOMy/Dt59EAaMhbs+hHtWw1W/gKlzYdb/wN0r4O6VMOgieH8uvPg1OFXudOV+p93wF5Fw4GlgFpAD3CIiOa2a3QkcV9WhwJPA4+7tNcB/Av/qtYoDzFPLC4kMD+O7k4c4XYrxgbHpfZiSncSfVu3ldJ19+veq40Xw7OVQtAaufgJuexvScttuO3AsfPNVmPM0HMqDZ6dDeaFPy/V3nnzynwgUqupeVa0DFgBzWrWZA7zovvwaMF1ERFVPqeoaXH8EQk7+0Wre+LSYb16YQXKvGKfLMT7ygxlZVJyq40+rbOSP1xzfDy9cDafK4Na3YMKd0N5U6CJwwbfg9neg9iT8+Sr7A9CCJ+GfCrSctLzYva3NNqraAFQCCZ4WISJ3i0ieiOSVlZV5eje/pqo89u4OesVE8sD0oU6XY3xo3KC+XD16AH9YuYejlSH5uce7qo64um7qTsLtf4fMSzp2//QJ8O13oanR9TgnDnRPnQHGLw74quozqpqrqrlJSUlOl+MVKwrKWL27nAemZ9En1kb4hJq5s4bT2KT88h/5TpcS2OpOwcs3wekKuPUNGHB+5x4nebirm6juFPz1666DxCHOk/A/BLRccSTNva3NNiISAcQDFd4oMBDVNzYx792dDE6M49aLMpwuxzggvV8sd0zK5PVPi9l2yIKmU5qa4PXvwtGtcMMLkDq+a4/XfxTc9BJU7IZXv+36TyCEeRL+G4AsERksIlHAzcCiVm0WAbe7L98ALNMQnuB8wfoDFJae5KFZw4mK8It/rowD7ps2lL6xUfzXOztsvv/OWP1LyH8Xrvw5ZF/hncccMsV1sHjPMlj2X955zADVbjK5+/DvB5YAO4FXVHW7iDwqIrPdzZ4DEkSkEHgQ+Hw4qIgUAU8A3xaR4jZGCgWVyjP1PLl0NxcN6cflOSlOl2Mc1Dsmkh9dns26fcdYsr3E6XICS+FSWP5zOP8muPAe7z72+NtdJ4CteRJ2/t27jx1AxN8+keTm5mpeXp7TZXTa3Ne38EreQRbdP4lRqXZ2YahraGzi6t+soaqmniU/mkxvm9epfVWH4Q+ToGd/uGspRMV6fx8NtfD8TKjYA/esgr6Z3t+Hj4nIRlU9y9jXL7M+CS9as7ucBRsO8t3JQyz4DQAR4WH8z/WjKamq4b8X73K6HP/X1Aiv3wX1NXDjn7sn+AEiouHGF1yXX/sONNR1z378mIW/l5yqbWDuG1sYkhjHj2ZkO12O8SMXDOrLXZcO4eX1B/io0M40PafVv4L9a+HqX0FSN/8e9c2EOb+FQxth+bzu3ZcfsvD3kv9dks+hE2f4xQ3nExMZ7nQ5xs88eHk2gxPj+Pc3ttiZv2dzYK313mYAAA6gSURBVB2s+B9XP//YW3yzz5w5rv7/tf8He1f4Zp9+wsLfCzYUHePPHxVx+8WZ5Gb2c7oc44diIsN5/PrzOXjsDL9438b+f0lNJbxxF8SnwVW/9O2+r/w5JGbBm/fA6WO+3beDLPy76PipOn64YBNpfXvwb1cOc7oc48cmDu7H7Rdn8OePili2y0b/fMG7P4bKQ3D9sxDT27f7jopz7fdUOSz6l5BZE8DCvwuampQfvbKJsupanv7GOOKibXkEc24PXTWCnAG9+dHCzRw8Zgu+A7B5IWx91TUrZ/pEZ2oYMAZmPAK73oGNLzhTg49Z+HfBU8sLWZFfxsNfy2FMeh+nyzEBICYynD98azxNqtz7143U1If2WaZU7HFNzzzoYrj0x87WctF9MGQavP8QlO50thYfsPDvpFUFZTy5tIBrL0jlmxcOcrocE0AGJcTyxNfHsu1QFT/7+w6ny3FOQx28fieERcB1f4IwhwdKhIXBtX+EqJ7w2p1Qf8bZerqZhX8n7Ck7yQ8WfEZ2ci/mXTsKaW9qWWNauTwnhXunnsfL6w/wUqgu/LL0p3D4M5j9W+iT3m5zn+iV4voDULrd9R9AELPw76AjlWe47bn1hInwx1vHExtl/fymc358eTaXDU/m4be38e6WI06X41u73oVPnoaJ34Oc2e2396WsGXDJD119/1tfc7qabmPh3wHHT9Vx23PrqTxTz4vfmUhmYpzTJZkAFhEextPfGEduRl9+uPAzVu8OjrUs2lWxB96817UM4xV+OrnaZf8B6RfBogegNDjPzLbw99Cp2gbu+PMG9h87zZ9uy7XpG4xX9IgK59nbJ3BeUk++99JGNh084XRJ3avuFCy81dW//vW/uKZZ8Efhka7pH6JiYeE3oabK6Yq8zsLfA8dP1fGt59axpfgEv73lAi4+z+NFyoxpV3yPSP7ynYkk9ozm1mfX8fGeIF0Ko6nJdSJV6Q7XuPq+fr7WRe+BcOOLcGyfa76hIJv/38K/HcXHT3P9Hz5i++EqfvfN8Vw5sr/TJZkglNw7hgV3X0T/+Bhuf349i7cG4TGA5fNg5yK44jEYOsPpajyTeQlc9b+wewn84z+drsarLPzPYdfRKq7//UeUV9cy/84LmTnKgt90n4F9evDqPRdzflo89/3tU178qMjpkrwn73nX4izjboOL73O6mo6ZcKdrTYFPnoaPf+d0NV5j4X8Wr28s5rrffYQgvHrPV5g42ObsMd2vT2wU8++6kOnDU3hk0Xb+7dXNgT8R3Pa3XNM3ZF3pWkUrEIdGX/lzGDEbljwEmxc4XY1XWPi3cqq2gQdf2cSPX93MqNR43rrvEob17+V0WSaEuM4CHse/XDaU1z4tZvZTa9l1NEAPOO78u+tErrQJrvn5wwN0MZuwcNeJaIMnw1v3wpZXna6oyyz8W1i3t4KvPbWGNz87xA+mZ/Hyd119sMb4WkR4GD++Yhjz77yQyjP1zHlqLc+u3kt9Y5PTpXlu62uuhdIHXgDffK37FmbxlcgYuGUBZFwCb94Nn77kdEVdYuEPlFTV8MMFn3HTM59QW9/E3+66iB9dnk14WAD+e2qCyiVDE1n8wKVcMjSRx97dyVd/s8b/RwOpwtrfuD7xp18E33rD9zN1dpeoOPjGQhgyFRbdDyt/EbCzgIb0Gr6VZ+r5y0dF/GHlHuoble9NGcL3pw6lR5QtxmL8i6rywY4SHn1nB8XHz3D16AE8MD3L/7ok607D338AW1+BnGtcUyVEBuF/zw11rumftyxwHQu45ncQ7ex70dE1fEMy/Muqa3l+7T5e+ng/J2sbmDEihf+4eoSdsWv8Xk19I79fsYc/rd7L6bpGLs9J4ftTz+OCQX2dLg0OfQpvfg/Kd8O0n7hm6QwL4s4FVfjot7D0EeiTAdc949yU1Fj4n1VDYxOrdpfx2sZilu4opb6piatHD+DeqecxcqCdrWsCy/FTdbz4cREvrC2i8kw9o1PjuWF8GrPHDKRvXJRvi6mpgpWPwye/h54pcM3TcN5lvq3BSUVrXSevVRVD7p1w2U+gh+//GFv4t3CmrpGP9pSzbFcp/9hRQll1Lf3iopgzdiC3XpTBkKSeXtmPMU45WdvAq3kHeTWvmB1HqogKD2NydhKXDU/msuHJ3TtgobbaNX5/7f+5lj8cdxtc/jNHgs9xNVXw4aOQ9xxE94av3A8T74YY332wDOnwr66pZ9PBE2zcf5yN+4+zft8xahuaiI0KZ3JWEteOS2XasGSiIoL4X1ETsrYfruT1jYdYsv0oh0645qIfMaA3EzP7Mi6jL+Mz+pLap0fXpiBXdU3DvPll2PQy1FXDedNdE6GljvPSMwlgR7fCsseg4H2IjIMxN8GYb0Babref39DR8EdV2/0CZgL5QCEwt43bo4GF7tvXAZktbnvIvT0fuLK9fY0fP147Y93eCs2c+45m/Ps7mjn3Hb3yyZX6yNvbdGV+qdbUN3TqMf3R/PnzNSMjQ0VEMzIydP78+Tp//nxNSEhQwKtf99577+f7AzQ8PFwBzcjI0HvvvfdLdQQST14zEQm456Wq2tTUpLuOVOnvlhfqLc98rCP+8z3N+HfX78ZPF23r6IOpnihW3fq66t9/pPrkKNVHeqs+mqj62l2qBzd0ud62fqbP1q7lexYXF6cJCQmO/gyetfZDn6m+cY/qo0mu1+tXOaqLHlDd8qrq8QOu19XLgDz1IM+bvzwJ/nBgDzAEiAI2Azmt2nwf+IP78s3AQvflHHf7aGCw+3HCz7W/zoZ/5Zk6/fUHBbq6oEyrztR16jH83fz58zU2NvYLARUZGakRERFeD/7mL08fOzY2NmCCcv78+RoZGenxaxAoz+ts6hsadWvxCX3xo336UWF5243KC1UL/qH66XzVlb9Qfft+1eevUn18sCu8Humt+tgA1b/drLrxRdXTx7xSW1s/0239LHnynvn6Z9Cj2s+cUP3sr6ovf0N1Xuo/X8v/yVB9fpbqW/eprnhc9dOXVPOXqJYVdLqejoZ/u90+InIx8FNVvdJ9/SEAVf3vFm2WuNt8LCIRwFEgCZjbsm3Ldmfbny+HegaazMxM9u/331WfMjIyKCoqcrqMdnX0dQyU59Ulb30fNv31n9djEyHhPEjMgv7nu7ot+p/v9TN0z/ZetH7NPX3PfPleeVr75xoboGQrFOfB0S1QVgDH9sKp0n+2GXmt60zoTuhot48ny1ClAgdbXC8GLjxbG1VtEJFKIMG9/ZNW901to+i7gbsBBg2y9XDP5sCBA06XcE7+Xl+zjtYZKM+rS77yAIz/NsQlQq8BENnDJ7s922vberun74Ev3ytPa/9ceITrbOeBF3xxe30NVB+BU+U+PQvaL458quozqpqrqrlJSUlOl+O3/P0Po7/X16yjdQbK8+qS5OGuMer9hvgs+OHsr23r7Z6+B758rzytvV2RMdBvMKRPgJSRXqjMM56E/yGg5erKae5tbbZxd/vEAxUe3td4aN68ecTGfvGTQWRkJBER3beOsKePHRsby7x587qtDm+aN28ekZGed18EyvMKRG39TLf1s+TJe+brn0FPa/db7R0UwNU1tBfXAdvmA74jW7W5jy8e8H3FfXkkXzzgu5duOuAbKmy0j3cE82ifQBOUo30cgLcP+AKIyFXAr3GN/HleVeeJyKPunS0SkRjgJeAC4Bhws6rudd/3J8B3gAbgh6r63rn2ZQd8jTGm40L6JC9jjAlVHQ1/vzjga4wxxrcs/I0xJgRZ+BtjTAiy8DfGmBDkdwd8RaQM8N85DL4oESh3uoguCPT6IfCfQ6DXD4H/HAK9fnA9hzhV9fgsWb8L/0AiInkdObrubwK9fgj85xDo9UPgP4dArx869xys28cYY0KQhb8xxoQgC/+uecbpAroo0OuHwH8OgV4/BP5zCPT6oRPPwfr8jTEmBNknf2OMCUEW/sYYE4Is/DtBRG4Uke0i0iQiua1ue0hECkUkX0SudKpGT4nIT0XkkIhscn9d5XRNnhCRme7XuFBE5jpdT2eISJGIbHW/7gExm6GIPC8ipSKyrcW2fiLygYjsdn/v62SN53KW+gPmd0BE0kVkuYjscGfQD9zbO/weWPh3zjbgOmBVy40ikoNrPYORwEzgdyIS7vvyOuxJVR3r/lrsdDHtcb+mTwOzgBzgFvdrH4imuV/3QBln/mdcP9stzQU+VNUs4EP3dX/1Z75cPwTO70AD8GNVzQEuAu5z/+x3+D2w8O8EVd2pqvlt3DQHWKCqtaq6DygEJvq2upAwEShU1b2qWgcswPXam26mqqtwrdnR0hzgRfflF4FrfFpUB5yl/oChqkdU9VP35WpgJ6510Tv8Hlj4e1dbi91/acF6P3S/iGxx/0vst/+ytxCor3NrCvxDRDaKyN1OF9MFKap6xH35KJDiZDGdFGi/A4hIJq4FtNbRiffAwv8sRGSpiGxr4yvgPmG281x+D5wHjAWOAL9ytNjQMklVx+HqvrpPRCY7XVBXuZcTDLTx4wH3OyAiPYHXca2OWNXyNk/fg+5b+TvAqeqMTtzNLxes9/S5iMifgHe6uRxv8MvXuaNU9ZD7e6mIvImrO2vVue/ll0pEZICqHhGRAUCp0wV1hKqWNF8OhN8BEYnEFfx/VdU33Js7/B7YJ3/vWgTcLCLRIjIYyALWO1zTObl/UJpdi+tgtr/bAGSJyGARicJ1kH2RwzV1iIjEiUiv5svAFQTGa9+WRcDt7su3A287WEuHBdLvgIgI8BywU1WfaHFTh98DO8O3E0TkWuC3QBJwAtikqle6b+vQgvVOE5GXcP27q0AR8L0WfYd+yz0c79dAOPC8qs5zuKQOEZEhwJvuqxHA3wLhOYjIy8BUXFMIlwCPAG8BrwCDcE3H/nVV9cuDqmepfyoB8jsgIpOA1cBWoMm9+f/h6vfv0Htg4W+MMSHIun2MMSYEWfgbY0wIsvA3xpgQZOFvjDEhyMLfGGNCkIW/McaEIAt/Y4wJQf8fqfV3v5oIN5YAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "np.random.seed(1)\n",
    "mus, sigmas, pis = EM(data, 2, 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As a last example, let's create an unbalanced dataset of 5 groups, whose centers of mass are not too separated."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iteration: 0. Means: [ 0 -4  1  1 10]. Stds: [1.3013605  1.77273915 1.15292982 1.57863393 1.009008  ]\n",
      "Iteration: 1. Means: [ 0.63949834 -4.74743948  1.9755868   3.63147449 11.76728246]. Stds: [1.99556008 0.98690515 0.9662139  2.26759397 2.70258213]\n",
      "Iteration: 2. Means: [ 0.3399088  -4.77766925  1.97193828  4.09558833 12.00616538]. Stds: [2.20943897 0.91822565 0.36918494 2.36516684 2.60966006]\n",
      "Iteration: 3. Means: [-0.58500569 -4.81424257  1.96985811  4.81864944 12.29858313]. Stds: [2.30067677 0.8730495  0.15324352 2.32813283 2.34599285]\n",
      "Iteration: 4. Means: [-1.55519465 -4.84805389  1.97133889  5.56556963 12.60669537]. Stds: [1.81317922 0.83194146 0.14969215 1.86652725 2.00199439]\n",
      "Iteration: 5. Means: [-2.02426315 -4.89665845  1.97036822  5.98423605 12.85677228]. Stds: [1.28360143 0.77518191 0.15302735 1.29172089 1.64517194]\n",
      "Iteration: 6. Means: [-2.17629966 -4.95008724  1.96998797  6.10693648 12.93535896]. Stds: [1.03266447 0.7129915  0.15506709 1.06434822 1.51012356]\n",
      "Iteration: 7. Means: [-2.2243756  -4.98477944  1.97000338  6.10799825 12.93838887]. Stds: [0.96353911 0.67093312 0.1552354  1.03306244 1.49992164]\n",
      "Iteration: 8. Means: [-2.27245553 -5.0047493   1.97000276  6.10529293 12.93584715]. Stds: [0.96166065 0.64807445 0.15524231 1.02840244 1.50256154]\n",
      "Iteration: 9. Means: [-2.3213795  -5.01846969  1.97000281  6.1041628  12.93449587]. Stds: [0.97810686 0.63395697 0.15524296 1.0270036  1.50423681]\n",
      "Iteration: 10. Means: [-2.36701513 -5.02922633  1.97000321  6.10368389 12.93389843]. Stds: [0.9981861  0.62367528 0.15524289 1.02644617 1.5049934 ]\n",
      "Iteration: 11. Means: [-2.40845111 -5.0381367   1.97000372  6.10347872 12.93364021]. Stds: [1.01796711 0.61553603 0.15524265 1.02621026 1.50532166]\n",
      "Iteration: 12. Means: [-2.44585547 -5.04568707  1.97000428  6.10339052 12.93352894]. Stds: [1.03648033 0.60885238 0.15524232 1.02610916 1.50546324]\n",
      "Iteration: 13. Means: [-2.47965502 -5.05216077  1.97000487  6.10335256 12.93348101]. Stds: [1.05356832 0.6032562  0.15524196 1.0260657  1.50552425]\n",
      "Iteration: 14. Means: [-2.5103043  -5.05775726  1.97000547  6.10333622 12.93346037]. Stds: [1.06930285 0.59850497 0.15524159 1.02604699 1.50555053]\n",
      "Iteration: 15. Means: [-2.53822493 -5.06262988  1.97000607  6.10332918 12.93345147]. Stds: [1.08381782 0.59442173 0.1552412  1.02603894 1.50556185]\n",
      "Iteration: 16. Means: [-2.56379239 -5.0669008   1.97000668  6.10332615 12.93344764]. Stds: [1.09725786 0.59087173 0.15524082 1.02603547 1.50556673]\n",
      "Iteration: 17. Means: [-2.58733645 -5.07066894  1.97000728  6.10332484 12.93344599]. Stds: [1.10976157 0.58775025 0.15524042 1.02603397 1.50556883]\n",
      "Iteration: 18. Means: [-2.60914544 -5.07401508  1.97000788  6.10332428 12.93344528]. Stds: [1.12145627 0.58497498 0.15524003 1.02603333 1.50556974]\n",
      "Iteration: 19. Means: [-2.62947147 -5.07700556  1.97000848  6.10332404 12.93344497]. Stds: [1.13245699 0.58248052 0.15523964 1.02603305 1.50557013]\n",
      "Iteration: 20. Means: [-2.64853553 -5.07969519  1.97000907  6.10332394 12.93344484]. Stds: [1.14286706 0.58021435 0.15523925 1.02603293 1.50557029]\n",
      "Iteration: 21. Means: [-2.66653234 -5.08212944  1.97000966  6.10332389 12.93344479]. Stds: [1.15277938 0.57813382 0.15523886 1.02603288 1.50557037]\n",
      "Iteration: 22. Means: [-2.68363474 -5.0843462   1.97001026  6.10332387 12.93344476]. Stds: [1.16227799 0.57620381 0.15523846 1.02603285 1.5055704 ]\n",
      "Iteration: 23. Means: [-2.6999976  -5.0863772   1.97001085  6.10332387 12.93344475]. Stds: [1.1714396  0.57439498 0.15523806 1.02603284 1.50557041]\n",
      "Iteration: 24. Means: [-2.71576146 -5.08824911  1.97001145  6.10332386 12.93344475]. Stds: [1.18033514 0.5726824  0.15523766 1.02603284 1.50557042]\n",
      "Iteration: 25. Means: [-2.73105573 -5.08998442  1.97001205  6.10332386 12.93344474]. Stds: [1.18903125 0.57104438 0.15523725 1.02603283 1.50557042]\n",
      "Iteration: 26. Means: [-2.74600177 -5.09160217  1.97001267  6.10332387 12.93344474]. Stds: [1.19759176 0.56946168 0.15523683 1.02603283 1.50557042]\n",
      "Iteration: 27. Means: [-2.76071577 -5.0931185   1.9700133   6.10332387 12.93344474]. Stds: [1.2060791  0.56791663 0.1552364  1.02603283 1.50557042]\n",
      "Iteration: 28. Means: [-2.77531161 -5.09454707  1.97001394  6.10332387 12.93344474]. Stds: [1.21455578 0.56639255 0.15523596 1.02603282 1.50557042]\n",
      "Iteration: 29. Means: [-2.7899038  -5.09589943  1.97001461  6.10332387 12.93344474]. Stds: [1.22308601 0.56487308 0.1552355  1.02603282 1.50557042]\n",
      "Iteration: 30. Means: [-2.80461067 -5.09718522  1.97001531  6.10332388 12.93344474]. Stds: [1.23173742 0.56334152 0.15523502 1.02603282 1.50557042]\n",
      "Iteration: 31. Means: [-2.81955794 -5.09841232  1.97001604  6.10332388 12.93344474]. Stds: [1.24058305 0.56178016 0.15523451 1.02603282 1.50557042]\n",
      "Iteration: 32. Means: [-2.83488288 -5.09958688  1.97001681  6.10332389 12.93344474]. Stds: [1.24970372 0.56016941 0.15523397 1.02603281 1.50557042]\n",
      "Iteration: 33. Means: [-2.85073941 -5.10071323  1.97001764  6.10332389 12.93344474]. Stds: [1.25919091 0.55848684 0.15523339 1.02603281 1.50557042]\n",
      "Iteration: 34. Means: [-2.86730442 -5.10179366  1.97001853  6.1033239  12.93344474]. Stds: [1.26915029 0.55670578 0.15523277 1.0260328  1.50557043]\n",
      "Iteration: 35. Means: [-2.88478593 -5.10282789  1.97001949  6.10332391 12.93344474]. Stds: [1.27970618 0.55479355 0.15523208 1.02603279 1.50557043]\n",
      "Iteration: 36. Means: [-2.90343358 -5.10381239  1.97002055  6.10332392 12.93344474]. Stds: [1.29100706 0.55270893 0.15523133 1.02603278 1.50557043]\n",
      "Iteration: 37. Means: [-2.9235525  -5.1047391   1.97002172  6.10332393 12.93344474]. Stds: [1.30323246 0.55039849 0.15523049 1.02603277 1.50557043]\n",
      "Iteration: 38. Means: [-2.94552141 -5.10559357  1.97002304  6.10332395 12.93344474]. Stds: [1.31660114 0.54779132 0.15522954 1.02603276 1.50557043]\n",
      "Iteration: 39. Means: [-2.96981597 -5.10635212  1.97002454  6.10332398 12.93344473]. Stds: [1.33138015 0.54479117 0.15522845 1.02603274 1.50557043]\n",
      "Iteration: 40. Means: [-2.9970376  -5.10697768  1.97002626  6.10332401 12.93344473]. Stds: [1.34789264 0.54126484 0.15522719 1.02603272 1.50557043]\n",
      "Iteration: 41. Means: [-3.02794525 -5.10741409  1.97002825  6.10332405 12.93344473]. Stds: [1.36651913 0.53702497 0.15522571 1.02603268 1.50557044]\n",
      "Iteration: 42. Means: [-3.06347951 -5.10757941  1.9700306   6.10332411 12.93344473]. Stds: [1.38767873 0.531806   0.15522396 1.02603264 1.50557044]\n",
      "Iteration: 43. Means: [-3.10474862 -5.10736166  1.97003337  6.1033242  12.93344472]. Stds: [1.41176038 0.52523405 0.15522186 1.02603257 1.50557044]\n",
      "Iteration: 44. Means: [-3.15290042 -5.10662889  1.97003663  6.10332434 12.93344472]. Stds: [1.43894497 0.51680256 0.15521936 1.02603248 1.50557045]\n",
      "Iteration: 45. Means: [-3.20872742 -5.10528421  1.97004041  6.10332453 12.93344471]. Stds: [1.46882924 0.50589661 0.15521642 1.02603234 1.50557046]\n",
      "Iteration: 46. Means: [-3.27180901 -5.10342127  1.97004459  6.10332481 12.9334447 ]. Stds: [1.49981635 0.49197239 0.1552131  1.02603215 1.50557047]\n",
      "Iteration: 47. Means: [-3.33931051 -5.10159445  1.97004878  6.10332519 12.93344469]. Stds: [1.5286088  0.47502937 0.15520969 1.0260319  1.50557049]\n",
      "Iteration: 48. Means: [-3.40567869 -5.10093486  1.97005221  6.10332559 12.93344467]. Stds: [1.55088444 0.45620319 0.15520683 1.02603162 1.50557051]\n",
      "Iteration: 49. Means: [-3.4650215  -5.10253533  1.97005394  6.10332589 12.93344464]. Stds: [1.56388279 0.43758218 0.15520531 1.02603141 1.50557054]\n",
      "Iteration: 50. Means: [-3.5145279  -5.10642076  1.97005353  6.10332596 12.93344462]. Stds: [1.56850689 0.42089118 0.15520552 1.02603134 1.50557056]\n",
      "Iteration: 51. Means: [-3.55471332 -5.1115822   1.97005147  6.10332583 12.93344461]. Stds: [1.56796949 0.40670004 0.15520708 1.02603141 1.50557057]\n",
      "Iteration: 52. Means: [-3.58731875 -5.1169189   1.97004871  6.10332562 12.93344461]. Stds: [1.56518661 0.39487735 0.15520923 1.02603155 1.50557057]\n",
      "Iteration: 53. Means: [-3.61395984 -5.12176089  1.97004595  6.10332541 12.93344462]. Stds: [1.56184652 0.38512939 0.15521139 1.02603169 1.50557056]\n",
      "Iteration: 54. Means: [-3.63583601 -5.12584365  1.97004352  6.10332524 12.93344463]. Stds: [1.55870047 0.37716798 0.15521328 1.02603181 1.50557055]\n",
      "Iteration: 55. Means: [-3.65381225 -5.12914217  1.97004148  6.10332511 12.93344464]. Stds: [1.55600364 0.37072497 0.15521486 1.02603191 1.50557054]\n",
      "Iteration: 56. Means: [-3.66854252 -5.13174155  1.97003984  6.103325   12.93344465]. Stds: [1.55379104 0.36555138 0.15521613 1.02603199 1.50557053]\n",
      "Iteration: 57. Means: [-3.68055494 -5.13376202  1.97003852  6.10332493 12.93344466]. Stds: [1.55201441 0.36142214 0.15521714 1.02603205 1.50557052]\n",
      "Iteration: 58. Means: [-3.69029677 -5.13532218  1.97003748  6.10332487 12.93344467]. Stds: [1.55060287 0.35814094 0.15521794 1.02603209 1.50557051]\n",
      "Iteration: 59. Means: [-3.69815442 -5.13652432  1.97003665  6.10332482 12.93344467]. Stds: [1.54948705 0.35554174 0.15521858 1.02603213 1.50557051]\n",
      "Iteration: 60. Means: [-3.70446141 -5.13745098  1.970036    6.10332479 12.93344468]. Stds: [1.54860705 0.35348727 0.15521908 1.02603215 1.5055705 ]\n",
      "Iteration: 61. Means: [-3.70950268 -5.13816657  1.97003549  6.10332476 12.93344468]. Stds: [1.54791375 0.35186588 0.15521947 1.02603217 1.5055705 ]\n",
      "Iteration: 62. Means: [-3.71351834 -5.13872044  1.97003508  6.10332474 12.93344468]. Stds: [1.54736781 0.35058767 0.15521978 1.02603219 1.50557049]\n",
      "Iteration: 63. Means: [-3.71670809 -5.13915018  1.97003476  6.10332472 12.93344468]. Stds: [1.54693802 0.3495808  0.15522002 1.0260322  1.50557049]\n",
      "Iteration: 64. Means: [-3.71923609 -5.13948435  1.97003451  6.10332471 12.93344468]. Stds: [1.54659973 0.34878811 0.15522021 1.02603221 1.50557049]\n",
      "Iteration: 65. Means: [-3.72123606 -5.13974473  1.97003431  6.1033247  12.93344468]. Stds: [1.54633352 0.34816431 0.15522036 1.02603222 1.50557049]\n",
      "Iteration: 66. Means: [-3.72281604 -5.13994795  1.97003416  6.10332469 12.93344469]. Stds: [1.54612405 0.34767358 0.15522048 1.02603222 1.50557049]\n",
      "Iteration: 67. Means: [-3.72406285 -5.14010678  1.97003403  6.10332469 12.93344469]. Stds: [1.54595927 0.34728761 0.15522058 1.02603223 1.50557049]\n",
      "Iteration: 68. Means: [-3.72504587 -5.14023106  1.97003394  6.10332468 12.93344469]. Stds: [1.54582966 0.34698409 0.15522065 1.02603223 1.50557049]\n",
      "Iteration: 69. Means: [-3.72582037 -5.14032838  1.97003386  6.10332468 12.93344469]. Stds: [1.54572772 0.34674545 0.15522071 1.02603224 1.50557049]\n",
      "Iteration: 70. Means: [-3.72643026 -5.14040466  1.9700338   6.10332468 12.93344469]. Stds: [1.54564757 0.34655783 0.15522075 1.02603224 1.50557049]\n",
      "Iteration: 71. Means: [-3.72691032 -5.14046448  1.97003376  6.10332467 12.93344469]. Stds: [1.54558454 0.34641034 0.15522079 1.02603224 1.50557049]\n",
      "Iteration: 72. Means: [-3.72728806 -5.14051141  1.97003372  6.10332467 12.93344469]. Stds: [1.54553499 0.34629441 0.15522081 1.02603224 1.50557049]\n",
      "Iteration: 73. Means: [-3.72758521 -5.14054824  1.97003369  6.10332467 12.93344469]. Stds: [1.54549604 0.34620328 0.15522084 1.02603224 1.50557049]\n",
      "Iteration: 74. Means: [-3.72781892 -5.14057716  1.97003367  6.10332467 12.93344469]. Stds: [1.54546542 0.34613165 0.15522085 1.02603224 1.50557049]\n",
      "Iteration: 75. Means: [-3.7280027  -5.14059986  1.97003365  6.10332467 12.93344469]. Stds: [1.54544135 0.34607535 0.15522087 1.02603224 1.50557049]\n",
      "Iteration: 76. Means: [-3.72814719 -5.14061769  1.97003364  6.10332467 12.93344469]. Stds: [1.54542243 0.34603111 0.15522088 1.02603224 1.50557049]\n",
      "Iteration: 77. Means: [-3.72826079 -5.1406317   1.97003363  6.10332467 12.93344469]. Stds: [1.54540756 0.34599633 0.15522089 1.02603224 1.50557049]\n",
      "Iteration: 78. Means: [-3.7283501  -5.1406427   1.97003362  6.10332467 12.93344469]. Stds: [1.54539587 0.345969   0.15522089 1.02603224 1.50557049]\n",
      "Iteration: 79. Means: [-3.7284203  -5.14065135  1.97003361  6.10332467 12.93344469]. Stds: [1.54538669 0.34594752 0.1552209  1.02603225 1.50557049]\n",
      "Iteration: 80. Means: [-3.72847548 -5.14065814  1.9700336   6.10332467 12.93344469]. Stds: [1.54537947 0.34593064 0.1552209  1.02603225 1.50557049]\n",
      "Iteration: 81. Means: [-3.72851885 -5.14066348  1.9700336   6.10332467 12.93344469]. Stds: [1.54537379 0.34591737 0.15522091 1.02603225 1.50557049]\n",
      "Iteration: 82. Means: [-3.72855294 -5.14066767  1.9700336   6.10332467 12.93344469]. Stds: [1.54536933 0.34590694 0.15522091 1.02603225 1.50557049]\n",
      "Iteration: 83. Means: [-3.72857973 -5.14067097  1.97003359  6.10332466 12.93344469]. Stds: [1.54536583 0.34589875 0.15522091 1.02603225 1.50557049]\n",
      "Iteration: 84. Means: [-3.72860079 -5.14067356  1.97003359  6.10332466 12.93344469]. Stds: [1.54536307 0.34589231 0.15522091 1.02603225 1.50557049]\n",
      "Iteration: 85. Means: [-3.72861734 -5.14067559  1.97003359  6.10332466 12.93344469]. Stds: [1.54536091 0.34588724 0.15522091 1.02603225 1.50557049]\n",
      "Iteration: 86. Means: [-3.72863034 -5.14067719  1.97003359  6.10332466 12.93344469]. Stds: [1.54535921 0.34588327 0.15522091 1.02603225 1.50557049]\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+j8jraAAAgAElEQVR4nO3deXhb5Zn38e8tyfIex1v2xAkQoGELxISmTKHsyxRCW9qGUqAD00AKLZ2WTmFogabDwMDVTjtv2QJNSxtoWqCFTCcMZS9bIGENCdlXZ3USJ453Lff7h3Qc2ZGsI1u2LOX+XJcvSWexHh1ZPz2+z3POEVXFGGNM7vJkugHGGGP6lwW9McbkOAt6Y4zJcRb0xhiT4yzojTEmx/ky3YDuqqqqdPz48ZluhjHGZJV33313l6pWx5s36IJ+/PjxLFmyJNPNMMaYrCIiGxPNs9KNMcbkOAt6Y4zJcRb0xhiT4yzojTEmx1nQG2NMjrOgN8aYHGdBb4wxOc6C3mSVQDjAE6ueoCPUkemmGJM1XAW9iJwvIitFZI2I3Bxn/nUislREPhCR10VkUsy8W6LrrRSR89LZeHPoWbBmAbPfms38FfMz3RRjskbSoBcRL3AfcAEwCbgsNsijHlfV41R1MnAP8PPoupOAGcAxwPnA/dHfZ0yv7GnbA8Cu1l0Zbokx2cNNj34qsEZV16lqBzAfmB67gKo2xjwsBpzLVk0H5qtqu6quB9ZEf58xvdIWagMgrOEMt8SY7OHmXDejgc0xj+uAU7ovJCLXA98D/MCZMesu6rbu6F611BigOdAMQFOgKcMtMSZ7pG1nrKrep6qHAz8EfpTKuiIyU0SWiMiS+vr6dDXJ5CAn6FuCLRluiTHZw03QbwHGxjweE52WyHzgklTWVdU5qlqrqrXV1XHPsmkMAG3Bti63xpjk3AT9YmCiiEwQET+RnasLYhcQkYkxD/8RWB29vwCYISL5IjIBmAi80/dmm0NVa7C1y60xJrmkNXpVDYrIDcBzgBeYq6rLRGQ2sERVFwA3iMjZQABoAK6KrrtMRP4ELAeCwPWqGuqn12IOAc7O2PZQe4ZbYkz2cHXhEVVdCCzsNu22mPs39rDuncCdvW2gMbHag5GAt9KNMe7ZkbEmqzg9eTsy1hj3LOhNVnEC3ko3xrhnQW+ySke4o8utMSY5C3qTVax0Y0zqLOhNVgmEAoAFvTGpsKA3WSUQjga9lW6Mcc2C3mQVpycfDAdR1SRLG2PAgt5kEVUlEA7gkcifbTAczHCLjMkOFvQma4Q0hKIU+4qBA2UcY0zPLOhN1nCCvTCvsMtjY0zPLOhN1nCCvchXBNjIG2PcsqA3WcMZWlnoi/TorUZvjDsW9CZrdJZufFa6MSYVFvQmazg9+KK8oi6PjTE9s6A3WcN69Mb0jgW9yRpOD96C3pjUWNCbrNG9R2+lG2PcsaA3WcN69Mb0jgW9yRpO0Bf4Cro8Nsb0zILeZI3OoPda0BuTCgt6kzWsR29M71jQm6wR1G49erWgN8YNC3qTNZydr06PPhQOZbI5xmQNC3qTNZxgd3r0NurGGHdcBb2InC8iK0VkjYjcHGf+90RkuYh8JCIvikhNzLyQiHwQ/VmQzsabQ0v3Gn1IrUdvjBu+ZAuIiBe4DzgHqAMWi8gCVV0es9j7QK2qtojILOAe4KvRea2qOjnN7TaHoM4ave2MNSYlbnr0U4E1qrpOVTuA+cD02AVU9WVVbYk+XASMSW8zjTm4dGM1emPccRP0o4HNMY/rotMSuQZ4NuZxgYgsEZFFInJJvBVEZGZ0mSX19fUummQORU5NPt+bD9ioG2PcSlq6SYWIfB2oBU6PmVyjqltE5DDgJRFZqqprY9dT1TnAHIDa2lpNZ5tM7nBq8vm+aNBb6cYYV9z06LcAY2Mej4lO60JEzgZuBS5W1XZnuqpuid6uA14BTuxDe80hzAn2zh69Bb0xrrgJ+sXARBGZICJ+YAbQZfSMiJwIPEQk5HfGTC8Xkfzo/SrgVCB2J64xrjk1eSfobdSNMe4kLd2oalBEbgCeA7zAXFVdJiKzgSWqugC4FygBnhARgE2qejHwKeAhEQkT+VK5u9toHWNcs1E3xvSOqxq9qi4EFnabdlvM/bMTrPcmcFxfGmiMw+nR+8SHV7wW9Ma4ZEfGmqzh9Og94okEvY26McYVC3qTNULhED7xISL4PD7r0RvjkgW9yRpBDeLzRKqNXo+XsIYz3CJjsoMFvckaoXAIr8cLROr01qM3xh0LepM1guEgXokEvddjO2ONccuC3mSNkIY6Szc+j8/G0RvjkgW9yRpdevTitZOaGeOSBb3JGiGNqdF7fDa80hiXLOhN1ujeo7cavTHuWNCbrBEKh8jz5AHRGr2VboxxxYLeZI2gdqvR285YY1yxoDdZo8s4eqvRG+OaBb3JGiEN2agbY3rBgt5kje6nQLDSjTHuWNCbrBEKH+jR+8R2xhrjlgW9yRohDeGRyJ+snQLBGPcs6E3WiB1eaaNujHHPgt5kjaAGO0fdWI3eGPcs6E3W6F6jt9KNMe5Y0JusEXuuG+vRG+OeBb3JGsFwEJ9Eh1faOHpjXLOgN1mj+9krrUdvjDsW9CZrxNbo7eyVxrjnKuhF5HwRWSkia0Tk5jjzvyciy0XkIxF5UURqYuZdJSKroz9XpbPx5tDS5RQINo7eGNeSBr2IeIH7gAuAScBlIjKp22LvA7WqejzwJHBPdN0K4HbgFGAqcLuIlKev+eZQEgzHnAJBvIQ1nOEWGZMd3PTopwJrVHWdqnYA84HpsQuo6suq2hJ9uAgYE71/HvC8qu5R1QbgeeD89DTdHGrsClPG9I6boB8NbI55XBedlsg1wLOprCsiM0VkiYgsqa+vd9EkcyjqXqO3UTfGuJPWnbEi8nWgFrg3lfVUdY6q1qpqbXV1dTqbZHKInb3SmN5xE/RbgLExj8dEp3UhImcDtwIXq2p7Kusa44advdKY3nET9IuBiSIyQUT8wAxgQewCInIi8BCRkN8ZM+s54FwRKY/uhD03Os2YlHU/MjaoQVQ1w60yZvDzJVtAVYMicgORgPYCc1V1mYjMBpao6gIipZoS4AkRAdikqher6h4R+SmRLwuA2aq6p19eicl53Wv0EAl/52hZY0x8rj4hqroQWNht2m0x98/uYd25wNzeNtAYAFXtUqN3bm2IpTHJ2ZGxJis4gd69R28HTRmTnAW9yQrOCJvYA6ZipxtjErOgN1nB6bnHXkoQsJE3xrhgQW+ygtNzP6h0Y0fHGpOUBb3JCk6NPvaAqdjpxpjELOhNVnBKN7EHTIGVboxxw4LeZAWndNO9Rm+jboxJzoLeZAWn557nyQNs1I0xqbCgN1nB2ekaewoEsKA3xg0LepMVnB599xq9lW6MSc6C3mSFzuGVnoPPdWOM6ZkFvckKTs/d6cnbAVPGuGdBb7JC9wOmOodXWo/emKQs6E1W6KzRd9sZazV6Y5KzoDdZwU6BYEzvWdCbrNBZo+9+PvqwnQLBmGQs6E1WsB69Mb1nQW+yglOj735SMxt1Y0xyFvQmK3QeGRvnmrHGmJ5Z0Jus4PTcPZ7oSc2sdGOMaxb0Jit0XkrQDpgyJmUW9CYrOD135zTFdq4bY9yzoDdZIdHOWLvClDHJWdCbrNBZuuk2jt5q9MYk5yroReR8EVkpImtE5OY4808TkfdEJCgil3abFxKRD6I/C9LVcHNo6ezROzV6sRq9MW75ki0gIl7gPuAcoA5YLCILVHV5zGKbgG8AN8X5Fa2qOjkNbTWHsO4XHnF69Da80pjkkgY9MBVYo6rrAERkPjAd6Ax6Vd0QnWcFU9Mvul94pHN4pe2MNSYpN6Wb0cDmmMd10WluFYjIEhFZJCKXxFtARGZGl1lSX1+fwq/OEoHWTLcg63Wv0dulBI1xbyB2xtaoai3wNeAXInJ49wVUdY6q1qpqbXV19QA0aQBtXgx318Bzt2a6JVnN6bkfdD56q9Ebk5SboN8CjI15PCY6zRVV3RK9XQe8ApyYQvuy3ztzINQObz9kPfs+cHruzjh6Ox+9Me65CfrFwEQRmSAifmAG4Gr0jIiUi0h+9H4VcCoxtf1DwobXIX8IhANQtyTTrclaTs89z5MHRAJfECvdGONC0qBX1SBwA/Ac8AnwJ1VdJiKzReRiABE5WUTqgC8DD4nIsujqnwKWiMiHwMvA3d1G6+S2pnrYvxVOvibyeNuHmW1PFus+6sa5b0FvTHJuRt2gqguBhd2m3RZzfzGRkk739d4EjutjG7PXrpWR2/Gfhfd+D7tWZbY9WSwUDiFIZ+kGInV6q9Ebk5wdGdufdq+J3FZNhMojDjw2KQuGg1168xDp0QfCgQy1yJjsYUHfnxo2gCcPhoyG8vHQsDHTLcpaIQ11jrRxeMVKN8a4YUHfn/ZugrLR4PFCeQ00boFgR6ZblZWC4WDnGHqHz2OlG2PcsKDvT/u2QFl0ZOqQ0YDC/m0ZbVK2CmmoS30eIjV6O6mZMclZ0Penxq0wZFTkfln0YOJG14cgmBihcOigHr3X47UevTEuWND3F4323ktHRh6XRgN///bMtSmLBTVoNXpjesmCvr+07IkcJNUZ9CMitxb0vWI1emN6z4K+vzRFA710eOS2sBy8/gPTTUpCGjp4eKV4rUZvjAsW9P2laUfktiQa9CJQPCxytKxJWSgc6jyhmcPr8dq5boxxwYK+vziBXjzswLSSYdC8MzPtyXIJSzdWozcmKQv6/uIEenHVgWklww709E1Kgho8qEfvE5/16I1xwYK+vzTXR2ryBWUHphVVQfPuzLUpi9nwSmN6z4K+vzTvguLqSG3eUVwV+QJQzVy7slTcc93YzlhjXLGg7y/Nu6Coouu04qrIkMu2fZlpUxaLd64bG15pjDsW9P2lZXekVBPLedxi5ZtUxdsZa6NujHHHgr6/tOzuuiMWoKgyOm/PwLcny8XbGZsneTbqxhgXLOj7S8tuKOxeunGCftfAtyfLhcJxDpiy89Eb44oFfX8IBaC98eAevRP81qNPWaIavZVujEnOgr4/OEFeWN51ulO6abWgT1XcGr2d1MwYVyzo+4Ozs7X7qJv80sgVp2xnbMoSHRlrPXpjkrOg7w9Oj93pwTtEIr18K92kLF7Q53nybHilMS5Y0PeHztJNxcHziiqsdNML8Ubd2AFTxrhjQd8fOnv0cYK+sAJaGga2PTkgUenGRt0Yk5yroBeR80VkpYisEZGb48w/TUTeE5GgiFzabd5VIrI6+nNVuho+qFmPPu3inevGavTGuJM06EXEC9wHXABMAi4TkUndFtsEfAN4vNu6FcDtwCnAVOB2Eek2FCUHtTaArwD8RQfPKyyPzDcpSXRkrNXojUnOTY9+KrBGVdepagcwH5geu4CqblDVj4Bwt3XPA55X1T2q2gA8D5yfhnYPbq0NBw+tdDg7Y+3EZimJd81Yn/isRm+MC26CfjSwOeZxXXSaG67WFZGZIrJERJbU1+fAFZh6CvqiCgi1Q6BlYNuU5QLhQNxRN2ENE9bu/QtjTKxBsTNWVeeoaq2q1lZXV2e6OX2XrEcP0Lp34NqT5VQ14c5YwOr0xiThJui3AGNjHo+JTnOjL+tmL1dBb3V6t5yjXy3ojekdN0G/GJgoIhNExA/MABa4/P3PAeeKSHl0J+y50Wm5zYI+rZwgTxT0NsTSmJ4lDXpVDQI3EAnoT4A/qeoyEZktIhcDiMjJIlIHfBl4SESWRdfdA/yUyJfFYmB2dFpua22AwqHx53UGfe5vhnRxgj7Pk9dluvXojXHHl3wRUNWFwMJu026Lub+YSFkm3rpzgbl9aGN26WiBYFv8MfRgPfpeSNajt6A3pmeDYmdsTnECPGHppqLrciYpZwhlvOGVsfONMfFZ0KdbsqDPKwRvvgV9CqxHb0zfWNCnW2fQJ6jR2xksU+bsbI03jh4s6I1JxoI+3TqDPkGNHiJB32bj6N1KFvQ26saYnlnQp1uy0o0zz85g6ZqNujGmbyzo080J+ninKHYUltvwyhQkCnor3RjjjgV9urU2RC4XmBfnzJWOwnI7BUIKEpVu7IApY9yxoE+31j2R3rxI4mUKh9qomxR09ui93Xr0XqvRG+OGBX26tTb0vCMWIl8EwVYItA5Mm7JcZ48+0Th6K90Y0yML+nRr3Zt4aKXDjo5NSaJx9J09+pD16I3piQV9urXsSd6jt6NjU+IEud/r7zK9c3ilWtAb0xML+nRrbYCiJFdLdHr0dtCUK07pJtGoG+vRG9MzC/p0a93T8xh6ODD00oZYupIs6K1Gb0zPLOjTKdmZKx3OfOvRu9IZ9DbqxphesaBPJ6eH3tPBUrHzrUfvSqJRN06PviPUMeBtMiabWNCnk9NDT9ajzysEX6H16F1KtjPWSjfG9MyCPp3c9uidZWzUjSvJavQdYevRG9MTC/p0atkduS2qTL5sYYX16F1KVKO3UyAY444FfTo5we0m6IsqDnwxmB45pZvuPXoRIc+TZzV6Y5KwoE+nzhp9kuGVEPkysJ2xrnSEO/CJD48c/Oea58mzHr0xSVjQp1PLLsgvg24lhriKKqF5V/+3KQd0hDoOKts4/F6/9eiNScKCPp1adkOxi7INRIK+bS+EbMRIMh2hjoNG3Dj8Hr/16I1JwoI+nVp2u6vPAxRXRW6tfJNUIBzA74kf9Hleq9Ebk4yroBeR80VkpYisEZGb48zPF5E/Rue/LSLjo9PHi0iriHwQ/Xkwvc0fZJpTCHpnOSvfJNUR6jhoR6zDSjfGJOdLtoCIeIH7gHOAOmCxiCxQ1eUxi10DNKjqESIyA/hP4KvReWtVdXKa2z04teyCUSe4W9bp0bdY0CfTEe65dGPj6I3pmZse/VRgjaquU9UOYD4wvdsy04FHo/efBM4S6ekSSzlINdI7L652t7yznPXok+oIdZDvzY87L9+bb2evNCYJN0E/Gtgc87guOi3uMqoaBPYBTg1jgoi8LyKvishn+9jewattH4QDUFTlbnlnOQv6pHraGZvnzbMevTFJJC3d9NE2YJyq7haRKcDTInKMqjbGLiQiM4GZAOPGjevnJvWT5vrIbckwd8sXVYB4DqxnEuoIJ67R53vzaQo0DXCLjMkubnr0W4CxMY/HRKfFXUZEfEAZsFtV21V1N4CqvgusBY7s/gSqOkdVa1W1trraZeljsHECu9hlj97jjY6l39l/bcoR7aF2CnwFcef5vX4r3RiThJugXwxMFJEJIuIHZgALui2zALgqev9S4CVVVRGpju7MRUQOAyYC69LT9EGmKRrYxS579BCp0zdZjz6Znko3+d582kJtA9wiY7JL0tKNqgZF5AbgOcALzFXVZSIyG1iiqguAXwO/F5E1wB4iXwYApwGzRSQAhIHrVDU3B453lm6Gu1+nZJj16F1oD7X3uDO2Pdg+wC0yJru4qtGr6kJgYbdpt8XcbwO+HGe9p4Cn+tjG7NC0A8Tr7hTFjpLhsOmt/mtTjmgPJg56v9dPe8iC3pie2JGx6bJ/e6QU4/G6X6dkWKTko9p/7coBbaG2hEFf4C2woDcmCQv6dGnaAaUplG0ASkZErjHbtrd/2pQjko2jt6A3pmcW9Omyf3skuFNROuLAuiahtlBbwlE3+b58AuEAoXBogFtlTPawoE+X/dsOBLdbpSMPrGviCoQDBMNBCrzxg77QWwiQNb16VSXY0EBg2zbCHXaglxkY/X3A1KEh2BEZdTNkVGrrOV8MjRb0iTgjanrq0QO0BlspyisasHalqm3lKhrm/Z79L79CaFf0aGivl8LjjqPskkso+8IlePLjl6eM6SsL+nRweuROD90t54th/9b0tieHOGPkE/XonemtwdYBa1MqQo2N7LjnHvY9+RRSWEjpmWdSePxxSGEhgbotNL36KtvvuIPdc+Yw4o7bKTnttEw3edAKh8Lsq2+lqaGdQHsIj0coKMmjrLqQwtL4x1mYCAv6dGiMHihcNia19fIKI0fH7ut+oLFxtAYiAV6YVxh3vjO9LTj4DppqW76cum9/h8D27VT80z9Rdd21eMvKuixT/S/fpeWtt9j+H//B5pnXUvGNbzDspu8jPvtoAjTuamXNuzvZ+PFudmxoJBQIx12upDyf0UeWM2FyFTXHVuLLS2H02yHA/prSYV8vgx5gyGjYVxd3VlsgxLZ9bext6SAUVgryvFSV5DOsNB+P59A4OWhLsAWAQl+CoI/W6Adbj77p1Vep++6/4C0rY/xj8yicHP9M3SJC8Wc+w4SnnmLnf97Dnt/+lo716xn9i//CUxj/Nec6VWXTsj188MIm6lY0AFA1toRjTxtN1dgSSssL8Bf6CIeU1qYO9u5oYceGRjZ+vJuVb28nv8jHpFNHcfyZYykpt3IYWNCnx75NkdveBP3QcbB7DQCNbQFeXrGT11bv4r2NDazf3Rx3iH1BnodJI4dw8oQKzjhqGCePr8Cbo8HvBHii0o1Tl3e+EAaDxv97ji033UTBkUcy9qEH8bk4f5MnP58Rt/2Y/CMnsv0ns9n0zW8y9sGH8JYUD0CLB4/NK/aw6C9r2blxP8VD85l60QSO+vQIhlT28KV3XOQmHApTt7KB5a9v5YMXNvHRy3Ucc9ooai8cT2HJoV3asaBPh72bIiUYf+ofSi0bS3jNS3x73hJe+KSejlCY8qI8asdXcNEJoxhXUURFsR+vR2gLhNi5v5119c18VLeXua+v56FX11Fdms+XThrD5aeMY2zF4N0h2RtOgBfnxd+2Rb7I6x0sPfrG559ny/e/T+EJJzB2zkN4S0pSWr98xgy8Q4aw5Qf/yubrrmXcww8fEj37ffWtvP7EajZ8tIuSinzOuOJojjplBF6f+4GBHq+HcZMqGTepkn31rbz7fxtY+nIdK97aztTPT+C4z43G4z00Bxpa0KdDwwYoH5/SKqGw8tePtrL1oyCzgi2sWLuer50yiYtOGMnkseWueuhN7UFeWbmTp9/fysOvrWPO39dywXEjueGMI/jUyCG9ey2DTEsgEvSJRtQ405sDzQPWpkSa3niDrd/7PoXHHsvYOXN63RsfcuGFAGy56QfUffs7jL3/PsSfmz3ScCjMBy9s5p2/rsfjEaZ94XCOP3NMn2vsZdWFnHnFp5h81jhef3I1rz+xmhWLtnHmFZ+ielxpmlqfPSzo06FhA4ye4mpRVeWFT3Zy73MrWLWjicvLI0Msn71yDPkTjknpaUvyfXz++FF8/vhRbNvXyqNvbuSxRRv534+28Y/Hj+QH5x7F+Krs/tffCfBEPXpneqaDvnXpUuq+/R38hx0W7cn3bbsPufBCwi0tbPvRj9l6y78x6t57EE9u9UZ3b23ipUc/YefG/Rw2uZrPfnUiJeXxS3S9VTGqmIu+fQJr36vntT+u4om7lzDl/BpqLxyf0n8L2c6Cvq+CHZHSzXFfSbro8q2NzP7rMhat28NhVcX86msncuGIiXD/T8lv3AhM63UzRpYVcvMFRzPr9MN5+LV1/Pr19fxt2XaunDaeG8+eyJCC+BfuGOyci4okCvqSvJIuy2VCx4YNbL72Onzl5Yx9eM5BI2t6a+illxLc00D9z3+Or6qK4bfcnJbfm2nhsPLhC5tZtGAt/gIf533zWI6YksLpvVMkIhwxZRhjji7njSdWs2ThBjYs3cXZ35hE5ejUSmvZyoK+rxo2gIah8vCEi+xrCfCz51cyb9FGygrz+On0Y7hs6jh8Xk/ki0K8sGt1WppTVpTHTecdxZXTavjZ31Yx9431PPPBFm6+4FN88cTRWTdap6kjEuClefH/3S70FeIRT+dyAy1YX8+mb84EVcY+8jB5w9IbWJXf/GeCO3ey59FH8Q0bRuU1V6f19w+0xt2tvPjbT9i6ei8TTqjic5cfTdGQgSlLFRTncdY3JnHYidW8PG8Ff7prMdMuOZwTzhyLZNnnIlUW9H21a2XktuqgC2ehqjz13hbuWvgJDS0dXPHpGr53zlGUFcX0rn1+qJhw4PekybAhBfznpcdzxbQabnvmY2564kPmv7OJf//CsRw9Invq902BJvK9+eR54/9HIiKU5JWwv2P/ALcMQk1NbLr2WoK7dlHz29+QP2FC2p9DRBj+b7cQ3LWLnffei6+6irKLL0778/Q3VWXVOzv4+x9WosCZVx7N0dNGIjLwATvhhGpGHFbGy/NW8MaTa9iwdBdnXTWJ0or0lo0GEwv6vtr5SeS2W9Cv2rGfHz39Me+s38NJ44byu2umcsyoBP/SVx8NO1f0S/OOHV3Gk9d9hiffreOuZz/hH//7da4+dTw3nn0kJfmD/+1v7GhkiL/nL6ZSfyn7AwMb9OH2duquv4H2lasYe/99FJ5wQr89l3g8jLrnP9nc0MDWf7sVb1kZJaef3m/Pl25tTQFeeXwla9/bycjDyzjrG5Moq87sSKLCUj8XXHccn7y5jdf+tJr5P32H02YcyZFTh2fky6e/Df5P+mC3fWlkxE1+pNbX3B7kv19cza9fX09JgY+7v3gcX6kd23PJZPgxsHIhdLSAP/3DIz0e4Ssnj+WcScO557kVPPzaev7nw238+POTuPC4EYP6D7uxPXnQl+WXsa993wC1CLSjgy3f/Rda3n6bUffeMyCh6/H7GXPfr9h05VXUfedGxs6ZQ/EpU/v9eftq3Qf1vPL4StqbA3z6ksM48dyaQVM+FBEmnTqK0UcO5cXffsILv1nOuvfrOf1rRw1YOWmgHDq7nfvL9o9g5AmoKs98sIUzf/YKD/19HV88aTQvfu90Zkwdl/wPe8TxkTr/jo/7tanlxX7u+uLxPDXrM1QU+7n+8ff4+q/fZtWOgS97uNXQ3sDQgqE9LjM0fyh7B+ic/trRwZbv30TTyy8z4vbbKLvoogF5XgBvSUlkP8DYMWyeNYuWxYsH7LlT1dLYwd8e+ZhnH1xK0RA/X76llinnjx80IR+rrLqIS75/EtO+eDgbP97N4z9ZxIpF29AcuiCQBX1fNO+Ghg1sKTqaLz3wJjfO/4BhpQX8+Vuf4Z5LT6CyxOXh187QzLol/dfWGFNqyllww6n8dPoxLK3bxwW/fI0fP/0xu5sG36l+97TtoaKg58szlheU01Ry8JoAABALSURBVNDe0O9tCbe1UfedG9n//PMM/7dbKL/ssn5/zu58FRXU/OY35I0YwaaZ19L0xhsD3oaehMPKste28Pgdi1j7QT1TL5rAl2+ppWrM4B677vEIJ51bw1d/dDLlw4t48befsOCXH9CwPfPHZ6SDBX0fbFn6EgA3vpHP5oZW7vnS8Txz/amcNK48tV80ZCSUjR3Q68f6vB6umDaeV39wBpefMo7H39nE6fe+wi9eWMX+tsCAtSOZ3a27qSyo7HGZqoIqdrfu7tceWLChgU3X/DNNr77KiNtvo+LKK/vtuZLxVVdT87tH8Y8bx+brZrFvwYKMtSXW1tV7efLuJbzy2EoqR5fw1VuncvI/TsCbRUejlo8o5os3TeH0y45k58b9zJ/9Dq8/uZq25sHzmegNq9H3wvKtjdz/yhqmfvJHLvXkc/oZ53H16UdT3Jedm+M/G6nTh0OpXXe2j8qL/cyefixXThvPvc+t4BcvrOY3b2zgmn+YwFXTxncdITTA2oJtNHY0Mqyo5yGL1UXVtIUiy5blp2cMe5d2rFhB3be/Q3DHDkb//GcMueCCtD9HqnxVVdT8/nfUffs7bP3XH9K+ejXVN96YkbNe1m/ezzv/s54NH+2ieGg+51w9iYknZ+9OTfEIx54+hsNOHMaip9fy4YubWfHmNk46r4ZjTx+NvyD7YjP7WpwhobDy8oqdPPrWBl5bvYtiv4c7C5fiG3s63z732L4/wcRz4MPHYfPbUPOZvv++FB0xrISHrqjlo7q9/PeLq/n586t48NW1XDplDFdOq+GIYQP/r/e25sh5/kcU93zlrpHFkesAbG/entag13CYhsf/wM5778U7ZAg1v3s04VkoM8E7ZAjjHp7D9jv/g90PP0LLe+8z6u678I8d2+/PrapsXb2XD57fxIalu/EX+jhl+mGccNZY8vy5cYrgoiF+zrzyUxx/5hgWPb2Ot/6ylvef38TxZ4zhmM+Ozqodthb0SazasZ9nPtjCn9/bwrZ9bQwfks8PzjuKq0ZtpeQP2+G4O9LzRBPPAV8hfPTHjAS94/gxQ3nkqpNZsb2ROX9fx/x3NvO7tzZSW1POl6aM4YJjRzC0aGD+wDfv3wzAmNKezwrqzN/YuJGjKo5Ky3O3LlvGjjv/g9b33qP4tM8y6q678FX2XELKBPH7GfmTOyiqncL2n8xm3cXTqbruOiquuhJPQfrHhbc2dbB68Q6Wv76V3VuaKSjOY+pFEzj+jDHkZ/C/v/5UNaaUz99wAtvX7ePd/9vIO/+znnef3cjhJ1Vz9GdGMvrI8kG5kzmWDLY9y7W1tbpkycDslIynLRDivU0N/H3VLp5fvp219c14BD47sZoZJ4/l7EnDyfN64A9fg42vw/c+6dVZK+N6+nr4+Cn47kdQ0n+HhKdiV1M7T75bx5+WbGZdfTM+j/Dpwyo58+hhnHZkFYdXl/Tbv+iPLH2EX773S16f8XqPPfWWQAuffvzTXHfCdXxr8rd6/XyqStuHH7J77m/Y/7e/4a2oYNhNN1H2hUuyogwR2LaN7XfeSdMLL+IbPpzKq/+Jsi99KeUzaHbXuKuVTct2s+7DXdStaEDDSvW4Uo49bTRHTh2OL0d68G41bG9m6StbWPn2djpagxSV+TlscjXjj69i1MShGfuPRkTeVdXauDNVNekPcD6wElgD3Bxnfj7wx+j8t4HxMfNuiU5fCZyX7LmmTJmiA6U9ENKV2xv16ffr9M7/Xa5ffuBNnXjrQq354V+15gfP6PAZd2rNWVfog795rMt6Lz/4r6q3D9Efn5avNTU1Om/ePD3rrLMU6NPPxAqPdvyoVP94aWGX6V6vVwGtqanRWbNmaU1NTcLpIhL38bx583TevHlx1503b17na3OWiV1PVTUcDutHm/fqXQs/0drbnolsox/+VWtufFzPv/PP+quXVutLK3boloYWDYfDaXl/bnjhBq19sNbVtjvi34/Qf/jlP6T8HOFQSFuXLdP6Bx7QtZ+/SJcfdbSumFKrO37xC338kUcSbsOepsW+R86Px+NRv9/f+bi4uFiLi4s7H4tI5/3Kysou74lj1qxZne+b1+vVWbNmxX1NTW8t0vWXX67Ljzpalx57nN53xBF6XmmpHtPtvY6nvTWg29bu1V/f82ed9fmf6h2XzdNfXfui/uraF/X3P3pT3/zzGq3fvL/H3xFvG/XU9ti/y9jt4PF4uvyNxvv7rays7LJdPR7PQdtl3rx5WllZ2eX9KC4u1srKShURrays7Lzf0/vc/bUdPuEInftfT+rCBz7SB294WX917Yt6//Uv6VP3LtE3/7xaVy/ZoQ3bmzUUSs/nIRlgiSbI1aQ9ehHxAquAc4A6YDFwmaouj1nmW8DxqnqdiMwAvqCqXxWRScAfgKnAKOAF4EhVDSV6vt726FWVtkCYtkCI1kCIlo4g+9uCNLYF2dcaYE9TO7uaOtje2MbWva1sbmhh6942QuHI6/d7PRwzeghFLdtZ+Jtfsm/NErQjco7zoqIi5syZw+WXX85Lc25l2sb/x/L6MKfObaY94SvpnR+e6ufuswuY824H1/41fZfH8/v9qCqBwMGjB5zXBzBz5kxaWloOmnf55ZcD8NhjjzFz5kzafSUUjp9M/thjKRjzKXxDD1wvt8jvZVxFEWPKCxlZVsiw0nwqS/KpKM5jSGEeQwryKM73Uez3kp/npTDPiz/OmQS/cutXeP7V59n7RvIx8mXTytCQcvmUy7n//vu7zAs1NRHau49QQwPBXfUEtm4lsGkTbatW0fbxMsL7I8cRFE6eTNkl0xny+YuY/8zTB22LeNswLy8PEaGjoyNpG1Ph9/uZO3du53b/1re+xQMPPHDQcrNmzTro9Tr+cu+9LPvlf3N6aQUl/iICHj915FF49HEMO24KwZJK2n0ltAb9NDXD/r0BmvcdeB2NLQ2s37GM1Vs/ZP2uj/n3e2/j61+/vMd2O38fsdvN4/EQDh98CcBZs2Zx6qmnHrR8PKluZ2e7PPbYY1x99dUpvT/x3ueioiKuuuoqHn300bifj69+eQZbVu+lbkUDW1c1sGtzE+Fotnh9HoZUF1JWVUBJeQHFQ/MpLM2jsNRPfpEPf6EPf4GPvHwveQXeXv9H0FOP3k3QTwPuUNXzoo9vAVDVu2KWeS66zFsi4gO2A9XAzbHLxi6X6Pl6G/Q797cx9c4Xe1zG6xGqS/IZObSAMeVFjK8s4vDqEo4cXsrE4SXkeT2MHz+ejRs3HrRuTU0NGzZs4JRJ4/j+Mbv41v+2sbu1f8pePzrNz7qGMI8vDfbL74+npqYGoMfXDiTePkcczR+fe51VO5tYV9/E5j0t1DW0sm1fG/taex6a9unDKpg/8+Azd/p8PkKh1L5JvV4vwWDX7bbuC1+k/ZNPukyTwkLyDz+cgkmTKJpyEkXTpnU5IVmi1zmQYrd7om0R7/U6xo8fj7e9lO9N/2XC55BwCH/HPgra9lA1+QiqJx/Gv/74Rt5fsYiGpp0J25NIKtvN6/UyZsyYftnOznZJ5/vo9XrjvgfxtkswEGLP1mZ2b2liz7YW9u1soXF3G00NbbQ3J/5cD6sp5cu3nNyr9vU16C8FzlfVf44+vgI4RVVviFnm4+gyddHHa4FTgDuARao6Lzr918Czqvpkt+eYCcwEGDdu3JTevDFtgRBz31hPgc9Lkd9LUb6P0nwfpQU+hhblUVGcz9DCvKQ7TTweT9zx2CJCOBxOOD/bOTXonl47JN8+8bQHQzQ0B9jT3MG+1gD72wI0dwRpbg/RFggxbEgBF58wKmGbUtW9fY3PPku4pQXv0KH4qqrwjRiJr7qqx/O7D4b3OXab9rQtErXT4/FQWljOSYd/jkCwnfZAK20dLbR0NPH2m6+Q17Qbb2M9oYa9hPbtpfSMM8gbNapX73Hsc6ay3USk37azqg7I++hmu8QKBkK07g/Q1hSgrTlAR2uQjrYQwY4Q+cU+jjy551FmPbSj9zV64FLgkZjHVwC/6rbMx8CYmMdrgSrgV8DXY6b/Gri0p+cbyBp9PN1rq85PTU1Nj/Oz/aempibpa3ezfdLJqcOm8uP1etPy3IPhfY7dpom2RU+vt7fvVV/e41S2m9fr7bft7GyXdP7+RO9Bf/zt9wY91OjdBP004LmYx7cAt3Rb5jlgWvS+D9gFSPdlY5dL9JPpoJ83b54WFRV1eSOLioq67IzpPj8bfvx+v+bl5cWd57y+ZK/dzfZJp1mzZqX8OhPtoExVvNcZbxvm5eV12RmYzvcrdpsm2hY9vd7evld9eY/jrevsVI3Xdrefp1S3s7Nd5s2bl/L7E+99Lioq0lmzZg3Y335v0Meg9wHrgAmAH/gQOKbbMtcDD0bvzwD+FL1/THT5/Oj66wBvT8+X6aBXTTzypKf56Rh14/yRdf9gDIZRN6lsn3RyG/Y9jULpLTcjbAbjqJtkr6G3r70v2y0XR930999+qnoKelfj6EXkQuAXgBeYq6p3isjs6C9eICIFwO+BE4E9wAxVXRdd91bgaiAIfFdVn+3puTI9jt4YY7JRn3bGDjQLemOMSV1PQZ89p5UzxhjTKxb0xhiT4yzojTEmx1nQG2NMjht0O2NFpB7I7LHn/aeKyDEGhwJ7rbnJXuvgVaOq1fFmDLqgz2UisiTRXvFcY681N9lrzU5WujHGmBxnQW+MMTnOgn5gzcl0AwaQvdbcZK81C1mN3hhjcpz16I0xJsdZ0BtjTI6zoB9gInKHiGwRkQ+iPxdmuk3pJiLni8hKEVkjIjdnuj39SUQ2iMjS6HuZU2fjE5G5IrIzegU5Z1qFiDwvIqujt+WZbGO6JHitOfNZtaDPjP9S1cnRn4WZbkw6RS8mfx9wATAJuCx6kfhcdkb0vcyJMdcxfguc323azcCLqjoReDH6OBf8loNfK+TIZ9WC3qTbVGCNqq5T1Q5gPjA9w20yvaCqfydyfYlY04FHo/cfBS4Z0Eb1kwSvNWdY0GfGDSLyUfTfxZz41zfGaGBzzOO66LRcpcDfROTd6EXuc91wVd0Wvb8dGJ7JxgyAnPisWtD3AxF5QUQ+jvMzHXgAOByYDGwDfpbRxpq++gdVPYlIqep6ETkt0w0aKNHL1+Xy+Oyc+az6Mt2AXKSqZ7tZTkQeBv7az80ZaFuAsTGPx0Sn5SRV3RK93SkifyFSuvp7ZlvVr3aIyEhV3SYiI4GdmW5Qf1HVHc79bP+sWo9+gEU/HI4vAB8nWjZLLQYmisgEEfETuVj8ggy3qV+ISLGIlDr3gXPJvfezuwXAVdH7VwHPZLAt/SqXPqvWox9494jIZCL/8m4Ars1sc9JLVYMicgPwHAcuJr8sw83qL8OBv4gIRD5Lj6vq/2W2SekjIn8APgdUiUgdcDtwN/AnEbmGyOnEv5K5FqZPgtf6uVz5rNopEIwxJsdZ6cYYY3KcBb0xxuQ4C3pjjMlxFvTGGJPjLOiNMSbHWdAbY0yOs6A3xpgc9/8BCMonEgh7af0AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "np.random.seed(1)\n",
    "num_samples = (35,20,5,15,20)\n",
    "centers = (-5,2,-2,6,13)\n",
    "spreads = (5,1,4,7,8)\n",
    "\n",
    "data = generate_data(num_samples,centers,spreads)\n",
    "mus, sigmas, pis = EM(data, 5, 200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It took EM 86 iterations to converge. The means found by EM (-5.14,-3.72,1.97,6.10,12.93) are not too far from the actual ones (-5,-2,2,6,13)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}