IBMDeveloperSkillsNetwork-ML0101EN-SkillsNetwork_labs_Module 3_Multi-class_Classification.jupyterlite.ipynb

{
  "metadata": {
    "kernelspec": {
      "name": "python",
      "display_name": "Pyolite",
      "language": "python"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "python",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8"
    },
    "toc-showcode": false
  },
  "nbformat_minor": 4,
  "nbformat": 4,
  "cells": [
    {
      "cell_type": "markdown",
      "source": "<p style=\"text-align:center\">\n    <a href=\"https://skills.network/?utm_medium=Exinfluencer&utm_source=Exinfluencer&utm_content=000026UJ&utm_term=10006555&utm_id=NA-SkillsNetwork-Channel-SkillsNetworkCoursesIBMDeveloperSkillsNetworkML0101ENSkillsNetwork20718538-2022-01-01\" target=\"_blank\">\n    <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/assets/logos/SN_web_lightmode.png\" width=\"200\" alt=\"Skills Network Logo\"  />\n    </a>\n</p>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "# **Softmax Regression ,One-vs-All & One-vs-One for Multi-class Classification**\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Estimated time needed: **1** hour\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "In this lab, we will study how to convert a linear classifier into a multi-class classifier, including multinomial logistic regression or softmax regression, One vs. All (One-vs-Rest) and One vs. One\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## **Objectives**\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "After completing this lab you will be able to:\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "*   Understand and apply some theory behind:\n    *   Softmax regression\n    *   One vs. All (One-vs-Rest)\n    *   One vs. One\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## **Introduction**\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "In Multi-class classification, we classify data into multiple class labels . Unlike classification trees and k-nearest neighbour, the concept of  Multi-class classification for linear classifiers is not as straightforward. We can convert logistic regression to Multi-class classification using multinomial logistic regression or softmax regression; this is a generalization of logistic regression, this will not work for support vector machines. One vs. All (One-vs-Rest) and One vs. One are two other multi-class classification techniques can covert any two-class classifier to a multi-class classifier.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "***\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## **Install and Import the required libraries**\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "For this lab, we are going to be using several Python libraries such as scit-learn, numpy and matplotlib  for visualizations. Some of these libraries might be installed in your lab environment, others may need to be installed by you by removing the hash signs. The cells below will install these libraries when executed.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "import piplite\nawait piplite.install(['pandas'])\nawait piplite.install(['matplotlib'])\nawait piplite.install(['numpy'])\nawait piplite.install(['scikit-learn'])\nawait piplite.install(['scipy'])\n",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "\nfrom pyodide.http import pyfetch\n\nasync def download(url, filename):\n    response = await pyfetch(url)\n    if response.status == 200:\n        with open(filename, \"wb\") as f:\n            f.write(await response.bytes())\n",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "import numpy as np\nimport matplotlib.pyplot as plt\nfrom sklearn import datasets\nfrom sklearn.svm import SVC\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.metrics import accuracy_score\nimport pandas as pd",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "## Utility Function\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "This functions Plots different decision boundary\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "plot_colors = \"ryb\"\nplot_step = 0.02\n\ndef decision_boundary (X,y,model,iris, two=None):\n    x_min, x_max = X[:, 0].min() - 1, X[:, 0].max() + 1\n    y_min, y_max = X[:, 1].min() - 1, X[:, 1].max() + 1\n    xx, yy = np.meshgrid(np.arange(x_min, x_max, plot_step),\n                         np.arange(y_min, y_max, plot_step))\n    plt.tight_layout(h_pad=0.5, w_pad=0.5, pad=2.5)\n    \n    Z = model.predict(np.c_[xx.ravel(), yy.ravel()])\n    Z = Z.reshape(xx.shape)\n    cs = plt.contourf(xx, yy, Z,cmap=plt.cm.RdYlBu)\n    \n    if two:\n        cs = plt.contourf(xx, yy, Z,cmap=plt.cm.RdYlBu)\n        for i, color in zip(np.unique(y), plot_colors):\n            \n            idx = np.where( y== i)\n            plt.scatter(X[idx, 0], X[idx, 1], label=y,cmap=plt.cm.RdYlBu, s=15)\n        plt.show()\n  \n    else:\n        set_={0,1,2}\n        print(set_)\n        for i, color in zip(range(3), plot_colors):\n            idx = np.where( y== i)\n            if np.any(idx):\n\n                set_.remove(i)\n\n                plt.scatter(X[idx, 0], X[idx, 1], label=y,cmap=plt.cm.RdYlBu, edgecolor='black', s=15)\n\n\n        for  i in set_:\n            idx = np.where( iris.target== i)\n            plt.scatter(X[idx, 0], X[idx, 1], marker='x',color='black')\n\n        plt.show()\n",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "This function will plot the probability of belonging to each class; each column is the probability of belonging to a class the row number is the sample number.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "def plot_probability_array(X,probability_array):\n\n    plot_array=np.zeros((X.shape[0],30))\n    col_start=0\n    ones=np.ones((X.shape[0],30))\n    for class_,col_end in enumerate([10,20,30]):\n        plot_array[:,col_start:col_end]= np.repeat(probability_array[:,class_].reshape(-1,1), 10,axis=1)\n        col_start=col_end\n    plt.imshow(plot_array)\n    plt.xticks([])\n    plt.ylabel(\"samples\")\n    plt.xlabel(\"probability of 3 classes\")\n    plt.colorbar()\n    plt.show()",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "In ths lab we will use the  iris dataset,  it consists of 3 different types of irises’ (Setosa y=0, Versicolour y=1, and Virginica y=2) petal and sepal length, stored in a 150x4 numpy.ndarray\n\nThe rows being the samples and the columns being: Sepal Length, Sepal Width, Petal Length and Petal Width.\n\nThe below plot uses the secoond two features\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "pair=[1, 3]\niris = datasets.load_iris()\nX = iris.data[:, pair]\ny = iris.target\nnp.unique(y)",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "plt.scatter(X[:, 0], X[:, 1], c=y, cmap=plt.cm.RdYlBu)\nplt.xlabel(\"sepal width (cm)\")\nplt.ylabel(\"petal width\")",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "## **Softmax Regression**\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "SoftMax regression is similar to logistic regression, the softmax function  convernts the actual distances i.e. dot products of $x$ with each of the parameters $\\theta_i$ for the $K$ classes. This is converted to probabilities using the following :\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "$softmax(x,i) = \\frac{e^{ \\theta_i^T \\bf x}}{\\sum\\_{j=1}^K e^{\\theta_j^T x}} $\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "The training procedure is almost identical to logistic regression. Consider the three-class example where $y \\in {0,1,2}$ we would like to classify $x\\_1$. We can use the softmax function to generate a probability of how likely the sample belongs to each class:\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "$\\[softmax(x\\_1,0),softmax(x\\_1,1),softmax(x\\_1,2)]=\\[0.97,0.2,0.1]$\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "The index of each probability is the same as the class. We can make a prediction using the argmax function:\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "$\\hat{y}=argmax_i  {softmax(x,i)}$\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "For the above example, we can make a prediction as follows:\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "$\\hat{y}=argmax_i  {\\[0.97,0.2,0.1]}=0$\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "<code>sklearn</code> does this automatically, but we can verify the prediction step, we fit the model:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "lr = LogisticRegression(random_state=0).fit(X, y)",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We generate the probability using the method <ocde>predict_proba</code>\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "probability=lr.predict_proba(X)\n",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can plot the probability of belonging to each class; each column is the probability of belonging to a class the row number is the sample number.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "plot_probability_array(X,probability)",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "Here is the output for the first sample:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "probability[0,:]",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "we see it sums to one\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "probability[0,:].sum()",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "we can apply the $argmax$ function\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "np.argmax(probability[0,:])",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can apply the $argmax$ function to each sample\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "softmax_prediction=np.argmax(probability,axis=1)\nsoftmax_prediction",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can verify that sklearn does this under the hood by comparing it to the output of the method <code> predict  </code>.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "yhat =lr.predict(X)\naccuracy_score(yhat,softmax_prediction)",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can't use Softmax regression for SVMs let explore two methods of Multi-class Classification. that we can apply to SVM.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## SVM\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Sklean performs Multi-class Classification automatically, we can apply the method and calculate the accuracy. Train a SVM classifier with the `kernel` set to `linear`, `gamma` set to `0.5`, and the `probability` paramter set to `True`, then train the model using the `X` and `y` data.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "\n#ADD CODE\n\n#ADD CODE\n",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<details><summary>Click here for the solution</summary>\n\n```python\nmodel = SVC(kernel='linear', gamma=.5, probability=True)\n\nmodel.fit(X,y)\n\n```\n\n</details>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Find the `accuracy_score` on the training data\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "<details><summary>Click here for the solution</summary>\n\n```python\nyhat = model.predict(X)\n\naccuracy_score(y,yhat)\n\n```\n\n</details>\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "We can plot the decision_boundary.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "decision_boundary (X,y,model,iris)",
      "metadata": {
        "trusted": true
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "Let's implement on vs One vs. All and One vs. One our self's.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## One vs. All (One-vs-Rest)\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "For one-vs-All classification, if we have K classes, we use K  two-class classifier models—the number of class labels present in the dataset is equal to the number of generated classifiers. First, we create an artificial class we will call this \"dummy\" class. For each classifier, we split the data into two classes.  We take the class samples we would like to classify; the rest of the samples will be labelled as a dummy class. We repeat the process for each class. To make a  classification, we use the classifier with the highest probability, disregarding the dummy class.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "### Train Each Classifier\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Here we train three classifiers and place them in the list <code>my_models</code>. For each class we take the class samples we would like to classify, and the rest will be labelled as a dummy class. We repeat the process for each class. For each classifier, we plot the decision regions. The class we are interested in is in red, and the dummy class is in blue. Similarly, the class samples are marked in blue, and the dummy samples are marked with a black x.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "#dummy class\ndummy_class=y.max()+1\n#list used for classifiers \nmy_models=[]\n#iterate through each class\nfor class_ in np.unique(y):\n    #select the index of our  class\n    select=(y==class_)\n    temp_y=np.zeros(y.shape)\n    #class, we are trying to classify \n    temp_y[y==class_]=class_\n    #set other samples  to a dummy class \n    temp_y[y!=class_]=dummy_class\n    #Train model and add to list \n    model=SVC(kernel='linear', gamma=.5, probability=True)    \n    my_models.append(model.fit(X,temp_y))\n    #plot decision boundary \n    decision_boundary (X,temp_y,model,iris)\n",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "For each sample we calculate the  probability of belonging to each class, not including the dummy class.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "probability_array=np.zeros((X.shape[0],3))\nfor j,model in enumerate(my_models):\n\n    real_class=np.where(np.array(model.classes_)!=3)[0]\n\n    probability_array[:,j]=model.predict_proba(X)[:,real_class][:,0]",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "here is the probability of belonging to each class for the first sample.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "probability_array[0,:]",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "As each is the probability of belonging to the actual class and not the dummy class is does not sum to one.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "probability_array[0,:].sum()",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can plot the probability of belonging to the class. The row number is the sample number.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "plot_probability_array(X,probability_array)",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We can apply the $argmax$ function to each sample to find the class\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "one_vs_all=np.argmax(probability_array,axis=1)\none_vs_all",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "we can calculate the accuracy\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "accuracy_score(y,one_vs_all)",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We see the accuracy is less than the one obtained by sklearn, and this is because for SVM sklearn uses one vs one; let's verify it by comparing the outputs.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "accuracy_score(one_vs_all,yhat)",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "we see that the output are different, now lets implement one vs one\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## One vs One\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "In One-vs-One classification, we split up the data into each class; we then train a two-class classifier on each pair of classes. For example, if we have class 0,1,2, we would train one classifier on the samples that are class 0 and class 1,  a second classifier on samples that are of class 0 and class 2 and a final classifier on samples of class 1 and class 2.\n\nFor  $K$ classes, we have to train  $K(K-1)/2$  classifiers. So if $K=3$, we have $(3x2)/2=3 $classes.\n\nTo perform classification on a sample, we perform a majority vote and select the class with the most predictions.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "here we list each class.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "classes_=set(np.unique(y))\nclasses_\n   ",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "determine the number of classifiers:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "K=len(classes_)\nK*(K-1)/2",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We then train a two-class classifier on each pair of classes. We plot the different training points for each of the two classes\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "pairs=[]\nleft_overs=classes_.copy()\n#list used for classifiers \nmy_models=[]\n#iterate through each class\nfor class_ in classes_:\n    #remove class we have seen before \n    left_overs.remove(class_)\n    #the second class in the pair\n    for second_class in left_overs:\n        pairs.append(str(class_)+' and '+str(second_class))\n        print(\"class {} vs class {} \".format(class_,second_class) )\n        temp_y=np.zeros(y.shape)\n        #find classes in pair \n        select=np.logical_or(y==class_ , y==second_class)\n        #train model \n        model=SVC(kernel='linear', gamma=.5, probability=True)  \n        model.fit(X[select,:],y[select])\n        my_models.append(model)\n        #Plot decision boundary for each pair and corresponding Training samples. \n        decision_boundary (X[select,:],y[select],model,iris,two=True)\n   \n        \n    ",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": "pairs",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "As we can see, our data is left-skewed, containing greater number of '5' star reviews.\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Here, we are plotting the distribution of text length.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "pairs\nmajority_vote_array=np.zeros((X.shape[0],3))\nmajority_vote_dict={}\nfor j,(model,pair) in enumerate(zip(my_models,pairs)):\n\n    majority_vote_dict[pair]=model.predict(X)\n    majority_vote_array[:,j]=model.predict(X)",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "In the following table, each column is the output of a classifier for each  pair of classes; the output is the prediction:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "pd.DataFrame(majority_vote_dict).head(10)",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "To perform classification on a sample, we perform a majority vote i.e. select the class with the most predictions. We repeat the process for each sample.\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "one_vs_one=np.array([np.bincount(sample.astype(int)).argmax() for sample  in majority_vote_array]) \none_vs_one\n    ",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "We calculate the accuracy:\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "accuracy_score(y,one_vs_one)",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "we compare it to sklearn , it's the same!\n",
      "metadata": {}
    },
    {
      "cell_type": "code",
      "source": "accuracy_score(yhat,one_vs_one)",
      "metadata": {},
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": "***\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## Author\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Joseph Santarcangelo\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "### Other Contributors\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Azim Hirjani\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "## Change Log\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "| Date (YYYY-MM-DD) | Version | Changed By | Change Description      |\n| ----------------- | ------- | ---------- | ----------------------- |\n| 2020-07-20        | 0.2     | Azim       | Modified Multiple Areas |\n| 2020-07-17        | 0.1     | Azim       | Created Lab Template    |\n",
      "metadata": {}
    },
    {
      "cell_type": "markdown",
      "source": "Copyright © 2020 IBM Corporation. All rights reserved.\n",
      "metadata": {}
    }
  ]
}