Week_6_assignment(final_assignment).ipynb

{
    "cells": [
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "<a href=\"https://www.bigdatauniversity.com\"><img src=\"https://ibm.box.com/shared/static/cw2c7r3o20w9zn8gkecaeyjhgw3xdgbj.png\" width=\"400\" align=\"center\"></a>\n\n<h1 align=\"center\"><font size=\"5\">Classification with Python</font></h1>"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "In this notebook we try to practice all the classification algorithms that we learned in this course.\n\nWe load a dataset using Pandas library, and apply the following algorithms, and find the best one for this specific dataset by accuracy evaluation methods.\n\nLets first load required libraries:"
        },
        {
            "cell_type": "code",
            "execution_count": 1,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [],
            "source": "import itertools\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib.ticker import NullFormatter\nimport pandas as pd\nimport numpy as np\nimport matplotlib.ticker as ticker\nfrom sklearn import preprocessing\n%matplotlib inline"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### About dataset"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "This dataset is about past loans. The __Loan_train.csv__ data set includes details of 346 customers whose loan are already paid off or defaulted. It includes following fields:\n\n| Field          | Description                                                                           |\n|----------------|---------------------------------------------------------------------------------------|\n| Loan_status    | Whether a loan is paid off on in collection                                           |\n| Principal      | Basic principal loan amount at the                                                    |\n| Terms          | Origination terms which can be weekly (7 days), biweekly, and monthly payoff schedule |\n| Effective_date | When the loan got originated and took effects                                         |\n| Due_date       | Since it\u2019s one-time payoff schedule, each loan has one single due date                |\n| Age            | Age of applicant                                                                      |\n| Education      | Education of applicant                                                                |\n| Gender         | The gender of applicant                                                               |"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets download the dataset"
        },
        {
            "cell_type": "code",
            "execution_count": 3,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "--2020-05-09 01:02:25--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 23101 (23K) [text/csv]\nSaving to: \u2018loan_train.csv\u2019\n\n100%[======================================>] 23,101      --.-K/s   in 0.002s  \n\n2020-05-09 01:02:25 (11.5 MB/s) - \u2018loan_train.csv\u2019 saved [23101/23101]\n\n"
                }
            ],
            "source": "!wget -O loan_train.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_train.csv"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Load Data From CSV File  "
        },
        {
            "cell_type": "code",
            "execution_count": 4,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>9/8/2016</td>\n      <td>9/22/2016</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/9/2016</td>\n      <td>10/8/2016</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30       9/8/2016   \n1           2             2     PAIDOFF       1000     30       9/8/2016   \n2           3             3     PAIDOFF       1000     15       9/8/2016   \n3           4             4     PAIDOFF       1000     30       9/9/2016   \n4           6             6     PAIDOFF       1000     30       9/9/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   45  High School or Below    male  \n1  10/7/2016   33              Bechalor  female  \n2  9/22/2016   27               college    male  \n3  10/8/2016   28               college  female  \n4  10/8/2016   29               college    male  "
                    },
                    "execution_count": 4,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df = pd.read_csv('loan_train.csv')\ndf.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 5,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "(346, 10)"
                    },
                    "execution_count": 5,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.shape"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Convert to date time object "
        },
        {
            "cell_type": "code",
            "execution_count": 6,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  \n0 2016-10-07   45  High School or Below    male  \n1 2016-10-07   33              Bechalor  female  \n2 2016-09-22   27               college    male  \n3 2016-10-08   28               college  female  \n4 2016-10-08   29               college    male  "
                    },
                    "execution_count": 6,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['due_date'] = pd.to_datetime(df['due_date'])\ndf['effective_date'] = pd.to_datetime(df['effective_date'])\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Data visualization and pre-processing\n\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Let\u2019s see how many of each class is in our data set "
        },
        {
            "cell_type": "code",
            "execution_count": 7,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "PAIDOFF       260\nCOLLECTION     86\nName: loan_status, dtype: int64"
                    },
                    "execution_count": 7,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['loan_status'].value_counts()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "260 people have paid off the loan on time while 86 have gone into collection \n"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Lets plot some columns to underestand data better:"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "# notice: installing seaborn might takes a few minutes\n!conda install -c anaconda seaborn -y"
        },
        {
            "cell_type": "code",
            "execution_count": 9,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAG4xJREFUeJzt3XucFOWd7/HPV5wVFaIioyKIMyKKqGTAWY3XJbCyqPF2jAbjUdx4DtFoXDbxeMt5aTa+1nghMclRibhyyCaKGrKgSxINUTmKiRfAEcELITrqKCAQN8YgBPB3/qiaSYM9zKV7pmu6v+/Xq15T9VTVU7+umWd+XU9XP6WIwMzMLGt2KHUAZmZm+ThBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBmZlZJjlBdRFJe0u6T9LrkhZJ+q2kM4tU92hJc4tRV3eQNF9SfanjsNIop7YgqVrSs5JekHR8Fx7nw66quydxguoCkgTMAZ6MiAMi4ghgAjCoRPHsWIrjmpVhWxgLvBoRIyPiqWLEZK1zguoaY4C/RMQPmwsi4s2I+D8AknpJulXS85KWSPpyWj46vdqYJelVSfemDRxJ49OyBcB/a65X0q6Spqd1vSDp9LT8Qkk/lfSfwK8KeTGSZkiaKumJ9F3w36XHfEXSjJztpkpaKGmZpH9ppa5x6TvoxWl8fQqJzTKvbNqCpDrgFuBkSQ2Sdm7t71lSo6Qb03ULJY2S9Kik30u6ON2mj6TH0n1fao43z3H/V875yduuylZEeCryBFwO3Lad9ZOA/53O7wQsBGqB0cAfSd5d7gD8FjgO6A28DQwFBDwIzE33vxH47+n87sByYFfgQqAJ6NdKDE8BDXmmv8+z7Qzg/vTYpwMfAIenMS4C6tLt+qU/ewHzgRHp8nygHugPPAnsmpZfBVxX6t+Xp66byrAtXAjcns63+vcMNAKXpPO3AUuAvkA18F5aviPwqZy6VgBKlz9Mf44DpqWvdQdgLnBCqX+v3TW566cbSLqDpHH9JSL+luSPboSkz6eb7EbS4P4CPBcRTel+DUAN8CHwRkT8Li3/CUnDJq3rNElXpMu9gcHp/LyI+EO+mCKio/3n/xkRIeklYHVEvJTGsiyNsQE4R9IkkoY3ABhO0jCbfSYtezp9M/w3JP94rEKUSVto1tbf88Ppz5eAPhHxJ+BPkjZI2h34M3CjpBOAj4GBwN7Aqpw6xqXTC+lyH5Lz82QnY+5RnKC6xjLgrOaFiLhUUn+Sd4eQvBv6akQ8mruTpNHAxpyiLfz1d9TaoIkCzoqI17ap6yiSBpB/J+kpknd027oiIn6dp7w5ro+3ifFjYEdJtcAVwN9GxPtp11/vPLHOi4hzW4vLyk45toXc423v73m7bQY4j+SK6oiI2CSpkfxt5tsRcdd24ihb/gyqazwO9JZ0SU7ZLjnzjwKXSKoCkHSQpF23U9+rQK2kIelyboN4FPhqTv/8yPYEGBHHR0Rdnml7DXJ7PkXyT+CPkvYGTsqzzTPAsZIOTGPdRdJBnTye9Qzl3BYK/XvejaS7b5OkzwL759nmUeBLOZ9tDZS0VweO0aM5QXWBSDqPzwD+TtIbkp4DfkTSRw3wb8DLwGJJS4G72M7VbERsIOnG+Hn6wfCbOatvAKqAJWldNxT79bRHRLxI0g2xDJgOPJ1nmzUkffgzJS0haeDDujFM62bl3BaK8Pd8L1AvaSHJ1dSreY7xK+A+4Ldp9/os8l/tlaXmD+TMzMwyxVdQZmaWSU5QZmaWSU5QZmaWSU5QZmaWSZlIUOPHjw+S7zZ48lQuU9G4fXgqs6ndMpGg1q5dW+oQzDLL7cMqVSYSlJmZ2bacoMzMLJOcoMzMLJM8WKyZlZVNmzbR1NTEhg0bSh1KRevduzeDBg2iqqqq03U4QZlZWWlqaqJv377U1NSQjhtr3SwiWLduHU1NTdTW1na6HnfxmVlZ2bBhA3vuuaeTUwlJYs899yz4KtYJyirG/gMGIKko0/4DBpT65dh2ODmVXjF+B+7is4rx1qpVNO07qCh1DXq3qSj1mFnrfAVlZmWtmFfO7b167tWrF3V1dRx22GGcffbZrF+/vmXd7NmzkcSrr/718U+NjY0cdthhAMyfP5/ddtuNkSNHcvDBB3PCCScwd+7creqfNm0aw4YNY9iwYRx55JEsWLCgZd3o0aM5+OCDqauro66ujlmzZm0VU/PU2NhYyGntFr6CMrOyVswrZ2jf1fPOO+9MQ0MDAOeddx4//OEP+drXvgbAzJkzOe6447j//vv55je/mXf/448/viUpNTQ0cMYZZ7DzzjszduxY5s6dy1133cWCBQvo378/ixcv5owzzuC5555jn332AeDee++lvr6+1Zh6ijavoCRNl/Re+oTK5rJvSnpHUkM6nZyz7hpJKyS9JukfuipwM7Oe4Pjjj2fFihUAfPjhhzz99NPcc8893H///e3av66ujuuuu47bb78dgJtvvplbb72V/v37AzBq1CgmTpzIHXfc0TUvoITa08U3Axifp/y2iKhLp18ASBoOTAAOTfe5U1KvYgVrZtaTbN68mV/+8pccfvjhAMyZM4fx48dz0EEH0a9fPxYvXtyuekaNGtXSJbhs2TKOOOKIrdbX19ezbNmyluXzzjuvpStv3bp1AHz00UctZWeeeWYxXl6Xa7OLLyKelFTTzvpOB+6PiI3AG5JWAEcCv+10hGZmPUxzMoDkCuqiiy4Cku69yZMnAzBhwgRmzpzJqFGj2qwvYvuDgEfEVnfNlUsXXyGfQV0m6QJgIfD1iHgfGAg8k7NNU1r2CZImAZMABg8eXEAYZuXH7aNny5cM1q1bx+OPP87SpUuRxJYtW5DELbfc0mZ9L7zwAocccggAw4cPZ9GiRYwZM6Zl/eLFixk+fHhxX0QGdPYuvqnAEKAOWAl8Jy3Pd+N73tQfEdMioj4i6qurqzsZhll5cvsoP7NmzeKCCy7gzTffpLGxkbfffpva2tqt7sDLZ8mSJdxwww1ceumlAFx55ZVcddVVLV13DQ0NzJgxg6985Std/hq6W6euoCJidfO8pLuB5nsgm4D9cjYdBLzb6ejMzAo0eJ99ivq9tcHpnXIdNXPmTK6++uqtys466yzuu+8+rrrqqq3Kn3rqKUaOHMn69evZa6+9+MEPfsDYsWMBOO2003jnnXc45phjkETfvn35yU9+woAy/PK42urbBEg/g5obEYelywMiYmU6/8/AURExQdKhwH0knzvtCzwGDI2ILdurv76+PhYuXFjI6zBrk6SiflG3jbZTtKEM3D465pVXXmnpDrPSauV30e620eYVlKSZwGigv6Qm4HpgtKQ6ku67RuDLABGxTNKDwMvAZuDStpKTmZlZPu25i+/cPMX3bGf7fwX+tZCgzMzMPNSRmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZW1fQcNLurjNvYd1L6RPVatWsWECRMYMmQIw4cP5+STT2b58uUsW7aMMWPGcNBBBzF06FBuuOGGlq8szJgxg8suu+wTddXU1LB27dqtymbMmEF1dfVWj9B4+eWXAVi+fDknn3wyBx54IIcccgjnnHMODzzwQMt2ffr0aXkkxwUXXMD8+fP53Oc+11L3nDlzGDFiBMOGDePwww9nzpw5LesuvPBCBg4cyMaNGwFYu3YtNTU1HfqdtJcft2FmZW3lO29z1HWPFK2+Z7+Vb+zsrUUEZ555JhMnTmwZtbyhoYHVq1dz4YUXMnXqVMaNG8f69es566yzuPPOO1tGiuiIL3zhCy2jnDfbsGEDp5xyCt/97nc59dRTAXjiiSeorq5uGX5p9OjRTJkypWW8vvnz57fs/+KLL3LFFVcwb948amtreeONNzjxxBM54IADGDFiBJA8W2r69OlccsklHY65I3wFZWZWZE888QRVVVVcfPHFLWV1dXUsX76cY489lnHjxgGwyy67cPvtt3PTTTcV7dj33XcfRx99dEtyAvjsZz/b8kDEtkyZMoVrr72W2tpaAGpra7nmmmu49dZbW7aZPHkyt912G5s3by5a3Pk4QZmZFdnSpUs/8UgMyP+ojCFDhvDhhx/ywQcfdPg4ud12dXV1fPTRR60eu73a8ziPwYMHc9xxx/HjH/+408dpD3fxmZl1k20fi5GrtfLtydfFV6h8MeYru/baaznttNM45ZRTinr8XL6CMjMrskMPPZRFixblLd92XMXXX3+dPn360Ldv3y49dkf23zbGfI/zOPDAA6mrq+PBBx/s9LHa4gRlZlZkY8aMYePGjdx9990tZc8//zxDhw5lwYIF/PrXvwaSBxtefvnlXHnllUU79he/+EV+85vf8POf/7yl7JFHHuGll15q1/5XXHEF3/72t2lsbASgsbGRG2+8ka9//euf2PYb3/gGU6ZMKUrc+biLz8zK2oCB+7XrzruO1NcWScyePZvJkydz00030bt3b2pqavje977HQw89xFe/+lUuvfRStmzZwvnnn7/VreUzZszY6rbuZ55JngE7YsQIdtghuaY455xzGDFiBA888MBWz5O68847OeaYY5g7dy6TJ09m8uTJVFVVMWLECL7//e+36/XV1dVx8803c+qpp7Jp0yaqqqq45ZZbWp4QnOvQQw9l1KhR7X50fUe163EbXc2PE7Du4MdtVAY/biM7Cn3cRptdfJKmS3pP0tKcslslvSppiaTZknZPy2skfSSpIZ1+2N5AzMzMcrXnM6gZwLbXx/OAwyJiBLAcuCZn3e8joi6dLsbMzKwT2kxQEfEk8Idtyn4VEc3f0HqG5NHuZmaZkIWPLipdMX4HxbiL70vAL3OWayW9IOn/STq+tZ0kTZK0UNLCNWvWFCEMs/Lh9tF5vXv3Zt26dU5SJRQRrFu3jt69exdUT0F38Un6Bsmj3e9Ni1YCgyNinaQjgDmSDo2IT3xFOiKmAdMg+RC4kDjMyo3bR+cNGjSIpqYmnNhLq3fv3gwaVFjnWqcTlKSJwOeAsZG+VYmIjcDGdH6RpN8DBwG+BcnMukVVVVXLOHLWs3Wqi0/SeOAq4LSIWJ9TXi2pVzp/ADAUeL0YgZqZWWVp8wpK0kxgNNBfUhNwPcldezsB89LxmZ5J79g7AfiWpM3AFuDiiPhD3orNzMy2o80EFRHn5im+p5Vtfwb8rNCgzMzMPBafmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllkhOUmZllUrsSlKTpkt6TtDSnrJ+keZJ+l/7cIy2XpB9IWiFpiaRRXRW8mZmVr/ZeQc0Axm9TdjXwWEQMBR5LlwFOInmS7lBgEjC18DDNzKzStCtBRcSTwLZPxj0d+FE6/yPgjJzyf4/EM8DukgYUI1gzM6schXwGtXdErARIf+6Vlg8E3s7Zrikt24qkSZIWSlq4Zs2aAsIwKz9uH2Zdc5OE8pTFJwoipkVEfUTUV1dXd0EYZj2X24dZYQlqdXPXXfrzvbS8CdgvZ7tBwLsFHMfMzCpQIQnqYWBiOj8ReCin/IL0br7PAH9s7go0MzNrrx3bs5GkmcBooL+kJuB64CbgQUkXAW8BZ6eb/wI4GVgBrAf+scgxm5lZBWhXgoqIc1tZNTbPtgFcWkhQZmZmHknCzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyyQnKzMwyqV2jmecj6WDggZyiA4DrgN2B/wk0P6f62oj4RacjNDOzitTpBBURrwF1AJJ6Ae8As0me/3RbREwpSoRmZlaRitXFNxb4fUS8WaT6zMyswhUrQU0AZuYsXyZpiaTpkvbIt4OkSZIWSlq4Zs2afJuYVSy3D7MiJChJfwOcBvw0LZoKDCHp/lsJfCfffhExLSLqI6K+urq60DDMyorbh1lxrqBOAhZHxGqAiFgdEVsi4mPgbuDIIhzDzMwqTDES1LnkdO9JGpCz7kxgaRGOYWZmFabTd/EBSNoFOBH4ck7xLZLqgAAat1lnZmbWLgUlqIhYD+y5Tdn5BUVkZmaGR5IwM7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMKug2c7OeRL2qGPRuU9HqMrOu5QRlFSO2bOKo6x4pSl3Pfmt8Ueoxs9a5i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMzDKp4NvMJTUCfwK2AJsjol5SP+ABoIbkmVDnRMT7hR7LzMwqR7GuoD4bEXURUZ8uXw08FhFDgcfSZasw+w8YgKSCp/0HDGj7YGZWdrrqi7qnA6PT+R8B84GruuhYllFvrVpF076DCq6nWKM/mFnPUowrqAB+JWmRpElp2d4RsRIg/bnXtjtJmiRpoaSFa9asKUIYZuXD7cOsOAnq2IgYBZwEXCrphPbsFBHTIqI+Iuqrq6uLEIZZ+XD7MCtCgoqId9Of7wGzgSOB1ZIGAKQ/3yv0OGZmVlkKSlCSdpXUt3keGAcsBR4GJqabTQQeKuQ4ZmZWeQq9SWJvYLak5rrui4hHJD0PPCjpIuAt4OwCj2NmZhWmoAQVEa8Dn85Tvg4YW0jdZmZW2TyShJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZZITlJmZZfIJ2F31RF0zM+tBsvgEbF9BmZlZJnU6QUnaT9ITkl6RtEzSP6Xl35T0jqSGdDq5eOGamVmlKKSLbzPw9YhYnD60cJGkeem62yJiSuHhmZlZpep0goqIlcDKdP5Pkl4BBhYrMDMzq2xF+QxKUg0wEng2LbpM0hJJ0yXt0co+kyQtlLRwzZo1xQjDrGy4fZgVIUFJ6gP8DJgcER8AU4EhQB3JFdZ38u0XEdMioj4i6qurqwsNw6ysuH2YFZigJFWRJKd7I+I/ACJidURsiYiPgbuBIwsP08zMKk0hd/EJuAd4JSK+m1Oe+y2tM4GlnQ/PzMwqVSF38R0LnA+8JKkhLbsWOFdSHRBAI/DlgiI0M7OKVMhdfAsA5Vn1i86HY2ZmlvBIEmZmlkkei8+6jHpVFWVcLvWqKkI0ZtbTOEFZl4ktmzjqukcKrufZb40vQjRm1tO4i8/MzDLJCcrMzDLJCcrMzDLJCcrMzDLJCcrMrJtl8fHqWeS7+MzMulkWH6+eRb6CMjOzTHKCMjOzTHIXn5mZZXLkFycoMzPL5Mgv7uIzM7NM6rIEJWm8pNckrZB0daH1+bZMM7PK0iVdfJJ6AXcAJwJNwPOSHo6Ilztbp2/LNDOrLF31GdSRwIqIeB1A0v3A6UCnE1TW7D9gAG+tWlVwPYP32Yc3V64sQkTlTcr3bEzLIreNthXrhoQdelWVddtQRBS/UunzwPiI+B/p8vnAURFxWc42k4BJ6eLBwGtFD6T9+gNrS3j8Qjj20mgr9rUR0elPizPUPsr5d5Rl5Rx7u9tGV11B5UvpW2XCiJgGTOui43eIpIURUV/qODrDsZdGV8eelfbh31FpOPZEV90k0QTsl7M8CHi3i45lZmZlqKsS1PPAUEm1kv4GmAA83EXHMjOzMtQlXXwRsVnSZcCjQC9gekQs64pjFUnJu1IK4NhLoyfH3hE9+XU69tIoWuxdcpOEmZlZoTyShJmZZZITlJmZZVLFJChJvSS9IGluulwr6VlJv5P0QHozB5J2SpdXpOtrShz37pJmSXpV0iuSjpbUT9K8NPZ5kvZIt5WkH6SxL5E0qsSx/7OkZZKWSpopqXdWz7uk6ZLek7Q0p6zD51nSxHT730ma2J2vobPcNkoSu9tGO1RMggL+CXglZ/lm4LaIGAq8D1yUll8EvB8RBwK3pduV0veBRyJiGPBpktdwNfBYGvtj6TLAScDQdJoETO3+cBOSBgKXA/URcRjJzTITyO55nwFs++XBDp1nSf2A64GjSEZTub654Wac20Y3ctvoQNuIiLKfSL6H9RgwBphL8kXitcCO6fqjgUfT+UeBo9P5HdPtVKK4PwW8se3xSUYVGJDODwBeS+fvAs7Nt10JYh8IvA30S8/jXOAfsnzegRpgaWfPM3AucFdO+VbbZXFy23DbaGfMJWkblXIF9T3gSuDjdHlP4L8iYnO63ETyRwN//eMhXf/HdPtSOABYA/zftAvm3yTtCuwdESvTGFcCe6Xbt8Seyn1d3Soi3gGmAG8BK0nO4yJ6xnlv1tHznJnz3wFuG93MbWOr8u0q+wQl6XPAexGxKLc4z6bRjnXdbUdgFDA1IkYCf+avl9L5ZCb29PL9dKAW2BfYleTyf1tZPO9taS3WnvQa3DbcNrpCUdtG2Sco4FjgNEmNwP0kXRnfA3aX1PxF5dyhmFqGaUrX7wb8oTsDztEENEXEs+nyLJJGuVrSAID053s522dliKm/B96IiDURsQn4D+AYesZ5b9bR85yl898ebhul4bbRzvNf9gkqIq6JiEERUUPyQeTjEXEe8ATw+XSzicBD6fzD6TLp+scj7TTtbhGxCnhb0sFp0ViSR5bkxrht7Bekd9J8Bvhj82V4CbwFfEbSLpLEX2PP/HnP0dHz/CgwTtIe6bvkcWlZJrltuG0UoHvaRik+JCzVBIwG5qbzBwDPASuAnwI7peW90+UV6foDShxzHbAQWALMAfYg6X9+DPhd+rNfuq1IHhT5e+AlkruEShn7vwCvAkuBHwM7ZfW8AzNJPg/YRPJu76LOnGfgS+lrWAH8Y6n/5jvw+t02ujd2t412HNtDHZmZWSaVfRefmZn1TE5QZmaWSU5QZmaWSU5QZmaWSU5QZmaWSU5QGSZpi6SGdMTjn0rapZXtfiFp907Uv6+kWQXE1yipf2f3N+sst43K4NvMM0zShxHRJ52/F1gUEd/NWS+S3+HHrdXRxfE1knzPYW0pjm+Vy22jMvgKqud4CjhQUo2SZ9/cCSwG9mt+t5az7m4lz5r5laSdASQdKOnXkl6UtFjSkHT7pen6CyU9JOkRSa9Jur75wJLmSFqU1jmpJK/erHVuG2XKCaoHSMffOonkm9kABwP/HhEjI+LNbTYfCtwREYcC/wWclZbfm5Z/mmTcr3zDvBwJnEfyDf2zJdWn5V+KiCOAeuBySaUeSdkMcNsod05Q2bazpAaS4VzeAu5Jy9+MiGda2eeNiGhI5xcBNZL6AgMjYjZARGyIiPV59p0XEesi4iOSASyPS8svl/Qi8AzJgI9DC35lZoVx26gAO7a9iZXQRxFRl1uQdK3z5+3sszFnfguwM/mHus9n2w8kQ9JoktGXj46I9ZLmk4wNZlZKbhsVwFdQFSAiPgCaJJ0BIGmnVu56OlFSv7Rv/gzgaZKh/d9PG+Aw4DPdFrhZF3PbyDYnqMpxPkl3xBLgN8A+ebZZQDKycgPws4hYCDwC7JjudwNJV4ZZOXHbyCjfZm5AcqcSyW2xl5U6FrMscdsoHV9BmZlZJvkKyszMMslXUGZmlklOUGZmlklOUGZmlklOUGZmlklOUGZmlkn/H+LDZoiBEQ8dAAAAAElFTkSuQmCC\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "import seaborn as sns\n\nbins = np.linspace(df.Principal.min(), df.Principal.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'Principal', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        },
        {
            "cell_type": "code",
            "execution_count": 10,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGfZJREFUeJzt3XuQVOW57/HvTxgdFbygo4yMwKgoopIBZ3tDDYJy2N49XuKOR7GOJx4Naqjo8ZZTVrLdZbyVmhwvkUQLK1HUmA26SUWDCidi4gVwRBBv0UFHQS7RKAchgs/5o9fMHqBhembWTK/u+X2qVnWvt1e/61lMvzy93vX2uxQRmJmZZc02xQ7AzMwsHycoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCeolEjaU9Ijkt6XNE/SXySdkVLdoyXNSKOu7iBptqT6YsdhxVdO7UJSlaSXJb0m6Zgu3M/qrqq71DhBpUCSgOnAnyJin4g4FDgXqClSPL2LsV+z1sqwXYwF3oqIERHxQhox2dY5QaVjDPCPiPhFc0FELImI/wMgqZek2yS9KmmBpP+ZlI9OzjaekPSWpIeTRo2k8UnZHOC/NtcraUdJDyZ1vSbptKT8Qkm/lfQfwB87czCSpki6T9Ks5Jvvt5N9LpY0pdV290maK2mRpJ9soa5xybfm+Ul8fToTm5WUsmkXkuqAW4ETJTVI2n5Ln21JjZJuSl6bK2mkpGck/VXSJck2fSQ9l7z3jeZ48+z3f7X698nbxspaRHjp5AJcAdy5ldcvBv538nw7YC5QC4wG/k7uG+U2wF+Ao4FK4CNgCCDgcWBG8v6bgP+WPN8FeAfYEbgQaAL6bSGGF4CGPMvxebadAjya7Ps04AvgkCTGeUBdsl2/5LEXMBsYnqzPBuqB3YE/ATsm5dcANxT77+Wle5YybBcXAncnz7f42QYagUuT53cCC4C+QBWwPCnvDezUqq73ACXrq5PHccDk5Fi3AWYAxxb779qdi7uCuoCke8g1qH9ExD+R+6ANl3RWssnO5BrZP4BXIqIpeV8DMBhYDXwQEe8m5b8h15hJ6jpV0lXJeiUwMHk+MyL+li+miGhvn/l/RERIegP4NCLeSGJZlMTYAJwj6WJyja0aGEauMTY7Iil7MfkCvC25/2ysByqTdtGsrc/2U8njG0CfiPgS+FLSWkm7AP8PuEnSscA3wABgT2BZqzrGJctryXofcv8+f+pgzCXHCSodi4Azm1ciYqKk3cl9I4TcN6DLI+KZ1m+SNBpY16poA//5N9nSJIkCzoyItzep63ByH/r8b5JeIPctblNXRcSzecqb4/pmkxi/AXpLqgWuAv4pIj5Luv4q88Q6MyL+ZUtxWVkrx3bRen9b+2xvtf0A55E7ozo0Ir6W1Ej+9vPTiLh/K3GUNV+DSsfzQKWkS1uV7dDq+TPApZIqACTtL2nHrdT3FlArad9kvXUjeAa4vFWf/IhCAoyIYyKiLs+ytUa4NTuRa/h/l7Qn8M95tnkJGCVpvyTWHSTt38H9Wekp53bR2c/2zuS6+76WdBwwKM82zwD/vdW1rQGS9mjHPkqeE1QKItdhfDrwbUkfSHoFeIhcvzTAr4A3gfmSFgL3s5Wz14hYS67r4vfJxeAlrV6+EagAFiR13Zj28RQiIl4n1/WwCHgQeDHPNivI9dtPlbSAXKMe2o1hWhGVc7tI4bP9MFAvaS65s6m38uzjj8AjwF+SrvYnyH+2V7aaL8qZmZllis+gzMwsk5ygzMwsk5ygzMwsk5ygzMwsk7o1QY0fPz7I/Y7Bi5dyXTrN7cRLD1gK0q0JauXKld25O7OS5HZiluMuPjMzyyQnKDMzyyQnKDMzyyRPFmtmZefrr7+mqamJtWvXFjuUHq2yspKamhoqKio69H4nKDMrO01NTfTt25fBgweTzB9r3SwiWLVqFU1NTdTW1naoDnfxmVnZWbt2LbvttpuTUxFJYrfdduvUWawTVDcaVF2NpFSWQdXVxT4cs0xzciq+zv4N3MXXjT5ctoymvWpSqavmk6ZU6jEzyyqfQZlZ2Uuz96LQHoxevXpRV1fHwQcfzNlnn82aNWtaXps2bRqSeOut/7wNVGNjIwcffDAAs2fPZuedd2bEiBEccMABHHvsscyYMWOj+idPnszQoUMZOnQohx12GHPmzGl5bfTo0RxwwAHU1dVRV1fHE088sVFMzUtjY2Nn/lm7nM+gzKzspdl7AYX1YGy//fY0NDQAcN555/GLX/yCH/7whwBMnTqVo48+mkcffZQf//jHed9/zDHHtCSlhoYGTj/9dLbffnvGjh3LjBkzuP/++5kzZw6777478+fP5/TTT+eVV16hf//+ADz88MPU19dvMaZS4DMoM7Mudswxx/Dee+8BsHr1al588UUeeOABHn300YLeX1dXxw033MDdd98NwC233MJtt93G7rvvDsDIkSOZMGEC99xzT9ccQJE4QZmZdaH169fzhz/8gUMOOQSA6dOnM378ePbff3/69evH/PnzC6pn5MiRLV2CixYt4tBDD93o9fr6ehYtWtSyft5557V05a1atQqAr776qqXsjDPOSOPwupS7+MzMukBzMoDcGdRFF10E5Lr3Jk2aBMC5557L1KlTGTlyZJv1RWx9EvCI2GjUXDl08RWUoCQ1Al8CG4D1EVEvqR/wGDAYaATOiYjPuiZMM7PSki8ZrFq1iueff56FCxciiQ0bNiCJW2+9tc36XnvtNQ488EAAhg0bxrx58xgzZkzL6/Pnz2fYsGHpHkSRtaeL77iIqIuI5pR8LfBcRAwBnkvWzcxsC5544gkuuOAClixZQmNjIx999BG1tbUbjcDLZ8GCBdx4441MnDgRgKuvvpprrrmmpeuuoaGBKVOm8P3vf7/Lj6E7daaL7zRgdPL8IWA2cE0n4zEzS93A/v1T/e3gwGSkXHtNnTqVa6/d+Lv8mWeeySOPPMI112z83+cLL7zAiBEjWLNmDXvssQc///nPGTt2LACnnnoqH3/8MUcddRSS6Nu3L7/5zW+oLrMf8Kutfk0ASR8An5G7E+L9ETFZ0ucRsUurbT6LiF3zvPdi4GKAgQMHHrpkyZLUgi81klL9oW4hfzvrdh366bzbSboWL17c0h1mxbWFv0VB7aTQLr5RETES+GdgoqRjCw0uIiZHRH1E1FdVVRX6NrMexe3EbHMFJaiI+CR5XA5MAw4DPpVUDZA8Lu+qIM3MrOdpM0FJ2lFS3+bnwDhgIfAUMCHZbALwZFcFaWZmPU8hgyT2BKYl4+t7A49ExNOSXgUel3QR8CFwdteFaWZmPU2bCSoi3ge+lad8FTC2K4IyMzPzVEdmZpZJTlBmVvb2qhmY6u029qoZWNB+ly1bxrnnnsu+++7LsGHDOPHEE3nnnXdYtGgRY8aMYf/992fIkCHceOONLT8bmTJlCpdddtlmdQ0ePJiVK1duVDZlyhSqqqo2uoXGm2++CcA777zDiSeeyH777ceBBx7IOeecw2OPPdayXZ8+fVpuyXHBBRcwe/ZsTj755Ja6p0+fzvDhwxk6dCiHHHII06dPb3ntwgsvZMCAAaxbtw6AlStXMnjw4Hb9TQrhufgKMKi6mg+XLSt2GGbWQUs//ojDb3g6tfpe/tfxbW4TEZxxxhlMmDChZdbyhoYGPv30Uy688ELuu+8+xo0bx5o1azjzzDO59957W2aKaI/vfOc7LbOcN1u7di0nnXQSd9xxB6eccgoAs2bNoqqqqmX6pdGjR3P77be3zNc3e/bslve//vrrXHXVVcycOZPa2lo++OADTjjhBPbZZx+GDx8O5O4t9eCDD3LppZe2O+ZCOUEVIK17yfguuGY9x6xZs6ioqOCSSy5pKaurq+OBBx5g1KhRjBs3DoAddtiBu+++m9GjR3coQeXzyCOPcOSRR7YkJ4Djjjuu4PfffvvtXH/99dTW1gJQW1vLddddx2233cavf/1rACZNmsSdd97J9773vVRizsddfGZmXWDhwoWb3RID8t8qY99992X16tV88cUX7d5P6267uro6vvrqqy3uu1CF3M5j4MCBHH300S0Jqyv4DMrMrBtteluM1rZUvjX5uvg6K1+M+cquv/56Tj31VE466aRU99/MZ1BmZl3goIMOYt68eXnL586du1HZ+++/T58+fejbt2+X7rs97980xny389hvv/2oq6vj8ccf7/C+tsYJysysC4wZM4Z169bxy1/+sqXs1VdfZciQIcyZM4dnn30WyN3Y8IorruDqq69Obd/f/e53+fOf/8zvf//7lrKnn36aN954o6D3X3XVVfz0pz+lsbERgMbGRm666SauvPLKzbb90Y9+xO23355K3JtyF5+Zlb3qAXsXNPKuPfW1RRLTpk1j0qRJ3HzzzVRWVjJ48GDuuusunnzySS6//HImTpzIhg0bOP/88zcaWj5lypSNhnW/9NJLAAwfPpxttsmdV5xzzjkMHz6cxx57bKP7Sd17770cddRRzJgxg0mTJjFp0iQqKioYPnw4P/vZzwo6vrq6Om655RZOOeUUvv76ayoqKrj11ltb7hDc2kEHHcTIkSMLvnV9exR0u4201NfXx6anjaUgrdtk1HzS5NttlL8O3W6jtVJtJ1ni221kR3fcbsPMzKxbOUGZmVkmOUGZWVlyF3jxdfZv4ARlZmWnsrKSVatWOUkVUUSwatUqKisrO1yHR/GZWdmpqamhqamJFStWFDuUHq2yspKamo4PDHOCKlHb0bFfneczsH9/lixdmkpdZllQUVHRMo+clS4nqBK1DlIdsm5mljUFX4OS1EvSa5JmJOu1kl6W9K6kxyRt23VhmplZT9OeQRI/ABa3Wr8FuDMihgCfARelGZiZmfVsBSUoSTXAScCvknUBY4Ankk0eAk7vigDNzKxnKvQM6i7gauCbZH034POIWJ+sNwED8r1R0sWS5kqa6xE1Zvm5nZhtrs0EJelkYHlEtJ67Pd/wsbw/OIiIyRFRHxH1VVVVHQzTrLy5nZhtrpBRfKOAUyWdCFQCO5E7o9pFUu/kLKoG+KTrwjQzs56mzTOoiLguImoiYjBwLvB8RJwHzALOSjabADzZZVGamVmP05mpjq4BfijpPXLXpB5IJyQzM7N2/lA3ImYDs5Pn7wOHpR+SmZmZJ4s1M7OMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMcoIyM7NMajNBSaqU9Iqk1yUtkvSTpLxW0suS3pX0mKRtuz5cMzPrKQo5g1oHjImIbwF1wHhJRwC3AHdGxBDgM+CirgvTzMx6mjYTVOSsTlYrkiWAMcATSflDwOldEqGZmfVIBV2DktRLUgOwHJgJ/BX4PCLWJ5s0AQO28N6LJc2VNHfFihVpxGxWdtxOzDZXUIKKiA0RUQfUAIcBB+bbbAvvnRwR9RFRX1VV1fFIzcqY24nZ5to1ii8iPgdmA0cAu0jqnbxUA3ySbmhmZtaTFTKKr0rSLsnz7YHjgcXALOCsZLMJwJNdFaSZmfU8vdvehGrgIUm9yCW0xyNihqQ3gUcl/RvwGvBAF8ZpZmY9TJsJKiIWACPylL9P7nqUmZlZ6jyThJmZZZITlJmZZZITlJmZZZITlJmZZVLZJqhB1dVISmUxM7PuV8gw85L04bJlNO1Vk0pdNZ80pVKPmZkVrmzPoMzMrLQ5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSY5QZmZWSa1maAk7S1plqTFkhZJ+kFS3k/STEnvJo+7dn24ZmbWUxRyBrUeuDIiDgSOACZKGgZcCzwXEUOA55J1MzOzVLSZoCJiaUTMT55/CSwGBgCnAQ8lmz0EnN5VQZqZWc/TrmtQkgYDI4CXgT0jYinkkhiwxxbec7GkuZLmrlixonPRmpUptxOzzRWcoCT1AX4HTIqILwp9X0RMjoj6iKivqqrqSIxmZc/txGxzBSUoSRXkktPDEfHvSfGnkqqT16uB5V0TopmZ9USFjOIT8ACwOCLuaPXSU8CE5PkE4Mn0w7PusB20edv7QpZB1dXFPhQzKyOF3PJ9FHA+8IakhqTseuBm4HFJFwEfAmd3TYjW1dYBTXvVdLqemk+aOh+MmVmizQQVEXMAbeHlsemGk03qVZHKf77qvW1q/4mrV0Uq9ZiZZVUhZ1A9Xmz4msNveLrT9bz8r+NTqae5LjOzcuapjszMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJOcoMzMLJPKdiaJtKYnMjOz4ijbBJXW9ETgaYXMzIrBXXxmZpZJTlBmZpZJTlBmZpZJZXsNqtylOQjE95ayrBlUXc2Hy5Z1up7tt+nFV99sSCEiGNi/P0uWLk2lLiuME1SJ8iAQK2cfLluW2l2e06inuS7rXm128Ul6UNJySQtblfWTNFPSu8njrl0bppmZ9TSFXIOaAmz6Ffta4LmIGAI8l6xbD7cdICmVZVB1dbEPx8yKrM0uvoj4k6TBmxSfBoxOnj8EzAauSTEuK0HrwN0pZpaajo7i2zMilgIkj3tsaUNJF0uaK2nuihUrOrg7s/JWDu1kUHV1amfQZtANgyQiYjIwGaC+vj66en9mpagc2klaAxvAZ9CW09EzqE8lVQMkj8vTC8nMzKzjCeopYELyfALwZDrhmJmZ5RQyzHwq8BfgAElNki4CbgZOkPQucEKybmZmlppCRvH9yxZeGptyLGZmZi0yNRefRwGZmVmzTE115FFAZmbWLFMJyoojrYlnPemsmaXJCcpSm3jWk86aWZoydQ3KzMysmROUmZllkhOUmZllkhOUmZllkhOUZZLvLdU9/NtDyzKP4rNM8r2luod/e2hZ5gRlqUnr91TNdZlZz+YEZalJ6/dU4N9UmZmvQZmZWUb5DMoyKc3uwm16VaRyEX9g//4sWbo0hYjKU6pdvL239fRbBRhUXc2Hy5alUlcWP99OUJZJaXcXpjEQwIMAti7tv5mn32pbuQ9ycRefmZllUqbOoNLsIjAzs9KWqQTlUWBmZtasUwlK0njgZ0Av4FcRcXMqUZmlqBzvd5XmxXErTFqDbQC26V3BN+u/TqWuctbhBCWpF3APcALQBLwq6amIeDOt4MzSUI73u0rr4ri71Av3jQfudLvODJI4DHgvIt6PiH8AjwKnpROWmZn1dIqIjr1ROgsYHxH/I1k/Hzg8Ii7bZLuLgYuT1QOAtzsebovdgZUp1JMFPpZs6uixrIyIdp9qdVE7Af9NsqqnH0tB7aQz16DydcZulu0iYjIwuRP72XzH0tyIqE+zzmLxsWRTdx9LV7QT8N8kq3wshelMF18TsHer9Rrgk86FY2ZmltOZBPUqMERSraRtgXOBp9IJy8zMeroOd/FFxHpJlwHPkBtm/mBELEotsq1LvSukiHws2VQux1IuxwE+lqzqsmPp8CAJMzOzruS5+MzMLJOcoMzMLJMyn6Ak7S1plqTFkhZJ+kFS3k/STEnvJo+7FjvWtkiqlPSKpNeTY/lJUl4r6eXkWB5LBp1knqRekl6TNCNZL8njAJDUKOkNSQ2S5iZlJfMZczvJtnJpK93dTjKfoID1wJURcSBwBDBR0jDgWuC5iBgCPJesZ906YExEfAuoA8ZLOgK4BbgzOZbPgIuKGGN7/ABY3Gq9VI+j2XERUdfqNx2l9BlzO8m2cmor3ddOIqKkFuBJcvP/vQ1UJ2XVwNvFjq2dx7EDMB84nNyvsHsn5UcCzxQ7vgLir0k+jGOAGeR+uF1yx9HqeBqB3TcpK9nPmNtJdpZyaivd3U5K4QyqhaTBwAjgZWDPiFgKkDzuUbzICpec6jcAy4GZwF+BzyNifbJJEzCgWPG1w13A1cA3yfpulOZxNAvgj5LmJdMOQel+xgbjdpIl5dRWurWdZOp+UFsjqQ/wO2BSRHyR1rT33S0iNgB1knYBpgEH5tuse6NqH0knA8sjYp6k0c3FeTbN9HFsYlREfCJpD2CmpLeKHVBHuJ1kSxm2lW5tJyWRoCRVkGt0D0fEvyfFn0qqjoilkqrJfdMqGRHxuaTZ5K4X7CKpd/KNqhSmjBoFnCrpRKAS2Inct8RSO44WEfFJ8rhc0jRys/WX1GfM7SSTyqqtdHc7yXwXn3JfAR8AFkfEHa1eegqYkDyfQK7PPdMkVSXfCJG0PXA8uQuns4Czks0yfywRcV1E1ETEYHJTXD0fEedRYsfRTNKOkvo2PwfGAQspoc+Y20k2lVNbKUo7KfZFtwIuyh1N7vR3AdCQLCeS68d9Dng3eexX7FgLOJbhwGvJsSwEbkjK9wFeAd4DfgtsV+xY23FMo4EZpXwcSdyvJ8si4EdJecl8xtxOsr+UelspRjvxVEdmZpZJme/iMzOznskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJyszMMskJqsRJmp5M3LioefJGSRdJekfSbEm/lHR3Ul4l6XeSXk2WUcWN3qx7uJ2UJv9Qt8RJ6hcRf0umhHkV+C/Ai8BI4EvgeeD1iLhM0iPAvRExR9JAclP855uE06ysuJ2UppKYLNa26gpJZyTP9wbOB/5vRPwNQNJvgf2T148HhrWa4XonSX0j4svuDNisCNxOSpATVAlLpu8/HjgyItYksz6/Tf5bE0CuS/fIiPiqeyI0Kz63k9Lla1ClbWfgs6TRDSV3S4IdgG9L2lVSb+DMVtv/EbiseUVSXbdGa1YcbiclygmqtD0N9Ja0ALgReAn4GLiJ3N1UnwXeBP6ebH8FUC9pgaQ3gUu6P2Szbud2UqI8SKIMSeoTEauTb4bTgAcjYlqx4zLLEreT7PMZVHn6saQGcvfS+QCYXuR4zLLI7STjfAZlZmaZ5DMoMzPLJCcoMzPLJCcoMzPLJCcoMzPLJCcoMzPLpP8PlTlGZbaTvVAAAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "bins = np.linspace(df.age.min(), df.age.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'age', bins=bins, ec=\"k\")\n\ng.axes[-1].legend()\nplt.show()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Pre-processing:  Feature selection/extraction"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Lets look at the day of the week people get the loan "
        },
        {
            "cell_type": "code",
            "execution_count": 11,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAADQCAYAAABStPXYAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAGepJREFUeJzt3XmcVPW55/HPV2gvIriC2tIBWkQQldtgR+OCQUh4EdzwuoTEKGTMdTQuYQyDSzImN84YF8YlcSVq8EbEhUTMJTcaVIjgztKCiCFebbEVFJgYYxQFfeaPOt1poKGr6VPU6erv+/WqV1edOud3ntNdTz91fnXq91NEYGZmljU7FDsAMzOzprhAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlAmZlZJrlApUTS3pLuk/S6pAWSnpV0ckptD5U0M422tgdJcyRVFzsOK75SygtJ3SU9L2mRpCEF3M+HhWq7rXGBSoEkATOApyJiv4g4FBgDVBQpno7F2K9ZYyWYF8OBVyNiUETMTSMm2zoXqHQMAz6NiNvrF0TEmxHxcwBJHSRdJ+lFSYsl/fdk+dDkbGO6pFclTU2SGkkjk2XzgH+pb1fSzpLuTtpaJOmkZPk4SQ9J+g/gD605GElTJN0maXbyzvfLyT6XSZrSaL3bJM2XtFTSv22hrRHJu+aFSXxdWhObtSklkxeSqoBrgVGSaiTttKXXtqRaSVclz82XNFjSY5L+S9K5yTpdJD2RbLukPt4m9vs/G/1+msyxkhYRvrXyBlwE3LCV588Bfpjc/ydgPlAJDAX+Su4d5Q7As8DRQCfgLaAvIOBBYGay/VXAt5L7uwHLgZ2BcUAdsMcWYpgL1DRx+0oT604B7k/2fRLwAXBIEuMCoCpZb4/kZwdgDjAweTwHqAa6AU8BOyfLLwGuKPbfy7ftcyvBvBgH3Jzc3+JrG6gFzkvu3wAsBroC3YH3kuUdgV0atfUaoOTxh8nPEcDk5Fh3AGYCxxT777o9b+4KKgBJt5BLqE8j4ovkXmgDJZ2arLIruST7FHghIuqS7WqA3sCHwBsR8edk+b3kkpmkrRMlTUgedwJ6JvdnRcT/ayqmiGhpn/l/RERIWgK8GxFLkliWJjHWAKdLOodcspUDA8glY70vJcueTt4A70jun421QyWSF/Wae23/Nvm5BOgSEX8D/iZpnaTdgL8DV0k6Bvgc6AHsDaxq1MaI5LYoedyF3O/nqW2Muc1xgUrHUuCU+gcRcb6kbuTeEULuHdCFEfFY440kDQU+abToM/7xN9nSIIkCTomIP23S1uHkXvRNbyTNJfcublMTIuLxJpbXx/X5JjF+DnSUVAlMAL4YEX9Juv46NRHrrIj4xpbispJWinnReH9be21vNX+AM8idUR0aEesl1dJ0/vw0Iu7YShwlzZ9BpeNJoJOk8xot69zo/mPAeZLKACQdIGnnrbT3KlApqU/yuHESPAZc2KhPflA+AUbEkIioauK2tSTcml3IJf5fJe0NfK2JdZ4DjpK0fxJrZ0kHbOP+rO0p5bxo7Wt7V3LdfeslHQv0amKdx4D/1uizrR6S9mrBPto8F6gURK7DeDTwZUlvSHoBuIdcvzTAncArwEJJLwN3sJWz14hYR67r4nfJh8FvNnr6SqAMWJy0dWXax5OPiHiJXNfDUuBu4Okm1llNrt9+mqTF5JK6/3YM04qolPMihdf2VKBa0nxyZ1OvNrGPPwD3Ac8mXe3Tafpsr2TVfyhnZmaWKT6DMjOzTHKBMjOzTHKBMjOzTHKBMjOzTNquBWrkyJFB7nsMvvlWqrdWc5741g5uedmuBWrNmjXbc3dmbZLzxCzHXXxmZpZJLlBmZpZJLlBmZpZJHizWzErO+vXrqaurY926dcUOpV3r1KkTFRUVlJWVbdP2LlBmVnLq6uro2rUrvXv3Jhk/1raziGDt2rXU1dVRWVm5TW24i8/MSs66devYc889XZyKSBJ77rlnq85iXaCs5PUqL0dSq2+9ysuLfSjWAi5Oxdfav4G7+KzkrVi1irp9K1rdTsU7dSlEY2b58hmUmZW8tM6iW3I23aFDB6qqqjj44IM57bTT+Oijjxqee/jhh5HEq6/+Yxqo2tpaDj74YADmzJnDrrvuyqBBg+jXrx/HHHMMM2fO3Kj9yZMn079/f/r3789hhx3GvHnzGp4bOnQo/fr1o6qqiqqqKqZPn75RTPW32tra1vxaCy6vMyhJ/wP4DrkhKpYA3wbKgfuBPYCFwJkR8WmB4jQz22ZpnUXXy+dseqeddqKmpgaAM844g9tvv52LL74YgGnTpnH00Udz//338+Mf/7jJ7YcMGdJQlGpqahg9ejQ77bQTw4cPZ+bMmdxxxx3MmzePbt26sXDhQkaPHs0LL7zAPvvsA8DUqVOprq7eYkxtQbNnUJJ6ABcB1RFxMNABGANcA9wQEX2BvwBnFzJQM7O2asiQIbz22msAfPjhhzz99NPcdddd3H///XltX1VVxRVXXMHNN98MwDXXXMN1111Ht27dABg8eDBjx47llltuKcwBFEm+XXwdgZ0kdQQ6AyuBYeSmIIbcNM6j0w/PzKxt27BhA7///e855JBDAJgxYwYjR47kgAMOYI899mDhwoV5tTN48OCGLsGlS5dy6KGHbvR8dXU1S5cubXh8xhlnNHTlrV27FoCPP/64YdnJJ5+cxuEVVLNdfBHxtqRJwArgY+APwALg/YjYkKxWB/RoantJ5wDnAPTs2TONmM1KjvOk9NQXA8idQZ19dq6Tadq0aYwfPx6AMWPGMG3aNAYPHtxsexFbHwQ8Ija6aq4UuviaLVCSdgdOAiqB94GHgK81sWqTv72ImAxMBqiurs57mHWz9sR5UnqaKgZr167lySef5OWXX0YSn332GZK49tprm21v0aJFHHjggQAMGDCABQsWMGzYsIbnFy5cyIABA9I9iCLLp4vvK8AbEbE6ItYDvwGOBHZLuvwAKoB3ChSjmVlJmD59OmeddRZvvvkmtbW1vPXWW1RWVm50BV5TFi9ezJVXXsn5558PwMSJE7nkkksauu5qamqYMmUK3/3udwt+DNtTPlfxrQC+JKkzuS6+4cB8YDZwKrkr+cYCjxQqSDOz1ui5zz6pfo+tZ3KlXEtNmzaNSy+9dKNlp5xyCvfddx+XXHLJRsvnzp3LoEGD+Oijj9hrr7342c9+xvDhwwE48cQTefvttznyyCORRNeuXbn33nspL7Evk6u5fk0ASf8GfB3YACwid8l5D/5xmfki4FsR8cnW2qmuro758+e3NmazFpGU2hd188iXVg9f4DxpvWXLljV0h1lxbeFvkVee5PU9qIj4EfCjTRa/DhyWz/ZmZmYt5ZEkzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzKzk7VvRM9XpNvatyG84qlWrVjFmzBj69OnDgAEDGDVqFMuXL2fp0qUMGzaMAw44gL59+3LllVc2fIVhypQpXHDBBZu11bt3b9asWbPRsilTptC9e/eNptB45ZVXAFi+fDmjRo1i//3358ADD+T000/ngQceaFivS5cuDVNynHXWWcyZM4fjjz++oe0ZM2YwcOBA+vfvzyGHHMKMGTManhs3bhw9evTgk09y3yxas2YNvXv3btHfJB+esNDMSt7Kt9/i8CseTa29538ystl1IoKTTz6ZsWPHNoxaXlNTw7vvvsu4ceO47bbbGDFiBB999BGnnHIKt956a8NIES3x9a9/vWGU83rr1q3juOOO4/rrr+eEE04AYPbs2XTv3r1h+KWhQ4cyadKkhvH65syZ07D9Sy+9xIQJE5g1axaVlZW88cYbfPWrX2W//fZj4MCBQG5uqbvvvpvzzjuvxTHny2dQZmYFMHv2bMrKyjj33HMbllVVVbF8+XKOOuooRowYAUDnzp25+eabufrqq1Pb93333ccRRxzRUJwAjj322IYJEZszadIkLr/8ciorKwGorKzksssu47rrrmtYZ/z48dxwww1s2LBhS820mguUmVkBvPzyy5tNiQFNT5XRp08fPvzwQz744IMW76dxt11VVRUff/zxFvedr3ym8+jZsydHH300v/rVr7Z5P81xF5+Z2Xa06bQYjW1p+dY01cXXWk3F2NSyyy+/nBNPPJHjjjsu1f3X8xmUmVkBHHTQQSxYsKDJ5ZuOtfj666/TpUsXunbtWtB9t2T7TWNsajqP/fffn6qqKh588MFt3tfWuECZmRXAsGHD+OSTT/jFL37RsOzFF1+kb9++zJs3j8cffxzITWx40UUXMXHixNT2/c1vfpNnnnmG3/3udw3LHn30UZYsWZLX9hMmTOCnP/0ptbW1ANTW1nLVVVfx/e9/f7N1f/CDHzBp0qRU4t6Uu/jMrOSV9/hCXlfetaS95kji4YcfZvz48Vx99dV06tSJ3r17c+ONN/LII49w4YUXcv755/PZZ59x5plnbnRp+ZQpUza6rPu5554DYODAgeywQ+684vTTT2fgwIE88MADG80ndeutt3LkkUcyc+ZMxo8fz/jx4ykrK2PgwIHcdNNNeR1fVVUV11xzDSeccALr16+nrKyMa6+9tmGG4MYOOuggBg8enPfU9S2R13QbafE0AlYMnm6j/fF0G9nRmuk23MVnZmaZlKkC1au8PLVvevcqsZklzczam0x9BrVi1apUumKAVKd3NrO2Z2uXc9v20dqPkDJ1BmVmloZOnTqxdu3aVv+DtG0XEaxdu5ZOnTptcxuZOoMyM0tDRUUFdXV1rF69utihtGudOnWiomLbe8VcoMys5JSVlTWMI2dtl7v4zMwsk1ygzMwsk1ygzMwsk1ygzMwsk1ygzMwsk/IqUJJ2kzRd0quSlkk6QtIekmZJ+nPyc/dCB2tmZu1HvmdQNwGPRkR/4J+BZcClwBMR0Rd4InlsZmaWimYLlKRdgGOAuwAi4tOIeB84CbgnWe0eYHShgjQzs/YnnzOo/YDVwC8lLZJ0p6Sdgb0jYiVA8nOvpjaWdI6k+ZLm+1vdZk1znphtLp8C1REYDNwWEYOAv9OC7ryImBwR1RFR3b17920M06y0OU/MNpdPgaoD6iLi+eTxdHIF611J5QDJz/cKE6KZmbVHzRaoiFgFvCWpX7JoOPAK8FtgbLJsLPBIQSI0M7N2Kd/BYi8EpkraEXgd+Da54vagpLOBFcBphQnRrHXUoSyV+cHUoSyFaMwsX3kVqIioAaqbeGp4uuGYpS8+W8/hVzza6nae/8nIFKIxs3x5JAkzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8skFygzM8ukvAuUpA6SFkmamTyulPS8pD9LekDSjoUL08zM2puWnEF9D1jW6PE1wA0R0Rf4C3B2moGZmVn7lleBklQBHAfcmTwWMAyYnqxyDzC6EAGamVn7lO8Z1I3ARODz5PGewPsRsSF5XAf0aGpDSedImi9p/urVq1sVrFmpcp6Yba7ZAiXpeOC9iFjQeHETq0ZT20fE5Iiojojq7t27b2OYZqXNeWK2uY55rHMUcKKkUUAnYBdyZ1S7SeqYnEVVAO8ULkwzM2tvmj2DiojLIqIiInoDY4AnI+IMYDZwarLaWOCRgkVpZmbtTmu+B3UJcLGk18h9JnVXOiGZmZnl18XXICLmAHOS+68Dh6UfkpmZmUeSMDOzjHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKBMjOzTHKB2o56lZcjKZVbr/LyYh+OmVlBtWg+KGudFatWUbdvRSptVbxTl0o7ZmZZ5TMoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLJBcoMzPLpGYLlKQvSJotaZmkpZK+lyzfQ9IsSX9Ofu5e+HDNzKy9yOcMagPw/Yg4EPgScL6kAcClwBMR0Rd4InlsZmaWimYLVESsjIiFyf2/AcuAHsBJwD3JavcAowsVpJmZtT8t+gxKUm9gEPA8sHdErIRcEQP22sI250iaL2n+6tWrWxetWYlynphtLu8CJakL8GtgfER8kO92ETE5Iqojorp79+7bEqNZyXOemG0urwIlqYxccZoaEb9JFr8rqTx5vhx4rzAhmplZe5TPVXwC7gKWRcT1jZ76LTA2uT8WeCT98MzMrL3KZ8LCo4AzgSWSapJllwNXAw9KOhtYAZxWmBDNzKw9arZARcQ8QFt4eni64ZiZWTH0Ki9nxapVqbTVc599eHPlyla34ynfzcyMFatWUbdvRSptVbxTl0o7HurIMqlXeTmSUrmVorR+P73Ky4t9KGZb5DMoy6QsvpvLkrR+P6X4u7HS4TMoMzPLpJI9g/onSK17J60P/Cx/6lDmd/dm7VzJFqhPwF1EbVh8tp7Dr3g0lbae/8nIVNoxs+3LXXxmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJLlBmZpZJJTvUkZmZ5S/N8S/VoSyVdlygzMwsk+NfuovPrB2rH/Xfkx9aFvkMyqwd86j/lmU+gzIzs0xygbLU7FvRM7XuIjMzd/FZala+/VbmPmQ1s7YrUwUqi5c5mtn216u8nBWrVrW6nZ777MObK1emEJEVQ6YKVBYvc8yq+quv0uAktqxZsWpVKhdv+MKNtq1VBUrSSOAmoANwZ0RcnUpU1ixffWVmpW6bL5KQ1AG4BfgaMAD4hqQBaQVmZtZaWf2eV6/y8lRi6tyhY0lfmNSaM6jDgNci4nUASfcDJwGvpBGYmVlrZbWnIc0uzCweX1oUEdu2oXQqMDIivpM8PhM4PCIu2GS9c4Bzkof9gD9tpdluwJptCqht8PG1bfkc35qIaPEHoC3Mk3xjact8fG1bc8eXV5605gyqqXPCzapdREwGJufVoDQ/IqpbEVOm+fjatkIeX0vypNCxZIGPr21L6/ha80XdOuALjR5XAO+0LhwzM7Oc1hSoF4G+kiol7QiMAX6bTlhmZtbebXMXX0RskHQB8Bi5y8zvjoilrYwn7y6ONsrH17Zl6fiyFEsh+PjatlSOb5svkjAzMyskDxZrZmaZ5AJlZmaZlJkCJWmkpD9Jek3SpcWOJ02SviBptqRlkpZK+l6xY0qbpA6SFkmaWexYCkHSbpKmS3o1+TseUaQ4nCdtXCnnStp5konPoJJhk5YDXyV3+fqLwDcioiRGpZBUDpRHxEJJXYEFwOhSOT4ASRcD1cAuEXF8seNJm6R7gLkRcWdy1WrniHh/O8fgPCkBpZwraedJVs6gGoZNiohPgfphk0pCRKyMiIXJ/b8By4AexY0qPZIqgOOAO4sdSyFI2gU4BrgLICI+3d7FKeE8aeNKOVcKkSdZKVA9gLcaPa6jxF6Y9ST1BgYBzxc3klTdCEwEPi92IAWyH7Aa+GXSNXOnpJ2LEIfzpO0r5VxJPU+yUqDyGjaprZPUBfg1MD4iPih2PGmQdDzwXkQsKHYsBdQRGAzcFhGDgL8Dxfj8x3nShrWDXEk9T7JSoEp+2CRJZeSSbmpE/KbY8aToKOBESbXkupyGSbq3uCGlrg6oi4j6d/PTySViMeJwnrRdpZ4rqedJVgpUSQ+bpNxkK3cByyLi+mLHk6aIuCwiKiKiN7m/25MR8a0ih5WqiFgFvCWpX7JoOMWZVsZ50oaVeq4UIk8yMeV7gYZNypKjgDOBJZJqkmWXR8R/FjEma5kLgalJYXgd+Pb2DsB5Ym1AqnmSicvMzczMNpWVLj4zM7ONuECZmVkmuUCZmVkmuUCZmVkmuUCZmVkmuUBlgKQfS5qQYnv9JdUkw430SavdRu3PkVSddrtmW+M8aX9coErTaOCRiBgUEf9V7GDMMsp5knEuUEUi6QfJvD6PA/2SZf8q6UVJL0n6taTOkrpKeiMZAgZJu0iqlVQmqUrSc5IWS3pY0u6SRgHjge8kc+tMlHRRsu0Nkp5M7g+vH2ZF0ghJz0paKOmhZCw0JB0q6Y+SFkh6LJkOofEx7CDpHkn/e7v94qxdcZ60by5QRSDpUHJDnQwC/gX4YvLUbyLiixHxz+SmGjg7mXZgDrkh+km2+3VErAf+HbgkIgYCS4AfJd+6vx24ISKOBZ4ChiTbVgNdkiQ+GpgrqRvwQ+ArETEYmA9cnKzzc+DUiDgUuBv4P40OoyMwFVgeET9M8ddjBjhPLCNDHbVDQ4CHI+IjAEn146kdnLzL2g3oQm5IG8jNHTMRmEFu6JB/lbQrsFtE/DFZ5x7goSb2tQA4VLkJ4D4BFpJLwCHARcCXgAHA07mh0NgReJbcu9WDgVnJ8g7Aykbt3gE8GBGNk9EsTc6Tds4FqniaGmNqCrkZRF+SNA4YChART0vqLenLQIeIeDlJvOZ3ErFeudGTvw08AywGjgX6kHv32QeYFRHfaLydpEOApRGxpSmbnwGOlfR/I2JdPrGYbQPnSTvmLr7ieAo4WdJOyTu2E5LlXYGVSbfBGZts8+/ANOCXABHxV+Avkuq7Jc4E/kjTngImJD/nAucCNZEbiPE54ChJ+wMk/fkHAH8Cuks6IlleJumgRm3eBfwn8JAkv9GxQnCetHMuUEWQTGv9AFBDbu6buclT/4vcDKKzgFc32WwqsDu55Ks3FrhO0mKgCvjJFnY5FygHno2Id4F19fuMiNXAOGBa0s5zQP9kSvFTgWskvZTEeuQmx3E9ua6QX0nya8lS5Twxj2beRkg6FTgpIs4sdixmWeU8KS0+5WwDJP0c+BowqtixmGWV86T0+AzKzMwyyf2hZmaWSS5QZmaWSS5QZmaWSS5QZmaWSS5QZmaWSf8feZ3K8s9z83MAAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x216 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "df['dayofweek'] = df['effective_date'].dt.dayofweek\nbins = np.linspace(df.dayofweek.min(), df.dayofweek.max(), 10)\ng = sns.FacetGrid(df, col=\"Gender\", hue=\"loan_status\", palette=\"Set1\", col_wrap=2)\ng.map(plt.hist, 'dayofweek', bins=bins, ec=\"k\")\ng.axes[-1].legend()\nplt.show()\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "We see that people who get the loan at the end of the week dont pay it off, so lets use Feature binarization to set a threshold values less then day 4 "
        },
        {
            "cell_type": "code",
            "execution_count": 12,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>male</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>female</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>male</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below    male          3        0  \n1 2016-10-07   33              Bechalor  female          3        0  \n2 2016-09-22   27               college    male          3        0  \n3 2016-10-08   28               college  female          4        1  \n4 2016-10-08   29               college    male          4        1  "
                    },
                    "execution_count": 12,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['weekend'] = df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## Convert Categorical features to numerical values"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets look at gender:"
        },
        {
            "cell_type": "code",
            "execution_count": 13,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "Gender  loan_status\nfemale  PAIDOFF        0.865385\n        COLLECTION     0.134615\nmale    PAIDOFF        0.731293\n        COLLECTION     0.268707\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 13,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "86 % of female pay there loans while only 73 % of males pay there loan\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets convert male to 0 and female to 1:\n"
        },
        {
            "cell_type": "code",
            "execution_count": 14,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>0</td>\n      <td>0</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>45</td>\n      <td>High School or Below</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>2</td>\n      <td>2</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>33</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>3</td>\n      <td>3</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>15</td>\n      <td>2016-09-08</td>\n      <td>2016-09-22</td>\n      <td>27</td>\n      <td>college</td>\n      <td>0</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>4</td>\n      <td>4</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>28</td>\n      <td>college</td>\n      <td>1</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>6</td>\n      <td>6</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-09</td>\n      <td>2016-10-08</td>\n      <td>29</td>\n      <td>college</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           0             0     PAIDOFF       1000     30     2016-09-08   \n1           2             2     PAIDOFF       1000     30     2016-09-08   \n2           3             3     PAIDOFF       1000     15     2016-09-08   \n3           4             4     PAIDOFF       1000     30     2016-09-09   \n4           6             6     PAIDOFF       1000     30     2016-09-09   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   45  High School or Below       0          3        0  \n1 2016-10-07   33              Bechalor       1          3        0  \n2 2016-09-22   27               college       0          3        0  \n3 2016-10-08   28               college       1          4        1  \n4 2016-10-08   29               college       0          4        1  "
                    },
                    "execution_count": 14,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ndf.head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## One Hot Encoding  \n#### How about education?"
        },
        {
            "cell_type": "code",
            "execution_count": 15,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "education             loan_status\nBechalor              PAIDOFF        0.750000\n                      COLLECTION     0.250000\nHigh School or Below  PAIDOFF        0.741722\n                      COLLECTION     0.258278\nMaster or Above       COLLECTION     0.500000\n                      PAIDOFF        0.500000\ncollege               PAIDOFF        0.765101\n                      COLLECTION     0.234899\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 15,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "#### Feature befor One Hot Encoding"
        },
        {
            "cell_type": "code",
            "execution_count": 16,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>education</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>High School or Below</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>Bechalor</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender             education\n0       1000     30   45       0  High School or Below\n1       1000     30   33       1              Bechalor\n2       1000     15   27       0               college\n3       1000     30   28       1               college\n4       1000     30   29       0               college"
                    },
                    "execution_count": 16,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "df[['Principal','terms','age','Gender','education']].head()"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "#### Use one hot encoding technique to conver categorical varables to binary variables and append them to the feature Data Frame "
        },
        {
            "cell_type": "code",
            "execution_count": 17,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    },
                    "execution_count": 17,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "Feature = df[['Principal','terms','age','Gender','weekend']]\nFeature = pd.concat([Feature,pd.get_dummies(df['education'])], axis=1)\nFeature.drop(['Master or Above'], axis = 1,inplace=True)\nFeature.head()\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Feature selection"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Lets defind feature sets, X:"
        },
        {
            "cell_type": "code",
            "execution_count": 18,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>45</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>33</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>15</td>\n      <td>27</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>28</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   45       0        0         0                     1   \n1       1000     30   33       1        0         1                     0   \n2       1000     15   27       0        0         0                     0   \n3       1000     30   28       1        1         0                     0   \n4       1000     30   29       0        1         0                     0   \n\n   college  \n0        0  \n1        0  \n2        1  \n3        1  \n4        1  "
                    },
                    "execution_count": 18,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X = Feature\nX[0:5]"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "What are our lables?"
        },
        {
            "cell_type": "code",
            "execution_count": 19,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n      dtype=object)"
                    },
                    "execution_count": 19,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "y = df['loan_status'].values\ny[0:5]"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "## Normalize Data "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Data Standardization give data zero mean and unit variance (technically should be done after train test split )"
        },
        {
            "cell_type": "code",
            "execution_count": 20,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  return self.partial_fit(X, y)\n/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:1: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  if __name__ == '__main__':\n"
                },
                {
                    "data": {
                        "text/plain": "array([[ 0.51578458,  0.92071769,  2.33152555, -0.42056004, -1.20577805,\n        -0.38170062,  1.13639374, -0.86968108],\n       [ 0.51578458,  0.92071769,  0.34170148,  2.37778177, -1.20577805,\n         2.61985426, -0.87997669, -0.86968108],\n       [ 0.51578458, -0.95911111, -0.65321055, -0.42056004, -1.20577805,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.48739188,  2.37778177,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679],\n       [ 0.51578458,  0.92071769, -0.3215732 , -0.42056004,  0.82934003,\n        -0.38170062, -0.87997669,  1.14984679]])"
                    },
                    "execution_count": 20,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "X= preprocessing.StandardScaler().fit(X).transform(X)\nX[0:5]"
        },
        {
            "cell_type": "code",
            "execution_count": 21,
            "metadata": {},
            "outputs": [],
            "source": "def comparelist(list1,list2):\n    '''\n    for list list1 and list2\n    combined two lists into one list\n    '''\n    result = []\n    for i in range(len(list1)):\n        result.append((list1[i],list2[i],list1[i] == list2[i]))\n    return result\n"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "# Classification "
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "Now, it is your turn, use the training set to build an accurate model. Then use the test set to report the accuracy of the model\nYou should use the following algorithm:\n- K Nearest Neighbor(KNN)\n- Decision Tree\n- Support Vector Machine\n- Logistic Regression\n\n\n\n__ Notice:__ \n- You can go above and change the pre-processing, feature selection, feature-extraction, and so on, to make a better model.\n- You should use either scikit-learn, Scipy or Numpy libraries for developing the classification algorithms.\n- You should include the code of the algorithm in the following cells."
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# K Nearest Neighbor(KNN)\nNotice: You should find the best k to build the model with the best accuracy.  \n**warning:** You should not use the __loan_test.csv__ for finding the best k, however, you can split your train_loan.csv into train and test to find the best __k__."
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Steps:\n+ Train Test Split\n+ Import library\n+ Classifier implementing the k-nearest neighbors vote.(start with k=4)\n+ Pridicting\n+ Accuracy evaluation\n+ What about other K? find best K and try again"
        },
        {
            "cell_type": "code",
            "execution_count": 22,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "(346, 8)\nTrain set: (276, 8) (276,)\nTest set: (70, 8) (70,)\n"
                }
            ],
            "source": "# Train Test Split\nprint(X.shape)\nfrom sklearn.model_selection import train_test_split\nX_train, X_test, y_train, y_test = train_test_split( X, y, test_size=0.2, random_state=4)\nprint ('Train set:', X_train.shape,  y_train.shape)\nprint ('Test set:', X_test.shape,  y_test.shape)\n"
        },
        {
            "cell_type": "code",
            "execution_count": 23,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n           metric_params=None, n_jobs=None, n_neighbors=4, p=2,\n           weights='uniform')\n['PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF']\nTrain set Accuracy:  0.8152173913043478\nTest set Accuracy:  0.6857142857142857\n"
                }
            ],
            "source": "\nfrom sklearn.neighbors import KNeighborsClassifier\n\nk = 4\n\n#Train Model and Predict  \nneigh = KNeighborsClassifier(n_neighbors = k).fit(X_train,y_train)\nprint(neigh)\n\n# Predicting\nyhat = neigh.predict(X_test)\nprint(yhat[0:5])\n\n# Accuracy evaluation\nfrom sklearn import metrics\nprint(\"Train set Accuracy: \", metrics.accuracy_score(y_train, neigh.predict(X_train)))\nprint(\"Test set Accuracy: \", metrics.accuracy_score(y_test, yhat))"
        },
        {
            "cell_type": "code",
            "execution_count": 24,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "[0.67142857 0.65714286 0.71428571 0.68571429 0.75714286 0.71428571\n 0.78571429 0.75714286 0.75714286 0.67142857 0.7        0.72857143\n 0.7        0.7        0.68571429 0.72857143 0.72857143 0.72857143\n 0.7       ]\n"
                },
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJztnXl8VOXVx79nliQkJMjmQhBBAQUFAcMiWEWpCL5WRfsqaq1o1bogqK0V+9pqW1uXWvettCpqFepKbZVFAReosikCsggiyuKCIEsg28w87x8zE4cwk9xZ7tw7k/P9fOaT5M5dzkzuveee8/yec8QYg6IoiqK4DY/TBiiKoihKPNRBKYqiKK5EHZSiKIriStRBKYqiKK5EHZSiKIriStRBKYqiKK5EHZSiKIriStRBKYqiKK5EHZSiKIriSnxOG5Ap2rVrZzp37uy0GYqiKEoTLF68+FtjTPum1ssbB9W5c2cWLVrktBmKoihKE4jI51bW0xSfoiiK4krUQSmKoiiuRB2UoiiK4kryZgxKURQlXerq6ti4cSPV1dVOm5IXFBUV0bFjR/x+f0rbq4NSFEWJsHHjRkpLS+ncuTMi4rQ5OY0xhq1bt7Jx40a6dOmS0j40xacoihKhurqatm3bqnPKACJC27Zt04pG1UEpiqLEoM4pc6T7XaqDUhRFUVyJOigl5wkEQ06boCgZ5ZVXXkFEWLVqldOmOIo6KCWnCQRDfLenjlDIOG2KomSMyZMnc9xxxzFlyhRbjxMMBm3df7qoik/JaWoCIULGsL2qjjYlBU6bo+QR106/liVfLcnoPvsc2If7RtzX6DqVlZXMmzePOXPmcPrpp3PrrbfWv3fXXXfxzDPP4PF4GDlyJHfccQdr167liiuuYMuWLXi9Xl544QU2bNjA3XffzX/+8x8Axo4dS0VFBWPGjKFz585ccsklzJw5k7Fjx7Jr1y4mTpxIbW0tXbt25ZlnnqG4uJivv/6aK664gnXr1gHw6KOPMm3aNNq1a8f48eMB+L//+z8OOOAAxo0bl9HvKYo6KCWnqQmE03t1wRC7qusoLUptvoWiuIWpU6cyYsQIunfvTps2bfjggw/o168f06ZNY+rUqcyfP5/i4mK2bdsGwAUXXMCECRMYNWoU1dXVhEIhNmzY0OgxioqKmDt3LgBbt27lsssuA+Dmm2/m8ccf55prrmHcuHGccMIJvPLKKwSDQSorK+nQoQNnnXUW48ePJxQKMWXKFBYsWGDbd6EOSslZQiFDXcz4057aIH6vhyK/10GrlHyhqUjHLiZPnsy1114LwOjRo5k8eTL9+vXjzTff5OKLL6a4uBiANm3asGvXLjZt2sSoUaOAsOOxwrnnnlv/+/Lly7n55pvZvn07lZWVnHLKKQDMnj2bp59+GgCv10urVq1o1aoVbdu25cMPP+Trr7+mb9++tG3bNmOfvSHqoJScJRo9xbKzqg6fR/B5dXhVyT22bt3K7NmzWb58OSJCMBhERLjrrrswxuwj2zYm/tirz+cjFPr++mg4F6mkpKT+9zFjxjB16lSOPvpoJk2axFtvvdWojZdeeimTJk3iq6++4pJLLknyEyaHXsVKzlIT2HeA1wDbq+oSXriK4mZefPFFfvrTn/L555+zfv16NmzYQJcuXZg7dy7Dhw/niSeeYM+ePQBs27aNsrIyOnbsyNSpUwGoqalhz549HHLIIaxYsYKamhp27NjBrFmzEh5z165dHHTQQdTV1fHss8/WLx82bBiPPvooEBZT7Ny5E4BRo0Yxffp0Fi5cWB9t2YU6KCUnMcZQGyeCAgiGDDurAlm2SFHSZ/LkyfXpuihnn302zz33HCNGjOD000+noqKCPn36cPfddwPwzDPP8MADD9C7d28GDx7MV199xcEHH8w555xD7969ueCCC+jbt2/CY/7hD39g4MCBnHzyyRxxxBH1y++//37mzJlDr169OOaYY/j4448BKCgo4MQTT+Scc87B67U3nS758qRZUVFhtGFh86G6LsiOqrpG1ykt8lFcoFlsxTorV66kR48eTpvhakKhEP369eOFF16gW7duTa4f7zsVkcXGmIqmttUISslJ4o0/NaSyOpAwylIUJXlWrFhB165dGTZsmCXnlC76eKnkJPHGnxpigB1VdbQtKcDj0fpqipIuPXv2rJ8XlQ00glJyjtpACKuZ6ZAxTaYCFUVxJ+qglJzDSvQUS20wRGWNiiYUJddQB6XkHFbGnxqyuyZAdZ27644pirI36qCUnCIQDBFMsTDszuq6lLdVFCX7qINScopUoqcoxsD2PbU6iVexzNc7qzP6yhTvvfdeff28RDz22GP06tWLPn36cNxxx7FixYqkj7N+/Xqee+65hO8PHToUO6f3qINScop0HBRAIGTYWa3jUYr7eeuttxgzZkzc96ZPn86IESMa3f78889n2bJlLFmyhF/96ldcf/31SdvQlIOyG3VQSs7QsDhsqlTXBamq1fEoJXeZNWsWP/zhDxtdp6ysrP733bt319fxu+eee+pr6C1btoyjjjqKPXv28Pbbb9OnTx/69OlD37592bVrFxMmTODdd9+lT58+3HvvvVRVVTF69Gh69+7NueeeS1VVlX0fEp0HpeQQ6UZPseyqrsPnFfxaVFbJMb799lv8fj+tWrVqct2HH36Ye+65h9raWmbPng3Atddey9ChQ3nllVf44x//yF//+leKi4u5++67efjhhxkyZAiVlZUUFRVxxx137NVX6p577qG4uJilS5eydOlS+vXrZ+tn1atTyRmSlZc3hgG2aydexYUMHDiQPn36cOmll/Lqq6/WRzUzZswAYObMmQwfPtzSvq6++mo+/fRT7rzzTm677TYAPB4PkyZN4sILL+SEE05gyJAhAAwZMoTrr7+eBx54gO3bt+Pz7Ru/vPPOO/zkJz8BoHfv3vTu3TsTHzkh6qCUnKCx4rCpEjKGndU6iVdxF/Pnz2fJkiX8/e9/5/TTT2fJkiUsWbKkvnL4tGnT6sefLr74Yvr06cOpp57a6D5Hjx5dX/EcYM2aNbRs2ZLNmzfXL5swYQJ///vfqaqqYtCgQaxatSruvhq2/LATdVBKTlATCGFHrFMT0Em8Su5gjGHp0qX06dMHgCeffJIlS5bw+uuv77PumjVr6n9/7bXX6mvn7dixg/Hjx/POO++wdetWXnzxRQA+/fRTevXqxY033khFRQWrVq2itLSUXbt21e/n+OOPr2/JsXz5cpYuXWrbZwUdg1JyhEyOPzVkd00Av1co9GknXmVvDiiz1qE2WyxevJi+fftaimIeeugh3nzzTfx+P61bt+app54C4LrrruOqq66ie/fuPP7445x44okcf/zx3HfffcyZMwev10vPnj0ZOXIkHo8Hn8/H0UcfzZgxY7jyyiu5+OKL6d27N3369GHAgAG2fl5tt6HkBFt21RCy8VwVgbYlhXi1qGyzxu3tNm677Ta6du3K6NGjnTbFMum029AISkmKmkAw65FGbSBkq3OC8CTerbtr8GQov962pCCruXqleXDzzTc7bUJWUQelWMZEKoO3KRZ8WZRnZ1K91xjGQDBDjrAuaCjwqYNSlHRQkYRimdpguM3F7prsTnK1c/zJLmozMKFYcYZ8GfZwA+l+l+qgFMtEHUV1IEggSzfgdIrDOol28s1NioqK2Lp1qzqpDGCMYevWrRQVpS400RSfYpmauu9vurtrgrQqtv/5JhejJwg7VmOMjkPlGB07dmTjxo1s2bLFaVPygqKiIjp27Jjy9rY6KBEZAdwPeIG/G2PuaPB+J+ApYL/IOhOMMa9H3rsJ+BkQBMYZY2bYaavSOHXBvYUK1YEgJUGv7WNRueqgDOE0n0rXcwu/30+XLl2cNkOJYNvdRUS8wMPASKAncJ6I9Gyw2s3A88aYvsBo4JHItj0jfx8JjAAeiexPcYh4jsLuCa6ZKg7rFHVBTRMpSjrY+fg7AFhrjFlnjKkFpgBnNFjHANGSu62AaN2NM4ApxpgaY8xnwNrI/hSHqInTjbYmELLVgeS60EDHoRQlPex0UOXAhpi/N0aWxXIr8BMR2Qi8DlyTxLaIyOUiskhEFmnO2D6CIUMggVBht41RVK63aI+OQymKkhp2Oqh4o8MNr9bzgEnGmI7AqcAzIuKxuC3GmInGmApjTEX79u3TNliJT2ORgF1RlB3FYbNNdBxKUZTUsNNBbQQOjvm7I9+n8KL8DHgewBjzHlAEtLO4rZIlmpooa0cUVRu0pzhstsl1J6soTmKng1oIdBORLiJSQFj08GqDdb4AhgGISA/CDmpLZL3RIlIoIl2AbsACG21VEmAlkqkJhDJ+I85V9V5DVCihKKljm8zcGBMQkbHADMIS8ieMMR+LyO+BRcaYV4FfAH8TkesIZ0TGmHDS/mMReR5YAQSAq40xuT0gkaNYbXOxuyZAga8gc8etyw8HpfOhFCV1tJq50ig7quosixVaFxdQ4Es/KK8Lhti2uzbt/biF/Yr9Oh9KUWKwWs1cSx0pjZJModZMjUXlS3ovio5DKUpqqINSElIbCBeHtbx+MJSRyuPx5lzlMjoOpSipoQ5KSUgqzibdSueNzbnKVep0PpSipIQ6KCUhqaTa6tKMorLV+ynb5FvaUlGygTooJS7BkEm5zUU6UVS+qPcakss1BRXFKdRBKXFJJ5JJNYoKhUzSlRdqAjVsq9qa9LGyjQolFCV51EEpcUk3kqmsTl7Rl0pZoD/OvZnBT/Ziw87Pk942mwRCRsehFCVJ1EEp+5BKJNOQQMgkXew1WadojOH1T19le813XDP9ZwRD7h6/0nEoRUkOdVDKPmSqwGky86KMMUmnBddsW83GnV9wQqdhvL9pHg8s/HOyJmYVHYdSlORQB6XsQ6aECslEUakUh529Ptxk+Z7hjzLq8HO4+73bWPzl/CT3kj10HEpRksPWlu9KblITzFyqrLImQJG/6TI/qaS/Zq+fyeFte1JeejB3nHQ/C798n6umXcysn8ynZUFpKubaSiBkCIUMHk/u1uWLRrlauqlxdlTVuS5ibuH3UlKYW7d8jaCUvagJBJOqHtEUQYtRVLJR2+7aSt7fNJeTOg8HoFXRfjw84gk27PycX8+5PiVbs0Eu94cKhgw7qurYUVWX8hSE5kBtIER1XbB+qoZbXpU1gZyL4tVBKXthx0B+ZRNjUXXBEKEkveLcDW9RG6xlWOdT6pcNLB/C+AE38vyKfzB19Qsp2Wo3ueygdlTVYQwYE/5diY+dXabTZUdVHaEcerhQB6XshR0TZZuKolJxirPWz6TE35IB5YP3Wv6LQb/mmIMG8KtZ17hSel6XY0+wUXZV752yqguG2FWtTqohtYGQqx9CQsbk1MOFOiilnkAKkYxVGouiki0Oa4xh9voZHN/pRAq8e/eg8nl8PDJiEiETcqX0PDoOlUtU1wXZU7vv97inNpj0VIJ8x83RU5TaYKjJrIZbUAel1GPnPJ1gyFAV5yaXSnHYqLz8pJj0XiyH7NeF20+8z7XSczc/YTckGDLsbCRS2lmt41FR3B49xbK7JpATdS/VQSn12D2RNN5TWyoXSVReflKX4QnX+XGP81wrPc+Vm5iJpIMaC6qNge17arVKBrkRPcWSC2IXdVAKEK4eYbcsNmT2jaJSGfOa9dmMenl5IkSEO4c9wEGl5Vw17WJ21exM+jh2kStKql01AUvnRCBk2JVjN+dMk0vRU5RcELuog1KA7JXhqawJ1D9tp1JSaXdtJfM3z6uXlzdGWWErHhnxpOuk58EcGIeqrgvGTckmoqqZj0flWvQUxe1iF3VQCpC9PkwhY6iK3MhSeeKMJy9vjAHlg7l2wAReWPksU1c/n/Tx7MLNT9uBYKjRcadE7KyqI+Diz2UXuRg9xeJmsYs6KAVjTFbTTrtrguHae6mk9xLIyxvj+kE3UXHQQH41a5xrpOduvaFZGXdKuC3RuVLujg4zTa5GT7G4VeyiDkqhJpB8Hbx0CBnDntpg0lFbY/LyxvB5fDw8Miw9HzvtEgIh528obh2H2lUTSFpVGUsgZNiZQquVXCXXo6co0fEotz1cqINyMeGyQ/afME60gdhdE0jaKTYlL2+MQ1p15o6T7mf+5v/ywALnpeduHIdKdtypsf24NWWUaXJlPpEV6lw4P0odlIvZXRNkZ5X9J4wTT/Op3JpnrZ8ONC4vb4wf9ziPs444l7+8/0cWbX4/pX1kEjc9eQeCIXZmUNHVHMajagJB1xWETRe3jUepg3IpgWCIumCI6kCQPbX2OalU6uA5xezPZjYpL2+KO066nw6lHV0hPXeLgzLGsL2qLqNpXgPhfebIuZUKu2vccyPPJG4aj1IH5VKqY6Kaympr81FSIVe6vCYjL2+MssJWPDziCTbu+oKb5lyXIetSwy3jUDurA7bckIJ5PB6Vj9FTFDdNvlYH5VJixwLsVEclWwfPKZKVlzfGgPLBXDfwJl5c+RyvrPpnBqxLjWgbBCexe/5Spsa13Ea+Rk9R3DL5Wh2UC6kJBPdJu0V78WSSVOrgOUUq8vLGuG7gBPofNIhfzRrHFzuck547+RSerUmaDSuh5zr5HD3F4obJ1+qgXEh1bfyTvyYQyuh4VC4Ui4TwGMmsz6YnLS9vDJ/Hx0MjnwRg7PSLHZOeO5VirZ/vlI1j4U4Jc6rke/QUi9NiF3VQLsMY06jjyOR4lB29n+zgk22r2LRrQ0ry8sYIS8/vY8Hm97h/wV0Z3bdVnHoS31llz7hTIoIhkxVFqt00l+gpitMPF+qgXEZ1XeOTZg2wfU/6XTGNsb84bKawUr08Vc7ucR5nHzGae97/kyPScyfGofbUBqh2IHq2W5GaDZpT9BTFycnXPkeOqiSkykLON2TCPXr2K0493ZVK9YiqQBWvrHqec3pegM+TvVMnE/Lyxrg9EkVdOW0M5x15UUb2OeKw0+jZvpeldeuCIbweb0aOa+VYlQ4q6yqrA/i9Hvze3Hs2bm7RUyzVdUEKvB5aFGTnPI2iDspFROc+WaEmEGJ3TYCSwtT+hamk957/+B/cOHscYDj/qDEpHTdZdtdW8v6muVza92rbjlFW2IpHR07igqmjuOu932dkn+9tfJcXfvy6pXVrAiGK/PZf+KGQYfue7Iw7JSKaAWhbUoDHIw5akjxOOnY3sKu6Dr9X8GXx4UIdlIuoTnLAfHdN+Gm0wJf8CVMTTD5VMSuSantw4d2c2/PCrDz1v7thDnWhOoZ1GWHrcSo6DGLllZsykmv/zdu/ZPLyp6kL1uH3+ptcP1vzoXZW17liUnYmMgDZproumDOKV7uITr5uW1KASHYeLtRBuYhk54tEBzCTfRqtDYSSrlZdE6hh7oa3OHS/rqzbvpZ/r3mZMw//3+R2kgKzP4vIyzsca/uxPOKBDFx3g8qP44klj7FsyxL6Hdi/yfVDJjwO5bUxothTG3DVpOyaQIjte2ozcqMTgWK/19Yn+3yoWJ4JgiHD7togLVPM3CSLrbGaiIwQkdUislZEJsR5/14RWRJ5fSIi22PeC8a896qddrqBeHOfrBB9Gk32WMkyf9M89tTt5rfH/4lubQ7nwYV3267sMcYwK4Xq5U4zqHwIEP7OrGJnFFUbcHbcKRE1gVB9Ydl0XlW1QbbtrrVNgKHR095kMwq3zUGJiBd4GBgJ9ATOE5GesesYY64zxvQxxvQBHgRejnm7KvqeMeZ0u+x0C4nmPlkhOh6VzPrJMmv9DAq8Bfyg00lc0/8GPt6ytD7lZxd2ycvtZv+SA+my32HJOSibBt9DoezNd3ISA+yqDrBtd23G5+1o9OQcdkZQA4C1xph1xphaYApwRiPrnwdMttEe19LU3CcrVNYELD2FB4KhlGTNs9fP5NjyH1DiL2HU4efQsawT98+/09Yoyk55ud0MLB/Cgk3vETLWbpZ2RVBuGXfKFnXBENt212bMqWj05Cx2OqhyYEPM3xsjy/ZBRA4BugCzYxYXicgiEXlfRM60z0znaWruk1V2VDU9PyqV6OmLHZ+zZtuqekfh9/q56pjrWPjl+7yfRJSQLHbLy+1kYPkQtlVvZc221ZbWj45DZZLdNe4ad8oWhvADWyaiKY2enMVOBxVv9DPRFTgaeNEYExtGdDLGVADnA/eJyGH7HEDk8ogTW7Rly5b0LXYIK3OfrBAyTdfrS+WGVR/JxKTazjvqItoV788DNlVgqKzdxfub5makOKwTOD0OVZtk2jcfSTea0ujJeex0UBuB2EffjsDmBOuOpkF6zxizOfJzHfAW0LfhRsaYicaYCmNMRfv27TNhc9ZJZu6TFWqDiW9MoVBq1SNmr5/JwWWH0LV19/plLXwt+Hm/a5jz+Rt89PUHKdubiLkb3qIuVMdJXXLTQXVudSj7Fx/oiINqLuNOVkgnmmruDt4N2OmgFgLdRKSLiBQQdkL7qPFE5HCgNfBezLLWIlIY+b0dMARYYaOtjpHs3CcrJBqPSiV6isrLT+p8yj6S4DG9L6essJUtLdSzKS+3AxFhYPngpFKgmRJKNLdxJyskG01p9OQObHNQxpgAMBaYAawEnjfGfCwivxeRWFXeecAUs/doew9gkYh8BMwB7jDG5KWDsqtXTrzxqHTk5cPiRDKlhWX8rM+VvL72X3yydVXKtjYkV+XlDRlYPoRNuzawcecXltYPGZORMZPmOO5khWSiKY2e3IGt86CMMa8bY7obYw4zxvwxsuy3xphXY9a51RgzocF2/zXG9DLGHB35+biddjpFqnOfrNBwPMoYk1IKKSovH3LwCXHfv7Tv1RT5WvDQor+kbGtDclVe3pBUxqHqgqmfDzruZI2moimNntxD7lVszCPSmftkhdpgiMrIRVgbTE0pGCsvj0fbFu24sNclvLRycsYa/836bDqQm/LyWHq0O4rSgjLmb/qv5W1SHYfScafkaCyaqlQn7xrUQTlEJuY+WSGc8glmRF6eiCuOGY9HPDy6+N5UzdyL2etnckTbI3NSXh6L1+Olf4djszIOpeNOqdEwmqquC2a9/YmSGHVQDpGpuU9W2FFVl1L18njy8nh0KO3IOT1/wnPLJ/HN7q9SsjFKZe0u5m+ax0mdczt6ijKofAifbFvJtqqtltZPZRxKx53SIzaa0ujJXaiDcohMzX2ygjGp1c+KJy9PxNUV11MXqmPiBw+lYmI9uS4vb8jA8sEALNhsPc2XzDiUjjtljroUq6wo9tGkgxKRsSLSOhvGNBcyPffJDqLy8mFdRliqOH1o666c3v1sJi2dyPbq71I+bq7LyxvS54AKCr2FyaX5LEZDOu6k5DtWIqgDgYUi8nykOnludRlzIdmMnlIlKi9PJtU2rv8NVNbu4smP/prSMfNFXh5Loa+QPgcck5SSz2qvLh13UvKdJh2UMeZmoBvwODAGWCMif4pXekixRnUK40HZpil5eTx6tu/FyV1O5W8fPMTuut1JH3P11pUpycvd/sQ0sHwIy75ZYvk7MQZL83R03EnJdyyNQUUm0X4VeQUIV354UUTsKcSWx9g59ymTzF4/o1F5eSLGD7iBbdVbeXbZEykdE5KTl4tAWYumu9Y6ycDyIQRCAT74coHlbRpT8+m4k9JcsDIGNU5EFgN3AfOAXsaYK4FjgLNtti/vsHvuUyYIy8tXpzQPqaLDIAZ3PJ5HF99HTaAmqW1TkZcXeD0U+b0U+e1vP58q/TsMQpCkxqHqAvEfYnTcSWlOWImg2gFnGWNOMca8YIypAzDGhIDTbLUuz8jW3Kd0sSovT8S4ATfwZeVmXlz5nOVtUpWXF/jCp3BpoQ+PS4dHywpbcWT73hkZh9JxJ6U5YcVBvQ5si/4hIqUiMhDAGLPSLsPykWzOfUqH2etn0qmssyV5eTxO6DSM3vv346FFfyEYsuaQU5WXF3jDp7DHI5QW+ZK2NVsMLB/M4i8XUBdsvB1KlHjjUDrupDQ3rDioR4HKmL93R5YpSZIL6r366uVd9q1ebhURYfyAG/hs+6f8e83LlraZ9dmMpOXlHhF83u9PYTen+gaWD6EqsIdlW5ZY3iZ2HErHnZTmiBUHJbGVxiOpPfc+qrqUXJj7BKnJy+MxsuvpdGtzOA8s+HOTbeGNMcxOQV4eTe/F4tZUXzoNDHXcSWmuWHE060RkHN9HTVcB6+wzKT/JhegJYNb66UnLy+PhEQ/X9L+BcTMu5c3PpnPyoSMTrhuWl2/kuoE3JXWMwjgOKprqa6qzcLbZv+RAuux3GPM3zePKY661tE00gnLTuNPDi+4hFApyzYAbnDbFtXy+Yz3jZ1xGTbDaaVP2oke7o7jzpAfwe92teo3FioO6AngAuJlw2apZwOV2GpWP5MLcJ2i6enkyjDr8HO567/fcv+BOfthIRYrvRRlJCiS88RMARX4vNXUhql0mSBlYPoSZn75GyITwSNPJC2MidRRdMu4089PX+MO7/wdA1zaHM7Lr6U1s0Ty5d/7tfPjVQgan+ZCXSeqCtTy3fBIHlBzEjYN/67Q5lmnSQRljviHcDTevMcbw3Z46yop8e41rZIJcmfsUlZf/pNclGdmf3+vn6orruWn2tby3aS6DO/4g7npReXmH0o6W9+3zCB5P4lReaZGPmt1B3PS1DywfwpSPn2bNttUc3raHpW2qXRJ5f135Jde+cQVHtT8aj3j5xRtX0ffACg5s2cFp01zFxp1f8OLK57io92X88cR7nDZnL8bPuJz7F9zJCZ1OYlDH45w2xxJW5kEVicjVIvKIiDwRfWXDuGyTbFtoq+TC3Cf4PpIZ1nlExvY5+sif0r74AB5YEH9Od7ry8kR4PEJZkbtSGamMQ7mBkAkxfublVNXt5tFTn+KRkU9SFajimumXEjK5cW5ni0cX3wdgOY2bTf544l/oVNaZq6dfwo7q7U6bYwkrocIzhOvxnQK8DXQEdtlplJMk0xba0v5yZO4TfC8vP6x1t4zts4WvBT/vdw1vff4mS75avM/7734xJzV5eRMOCiKqPp97VH2dWx3K/sUH5pyD+tuHD/HW52/yuxPuolubw+napju3Db2bdzfM4bHF9zttnmvYsucbnl32JD/ucT4dyzo5bc4+tCwo5ZGRT/JV5WZunD2uSfGSG7DFWe3DAAAgAElEQVTioLoaY34D7DbGPAX8D9DLXrOcJxpN7alNL5rKlblPNYEa3v1iTlry8kRc1PsyWhXux4ML/7zPe7PXJ1+9XEg8/tSQ0iIfbhH1iQgDywcnVVHCaZZ/8xF/nPsbRhx2Ghf2+ln98vOPGsP/dD2D2+fdwtKvP3TQQvfwtw8eoiZYw9iKXzhtSkL6HTSAG479DVNXv8ALSUykdworV3lUDrVdRI4CWgGdbbPIRRhgV3U4mkq1T0yuqPfe3zSXqsAeWxoFlhaWcUmfK3lt7b9YvfX7ud1RefkJh5yUlLzc7/VYdqJuS/UNLB/Cpl0b2LjzC6dNaZI9dXu4ctoYWhe14S8nP7rXdy4i3H3yI7Qrbs9V08akVBw4n9hRvZ0nP/orp3UbRdc2qU1wzxbX9P8lg8qP46bZ17J+u7sF2VYc1MRIP6ibgVeBFcCdtlrlMuqCIbZW1iQdTeXK3CcIjz9lQl6eiEv7XkULXzEPLfxL/bKovDzZkkp+C+m9WNyU6sulcajfvXMTa7at4sFTHqdti3b7vN+6qA0PnvI4n363hlvfvtEBC93DpKUT2VW7k3E5IL/3erw8NOJxfB4fV00bY7m6iRM0eqWLiAfYaYz5zhjzjjHmUGPM/saY1Br+5DDRaOq7JKKpXImeILPy8ni0bdGOn/b+GS+vmsIXOz6PHDOz8vLGcEuqr0e7oygtKGP+Jusddp1gxqf/4amlE7nymGs5/pCTEq53XKehXFVxHc8se5zX1/4rixa6hz11e5j4wUOc1PkUeu3fx2lzLNGxrBN3DXuQD75ayD3z/+S0OQlp9EqPVI0YmyVbcoLaJKKpXJn7FJWXD7O5zfoVx4zHIx4eWXwvEO45lay8XMSaQKIhbkn1eT1e+nc41tXjUF9Xfsl1b1xJr/37MGHwrU2uf+PgW+i9fz9+8cZVfFm5yX4DXcZzyyextWoL43MgeorljMN/zLk9L+T+BXfx/sa5TpsTFytX+hsi8ksROVhE2kRftlvmYqxEU7ky9wnSr15ulYNalnNuzwuZvHwS675by4JN/81K9BTFLam+QeVD+GTbSrZVbXXalH0ImRDjZlxGVd1uHhk5iUJfYZPbFHgLeGTkk1QHqhk3/bJmJT2vDdby6OJ7GVQ+hIGR9G0u4XbpuZWr/RLgauAdYHHktchOo3KFaDRVVbtvKi9X5j6BPfLyRFzd/3rqQnVc9toFtsnLG8MNqb6B5YMBWLDZfWm+iR88yNtfzKqXlFuluUrPX1o1hU27NjJuwK+cNiUlWhaU8sipk1wrPbfS8r1LnNeh2TAuFzCEa6XFRlOhUO7MfaoOVNsmL49Hl/0O4/TuZ/PxlqW0LChNSl4O6UVQ4I5UX58DKij0Frouzbf8m4/407zf7iMpt0pzk54HQ0EeWng3vfbvw4mHnOy0OSnT78D+rpWeW6kk8dN4r2wYl0vERlM1gfTnPlUFqjjluSH88+N/ZMS+RMzfNM82eXkixvUP5+qTrV7esL1GqhT5vXELzWaLQl8hfQ+scJWSLywpvyiupNwqzU16/traqXz63RrG9b8hKw93duJW6bmVq7R/zOsHwK2AVomMQzSa2lWdvmxz5qev8dHXHzBh9jg+2boqfeMSYLe8PB492/fi/uF/44ZjkytamW56L5ayIr+jqb6B5UNY9s0S19zEb31nAmu2rU4oKbdK66I2PDjiibyXnhtjeGDBn+naujundj3DaXPSxq3ScyspvmtiXpcBfQHrj73NkExkcV9e9U/aFx9Asb8lV067iJpATQb2ui+z189kcMfjbZOXJ+LcI39Cj3ZHJrVNJqMep1N9A8uHEAgF+ODLBY7ZEGX6p//m6aV/a1JSbpXjDj6Bqyuuz2vp+ez1M1m+5SPG9v8lXo/zwptM4EbpeSpX/B7A/tH0Zsy2qq3MXj+Ds444l3uHP8bHW5byp3mZL5EflZdnM72XDumOPzXEyVRf/4MG4RGP4+NQX1Vu5vqZYUn5TUN+l7H9/mrwb/Naev7AgrsoLz2Ys4/Ir0YPbpOeWxmD+reIvBp5/QdYDeTnY5FL+PcnL1MXquPHPc5j+KGncvHRP+evHzzAW5+/mdHjZEtengmaaq+RKk6l+koLyziyfW9Hx6HqJeWBPTwyclJS44FNkc/S8/c3zmX+5v9yVcV1OdX8zypukp5beXy8G/hL5HU7cLwxZoKtVjVzXl41hW5tjuCo9kcD8Nvjb+fwtj0ZN/0yvt2zJWPHmfXZjKzJy9Mlk+NPsTiZ6hvYYTCLv1zgWL7/rx88wDtfzOb3J/w5KUm5VWKl59E2FPnA/Qv/TLvi/Tn/qDFOm2ILUen517u/5FezrnFUem7lqv8CmG+MedsYMw/YKiKdbbWqGfPFjs+Zv/m//LjHefXKoBa+Fjw6chI7ar7j+jeuzMgJUx2oZu6Gt7ImL08XuxwUOJfqG1g+hKrAHpZ9k31J9rJvlvCnub/l1K6nZ6xBZTyi0vM75t2aF9LzpV9/yJz1M7m871ha+Fo4bY5tRKXn//rkRZ5f8axjdli5Kl8AYuPzYGSZYgOvrP4nAKMOP3ev5T3b9+Lm425j5rrXeGrp39I+jhPy8lRJpr1GqpQ6kOqLFo7N9jjUnro9XDVtDG1btOPuHz5i6wNKrPT8ymkXuUa1mCoPLPwzpQVljDn6cqdNsZ2xFb9gUPlx/HrOdXy2/VNHbLBy1fuMMbXRPyK/q4rPBowxvLhyMgM7DKZTq0P2ef9nfa/ixENO5ta3b9yrbUUqzF4/g0JvYdLycq8N40BNkUx7jVTxeoT9WhRQVuTPyMsK7UsO4ND9uma9cOyt70xg7bZPeHDE47Rp0db240Wl5+u+W8stb+VmxQWANdtW89qaqVzS5wrKCls5bY7txErPr552sSOpaCsOaouI1M97EpEzgG/tM6n5snzLR6zZtoqzEiiDPOLh/lMmUlJQylXTxqQlPZ+9fibHdky+enlZkZ9suyg703sNj9OiwJuRl1VHPrB8CAs2/zdrIoJYSfkPOp2YlWPC99Lzfyx/Imel5w8t/AtFviIu7Xu106ZkDael51au/CuAX4vIFyLyBXAj8HMrOxeRESKyWkTWisg+wgoRuVdElkRen4jI9pj3LhKRNZHXRVY/UC7z4srJ+D1+ftT9rITr7F9yYIz0/DcpHefzHetTkpd7RCjweSj0Z3feR7YcVCbxW0xJDiwfwnfV22ydjB0lKinvvX9fJgy51fbjNSSXpecbdn7OS6smc8FRF9O+eH+nzckqTkrPrUzU/dQYMwjoCRxpjBlsjFnb1HYi4gUeBkZGtj1PRHo22Pd1xpg+xpg+wIPAy5Ft2wC3AAOBAcAtkaaJeUswFGTq6uc5qfMpTaZdvpeeP5iS9DxVeXnUUZQUZM9BiVi/2bsJq2Nm2WpgGCspf3jkkxmVlFulwFvAo6dOojpQzTXTL80p6XlUhXhlxbUOW+IMsdLz7VmUnluZB/UnEdnPGFNpjNklIq1F5DYL+x4ArDXGrIuMW00BGqsJch4wOfL7KcAbxphtxpjvgDeAERaOmbPM2/A2X+/+irN7WJv4l470fPZnqVUvjyrdfF5P1lRvhd7cnKXv91pL8R3SqgsHlBzI/M32Oii7JeVWOax1N24bejdzN7zFgwvv5ts9W9J+2d22ZMvur3lu2ST+t8cFlJcebOux3Eqs9Py6GVdnTXrus7DOSGPMr6N/GGO+E5FTCbeAb4xyYEPM3xsJR0T7ICKHAF2A2Y1sW27B1pzlpVVTaFlQysmHnmpp/aj0fOTkH3DdzCt4+owXLQkJovLyc4+8MGnhQWxU0KLAS03A/ifgXEzvQdiJi0BT17GIMLB8iK0RVLYk5VY5/6gxzF4/g9vn3cLt827JyD5P6nwK9w1/jP1LDszI/mKZ+OFD1ARrGNv/Fxnfdy4RlZ7fPu8WpiwfxXm9zrP9mFYclFdECo0xNQAi0gJouosZccfSE12uo4EXjTHRHhWWthWRy4HLATp16mTBJHdSFajitbVTOa3bqKTmVkSl5795+wYmLZ3IxUc3PTQYlZcPSzK917CSQ6HPi88TIJCgYWOmyFUHBeD3eKgNNu3EB5YP4dVPXmLDzs85uGxf9WY6ZFNSbhUR4YFT/s6wLiOpDlSlvb8te77h0UX3MvTpCu4++eGMFm/dUb2dSR9N5Efdz8qJCe12M7biF/i8wqndrD1Ip4sVB/UPYJaIPEnYSVwCPG1hu41AbDzcEdicYN3RhJsixm47tMG2bzXcyBgzEZgIUFFR4a5OW0kw89PXqKzdlVJdr0v7Xs3sz9/gd29PYHDH4zm8bY9G109VXh7PUZQU+thRZZ/01CPiiKw9U/h91hzU9+NQ/824g7r17RtZu+0Tnj/7taxIyq1SUtCS84/KnPbprCPO5eppl3DJv0cz+sifctvQu2lZUJr2fictnciu2p31LWKaO16Pl18eOyFr1VesiCTuAm4DegBHAn8wxtxpYd8LgW4i0kVECgg7oVcbriQihwOtgfdiFs8AhkfGu1oDwyPL8pKXV/2TA0oOZHDH45PeVkS4f/hfLUvPo/LyYn9xUscpjNMqvdDnwWPjE3kuR09gfRzqiLZHUlbYKuNpvmlrX+XpZX/PuqTcCbq1OZz/jJ7DtQNu5PkV/+CkZwawIM35ZXvq9vDXDx5kWJcRHLX/0RmyVEkGS3cAY8x0Y8wvjTG/ACpF5GEL2wSAsYQdy0rgeWPMxyLy+9h5VYTFEVNMzKibMWYb8AfCTm4h8PvIsrxjW9VWZq2fzqgjzk25bL9V6Xmq8nIh/s1WRCi2UdHnZFPBTGBVyef1eOnf4diMVpT4qnIzv3jjKsck5U5Q4C1gwpBbmfq/bwBw5gsnc8e8W1OeYPrc8ifZVvUt4zV6cgxLV5CI9BGRO0VkPeFoytKkDWPM68aY7saYw4wxf4ws+60x5tWYdW6NV3zWGPOEMaZr5PWkpU+Tg/z7k5cJhAJpl+23Ij1PR16eaOyihd9r28Rdu8sb2Y2I4LOYohzUYTBrtq1ia1X6c+BDJsQ10y+lKlCV8SrlucCA8sHMvnAB5/a8kPsW3Mn/TBnKmm2rk9pHbbCWRxbdy6Dy4xhQPtgmS5WmSHgHEJHuIvJbEVkJPER4XEiMMScaYx7MmoV5zkurJtO9TY/6yuXp0JT0PFV5eWOpNo9HKLIhirKrvUa28VuMAgdGxqHSTUsBPLb4ft7dMIc/DP0zXdt0T3t/uUjLglLuHf4YT/xoCht3fs7wZ4/liSWPWZZHv7hyMpsrNzF+QO6WZsoHGrt6VgHDgB8ZY46LOKVgI+srSfL5jvUs2PweZ/cYnRF1VWzV8+tmXrHXxRiVlw9LoXp5U5FMsQ2VJXJ9/CmK1Sjw6AOOodBbmHaab+nXH3L7vFs4tevpXHDUxWntKx84tesZzLlwIcd2PJ5fz7mO86eeydeVXza6TTAU5KGFd9N7/74MPeSHWbJUiUdjV8/ZwFfAHBH5m4gMI778W0mRV1bFr1yeDlHp+Rufvc6kpRPrl39fvTy59J5HBF8TN1k7Ju7mi4OyWgWj0FdI3wP7pyWU2F2323WScjdwQMuDePbMV7j9pPt4f+O7nPhM/0brAf5nzSus276WcQNu0O/QYRJePcaYV4wx5wJHEJZ4XwccICKPioj7ezS4HGMML61KXLk8HS7tezUndh7O796ewKpvVwCZlZfHo7jAyowFa2SjvUa28HrEchuPQeVDWPbNEnbXVqZ0rFvfvpFPv1uTtSrluYSIcPHRP+eNC96nY9khXPLv0Vw78+dU1u7aaz1jDA8sDFfbyOR8KiU1rMjMdxtjnjXGnEZ4PtISQDvqpsmyb5awZttqzu6R+dnYIsIDw7+vel4dqGbWZzNSlJdbcxQFPk/GauZlo71GNrHqbAeWDyFogiz+ckHSx3h97b94ZtnjzUJSng5d23TntdFvcd3ACXHl6LPWz+DjLUsZW/FLPJIfD0m5TFL/gUhtvL8aY06yy6DmwkurpjRZuTwd2pccwH3D/8qKb5cxdvolrP3uk5SaEyYTyWRKcp4v6b0oVh13xUED8Ygn6XGoLys3NTtJeTr4vX5uHHwL/zrnTUSEM184mdvn3UJtsJb7599JeenBnHVE5tLuSurk150gR4hWLh/W5RRaF7Wx7TgnHzqSS/pcwX/WvAIkLy9PVklX5PdmZOJuc3VQpYVlHNX+6KQKx4ZMiHHTL6M6UN0sJeXp0L/Dscz6yfz6VhJDnz6GhV++z9UV1+P3ZqdSgtI4+XUnyBGilcsTNSbMJL/5wZ84vG1PDmvdLaPy8kSkG0XlanuNxvB7xbK6aGD5YD74cgG1wdqmV0Yl5ekSlaM/+aN/sr1mO/sXH8h5GSzBpKRH5ka2Fcu8uGoypQVlliuXp0MLXwumnvMG1YGq5OXlKTqo3bWBJqt4JyJX22s0hkSUkHUWC8f+7cOHWfbNhxxzUNzi//WopDxzjOx6Osd2/AHVgaqkCjYr9pJfj6o5QFWgitfX/ov/6XZm1i6E1kVtOKhlct1KUlXSiQgt0pgXlW/pvShW6/IN7BCuWjC/iQm7UUl5u+L2KinPEPsVtebAlh2cNkOJIT/vBi4mWrn8x0fY30slHdJR0qUjOc9fB2Xtc7UvOYDDWndr0kHVS8pPUUm5kr/k593Axby0agoHlhzEsR1/4LQpjZKOo/B6hKI41c+tbJfL7TUaI5lxtYHlQ1iw+b8JW6JHJeVXVVzHcZ2GZshCRXEf6qCyyNaqb5m9fkZalcuzRbqRTHFh8p8vX6MnCDtfqwrHgR2G8F31Nj7Zum9N5lhJ+Y2DM9ONVlHcSv7eEVxIpiqX200mlHR+b/ITd/OlekQirH6+7xsY7i03V0m50tzI7zuCy3h51RQOb9uTI9v3dtqURsmUki5ZyXm+Oyi/z1oE1alVZw4oOXCf+VCPLr5PJeVKsyK/7wguor5y+RGZqVxuJ5lKtRX5vZbHlPxeT16012gMqxGliDCwfMheEdTSrz/kjnm3qqRcaVboPKgsUV+5/IhzHLakaTI5FlRc4GVXdaDJ9azKsHMZv9eDAFamiA0sH8Krn7zEhp2f06ZFO66cdpFKynMIj7hP8BMIhVKen+gU6qCyQLRy+aDyIRxcltnK5Zkm00q6Fn4vlTVNT9zNZ4FELFYn7H4/DvVf3t84l3XfreWFs193XFJeXOBFRNhd0/RDR3OmdbG/yTY12aa6LsiOqjqnzUgKdVBZIFq5/PJh1zhtSpNk2lGICMUFvkZvaPnUXqMp/F6hzkLbzyPaHklZYSvuX3AXa7at4uqK6x2XlPs8QsvC8C2jpi5IIJRjj+NZotDncZ1zgnDKvaYuRHUgd/rOuu9bdIhpa6ax5KvFtuz7pVWT8Xv8nNZ9lC37zyR2OIpiv7fRWnT51l6jMayOQ3k9Xvp3OJY121bRe/9+jkvKBShr4UdEEJHw745a5F4y2Rst05QW+TJS0DlbqIMiLN+9afZNnPbPodw3/04CocylL8KVy1/gh11G2Fq5PFPY4aA8HqGwkfJHzSW9B8l9v8M6n0JpQRmPjHzScUl5caFvL+fq93ooKXTvjdgpfB5x9fns8QhlLXLn/+bebzKLeMTDnJ/O4bRuo7jjv7cy6vmT+Xz7ZxnZdzYrl6eLnUq6xiTnbr6gM40niTG+i4/+OR9d/pnjkvLY1F4sJQ2clkJOOO1Cn5eiNOplZhM9uyK0btGax059mkdGPsnqbSs56R8DeG75U5g0ZS/ZrFyeLnY6Cr/XEzd6yMf2Gk2RjNw82Q7ImUaAVi0S90YqK/Jpqi+CRyRnbvxlOZLqa153BgucdcRoZv9kAUcf0I/r37iCS/59Lt/u2ZLSvvbU7eH1tf/itG6jKPIVZdjSzGO3UCFe+aN8bK/RFLkkqS8p9DU64O/TVF89JSmU93KK8Dii+/9v6qDi0LGsEy/+eBq3HH87s9bP4MRn+vPmZ9OT3s/MdeHK5W4vbQQRJZ3NqbZC374Td5tTei9KrkSMVseZNNUXzgSk02bGCQp9Xlqk2WDUbpr3WdUIHvFw5THXMv28ubRt0Y6fTB3FjbPGs6duj+V9vLzqnxzUsoPrK5dD9hxFSQOFU3N1UG6PoYRwGsgqzT3V18LvzUklammhz3UTimNpfneHJOnZvhfTz5/Lz/uN46mlEzn52UGW5OjRyuVnHn6O6yuXQ/YcRZHfQ/Q6zuf2Gk3h9oijZVHjqb2G+LweWibh0PIJwd3S8sYQEcqKEo8xOo27rxKXUOQr4ncn3MmLZ0+jqm4Pp/1zKPfOv6NROXq0cvmPe7i7MWGUbE2UjU7cheYZPUXxu/izF3g9Kd1wiwt8zWbCdSyFSdScdCMFPk/ShZ2zRfM7m9LguE5DmX3hQn7U7Szu/O/vOPP5H7J++7q46760cjKHt+1Jz3a9smxl8nhEsjrzPTpxtznezKK4VSgRnZCbKs1xAq9bb+7J0NKlqb7me4dIkf2KWvPoqU/xyMgn+WTbKob9YyDPLZ+0lxz98+2fsfDL9/lxDlQuh+xHMh6PUFTgbd4OyuPOz96yKL0bldcjlLo4ZZRpClLoe+ZG3Jrqy/1v1iGicvQ+Bx7D9W9cuZcc/eXV4crlZ6ZZudyXpSeaQgfSTaWFvrxvr9EYyUzYzRappvYa0qIZPXyk0jnarbgx1dc8ziKb6FjWiRfOfn0fOfrLq6YwqPy4tCuXlxT6snLCOHEzyYXI0m7c9OQtkl5qryHhun0Z250r8XmEQp+7bujp0rLQl7UHYyu45wrJUWLl6O2K2/OTqaNYs2112nOfhHBkY3du2OeRZh3JOImboozSQn9GzzOvx50po0ySq8q9xnBbIWD3XCE5Ts/2vZh23rtcccx4jmh7JD/qflZa+yv0eesrRzdWaiZdmrOSzmncIpQo9HlsmbBZ5Pc6kj7OBuGyRvn52fxeD8UuqQ7iDivyhCJfEbcefwccn/6+CmNO/uiMfjuaxKmDcg6fNzwnzMkupyLYGumUFfn5dndNznVybYpo48Z8pWWhzxU9v/Tu5EJE9hUu2JEbbu5Sbzfg9PdfVuS3NcXrycNUn5B7ZY1SoZULUn22Xh0iMkJEVovIWhGZkGCdc0RkhYh8LCLPxSwPisiSyOtVO+10G9H0XkMyfcI0p0aBbsVJoURRltouFPm9FOWRmKCowNssxm3dUAjYtqOLiBd4GDgZ2AgsFJFXjTErYtbpBtwEDDHGfCci+8fsosoY08cu+9xMotx29ISpzFCqT9N7zuOUgxIJd1fNFqVFPmp2B/Mi1dewnmQ+U1LooyYQoi4YcuT4dl4dA4C1xph1xphaYApwRoN1LgMeNsZ8B2CM+cZGe3KCcHov8dNmJitHq4NyHqeEEnan9hqSL6m+ojgV+fMdJwsB23mHKgc2xPy9MbIslu5AdxGZJyLvi8iImPeKRGRRZPmZNtrpKqzMq8jECdMcGwW6ERHJ+ryTbKX29jmuP3c6uSYinybmWsXJQsB2HjXeVdcwwPcB3YChQEfgXRE5yhizHehkjNksIocCs0VkmTHm070OIHI5cDlAp06dMm2/I1iRrkZPmF3Vqaf6mmOjQLfi93kI1AazciyPSFZTew0pLfRRGwgRysFcnz9PyhqlQnGBj+q67Kf67Py2NwIHx/zdEdgcZ51/GWPqjDGfAasJOyyMMZsjP9cBbwF9Gx7AGDPRGFNhjKlo37595j9BlmkqvRdLupWjNb3nHrKp5CstcrbElMeTG51c4+G2MkDZxglVn51XxkKgm4h0EZECYDTQUI03FTgRQETaEU75rROR1iJSGLN8CLCCPCfZ9Ec6M77VQbmHbD2VuyXFVuhQijEdvB7JOZszjdcjWU/12XZlGGMCwFhgBrASeN4Y87GI/F5ETo+sNgPYKiIrgDnADcaYrUAPYJGIfBRZfkes+i9fSVaKm2rl6ObcKNCNeD2Cx2a5v0ckqQ65dlMWqZoukJGX3TQn5V5jZLvnl5gczAXHo6KiwixatCjl7Y0xfLOrJoMWJYdHhPalhSltu31PLTUB67nhFgXevFBU5RPJ/g+TZb9if94VNo2lsiZgS6UVCKfe27cs1DmDGUREFhtjKppaT/M8LqEwjbpepUXJVY52unqBsi92pvlaFHjz2jlBuNJK6+ICWyLR4gKfOieH0DuVS0hnpn0ylaOjVdIVd2GXg/J6hFKXFP60mwKfh7YlBRk9v4VwB2jFGfRO5QI8ImmLFqyWk/FpeSNX4veKLWMpZUX+ZvX/9niE/YoLKM3Q5NLmUtbIraiDcgGZKttfWuRrMtWn6j13IiL4MhxFFRd4m+3/u7jAR+uSgrTFQBo9OUvzPHtdRqbkq1bKyej4k3vJZNkjn0do2UxSe4nwe8Mpv1TT54U+T8YfGpTk0G/fYbweyej4Q2OpPhGNoNxMps4DIdpyXVNTIkKrYn841ZnktvnYMTfX0LuVw9gx+a+0yBdXzaTRk7vJ1P+nOIMFhfOFFgVe2pQUWK576Pd69GHOBeh/wGGKbLgIEpWT0QvO3XgyMGHX7/U0+9ReInxeD21KCiy1t2/uZY3cgt6xHMTryfzAeJRCn3efC1EjKPeTzv9IwFXVItyISHictlWLxHMHPaJljdyC3rEcxO6LoLTw+1SfxwaVmJJ5/L7UI6iWRT79H1ukyO+lbUlh3FRoSTNsqeFW9Gx2EDvSe7GICK1ahFV9mt7LDVIdOyrwenRQP0m8HqFNScFe6TwRaKHRk2vQu5ZD+GxM78VS4PNESt3ovzoX8Hs9SavNoqo9JTVKi/zsV+yvd06qfnQP+sjlENnMcTeXUjf5gt/roTaJxnClRX6tTp8mhT4v7Ur0Ic5t6H/EIbLpoEREnwpzCH8S0W5hJEJW0sfjES1r5DLUQTmA3+vRJ14lIVbn6oiQUj8wRckV1EE5gI4HKY1hVWpepqk9JXv8QE4AAAy6SURBVM/RO6UD6BwLpTE8FjoeF+Vg23RFSRZ1UFlG03uKFRqTm4dTeyp8UfIfdVBZJlOtNZT8prE0X1mRXwfzlWaB3i2zTDqdc5XmQ6LWG0V+Te0pzQd1UFmkwOvRJ1/FEuHOx3sv84horT2lWaEOKovok6+SDH7P3pdnWQufzmdTmhXqoLKIysuVZIidsBsuV6UPOErzQu+YWULTe0qyRMehvB7RclVKs0QdVJbQ9J6SLAWRwrFlRdq+XWmeqIOKYOcNQND0npI8IkJZC7+2SlGaLXrmx7BfsT/tltvxKPBpek9JDY28leaMOqgYCn1e2pYUZDza0ZuMoihK8qiDaoDHI+xXXEBpkS/pxnHx0PSeoihKauidMwHFBT5alxSkXTevwOfRAW5FUZQUUAfVCH6vh7YlBWml6DS9pyiKkhrqoJpARGjVwk+rFv6kU36a3lMURUkdvXtapMjvpU1JgeVupxAWXWh6T1EUJTXUQSWBz+uhTUkBxQXW0naF2lpDURQlZfQOmiQiQmmRn/2K/ftUm957PU3vKYqipIPeQVOk0OelXUlhwsZymt5TFEVJD1sdlIiMEJHVIrJWRCYkWOccEVkhIh+LyHMxyy8SkTWR10V22pkqHo/QuqSAloX7zpnSzrmKoijpYVuJZBHxAg8DJwMbgYUi8qoxZkXMOt2Am4AhxpjvRGT/yPI2wC1ABWCAxZFtv7PL3nQoKfRR4POwo6qOYMgg0njLbkVRFKVp7LyLDgDWGmPWGWNqgSnAGQ3WuQx4OOp4jDHfRJafArxhjNkWee8NYISNtqZN/Zwpn1fTe4qiKBnATgdVDmyI+XtjZFks3YHuIjJPRN4XkRFJbOs6RIRWxX5ty60oipIB7LyTxgshTJzjdwOGAh2Bd0XkKIvbIiKXA5cDdOrUKR1bM4pGT4qiKOljZwS1ETg45u+OwOY46/zLGFNnjPkMWE3YYVnZFmPMRGNMhTGmon379hk1XlEURXEWOx3UQqCbiHQRkQJgNPBqg3WmAicCiEg7wim/dcAMYLiItBaR1sDwyDJFURSlmWBbis8YExCRsYQdixd4whjzsYj8HlhkjHmV7x3RCiAI3GCM2QogIn8g7OQAfm+M2WaXrYqiKIr7EGP2GdrJSSoqKsyiRYucNkNRFEVpAhFZbIypaGo9nayjKIqiuBJ1UIqiKIorUQelKIqiuBJ1UIqiKIorUQelKIqiuJK8UfGJyBbgc6ftSJJ2wLdOG5EkuWgzqN3ZRu3OLrlm9yHGmCarK+SNg8pFRGSRFamlm8hFm0HtzjZqd3bJVbubQlN8iqIoiitRB6UoiqK4EnVQzjLRaQNSIBdtBrU726jd2SVX7W4UHYNSFEVRXIlGUIqiKIorUQelKIqiuBJ1UDYiIgeLyBwRWSkiH4vI+DjrDBWRHSKyJPL6rRO2NkRE1ovIsohN+5SJlzAPiMhaEVkqIv2csLOBTYfHfI9LRGSniFzbYB1XfN8i8oSIfCMiy2OWtRGRN0RkTeRn6wTbXhRZZ42IXJQ9qxPa/WcRWRU5D14Rkf0SbNvoOWUnCey+VUQ2xZwLpybYdoSIrI6c6xOyZ3VCu/8ZY/N6EVmSYFvHvu+MYYzRl00v4CCgX+T3UuAToGeDdYYC/3Ha1ji2rwfaNfL+qcA0QIBBwHynbW5gnxf4ivCEQNd938DxQD9gecyyu4AJkd8nAHfG2a4N4aaebYDWkd9bO2z3cMAX+f3OeHZbOaccsPtW4JcWzqNPgUOBAuCjhtdwtu1u8P5fgN+67fvO1EsjKBsxxnxpjPkg8vsuYCVQ7qxVGeMM4GkT5n1gPxE5yGmjYhgGfGqMcWV1EWPMO0DDJpxnAE9Ffn8KODPOpqcAbxhjthljvgPeAEbYZmgD4tltjJlpjAlE/nwf6Jgte6yS4Pu2wgBgrTFmnTGmFphC+P+UFRqzW0QEOAeYnC17so06qCwhIp2BvsD8OG8fKyIficg0ETkyq4YlxgAzRWSxiFwe5/1yYEPM3xtxl/MdTeIL143fN8ABxpgvIfxwA+wfZx23f++XEI6s49HUOeUEYyOpyScSpFTd/H3/APjaGLMmwftu/L6TQh1UFhCRlsBLwLXGmJ0N3v6AcBrqaOBBYGq27UvAEGNMP2AkcLWIHN/gfYmzjSvmLIhIAXA68EKct936fVvFzd/7/wEB4NkEqzR1TmWbR4HDgD7Al4TTZQ1x7fcNnEfj0ZPbvu+kUQdlMyLiJ+ycnjXGvNzwfWPMTmNMZeT31wG/iLTLspn7YIzZHPn5DfAK4VRHLBuBg2P+7ghszo51TTIS+MAY83XDN9z6fUf4Opomjfz8Js46rvzeI2KN04ALTGQApCEWzqmsYoz52hgTNMaEgL8lsMet37cPOAv4Z6J13PZ9p4I6KBuJ5IgfB1YaY+5JsM6BkfUQkQGE/ydbs2dlXJtKRKQ0+jvhQfDlDVZ7FfhpRM03CNgRTU+5gIRPlm78vmN4FYiq8i4C/hVnnRnAcBFpHUlJDY8scwwRGQHcCJxujNmTYB0r51RWaTBmOor49iwEuolIl0hkPprw/8lpfgisMsZsjPemG7/vlHBapZHPL+A4wumApcCSyOtU4Argisg6Y4GPCauD3gcGu8DuQyP2fBSx7f8iy2PtFuBhwgqnZUCF03ZH7Com7HBaxSxz3fdN2IF+CdQRfkr/GdAWmAWsifxsE1m3Avh7zLaXAGsjr4tdYPdawuM00XP8sci6HYDXGzunHLb7mci5u5Sw0zmood2Rv08lrMD91A12R5ZPip7TMeu65vvO1EtLHSmKoiiuRFN8iqIoiitRB6UoiqK4EnVQiqIoiitRB6UoiqK4EnVQiqIoiitRB6XkDSJiROQvMX//UkRuzdC+J4nIjzOxryaO878Srn4/p8HyzpHPd03MsodEZEwT+8uW3S+KyKGR39dHJz+LyDEi8pmI9BWR00Tkd3bbouQP6qCUfKIGOMtFlSEAEBFvEqv/DLjKGHNinPe+AcZHJozaTqRagZX1jgS8xph1DZb3Bl4EzjXGfAi8BpwuIsUZN1bJS9RBKflEAJgIXNfwjYaRhIhURn4OFZG3ReR5EflERO4QkQtEZEGkl85hMbv5oYi8G1nvtMj2Xgn3Q1oYKTr685j9zhGR5whPBm1oz3mR/S8XkTsjy35LeHL3YyLy5zifbwvhCbz79IASkcsiNnwkIi81cALx7C4SkScjNnwoIidGlo8RkRdE5N+EC40eJCLvSLin0HIR+UEcuy5g36oXPQjXObzQGLMAwIQnXb5FuCSSojSJOigl33gYuEBEWiWxzdHAeKAXcCHQ3RgzAPg7cE3Mep2BE4D/IexEighHPDuMMf2B/sBlItIlsv4AwjP4e8YeTEQ6EO6bdBLhQqX9ReRMY8zvgUWE69ndkMDWO4BfxInKXjbG9DfhIrgrI3Y1ZvfVAMaYXoRLQz0VWQ5wLHCRMeYk4HxghjGmT+R7itccbwiwuMGyfwFjjTFzGyxfRLgKt6I0iTooJa8w4WrxTwPjkthsoQn37qohXM5mZmT5MsI39yjPG2NCJtzeYB1wBOEaZz+VcFfT+YTLFXWLrL/AGPNZnOP1B94yxmwx4T5KzxJuTGfl830GLCDsOGI5KhIlLSMc0cS2EYln93GES/1gjFkFfA50j6z/hjEm2oNoIXBxZCyvlwn3NWvIQYSju1jeBC6N40i/IVySR1GaRB2Uko/cRziCKIlZFiByvkeKxcaO49TE/B6K+TsExI7DNKwLZgjXJLzGGNMn8upijIk6uN0J7IvXwiEZ/kS4OGvs9TuJcMTSC/gdUBTzXiK7E1Fvtwk3zDse2AQ8IyI/jbN+VYPjQbjmIcAjDZYXRdZXlCZRB6XkHZGn/+fZO821Hjgm8vsZgD+FXf+viHgi41KHAqsJVxK/UsJtVRCR7pHq0Y0xHzhBRNpFIozzgLetGhGJeFaw91hOKfBlxI4LLNj9TnQ9EekOdIos3wsROQT4xhjzN8KV+fvFMWkl0LXBslDkcx0uIr+PWd6dXKyqrTiCOiglX/kLEKvm+xthp7AAGEji6KYxVhN2JNMIV5KuJjxOtQL4QESWA39l76hrH0y4LclNwBzC1aY/MMbEa63RGH9k79bqvyHs+N4AVlmw+xHAG0kJ/hMYE0lxNmQosEREPgTOBu6Ps85rkfX2IrK/Mwgr966OLD4xsr6iNIlWM1cUJS1EpAVhZzvEGBNsZL0DgOeMMcOyZpyS06iDUhQlbUTkFMKNOb9oZJ3+QJ0xJp4SUFH2QR2UoiiK4kp0DEpRFEVxJeqgFEVRFFeiDkpRFEVxJeqgFEVRFFeiDkpRFEVxJf8P3jbjYC/jI4wAAAAASUVORK5CYII=\n",
                        "text/plain": "<Figure size 432x288 with 1 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "# What about other K? # find best K\nKs = 20\nmean_acc = np.zeros((Ks-1))\nstd_acc = np.zeros((Ks-1))\nConfustionMx = [];\nfor n in range(1,Ks):\n    \n    #Train Model and Predict  \n    neigh = KNeighborsClassifier(n_neighbors = n).fit(X_train,y_train)\n    yhat=neigh.predict(X_test)\n    mean_acc[n-1] = metrics.accuracy_score(y_test, yhat)\n\n    \n    std_acc[n-1]=np.std(yhat==y_test)/np.sqrt(yhat.shape[0])\n\nprint(mean_acc)\nplt.plot(range(1,Ks),mean_acc,'g')\nplt.fill_between(range(1,Ks),mean_acc - 1 * std_acc,mean_acc + 1 * std_acc, alpha=0.10)\nplt.legend(('Accuracy ', '+/- 3xstd'))\nplt.ylabel('Accuracy ')\nplt.xlabel('Number of Nabors (K)')\nplt.tight_layout()\nplt.show()"
        },
        {
            "cell_type": "code",
            "execution_count": 25,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "The best accuracy was with 0.7857142857142857 with k= 7\n"
                }
            ],
            "source": "print( \"The best accuracy was with\", mean_acc.max(), \"with k=\", mean_acc.argmax()+1) "
        },
        {
            "cell_type": "code",
            "execution_count": 27,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "KNeighborsClassifier(algorithm='auto', leaf_size=30, metric='minkowski',\n           metric_params=None, n_jobs=None, n_neighbors=7, p=2,\n           weights='uniform')\n['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF']\nTrain set Accuracy:  0.8079710144927537\nTest set Accuracy:  0.7857142857142857\n"
                }
            ],
            "source": "# use best K and get the model\nk = mean_acc.argmax()+1 # 7\n\n#Train Model and Predict  \nKNN_model = KNeighborsClassifier(n_neighbors = k).fit(X_train,y_train)\nprint(neigh)\n\n# Predicting\nyhat = KNN_model.predict(X_test)\nprint(yhat[0:5])\n\n# Accuracy evaluation\nfrom sklearn import metrics\nprint(\"Train set Accuracy: \", metrics.accuracy_score(y_train, neigh.predict(X_train)))\nprint(\"Test set Accuracy: \", metrics.accuracy_score(y_test, yhat))"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "comparelist(y_test,yhat)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Decision Tree"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Steps:\n+ Train Test Split\n+ Modeling\n+ Pridicting\n+ Accuracy evaluation\n+ Visualization"
        },
        {
            "cell_type": "code",
            "execution_count": 28,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "(346, 8) (346,)\nTrain set: (242, 8) (242,)\nTest set: (104, 8) (104,)\n"
                },
                {
                    "data": {
                        "text/plain": "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=4,\n            max_features=None, max_leaf_nodes=None,\n            min_impurity_decrease=0.0, min_impurity_split=None,\n            min_samples_leaf=1, min_samples_split=2,\n            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n            splitter='best')"
                    },
                    "execution_count": 28,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "from sklearn.model_selection import train_test_split\nX_trainset, X_testset, y_trainset, y_testset = train_test_split(X, y, test_size=0.3, random_state=3)\nprint(X.shape,y.shape)\nprint ('Train set:', X_trainset.shape,  y_trainset.shape)\nprint ('Test set:', X_testset.shape,  y_testset.shape)\n\nfrom sklearn.tree import DecisionTreeClassifier\nanswerTree = DecisionTreeClassifier(criterion=\"entropy\", max_depth = 4)\nanswerTree # it shows the default parameters"
        },
        {
            "cell_type": "code",
            "execution_count": 29,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "DecisionTreeClassifier(class_weight=None, criterion='entropy', max_depth=4,\n            max_features=None, max_leaf_nodes=None,\n            min_impurity_decrease=0.0, min_impurity_split=None,\n            min_samples_leaf=1, min_samples_split=2,\n            min_weight_fraction_leaf=0.0, presort=False, random_state=None,\n            splitter='best')"
                    },
                    "execution_count": 29,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# fit\nanswerTree.fit(X_trainset,y_trainset)"
        },
        {
            "cell_type": "code",
            "execution_count": 30,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF']\n['PAIDOFF' 'PAIDOFF' 'COLLECTION' 'COLLECTION' 'PAIDOFF']\n"
                }
            ],
            "source": "# pridicting\npredTree = answerTree.predict(X_testset)\nprint (predTree [0:5])\nprint (y_testset [0:5])"
        },
        {
            "cell_type": "code",
            "execution_count": 31,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "DecisionTrees's Accuracy:  0.6538461538461539\n"
                }
            ],
            "source": "# evaluation\nfrom sklearn import metrics\nimport matplotlib.pyplot as plt\nprint(\"DecisionTrees's Accuracy: \", metrics.accuracy_score(y_testset, predTree))"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "comparelist(y_testset,predTree)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Support Vector Machine"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Steps:\n+ Data pre-processing and selection\n+ Train Test Split\n+ Modeling\n+ evaluation"
        },
        {
            "cell_type": "code",
            "execution_count": 135,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/svm/base.py:196: FutureWarning: The default value of gamma will change from 'auto' to 'scale' in version 0.22 to account better for unscaled features. Set gamma explicitly to 'auto' or 'scale' to avoid this warning.\n  \"avoid this warning.\", FutureWarning)\n"
                },
                {
                    "data": {
                        "text/plain": "SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0,\n  decision_function_shape='ovr', degree=3, gamma='auto_deprecated',\n  kernel='rbf', max_iter=-1, probability=False, random_state=None,\n  shrinking=True, tol=0.001, verbose=False)"
                    },
                    "execution_count": 135,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# Modeling (SVM with Scikit-learn)\n\nfrom sklearn import svm\nclf = svm.SVC(kernel='rbf')\nclf.fit(X_train, y_train) "
        },
        {
            "cell_type": "code",
            "execution_count": 136,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "array(['COLLECTION', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF'],\n      dtype=object)"
                    },
                    "execution_count": 136,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# \nyhat = clf.predict(X_test)\nyhat [0:5]\n"
        },
        {
            "cell_type": "code",
            "execution_count": 137,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.metrics import classification_report, confusion_matrix\nimport itertools"
        },
        {
            "cell_type": "code",
            "execution_count": 138,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "              precision    recall  f1-score   support\n\n  COLLECTION       0.36      0.27      0.31        15\n     PAIDOFF       0.81      0.87      0.84        55\n\n   micro avg       0.74      0.74      0.74        70\n   macro avg       0.59      0.57      0.57        70\nweighted avg       0.72      0.74      0.73        70\n\nConfusion matrix, without normalization\n[[48  7]\n [11  4]]\nf1_score:\n0.7275882012724117\njaccard_similarity_score:\n0.7428571428571429\n"
                },
                {
                    "data": {
                        "image/png": "iVBORw0KGgoAAAANSUhEUgAAAVMAAAEmCAYAAADfpHMGAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XecXFXdx/HPN6ElJqGForTQAiIaDMKD1IDCE5CqdFARNAIPqCBIVRBEkSJdFKQqJYhGQugGUJAiBEMAKaFFkFBC7yX8nj/OGZhsdnfu7t7Zmcl+37zuKzP33rn3txvml3POPUURgZmZ9Uy/RgdgZjYncDI1MyuBk6mZWQmcTM3MSuBkamZWAidTM7MSOJlaw0gaIOlKSa9K+mMPrrOLpOvLjK1RJK0n6eFGx2FdJ/cztVok7QzsD6wMvA5MBo6JiFt7eN2vA/sCa0fEBz0OtMlJCmDFiHi00bFY+VwytU5J2h84Gfg5sBiwNPBrYKsSLr8M8EhfSKRFSJqr0TFYD0SEN2/tbsD8wBvAdp2cMy8p2T6Tt5OBefOxUcDTwA+B54HpwLfysZ8C7wHv53vsARwJ/KHq2sOAAObK73cDHieVjp8Adqnaf2vV59YG7gJezX+uXXXsZuBo4B/5OtcDQzv42Srx/6gq/q2BzYBHgJeAQ6vOXxO4HXgln3s6ME8+9vf8s7yZf94dqq5/EPAs8PvKvvyZ5fM9Rub3nwJmAKMa/f+Gt9k3l0ytM18E5gPGdXLOYcBawGrACFJCObzq+OKkpLwEKWGeIWnBiDiCVNodGxGDIuKczgKR9AngVGDTiBhMSpiT2zlvIeCqfO7CwK+AqyQtXHXazsC3gEWBeYADOrn14qTfwRLAT4CzgV2B1YH1gJ9IWi6fOxPYDxhK+t19CdgbICLWz+eMyD/v2KrrL0QqpY+pvnFEPEZKtBdJGgicB5wfETd3Eq81iJOpdWZhYEZ0Xg3fBTgqIp6PiBdIJc6vVx1/Px9/PyKuJpXKVupmPB8Cq0oaEBHTI+KBds75CjA1In4fER9ExCXAQ8AWVeecFxGPRMTbwGWkfwg68j6pffh94FJSojwlIl7P938A+BxAREyKiDvyfZ8EfgtsUOBnOiIi3s3xzCIizgamAncCnyT942VNyMnUOvMiMLRGW96ngGlV76flfR9do00yfgsY1NVAIuJNUtV4T2C6pKskrVwgnkpMS1S9f7YL8bwYETPz60qye67q+NuVz0saLmmCpGclvUYqeQ/t5NoAL0TEOzXOORtYFTgtIt6tca41iJOpdeZ24B1SO2FHniFVUSuWzvu6401gYNX7xasPRsR1EbExqYT2ECnJ1IqnEtN/uxlTV5xJimvFiBgCHAqoxmc67U4jaRCpHfoc4MjcjGFNyMnUOhQRr5LaCc+QtLWkgZLmlrSppOPyaZcAh0taRNLQfP4funnLycD6kpaWND9wSOWApMUkbZnbTt8lNRfMbOcaVwPDJe0saS5JOwCrABO6GVNXDAZeA97Ipea92hx/Dlhutk917hRgUkR8m9QW/JseR2l14WRqnYqIX5H6mB4OvAA8BewD/CWf8jPgbmAKcB9wT97XnXvdAIzN15rErAmwH6lXwDOkJ9wbkB/utLnGi8Dm+dwXSU/iN4+IGd2JqYsOID3cep1Uah7b5viRwAWSXpG0fa2LSdoKGE1q2oD09zBS0i6lRWylcad9M7MSuGRqZlYCJ1MzsxI4mZqZlcDJ1MysBJ5YoRdprgGheQY3Oow+63MrL9XoEPq0e/91z4yIWKSMa/UfskzEB7MNGJtNvP3CdRExuox71uJk2os0z2DmXalmjxirk+v/dlKjQ+jTFhsyT9uRad0WH7xd6Lv0zuQzao1AK42TqZm1Hgn69W90FLNwMjWz1qTmeuTjZGpmrUm1pj3oXU6mZtaCXM03M+s54Wq+mVnPydV8M7NSuJpvZtZTarpqfnNFY2ZWhEjV/Fpb0ctJ/SX9S9KE/P58SU9Impy3ztYJA1wyNbOWJOhXavr6PvAgMKRq34ERcXnRC7hkamatqZ9qbwVIWpK0qu3vehROTz5sZtYQla5Rtba0uu7dVduYdq52Mml5mw/b7D9G0hRJJ0mat1ZIruabWQsq3Gl/RkR8ocOrSJsDz0fEJEmjqg4dQloSfB7gLOAg4KjObuSSqZm1pnIeQK0DbCnpSeBSYCNJf4iI6ZG8C5wHrFnrQk6mZtaailXzOxURh0TEkhExDNgRuDEidpX0SQBJArYG7q91LVfzzaz11H8KvoskLUJqnZ3Mx8ttd8jJ1MxaU8nDSSPiZuDm/Hqjrn7eydTMWlDzjYByMjWz1iM8Nt/MrOdcMjUzK4en4DMzK4Gr+WZmPSRX883MyuFqvplZzwjo188lUzOznlHemoiTqZm1ICFX883Mes7VfDOzErhkambWU03YZtpc5WQzswKE6NevX82t8PVmX510WUl3SpoqaaykeWpdw8nUzFqSpJpbF1RWJ634JXBSRKwIvAzsUesCTqZm1pLKSqZtVyfNs+tvBFSWeb6ANNt+p9xmamatR6BiSzkPlXR31fuzIuKsNudUVicdnN8vDLwSER/k908DS9S6kZOpmbUcFe9n2p3VSdu7cNS6kZOpmbWkkrpGVVYn3QyYDxhCKqkuIGmuXDpdEnim1oXcZmpmrSdX82tttXSwOukuwE3Atvm0bwJX1LqWk6mZtaSSn+a3dRCwv6RHSW2o59T6gKv5ZtaSyh4B1WZ10seBNbvyeZdMrUP9+onbLzmIP52SlgwfteZwbrv4IO649GAmnrsfyy01tMER9g2PTn2Yjdb5wkfb8ksszG/POLXRYTWUqF3FL/i0vzQumVqH9tl5Qx5+4jkGf2I+AE49dEe22++3PPzEc4zZbj0O/vZoxhzxhwZHOedbYcWVuPEfqXfPzJkzGbHSMDbbYqsGR9Vgar6x+S6ZWruWWHQBRq/7Gc4bd9tH+yKCITmxDhk8gOkvvNqo8PqsW26+kWHLLsdSSy/T6FAars5tpl3mkqm16/gDv8Zhp/yFQQPn+2jf3kddzLjT9uadd9/jtTffYYNvnNjACPumcX+6jG223aHRYTSF3q7G19IyJVNJMyVNlnS/pD9KGlh1bBtJIWnlqn3DJN2fX4+S9GqeyOBhSX/PnXWrrz9G0kN5+6ekdauO3Zw/Nzlv27aJqbINq/fvoTdsut6qPP/S6/zrwadm2b/vLhuyzb6/ZoXRP+b3V9zBL3/41QZF2De99957XH/1BLbY5muNDqUpuGTafW9HxGoAki4C9gR+lY/tBNxK6id2ZAefvyUiNs+fXw34i6S3I2JiTqzfBdaNiBmSRubja0bEs/nzu0TE3W2u+VFMc5IvrrYcm2/wWUav+xnmnWduhnxiPv586p6sNGwx7rp/GgCXX38PV5yxd4Mj7Vsm3nAtnx3xeRZddLFGh9JwjUiWtbRMybSNW4AVACQNIo1i2IOUTGuKiMnAUcA+eddBwIERMSMfv4c0ucH/lRt2a/jJaeNZYfSPWfkrR/CNg8/j5rseYbv9zmLIoAGssPSiAGy01so8/MRzDY60bxn3x7Fss52r+BVlTsFXhlYqmQIgaS5gU+DavGtr4NqIeETSS5JG5mRYyz3Agfn1Z4BJbY7fTRr5UHGRpLfz6y9FxIvAAEmT874nImKbduIdA4wBYO5BBcJqTjNnfsj/HX0xl5zwbT6MD3nltbf57pF+kt9b3nrrLf5+00ROOOXXjQ6leTRXwbSlkml14rqFj0ck7EQaSwtwaX5fJJnW+qsQs05u0K1qfp6h5iyAfgMXrTlZQrO5ZdJUbpk0FYDxN01h/E1TGhxR3zRw4EAemvZs7RP7kGar5rdSMp0tcUlamDTv4KqSAugPhKQfFbje5/l4Mth/A6sDN1YdH5n3m1mTkdKgkmbSqm2mFdsCF0bEMhExLCKWAp4A1u3sQ5I+B/wYOCPvOg74ZU7OlQdUuwGuU5k1pdpP8v00v2t2Ao5ts+9PwM6kZQeqrSfpX8BA4HngexExESAixktaArgtl3BfB3aNiOl1jd7Muq3Javmtk0wjYranNxExqp191YOWV837bgbmr3H9M4EzOzg22306isnMeoGr+WZmPSdSMq211byONF8epHOvpAck/TTvP1/SE1UDcmr2J2+ZkqmZWbWSqvnvAhtFxBuS5gZulXRNPnZgRFzeyWdn4WRqZq2npGp+RATwRn47d9661YXR1Xwzazmi8Nj8oZLurtrGzHYtqX/uw/48cENE3JkPHSNpiqSTJM1bKyaXTM2sBZWzOilARMwEVpO0ADBO0qrAIcCzwDykQTcHkYagd8glUzNrSWU8gKoWEa+Qli0ZHRHTI3kXOI8CS5g4mZpZ61F6AFVrq3kZaZFcIkXSAODLwEOSPpn3iTT/x/21ruVqvpm1nEqbaQk+CVwgqT+pcHlZREyQdKOkRfKtJpOm/OyUk6mZtaSSnuZPIc3T0Xb/Rl29lpOpmbUkDyc1M+upJlyd1MnUzFqO6PrT+npzMjWzltRkBVMnUzNrTa7mm5n1UDPOtO9kamYtySVTM7MSNFkudTI1sxbkar6ZWc+p+KxRvabDZCppSGcfjIjXyg/HzKyYJsulnZZMHyDNOF0dcuV9AEvXMS4zs071b5Vqfl6D3sys6agJh5MWms9U0o6SDs2vl5S0en3DMjPrXD/V3mrpZHXSZSXdKWmqpLGS5qkZT4GbnQ5sCHw973oL+E3tMM3M6qekmfYrq5OOAFYDRktaC/glcFJErAi8DOxRM54CN1s7Ir4LvAMQES+R1kUxM2sIkZ/o1/ivlrw0SXurk24EVJZ5voA0236niiTT9yX1yzdA0sLAhwU+Z2ZWNwWr+V1enRR4DHglIj7IpzwNLFErniL9TM8A/gQsktsTtgd+WuSHNTOrCxWuxnd5dVLg0+2dVutGNZNpRFwoaRJpoSmA7SKi5uJSZmb1IqBfyU/zI+IVSTcDawELSJorl06XBJ6p9fmiq5P2B94H3uvCZ8zM6qaOq5M+CNwEbJtP+yZwRa1rFXmafxhwCfApUoa+WNIhtcM0M6uPyhR8JTzN/yRwk6QpwF3ADRExATgI2F/So8DCwDm1LlSkzXRXYPWIeCv9EDoGmAT8okikZmb1UEY1v5PVSR8H1uzKtYok02ltzpsLeLwrNzEzK1tzjX/qfKKTk0hPsN4CHpB0XX6/CXBr74RnZjY70UJj84HKE/sHgKuq9t9Rv3DMzApQC03BFxE1G1zNzBqlyXJp7TZTScsDxwCrAPNV9kfE8DrGZWbWoWas5hfpM3o+cB4p/k2By4BL6xiTmVlNylX9zrbeVCSZDoyI6wAi4rGIOJw0i5SZWcOowNabinSNelcpxT8maU/gv8Ci9Q3LzKxjUvNV84sk0/2AQcD3SG2n8wO71zMoM7NaWuZpfkVE3Jlfvs7HE0SbmTVUk+XSTjvtj6OTaaci4qt1icjMrAZJLVXNP73XougjVllxSS6/+peNDqPPGjJg7kaHYCVqmWp+REzszUDMzLqijLlAJS0FXAgsTlpB5KyIOEXSkcB3gBfyqYdGxNWdXavIAygzs6ZSYqf9D4AfRsQ9kgYDkyTdkI+dFBEnFL2Qk6mZtaQycmlETAem59evS3qQAus9tRtP0RMlzdudG5iZlS3NpF/uCChJw0hzm1Z6MO0jaYqkcyUtWOvzRWbaX1PSfcDU/H6EpNO6FKWZWcn696u9UWB1UgBJg0gLh/4gIl4DzgSWB1YjlVxPrBVPkWr+qcDmwF8AIuJeSR5OamYN04UF9WquTippblIivSgi/gwQEc9VHT8bmFDrRkWq+f0iYlqbfTMLfM7MrG76FdhqyUPlzwEejIhfVe3/ZNVp2/Dx/M4dKlIyfUrSmkBI6g/sCzxS4HNmZnVRYqf9dUgjO++TNDnvOxTYSdJqpIFLTwLfrXWhIsl0L1JVf2ngOeCveZ+ZWcOU0Wc/Im6l/QmmOu1T2p4iY/OfB3bs6oXNzOqpyUaTFppp/2zaGaMfEe0+FTMzq7dmnGm/SDX/r1Wv5yM1xj5Vn3DMzApQC5ZMI2Js9XtJvwdu6OB0M7NeoV6fS79z3RlOuiywTNmBmJkVJWCuMmY6KVGRNtOX+bjNtB/wEnBwPYMyM6ulZabgg486tI4grfsE8GFEdDhhtJlZb0gjoBodxaw6TaYREZLGRcTqvRWQmVlNTbigXpFWh39KGln3SMzMCqqUTGttvamzNaDmiogPgHWB70h6DHiT9HNERDjBmlnDNFmTaafV/H8CI4GteykWM7NChOjfZNm0s2QqgIh4rJdiMTMrpsU67S8iaf+ODlZPV2Vm1tsKzmfaazpLpv2BQbQ/o4qZWcOUNTa/k9VJFwLGAsNIU/BtHxEvd3atzpLp9Ig4qsfRmpnVQUkF045WJ90NmBgRx0o6mDRQ6aDOLtRZ1yiXSM2sKYlyZtqPiOkRcU9+/TpQWZ10K+CCfNoFFHgQ31nJ9EsFYjEz630q3GY6VNLdVe/Pioiz2r3krKuTLpaXgSYipktatNaNOkymEfFSkUjNzHpbmQvqweyrk3Zn3H+TzbtiZlaMCmyFrtPO6qTAc5VF9fKfz9e6jpOpmbUg0a9f7a3mVTpYnRQYD3wzv/4mcEWta3VnPlMzs4aqPIAqQUerkx4LXCZpD+A/wHa1LuRkamYtqYz5TDtZnRS6+BDeydTMWk/xp/m9xsnUzFpOidX80jiZmllLaqllS8zMmlUrzRplZtaUUjW/ubKpk6mZtaQmq+U7mZpZK5Kf5puZ9ZSr+WZmZVDzVfObrauWNYnD9tuLdT47jC02XOOjfdde+Wc2H/UFVlliMPffe08Do+t7Zs6cyVpf+Dxf3WrzRofSNPpJNbdejadX72YtY+sdduGsi/4yy74VV16F0353MV9Ya50GRdV3nX7qKaz06U83Ooymkabgq731JidTa9caa63LAgsuOMu+5VdcmWVXGN6giPqup59+mmuvuYpv7f7tRofSVFTgv97kNlOzJnfgD3/AMb84jjfeeL3RoTSVZnuaX9eSqaTFJV0q6TFJ/5Z0taThkj4j6UZJj0iaKunHeV5BJO0m6fR2rvWkpKFt9u0m6QVJk6u2VfKx4fl+j0p6UNJlknaoOu8NSQ/n1xdKGiVpQtW1t5Y0RdJDku6TtHXVsfMl/VfSvPn9UElP1unXaH3Y1VdNYNFFFmXk6qs3OpSmUlY1X9K5kp6XdH/VviPz97uSKzYrElPdkmlOjuOAmyNi+YhYhTRP4GKkiVePjYjhwAhgbWDvbt5qbESsVrX9W9J8wFXAmRGxQkR8GjgTeKByHnA3sEt+/402sY8ATgC2ioiVgS2BEyR9ruq0mcDu3YzZrJDbb/sHEyaMZ6UVhvGNXXbk5ptu5Fvf2LXRYTWBIpX8QiXX84HR7ew/qSqnXF3kQvUsmW4IvB8Rv6nsiIjJwHDgHxFxfd73FrAPaSnVsuwM3B4RV1bd+6aIuL+Tz1Q7APh5RDyRP/sE8AvgwKpzTgb2k+SmEqubo4/5BY89+TQPP/okF150KaM23IjzLvxDo8NqvAKl0iIl04j4O1DKenf1TKarApPa2f+Ztvsj4jFgkKQh3bhPddV9sqQBndy7qNliJJVkP1P1/j/AraRZujskaYykuyXd/fKLM3oQUu/64V67seMWG/HkY1MZtfpwLr/4Am64ZjyjVh/O5En/ZM+vf41v77RVo8O0PqqyoF6BrlFDK9+/vI0peIt9cjPfuZIWrH16Yx5ACYgOjnW0vzNjI2KfWW7Q84bp9mJsb9/PSU0WV3V0obys7FkAq44Y2Z2fryFOPPP8dvdvvOmWvRuIfWT9DUax/gajGh1G0yj4LS+0OmkbZwJHk77vRwMnUqBJr54l0weA9lrNHwBm+eEkLQe8ERFlPa7s6N5d+Xzbv4CRwL+rd0TEo8BkYPse3MvMukFSza07IuK5iJgZER8CZwNrFvlcPZPpjcC8kr5T2SFpDWAqsK6kL+d9A4BTgeNKvPfFwNqSvlJ179GSPlvw8ycAh0galj87jPTw7MR2zj2G1MZqZr1Iqr1177ppiedsG6DQs5a6JdOIiBzIxrlr1APAkcAzwFbA4ZIeBu4D7gKqu0PtJunpqm3JvH9K1b7Ksqxt20zXjoi3gc2BfXPXq38Du1Fg7esc+2TgIOBKSQ8BVwI/yvvbnvsA4LGVZr1MBbaa15AuAW4HVsp5ZQ/guNwdcgrpQfp+heJJOc96w6ojRsbl197S6DD6rGGLfKLRIfRpA+bWpG60X7Zrlc9+Pi4c/7ea562x3Pyl3bMWd+sxs9bThLNGOZmaWUtqslzqZGpmraj7T+vrxcnUzFpSk+VSJ1Mzaz1Fn9b3JidTM2tJruabmZWgyXKpk6mZtaYmy6VOpmbWguRqvplZjwlX883MStFkudTJ1Mxak6v5ZmYlaLJcWt/VSc3M6qWkKfjaW510IUk35Ok7byi6bImTqZm1nPQAqpSZ9s9n9tVJDwYmRsSKwEQKLvbpZGpmrafALPtFcmkHq5NuBVyQX18AbF0kJLeZmllLKthkOlTS3VXvz8qLXHZmsYiYDhAR0yUtWuRGTqZm1oIKV+O7szppt7iab2YtqV4L6gHPVRbVy38WWjvOydTMWk6RJ/k96Dk1Hvhmfv1N4IoiH3I138xaUhmd9vPqpKNIbatPA0cAxwKX5ZVK/wNsV+RaTqZm1pLK6LQfETt1cOhLXb2Wk6mZtaQmGwDlZGpmLchT8JmZ9Zyn4DMzK0mT5VInUzNrTf2arGjqZGpmram5cqmTqZm1pibLpU6mZtZ6JFfzzczK0Vy51MnUzFpTk+VSJ1Mza0VyNd/MrKeasdO+p+AzMyuBS6Zm1pLKquZLehJ4HZgJfNDdmfmdTM2s9fRsJv32bBgRM3pyASdTM2s5PZxJvy7cZmpmLUlSzY28OmnVNqadSwVwvaRJHRwvxCVTM2tJBav5RVYnXScinslLOt8g6aGI+HtX43HJ1MxaUlkL6kXEM/nP54FxwJrdicfJ1MxaUsFqfq1rfELS4MprYBPg/m7FExHd+Zx1g6QXgGmNjqMHhgI9euJpPdLqv/9lImKRMi4k6VrS76OWGRExupPrLEcqjUJq9rw4Io7pVkxOplaUpLu72wfPes6//+bmar6ZWQmcTM3MSuBkal1xVqMD6OP8+29ibjM1MyuBS6ZmZiVwMjUzK4GTqVkfI8nf+zrwL9VKJ2llSQMaHYfNStJIgIj40Am1fP6FWqkkjQauApZodCw2m8MlTQQn1HrwL9NKI2lz4BDguxHxaB7rbM1je+BlSVeCE2rZ/Iu0UkgaQhrjfGdE/FXSMGCcpE83NLA+TtI6kkZKWjAiPgC2A96UdA04oZbJ/UytxyStkEui/wtcBBwBbA1cGRGnNja6vkvSEsDNwGLALcBk4DLgceBkYP6I2Daf2y8iPmxQqHME/4tkPSJpM+B8SctFxHXAzsCvgOcqiVRS/0bG2FdFxH+B44G/AVeT2rG/A1wL3AlsLGlsPteJtIecTK3bckn0aOAnEfG4pAERcT2wGbCFpK/lU/1F7UWSFpa0JEBEnAVcCQwHLge+B5xIWqpjGvA/kj7VqFjnJF62xLpF0jLAr4HjIuJGSUsDp0saExETJW0PXCxpvoi4qLHR9h35IeDh+fUTwJPAYcBupAdQHwBXRMRMSeOAfnmGeeshJ1PrloiYlr+MX5Q0jfQFHhcRz+b2t+sk7QacIWk88Ea4gb6uJG1MamLZA5gCfC6/vhTYEegPbAsMkHRDT5c2tln5AZR1SX5qP29EvJDfHwx8G/hjRByS9/UHIj8p/kREvNm4iPsOSYcBT0TExVX7FiZV6x+PiKMkHQ4sChziv5dyuWRqhUn6CvAjYG5JTwK3AccBbwHrSfos8O9chawswPNWQ4LtmxYHFgE+SqYR8aKkS4Fv5Pc/k7SQE2n5/ADKCskPm04GjgK2ASaQqpEn5af29wI/AUZAKpZW/2n1IelzkvbLb69JuzR/m9MmAZ+UNBQgIl7qzRj7CidTqyl36v4ScGhETIyI54BLgDNI7W97RMTPgOeA70uat4Hh9hmShpMWldskt0/fBKwC/EjSAlWnjiZ919/u9SD7EFfzrabc9jkAWAFSm2iuyk8B/glsCJwTEftIWiQi3m1kvH1BngPhAGBf0kOnfYE3SE/sxwLHSJoPeBDYHdjBVfv6cjK1oh4glXrIiXSuiPggL7m7vaRBEfFG5cGU1U9ucjkB2DsiHsy9KURKqAK2BNYgrQH/HvC1iHiwUfH2FU6m1i5JI4ClgPlJ7aEXAv+UdHxEHJjHeUP6wkLqv2h1lhPpWNKopltyN7S3JN2ST9kHGBwR55KGkFovcTK12eQhoieQhiB+HpgJXAd8EfiHpIHA68CzwJ7AthHxToPC7TMkfRE4lTRkdztS9f544JmIeDsn1AAOzk0xZzcu2r7H/UxtFrn700+A/SLitpw4VwZ+Q+r8fSbwtbzvXeBPEfHvRsXbV+TZtz4LTI+IWyQtRFqt9D/A8RExPZ83EPgfYGpEPN2wgPsgJ1MDUn8aYGFS1fBvEbFnm+NrAD8FvhcRj1Y+465P9Zf/gTsMOIc0zeFrub16fuB3wFNUJVRrDHeNsoqheXjhj4AFJf1fm+NTSd1wVqnscCKtP0mbkPr3HgCcGxEv5UTaPyJeJY0++xRwpKTFGxlrX+dkakhaBLhZ0tci4krgfFLfxb0q50TEK8BdwMuNibLP2hI4LCJuI39fc41gZn749CqwFzAvqb3UGsTJ1MjdmY4ADpO0ZURcQ2oj3bSSUCXtDKwHPNG4SPuOquG4y1ftbjuqbJk87eHLwLfyYAprED/NNwAi4nJJH5A6exMR4/P3eTdJXyZVJXf2Q436k7Q+8A5pQMQ1wLqSboyIGVUDJgYCWwF/AN52k0vjuWTaR0naVNL+klat7IuIv5Ce5B9TVUK9mDRD+5iIuL9B4fYZuY30fD4u6NwNDAS2lDQ0Imbm/VuR5kjwd7hJuGTaB+Wx9psCuwJrSHqf9IDjrYgYl6fQOzJP7HyZpL96KGL95af2RwK75m5piwD3AX/nt/tdAAAHP0lEQVQFRpHmjr0VWIi0/Mj2nti5efhftT6k0g6X1/u5BngE2I805PAg4ERJwyLiclIb6r6SBjuR1l9+En8kaXXX2yQtBtwO/G9EXAqcS5r9aWNSk8u2rik0F/cz7UMq4+mr3l8G3BcRR0v6PvBz4GHSaKfrSF9szzTUCyQNIg2GWBWYAWwO/D6v4WQtwMm0j8hLWuxOGmf/UET8JQ9P3IzULncSaQLht4B1gT/7YVPvqAx+yNPmbUaasOTpiNiu6pyvAq8BN3ol0ebkZNoH5OnajgJ+T1qyYkngNNL66eOA9YGNI+LGfP4sJVirD0mLVbozVSXUQcAWwDrApIg4T9K2wM+ALSJiagNDtk64zXQOl8dwXw0cHRGnkcZzDwSWzR3xfwBMJM17CYATaX0pGUqahWtnSH1Hc0J9g7Su/W3Aakrr2h8AbONE2tycTOdweYmKLYBjJQ2JiKeA94Gh+YHUNNKopnWqOopbHUUygzRd3pGStqvsz/1IXyZNKnMvMAj4jucjbX7uGtUHRMRVkj4EJkm6jlQyvTB39H5F0hWkKqXbfHqBPl699crcLe3MXCq9DKi0h34FGALsFBGvNSpWK85tpn1IHsl0PbB4RDwvaWBEePXQXiJpJdJopf/kvr6VJWFGk6Y2PCj36x1DGjyxUUQ80sCQrQucTPsYSZuSJn7e0B2+e0+eLu8gYAHgFxHxVFUJtZJQTwXuIE3CvW1E3Nu4iK2rnEz7IElbkTrlf4HchNfgkPoESWsCWwPzASfnEurcwId5vP1o4JfA1yNiSiNjta5zMu2jKgvgNTqOOZ3ScszzA68AjwHDgDHAPMCpEfFkPm8f0jIwE7wETGtyMjWrE0lbkkaVTSNNn7cYqVP+QqQBFHMDB5JGPp0CbBYR/2pMtNZTTqZmdSBpHdIyI7tExKS879ekOWE3ILWdfgv4MrAMaQz+fQ0K10rgfqZm9bE48LuImCRpXoCI2Bu4FRhPmmR7PGnOUifSOYBLpmYlkvSF/HJrYI2I+N+8f+6IeD8/cBoH7BkRT0uaNyLebVS8Vh6XTM1KkucjPZe01MgE4ClJW+W1mt7Pcx68T/rezQ/gRDrn8AgosxJI2oD0EGmXiLhT0nykiWQ2JCXPcXlV0e1I85G+0LhorR5czTcrgaT9gZkRcYqkeSLivTyZye6k5bGXJU1esjkp4bof6RzGJVOzHqhMnUdKlq/m3e/nqv0MSacCI0hzxN5Heij1WIPCtTpym6lZD1SNHhsHrCVp9cq+3Eb6DrAGMDEirncinXM5mZqV405St6cdckL9MLeR7gB8E3ipseFZvbnN1KwkkpYA9gC+BNwFvANsixe/6xOcTM1KJGkAsDppZNN04CZPo9c3OJmamZXAbaZmZiVwMjUzK4GTqZlZCZxMzcxK4GRqZlYCJ1MzsxI4mVrdSJopabKk+yX9UdLAHlxrlKQJ+fWWkg7u5NwFJO3djXscKemAovvbnHO+pG27cK9hktyRfw7iZGr19HZErBYRqwLvAXtWH1TS5f8HI2J8RBzbySkLAF1OpmY94WRqveUWYIVcInswr4d0D7CUpE0k3S7pnlyCHQQgabSkhyTdCny1ciFJu0k6Pb9eTNI4SffmbW3gWGD5XCo+Pp93oKS7JE2R9NOqax0m6WFJfwVWqvVDSPpOvs69kv7UprT9ZUm3SHpE0ub5/P6Sjq+693d7+ou05uRkanUnaS5gU9IUdJCS1oUR8XngTeBw4MsRMRK4G9g/T658NrAFaRG6xTu4/KnA3yJiBDASeAA4GHgsl4oPlLQJsCKwJrAasLqk9SWtDuwIfJ6UrNco8OP8OSLWyPd7kDQWv2IYabG8rwC/yT/DHsCrEbFGvv53JC1b4D7WYjyfqdXTAEmT8+tbSKt1fgqYFhF35P1rkSZP/ockSOvJ3w6sDDwREVMBJP2BtN58WxsB3wCIiJnAq5IWbHPOJnmrLKM8iJRcB5NmwH8r32N8gZ9pVUk/IzUlDAKuqzp2WUR8CEyV9Hj+GTYBPlfVnjp/vrfH689hnEytnt6OiNWqd+SE+Wb1LuCGiNipzXmrkdaaL4OAX0TEb9vc4wfduMf5wNYRca+k3YBRVcfaXivyvfeNiOqki6RhXbyvNTlX863R7gDWkbQCgKSBkoYDDwHLSlo+n7dTB5+fCOyVP9tf0hDgdVKps+I6YPeqttglJC0K/B3YRtIASYNJTQq1DAam51VGd2lzbDtJ/XLMywEP53vvlc9H0nBJnyhwH2sxLplaQ0XEC7mEd4ny+vLA4RHxiKQxwFWSZpAmXl61nUt8HzhL0h7ATGCviLhd0j9y16Nrcrvpp4Hbc8n4DWDXiLhH0lhgMjCN1BRRy49JE0FPI7UBVyfth4G/AYuRlnJ+R9LvSG2p9yjd/AXSMtA2h/EUfGZmJXA138ysBE6mZmYlcDI1MyuBk6mZWQmcTM3MSuBkamZWAidTM7MS/D+wnJZXgOrrowAAAABJRU5ErkJggg==\n",
                        "text/plain": "<Figure size 432x288 with 2 Axes>"
                    },
                    "metadata": {
                        "needs_background": "light"
                    },
                    "output_type": "display_data"
                }
            ],
            "source": "def plot_confusion_matrix(cm, classes,\n                          normalize=False,\n                          title='Confusion matrix',\n                          cmap=plt.cm.Blues):\n    \"\"\"\n    This function prints and plots the confusion matrix.\n    Normalization can be applied by setting `normalize=True`.\n    \"\"\"\n    if normalize:\n        cm = cm.astype('float') / cm.sum(axis=1)[:, np.newaxis]\n        print(\"Normalized confusion matrix\")\n    else:\n        print('Confusion matrix, without normalization')\n\n    print(cm)\n\n    plt.imshow(cm, interpolation='nearest', cmap=cmap)\n    plt.title(title)\n    plt.colorbar()\n    tick_marks = np.arange(len(classes))\n    plt.xticks(tick_marks, classes, rotation=45)\n    plt.yticks(tick_marks, classes)\n\n    fmt = '.2f' if normalize else 'd'\n    thresh = cm.max() / 2.\n    for i, j in itertools.product(range(cm.shape[0]), range(cm.shape[1])):\n        plt.text(j, i, format(cm[i, j], fmt),\n                 horizontalalignment=\"center\",\n                 color=\"white\" if cm[i, j] > thresh else \"black\")\n\n    plt.tight_layout()\n    plt.ylabel('True label')\n    plt.xlabel('Predicted label')\n    \n##    \n# Compute confusion matrix\ncnf_matrix = confusion_matrix(y_test, yhat, labels=['PAIDOFF','COLLECTION'])\nnp.set_printoptions(precision=2)\n\nprint (classification_report(y_test, yhat))\n\n# Plot non-normalized confusion matrix\nplt.figure()\nplot_confusion_matrix(cnf_matrix, classes=['PAIDOFF','COLLECTION'],normalize= False,  title='Confusion matrix')    \n\nfrom sklearn.metrics import f1_score\nprint('f1_score:')\nprint(f1_score(y_test, yhat, average='weighted') )\n\nfrom sklearn.metrics import jaccard_similarity_score\nprint(\"jaccard_similarity_score:\")\nprint(jaccard_similarity_score(y_test, yhat))"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": "comparelist(y_test,yhat)"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Logistic Regression"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "Steps:\n+ Data pre-processing and selection\n+ Train Test Split\n+ Modeling\n+ evaluation"
        },
        {
            "cell_type": "code",
            "execution_count": 34,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "Train set: (276, 8) (276,)\nTest set: (70, 8) (70,)\n"
                },
                {
                    "data": {
                        "text/plain": "LogisticRegression(C=0.01, class_weight=None, dual=False, fit_intercept=True,\n          intercept_scaling=1, max_iter=100, multi_class='warn',\n          n_jobs=None, penalty='l2', random_state=None, solver='liblinear',\n          tol=0.0001, verbose=0, warm_start=False)"
                    },
                    "execution_count": 34,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "\n\n# Train/Test dataset\nfrom sklearn.model_selection import train_test_split\nX_train_LR, X_test_LR, y_train_LR, y_test_LR = train_test_split( X, y, test_size=0.2, random_state=4)\nprint ('Train set:', X_train_LR.shape,  y_train_LR.shape)\nprint ('Test set:', X_test_LR.shape,  y_test_LR.shape)\n\nfrom sklearn.linear_model import LogisticRegression\nfrom sklearn.metrics import confusion_matrix\nLR = LogisticRegression(C=0.01, solver='liblinear').fit(X_train_LR,y_train_LR)\nLR"
        },
        {
            "cell_type": "code",
            "execution_count": 151,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "['COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'COLLECTION' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'COLLECTION'\n 'COLLECTION' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF']\n['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'COLLECTION' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'COLLECTION'\n 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'COLLECTION' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'COLLECTION'\n 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF']\n"
                }
            ],
            "source": "yhat_LR = LR.predict(X_test_LR)\nprint(yhat_LR)\nprint(y_test_LR)"
        },
        {
            "cell_type": "code",
            "execution_count": 149,
            "metadata": {},
            "outputs": [],
            "source": "yhat_prob_LR = LR.predict_proba(X_test_LR)"
        },
        {
            "cell_type": "code",
            "execution_count": 150,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "jaccard_similarity_score\n0.6857142857142857\nlog_loss\n0.5772287609479654\n"
                }
            ],
            "source": "# evaluation\nfrom sklearn.metrics import jaccard_similarity_score\nprint('jaccard_similarity_score')\nprint(jaccard_similarity_score(y_test_LR, yhat_LR))\n\nfrom sklearn.metrics import log_loss\nprint('log_loss')\nprint(log_loss(y_test_LR, yhat_prob_LR))"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": ""
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Model Evaluation using Test set"
        },
        {
            "cell_type": "code",
            "execution_count": 33,
            "metadata": {},
            "outputs": [],
            "source": "from sklearn.metrics import jaccard_similarity_score\nfrom sklearn.metrics import f1_score\nfrom sklearn.metrics import log_loss"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "First, download and load the test set:"
        },
        {
            "cell_type": "code",
            "execution_count": 34,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "--2020-05-09 01:13:41--  https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv\nResolving s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)... 67.228.254.196\nConnecting to s3-api.us-geo.objectstorage.softlayer.net (s3-api.us-geo.objectstorage.softlayer.net)|67.228.254.196|:443... connected.\nHTTP request sent, awaiting response... 200 OK\nLength: 3642 (3.6K) [text/csv]\nSaving to: \u2018loan_test.csv\u2019\n\n100%[======================================>] 3,642       --.-K/s   in 0s      \n\n2020-05-09 01:13:41 (179 MB/s) - \u2018loan_test.csv\u2019 saved [3642/3642]\n\n"
                }
            ],
            "source": "!wget -O loan_test.csv https://s3-api.us-geo.objectstorage.softlayer.net/cf-courses-data/CognitiveClass/ML0101ENv3/labs/loan_test.csv"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "### Load Test set for evaluation "
        },
        {
            "cell_type": "code",
            "execution_count": 35,
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/8/2016</td>\n      <td>10/7/2016</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>9/9/2016</td>\n      <td>9/15/2016</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>9/10/2016</td>\n      <td>10/9/2016</td>\n      <td>26</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>9/11/2016</td>\n      <td>9/25/2016</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30       9/8/2016   \n1           5             5     PAIDOFF        300      7       9/9/2016   \n2          21            21     PAIDOFF       1000     30      9/10/2016   \n3          24            24     PAIDOFF       1000     30      9/10/2016   \n4          35            35     PAIDOFF        800     15      9/11/2016   \n\n    due_date  age             education  Gender  \n0  10/7/2016   50              Bechalor  female  \n1  9/15/2016   35       Master or Above    male  \n2  10/9/2016   43  High School or Below  female  \n3  10/9/2016   26               college    male  \n4  9/25/2016   29              Bechalor    male  "
                    },
                    "execution_count": 35,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df = pd.read_csv('loan_test.csv')\ntest_df.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 36,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>2016-09-09</td>\n      <td>2016-09-15</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>female</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>26</td>\n      <td>college</td>\n      <td>male</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>2016-09-11</td>\n      <td>2016-09-25</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>male</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30     2016-09-08   \n1           5             5     PAIDOFF        300      7     2016-09-09   \n2          21            21     PAIDOFF       1000     30     2016-09-10   \n3          24            24     PAIDOFF       1000     30     2016-09-10   \n4          35            35     PAIDOFF        800     15     2016-09-11   \n\n    due_date  age             education  Gender  \n0 2016-10-07   50              Bechalor  female  \n1 2016-09-15   35       Master or Above    male  \n2 2016-10-09   43  High School or Below  female  \n3 2016-10-09   26               college    male  \n4 2016-09-25   29              Bechalor    male  "
                    },
                    "execution_count": 36,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# Pre-processing: Feature selection/extraction\n\n# Convert to date time object\ntest_df['due_date'] = pd.to_datetime(test_df['due_date'])\ntest_df['effective_date'] = pd.to_datetime(test_df['effective_date'])\ntest_df.head()\n"
        },
        {
            "cell_type": "code",
            "execution_count": 37,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "PAIDOFF       40\nCOLLECTION    14\nName: loan_status, dtype: int64"
                    },
                    "execution_count": 37,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df['loan_status'].value_counts()"
        },
        {
            "cell_type": "code",
            "execution_count": 38,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>2016-09-09</td>\n      <td>2016-09-15</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>male</td>\n      <td>4</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>female</td>\n      <td>5</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>26</td>\n      <td>college</td>\n      <td>male</td>\n      <td>5</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>2016-09-11</td>\n      <td>2016-09-25</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>male</td>\n      <td>6</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30     2016-09-08   \n1           5             5     PAIDOFF        300      7     2016-09-09   \n2          21            21     PAIDOFF       1000     30     2016-09-10   \n3          24            24     PAIDOFF       1000     30     2016-09-10   \n4          35            35     PAIDOFF        800     15     2016-09-11   \n\n    due_date  age             education  Gender  dayofweek  \n0 2016-10-07   50              Bechalor  female          3  \n1 2016-09-15   35       Master or Above    male          4  \n2 2016-10-09   43  High School or Below  female          5  \n3 2016-10-09   26               college    male          5  \n4 2016-09-25   29              Bechalor    male          6  "
                    },
                    "execution_count": 38,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df['dayofweek'] = test_df['effective_date'].dt.dayofweek\ntest_df.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 39,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>female</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>2016-09-09</td>\n      <td>2016-09-15</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>male</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>female</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>26</td>\n      <td>college</td>\n      <td>male</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>2016-09-11</td>\n      <td>2016-09-25</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>male</td>\n      <td>6</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30     2016-09-08   \n1           5             5     PAIDOFF        300      7     2016-09-09   \n2          21            21     PAIDOFF       1000     30     2016-09-10   \n3          24            24     PAIDOFF       1000     30     2016-09-10   \n4          35            35     PAIDOFF        800     15     2016-09-11   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   50              Bechalor  female          3        0  \n1 2016-09-15   35       Master or Above    male          4        1  \n2 2016-10-09   43  High School or Below  female          5        1  \n3 2016-10-09   26               college    male          5        1  \n4 2016-09-25   29              Bechalor    male          6        1  "
                    },
                    "execution_count": 39,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df['weekend'] = test_df['dayofweek'].apply(lambda x: 1 if (x>3)  else 0)\ntest_df.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 40,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "Gender  loan_status\nfemale  PAIDOFF        0.727273\n        COLLECTION     0.272727\nmale    PAIDOFF        0.744186\n        COLLECTION     0.255814\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 40,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# Convert Categorical features to numerical values\ntest_df.groupby(['Gender'])['loan_status'].value_counts(normalize=True)\n"
        },
        {
            "cell_type": "code",
            "execution_count": 41,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Unnamed: 0</th>\n      <th>Unnamed: 0.1</th>\n      <th>loan_status</th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>effective_date</th>\n      <th>due_date</th>\n      <th>age</th>\n      <th>education</th>\n      <th>Gender</th>\n      <th>dayofweek</th>\n      <th>weekend</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1</td>\n      <td>1</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-08</td>\n      <td>2016-10-07</td>\n      <td>50</td>\n      <td>Bechalor</td>\n      <td>1</td>\n      <td>3</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>5</td>\n      <td>5</td>\n      <td>PAIDOFF</td>\n      <td>300</td>\n      <td>7</td>\n      <td>2016-09-09</td>\n      <td>2016-09-15</td>\n      <td>35</td>\n      <td>Master or Above</td>\n      <td>0</td>\n      <td>4</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>21</td>\n      <td>21</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>43</td>\n      <td>High School or Below</td>\n      <td>1</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>24</td>\n      <td>24</td>\n      <td>PAIDOFF</td>\n      <td>1000</td>\n      <td>30</td>\n      <td>2016-09-10</td>\n      <td>2016-10-09</td>\n      <td>26</td>\n      <td>college</td>\n      <td>0</td>\n      <td>5</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>35</td>\n      <td>35</td>\n      <td>PAIDOFF</td>\n      <td>800</td>\n      <td>15</td>\n      <td>2016-09-11</td>\n      <td>2016-09-25</td>\n      <td>29</td>\n      <td>Bechalor</td>\n      <td>0</td>\n      <td>6</td>\n      <td>1</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Unnamed: 0  Unnamed: 0.1 loan_status  Principal  terms effective_date  \\\n0           1             1     PAIDOFF       1000     30     2016-09-08   \n1           5             5     PAIDOFF        300      7     2016-09-09   \n2          21            21     PAIDOFF       1000     30     2016-09-10   \n3          24            24     PAIDOFF       1000     30     2016-09-10   \n4          35            35     PAIDOFF        800     15     2016-09-11   \n\n    due_date  age             education  Gender  dayofweek  weekend  \n0 2016-10-07   50              Bechalor       1          3        0  \n1 2016-09-15   35       Master or Above       0          4        1  \n2 2016-10-09   43  High School or Below       1          5        1  \n3 2016-10-09   26               college       0          5        1  \n4 2016-09-25   29              Bechalor       0          6        1  "
                    },
                    "execution_count": 41,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "# convert male to 0 and female to 1\ntest_df['Gender'].replace(to_replace=['male','female'], value=[0,1],inplace=True)\ntest_df.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 42,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "education             loan_status\nBechalor              PAIDOFF        1.000000\nHigh School or Below  PAIDOFF        0.523810\n                      COLLECTION     0.476190\nMaster or Above       PAIDOFF        1.000000\ncollege               PAIDOFF        0.826087\n                      COLLECTION     0.173913\nName: loan_status, dtype: float64"
                    },
                    "execution_count": 42,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df.groupby(['education'])['loan_status'].value_counts(normalize=True)"
        },
        {
            "cell_type": "code",
            "execution_count": 43,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>education</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>50</td>\n      <td>1</td>\n      <td>Bechalor</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>300</td>\n      <td>7</td>\n      <td>35</td>\n      <td>0</td>\n      <td>Master or Above</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>43</td>\n      <td>1</td>\n      <td>High School or Below</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>26</td>\n      <td>0</td>\n      <td>college</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>800</td>\n      <td>15</td>\n      <td>29</td>\n      <td>0</td>\n      <td>Bechalor</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender             education\n0       1000     30   50       1              Bechalor\n1        300      7   35       0       Master or Above\n2       1000     30   43       1  High School or Below\n3       1000     30   26       0               college\n4        800     15   29       0              Bechalor"
                    },
                    "execution_count": 43,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_df[['Principal','terms','age','Gender','education']].head()"
        },
        {
            "cell_type": "code",
            "execution_count": 44,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>50</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>300</td>\n      <td>7</td>\n      <td>35</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>43</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>26</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>800</td>\n      <td>15</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   50       1        0         1                     0   \n1        300      7   35       0        1         0                     0   \n2       1000     30   43       1        1         0                     1   \n3       1000     30   26       0        1         0                     0   \n4        800     15   29       0        1         1                     0   \n\n   college  \n0        0  \n1        0  \n2        0  \n3        1  \n4        0  "
                    },
                    "execution_count": 44,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "Test_Feature  = test_df[['Principal','terms','age','Gender','weekend']]\nTest_Feature  = pd.concat([Test_Feature,pd.get_dummies(test_df['education'])], axis=1)\nTest_Feature.drop(['Master or Above'], axis = 1,inplace=True)\nTest_Feature.head()"
        },
        {
            "cell_type": "code",
            "execution_count": 45,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/html": "<div>\n<style scoped>\n    .dataframe tbody tr th:only-of-type {\n        vertical-align: middle;\n    }\n\n    .dataframe tbody tr th {\n        vertical-align: top;\n    }\n\n    .dataframe thead th {\n        text-align: right;\n    }\n</style>\n<table border=\"1\" class=\"dataframe\">\n  <thead>\n    <tr style=\"text-align: right;\">\n      <th></th>\n      <th>Principal</th>\n      <th>terms</th>\n      <th>age</th>\n      <th>Gender</th>\n      <th>weekend</th>\n      <th>Bechalor</th>\n      <th>High School or Below</th>\n      <th>college</th>\n    </tr>\n  </thead>\n  <tbody>\n    <tr>\n      <th>0</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>50</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>1</th>\n      <td>300</td>\n      <td>7</td>\n      <td>35</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>2</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>43</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n    </tr>\n    <tr>\n      <th>3</th>\n      <td>1000</td>\n      <td>30</td>\n      <td>26</td>\n      <td>0</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n      <td>1</td>\n    </tr>\n    <tr>\n      <th>4</th>\n      <td>800</td>\n      <td>15</td>\n      <td>29</td>\n      <td>0</td>\n      <td>1</td>\n      <td>1</td>\n      <td>0</td>\n      <td>0</td>\n    </tr>\n  </tbody>\n</table>\n</div>",
                        "text/plain": "   Principal  terms  age  Gender  weekend  Bechalor  High School or Below  \\\n0       1000     30   50       1        0         1                     0   \n1        300      7   35       0        1         0                     0   \n2       1000     30   43       1        1         0                     1   \n3       1000     30   26       0        1         0                     0   \n4        800     15   29       0        1         1                     0   \n\n   college  \n0        0  \n1        0  \n2        0  \n3        1  \n4        0  "
                    },
                    "execution_count": 45,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_X = Test_Feature\ntest_X[0:5]"
        },
        {
            "cell_type": "code",
            "execution_count": 46,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "array(['PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF',\n       'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'PAIDOFF', 'COLLECTION',\n       'COLLECTION', 'COLLECTION', 'COLLECTION', 'COLLECTION',\n       'COLLECTION', 'COLLECTION', 'COLLECTION', 'COLLECTION',\n       'COLLECTION', 'COLLECTION', 'COLLECTION', 'COLLECTION',\n       'COLLECTION'], dtype=object)"
                    },
                    "execution_count": 46,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_y = test_df['loan_status'].values\ntest_y"
        },
        {
            "cell_type": "code",
            "execution_count": 47,
            "metadata": {},
            "outputs": [
                {
                    "name": "stderr",
                    "output_type": "stream",
                    "text": "/opt/conda/envs/Python36/lib/python3.6/site-packages/sklearn/preprocessing/data.py:645: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  return self.partial_fit(X, y)\n/opt/conda/envs/Python36/lib/python3.6/site-packages/ipykernel/__main__.py:3: DataConversionWarning: Data with input dtype uint8, int64 were all converted to float64 by StandardScaler.\n  app.launch_new_instance()\n"
                }
            ],
            "source": "# Normalize Data\n\ntest_X = preprocessing.StandardScaler().fit(test_X).transform(test_X)\n"
        },
        {
            "cell_type": "code",
            "execution_count": 48,
            "metadata": {},
            "outputs": [
                {
                    "data": {
                        "text/plain": "array([[ 0.49362588,  0.92844966,  3.05981865,  1.97714211, -1.30384048,\n         2.39791576, -0.79772404, -0.86135677],\n       [-3.56269116, -1.70427745,  0.53336288, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404, -0.86135677],\n       [ 0.49362588,  0.92844966,  1.88080596,  1.97714211,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588,  0.92844966, -0.98251057, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [-0.66532184, -0.78854628, -0.47721942, -0.50578054,  0.76696499,\n         2.39791576, -0.79772404, -0.86135677],\n       [-1.24479571, -0.78854628,  0.19650211, -0.50578054,  0.76696499,\n        -0.41702883,  1.25356634, -0.86135677],\n       [ 0.49362588, -0.78854628, -1.31937134, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588,  0.92844966,  0.02807173, -0.50578054,  0.76696499,\n         2.39791576, -0.79772404, -0.86135677],\n       [-0.66532184, -0.78854628, -0.81408019,  1.97714211,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912],\n       [ 0.49362588, -0.78854628,  0.87022365, -0.50578054,  0.76696499,\n        -0.41702883, -0.79772404,  1.16095912]])"
                    },
                    "execution_count": 48,
                    "metadata": {},
                    "output_type": "execute_result"
                }
            ],
            "source": "test_X[:10]\n"
        },
        {
            "cell_type": "code",
            "execution_count": 49,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "['PAIDOFF' 'PAIDOFF' 'COLLECTION' 'COLLECTION' 'COLLECTION' 'COLLECTION'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'COLLECTION' 'PAIDOFF']\nKNN Jaccard Score: 0.6666666666666666\nKNN Avg F1-score: 0.6328400281888654\n"
                }
            ],
            "source": "# KNN\n\nknn_predict = KNN_model.predict(test_X)\n# print(test_y)\nprint(knn_predict)\nknn_jaccard = jaccard_similarity_score(test_y, knn_predict)\nknn_f1 = f1_score(test_y, knn_predict, average='weighted')\nprint(\"KNN Jaccard Score:\",knn_jaccard)\nprint(\"KNN Avg F1-score:\",knn_f1)"
        },
        {
            "cell_type": "code",
            "execution_count": 172,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "['PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'COLLECTION'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'COLLECTION' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'COLLECTION' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF'\n 'PAIDOFF' 'PAIDOFF' 'COLLECTION' 'PAIDOFF' 'COLLECTION' 'COLLECTION'\n 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'PAIDOFF' 'COLLECTION'\n 'PAIDOFF' 'PAIDOFF']\ndecision_tree Jaccard Score: 0.6851851851851852\ndecision_tree Avg F1-score: 0.6719857206604195\n"
                }
            ],
            "source": "# Decision Tree\n\ndecision_tree_predict = answerTree.predict(test_X)\n# print(test_y)\nprint(decision_tree_predict)\ndecision_tree_jaccard = jaccard_similarity_score(test_y, decision_tree_predict)\ndecision_tree_f1 = f1_score(test_y, decision_tree_predict, average='weighted')\nprint(\"decision_tree Jaccard Score:\",decision_tree_jaccard)\nprint(\"decision_tree Avg F1-score:\",decision_tree_f1)"
        },
        {
            "cell_type": "code",
            "execution_count": 173,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "SVM Jaccard Score: 0.7962962962962963\nSVM Avg F1-score: 0.7583503077293734\n"
                }
            ],
            "source": "# SVM\n\nSVM_predict = clf.predict(test_X)\n# print(test_y)\n# print(SVM_predict)\nSVM_jaccard = jaccard_similarity_score(test_y, SVM_predict)\nSVM_f1 = f1_score(test_y, SVM_predict, average='weighted')\nprint(\"SVM Jaccard Score:\",SVM_jaccard)\nprint(\"SVM Avg F1-score:\",SVM_f1)"
        },
        {
            "cell_type": "code",
            "execution_count": 174,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "LR Jaccard Score: 0.7407407407407407\nLR Avg F1-score: 0.6604267310789049\n"
                }
            ],
            "source": "# LR\n\nLR_predict = LR.predict(test_X)\n# print(LR_predict)\nLR_jaccard = jaccard_similarity_score(test_y, LR_predict)\nLR_f1 = f1_score(test_y, LR_predict, average='weighted')# pos_label='PAIDOFF'\nprint(\"LR Jaccard Score:\",LR_jaccard)\nprint(\"LR Avg F1-score:\",LR_f1)\n\n"
        },
        {
            "cell_type": "code",
            "execution_count": 175,
            "metadata": {},
            "outputs": [
                {
                    "name": "stdout",
                    "output_type": "stream",
                    "text": "LR log loss 0.5672153379912981\n"
                }
            ],
            "source": "# LR log loss\nfrom sklearn.metrics import log_loss\nLR_predict2 = LR.predict_proba(test_X)\n\nprint('LR log loss',log_loss(test_y, LR_predict2))"
        },
        {
            "cell_type": "code",
            "execution_count": null,
            "metadata": {},
            "outputs": [],
            "source": ""
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "# Report\nYou should be able to report the accuracy of the built model using different evaluation metrics:"
        },
        {
            "cell_type": "markdown",
            "metadata": {},
            "source": "| Algorithm          | Jaccard | F1-score | LogLoss |\n|--------------------|---------|----------|---------|\n| KNN                | 0.66       | 0.63        | NA      |\n| Decision Tree      | 0.68       | 0.67       | NA      |\n| SVM                | 0.79       | 0.75        | NA      |\n| LogisticRegression | 0.74       | 0.66        | 0.56       |"
        },
        {
            "cell_type": "markdown",
            "metadata": {
                "button": false,
                "new_sheet": false,
                "run_control": {
                    "read_only": false
                }
            },
            "source": "<h2>Want to learn more?</h2>\n\nIBM SPSS Modeler is a comprehensive analytics platform that has many machine learning algorithms. It has been designed to bring predictive intelligence to decisions made by individuals, by groups, by systems \u2013 by your enterprise as a whole. A free trial is available through this course, available here: <a href=\"http://cocl.us/ML0101EN-SPSSModeler\">SPSS Modeler</a>\n\nAlso, you can use Watson Studio to run these notebooks faster with bigger datasets. Watson Studio is IBM's leading cloud solution for data scientists, built by data scientists. With Jupyter notebooks, RStudio, Apache Spark and popular libraries pre-packaged in the cloud, Watson Studio enables data scientists to collaborate on their projects without having to install anything. Join the fast-growing community of Watson Studio users today with a free account at <a href=\"https://cocl.us/ML0101EN_DSX\">Watson Studio</a>\n\n<h3>Thanks for completing this lesson!</h3>\n\n<h4>Author:  <a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a></h4>\n<p><a href=\"https://ca.linkedin.com/in/saeedaghabozorgi\">Saeed Aghabozorgi</a>, PhD is a Data Scientist in IBM with a track record of developing enterprise level applications that substantially increases clients\u2019 ability to turn data into actionable knowledge. He is a researcher in data mining field and expert in developing advanced analytic methods like machine learning and statistical modelling on large datasets.</p>\n\n<hr>\n\n<p>Copyright &copy; 2018 <a href=\"https://cocl.us/DX0108EN_CC\">Cognitive Class</a>. This notebook and its source code are released under the terms of the <a href=\"https://bigdatauniversity.com/mit-license/\">MIT License</a>.</p>"
        }
    ],
    "metadata": {
        "kernelspec": {
            "display_name": "Python 3.6",
            "language": "python",
            "name": "python3"
        },
        "language_info": {
            "codemirror_mode": {
                "name": "ipython",
                "version": 3
            },
            "file_extension": ".py",
            "mimetype": "text/x-python",
            "name": "python",
            "nbconvert_exporter": "python",
            "pygments_lexer": "ipython3",
            "version": "3.6.9"
        }
    },
    "nbformat": 4,
    "nbformat_minor": 2
}