Bayesian_Optimization_with_Hyperopt.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "EkFpZxZRW2pm"
   },
   "source": [
    "## Model with Bayesian Optimization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "xx8XChoSW_Vl"
   },
   "source": [
    "Bayesian optimization uses probability to find the minimum of a function. The final aim is to find the input value to a function which can gives us the lowest possible output value."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "9-Gsf2wQXFLv"
   },
   "source": [
    "In Hyperopt, Bayesian Optimization can be implemented giving 3 three main parameters to the function\n",
    "\n",
    "Objective Function = Defines the loss function to minimize.\n",
    "\n",
    "Domain Space = Defines the range of input values to test\n",
    "\n",
    "Optimization Algorithm = Defines the search algorithm to use to select the best input values to use in each new iteration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xHNGzH8fVXmC"
   },
   "outputs": [],
   "source": [
    "import hyperopt\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sd3ExkRrVfr4"
   },
   "outputs": [],
   "source": [
    "from hyperopt import hp,fmin,tpe,STATUS_OK,Trials\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "vSs75oWAWDeE"
   },
   "outputs": [],
   "source": [
    "from google.colab import files\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "bvG93zWtXTo-"
   },
   "outputs": [],
   "source": [
    "!pip install -U -q PyDrive\n",
    "from pydrive.auth import GoogleAuth\n",
    "from pydrive.drive import GoogleDrive\n",
    "from google.colab import auth\n",
    "from oauth2client.client import GoogleCredentials"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "kdw0N1zsXzZP"
   },
   "outputs": [],
   "source": [
    "auth.authenticate_user()\n",
    "gauth = GoogleAuth()\n",
    "gauth.credentials = GoogleCredentials.get_application_default()\n",
    "drive = GoogleDrive(gauth)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "iNXw5Pjkb0cU"
   },
   "outputs": [],
   "source": [
    "downloaded = drive.CreateFile({'id':'1Blpp94I9dOaMt7V4nB-0meR4FygacAwN'}) # replace the id with id of file you want to access\n",
    "downloaded.GetContentFile('Real_combine.csv') "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "jFU7oa4ocfAI"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "2VK7pNVAcitI"
   },
   "outputs": [],
   "source": [
    "df=pd.read_csv('Real_combine.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 415
    },
    "colab_type": "code",
    "id": "WH7OfRTtcqeD",
    "outputId": "aea4e51c-688c-411f-8296-f4bbd9302c54"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>T</th>\n",
       "      <th>TM</th>\n",
       "      <th>Tm</th>\n",
       "      <th>SLP</th>\n",
       "      <th>H</th>\n",
       "      <th>VV</th>\n",
       "      <th>V</th>\n",
       "      <th>VM</th>\n",
       "      <th>PM 2.5</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>26.7</td>\n",
       "      <td>33.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1012.4</td>\n",
       "      <td>60.0</td>\n",
       "      <td>5.1</td>\n",
       "      <td>4.4</td>\n",
       "      <td>13.0</td>\n",
       "      <td>284.795833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>3</td>\n",
       "      <td>29.1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>20.5</td>\n",
       "      <td>1011.9</td>\n",
       "      <td>49.0</td>\n",
       "      <td>5.8</td>\n",
       "      <td>5.2</td>\n",
       "      <td>14.8</td>\n",
       "      <td>219.720833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5</td>\n",
       "      <td>28.4</td>\n",
       "      <td>36.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>1011.3</td>\n",
       "      <td>46.0</td>\n",
       "      <td>5.3</td>\n",
       "      <td>5.7</td>\n",
       "      <td>11.1</td>\n",
       "      <td>182.187500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>7</td>\n",
       "      <td>25.9</td>\n",
       "      <td>32.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1011.8</td>\n",
       "      <td>56.0</td>\n",
       "      <td>6.1</td>\n",
       "      <td>6.9</td>\n",
       "      <td>11.1</td>\n",
       "      <td>154.037500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>9</td>\n",
       "      <td>24.8</td>\n",
       "      <td>31.1</td>\n",
       "      <td>20.6</td>\n",
       "      <td>1013.6</td>\n",
       "      <td>58.0</td>\n",
       "      <td>4.8</td>\n",
       "      <td>8.3</td>\n",
       "      <td>11.1</td>\n",
       "      <td>223.208333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>638</th>\n",
       "      <td>1277</td>\n",
       "      <td>28.5</td>\n",
       "      <td>33.4</td>\n",
       "      <td>20.9</td>\n",
       "      <td>1012.6</td>\n",
       "      <td>59.0</td>\n",
       "      <td>5.3</td>\n",
       "      <td>6.3</td>\n",
       "      <td>14.8</td>\n",
       "      <td>185.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>639</th>\n",
       "      <td>1279</td>\n",
       "      <td>24.9</td>\n",
       "      <td>33.2</td>\n",
       "      <td>14.8</td>\n",
       "      <td>1011.5</td>\n",
       "      <td>48.0</td>\n",
       "      <td>4.2</td>\n",
       "      <td>4.6</td>\n",
       "      <td>13.0</td>\n",
       "      <td>166.875000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>640</th>\n",
       "      <td>1281</td>\n",
       "      <td>26.4</td>\n",
       "      <td>32.0</td>\n",
       "      <td>20.9</td>\n",
       "      <td>1011.2</td>\n",
       "      <td>70.0</td>\n",
       "      <td>3.9</td>\n",
       "      <td>6.7</td>\n",
       "      <td>9.4</td>\n",
       "      <td>200.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>641</th>\n",
       "      <td>1283</td>\n",
       "      <td>20.8</td>\n",
       "      <td>25.0</td>\n",
       "      <td>14.5</td>\n",
       "      <td>1016.8</td>\n",
       "      <td>78.0</td>\n",
       "      <td>4.7</td>\n",
       "      <td>5.9</td>\n",
       "      <td>11.1</td>\n",
       "      <td>349.291667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>642</th>\n",
       "      <td>1285</td>\n",
       "      <td>23.3</td>\n",
       "      <td>28.0</td>\n",
       "      <td>14.9</td>\n",
       "      <td>1014.0</td>\n",
       "      <td>71.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>9.4</td>\n",
       "      <td>310.250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>643 rows × 10 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "     Unnamed: 0     T    TM    Tm     SLP     H   VV    V    VM      PM 2.5\n",
       "0             1  26.7  33.0  20.0  1012.4  60.0  5.1  4.4  13.0  284.795833\n",
       "1             3  29.1  35.0  20.5  1011.9  49.0  5.8  5.2  14.8  219.720833\n",
       "2             5  28.4  36.0  21.0  1011.3  46.0  5.3  5.7  11.1  182.187500\n",
       "3             7  25.9  32.0  20.0  1011.8  56.0  6.1  6.9  11.1  154.037500\n",
       "4             9  24.8  31.1  20.6  1013.6  58.0  4.8  8.3  11.1  223.208333\n",
       "..          ...   ...   ...   ...     ...   ...  ...  ...   ...         ...\n",
       "638        1277  28.5  33.4  20.9  1012.6  59.0  5.3  6.3  14.8  185.500000\n",
       "639        1279  24.9  33.2  14.8  1011.5  48.0  4.2  4.6  13.0  166.875000\n",
       "640        1281  26.4  32.0  20.9  1011.2  70.0  3.9  6.7   9.4  200.333333\n",
       "641        1283  20.8  25.0  14.5  1016.8  78.0  4.7  5.9  11.1  349.291667\n",
       "642        1285  23.3  28.0  14.9  1014.0  71.0  4.5  3.0   9.4  310.250000\n",
       "\n",
       "[643 rows x 10 columns]"
      ]
     },
     "execution_count": 15,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ndpVFGZZcrUr"
   },
   "outputs": [],
   "source": [
    "df.drop(['Unnamed: 0'],axis=1,inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 415
    },
    "colab_type": "code",
    "id": "bJSzBDsNdJuF",
    "outputId": "fd6e0dae-7749-4c1b-e992-328dbb3bbc58"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>T</th>\n",
       "      <th>TM</th>\n",
       "      <th>Tm</th>\n",
       "      <th>SLP</th>\n",
       "      <th>H</th>\n",
       "      <th>VV</th>\n",
       "      <th>V</th>\n",
       "      <th>VM</th>\n",
       "      <th>PM 2.5</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>26.7</td>\n",
       "      <td>33.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1012.4</td>\n",
       "      <td>60.0</td>\n",
       "      <td>5.1</td>\n",
       "      <td>4.4</td>\n",
       "      <td>13.0</td>\n",
       "      <td>284.795833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>29.1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>20.5</td>\n",
       "      <td>1011.9</td>\n",
       "      <td>49.0</td>\n",
       "      <td>5.8</td>\n",
       "      <td>5.2</td>\n",
       "      <td>14.8</td>\n",
       "      <td>219.720833</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>28.4</td>\n",
       "      <td>36.0</td>\n",
       "      <td>21.0</td>\n",
       "      <td>1011.3</td>\n",
       "      <td>46.0</td>\n",
       "      <td>5.3</td>\n",
       "      <td>5.7</td>\n",
       "      <td>11.1</td>\n",
       "      <td>182.187500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>25.9</td>\n",
       "      <td>32.0</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1011.8</td>\n",
       "      <td>56.0</td>\n",
       "      <td>6.1</td>\n",
       "      <td>6.9</td>\n",
       "      <td>11.1</td>\n",
       "      <td>154.037500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>24.8</td>\n",
       "      <td>31.1</td>\n",
       "      <td>20.6</td>\n",
       "      <td>1013.6</td>\n",
       "      <td>58.0</td>\n",
       "      <td>4.8</td>\n",
       "      <td>8.3</td>\n",
       "      <td>11.1</td>\n",
       "      <td>223.208333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>638</th>\n",
       "      <td>28.5</td>\n",
       "      <td>33.4</td>\n",
       "      <td>20.9</td>\n",
       "      <td>1012.6</td>\n",
       "      <td>59.0</td>\n",
       "      <td>5.3</td>\n",
       "      <td>6.3</td>\n",
       "      <td>14.8</td>\n",
       "      <td>185.500000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>639</th>\n",
       "      <td>24.9</td>\n",
       "      <td>33.2</td>\n",
       "      <td>14.8</td>\n",
       "      <td>1011.5</td>\n",
       "      <td>48.0</td>\n",
       "      <td>4.2</td>\n",
       "      <td>4.6</td>\n",
       "      <td>13.0</td>\n",
       "      <td>166.875000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>640</th>\n",
       "      <td>26.4</td>\n",
       "      <td>32.0</td>\n",
       "      <td>20.9</td>\n",
       "      <td>1011.2</td>\n",
       "      <td>70.0</td>\n",
       "      <td>3.9</td>\n",
       "      <td>6.7</td>\n",
       "      <td>9.4</td>\n",
       "      <td>200.333333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>641</th>\n",
       "      <td>20.8</td>\n",
       "      <td>25.0</td>\n",
       "      <td>14.5</td>\n",
       "      <td>1016.8</td>\n",
       "      <td>78.0</td>\n",
       "      <td>4.7</td>\n",
       "      <td>5.9</td>\n",
       "      <td>11.1</td>\n",
       "      <td>349.291667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>642</th>\n",
       "      <td>23.3</td>\n",
       "      <td>28.0</td>\n",
       "      <td>14.9</td>\n",
       "      <td>1014.0</td>\n",
       "      <td>71.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>3.0</td>\n",
       "      <td>9.4</td>\n",
       "      <td>310.250000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>643 rows × 9 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        T    TM    Tm     SLP     H   VV    V    VM      PM 2.5\n",
       "0    26.7  33.0  20.0  1012.4  60.0  5.1  4.4  13.0  284.795833\n",
       "1    29.1  35.0  20.5  1011.9  49.0  5.8  5.2  14.8  219.720833\n",
       "2    28.4  36.0  21.0  1011.3  46.0  5.3  5.7  11.1  182.187500\n",
       "3    25.9  32.0  20.0  1011.8  56.0  6.1  6.9  11.1  154.037500\n",
       "4    24.8  31.1  20.6  1013.6  58.0  4.8  8.3  11.1  223.208333\n",
       "..    ...   ...   ...     ...   ...  ...  ...   ...         ...\n",
       "638  28.5  33.4  20.9  1012.6  59.0  5.3  6.3  14.8  185.500000\n",
       "639  24.9  33.2  14.8  1011.5  48.0  4.2  4.6  13.0  166.875000\n",
       "640  26.4  32.0  20.9  1011.2  70.0  3.9  6.7   9.4  200.333333\n",
       "641  20.8  25.0  14.5  1016.8  78.0  4.7  5.9  11.1  349.291667\n",
       "642  23.3  28.0  14.9  1014.0  71.0  4.5  3.0   9.4  310.250000\n",
       "\n",
       "[643 rows x 9 columns]"
      ]
     },
     "execution_count": 17,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "nJwTqXH4kK0g"
   },
   "outputs": [],
   "source": [
    "X=df.iloc[:,:-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "MPyxVlEekQQG"
   },
   "outputs": [],
   "source": [
    "y=df.iloc[:,-1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "sjiCPIIIlXdQ"
   },
   "outputs": [],
   "source": [
    "#### Train Test Split\n",
    "from sklearn.model_selection import train_test_split\n",
    "X_train,X_test,y_train,y_test=train_test_split(X,y,test_size=0.20,random_state=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 415
    },
    "colab_type": "code",
    "id": "FtfP2JJVkYu3",
    "outputId": "536f6203-3990-4e5f-e11d-413311dc2dfc"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>T</th>\n",
       "      <th>TM</th>\n",
       "      <th>Tm</th>\n",
       "      <th>SLP</th>\n",
       "      <th>H</th>\n",
       "      <th>VV</th>\n",
       "      <th>V</th>\n",
       "      <th>VM</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>261</th>\n",
       "      <td>27.7</td>\n",
       "      <td>33.1</td>\n",
       "      <td>23.0</td>\n",
       "      <td>1005.8</td>\n",
       "      <td>68.0</td>\n",
       "      <td>5.5</td>\n",
       "      <td>8.3</td>\n",
       "      <td>22.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>470</th>\n",
       "      <td>25.8</td>\n",
       "      <td>32.5</td>\n",
       "      <td>20.0</td>\n",
       "      <td>1012.6</td>\n",
       "      <td>60.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>6.1</td>\n",
       "      <td>11.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>612</th>\n",
       "      <td>27.7</td>\n",
       "      <td>32.8</td>\n",
       "      <td>21.0</td>\n",
       "      <td>1013.1</td>\n",
       "      <td>66.0</td>\n",
       "      <td>4.7</td>\n",
       "      <td>5.4</td>\n",
       "      <td>13.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>552</th>\n",
       "      <td>23.2</td>\n",
       "      <td>31.0</td>\n",
       "      <td>12.9</td>\n",
       "      <td>1013.3</td>\n",
       "      <td>41.0</td>\n",
       "      <td>4.0</td>\n",
       "      <td>3.0</td>\n",
       "      <td>9.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td>25.2</td>\n",
       "      <td>31.0</td>\n",
       "      <td>19.0</td>\n",
       "      <td>1015.4</td>\n",
       "      <td>48.0</td>\n",
       "      <td>4.7</td>\n",
       "      <td>11.1</td>\n",
       "      <td>16.5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td>23.7</td>\n",
       "      <td>30.4</td>\n",
       "      <td>17.0</td>\n",
       "      <td>1015.8</td>\n",
       "      <td>46.0</td>\n",
       "      <td>5.1</td>\n",
       "      <td>5.2</td>\n",
       "      <td>14.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>359</th>\n",
       "      <td>33.6</td>\n",
       "      <td>40.0</td>\n",
       "      <td>25.0</td>\n",
       "      <td>1006.9</td>\n",
       "      <td>36.0</td>\n",
       "      <td>5.8</td>\n",
       "      <td>6.1</td>\n",
       "      <td>11.1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>192</th>\n",
       "      <td>24.9</td>\n",
       "      <td>30.4</td>\n",
       "      <td>19.0</td>\n",
       "      <td>1008.9</td>\n",
       "      <td>57.0</td>\n",
       "      <td>4.8</td>\n",
       "      <td>4.6</td>\n",
       "      <td>9.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>629</th>\n",
       "      <td>26.1</td>\n",
       "      <td>29.0</td>\n",
       "      <td>22.4</td>\n",
       "      <td>1001.2</td>\n",
       "      <td>87.0</td>\n",
       "      <td>5.0</td>\n",
       "      <td>14.1</td>\n",
       "      <td>22.2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>559</th>\n",
       "      <td>23.8</td>\n",
       "      <td>30.2</td>\n",
       "      <td>17.9</td>\n",
       "      <td>1010.6</td>\n",
       "      <td>55.0</td>\n",
       "      <td>4.5</td>\n",
       "      <td>3.7</td>\n",
       "      <td>7.6</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>514 rows × 8 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "        T    TM    Tm     SLP     H   VV     V    VM\n",
       "261  27.7  33.1  23.0  1005.8  68.0  5.5   8.3  22.2\n",
       "470  25.8  32.5  20.0  1012.6  60.0  4.5   6.1  11.1\n",
       "612  27.7  32.8  21.0  1013.1  66.0  4.7   5.4  13.0\n",
       "552  23.2  31.0  12.9  1013.3  41.0  4.0   3.0   9.4\n",
       "12   25.2  31.0  19.0  1015.4  48.0  4.7  11.1  16.5\n",
       "..    ...   ...   ...     ...   ...  ...   ...   ...\n",
       "9    23.7  30.4  17.0  1015.8  46.0  5.1   5.2  14.8\n",
       "359  33.6  40.0  25.0  1006.9  36.0  5.8   6.1  11.1\n",
       "192  24.9  30.4  19.0  1008.9  57.0  4.8   4.6   9.4\n",
       "629  26.1  29.0  22.4  1001.2  87.0  5.0  14.1  22.2\n",
       "559  23.8  30.2  17.9  1010.6  55.0  4.5   3.7   7.6\n",
       "\n",
       "[514 rows x 8 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "FFoDNcQadLR8"
   },
   "outputs": [],
   "source": [
    "from hyperopt import hp,fmin,tpe,STATUS_OK,Trials\n",
    "from sklearn.ensemble import RandomForestRegressor\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UlIdZwbadT-P"
   },
   "outputs": [],
   "source": [
    "# Hyperparameters range ===> Domain space\n",
    "\n",
    "parameters = {'criterion': hp.choice('criterion', ['mse', 'mae']),\n",
    "        'max_depth': hp.quniform('max_depth', 10, 1200, 10),\n",
    "        'max_features': hp.choice('max_features', ['auto', 'sqrt','log2', None]),\n",
    "        'min_samples_leaf': hp.uniform('min_samples_leaf', 0, 0.5),\n",
    "        'min_samples_split' : hp.uniform ('min_samples_split', 0, 1),\n",
    "        'n_estimators' : hp.choice('n_estimators', [10, 50, 300, 750, 1200,1300,1500])\n",
    "    }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "8yde7-uKd35G"
   },
   "outputs": [],
   "source": [
    "# Objective function \n",
    "\n",
    "def objective(parameters):\n",
    "    model = RandomForestRegressor(criterion = parameters['criterion'], max_depth = parameters['max_depth'],\n",
    "                                 max_features = parameters['max_features'],\n",
    "                                 min_samples_leaf = parameters['min_samples_leaf'],\n",
    "                                 min_samples_split = parameters['min_samples_split'],\n",
    "                                 n_estimators = parameters['n_estimators'], \n",
    "                                 )\n",
    "    \n",
    "    performance= cross_val_score(model, df.iloc[:,:-1], df.iloc[:,-1], cv = 5).mean()\n",
    "\n",
    "    # We aim to maximize performance of model, therefore we return it as a negative value\n",
    "    return {'loss': -performance, 'status': STATUS_OK }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "3qBXZYLzfAzD",
    "outputId": "879c8705-3aba-4a07-ab45-ac31efe16d02"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "100%|██████████| 80/80 [12:05<00:00,  9.07s/it, best loss: -0.45233420634261867]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import cross_val_score\n",
    "trials = Trials()\n",
    "best = fmin(fn= objective,space=parameters,algo= tpe.suggest,max_evals = 80,trials= trials)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "qJCEpDqznZCP",
    "outputId": "5d7807fe-6fb5-4e02-bcc9-77e2b9eb2345"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 26,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "best['criterion']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "lou7z0Gqnhxr"
   },
   "source": [
    "Hyperopt Fmin returns index values which are given in Domain space\n",
    "\n",
    "To get actual parameters we create a dictonary to map index values to parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 69
    },
    "colab_type": "code",
    "id": "1TtWCHlpfWhc",
    "outputId": "089f301a-9055-404c-a6c5-b8b57c327065"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "mae\n",
      "log2\n",
      "750\n"
     ]
    }
   ],
   "source": [
    "\n",
    "crit = {0: 'mse', 1: 'mae'}\n",
    "feat = {0: 'auto', 1: 'sqrt', 2: 'log2', 3: None}\n",
    "est = {0: 10, 1: 50, 2: 300, 3: 750, 4: 1200,5:1300,6:1500}\n",
    "\n",
    "\n",
    "print(crit[best['criterion']])\n",
    "print(feat[best['max_features']])\n",
    "print(est[best['n_estimators']])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "aQfzrwoskpWv"
   },
   "outputs": [],
   "source": [
    "\n",
    "trainedforest = RandomForestRegressor(criterion = crit[best['criterion']], max_depth = best['max_depth'], \n",
    "                                       max_features = feat[best['max_features']], \n",
    "                                       min_samples_leaf = best['min_samples_leaf'], \n",
    "                                       min_samples_split = best['min_samples_split'], \n",
    "                                       n_estimators = est[best['n_estimators']]).fit(X_train,y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "_nO56sdDlIT6"
   },
   "outputs": [],
   "source": [
    "y_prediction=trainedforest.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "iCy_SWCnjyHd"
   },
   "outputs": [],
   "source": [
    "# performance of hyperparameter tuned model\n",
    "import sklearn\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "h2OCs6YSos_8"
   },
   "outputs": [],
   "source": [
    "from sklearn import metrics"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 35
    },
    "colab_type": "code",
    "id": "9MDzEWKepPvp",
    "outputId": "e7ec8304-2bf9-4f8a-921b-039b52d05c8f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3745.1700713812247"
      ]
     },
     "execution_count": 53,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sklearn.metrics.mean_squared_error(y_test,y_prediction)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "c6Pitd5dpRbV"
   },
   "outputs": [],
   "source": [
    "from hyperopt import plotting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 285
    },
    "colab_type": "code",
    "id": "O16rJ0R0pyCs",
    "outputId": "1f9c69f3-570b-49a3-e7d9-b1736677b7a8"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.collections.PathCollection at 0x7f2091091a90>"
      ]
     },
     "execution_count": 57,
     "metadata": {
      "tags": []
     },
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD7CAYAAACRxdTpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3df5BddZnn8feTpsEOsnSQLBUaMgkWxhKdSbBX2MKhENQo/iC67AzUjOJqbYYd3RLXzRBWawyzbpkxq9ZabmlhySori1HQCOIuMAbXkZrgdkwCRIgEEKWNEAkBNRE74dk/7rnJ7dvn3Hvuvefc8z3nfF5Vqdz+3u7bT9/b/dzveb7P+R5zd0REpFrmFR2AiIhkT8ldRKSClNxFRCpIyV1EpIKU3EVEKkjJXUSkgromdzM73czuNrOfmNlOM/tgNL7BzB4ys/vM7FtmNh6NLzGzg2a2Pfr3hbx/CBERmc269bmb2SJgkbv/2MxOALYCq4DTgM3ufsjM/h7A3a82syXAd9z9lblGLiIiiY7p9gnuvgfYE93+jZk9CEy4+50tn7YFuLTfIE4++WRfsmRJv18uIlJLW7du/bW7L4y7r2tybxXNylcA97bd9V5gY8vHS81sG/Ac8FF3/8dOj7tkyRKmpqZ6CUVEpPbM7PGk+1IndzN7MXALcJW7P9cy/hHgEHBjNLQHWOzuT5vZq4FNZnZW69dEX7caWA2wePHitGGIiEgKqbplzGyURmK/0d2/2TL+HuCtwF94VLx39+fd/eno9lbgEeBl7Y/p7te5+6S7Ty5cGHtUISIifUrTLWPAl4AH3f3TLeNvAv4GeLu7H2gZX2hmI9HtM4AzgUezDlxERJKlKcucB7wLuN/Mtkdj/wn4LHAccFcj/7PF3a8Ezgf+zsxmgBeAK919X+aRi4hIojTdMj8ELOau7yZ8/i00SjgiIlKQnrplRETysGnbNBvu2MUv9x/k1PEx1qxcxqoVE0WHVWpK7iJSqE3bprnmm/dzcOYwANP7D3LNN+8HUIIfgPaWEZFCbbhj15HE3nRw5jAb7thVUETVoOQuIoX65f6DPY1LOkruIlKoU8fHehqXdFRzFymRKi48rlm5bFbNHWBsdIQ1K5cVGFX+8n4tldxFSqKqC4/N2Dsluqq9qQ3jtey65e8wTE5OujYOE+nsvPWbmY6pQ0+Mj3HP2gsLiGg42hMhNGb2n3jnq0qb4LN6Lc1sq7tPxt2nmbsMVdVmYMNU14XHTt00Zf3dGcZrqQVVGZrmDGx6/0Gco4eim7ZNFx1aKdRx4XHTtunYGS6U+01tGK+lkrsMjfqZB7Nm5TLGRkdmjVV54bE5GUhS5je1YbyWKsvI0FSxrDDMMlPSwiM0arhVK3XFTQaayv6mlmYReVBK7jI0p46PxR5il3UGVkT3yqoVE3O6SKrYQQOd3/TLvJja1P5aZk1lGenbpm3TnLd+M0vX3s556zd3rZ1XrawQQpkphBjykvSmPzE+VvrEPgxK7tKXfhZHV62Y4BPvfBUT42MYjT/SMs/AQigzhRBDXnqdDPQ62ag6lWWkL/22p+V9KDpMIZSZQoghL73UpatcnuqXkrv0pcozxrRCOG0+hBjylHYyUMVe+EEpuUtfqjxjTGsYHQ9liCEEmmzMpeQufan6jDGtEMpMIcRQNE025uq6oGpmp5vZ3Wb2EzPbaWYfjMZPMrO7zOzh6P8F0biZ2WfNbLeZ3WdmZ+f9Q8jwVW1xVMqtap1YWUgzcz8EfNjdf2xmJwBbzewu4D3A99x9vZmtBdYCVwNvBs6M/p0DfD76XypmmDNG7Ukjnag8NVfX5O7ue4A90e3fmNmDwARwCXBB9GlfAb5PI7lfAtzgje0mt5jZuJktih5HpGfqhJA0VJ6arac+dzNbAqwA7gVOaUnYvwJOiW5PAL9o+bInorH2x1ptZlNmNrV3794ew5Y6qfKJOiJ5SZ3czezFwC3AVe7+XOt90Sy9p43h3f06d59098mFCxf28qVSM+qEEOldqm4ZMxulkdhvdPdvRsNPNsstZrYIeCoanwZOb/ny06Ixkb6oE0KyVJf1mzTdMgZ8CXjQ3T/dctetwBXR7SuAb7eMvzvqmjkXeFb1dhmEOiEkK3W6pkCassx5wLuAC81se/TvYmA98AYzexh4ffQxwHeBR4HdwBeBv84+bKkTtV1KVuq0fpOmW+aHgCXcfVHM5zvw/gHjEplFnRCShTqt3+gMVZEC1KXuG5o6rd8ouUtPypSUQo1VffvFqdO2GdrPXVIr02JUyLHWqe4bmjqt32jmLqmVaVvVkGOtQt031KOiNOqyfqOZu6RWpqQUcqxJ9d2y1H1DPiqSo5TcJbUyJaWQYy17377KSuWg5C6plSkphRxr2eu+IR8VyVGquUtqZdpWNfRYy1z3rVM7YZlZ45yjYk1OTvrU1FTRYYhICu2tnNA4KirT0UdVmNlWd5+Mu08zdxHpSehHRdKg5C4iPStzWakutKAqIlJBmrlLLZT5pBuRfii5S+VpLxepI5VlpPJ00o3UkZK7VJ5OupE6UnKXygt5KwKRvCi5S6E2bZvmvPWbWbr2ds5bvzmXzadC3opAJC9dF1TN7HrgrcBT7v7KaGwj0PzLGAf2u/tyM1sCPAg0i5lb3P3KrIOWahjWQqdOupE6StMt82Xgc8ANzQF3//PmbTP7FPBsy+c/4u7LswpQqmuYe67rpBupmzQXyP5BNCOfw8wM+DPgwmzDkjrQQqcUqernPgxac/9T4El3f7hlbKmZbTOz/2tmfzrg40uFaaFTilKHC44MmtwvB25q+XgPsNjdVwD/AfhfZvbP4r7QzFab2ZSZTe3du3fAMKSMtNApRanDuQ99J3czOwZ4J7CxOebuz7v709HtrcAjwMvivt7dr3P3SXefXLhwYb9hSImV/aIVUl51KAkOsv3A64GH3P2J5oCZLQT2ufthMzsDOBN4dMAYpcKyXuiseh1VslGHC450nbmb2U3APwHLzOwJM3tfdNdlzC7JAJwP3Gdm24GbgSvdfV+WAYskqUMdVbJRh5Jgmm6ZyxPG3xMzdgtwy+BhSRmENkseZmtlmYT2OoWgDuc+aFdI6UuIOy0Ou45ahqTZ6XWCaie3bqp+7oOSu/QlxFnyMOuoIb65xUl6ndbdupPnD71QePxleIMsK+0tI30JsdtgmHXUsrTSJb0e+w/OFB6/1kjypeQufQnxBKRhtlaG+OYWp9fXY5jxl+UNsqxUlpG+rFm5bFZZAsLoNsiqjtqtXFCWVrqk1+lFo/N45sDMnM8fZvxleYMsKyX3lFQbnC3EboOsXqM09fRQ39zaJb1OQOHxl+UNsqzM3YuOgcnJSZ+amio6jETtf+zQ+EPQ2ZThyPI1Om/95tikMzE+xj1rj+6RV/Y3/KLj19/V4Mxsq7tPxt2nmXsKIXaGyGxZvkZpywVlb6UrOv4Qj/6qRMk9BdUGw5fla6RywfAU/QZTZeqWSSHEzpBhGcZl8LKQ5WtUh1PTpfqU3FOo6x97mfqQs3yNtFulVIHKMinUtTZYprWGrF8jlQuk7JTcU6rjH3vZ1hrq+BqJJFFZRhLVea1BpOyU3CVRXdcaRKpAZRlJVNe1BpEqUHKXjlTHFiknJXepjaJPtxcZJiV3qYWyXFxDJCtpLpB9vZk9ZWYPtIytM7NpM9se/bu45b5rzGy3me0ys5V5BS7SC+0dLnWTZub+ZeBzwA1t459x9//aOmBmrwAuA84CTgX+wcxe5u6HESlQ2Xr260Tlsnx0nbm7+w+AfSkf7xLga+7+vLs/BuwGXjNAfCKZUM9+mMq0xUXZDNLn/gEzuy8q2yyIxiaAX7R8zhPR2BxmttrMpsxsau/evQOEIdKdevbDpHJZfvpN7p8HXgosB/YAn+r1Adz9OnefdPfJhQsX9hmGSDraDCxMKpflp69uGXd/snnbzL4IfCf6cBo4veVTT4vGRAqnnv3waO/8/PQ1czezRS0fvgNodtLcClxmZseZ2VLgTOBHg4UoEray7HkfIpXL8tN15m5mNwEXACeb2RPAx4ALzGw54MDPgL8CcPedZvZ14CfAIeD96pSRkA3aqaH++cFoi4v86ALZUltZXKA57cW0i6RWw+rqdIFs7QoptZVFp0boC4JqNawvJXeprSwSc+j982o1rC8ld6mtLBJz6AuCoR9ZSH6U3KW2skjMoffPj88fjR0P5chC8qNdIaW2surUCLV/ftO2aX77+0NzxkdHLJgjC8mPkrvUUnsHyWf+fHmQCXoQG+7YxcwLc7vhjj/2mMr9rDKXkrvUTl1605Pq6s8enBlyJFIE1dyldurSQRJ6J4/kS8ldaqcuHSShd/JIvpTcpXbqMqMNvZNH8lX7mrtOza6fNSuXxW47UMUZbaidPJK/Wif3uiysyWx5bFalSYKEptbJvdPCmv4wqy3LGa0mCRKiWtfc67KwJvmqS/eNlEutk3tdFtYkX5okSIhqndzVKlZeIV39SJMECVGtk7taxcoptD3KNUmQENV6QRXUKlZGoS2E61JxEqLaJ3cpnxBr3JokSGi6lmXM7Hoze8rMHmgZ22BmD5nZfWb2LTMbj8aXmNlBM9se/ftCnsFLPanGLdJdmpr7l4E3tY3dBbzS3f8Y+ClwTct9j7j78ujfldmEma+QFuekO9W4RbrrWpZx9x+Y2ZK2sTtbPtwCXJptWMOjE1DKRzVuke6yqLm/F9jY8vFSM9sGPAd81N3/Me6LzGw1sBpg8eLFGYTRn9AW5yQd1bhFOhsouZvZR4BDwI3R0B5gsbs/bWavBjaZ2Vnu/lz717r7dcB1AJOTk3MvFzMkIS7OZUX7nYjUV9/J3czeA7wVuMjdHcDdnweej25vNbNHgJcBU4OHmo9Tx8eYjknkZV+cU7mp/EJ+cw45Nmno6yQmM3sT8DfA2939QMv4QjMbiW6fAZwJPJpFoHmp6uKc9jspt9BO1CpLbHJUmlbIm4B/ApaZ2RNm9j7gc8AJwF1tLY/nA/eZ2XbgZuBKd9+XU+yZqOpZqlUuN9VByG/OIccmR6Xplrk8ZvhLCZ97C3DLoEENWxUX55LKTSeOjRYQjfQq5DfnkGOTo2q9t0yVrVm5jNF5Nmf8d384pMPnEgj5RK2QY5OjlNwratWKCV78orkHZjOHXYfPJRDyWlDIsclR2lumwvYfmIkd1+Fz+EI+USvk2OQoJfcKq2qbZ12EvBYUcmzSoLJMhenwWaS+NHOvMB0+i9SXknvF6fBZpJ6U3CUoOq1dJBtK7jJLkclV++GIZEfJXY4oOrmWcfvlMhxplCFGyZ66ZeSIovcMKdtp7WXYQKsMMUo+lNzliKKTa9lOay/6zTCNMsQo+VBZpmAhHTLnedJTmp9zzcpls8pCEHZfftFvhmmUIUbJh2buBcrjkHmQi33nddJT2p+zbNsvl+FIowwxSj5qP3PftG2adbfuZP/Bxj4sC+aP8rG3nVXKBcRN26ZZc/MOZg43rlo4vf8ga27eAaRbEM3rpKdefs6s+/LzPDIqw5FGGWKUfNQ6uW/aNs2ab+xg5oWjl3B95sBMTwlxEFkfMl97284jib1p5rBz7W07U/8seZz0VFRpIO/unzKcAVyGGCUftU7uG+7YNSuxNzW3xc37DyDrGvczCbtAJo0PS1EbmA2jtbIMZwCXIUbJXulr7oPUmDvNHIex4FSXjb2K+jm1mCh1liq5m9n1ZvaUmT3QMnaSmd1lZg9H/y+Ixs3MPmtmu83sPjM7O6/gB12Q7DRzHMaCU9YLiOMJl9BLGh+WohZKtZgodWbuc8sScz7J7Hzgt8AN7v7KaOyTwD53X29ma4EF7n61mV0M/HvgYuAc4L+5+zmdHn9yctKnpqZ6Dv689ZtjD/cnxse4Z+2FXb8+ruYOMDpibLj0T0p3KBv384zOMzb86/L9LFlor7lD44gh5A6ctEJqoZXimNlWd5+Muy9Vzd3df2BmS9qGLwEuiG5/Bfg+cHU0foM33jW2mNm4mS1y9z29h97ZoIfdzT+GorplsqbFs9mq+nwUvU2ElMMgC6qntCTsXwGnRLcngF+0fN4T0VjmyT2LhbpeF5tCnzFp8Wy2Kj4fZdyDR4YvkwXVaJbevb7TwsxWm9mUmU3t3bu3r++7ZuUyRkds1tjoiOW2UKd9OrIzyEJ43WmhWNIYJLk/aWaLAKL/n4rGp4HTWz7vtGhsFne/zt0n3X1y4cKF/UfR/pbS01tMb+q+T0dWCVlvkoPRQrGkMUhyvxW4Irp9BfDtlvF3R10z5wLP5lFvh/g+9ZkXPLdkW+cZU5YJue5vkoOqSwutDCZVzd3MbqKxeHqymT0BfAxYD3zdzN4HPA78WfTp36XRKbMbOAD8m4xjPmLYybaok3HSyHstIMs6b53fJLNQ1YViyVbabpnLE+66KOZzHXj/IEGlNexkG+o+HXHdE1dt3M61t+3MrPMny4Sc9LrNM2Pp2tuVrFKo4kKxZKvUZ6gO+/A01F0L42bV0Nh2IKtadpZ13riFcIDD7qrBi2Sk1HvLFHF4GuKMqdPsOasWucyPWrosfMfFHXobqkhISp3cIcxk20keCSqpzNGURS07yzfSpA3b2rXGrRN3RHpT+uReJnklqLhZdaus1iCyeiNN+2bTGrdO3BHpjZJ7TuJm6HklqLhtFJpCWPBt1+1IA+bGrQ4bkd6k2jgsb/1uHJaVrEslSRtWJc2sDXhs/Vv6/n7t3zurnyWvGnfc8zM6Yhx/7DE8e3Am9nslbRI3YsYL7qrBSy0NvHFYleVRKkmaoY+YcTjmzTTL1s2sSid51rj7qd8nlZ6az6dq8CKzlboVMgt5nC2ZVCo47D6nBTDEsgmEdxZpexvqiM1tpdRZriJH1X7mnkctt2NN2RvbCu8/EF9+SDLsNsA8a9zt+85P7z/Imm90v25t61HJ0rW35xafSBXUfuaexyZMcSdXNc284Mw/9hgeW/8W7ll7YerEPuyNtvLcnGrdrTtj9wRad+vO1I+hzbNEOqt9cs/jLNdmCSFJ0uwyadfFfkokg+7gmOfZv+0dPa3jaWPW5lkindW+LJPXWa6rVkyw4Y5dqfe+iVvAXPONHVx7206eORCfDDu9SQy6GJr2ecm6XNR8vrrFnOfZyToTVqpArZA56uUankmtfp0kXSt20GvLprFp23RiX323/XZW/N2diW9Y7bKMOY0qX3dVqqdTK2TtyzJ56mWjsV4XAjuVIPI+4eejm+7nQxu3x5ZX0nSsfOxtZ8VuHBZn2AukoXUJifSr9mWZvKXtO09z1mbTRJdSQZ5bIW/aNs2NW37ecd+vbgk5rqRy4A+HYmfzw14g1ZmwUhVK7h0Ms/babX+YpmZi33DHLj60cXtsXEk7OC55yRgvvea7HHZnxIzLzzmdj69KXviNs+GOXV2vZJgmIbe/6SWVQ4a9QBryBVlEeqHknmDYuxC2z2ZPHBvlN88f4nBLy+DIPOOZ3z3PVRu3HxmLiytuZrzkJWPc88i+I1932J2vbvk5QGKCj3tz6zaD7Tchh3J1oVAvyCLSKy2oJshrUTLt0UD7iT7ddIurOWNvN2LGI5+4OPb7xyW5446Zl9jKuGD+aGZXfiqSumWkLLS3TB/yqL32cjSQds/ztHHFJfZO40kLiy8anTdnEzQD/uLcxT2XeEJVtmsEiMTpu1vGzJaZ2faWf8+Z2VVmts7MplvG504LSyDtGZC9nCzUSydGr28i3WrCcXuxdBpP+v77D8wc6QBqfr0Ddz+0t68zZgc92UpE4vWd3N19l7svd/flwKuBA8C3ors/07zP3b+bRaDDluYMyLhtAdbcvIPl194Zm6x6ORroZQEvTU348nNO72m805vbqhUTR56f9l0Ze0nORWyrIFIXWfW5XwQ84u6PZ/R4hUvTox43E5857Ow/OBObrE4cG439XnGJtNP+NK3Gx0ZTnWDz8VWv4i/PXXxkpj5ixl92KKXEfX+j8TOdt34z1962M/EoJO1sXD3lIvnJquZ+GXBTy8cfMLN3A1PAh939mfYvMLPVwGqAxYsXZxRGtrrVXtOUTlqT1e/+cGjO/aPzLHbWHdc98rqXL+Tuh/b2vdD38VWvSl0Xb/3+0/sPYhy9pnWnfvzmG1qadQX1lIvkZ+BuGTM7FvglcJa7P2lmpwC/ppEL/jOwyN3f2+kxQuyWSSPtlgFGcv/0gvmjbPvbN+YQXXZ62Roh6YIkcd08w9gmQaTK8t5+4M3Aj939SQB3f9LdD7v7C8AXgddk8D2ClLZ0cur4WOJs9JkD6XdCLEramXRrDT7NY2hnR5H8ZJHcL6elJGNmi1ruewfwQAbfI0jtdfkF80cZnRd/paWkBcpmHTvkBcWk2MfHRuesSUyk7DKC3vbeEZHeDFRzN7PjgTcAf9Uy/EkzW06jLPOztvsqJ+40+qQTYNpPCmqtYzc1a/QhJbikszbXvT3+hKVezvBUT7lIPgZK7u7+O+AlbWPvGiiikktKVnELpEl17NAWFHvZGiCUbQRE6k7bDxRIC4oiMgjt5x4oLSiKSF60t0yBVMIoN20wJiFTci+YFhTLadhbQov0SmUZkT5o6wQJnWbuAYk7zAeVbUKkrRMkdErugYg7zF9z8w5wjuzrrkP/cOhyfBI6lWUCkbTDZPsFO3ToHwZ1OknoNHMPRC+H8zr0L546nSR0Su4Fa9bZezmVTIf+YVCnk4RMyb1AcRehbjUyzzjcVpbRob+IpKGae4Hi6uxNC+aPznlxDPhXr9ZsUUS608w9J2nOXkyqnRsw/9hjeObAzKzx5oWoRUS6qV1yH8Yp42nPXkxqpxufP5q4Y2TaKyKJSL3Vqiyzads0a27eMeviGGtu3tHzxTG6XQA67dmLce10oyPGb38/91qrTRZ9fxGRTmqV3K+9bSczh2cvUM4cdq69bWfqx2jOyjtdPSnt2YtxVyI6/thj5vS2t3JQn7sEqdukR4arUmWZbiWX9hp2t/G4x//w13fMuU5o+9WTejl7sb2dbuna27vGoT73/mgXx/xoI7XwVGbmnmZGncXjJ10Aenr/wSPfa5CzF9P0sKvPvXd5/37UnTZSC8/Ayd3MfmZm95vZdjObisZOMrO7zOzh6P8Fg4faWZpfrvGx0divTRrv9vjtmslikAs/x70xtFKfe3+UfPKljdTCk1VZ5nXu/uuWj9cC33P39Wa2Nvr46oy+V6w0v1zr3n4Wa76xY1ZNe3Sese7tZ/X9+K1ayzP9nr3Yflr7iWOjmMH+AzMqJQxAySdf2kgtPHnV3C8BLohufwX4Pjkn9zS/XIPsB9LpgtatskgWOq09e0o++Vqzctmcs611lFmsLGruDtxpZlvNbHU0doq774lu/wo4JYPv01HaOveqFRPcs/ZCHlv/Fu5Ze2HqJNqtXNKkZBEm7eKYr0FKkZKPLGbur3X3aTP758BdZvZQ653u7mY2ZxUyeiNYDbB48eKBg8h7l77Wx5/efxCDOZt9KVmES7s45k9HnGExT+j+6OvBzNYBvwX+LXCBu+8xs0XA9909MetNTk761NRUZnHkKWmzrwXzR/nY284ayi+3WvpEBMDMtrr7ZNx9A5VlzOx4MzuheRt4I/AAcCtwRfRpVwDfHuT7hCSpa2b+sccMLbF3a+nTySQiMmjN/RTgh2a2A/gRcLu7/x9gPfAGM3sYeH30cSUkLZi29rnnqVtLn/q5RQQGrLm7+6PAn8SMPw1cNMhjh6pT18wwzsjr1tLXKfmrdCNSH5U5Q3VYOnXNDOOkmKRunOa4+rlFBJTce9Zs+UqSdxLt1tLXLfmLSD2UOrkXtXC4asUEEwUl0W79xOrnjqdFZqmb0u4KOaxd6JLaDos8I69TP7H6uefSjoVSR6VN7v0uHCYl6/bx1718Id/ZsYf9B49uBxyXFK69beeRLYOPOyaMA6FhnkxShp57LTJLHZU2ufdzGbqkGdzU4/u4Zev0rPGvbvl57GO0J4Xfz7xw5L79B2f40MbtTD2+j4+vSq7LV0VZZsRaZJY6CmOq2YcRs57GIXkGd9O9v+i6nW+rTm2HDty45ee1qOmWZRtdLTJLHZU2uSddNCNpHJJnap2+Jk63tsO6XAqvLDNiLTJLHZU2uSd1qySNQ/JMrdNsv12atkMIL8HloSwzYu1YKHVU2uTez2ws6WsuP+f0VNv5Lpg/OqftMOltIbQEl4cyzYj73epZpKxKu6DaT8tfp6+Z/KOT5nTL3P3Q3o6PvWrFBFOP7+PGLT+ftf1vqAkua2q7FAlXplv+9qtMW/7GKUM7oIhUT6ctf0s7cw+JLlIgIqEpbc1dRESSKbmLiFSQkruISAUpuYuIVJCSu4hIBQXRCmlme4HHB3iIk4FfZxRO3soUK5Qr3jLFCuWKt0yxQn3i/SN3Xxh3RxDJfVBmNpXU6xmaMsUK5Yq3TLFCueItU6ygeEFlGRGRSlJyFxGpoKok9+uKDqAHZYoVyhVvmWKFcsVbplhB8Vaj5i4iIrNVZeYuIiItSp3czexNZrbLzHab2dqi44ljZj8zs/vNbLuZTUVjJ5nZXWb2cPT/goJiu97MnjKzB1rGYmOzhs9Gz/V9ZnZ2IPGuM7Pp6PndbmYXt9x3TRTvLjNbOeRYTzezu83sJ2a208w+GI0H+fx2iDe459fMXmRmPzKzHVGs10bjS83s3iimjWZ2bDR+XPTx7uj+JcOKtUu8Xzazx1qe2+XReDa/C+5eyn/ACPAIcAZwLLADeEXRccXE+TPg5LaxTwJro9trgb8vKLbzgbOBB7rFBlwM/G/AgHOBewOJdx3wH2M+9xXR78RxwNLod2VkiLEuAs6Obp8A/DSKKcjnt0O8wT2/0XP04uj2KHBv9Jx9HbgsGv8C8O+i238NfCG6fRmwccjPbVK8XwYujfn8TH4Xyjxzfw2w290fdfc/AF8DLik4prQuAb4S3f4KsKqIINz9B8C+tuGk2C4BbvCGLcC4mS0aTqQNCfEmuQT4mrs/7+6PAbtp/M4MhbvvcfcfR7d/AzwITBDo89sh3iSFPb/Rc/Tb6MPR6J8DFwI3R+Ptz23zOb8ZuMish2trDqhDvEky+V0oc3KfAH7R8vETdP5lLIoDd5rZVjNbHeLugA0AAAKaSURBVI2d4u57otu/Ak4pJrRYSbGF/Hx/IDp8vb6lxBVMvFEZYAWNGVvwz29bvBDg82tmI2a2HXgKuIvGkcN+dz8UE8+RWKP7nwVeMqxY4+J19+Zz+1+i5/YzZnZce7yRvp7bMif3snitu58NvBl4v5md33qnN47DgmxZCjm2Fp8HXgosB/YAnyo2nNnM7MXALcBV7v5c630hPr8x8Qb5/Lr7YXdfDpxG44jh5QWH1FF7vGb2SuAaGnH/C+Ak4Oosv2eZk/s0cHrLx6dFY0Fx9+no/6eAb9H4RXyyeZgV/f9UcRHOkRRbkM+3uz8Z/eG8AHyRo6WBwuM1s1EaifJGd/9mNBzs8xsXb8jPbxTffuBu4F/SKF80ry7XGs+RWKP7TwSeHnKowKx43xSVwtzdnwf+Bxk/t2VO7v8PODNaIT+WxkLJrQXHNIuZHW9mJzRvA28EHqAR5xXRp10BfLuYCGMlxXYr8O5oJf9c4NmW8kJh2mqR76Dx/EIj3suiTomlwJnAj4YYlwFfAh5090+33BXk85sUb4jPr5ktNLPx6PYY8AYaawR3A5dGn9b+3Daf80uBzdFR01AkxPtQy5u80VgfaH1uB/9dGOaqcdb/aKwq/5RGve0jRccTE98ZNDoKdgA7mzHSqPd9D3gY+AfgpILiu4nGofYMjbre+5Jio7Fy/9+j5/p+YDKQeP9nFM990R/FopbP/0gU7y7gzUOO9bU0Si73AdujfxeH+vx2iDe45xf4Y2BbFNMDwN9G42fQeIPZDXwDOC4af1H08e7o/jOG/Nwmxbs5em4fAL7K0Y6aTH4XdIaqiEgFlbksIyIiCZTcRUQqSMldRKSClNxFRCpIyV1EpIKU3EVEKkjJXUSkgpTcRUQq6P8DyfF7okJ7f2EAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light",
      "tags": []
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "plt.scatter(y_test,y_prediction)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "g-5bl6Mwpzey"
   },
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "QTYagPzsp7sU"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "ncL-uCQgqASL"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "LCR0WgX1qb0m"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "NWuvvb4pkgvm"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "KL9NvUJM4qqF"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "AeFQuxKTrRrA"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "xFOtrKbcxsHd"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "GS21el5H2ff0"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "t-QrunLTyljH"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "YfCtxV9Dy7Zb"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "zzj9WqIb2Jg9"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "BTDhy9pT8ypH"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "FCILBgWi8zCu"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gM-AeSBu8z5W"
   },
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "meZDf3V891Ay"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "TvB2RbDNBNR3"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "UTYbJyQ183-H"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "cLpIQdld-mlV"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "c0TNQzON-rPF"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "mAEK9bSgA1ZO"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "-0Q3pm0XFjl_"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "X2Qa42IjFra-"
   },
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "tTqTGbkEGAEp"
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "name": "Bayesian Optimization with Hyperopt",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}