Feature_Engineering_1.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Feature_Engineering_1.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyMzlZYIglWugXYAlEHtNNrL",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/Chandradithya8/Feature_Engineering/blob/main/Feature_Engineering_1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pBCv4SyYkteY"
      },
      "source": [
        "Handling missing values\n",
        "\n",
        "1.mean/median/mode replacement\n",
        "\n",
        "2.Random sample replacement\n",
        "\n",
        "3.Capturing Nan values with a new feature\n",
        "\n",
        "4.End of distribution inputation\n",
        "\n",
        "5.Arbitrary imputation\n",
        "\n",
        "6.Frequent categories imputation\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6Vf2rN3SPNy6"
      },
      "source": [
        "# **HANDLE NUMERICAL MISSING VALUES**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "u_9YNDavlm2i"
      },
      "source": [
        "**1. mean/ median /mode replacement** - more frequently used"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "TzaWvOPEkz3X",
        "outputId": "e91819ff-bc9b-4a94-8dd3-7a4ebd1012e3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        }
      },
      "source": [
        "import pandas as pd\n",
        "df=pd.read_csv('/content/titanic.csv.txt',usecols=['Age','Fare','Survived'])\n",
        "df.head(6)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Survived</th>\n",
              "      <th>Age</th>\n",
              "      <th>Fare</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>22.0</td>\n",
              "      <td>7.2500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>38.0</td>\n",
              "      <td>71.2833</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>26.0</td>\n",
              "      <td>7.9250</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>35.0</td>\n",
              "      <td>53.1000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>8.0500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>8.4583</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Survived   Age     Fare\n",
              "0         0  22.0   7.2500\n",
              "1         1  38.0  71.2833\n",
              "2         1  26.0   7.9250\n",
              "3         1  35.0  53.1000\n",
              "4         0  35.0   8.0500\n",
              "5         0   NaN   8.4583"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 1
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Ssobx8wZn_xc",
        "outputId": "f35c257b-cdb1-4453-aa52-d7f26067bb3c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "df.shape"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(891, 3)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iJ-EqGsFn4bb",
        "outputId": "4581b4a9-1164-4dfb-c876-6833b0e1f0a2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "df.isnull().sum()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Survived      0\n",
              "Age         177\n",
              "Fare          0\n",
              "dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 3
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "XEY3UjUpl4a2",
        "outputId": "a164cda7-e89d-4256-d282-d107975868f7",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        }
      },
      "source": [
        "df['Age']=df.Age.fillna(df['Age'].median)\n",
        "df.head(6)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Survived</th>\n",
              "      <th>Age</th>\n",
              "      <th>Fare</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>22</td>\n",
              "      <td>7.2500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>38</td>\n",
              "      <td>71.2833</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>26</td>\n",
              "      <td>7.9250</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>35</td>\n",
              "      <td>53.1000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>35</td>\n",
              "      <td>8.0500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>0</td>\n",
              "      <td>&lt;bound method Series.median of 0      22.0\\n1 ...</td>\n",
              "      <td>8.4583</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Survived                                                Age     Fare\n",
              "0         0                                                 22   7.2500\n",
              "1         1                                                 38  71.2833\n",
              "2         1                                                 26   7.9250\n",
              "3         1                                                 35  53.1000\n",
              "4         0                                                 35   8.0500\n",
              "5         0  <bound method Series.median of 0      22.0\\n1 ...   8.4583"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "T4l6X1gExHys"
      },
      "source": [
        "Disadvantages- Distortion in variance"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Bh2GITopm8KU"
      },
      "source": [
        "2. Random sample inputation(**Less** distortion in variance) - more frequently used"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "ivMaKk1LnbWb",
        "outputId": "8c5ab5c0-30fa-4ebe-e34c-848f134f1b8a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        }
      },
      "source": [
        "import pandas as pd\n",
        "df=pd.read_csv('/content/titanic.csv.txt',usecols=['Age','Fare','Survived'])\n",
        "df.head(6)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Survived</th>\n",
              "      <th>Age</th>\n",
              "      <th>Fare</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>22.0</td>\n",
              "      <td>7.2500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>38.0</td>\n",
              "      <td>71.2833</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>26.0</td>\n",
              "      <td>7.9250</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>35.0</td>\n",
              "      <td>53.1000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>8.0500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>8.4583</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Survived   Age     Fare\n",
              "0         0  22.0   7.2500\n",
              "1         1  38.0  71.2833\n",
              "2         1  26.0   7.9250\n",
              "3         1  35.0  53.1000\n",
              "4         0  35.0   8.0500\n",
              "5         0   NaN   8.4583"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YxjUjgThm5Gn",
        "outputId": "5cdef22e-1603-4715-ffb9-f91bda3250c3",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        }
      },
      "source": [
        "df['Age'].dropna()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "0      22.0\n",
              "1      38.0\n",
              "2      26.0\n",
              "3      35.0\n",
              "4      35.0\n",
              "       ... \n",
              "885    39.0\n",
              "886    27.0\n",
              "887    19.0\n",
              "889    26.0\n",
              "890    32.0\n",
              "Name: Age, Length: 714, dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KZLQZI3QoNXU",
        "outputId": "fca98e6f-6cfd-483a-cd00-e8eb234c9d6f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 221
        }
      },
      "source": [
        "random_samples=df['Age'].dropna().sample(df['Age'].isnull().sum()) # Takes 177 random samples\n",
        "random_samples"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "842    30.0\n",
              "668    43.0\n",
              "434    50.0\n",
              "528    39.0\n",
              "806    39.0\n",
              "       ... \n",
              "67     19.0\n",
              "687    19.0\n",
              "494    21.0\n",
              "69     26.0\n",
              "92     46.0\n",
              "Name: Age, Length: 177, dtype: float64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "SUbMCz3YNZiO",
        "outputId": "2828b977-4aa5-41e9-d3c0-5f64a266584e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "random_samples.index"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Int64Index([842, 668, 434, 528, 806, 423,  99, 881, 623, 137,\n",
              "            ...\n",
              "            356,  15, 189, 700, 781,  67, 687, 494,  69,  92],\n",
              "           dtype='int64', length=177)"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "xAINCt3SOAZM",
        "outputId": "5056fd35-42a6-4a9a-b5e8-3aa87cbe89ad",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "import numpy as np\n",
        "for i in random_samples:\n",
        "  df['Age']=np.where(df['Age'].isnull(),i,df['Age'])\n",
        "print(df.head(6))\n",
        "print(df.isnull().sum())"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "   Survived   Age     Fare\n",
            "0         0  22.0   7.2500\n",
            "1         1  38.0  71.2833\n",
            "2         1  26.0   7.9250\n",
            "3         1  35.0  53.1000\n",
            "4         0  35.0   8.0500\n",
            "5         0  30.0   8.4583\n",
            "Survived    0\n",
            "Age         0\n",
            "Fare        0\n",
            "dtype: int64\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "eCOdRRs7u8p1"
      },
      "source": [
        "Disadvantage- Randomness wont work in every situation"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TaCbQ1L0x8rZ"
      },
      "source": [
        "**3.**  **Capturing NAN VALUES WITH A NEW FEATURE**\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eY-CEWPRx4xf",
        "outputId": "d13c3ff6-2080-4def-ac54-228d1fa0770e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        }
      },
      "source": [
        "import pandas as pd\n",
        "df=pd.read_csv('/content/titanic.csv.txt',usecols=['Age','Fare','Survived'])\n",
        "df.head(6)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Survived</th>\n",
              "      <th>Age</th>\n",
              "      <th>Fare</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>22.0</td>\n",
              "      <td>7.2500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>38.0</td>\n",
              "      <td>71.2833</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>26.0</td>\n",
              "      <td>7.9250</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>35.0</td>\n",
              "      <td>53.1000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>8.0500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>8.4583</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Survived   Age     Fare\n",
              "0         0  22.0   7.2500\n",
              "1         1  38.0  71.2833\n",
              "2         1  26.0   7.9250\n",
              "3         1  35.0  53.1000\n",
              "4         0  35.0   8.0500\n",
              "5         0   NaN   8.4583"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "jmaMoZnRy8B5",
        "outputId": "2f0a44a9-f0dd-4f4b-983b-b39359739f63",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        }
      },
      "source": [
        "import numpy as np\n",
        "df['Age_na']=np.where(df['Age'].isnull(),1,0)\n",
        "df.head(6)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Survived</th>\n",
              "      <th>Age</th>\n",
              "      <th>Fare</th>\n",
              "      <th>Age_na</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>22.0</td>\n",
              "      <td>7.2500</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>38.0</td>\n",
              "      <td>71.2833</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>26.0</td>\n",
              "      <td>7.9250</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>35.0</td>\n",
              "      <td>53.1000</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>8.0500</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>8.4583</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Survived   Age     Fare  Age_na\n",
              "0         0  22.0   7.2500       0\n",
              "1         1  38.0  71.2833       0\n",
              "2         1  26.0   7.9250       0\n",
              "3         1  35.0  53.1000       0\n",
              "4         0  35.0   8.0500       0\n",
              "5         0   NaN   8.4583       1"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WhCgFivq610x"
      },
      "source": [
        "Disadvantages- Creating additonal features(curse of dimentionality)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "LtdEUL9i8pJM"
      },
      "source": [
        "**4. End of distribution imputation**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "uYwHOR_-8PKN",
        "outputId": "ee6f5466-c79c-4009-cc28-067bcee97a55",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        }
      },
      "source": [
        "import pandas as pd\n",
        "%matplotlib inline\n",
        "df=pd.read_csv('/content/titanic.csv.txt',usecols=['Age','Fare','Survived'])\n",
        "df.head(6)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Survived</th>\n",
              "      <th>Age</th>\n",
              "      <th>Fare</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>22.0</td>\n",
              "      <td>7.2500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>38.0</td>\n",
              "      <td>71.2833</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>26.0</td>\n",
              "      <td>7.9250</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>35.0</td>\n",
              "      <td>53.1000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>8.0500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>8.4583</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Survived   Age     Fare\n",
              "0         0  22.0   7.2500\n",
              "1         1  38.0  71.2833\n",
              "2         1  26.0   7.9250\n",
              "3         1  35.0  53.1000\n",
              "4         0  35.0   8.0500\n",
              "5         0   NaN   8.4583"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wSz8TEVd9ZRu",
        "outputId": "d9498de3-abe6-4a58-ba3b-ad1c9b62e93d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        }
      },
      "source": [
        "df.Age.hist(bins=50) # taking the end of the distribution and replacing it with the nan values(after the third standard distribution)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fc1894e2860>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 14
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAD4CAYAAAD1jb0+AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAARZklEQVR4nO3db4wcd33H8feXhIDxUTsm4eTaUZ0qUVCUaxx8CkGg6i4p1PwRzoMoCrKQI7nyE6ChddU6rdQKqVWD2kB5gKpahMaq2lxCSprICGhqfK1aiYBNAk5i0qTBQKxgA3UMl0aUo98+2Dl8Ou/d7tztn/nZ75d0up3Z2bnP7Yw/nvvtzG5kJpKk8rxq2AEkSctjgUtSoSxwSSqUBS5JhbLAJalQFw7yh11yySW5adOmWo95+eWXWb16dX8CrVBTs5mrnqbmguZmM1c9K811+PDhH2bmpWfdkZkD+9qyZUvWdfDgwdqPGZSmZjNXPU3NldncbOaqZ6W5gEPZplMdQpGkQlngklQoC1ySCmWBS1KhLHBJKpQFLkmFssAlqVAWuCQVygKXpEIN9FJ69camPZ9vO//YXe8ZcBJJw9RVgUfEMeAnwM+B2cwcj4h1wP3AJuAYcGtmnupPTEnSQnWGUCYzc3NmjlfTe4ADmXklcKCaliQNyErGwLcB+6rb+4CbVx5HktStyC4+1Dgivg2cAhL4m8zcGxEvZeba6v4ATs1NL3jsLmAXwOjo6JapqalaAWdmZhgZGan1mEEZVrYjx0+3nT+2YQ3Q3OfMXPU1NZu56llprsnJycPzRj9+odsXMd+emccj4o3AoxHxrfl3ZmZGRNv/CTJzL7AXYHx8PCcmJmoFn56epu5jBmVY2W5f7EXM7RNAc58zc9XX1GzmqqdfuboaQsnM49X3k8BDwPXAiYhYD1B9P9nzdJKkRXUs8IhYHRGvn7sNvBN4EngE2FEttgN4uF8hJUln62YIZRR4qDXMzYXAP2TmFyPia8ADEbET+A5wa/9iSpIW6ljgmfk8cG2b+T8CbupHKElSZ15KL0mFssAlqVAWuCQVygKXpEJZ4JJUKAtckgrl+4GrJ3yPcmnwPAKXpEJZ4JJUKAtckgrlGPh5zHFrqWwegUtSoSxwSSqUBS5JhbLAJalQFrgkFcoCl6RCWeCSVCgLXJIKZYFLUqEscEkqlAUuSYWywCWpUBa4JBXKApekQlngklQoC1ySCmWBS1KhLHBJKpQFLkmFssAlqVBdF3hEXBARj0fE/mr68oh4LCKei4j7I+Ki/sWUJC1U5wj8DuDovOmPAZ/IzCuAU8DOXgaTJC2tqwKPiI3Ae4BPV9MB3Ag8WC2yD7i5HwElSe1FZnZeKOJB4M+B1wO/B9wOfKU6+iYiLgO+kJnXtHnsLmAXwOjo6JapqalaAWdmZhgZGan1mEEZVrYjx08vef/oKjjxypnpsQ1raq1nseWXk2n+upq6LZuaC5qbzVz1rDTX5OTk4cwcXzj/wk4PjIj3Aicz83BETNT9wZm5F9gLMD4+nhMT9VYxPT1N3ccMyrCy3b7n80vev3tslruPnNm0x7ZP1FrPYssvJ9P8dTV1WzY1FzQ3m7nq6VeujgUOvA14X0S8G3gt8EvAJ4G1EXFhZs4CG4HjPU8nSVpUxzHwzLwzMzdm5ibgNuDLmbkdOAjcUi22A3i4byklSWdZyXngfwD8bkQ8B7wBuKc3kSRJ3ehmCOUXMnMamK5uPw9c3/tIkqRueCWmJBXKApekQlngklQoC1ySCmWBS1KhLHBJKpQFLkmFssAlqVAWuCQVygKXpEJZ4JJUqFrvhaLz26YO70MuabA8ApekQlngklQoC1ySCuUYuIZisfH0Y3e9Z8BJpHJ5BC5JhbLAJalQFrgkFcoxcBXBMXPpbB6BS1KhLHBJKpQFLkmFssAlqVAWuCQVygKXpEJZ4JJUKAtckgplgUtSoSxwSSqUBS5JhbLAJalQHQs8Il4bEV+NiG9ExFMR8dFq/uUR8VhEPBcR90fERf2PK0ma080R+E+BGzPzWmAzsDUibgA+BnwiM68ATgE7+xdTkrRQxwLPlplq8tXVVwI3Ag9W8/cBN/cloSSprcjMzgtFXAAcBq4APgX8BfCV6uibiLgM+EJmXtPmsbuAXQCjo6NbpqamagWcmZlhZGSk1mMGZVjZjhw/veT9o6vgxCtnpsc2rKm1nrrLL2X+uuY/X7362YstX4f7WH3mqmeluSYnJw9n5vjC+V19oENm/hzYHBFrgYeAN3X7gzNzL7AXYHx8PCcmJrp9KADT09PUfcygDCvb7Yt8uMGc3WOz3H3kzKY9tn2i1nrqLr+U+eua/3z16mcvtnwd7mP1mauefuWqdRZKZr4EHATeCqyNiLmW2Agc73E2SdISujkL5dLqyJuIWAW8AzhKq8hvqRbbATzcr5CSpLN1M4SyHthXjYO/CnggM/dHxNPAVET8KfA4cE8fc0pt+VmZOp91LPDM/CZwXZv5zwPX9yOUJKkzr8SUpEJZ4JJUKAtckgplgUtSoSxwSSqUBS5JherqUnqVbbFzpQf9s3ePzS7rcvxeavdc7B6bZWLwUaQV8whckgplgUtSoSxwSSqUBS5JhbLAJalQFrgkFcoCl6RCWeCSVCgLXJIKZYFLUqEscEkqVDHvheJnHw7OMN87RVL3PAKXpEJZ4JJUKAtckgplgUtSoSxwSSqUBS5JhbLAJalQxZwHfi7zHHdJy+ERuCQVygKXpEJZ4JJUKAtckgrVscAj4rKIOBgRT0fEUxFxRzV/XUQ8GhHPVt8v7n9cSdKcbo7AZ4HdmXk1cAPwwYi4GtgDHMjMK4ED1bQkaUA6FnhmvpiZX69u/wQ4CmwAtgH7qsX2ATf3K6Qk6Wy1xsAjYhNwHfAYMJqZL1Z3fR8Y7WkySdKSIjO7WzBiBPhX4M8y83MR8VJmrp13/6nMPGscPCJ2AbsARkdHt0xNTdUKODMzw8jICEeOn257/9iGNbXW10tz2Vaq7u+22PJzRlfBiVdWHKvnusm13N95JesZXQVvXDe8/WgpvdrHes1c9aw01+Tk5OHMHF84v6sCj4hXA/uBL2Xmx6t5zwATmfliRKwHpjPzqqXWMz4+nocOHaoVfHp6momJiUZerTiXbaXq/m6dPjFn99gsdx9p3kW23eRa7u+8kvXsHpvlw9u31Vr/oPRqH+s1c9Wz0lwR0bbAuzkLJYB7gKNz5V15BNhR3d4BPLzsdJKk2ro5THsb8AHgSEQ8Uc37Q+Au4IGI2Al8B7i1PxElSe10LPDM/HcgFrn7pt7GkSR1yysxJalQFrgkFcoCl6RCNe9cM6lBenn6ahNPhVXZPAKXpEJZ4JJUKAtckgrlGLgape4l803Ty/yb9nye3WOz3L5gnY6Za45H4JJUKAtckgplgUtSoSxwSSqUBS5JhbLAJalQFrgkFcoCl6RCWeCSVCgLXJIKZYFLUqHO2fdC8b2XJZ3rPAKXpEJZ4JJUKAtckgp1zo6BnwtKf29sSf3lEbgkFcoCl6RCWeCSVCgLXJIKZYFLUqEscEkqlAUuSYXyPHCdkzyHXueDjkfgEfGZiDgZEU/Om7cuIh6NiGer7xf3N6YkaaFuhlDuBbYumLcHOJCZVwIHqmlJ0gB1LPDM/DfgvxfM3gbsq27vA27ucS5JUgeRmZ0XitgE7M/Ma6rplzJzbXU7gFNz020euwvYBTA6OrplamqqVsCZmRlGRkY4cvx02/vHNqxpO7/u8ssxl22lFsu6XKOr4MQrPV1lTzQ51xvX9WY/Ws62XGpd7Z6zXu7Dy9Wrfb/XztVck5OThzNzfOH8FRd4NX0qMzuOg4+Pj+ehQ4fq5GZ6epqJiYnaH9AwiA90mMu2Ur1+wW332Cx3H2ne69NNzvXh7dva3ter/W4pS62r3XPWhA8l6dW+32vnaq6IaFvgyz2N8ERErK9WvB44uexkkqRlWW6BPwLsqG7vAB7uTRxJUrc6/j0bEfcBE8AlEfEC8CfAXcADEbET+A5waz9DSk3TxPPM/RzY80/HAs/M9y9y1009ziJJqsFL6SWpUBa4JBWqeed0NcxSY533bl09wCTqpyaOaUudeAQuSYWywCWpUBa4JBXKMfDKIMZAHWfVMPT7bSg8/3x4PAKXpEJZ4JJUKAtckgpV/Bj4MMeVjxw/ze1tfr5jf6qj7j7sayma4xG4JBXKApekQlngklQoC1ySCmWBS1KhLHBJKpQFLkmFssAlqVDFX8gjaTDmX0C0e2y27UVsiy2/kBe79YZH4JJUKAtckgplgUtSoc67MXA/uEEavn5/CMT58iETHoFLUqEscEkqlAUuSYU678bAJbWcC6/VLPwdujk//VziEbgkFcoCl6RCWeCSVCjHwCU1xrDO3+7Vz11sPfduXV07UzdWdAQeEVsj4pmIeC4i9vQqlCSps2UXeERcAHwKeBdwNfD+iLi6V8EkSUtbyRH49cBzmfl8Zv4vMAVs600sSVInkZnLe2DELcDWzPytavoDwFsy80MLltsF7KomrwKeqfmjLgF+uKyQ/dfUbOaqp6m5oLnZzFXPSnP9SmZeunBm31/EzMy9wN7lPj4iDmXmeA8j9UxTs5mrnqbmguZmM1c9/cq1kiGU48Bl86Y3VvMkSQOwkgL/GnBlRFweERcBtwGP9CaWJKmTZQ+hZOZsRHwI+BJwAfCZzHyqZ8nOWPbwywA0NZu56mlqLmhuNnPV05dcy34RU5I0XF5KL0mFssAlqVCNLvCmXKofEZ+JiJMR8eS8eesi4tGIeLb6fvEQcl0WEQcj4umIeCoi7mhQttdGxFcj4htVto9W8y+PiMeqbXp/9QL4wEXEBRHxeETsb0quiDgWEUci4omIOFTNa8K2XBsRD0bEtyLiaES8tSG5rqqeq7mvH0fERxqS7Xeq/f7JiLiv+vfQ832ssQXesEv17wW2Lpi3BziQmVcCB6rpQZsFdmfm1cANwAer56gJ2X4K3JiZ1wKbga0RcQPwMeATmXkFcArYOYRsAHcAR+dNNyXXZGZunnfOcBO25SeBL2bmm4BraT1vQ8+Vmc9Uz9VmYAvwP8BDw84WERuA3wbGM/MaWid53EY/9rHMbOQX8FbgS/Om7wTuHGKeTcCT86afAdZXt9cDzzTgOXsYeEfTsgGvA74OvIXW1WgXttvGA8yzkdY/7BuB/UA0JNcx4JIF84a6LYE1wLepTnhoSq42Od8J/EcTsgEbgO8B62id6bcf+M1+7GONPQLnzJMw54VqXlOMZuaL1e3vA6PDDBMRm4DrgMdoSLZqmOIJ4CTwKPBfwEuZOVstMqxt+lfA7wP/V02/oSG5EvjniDhcvQUFDH9bXg78APjbasjp0xGxugG5FroNuK+6PdRsmXkc+Evgu8CLwGngMH3Yx5pc4MXI1n+pQzsfMyJGgH8EPpKZP55/3zCzZebPs/Xn7UZab372pmHkmC8i3guczMzDw87Sxtsz8820hg0/GBG/Pv/OIW3LC4E3A3+dmdcBL7NgSKIB+/9FwPuAzy68bxjZqjH3bbT+8/tlYDVnD8H2RJMLvOmX6p+IiPUA1feTwwgREa+mVd5/n5mfa1K2OZn5EnCQ1p+NayNi7gKyYWzTtwHvi4hjtN5B80ZaY7zDzjV35EZmnqQ1lns9w9+WLwAvZOZj1fSDtAp92Lnmexfw9cw8UU0PO9tvAN/OzB9k5s+Az9Ha73q+jzW5wJt+qf4jwI7q9g5a488DFREB3AMczcyPNyzbpRGxtrq9itbY/FFaRX7LsLJl5p2ZuTEzN9Hap76cmduHnSsiVkfE6+du0xrTfZIhb8vM/D7wvYi4qpp1E/D0sHMt8H7ODJ/A8LN9F7ghIl5X/Rude856v48N84WHLl4MeDfwn7TGTv9oiDnuozWW9TNaRyQ7aY2bHgCeBf4FWDeEXG+n9efhN4Enqq93NyTbrwGPV9meBP64mv+rwFeB52j9yfuaIW7XCWB/E3JVP/8b1ddTc/t7Q7blZuBQtS3/Cbi4CbmqbKuBHwFr5s0bejbgo8C3qn3/74DX9GMf81J6SSpUk4dQJElLsMAlqVAWuCQVygKXpEJZ4JJUKAtckgplgUtSof4f+O9oyaFCVmkAAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qIe1EURhBLZU",
        "outputId": "4cd9816e-ae6f-47e6-c3fc-7a162e534301",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "a=df.Age.mean()+3*(df.Age.std())\n",
        "a"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "73.27860964406095"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "e-Or_RjRB6K5",
        "outputId": "3908e6fa-78f5-4389-d94d-bcbea4e3ba40",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 350
        }
      },
      "source": [
        "import seaborn as sns\n",
        "sns.boxplot('Age',data=df) # finding the outliers"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.\n",
            "  FutureWarning\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fc17f416d68>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 16
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAEGCAYAAABbzE8LAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAANiElEQVR4nO3db2xd9XnA8e8Te7TB3krjoIwZVLe6FYg1K22sDlY0jQxWl0zttPGCaBp5UalvqiRFk6YiosXRwotK07ZgbZPQ1pF0E5v6Z1sFUboA0aTtBZ3dwhIItHdr2mIVkpoVloS1dfLbi3PseSZg+8b3Pifw/UhWfO+1z+/B99wvx8fxSZRSkCT13prsASTprcoAS1ISAyxJSQywJCUxwJKUpH8lH7x+/foyMjLSpVEk6c1pamrqB6WUKxffv6IAj4yMMDk5uXpTSdJbQER850L3ewpCkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpiQGWpCQGWJKSGGBJSmKAJSmJAZakJAZYkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpyYr+TTh138TEBO12u+frTk9PAzA8PNzztZej1Wqxffv27DGkVWWAG6bdbvPkseOcu3xdT9ftO/syAC/8qHm7RN/Zl7JHkLqiea82ce7ydbx63e09XXPtswcBer7ucszNJr3ZeA5YkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpiQGWpCQGWJKSGGBJSmKAJSmJAZakJAZYkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpiQGWpCQGWJKSGGBJSmKAJSmJAZakJAZYkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpiQGWpCQGWJKSGGBJSmKAJSmJAZakJAZYkpIYYElKYoAlKYkBlqQkBliSkvQkwBMTE0xMTPRiKUld5Gt5dfX3YpF2u92LZSR1ma/l1eUpCElKYoAlKYkBlqQkBliSkhhgSUpigCUpiQGWpCQGWJKSGGBJSmKAJSmJAZakJAZYkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpiQGWpCQGWJKSGGBJSmKAJSmJAZakJAZYkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpiQGWpCQGWJKSGGBJSmKAJSmJAZakJAZYkpIYYElKYoAlKYkBltSxyclJNm/ezNTUFAAzMzPs2LGDmZmZFW+r3W6zZcsWpqam2LFjB5OTk2zZsoV2u73k517MupnbNsCSOjY+Ps758+fZvXs3APv37+fo0aMcOHBgxdvau3cvZ86cYffu3Rw9epTx8XHOnDnD3r17l/zci1k3c9sGWFJHJicnOX36NACnT5/myJEjHDp0iFIKhw4dWtERY7vd5sSJE/PbKqXMb/vEiRNveBQ8MzPT8bpL6ea2AfpXdWuvY3p6mldffZWdO3f2YrlLWrvdZs2PS/YYjbLmf16h3f5v958GaLfbrF27FqiOfhe677775t8/d+4cBw4c4O67717Wdpc6yt27dy8PPvjgBR/bv38/58+f72jdpXRz27CMI+CI+GRETEbE5KlTp1ZtYUmXtrkj1Dmzs7PMzs7Ov3/48OFlb2vu6LeTxx999NGO111KN7cNyzgCLqU8ADwAMDo62tGh2fDwMAD79u3r5NPfUnbu3MnUf76YPUajnH/7z9B6zwb3nwZY+F3I4ODg/4twf3+Vk9nZWfr7+7ntttuWvd2RkZE3jOzIyMjrPnbrrbdy8ODBjtZdSje3DZ4DltShxacg7r33XtasqZLS19fHXXfdtext7dq1q+PHt23b1vG6S+nmtsEAS+rQ6Ogog4ODQHU0fMsttzA2NkZEMDY2xtDQ0LK31Wq15o9yBwcHiYj5bY+MjNBqtV73c4eGhjpedynd3DYYYEkXYXx8nDVr1rBnzx6gOmLcuHFjR0eKu3btYmBggD179rBx40bGx8cZGBhY8uj4YtfN3HZP/haEpDen0dFRHn/88fnbQ0ND3H///R1tq9Vq8cgjjwCwadMmgPnbS7mYdTO37RGwJCUxwJKUxABLUhIDLElJDLAkJTHAkpTEAEtSEgMsSUkMsCQlMcCSlMQAS1ISAyxJSQywJCUxwJKUxABLUhIDLElJDLAkJTHAkpTEAEtSEgMsSUkMsCQlMcCSlMQAS1ISAyxJSQywJCUxwJKUxABLUhIDLElJDLAkJTHAkpTEAEtSEgMsSUkMsCQlMcCSlMQAS1ISAyxJSQywJCUxwJKUpL8Xi7RarV4sI6nLfC2vrp4EePv27b1YRlKX+VpeXZ6CkKQkBliSkhhgSUpigCUpiQGWpCQGWJKSGGBJSmKAJSmJAZakJAZYkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpiQGWpCQGWJKSGGBJSmKAJSmJAZakJAZYkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpiQGWpCQGWJKSGGBJSmKAJSmJAZakJAZYkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpSX/2AHqtvrMvsfbZgz1ecwag5+suR9/Zl4AN2WNIq84AN0yr1UpZd3p6FoDh4SaGbkPa10XqJgPcMNu3b88eQVKPeA5YkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpiQGWpCQGWJKSGGBJSmKAJSmJAZakJAZYkpIYYElKYoAlKYkBlqQkBliSkhhgSUpigCUpiQGWpCRRSln+B0ecAr6zwjXWAz9Y4ef0SlNnc66Vaepc0NzZnGtlLnaud5VSrlx854oC3ImImCyljHZ1kQ41dTbnWpmmzgXNnc25VqZbc3kKQpKSGGBJStKLAD/QgzU61dTZnGtlmjoXNHc251qZrszV9XPAkqQL8xSEJCUxwJKUpKsBjoixiHguItoR8ZlurrXEHJ+LiJMRcWzBfesi4nBEfKv+850Jc10TEUci4pmIeDoidjZotrdHxNci4ql6tj31/e+OiCfq5/TvIuKyhNn6IuIbEfFwU2aq5zgREUcj4smImKzva8JzeUVEfDEino2I4xFxU0Pmurb+Ws29vRIRn27IbHfX+/2xiHiofj2s+n7WtQBHRB/wp8BHgeuBrRFxfbfWW8KDwNii+z4DPFZKeS/wWH2712aB3y2lXA/cCHyq/ho1YbYfAZtLKe8HbgDGIuJG4LPAH5dSWsB/AZ9ImG0ncHzB7SbMNOeWUsoNC/7OaBOey33AoVLKdcD7qb526XOVUp6rv1Y3AJuAs8DfZ88WEcPADmC0lPI+oA+4k27sZ6WUrrwBNwFfXXD7HuCebq23jHlGgGMLbj8HXFW/fxXwXNZsC2b6R+C2ps0GXA58HfhFqt8G6r/Qc9yjWa6melFuBh4GInumBbOdANYvui/1uQTeAXyb+gfuTZnrAnP+GvCvTZgNGAa+B6wD+uv97CPd2M+6eQpi7j9izvP1fU2xoZTy/fr9F4ANmcNExAjwAeAJGjJb/a3+k8BJ4DDwH8APSymz9YdkPKd/AvwecL6+PdSAmeYU4J8iYioiPlnfl/1cvhs4BfxVfdrmLyJioAFzLXYn8FD9fupspZRp4A+B7wLfB14GpujCfuYP4YBS/S8t7e/jRcQg8CXg06WUVxY+ljlbKeVcqb49vBr4EHBdxhxzIuLXgZOllKnMOd7AzaWUD1KddvtURPzywgeTnst+4IPAn5dSPgCcYdG39A3Y/y8DPgZ8YfFjGbPV55w/TvU/r58DBnjtKcxV0c0ATwPXLLh9dX1fU7wYEVcB1H+ezBgiIn6KKr5/U0r5cpNmm1NK+SFwhOrbrisior9+qNfP6YeBj0XECeBvqU5D7EueaV595EQp5STVucwPkf9cPg88X0p5or79RaogZ8+10EeBr5dSXqxvZ892K/DtUsqpUspPgC9T7Xurvp91M8D/Bry3/snhZVTfYnyli+ut1FeAbfX726jOv/ZURATwl8DxUsofNWy2KyPiivr9tVTnpo9ThfiOjNlKKfeUUq4upYxQ7U+Pl1J+O3OmORExEBE/Pfc+1TnNYyQ/l6WUF4DvRcS19V2/CjyTPdciW/m/0w+QP9t3gRsj4vL6NTr3NVv9/azLJ7NvB75Jde7w3l6eSF80x0NU53J+QnVE8Amqc4ePAd8CHgXWJcx1M9W3V/8OPFm/3d6Q2X4B+EY92zHg9+v73wN8DWhTfcv4tqTn9FeAh5syUz3DU/Xb03P7e0OeyxuAyfq5/AfgnU2Yq55tAJgB3rHgvvTZgD3As/W+/3ngbd3Yz/xVZElK4g/hJCmJAZakJAZYkpIYYElKYoAlKYkB1iUhIn4jIkpEpP42nrSaDLAuFVuBf6n/lN4UDLAar75Wxs1Uv0BzZ33fmoj4s/oat4cj4mBE3FE/tiki/rm+KM5X536tVWoaA6xLwceprmf7TWAmIjYBv0l1idHrgd+huk7F3LU1JoA7SimbgM8B92UMLS2lf+kPkdJtpbroDlQX4dlKte9+oZRyHnghIo7Uj18LvA84XP0aP31Uv4YuNY4BVqNFxDqqq55tjIhCFdRCdbWxC34K8HQp5aYejSh1zFMQaro7gM+XUt5VShkppVxD9S88vAT8Vn0ueAPVxXmg+tcUroyI+VMSEfHzGYNLSzHAarqtvPZo90vAz1Jd2e4Z4K+p/smkl0spP6aK9mcj4imqK8z9Uu/GlZbPq6HpkhURg6WU0xExRHWZwA+X6vq30iXBc8C6lD1cXzT+MuAPjK8uNR4BS1ISzwFLUhIDLElJDLAkJTHAkpTEAEtSkv8FCi149yDLXg0AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "sZdgl51eDJrJ",
        "outputId": "e233d018-6812-4755-9341-787ec1fac12c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 235
        }
      },
      "source": [
        "df['Age_na']=df.Age.fillna(a)\n",
        "df.head(6)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>Survived</th>\n",
              "      <th>Age</th>\n",
              "      <th>Fare</th>\n",
              "      <th>Age_na</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>22.0</td>\n",
              "      <td>7.2500</td>\n",
              "      <td>22.00000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>1</td>\n",
              "      <td>38.0</td>\n",
              "      <td>71.2833</td>\n",
              "      <td>38.00000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>1</td>\n",
              "      <td>26.0</td>\n",
              "      <td>7.9250</td>\n",
              "      <td>26.00000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>1</td>\n",
              "      <td>35.0</td>\n",
              "      <td>53.1000</td>\n",
              "      <td>35.00000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>35.0</td>\n",
              "      <td>8.0500</td>\n",
              "      <td>35.00000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>0</td>\n",
              "      <td>NaN</td>\n",
              "      <td>8.4583</td>\n",
              "      <td>73.27861</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "   Survived   Age     Fare    Age_na\n",
              "0         0  22.0   7.2500  22.00000\n",
              "1         1  38.0  71.2833  38.00000\n",
              "2         1  26.0   7.9250  26.00000\n",
              "3         1  35.0  53.1000  35.00000\n",
              "4         0  35.0   8.0500  35.00000\n",
              "5         0   NaN   8.4583  73.27861"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 17
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dXMdC8R8ELbI",
        "outputId": "5783d7a2-8c89-45b1-fa29-015b07b84364",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 282
        }
      },
      "source": [
        "df.Age_na.hist(bins=50)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fc17eea7e48>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 18
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD4CAYAAAAXUaZHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAUYklEQVR4nO3df5BlZ13n8ffXRCNOUzOJiV2zk6wdamMsyMjAdEUsXaqbKA7BIotlYaZSbEaiA1WguDVVOtEtwaWoyu7yQy0UHU0M7o/pICGQmqAYs2lRS8AeGDMTQiCBQdMbZyAJEztQKSZ89497erk2t3+cPvf0Pf3k/aq61fc+5557PnPu7U+ffu7pO5GZSJLK8h2jDiBJGj7LXZIKZLlLUoEsd0kqkOUuSQU6d9QBAC688MKcmJiovd5TTz3Fli1bhh+oIXPV19Vs5qqnq7mgu9ma5Dp69OhXMvOigQszc+SX3bt353rce++961qvbeaqr6vZzFVPV3Nldjdbk1zAXC7Tq07LSFKBLHdJKpDlLkkFstwlqUCWuyQVyHKXpAJZ7pJUIMtdkgq0arlHxC0RcToiTvSN3RYRx6rLyYg4Vo1PRMTX+5b9fpvhJUmDreXjB24F3gP8yeJAZv7s4vWIeCdwpu/+D2fmrmEFlKS2TRy8a9llJ2965QYmGZ5Vyz0zPxYRE4OWRUQArwFeNtxYkqQmItfw3+xV5X4kM69YMv5S4F2ZOdl3v/uBzwFPAv85M/96mcfcD+wHGB8f3z0zM1M7/MLCAmNjY7XXa5u56utqNnPV09VcsHK24/NnBo4D7Nyxta1IQLN9Nj09fXSxf5dq+qmQe4HDfbcfBf5tZj4WEbuBD0XECzLzyaUrZuYh4BDA5ORkTk1N1d747Ows61mvbeaqr6vZzFVPV3PBytn2rTQtc93gdYalrX227rNlIuJc4KeB2xbHMvPpzHysun4UeBj4gaYhJUn1NDkV8seBz2bmI4sDEXFRRJxTXX8ecBnwhWYRJUl1reVUyMPA3wGXR8QjEXFDteha/vWUDMBLgfuqUyM/ALwhMx8fZmBJ0urWcrbM3mXG9w0Yux24vXksSVIT/oWqJBXIcpekAlnuklQgy12SCmS5S1KBLHdJKpDlLkkFstwlqUCWuyQVyHKXpAJZ7pJUIMtdkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFchyl6QCWe6SVCDLXZIKtGq5R8QtEXE6Ik70jb01IuYj4lh1ubpv2Y0R8VBEPBgRP9lWcEnS8tZy5H4rsGfA+Lszc1d1+QhARDwfuBZ4QbXO70XEOcMKK0lam1XLPTM/Bjy+xse7BpjJzKcz84vAQ8CVDfJJktYhMnP1O0VMAEcy84rq9luBfcCTwBxwIDOfiIj3AB/PzP9Z3e9m4M8y8wMDHnM/sB9gfHx898zMTO3wCwsLjI2N1V6vbeaqr6vZzFVPV3PBytmOz59Zdr2dO7a2FQlots+mp6ePZubkoGXnrjPPe4G3AVl9fSfwujoPkJmHgEMAk5OTOTU1VTvE7Ows61mvbeaqr6vZzFVPV3PBytn2Hbxr2fVOXjd4nWFpa5+t62yZzDyVmc9k5jeBP+RbUy/zwCV9d724GpMkbaB1lXtEbO+7+Wpg8UyaO4FrI+K8iLgUuAz4ZLOIkqS6Vp2WiYjDwBRwYUQ8ArwFmIqIXfSmZU4CrwfIzPsj4v3AZ4CzwBsz85l2okuSlrNquWfm3gHDN69w/7cDb28SSpLUjH+hKkkFstwlqUCWuyQVyHKXpAJZ7pJUIMtdkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFchyl6QCWe6SVCDLXZIKZLlLUoEsd0kqkOUuSQWy3CWpQJa7JBXIcpekAq1a7hFxS0ScjogTfWP/PSI+GxH3RcQdEbGtGp+IiK9HxLHq8vtthpckDbaWI/dbgT1Lxu4GrsjMHwI+B9zYt+zhzNxVXd4wnJiSpDpWLffM/Bjw+JKxv8jMs9XNjwMXt5BNkrROw5hzfx3wZ323L42IT0fEX0XEvx/C40uSaorMXP1OERPAkcy8Ysn4rwOTwE9nZkbEecBYZj4WEbuBDwEvyMwnBzzmfmA/wPj4+O6ZmZna4RcWFhgbG6u9XtvMVV9Xs5mrnq7mgpWzHZ8/s+x6O3dsbSsS0GyfTU9PH83MyUHLzl1voIjYB/wUcFVWPyEy82ng6er60Yh4GPgBYG7p+pl5CDgEMDk5mVNTU7UzzM7Osp712mau+rqazVz1dDUXrJxt38G7ll3v5HWD1xmWtvbZuqZlImIP8CvAqzLza33jF0XEOdX15wGXAV8YRlBJ0tqteuQeEYeBKeDCiHgEeAu9s2POA+6OCICPV2fGvBT4LxHxDeCbwBsy8/GBDyxJas2q5Z6ZewcM37zMfW8Hbm8aSpLUjH+hKkkFstwlqUCWuyQVyHKXpAJZ7pJUIMtdkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFchyl6QCWe6SVCDLXZIKZLlLUoEsd0kqkOUuSQWy3CWpQJa7JBXIcpekAlnuklSgNZV7RNwSEacj4kTf2AURcXdEfL76en41HhHxOxHxUETcFxEvbiu8JGmwtR653wrsWTJ2ELgnMy8D7qluA7wCuKy67Afe2zymJKmONZV7Zn4MeHzJ8DXA+6rr7wP+Q9/4n2TPx4FtEbF9GGElSWsTmbm2O0ZMAEcy84rq9lczc1t1PYAnMnNbRBwBbsrMv6mW3QP8ambOLXm8/fSO7BkfH989MzNTO/zCwgJjY2O112ubuerrajZz1dPVXLBytuPzZ5Zdb+eOrW1FAprts+np6aOZOTlo2bmNUlUyMyNibT8lvrXOIeAQwOTkZE5NTdXe7uzsLOtZr23mqq+r2cxVT1dzwcrZ9h28a9n1Tl43eJ1haWufNTlb5tTidEv19XQ1Pg9c0ne/i6sxSdIGaVLudwLXV9evBz7cN/4fq7NmXgKcycxHG2xHklTTmqZlIuIwMAVcGBGPAG8BbgLeHxE3AF8CXlPd/SPA1cBDwNeAnxtyZknSKtZU7pm5d5lFVw24bwJvbBJKktSMf6EqSQWy3CWpQJa7JBXIcpekAlnuklQgy12SCmS5S1KBLHdJKpDlLkkFstwlqUCWuyQVyHKXpAJZ7pJUIMtdkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFchyl6QCrek/yB4kIi4Hbusbeh7wG8A24BeAL1fjv5aZH1l3QklSbesu98x8ENgFEBHnAPPAHcDPAe/OzHcMJaEkqbZhTctcBTycmV8a0uNJkhqIzGz+IBG3AJ/KzPdExFuBfcCTwBxwIDOfGLDOfmA/wPj4+O6ZmZna211YWGBsbKxB8naYq76uZjNXPV3NBStnOz5/Ztn1du7Y2lYkoNk+m56ePpqZk4OWNS73iPgu4P8CL8jMUxExDnwFSOBtwPbMfN1KjzE5OZlzc3O1tz07O8vU1FT90C0zV31dzWauerqaC1bONnHwrmXXO3nTK1tK1NNkn0XEsuU+jGmZV9A7aj8FkJmnMvOZzPwm8IfAlUPYhiSphmGU+17g8OKNiNjet+zVwIkhbEOSVMO6z5YBiIgtwE8Ar+8b/m8RsYvetMzJJcskSRugUbln5lPA9y4Ze22jRJKkxvwLVUkqkOUuSQWy3CWpQJa7JBXIcpekAlnuklQgy12SCmS5S1KBLHdJKpDlLkkFstwlqUCWuyQVyHKXpAJZ7pJUIMtdkgpkuUtSgSx3SSqQ5S5JBbLcJalAlrskFajRf5ANEBEngX8BngHOZuZkRFwA3AZMACeB12TmE023JUlam2EduU9n5q7MnKxuHwTuyczLgHuq25KkDdL4yH0Z1wBT1fX3AbPAr7a0LfWZOHjXwPGTN71yg5NIGqXIzGYPEPFF4AkggT/IzEMR8dXM3FYtD+CJxdt96+0H9gOMj4/vnpmZqb3thYUFxsbGGuVvwyhzHZ8/M3B8546tnd1f4HNZl7nqWynbct830PveaVOTfTY9PX20b8bkXxlGue/IzPmI+D7gbuAXgTv7yzwinsjM85d7jMnJyZybm6u97dnZWaamptaRul2jzLXSkXtX9xf4XNZlrvpWyrbc9w20/1tvk30WEcuWe+M598ycr76eBu4ArgRORcT2auPbgdNNtyNJWrtG5R4RWyLiuYvXgZcDJ4A7geuru10PfLjJdiRJ9TR9Q3UcuKM3rc65wP/OzD+PiL8H3h8RNwBfAl7TcDvapHyDVxqNRuWemV8AXjhg/DHgqiaPrdGylKXNzb9QlaQCWe6SVCDLXZIKZLlLUoEsd0kqkOUuSQVq64PD1DETB+/iwM6z7FtyiqOnNkpl8shdkgpkuUtSgSx3SSqQ5S5JBfINVQ3FSp+HLWnjWe7qFD+wTBoOy12bmj8MpMGcc5ekAlnuklQgy12SCmS5S1KBLHdJKpBny6hIK51375k0ejZY95F7RFwSEfdGxGci4v6IeHM1/taImI+IY9Xl6uHFlSStRZMj97PAgcz8VEQ8FzgaEXdXy96dme9oHk9tG9Vfli7d7qCPI95og/bFgZ1nmdr4KFJj6y73zHwUeLS6/i8R8QCwY1jBJEnrN5Q3VCNiAngR8Ilq6E0RcV9E3BIR5w9jG5KktYvMbPYAEWPAXwFvz8wPRsQ48BUggbcB2zPzdQPW2w/sBxgfH989MzNTe9sLCwuMjY01id+KUeY6Pn9m2WXjz4FTX2/2+Dt3bK293bVYLdswt1vnscafA993weD7j5Kv/fpWyrbS62i518uwNNln09PTRzNzctCyRuUeEd8JHAE+mpnvGrB8AjiSmVes9DiTk5M5NzdXe/uzs7NMTU117vNFFnONwkpz6Ad2nuWdx7t5gtRq2ZZ7LtfznkGdxzqw8yy/eN01tbfRtlG+xlbS1VywcrZRnl3VZJ9FxLLl3uRsmQBuBh7oL/aI2N53t1cDJ9a7DUnS+jQ5jPtR4LXA8Yg4Vo39GrA3InbRm5Y5Cby+UcJnsa79RiJp82hytszfADFg0UfWH0eSNAx+/IAkFchyl6QCWe6SVKBunhcnbQLDesPbN87VBstdm8KoPgNnmIb1b5g4eNfAz+Lxh4H6OS0jSQWy3CWpQM/KaRnnOCWVziN3SSrQs/LIfbMr4c1FSe3yyF2SCuSRu551/M1HzwYeuUtSgTxyl4asa78ZjPI/otDoWO4NLPdNc+ueLRucRG3qWllLa+G0jCQVyCP3NWj7yM0jQ41K3T/oa/v+Gp6iy31UpXl8/sy3fagT+IJWPXVfvx4kqJ/TMpJUoKKP3CVtjMXfGgZ9FPFK91/K326Hx3Lv49y6NFptl/6z6XuwtXKPiD3AbwPnAH+UmTe1tS1Jw7XZS3BQ/gM7z/JsOp5tZc49Is4Bfhd4BfB8YG9EPL+NbUmSvl1bP8auBB7KzC8ARMQMcA3wmZa2J6lgo5yjb/v/ym3rjx4jM4f/oBE/A+zJzJ+vbr8W+OHMfFPfffYD+6ublwMPrmNTFwJfaRi3Deaqr6vZzFVPV3NBd7M1yfX9mXnRoAUjm4DKzEPAoSaPERFzmTk5pEhDY676uprNXPV0NRd0N1tbudo6z30euKTv9sXVmCRpA7RV7n8PXBYRl0bEdwHXAne2tC1J0hKtTMtk5tmIeBPwUXqnQt6Smfe3sKlG0zotMld9Xc1mrnq6mgu6m62VXK28oSpJGi0/W0aSCmS5S1KBNmW5R8SeiHgwIh6KiIMjznJLRJyOiBN9YxdExN0R8fnq6/kjyHVJRNwbEZ+JiPsj4s1dyBYR3x0Rn4yIf6hy/WY1fmlEfKJ6Tm+r3ojfcBFxTkR8OiKOdCzXyYg4HhHHImKuGuvC62xbRHwgIj4bEQ9ExI+MOldEXF7tp8XLkxHxy6POVWX7T9Xr/kREHK6+H1p5jW26cu/gRxvcCuxZMnYQuCczLwPuqW5vtLPAgcx8PvAS4I3Vfhp1tqeBl2XmC4FdwJ6IeAnwX4F3Z+a/A54AbtjgXIveDDzQd7sruQCmM3NX3znRo34uoff5UX+emT8IvJDevhtprsx8sNpPu4DdwNeAO0adKyJ2AL8ETGbmFfRONrmWtl5jmbmpLsCPAB/tu30jcOOIM00AJ/puPwhsr65vBx7swH77MPATXcoGfA/wKeCH6f2F3rmDnuMNzHMxvW/6lwFHgOhCrmrbJ4ELl4yN9LkEtgJfpDoxoyu5lmR5OfC3XcgF7AD+CbiA3pmKR4CfbOs1tumO3PnWDlr0SDXWJeOZ+Wh1/Z+B8VGGiYgJ4EXAJ+hAtmrq4xhwGrgbeBj4amaere4yquf0t4BfAb5Z3f7ejuQCSOAvIuJo9dEdMPrn8lLgy8AfV1NZfxQRWzqQq9+1wOHq+khzZeY88A7gH4FHgTPAUVp6jW3Gct9UsvfjeGTnm0bEGHA78MuZ+WT/slFly8xnsvcr88X0PmTuBzc6w1IR8VPA6cw8Ouosy/ixzHwxvenIN0bES/sXjui5PBd4MfDezHwR8BRLpjpG+fqv5q5fBfzp0mWjyFXN8V9D74fivwG28O1TukOzGct9M3y0wamI2A5QfT09ihAR8Z30iv1/ZeYHu5QNIDO/CtxL71fRbRGx+Ed1o3hOfxR4VUScBGboTc38dgdyAf//qI/MPE1v/vhKRv9cPgI8kpmfqG5/gF7ZjzrXolcAn8rMU9XtUef6ceCLmfnlzPwG8EF6r7tWXmObsdw3w0cb3AlcX12/nt5894aKiABuBh7IzHd1JVtEXBQR26rrz6H3PsAD9Er+Z0aVKzNvzMyLM3OC3mvq/2TmdaPOBRARWyLiuYvX6c0jn2DEz2Vm/jPwTxFxeTV0Fb2P9R7567+yl29NycDoc/0j8JKI+J7q+3Nxf7XzGhvVGx0N35i4GvgcvbnaXx9xlsP05s++Qe9I5gZ6c7X3AJ8H/hK4YAS5fozer533Aceqy9Wjzgb8EPDpKtcJ4Deq8ecBnwQeovdr9HkjfE6ngCNdyVVl+Ifqcv/ia37Uz2WVYRcwVz2fHwLO70iuLcBjwNa+sS7k+k3gs9Vr/38A57X1GvPjBySpQJtxWkaStArLXZIKZLlLUoEsd0kqkOUuSQWy3CWpQJa7JBXo/wEh/h8Z/wxmaQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "CxAjcp2_Frzl",
        "outputId": "e020cad0-13cb-4595-cc28-cc1f644fedee",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 351
        }
      },
      "source": [
        "sns.boxplot('Age_na',data=df) # no outliers"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "/usr/local/lib/python3.6/dist-packages/seaborn/_decorators.py:43: FutureWarning: Pass the following variable as a keyword arg: x. From version 0.12, the only valid positional argument will be `data`, and passing other arguments without an explicit keyword will result in an error or misinterpretation.\n",
            "  FutureWarning\n"
          ],
          "name": "stderr"
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.axes._subplots.AxesSubplot at 0x7fc17edb7940>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 19
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAWAAAAEHCAYAAACQkJyuAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAMB0lEQVR4nO3dfYxl9V3H8c+X3dIuVGkpiLg03TaLEKyWtgTBVqPVKiWmmEgixBgSm/SfZt02JqbERNMY/yAxKllbk0Zbk2qq6ZM2pLEiVqP+Qd2loMtjR0tbNjxswYLKQwv8/OOcseNCgdnOvd/L8nolk5l7ZnbOd+ecee+9v9k5t8YYAWD5juseAOCFSoABmggwQBMBBmgiwABNtm/mg0855ZSxa9euBY0CcGw6cODA18YYpx65fVMB3rVrV/bv3791UwG8AFTVl59uuyUIgCYCDNBEgAGaCDBAEwEGaCLAAE0EGKCJAAM0EWCAJgIM0ESAAZoIMEATAQZoIsAATQQYoIkAAzQRYIAmAgzQRIABmmzqOeHotW/fvqytrXWP8bQOHTqUJNm5c2fzJMeO3bt3Z8+ePd1jsEAC/DyytraWGw/emidOOLl7lKfY9vCDSZJ7HnNKbYVtDz/QPQJL4LvleeaJE07OI2df3D3GU+y47TNJspKzPR+tfz05tlkDBmgiwABNBBigiQADNBFggCYCDNBEgAGaCDBAEwEGaCLAAE0EGKCJAAM0EWCAJgIM0ESAAZoIMEATAQZoIsAATQQYoIkAAzQRYIAmAgzQRIABmggwQBMBBmgiwABNBBigiQADNBFggCYCDNBEgAGaCDBAEwEGaCLAAE0EGKCJAAM0EWCAJgIM0ESAAZoIMEATAQZospQA79u3L/v27VvGrgC21CL7tX0hn/UIa2try9gNwJZbZL8sQQA0EWCAJgIM0ESAAZoIMEATAQZoIsAATQQYoIkAAzQRYIAmAgzQRIABmggwQBMBBmgiwABNBBigiQADNBFggCYCDNBEgAGaCDBAEwEGaCLAAE0EGKCJAAM0EWCAJgIM0ESAAZoIMEATAQZoIsAATQQYoIkAAzQRYIAmAgzQRIABmggwQBMBBmgiwABNBBigiQADNNm+jJ0cOnQojzzySPbu3buM3R2z1tbWctw3RvcYLMFxjz6UtbX/8j2zAtbW1rJjx46FfO5nvQdcVe+sqv1Vtf/w4cMLGQLghehZ7wGPMT6Y5INJct555x3V3a+dO3cmSa6++uqj+ePM9u7dmwP/cW/3GCzBky/57ux+zWm+Z1bAIh+FWAMGaCLAAE0EGKCJAAM0EWCAJgIM0ESAAZoIMEATAQZoIsAATQQYoIkAAzQRYIAmAgzQRIABmggwQBMBBmgiwABNBBigiQADNBFggCYCDNBEgAGaCDBAEwEGaCLAAE0EGKCJAAM0EWCAJgIM0ESAAZoIMEATAQZoIsAATQQYoIkAAzQRYIAmAgzQRIABmggwQBMBBmgiwABNti9jJ7t3717GbgC23CL7tZQA79mzZxm7Adhyi+yXJQiAJgIM0ESAAZoIMEATAQZoIsAATQQYoIkAAzQRYIAmAgzQRIABmggwQBMBBmgiwABNBBigiQADNBFggCYCDNBEgAGaCDBAEwEGaCLAAE0EGKCJAAM0EWCAJgIM0ESAAZoIMEATAQZoIsAATQQYoIkAAzQRYIAmAgzQRIABmggwQBMBBmgiwABNBBigiQADNNnePQCbs+3hB7Ljts90j/EU2x6+P0lWcrbno20PP5DktO4xWDABfh7ZvXt39wjf1qFDjydJdu4Uja1x2kofb7aGAD+P7Nmzp3sEYAtZAwZoIsAATQQYoIkAAzQRYIAmAgzQRIABmggwQBMBBmgiwABNBBigiQADNBFggCYCDNBEgAGaCDBAEwEGaCLAAE0EGKCJAAM0qTHGc//gqsNJvrzJfZyS5Gub/DPLsqqzmWtzVnWuZHVnM9fmfKdzvWqMceqRGzcV4KNRVfvHGOctdCdHaVVnM9fmrOpcyerOZq7NWdRcliAAmggwQJNlBPiDS9jH0VrV2cy1Oas6V7K6s5lrcxYy18LXgAF4epYgAJoIMECThQa4qi6qqturaq2q3rvIfT3LHB+qqvuq6uCGbSdX1bVV9cX59csb5nplVX2uqm6pqpurau8KzfaSqvp8Vd00z/a+efurq+r6+Zj+RVUd3zDbtqr6QlVdsyozzXPcWVX/VlU3VtX+edsqHMuXVdXHq+q2qrq1qi5ckbnOmr9W6y8PVdW7V2S298zn/cGq+uj8/bDl59nCAlxV25K8P8nbkpyT5PKqOmdR+3sWf5LkoiO2vTfJdWOMM5NcN99etseT/OoY45wkFyR51/w1WoXZHkvyljHG65Kcm+SiqrogyVVJfm+MsTvJfyZ5R8Nse5PcuuH2Ksy07ifGGOdu+D+jq3Asr07y12OMs5O8LtPXrn2uMcbt89fq3CRvTPJwkk91z1ZVO5P8SpLzxhivTbItyWVZxHk2xljIS5ILk3x2w+0rk1y5qP09h3l2JTm44fbtSU6f3z49ye1ds22Y6a+SvHXVZktyQpIbkvxwpt8G2v50x3hJs5yR6ZvyLUmuSVLdM22Y7c4kpxyxrfVYJjkpyZcy/8B9VeZ6mjl/Osk/r8JsSXYm+WqSk5Nsn8+zn1nEebbIJYj1v8S6u+Ztq+K0Mcbd89v3JDmtc5iq2pXk9Umuz4rMNj/UvzHJfUmuTfLvSb4+xnh8/pCOY/r7SX4tyZPz7VeswEzrRpK/qaoDVfXOeVv3sXx1ksNJPjwv2/xRVZ24AnMd6bIkH53fbp1tjHEoye8k+UqSu5M8mORAFnCe+SFckjH9k9b2//Gq6qVJPpHk3WOMhza+r3O2McYTY3p4eEaS85Oc3THHuqr62ST3jTEOdM7xDN48xnhDpmW3d1XVj218Z9Ox3J7kDUn+cIzx+iT/kyMe0q/A+X98krcn+diR7+uYbV5zviTTP17fl+TEPHUJc0ssMsCHkrxyw+0z5m2r4t6qOj1J5tf3dQxRVS/KFN8/G2N8cpVmWzfG+HqSz2V62PWyqto+v2vZx/RNSd5eVXcm+fNMyxBXN8/0f+Z7Thlj3JdpLfP89B/Lu5LcNca4fr798UxB7p5ro7cluWGMce98u3u2n0rypTHG4THGN5N8MtO5t+Xn2SID/C9Jzpx/cnh8pocYn17g/jbr00mumN++ItP661JVVSX54yS3jjF+d8VmO7WqXja/vSPT2vStmUJ8acdsY4wrxxhnjDF2ZTqf/m6M8YudM62rqhOr6rvW3860pnkwzcdyjHFPkq9W1Vnzpp9Mckv3XEe4PN9afkj6Z/tKkguq6oT5e3T9a7b159mCF7MvTnJHprXDX1/mQvoRc3w001rONzPdI3hHprXD65J8McnfJjm5Ya43Z3p49a9JbpxfLl6R2X4oyRfm2Q4m+Y15+2uSfD7JWqaHjC9uOqY/nuSaVZlpnuGm+eXm9fN9RY7luUn2z8fyL5O8fBXmmmc7Mcn9SU7asK19tiTvS3LbfO5/JMmLF3Ge+VVkgCZ+CAfQRIABmggwQBMBBmgiwABNBBigiQCzEqrq56pqVFXrrzvDMgkwq+LyJP80v4YXBAGm3Xwxojdn+g3Fy+Ztx1XVB+aLiF9bVZ+pqkvn972xqv5hvurYZ9evG/BtPvffV9VV88Xl76iqH52376qqf6yqG+aXH1nCXxX+n+3P/iGwcJdkumD4HVV1f1W9MdOVqHZlupj/92S6DsWH5osX7UtyyRjjcFX9QpLfTvLLz/D5t48xzq+qi5P8ZqaLrdyX5K1jjEer6sxMv65+3jN8DthyAswquDzTVc2S6Spnl2c6Nz82xngyyT1V9bn5/WcleW2Sa6frpGRbput8PJP1q8wdyBT1JHlRkj+oqnOTPJHk+7/zvwZsjgDTqqpOznRZyR+sqpEpqCPT5Ryf9o8kuXmMceEmdvPY/PqJfOucf0+SezM9Rc9xSR7d5OjwHbMGTLdLk3xkjPGqMcauMcYrMz2FzgNJfn5eCz4t09XPkunpak6tqguT6XrKVfUDR7Hfk5LcPd/D/qVM4YelEmC6XZ6n3tv9RJLvzXTp0FuS/Gmm56R7cIzxjUzRvqqqbsp0Cc+j+QHaB5JcMX+OszM9UwQslctRsrKq6qVjjP+uqldkug7rm8Z0gXE4JlgDZpVdMz8rx/FJfkt8Oda4B8wxoaren+l5uza6eozx4Y554LkQYIAmfggH0ESAAZoIMEATAQZo8r/yFHrWOA/BdQAAAABJRU5ErkJggg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": [],
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "atAMX5nWJ6Jt"
      },
      "source": [
        "5. Arbitrary imputation - Used on the basis of persons choice. No particular reason. We can the last of the distribution or least of the distribution ,based on the person's choice."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "r5gV2uRePXKU"
      },
      "source": [
        "# HANDLE CATEGORICAL MISSING VALUES\n",
        "1.FREQUENT CATEGORY IMPUTATION"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yLpJoQJfMOmp",
        "outputId": "1c78a02f-6bd8-4c2c-8e5f-8e30e596113a",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "import pandas as pd\n",
        "%matplotlib inline\n",
        "df=pd.read_csv('/content/train.csv.txt',usecols=['BsmtQual','FireplaceQu','GarageType','SalePrice'])\n",
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>BsmtQual</th>\n",
              "      <th>FireplaceQu</th>\n",
              "      <th>GarageType</th>\n",
              "      <th>SalePrice</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Gd</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>208500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>181500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>223500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>TA</td>\n",
              "      <td>Gd</td>\n",
              "      <td>Detchd</td>\n",
              "      <td>140000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>250000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  BsmtQual FireplaceQu GarageType  SalePrice\n",
              "0       Gd         NaN     Attchd     208500\n",
              "1       Gd          TA     Attchd     181500\n",
              "2       Gd          TA     Attchd     223500\n",
              "3       TA          Gd     Detchd     140000\n",
              "4       Gd          TA     Attchd     250000"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eXVdGY57Q3C1",
        "outputId": "4b3593ec-6e6d-4558-c378-bc39cd1a738f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "df.isnull().sum() # BsmtQual and GarageType  has less no of missing values. So replace with most frequent values."
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "BsmtQual        37\n",
              "FireplaceQu    690\n",
              "GarageType      81\n",
              "SalePrice        0\n",
              "dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "B2LyDfX6SR59",
        "outputId": "7089cdce-ad3d-47f6-a08b-d9050dc38d3e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 102
        }
      },
      "source": [
        "df.BsmtQual.value_counts()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TA    649\n",
              "Gd    618\n",
              "Ex    121\n",
              "Fa     35\n",
              "Name: BsmtQual, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 22
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MWUmz38JXBq8",
        "outputId": "d7fec8c1-ceaa-458f-c1c3-8c068a96907f",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "df.GarageType.value_counts()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Attchd     870\n",
              "Detchd     387\n",
              "BuiltIn     88\n",
              "Basment     19\n",
              "CarPort      9\n",
              "2Types       6\n",
              "Name: GarageType, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 23
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KbAs_RsxXEoC",
        "outputId": "813af6dc-e92b-4d73-8aca-b460d77c348c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "source": [
        "df.FireplaceQu.value_counts()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "Gd    380\n",
              "TA    313\n",
              "Fa     33\n",
              "Ex     24\n",
              "Po     20\n",
              "Name: FireplaceQu, dtype: int64"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 24
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hWa_RPecYvG-",
        "outputId": "782a9696-8756-4d4a-f159-2ef44a975761",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "df.FireplaceQu.value_counts().index[0]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Gd'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 25
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "kYbJGyF5cAKo",
        "outputId": "3b6811a7-4f31-4eda-ac07-728949eaea8b",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 35
        }
      },
      "source": [
        "df['GarageType'].mode()[0]"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            },
            "text/plain": [
              "'Attchd'"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 26
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "9kiA_2TuXaFl"
      },
      "source": [
        "def inpute_nan(df,variable):\n",
        "  most_frequent_value=df[variable].value_counts().index[0] # can also use mode\n",
        "  df[variable].fillna(most_frequent_value,inplace=True)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "OtD05DWyZBj-"
      },
      "source": [
        "for i in ['BsmtQual','FireplaceQu','GarageType']:\n",
        "  inpute_nan(df,i)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WM-I7VW9ZY4I",
        "outputId": "23145935-123f-4521-fb3f-374da496f69d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>BsmtQual</th>\n",
              "      <th>FireplaceQu</th>\n",
              "      <th>GarageType</th>\n",
              "      <th>SalePrice</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Gd</td>\n",
              "      <td>Gd</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>208500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>181500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>223500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>TA</td>\n",
              "      <td>Gd</td>\n",
              "      <td>Detchd</td>\n",
              "      <td>140000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>250000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  BsmtQual FireplaceQu GarageType  SalePrice\n",
              "0       Gd          Gd     Attchd     208500\n",
              "1       Gd          TA     Attchd     181500\n",
              "2       Gd          TA     Attchd     223500\n",
              "3       TA          Gd     Detchd     140000\n",
              "4       Gd          TA     Attchd     250000"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "zZJUK-5Uc_zd"
      },
      "source": [
        "Disadvantages- Not suitable for columns having more Nan values "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "t4csZ6hWekB1"
      },
      "source": [
        "2.Adding a variable to capture Nan (just used to display the importance of Nan)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Fx2XarRHeroo",
        "outputId": "1a47105b-d643-4417-dafb-f81f98e24259",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "import pandas as pd\n",
        "%matplotlib inline\n",
        "df=pd.read_csv('/content/train.csv.txt',usecols=['BsmtQual','FireplaceQu','GarageType','SalePrice'])\n",
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>BsmtQual</th>\n",
              "      <th>FireplaceQu</th>\n",
              "      <th>GarageType</th>\n",
              "      <th>SalePrice</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Gd</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>208500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>181500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>223500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>TA</td>\n",
              "      <td>Gd</td>\n",
              "      <td>Detchd</td>\n",
              "      <td>140000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>250000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  BsmtQual FireplaceQu GarageType  SalePrice\n",
              "0       Gd         NaN     Attchd     208500\n",
              "1       Gd          TA     Attchd     181500\n",
              "2       Gd          TA     Attchd     223500\n",
              "3       TA          Gd     Detchd     140000\n",
              "4       Gd          TA     Attchd     250000"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 30
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3EaLjlryeskd",
        "outputId": "2f044f0a-a14b-4bf7-9a20-be5d912fcf8c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "df['nan']=np.where(df['BsmtQual'].isnull(),1,0)\n",
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>BsmtQual</th>\n",
              "      <th>FireplaceQu</th>\n",
              "      <th>GarageType</th>\n",
              "      <th>SalePrice</th>\n",
              "      <th>nan</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Gd</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>208500</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>181500</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>223500</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>TA</td>\n",
              "      <td>Gd</td>\n",
              "      <td>Detchd</td>\n",
              "      <td>140000</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>250000</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  BsmtQual FireplaceQu GarageType  SalePrice  nan\n",
              "0       Gd         NaN     Attchd     208500    0\n",
              "1       Gd          TA     Attchd     181500    0\n",
              "2       Gd          TA     Attchd     223500    0\n",
              "3       TA          Gd     Detchd     140000    0\n",
              "4       Gd          TA     Attchd     250000    0"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 31
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "GYTLfKPIiaWg"
      },
      "source": [
        "3.Suppose if we have more frequent categories we replace Nan with a new category(most common technique)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "eYgk16tYi_2k",
        "outputId": "782bcad2-3048-4840-93ad-7b3b75466269",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "import pandas as pd\n",
        "%matplotlib inline\n",
        "df=pd.read_csv('/content/train.csv.txt',usecols=['BsmtQual','FireplaceQu','GarageType','SalePrice'])\n",
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>BsmtQual</th>\n",
              "      <th>FireplaceQu</th>\n",
              "      <th>GarageType</th>\n",
              "      <th>SalePrice</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Gd</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>208500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>181500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>223500</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>TA</td>\n",
              "      <td>Gd</td>\n",
              "      <td>Detchd</td>\n",
              "      <td>140000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>250000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  BsmtQual FireplaceQu GarageType  SalePrice\n",
              "0       Gd         NaN     Attchd     208500\n",
              "1       Gd          TA     Attchd     181500\n",
              "2       Gd          TA     Attchd     223500\n",
              "3       TA          Gd     Detchd     140000\n",
              "4       Gd          TA     Attchd     250000"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 32
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hTNrh7IsjGIc"
      },
      "source": [
        "def inpute_nan(df,variable):\n",
        "  df[variable+'_newvar']=np.where(df[variable].isnull(),'missing',df[variable])"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "KVYFJRUCjnKZ"
      },
      "source": [
        "for i in ['BsmtQual','FireplaceQu','GarageType']:\n",
        "  inpute_nan(df,i)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bBxc5R07jqhr",
        "outputId": "bafba6a9-ce79-46ee-83e4-1dfe4fa84dc4",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 204
        }
      },
      "source": [
        "df.head()"
      ],
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": [
              "<div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>BsmtQual</th>\n",
              "      <th>FireplaceQu</th>\n",
              "      <th>GarageType</th>\n",
              "      <th>SalePrice</th>\n",
              "      <th>BsmtQual_newvar</th>\n",
              "      <th>FireplaceQu_newvar</th>\n",
              "      <th>GarageType_newvar</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>Gd</td>\n",
              "      <td>NaN</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>208500</td>\n",
              "      <td>Gd</td>\n",
              "      <td>missing</td>\n",
              "      <td>Attchd</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>181500</td>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>223500</td>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>TA</td>\n",
              "      <td>Gd</td>\n",
              "      <td>Detchd</td>\n",
              "      <td>140000</td>\n",
              "      <td>TA</td>\n",
              "      <td>Gd</td>\n",
              "      <td>Detchd</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "      <td>250000</td>\n",
              "      <td>Gd</td>\n",
              "      <td>TA</td>\n",
              "      <td>Attchd</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>"
            ],
            "text/plain": [
              "  BsmtQual FireplaceQu  ... FireplaceQu_newvar  GarageType_newvar\n",
              "0       Gd         NaN  ...            missing             Attchd\n",
              "1       Gd          TA  ...                 TA             Attchd\n",
              "2       Gd          TA  ...                 TA             Attchd\n",
              "3       TA          Gd  ...                 Gd             Detchd\n",
              "4       Gd          TA  ...                 TA             Attchd\n",
              "\n",
              "[5 rows x 7 columns]"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 35
        }
      ]
    }
  ]
}