1_1_cah_extract_data_outperformers.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "94ae1e0b",
   "metadata": {
    "id": "94ae1e0b"
   },
   "source": [
    "### Extract data & train/test sets \n",
    "* pretrain data: all black + which + combinations\n",
    "    * Could supplemenet with more black/white from internet, and other jokes sources (reddit, jester) but overkill. \n",
    "    * Main goal is just to \"warm up\" a language/sentence model, generically. \n",
    "\n",
    "\n",
    "#### Check which pairs outperform their white cards baseline\n",
    "  * NOTE: This is underpowered! (unless we were to use a very high min # of picks) \n",
    "\n",
    "  * Save this as another dataset, without train/test split ? "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1fd1e26b",
   "metadata": {
    "id": "1fd1e26b"
   },
   "source": [
    "Each row of the file represents a white card that was given to a player in a round and how it performed. It also contains information about the round like how long it took and what black card was dealt.\n",
    "So columns that describe general round data will be exactly the same for all ten rows of that round. These rows are: fake_round_id, round_completion_seconds, round_skipped, black_card_text, and black_card_pick_num.\n",
    "But columns that describe a property of the white card will change over the ten rows of that round. These rows are: white_card_text, won, and winning_index.\n",
    "\n",
    "\n",
    "The columns:\n",
    "* fake_round_id: a unique integer for every round. This is to help you separate one round from another to find which 10 cards were presented in every round.\n",
    "* round_completion_seconds: the number of seconds it took for the user to pick a winner after being presented the cards for that round. You'll probably want to filter out users that decide too quickly by setting some minimum threshold for this. Some of these numbers can be insanely high as well since the user just left their browser open for hours or possibly days before picking a winner.\n",
    "* round_skipped: we have a red button in the top right of the lab page that says \"No Good Plays!\". If a user clicks this button, round_skipped will be \"True\" and no white card will be marked as a winner.\n",
    "* black_card_text: self explanatory.\n",
    "* black_card_pick_num: The number of white cards that the black card requires. Generally 1, but will be 2 for black cards like \"That's right, I killed _____. How, you ask? _____.\"\n",
    "* white_card_text: self explanatory.\n",
    "* won: True if this white card won the round.\n",
    "* winning_index: This is for storing the order that the winning card(s) were picked in. If the white card didn't win, this will just contain a null value (NULL). If the card won and the black card was a normal \"Pick 1\" this will always be 0. But if the black card was a \"Pick 2\", one of the winners will be 0 and the other will be 1. This is how you figure out which blank in pick 2 black cards was meant to be filled by which winning white card. The first blank is filled by the 0 and the second one by the 1."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa5e49f8",
   "metadata": {
    "id": "fa5e49f8"
   },
   "source": [
    "* sample groups (for test);\n",
    "\n",
    "* https://stackoverflow.com/questions/44007496/random-sampling-with-pandas-data-frame-disjoint-groups\n",
    "    ```\n",
    "    from sklearn.model_selection import GroupShuffleSplit\n",
    "\n",
    "    # Initialize the GroupShuffleSplit.\n",
    "    gss = GroupShuffleSplit(n_splits=1, test_size=0.5)\n",
    "\n",
    "    # Get the indexers for the split.\n",
    "    idx1, idx2 = next(gss.split(df, groups=df.ids))\n",
    "\n",
    "    # Get the split DataFrames.\n",
    "    df1, df2 = df.iloc[idx1], df.iloc[idx2]\n",
    "    ```\n",
    "\n",
    "* https://hippocampus-garden.com/pandas_group_sample/\n",
    "    ```\n",
    "    sampled_users = np.random.choice(df[\"user_id\"].unique(), 100)\n",
    "    df_sampled = df.groupby('user_id').filter(lambda x: x[\"user_id\"].values[0] in sampled_users)\n",
    "    df_sampled[\"user_id\"].nunique()\n",
    "    ```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "jEQaokJqxoes",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "jEQaokJqxoes",
    "outputId": "06469acb-93d0-4f32-c1a3-58798709302f"
   },
   "outputs": [],
   "source": [
    "# from google.colab import drive\n",
    "# drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "experienced-isolation",
   "metadata": {
    "id": "experienced-isolation"
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import seaborn as sns\n",
    "pd.set_option(\"display.precision\", 4)\n",
    "pd.set_option('display.width', 1024)\n",
    "pd.set_option('display.max_colwidth', None)\n",
    "%matplotlib inline \n",
    "from sklearn.linear_model import LogisticRegression, LogisticRegressionCV\n",
    "from sklearn.model_selection import train_test_split,GridSearchCV, cross_val_score, cross_val_predict\n",
    "from sklearn.preprocessing import OrdinalEncoder, LabelEncoder\n",
    "from sklearn.preprocessing import StandardScaler, MinMaxScaler\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.pipeline import make_pipeline\n",
    "from sklearn.metrics import classification_report, roc_auc_score  \n",
    "# https://stackoverflow.com/questions/53784971/how-to-disable-convergencewarning-using-sklearn\n",
    "from warnings import simplefilter\n",
    "from sklearn.exceptions import ConvergenceWarning\n",
    "simplefilter(\"ignore\", category=ConvergenceWarning)\n",
    "from sklearn.model_selection import GroupShuffleSplit\n",
    "\n",
    "ord_enc = OrdinalEncoder(handle_unknown=\"use_encoded_value\",unknown_value=-1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "8ef51172",
   "metadata": {
    "id": "8ef51172"
   },
   "outputs": [],
   "source": [
    "# from sentence_transformers import SentenceTransformer, util ## versions error? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6a0a6e7d",
   "metadata": {
    "id": "6a0a6e7d"
   },
   "outputs": [],
   "source": [
    "LOAD_CSV = False#False ## load raw data from csv instead of parquet (which has had some cols dropped)\n",
    "MIN_PAIR_FREQ = 4\n",
    "SAVE_TRAIN_DATA = False#True\n",
    "\n",
    "LOWERCASE_DATA = False\n",
    "DROP_DOUBLES = True ## drop white cards with pick 2 options ; and their blacks. (TODO: analyze interesting pairs)\n",
    "DROP_SKIPPED = True ## ignore rounds where skipped"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bridal-paris",
   "metadata": {
    "id": "bridal-paris"
   },
   "source": [
    "Load data\n",
    "\n",
    "* Note there are some near-duplicate cards (e.g. footballers - could minhash - https://github.com/chrisjmccormick/MinHash/blob/master/runMinHashExample.py , LSH/shingles : https://onestopdataanalysis.com/lsh/  (broken library?) , https://github.com/zyocum/dedup/blob/master/dedup.py ,\n",
    "\n",
    "`Datasketch` https://www.learndatasci.com/tutorials/building-recommendation-engine-locality-sensitive-hashing-lsh-python/ ,\n",
    "https://stackoverflow.com/questions/25114338/approximate-string-matching-using-lsh , \n",
    "\n",
    "SKLearn friendly?: http://ethen8181.github.io/machine-learning/recsys/content_based/lsh_text.html\n",
    "\n",
    "`dedupe` library - https://github.com/dedupeio/dedupe\n",
    "\n",
    "* Paraphrase mining using sentence bert - https://www.sbert.net/examples/applications/paraphrase-mining/README.html  (may be too semantic)\n",
    "\n",
    "* Example: \"10 football players with erections *barreling* towards you at full speed\" - 10 football players with erections **barrelling** towards you at full speed.\"\n",
    "\n",
    "Task modelling:\n",
    "* Ranking = most accurate\n",
    "* Regression/classification: can be done just over single cards, or between pairs. \n",
    "    * if pairs-wise: can use sentencebert/SNLI models (which assume pairs of sentences) - https://github.com/UKPLab/sentence-transformers\n",
    "    \n",
    "    \n",
    "Examples:\n",
    "\n",
    "* https://www.sbert.net/examples/training/sts/README.html#training-data\n",
    "* https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/nli/training_nli.py\n",
    "* https://github.com/UKPLab/sentence-transformers/blob/master/examples/training/nli/training_nli_v2.py\n",
    "\n",
    "\n",
    "* Keras example (not sentenceBert): https://keras.io/examples/nlp/semantic_similarity_with_bert/\n",
    "\n",
    "\n",
    "* Interpretability tool : https://github.com/cdpierse/transformers-interpret"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "sound-transcript",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "sound-transcript",
    "outputId": "29674835-a5a2-48e5-ad87-2445be6ec2bc"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2989550 rows\n",
      "173830 rounds skipped\n",
      "fake_round_id               298955\n",
      "round_completion_seconds      1909\n",
      "round_skipped                    2\n",
      "black_card_text                650\n",
      "black_card_pick_num              2\n",
      "white_card_text               2128\n",
      "won                              2\n",
      "ID_index                        10\n",
      "dtype: int64\n",
      "CPU times: total: 2.66 s\n",
      "Wall time: 2.14 s\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fake_round_id</th>\n",
       "      <th>round_completion_seconds</th>\n",
       "      <th>round_skipped</th>\n",
       "      <th>black_card_text</th>\n",
       "      <th>black_card_pick_num</th>\n",
       "      <th>white_card_text</th>\n",
       "      <th>won</th>\n",
       "      <th>ID_index</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>False</td>\n",
       "      <td>Hi MTV! My name is Kendra, I live in Malibu, I'm into _____, and I love to have a good time.</td>\n",
       "      <td>1</td>\n",
       "      <td>Going inside at some point because of the mosquitoes.</td>\n",
       "      <td>False</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>False</td>\n",
       "      <td>Hi MTV! My name is Kendra, I live in Malibu, I'm into _____, and I love to have a good time.</td>\n",
       "      <td>1</td>\n",
       "      <td>Being fat from noodles.</td>\n",
       "      <td>False</td>\n",
       "      <td>2</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>False</td>\n",
       "      <td>Hi MTV! My name is Kendra, I live in Malibu, I'm into _____, and I love to have a good time.</td>\n",
       "      <td>1</td>\n",
       "      <td>Letting this loser eat me out.</td>\n",
       "      <td>False</td>\n",
       "      <td>3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>False</td>\n",
       "      <td>Hi MTV! My name is Kendra, I live in Malibu, I'm into _____, and I love to have a good time.</td>\n",
       "      <td>1</td>\n",
       "      <td>That chicken from Popeyes.®</td>\n",
       "      <td>False</td>\n",
       "      <td>4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>1</td>\n",
       "      <td>24</td>\n",
       "      <td>False</td>\n",
       "      <td>Hi MTV! My name is Kendra, I live in Malibu, I'm into _____, and I love to have a good time.</td>\n",
       "      <td>1</td>\n",
       "      <td>A sorry excuse for a father.</td>\n",
       "      <td>False</td>\n",
       "      <td>5</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2989545</th>\n",
       "      <td>298955</td>\n",
       "      <td>7613</td>\n",
       "      <td>False</td>\n",
       "      <td>Oh my god! _____ killed Kenny!</td>\n",
       "      <td>1</td>\n",
       "      <td>Breastfeeding a ten-year-old.</td>\n",
       "      <td>False</td>\n",
       "      <td>6</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2989546</th>\n",
       "      <td>298955</td>\n",
       "      <td>7613</td>\n",
       "      <td>False</td>\n",
       "      <td>Oh my god! _____ killed Kenny!</td>\n",
       "      <td>1</td>\n",
       "      <td>Happy daddies with happy sandals.</td>\n",
       "      <td>False</td>\n",
       "      <td>7</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2989547</th>\n",
       "      <td>298955</td>\n",
       "      <td>7613</td>\n",
       "      <td>False</td>\n",
       "      <td>Oh my god! _____ killed Kenny!</td>\n",
       "      <td>1</td>\n",
       "      <td>Jerking off to a 10-second RealMedia clip.</td>\n",
       "      <td>False</td>\n",
       "      <td>8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2989548</th>\n",
       "      <td>298955</td>\n",
       "      <td>7613</td>\n",
       "      <td>False</td>\n",
       "      <td>Oh my god! _____ killed Kenny!</td>\n",
       "      <td>1</td>\n",
       "      <td>Getting naked and watching Nickelodeon.</td>\n",
       "      <td>False</td>\n",
       "      <td>9</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2989549</th>\n",
       "      <td>298955</td>\n",
       "      <td>7613</td>\n",
       "      <td>False</td>\n",
       "      <td>Oh my god! _____ killed Kenny!</td>\n",
       "      <td>1</td>\n",
       "      <td>Awesome pictures of planets and stuff.</td>\n",
       "      <td>False</td>\n",
       "      <td>10</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>2815720 rows × 8 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "         fake_round_id  round_completion_seconds  round_skipped                                                                               black_card_text  black_card_pick_num                                        white_card_text    won  ID_index\n",
       "0                    1                        24          False  Hi MTV! My name is Kendra, I live in Malibu, I'm into _____, and I love to have a good time.                    1  Going inside at some point because of the mosquitoes.  False         1\n",
       "1                    1                        24          False  Hi MTV! My name is Kendra, I live in Malibu, I'm into _____, and I love to have a good time.                    1                                Being fat from noodles.  False         2\n",
       "2                    1                        24          False  Hi MTV! My name is Kendra, I live in Malibu, I'm into _____, and I love to have a good time.                    1                         Letting this loser eat me out.  False         3\n",
       "3                    1                        24          False  Hi MTV! My name is Kendra, I live in Malibu, I'm into _____, and I love to have a good time.                    1                            That chicken from Popeyes.®  False         4\n",
       "4                    1                        24          False  Hi MTV! My name is Kendra, I live in Malibu, I'm into _____, and I love to have a good time.                    1                           A sorry excuse for a father.  False         5\n",
       "...                ...                       ...            ...                                                                                           ...                  ...                                                    ...    ...       ...\n",
       "2989545         298955                      7613          False                                                                Oh my god! _____ killed Kenny!                    1                          Breastfeeding a ten-year-old.  False         6\n",
       "2989546         298955                      7613          False                                                                Oh my god! _____ killed Kenny!                    1                      Happy daddies with happy sandals.  False         7\n",
       "2989547         298955                      7613          False                                                                Oh my god! _____ killed Kenny!                    1             Jerking off to a 10-second RealMedia clip.  False         8\n",
       "2989548         298955                      7613          False                                                                Oh my god! _____ killed Kenny!                    1                Getting naked and watching Nickelodeon.  False         9\n",
       "2989549         298955                      7613          False                                                                Oh my god! _____ killed Kenny!                    1                 Awesome pictures of planets and stuff.  False        10\n",
       "\n",
       "[2815720 rows x 8 columns]"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%%time\n",
    "#### Original processing of data, before I saved it as a parquet for faster subsequent loading: \n",
    "if LOAD_CSV:\n",
    "    df = pd.read_csv(DATA_PATH)\n",
    "    \n",
    "    ## remove punct/newlines\n",
    "    df[\"white_card_text\"] = df[\"white_card_text\"].replace(\"\\n\", \" \",regex=True).replace(\"\\t\", \" \",regex=True) # .replace(\"\\r\", \"\")\n",
    "    df[\"black_card_text\"] = df[\"black_card_text\"].replace(\"\\n\", \" \",regex=True).replace(\"\\t\", \" \",regex=True) # .replace(\"\\r\", \"\")\n",
    "\n",
    "    ## add number within group - may not actually be the read order shown on screen!! \n",
    "    df['ID_index'] = df.groupby('fake_round_id').cumcount() + 1\n",
    "\n",
    "    print(df[['black_card_text', 'black_card_pick_num', 'white_card_text',\"fake_round_id\"]].nunique())\n",
    "    print(df.describe())\n",
    "    \n",
    "    ## save to disk in compressed, binary format\n",
    "    df.drop([\"winning_index\"],axis=1).to_parquet('cah_lab_data.parquet')\n",
    "#     df.drop([\"winning_index\"],axis=1).to_feather('cah_lab_data.feather')\n",
    "    \n",
    "else:  # load data saved in above code previously\n",
    "    df = pd.read_parquet('cah_lab_data.parquet')\n",
    "#     df= pd.read_feather('cah_lab_data.feather')\n",
    "\n",
    "print(df.shape[0], \"rows\")\n",
    "print(df.loc[df[\"round_skipped\"]].shape[0], \"rounds skipped\")\n",
    "print(df.nunique())\n",
    "if DROP_SKIPPED:\n",
    "    df = df.loc[~df[\"round_skipped\"]]\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a2080567",
   "metadata": {
    "id": "a2080567"
   },
   "source": [
    "#### normalize \\____ 's \n",
    "* Rememebr to do this if evaluating\n",
    "* May not be needed!\n",
    "    * We also do this for consistency when doing find and replace , and for cases where there's no ___ in the text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e89b707",
   "metadata": {
    "id": "3e89b707"
   },
   "outputs": [],
   "source": [
    "df[\"has_\"] = df[\"black_card_text\"].str.contains(\"_{2,}\")\n",
    "### filter out cases with 2 blanks to fill ? \n",
    "\n",
    "df['black_card_text'] = df['black_card_text'].str.replace(\"_{2,}\",\"__\")\n",
    "## https://stackoverflow.com/questions/47696401/replace-character-of-column-value-with-string-from-another-column-in-pandas\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea306b22",
   "metadata": {
    "id": "ea306b22"
   },
   "source": [
    "### Pick 2s\n",
    "* Keep only subset for free text, and self join is slow.... \n",
    "* Keep only sample of combinations, as otherwise we get huuuge # combinations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9078dcc3",
   "metadata": {
    "id": "9078dcc3"
   },
   "outputs": [],
   "source": [
    "df_doubles = df.loc[df[\"black_card_pick_num\"]>1].filter(['black_card_text', 'white_card_text',\"won\",\"fake_round_id\"],axis=1).drop_duplicates().copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "15c70205",
   "metadata": {
    "id": "15c70205",
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "df = df.loc[df[\"black_card_pick_num\"]<2] ## keep non doubles only\n",
    "\n",
    "# df[[\"id_black\",\"id_white\"]] = ord_enc.fit_transform(df[[\"black_card_text\",\"white_card_text\"]])\n",
    "df[\"id_white\"] = ord_enc.fit_transform(df[\"white_card_text\"].values.reshape(-1, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "709907b7",
   "metadata": {
    "id": "709907b7"
   },
   "source": [
    "## df with all texts from data\n",
    "white, black and combinations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "19f09106",
   "metadata": {
    "id": "19f09106"
   },
   "outputs": [],
   "source": [
    "# df_text = pd.concat([df[\"black_card_text\"],\n",
    "#                        df[\"white_card_text\"],\n",
    "#                     df_doubles['text'],df_doubles['black_card_text']]).drop_duplicates()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c8fc04e2",
   "metadata": {
    "id": "c8fc04e2"
   },
   "source": [
    "Fill in \"punchlines\" into blanks (___) (or add at end of text for cards without ___)\n",
    "\n",
    "* NOTE: Previously I had tried a model with black and white text as 2 prompts (which meant the model needed to learn to fill in blanks); and a model of just the filled in answer (1 col). \n",
    "* Unclear what is \"easier\" to learn/model\n",
    "\n",
    "* Note this doesn't handle cases with double blanks (it will just input the same punchline twice)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fb471154",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 902
    },
    "id": "fb471154",
    "outputId": "43163df9-d7d8-4902-d101-4e339e85d83a"
   },
   "outputs": [],
   "source": [
    "df[\"has_\"] = df[\"black_card_text\"].str.contains(\"_{2,}\",regex=True)\n",
    "\n",
    "df['black_card_text'] = df['black_card_text'].str.replace(\"_{2,}\",\"__\",regex=True)\n",
    "## https://stackoverflow.com/questions/47696401/replace-character-of-column-value-with-string-from-another-column-in-pandas\n",
    "## https://datascience.stackexchange.com/questions/39345/how-to-replace-a-part-string-value-of-a-column-using-another-column\n",
    "# Remove characters from one column based on string of another column\n",
    "\n",
    "## remove \".\" from end of text\n",
    "df[\"white_card_text\"] = df[\"white_card_text\"].str.replace(\"\\.$\", '',regex=True)\n",
    "df['text'] = df.apply(lambda x:x['black_card_text'].replace(\"__\", x['white_card_text']), axis=1)\n",
    "\n",
    "## add answer at end of text, in cases where no ___ in black card\n",
    "df.loc[(df[\"has_\"]==False),\"text\"] = df[\"text\"] +\" \" + df[\"white_card_text\"] #+ \".\"\n",
    "\n",
    "df['text'] = df['text'].str.replace(\"..\",\".\",regex=False) ## fix double dots. Still levaes extra punct.. \n",
    "\n",
    "display(df)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6e8d5b6f",
   "metadata": {
    "id": "6e8d5b6f"
   },
   "source": [
    "##### Fit prior odds / encoder\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "3e3b036c",
   "metadata": {
    "id": "3e3b036c"
   },
   "outputs": [],
   "source": [
    "assert df[\"won\"].isna().sum()==0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0d013408",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "0d013408",
    "outputId": "14847505-0556-429f-c11f-e7bfde27954f"
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.1"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df[\"won\"].mean().round(2)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6373271e",
   "metadata": {
    "id": "6373271e"
   },
   "source": [
    "### check white prior as baseline model\n",
    "\n",
    "* 21% win rate by picking most popular answer, on FILTERED Data\n",
    "* On _full_ 3 million games (including different outcomes for same pair, i.e no filtering by # picks), _it does not better than random! _"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "bGlFJTJOytcz",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "bGlFJTJOytcz",
    "outputId": "e9c8d101-3521-44a2-9a56-beac2114466b"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Prior Acc: 0.20897578443074608\n"
     ]
    }
   ],
   "source": [
    "df_white_prior = df.groupby([\"white_card_text\"], as_index=False)[\"won\"].mean().rename(columns={\"won\":\"white_prior\"}).set_index(\"white_card_text\")\n",
    "df = df.join(df_white_prior,on=\"white_card_text\",how=\"left\")\n",
    "prior = df[\"white_prior\"].mean()\n",
    "df[\"white_prior\"] = df[\"white_prior\"].fillna(prior)\n",
    "\n",
    "print(\"Prior Acc:\",df.sort_values(\"white_prior\",ascending=False).groupby(\"fake_round_id\").head(1)[\"won\"].mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "rHw-lXIxkGgF",
   "metadata": {
    "id": "rHw-lXIxkGgF"
   },
   "source": [
    "### Filter (twice) by min pair count\n",
    "* Also later by higher thresh"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "TJApvDhNkEjL",
   "metadata": {
    "id": "TJApvDhNkEjL"
   },
   "outputs": [],
   "source": [
    "df[\"pair_count\"] = df.groupby(\"text\")[\"won\"].transform(\"count\") ## can be used to filter sentences occurring less than k times\n",
    "\n",
    "df[\"white_count\"] = df.groupby(\"white_card_text\")[\"won\"].transform(\"count\")\n",
    "df[\"black_count\"] = df.groupby(\"black_card_text\")[\"won\"].transform(\"count\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "f7GcJISokFS0",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "f7GcJISokFS0",
    "outputId": "af84f7fb-2cdc-4593-da62-c38d303fb1b3"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1949028\n"
     ]
    }
   ],
   "source": [
    "int(df.shape[0])\n",
    "df = df.loc[df[\"pair_count\"]>=3] # 3?  +- could filter rounds with no win?  (code in previous notebook)\n",
    "print(df.shape[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e457242",
   "metadata": {
    "id": "2e457242"
   },
   "source": [
    "### get min count/cooccurences of sentence pairs & White\n",
    "* Filter train +_ TEST for min K occurrences of a pair\n",
    "* Filter games/rounds where positive was removed ? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "d13824a2",
   "metadata": {
    "id": "d13824a2"
   },
   "outputs": [],
   "source": [
    "df[\"sum_won\"] = df.groupby(\"text\")[\"won\"].transform(\"sum\")\n",
    "df[\"pair_prior\"] = df.groupby(\"text\")[\"won\"].transform(\"mean\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "uVJch0bMh1Zr",
   "metadata": {
    "id": "uVJch0bMh1Zr"
   },
   "source": [
    "### some top picks/combs"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "3Yz4pSziiCVs",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "3Yz4pSziiCVs",
    "outputId": "eab2439b-1802-4969-ed1b-e44a1b6c52b5"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fake_round_id</th>\n",
       "      <th>round_completion_seconds</th>\n",
       "      <th>round_skipped</th>\n",
       "      <th>black_card_text</th>\n",
       "      <th>black_card_pick_num</th>\n",
       "      <th>white_card_text</th>\n",
       "      <th>won</th>\n",
       "      <th>ID_index</th>\n",
       "      <th>has_</th>\n",
       "      <th>id_white</th>\n",
       "      <th>text</th>\n",
       "      <th>white_prior</th>\n",
       "      <th>pair_count</th>\n",
       "      <th>white_count</th>\n",
       "      <th>black_count</th>\n",
       "      <th>sum_won</th>\n",
       "      <th>pair_prior</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>663810</th>\n",
       "      <td>66382</td>\n",
       "      <td>18</td>\n",
       "      <td>False</td>\n",
       "      <td>What has been making life difficult at the nudist colony?</td>\n",
       "      <td>1</td>\n",
       "      <td>Gay thoughts</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "      <td>844.0</td>\n",
       "      <td>What has been making life difficult at the nudist colony? Gay thoughts</td>\n",
       "      <td>0.2609</td>\n",
       "      <td>3</td>\n",
       "      <td>1234</td>\n",
       "      <td>5870</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>673916</th>\n",
       "      <td>67392</td>\n",
       "      <td>4</td>\n",
       "      <td>False</td>\n",
       "      <td>Match-maker, match-maker, make me a match. Find me __.</td>\n",
       "      <td>1</td>\n",
       "      <td>Thick Ukrainian ass</td>\n",
       "      <td>True</td>\n",
       "      <td>7</td>\n",
       "      <td>True</td>\n",
       "      <td>1940.0</td>\n",
       "      <td>Match-maker, match-maker, make me a match. Find me Thick Ukrainian ass.</td>\n",
       "      <td>0.1916</td>\n",
       "      <td>4</td>\n",
       "      <td>1216</td>\n",
       "      <td>6010</td>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>353603</th>\n",
       "      <td>35361</td>\n",
       "      <td>19</td>\n",
       "      <td>False</td>\n",
       "      <td>And today's soup is Cream of __.</td>\n",
       "      <td>1</td>\n",
       "      <td>Capitalism</td>\n",
       "      <td>True</td>\n",
       "      <td>4</td>\n",
       "      <td>True</td>\n",
       "      <td>518.0</td>\n",
       "      <td>And today's soup is Cream of Capitalism.</td>\n",
       "      <td>0.1837</td>\n",
       "      <td>3</td>\n",
       "      <td>1339</td>\n",
       "      <td>6280</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2961177</th>\n",
       "      <td>296118</td>\n",
       "      <td>10</td>\n",
       "      <td>False</td>\n",
       "      <td>Well if you'll excuse me, gentlemen, I have a date with __.</td>\n",
       "      <td>1</td>\n",
       "      <td>A horse with no legs</td>\n",
       "      <td>True</td>\n",
       "      <td>8</td>\n",
       "      <td>True</td>\n",
       "      <td>139.0</td>\n",
       "      <td>Well if you'll excuse me, gentlemen, I have a date with A horse with no legs.</td>\n",
       "      <td>0.1064</td>\n",
       "      <td>3</td>\n",
       "      <td>1241</td>\n",
       "      <td>6480</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2539524</th>\n",
       "      <td>253953</td>\n",
       "      <td>8</td>\n",
       "      <td>False</td>\n",
       "      <td>Match-maker, match-maker, make me a match. Find me __.</td>\n",
       "      <td>1</td>\n",
       "      <td>German dungeon porn</td>\n",
       "      <td>True</td>\n",
       "      <td>5</td>\n",
       "      <td>True</td>\n",
       "      <td>854.0</td>\n",
       "      <td>Match-maker, match-maker, make me a match. Find me German dungeon porn.</td>\n",
       "      <td>0.2520</td>\n",
       "      <td>4</td>\n",
       "      <td>1246</td>\n",
       "      <td>6010</td>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1064028</th>\n",
       "      <td>106403</td>\n",
       "      <td>18</td>\n",
       "      <td>False</td>\n",
       "      <td>What has my life become? I'm jerking off to a video called \"co-ed absolutely destroyed by __.\"</td>\n",
       "      <td>1</td>\n",
       "      <td>Cancer</td>\n",
       "      <td>True</td>\n",
       "      <td>9</td>\n",
       "      <td>True</td>\n",
       "      <td>517.0</td>\n",
       "      <td>What has my life become? I'm jerking off to a video called \"co-ed absolutely destroyed by Cancer.\"</td>\n",
       "      <td>0.1879</td>\n",
       "      <td>3</td>\n",
       "      <td>1288</td>\n",
       "      <td>6000</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1516922</th>\n",
       "      <td>151693</td>\n",
       "      <td>6</td>\n",
       "      <td>False</td>\n",
       "      <td>Click Here for __!!!</td>\n",
       "      <td>1</td>\n",
       "      <td>Hillary Clinton's emails</td>\n",
       "      <td>True</td>\n",
       "      <td>3</td>\n",
       "      <td>True</td>\n",
       "      <td>990.0</td>\n",
       "      <td>Click Here for Hillary Clinton's emails!!!</td>\n",
       "      <td>0.1279</td>\n",
       "      <td>3</td>\n",
       "      <td>1204</td>\n",
       "      <td>6430</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>282237</th>\n",
       "      <td>28224</td>\n",
       "      <td>8</td>\n",
       "      <td>False</td>\n",
       "      <td>I'm no doctor, but I'm pretty sure what you're suffering from is called \"__.\"</td>\n",
       "      <td>1</td>\n",
       "      <td>The last season of Game of Thrones</td>\n",
       "      <td>True</td>\n",
       "      <td>8</td>\n",
       "      <td>True</td>\n",
       "      <td>1852.0</td>\n",
       "      <td>I'm no doctor, but I'm pretty sure what you're suffering from is called \"The last season of Game of Thrones.\"</td>\n",
       "      <td>0.1017</td>\n",
       "      <td>3</td>\n",
       "      <td>1318</td>\n",
       "      <td>6280</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>353406</th>\n",
       "      <td>35341</td>\n",
       "      <td>15</td>\n",
       "      <td>False</td>\n",
       "      <td>IF you like __, YOU MIGHT BE A REDNECK.</td>\n",
       "      <td>1</td>\n",
       "      <td>Our dumb asshole president</td>\n",
       "      <td>True</td>\n",
       "      <td>7</td>\n",
       "      <td>True</td>\n",
       "      <td>1360.0</td>\n",
       "      <td>IF you like Our dumb asshole president, YOU MIGHT BE A REDNECK.</td>\n",
       "      <td>0.1628</td>\n",
       "      <td>4</td>\n",
       "      <td>1278</td>\n",
       "      <td>5950</td>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1431812</th>\n",
       "      <td>143182</td>\n",
       "      <td>9</td>\n",
       "      <td>False</td>\n",
       "      <td>I'm just gonna stay in tonight. You know, Netflix and __.</td>\n",
       "      <td>1</td>\n",
       "      <td>Fisting</td>\n",
       "      <td>True</td>\n",
       "      <td>3</td>\n",
       "      <td>True</td>\n",
       "      <td>790.0</td>\n",
       "      <td>I'm just gonna stay in tonight. You know, Netflix and Fisting.</td>\n",
       "      <td>0.2264</td>\n",
       "      <td>4</td>\n",
       "      <td>1272</td>\n",
       "      <td>6570</td>\n",
       "      <td>4</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>33625</th>\n",
       "      <td>3363</td>\n",
       "      <td>20</td>\n",
       "      <td>False</td>\n",
       "      <td>All classes today are canceled due to __.</td>\n",
       "      <td>1</td>\n",
       "      <td>The bomb strapped to my chest</td>\n",
       "      <td>True</td>\n",
       "      <td>6</td>\n",
       "      <td>True</td>\n",
       "      <td>1786.0</td>\n",
       "      <td>All classes today are canceled due to The bomb strapped to my chest.</td>\n",
       "      <td>0.1191</td>\n",
       "      <td>3</td>\n",
       "      <td>1293</td>\n",
       "      <td>5890</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2227288</th>\n",
       "      <td>222729</td>\n",
       "      <td>8</td>\n",
       "      <td>False</td>\n",
       "      <td>This holiday season, Tim Allen must overcome his fear of __ to save Christmas.</td>\n",
       "      <td>1</td>\n",
       "      <td>Capitalism</td>\n",
       "      <td>True</td>\n",
       "      <td>9</td>\n",
       "      <td>True</td>\n",
       "      <td>518.0</td>\n",
       "      <td>This holiday season, Tim Allen must overcome his fear of Capitalism to save Christmas.</td>\n",
       "      <td>0.1837</td>\n",
       "      <td>3</td>\n",
       "      <td>1339</td>\n",
       "      <td>5640</td>\n",
       "      <td>3</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         fake_round_id  round_completion_seconds  round_skipped                                                                                 black_card_text  black_card_pick_num                     white_card_text   won  ID_index   has_  id_white                                                                                                           text  white_prior  pair_count  white_count  black_count  sum_won  pair_prior\n",
       "663810           66382                        18          False                                       What has been making life difficult at the nudist colony?                    1                        Gay thoughts  True         1  False     844.0                                         What has been making life difficult at the nudist colony? Gay thoughts       0.2609           3         1234         5870        3         1.0\n",
       "673916           67392                         4          False                                          Match-maker, match-maker, make me a match. Find me __.                    1                 Thick Ukrainian ass  True         7   True    1940.0                                        Match-maker, match-maker, make me a match. Find me Thick Ukrainian ass.       0.1916           4         1216         6010        4         1.0\n",
       "353603           35361                        19          False                                                                And today's soup is Cream of __.                    1                          Capitalism  True         4   True     518.0                                                                       And today's soup is Cream of Capitalism.       0.1837           3         1339         6280        3         1.0\n",
       "2961177         296118                        10          False                                     Well if you'll excuse me, gentlemen, I have a date with __.                    1                A horse with no legs  True         8   True     139.0                                  Well if you'll excuse me, gentlemen, I have a date with A horse with no legs.       0.1064           3         1241         6480        3         1.0\n",
       "2539524         253953                         8          False                                          Match-maker, match-maker, make me a match. Find me __.                    1                 German dungeon porn  True         5   True     854.0                                        Match-maker, match-maker, make me a match. Find me German dungeon porn.       0.2520           4         1246         6010        4         1.0\n",
       "1064028         106403                        18          False  What has my life become? I'm jerking off to a video called \"co-ed absolutely destroyed by __.\"                    1                              Cancer  True         9   True     517.0             What has my life become? I'm jerking off to a video called \"co-ed absolutely destroyed by Cancer.\"       0.1879           3         1288         6000        3         1.0\n",
       "1516922         151693                         6          False                                                                            Click Here for __!!!                    1            Hillary Clinton's emails  True         3   True     990.0                                                                     Click Here for Hillary Clinton's emails!!!       0.1279           3         1204         6430        3         1.0\n",
       "282237           28224                         8          False                   I'm no doctor, but I'm pretty sure what you're suffering from is called \"__.\"                    1  The last season of Game of Thrones  True         8   True    1852.0  I'm no doctor, but I'm pretty sure what you're suffering from is called \"The last season of Game of Thrones.\"       0.1017           3         1318         6280        3         1.0\n",
       "353406           35341                        15          False                                                         IF you like __, YOU MIGHT BE A REDNECK.                    1          Our dumb asshole president  True         7   True    1360.0                                                IF you like Our dumb asshole president, YOU MIGHT BE A REDNECK.       0.1628           4         1278         5950        4         1.0\n",
       "1431812         143182                         9          False                                       I'm just gonna stay in tonight. You know, Netflix and __.                    1                             Fisting  True         3   True     790.0                                                 I'm just gonna stay in tonight. You know, Netflix and Fisting.       0.2264           4         1272         6570        4         1.0\n",
       "33625             3363                        20          False                                                       All classes today are canceled due to __.                    1       The bomb strapped to my chest  True         6   True    1786.0                                           All classes today are canceled due to The bomb strapped to my chest.       0.1191           3         1293         5890        3         1.0\n",
       "2227288         222729                         8          False                  This holiday season, Tim Allen must overcome his fear of __ to save Christmas.                    1                          Capitalism  True         9   True     518.0                         This holiday season, Tim Allen must overcome his fear of Capitalism to save Christmas.       0.1837           3         1339         5640        3         1.0"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sort_values(by=[\"pair_prior\"],ascending=False).head(12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "wSSSSKkyiDiw",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000
    },
    "id": "wSSSSKkyiDiw",
    "outputId": "10a2fe12-62af-463c-fa70-548acf32ec06"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>fake_round_id</th>\n",
       "      <th>round_completion_seconds</th>\n",
       "      <th>round_skipped</th>\n",
       "      <th>black_card_text</th>\n",
       "      <th>black_card_pick_num</th>\n",
       "      <th>white_card_text</th>\n",
       "      <th>won</th>\n",
       "      <th>ID_index</th>\n",
       "      <th>has_</th>\n",
       "      <th>id_white</th>\n",
       "      <th>text</th>\n",
       "      <th>white_prior</th>\n",
       "      <th>pair_count</th>\n",
       "      <th>white_count</th>\n",
       "      <th>black_count</th>\n",
       "      <th>sum_won</th>\n",
       "      <th>pair_prior</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1440673</th>\n",
       "      <td>144068</td>\n",
       "      <td>11</td>\n",
       "      <td>False</td>\n",
       "      <td>The secret to a lasting marriage is communication, communication, and __.</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>True</td>\n",
       "      <td>4</td>\n",
       "      <td>True</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>The secret to a lasting marriage is communication, communication, and Sluts, whores, and bitches.</td>\n",
       "      <td>0.275</td>\n",
       "      <td>7</td>\n",
       "      <td>1302</td>\n",
       "      <td>6070</td>\n",
       "      <td>3</td>\n",
       "      <td>0.4286</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>744071</th>\n",
       "      <td>74408</td>\n",
       "      <td>63</td>\n",
       "      <td>False</td>\n",
       "      <td>In what's being hailed as a major breakthrough, scientists have synthesized __ in the lab.</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>True</td>\n",
       "      <td>2</td>\n",
       "      <td>True</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>In what's being hailed as a major breakthrough, scientists have synthesized Sluts, whores, and bitches in the lab.</td>\n",
       "      <td>0.275</td>\n",
       "      <td>4</td>\n",
       "      <td>1302</td>\n",
       "      <td>5840</td>\n",
       "      <td>2</td>\n",
       "      <td>0.5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>880140</th>\n",
       "      <td>88015</td>\n",
       "      <td>20</td>\n",
       "      <td>False</td>\n",
       "      <td>Why my mustache smell so bad?</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>True</td>\n",
       "      <td>1</td>\n",
       "      <td>False</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>Why my mustache smell so bad? Sluts, whores, and bitches</td>\n",
       "      <td>0.275</td>\n",
       "      <td>4</td>\n",
       "      <td>1302</td>\n",
       "      <td>5750</td>\n",
       "      <td>1</td>\n",
       "      <td>0.2500</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1901136</th>\n",
       "      <td>190114</td>\n",
       "      <td>52</td>\n",
       "      <td>False</td>\n",
       "      <td>What makes life worth living?</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>False</td>\n",
       "      <td>7</td>\n",
       "      <td>False</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>What makes life worth living? Sluts, whores, and bitches</td>\n",
       "      <td>0.275</td>\n",
       "      <td>5</td>\n",
       "      <td>1302</td>\n",
       "      <td>5900</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2609031</th>\n",
       "      <td>260904</td>\n",
       "      <td>10</td>\n",
       "      <td>False</td>\n",
       "      <td>To prepare for his upcoming role, Daniel Day-Lewis immersed himself in the world of __.</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>False</td>\n",
       "      <td>2</td>\n",
       "      <td>True</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>To prepare for his upcoming role, Daniel Day-Lewis immersed himself in the world of Sluts, whores, and bitches.</td>\n",
       "      <td>0.275</td>\n",
       "      <td>3</td>\n",
       "      <td>1302</td>\n",
       "      <td>6110</td>\n",
       "      <td>2</td>\n",
       "      <td>0.6667</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1780039</th>\n",
       "      <td>178004</td>\n",
       "      <td>30</td>\n",
       "      <td>False</td>\n",
       "      <td>What made my first kiss so awkward?</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>True</td>\n",
       "      <td>10</td>\n",
       "      <td>False</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>What made my first kiss so awkward? Sluts, whores, and bitches</td>\n",
       "      <td>0.275</td>\n",
       "      <td>5</td>\n",
       "      <td>1302</td>\n",
       "      <td>6900</td>\n",
       "      <td>1</td>\n",
       "      <td>0.2000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>578031</th>\n",
       "      <td>57804</td>\n",
       "      <td>10</td>\n",
       "      <td>False</td>\n",
       "      <td>Welcome to Señor Frog's! Would you like to try our signature cocktail, \"__ on the Beach\"?</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>False</td>\n",
       "      <td>2</td>\n",
       "      <td>True</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>Welcome to Señor Frog's! Would you like to try our signature cocktail, \"Sluts, whores, and bitches on the Beach\"?</td>\n",
       "      <td>0.275</td>\n",
       "      <td>3</td>\n",
       "      <td>1302</td>\n",
       "      <td>6660</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>687974</th>\n",
       "      <td>68798</td>\n",
       "      <td>26</td>\n",
       "      <td>False</td>\n",
       "      <td>Why do I hurt all over?</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>False</td>\n",
       "      <td>5</td>\n",
       "      <td>False</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>Why do I hurt all over? Sluts, whores, and bitches</td>\n",
       "      <td>0.275</td>\n",
       "      <td>3</td>\n",
       "      <td>1302</td>\n",
       "      <td>5940</td>\n",
       "      <td>1</td>\n",
       "      <td>0.3333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2167747</th>\n",
       "      <td>216775</td>\n",
       "      <td>11</td>\n",
       "      <td>False</td>\n",
       "      <td>We do not shake with our left hands in this country. That is the hand we use for __.</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>True</td>\n",
       "      <td>8</td>\n",
       "      <td>True</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>We do not shake with our left hands in this country. That is the hand we use for Sluts, whores, and bitches.</td>\n",
       "      <td>0.275</td>\n",
       "      <td>4</td>\n",
       "      <td>1302</td>\n",
       "      <td>6190</td>\n",
       "      <td>2</td>\n",
       "      <td>0.5000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>823072</th>\n",
       "      <td>82308</td>\n",
       "      <td>6</td>\n",
       "      <td>False</td>\n",
       "      <td>Just once, I'd like to hear you say \"Thanks, Mom. Thanks for __.\"</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>False</td>\n",
       "      <td>3</td>\n",
       "      <td>True</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>Just once, I'd like to hear you say \"Thanks, Mom. Thanks for Sluts, whores, and bitches.\"</td>\n",
       "      <td>0.275</td>\n",
       "      <td>4</td>\n",
       "      <td>1302</td>\n",
       "      <td>6180</td>\n",
       "      <td>0</td>\n",
       "      <td>0.0000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2676252</th>\n",
       "      <td>267626</td>\n",
       "      <td>8</td>\n",
       "      <td>False</td>\n",
       "      <td>Uh, hey guys, I know this was my idea, but I'm having serious doubts about __.</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>True</td>\n",
       "      <td>3</td>\n",
       "      <td>True</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>Uh, hey guys, I know this was my idea, but I'm having serious doubts about Sluts, whores, and bitches.</td>\n",
       "      <td>0.275</td>\n",
       "      <td>3</td>\n",
       "      <td>1302</td>\n",
       "      <td>6210</td>\n",
       "      <td>1</td>\n",
       "      <td>0.3333</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2798462</th>\n",
       "      <td>279847</td>\n",
       "      <td>6</td>\n",
       "      <td>False</td>\n",
       "      <td>Alternative medicine is now embracing the curative powers of __.</td>\n",
       "      <td>1</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>False</td>\n",
       "      <td>3</td>\n",
       "      <td>True</td>\n",
       "      <td>1593.0</td>\n",
       "      <td>Alternative medicine is now embracing the curative powers of Sluts, whores, and bitches.</td>\n",
       "      <td>0.275</td>\n",
       "      <td>5</td>\n",
       "      <td>1302</td>\n",
       "      <td>5650</td>\n",
       "      <td>2</td>\n",
       "      <td>0.4000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "         fake_round_id  round_completion_seconds  round_skipped                                                                             black_card_text  black_card_pick_num             white_card_text    won  ID_index   has_  id_white                                                                                                                text  white_prior  pair_count  white_count  black_count  sum_won  pair_prior\n",
       "1440673         144068                        11          False                   The secret to a lasting marriage is communication, communication, and __.                    1  Sluts, whores, and bitches   True         4   True    1593.0                   The secret to a lasting marriage is communication, communication, and Sluts, whores, and bitches.        0.275           7         1302         6070        3      0.4286\n",
       "744071           74408                        63          False  In what's being hailed as a major breakthrough, scientists have synthesized __ in the lab.                    1  Sluts, whores, and bitches   True         2   True    1593.0  In what's being hailed as a major breakthrough, scientists have synthesized Sluts, whores, and bitches in the lab.        0.275           4         1302         5840        2      0.5000\n",
       "880140           88015                        20          False                                                               Why my mustache smell so bad?                    1  Sluts, whores, and bitches   True         1  False    1593.0                                                            Why my mustache smell so bad? Sluts, whores, and bitches        0.275           4         1302         5750        1      0.2500\n",
       "1901136         190114                        52          False                                                               What makes life worth living?                    1  Sluts, whores, and bitches  False         7  False    1593.0                                                            What makes life worth living? Sluts, whores, and bitches        0.275           5         1302         5900        0      0.0000\n",
       "2609031         260904                        10          False     To prepare for his upcoming role, Daniel Day-Lewis immersed himself in the world of __.                    1  Sluts, whores, and bitches  False         2   True    1593.0     To prepare for his upcoming role, Daniel Day-Lewis immersed himself in the world of Sluts, whores, and bitches.        0.275           3         1302         6110        2      0.6667\n",
       "1780039         178004                        30          False                                                         What made my first kiss so awkward?                    1  Sluts, whores, and bitches   True        10  False    1593.0                                                      What made my first kiss so awkward? Sluts, whores, and bitches        0.275           5         1302         6900        1      0.2000\n",
       "578031           57804                        10          False   Welcome to Señor Frog's! Would you like to try our signature cocktail, \"__ on the Beach\"?                    1  Sluts, whores, and bitches  False         2   True    1593.0   Welcome to Señor Frog's! Would you like to try our signature cocktail, \"Sluts, whores, and bitches on the Beach\"?        0.275           3         1302         6660        0      0.0000\n",
       "687974           68798                        26          False                                                                     Why do I hurt all over?                    1  Sluts, whores, and bitches  False         5  False    1593.0                                                                  Why do I hurt all over? Sluts, whores, and bitches        0.275           3         1302         5940        1      0.3333\n",
       "2167747         216775                        11          False        We do not shake with our left hands in this country. That is the hand we use for __.                    1  Sluts, whores, and bitches   True         8   True    1593.0        We do not shake with our left hands in this country. That is the hand we use for Sluts, whores, and bitches.        0.275           4         1302         6190        2      0.5000\n",
       "823072           82308                         6          False                           Just once, I'd like to hear you say \"Thanks, Mom. Thanks for __.\"                    1  Sluts, whores, and bitches  False         3   True    1593.0                           Just once, I'd like to hear you say \"Thanks, Mom. Thanks for Sluts, whores, and bitches.\"        0.275           4         1302         6180        0      0.0000\n",
       "2676252         267626                         8          False              Uh, hey guys, I know this was my idea, but I'm having serious doubts about __.                    1  Sluts, whores, and bitches   True         3   True    1593.0              Uh, hey guys, I know this was my idea, but I'm having serious doubts about Sluts, whores, and bitches.        0.275           3         1302         6210        1      0.3333\n",
       "2798462         279847                         6          False                            Alternative medicine is now embracing the curative powers of __.                    1  Sluts, whores, and bitches  False         3   True    1593.0                            Alternative medicine is now embracing the curative powers of Sluts, whores, and bitches.        0.275           5         1302         5650        2      0.4000"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.sort_values(by=[\"white_prior\"],ascending=False).head(12)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "e6d3e9de",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "id": "e6d3e9de",
    "outputId": "90e5011d-9032-4429-da52-e6c227ebf229"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1949028\n",
      "1429296\n",
      "fake_round_id               240182\n",
      "round_completion_seconds      1650\n",
      "round_skipped                    1\n",
      "black_card_text                430\n",
      "black_card_pick_num              1\n",
      "white_card_text               1964\n",
      "won                              2\n",
      "ID_index                        10\n",
      "has_                             2\n",
      "id_white                      1964\n",
      "text                        288765\n",
      "white_prior                   1857\n",
      "pair_count                      12\n",
      "white_count                    213\n",
      "black_count                    149\n",
      "sum_won                          9\n",
      "pair_prior                      40\n",
      "dtype: int64\n"
     ]
    }
   ],
   "source": [
    "print(df.shape[0])\n",
    "df = df.loc[df[\"pair_count\"]>=MIN_PAIR_FREQ]\n",
    "print(df.shape[0])\n",
    "print(df.nunique())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "yspNHvCu4NmW",
   "metadata": {
    "id": "yspNHvCu4NmW"
   },
   "source": [
    "### Get Outperforming pairs target: \n",
    "Pairs where the white card does better than it's prior win rate\n",
    "* Add +0.1 to smoothen. \n",
    "* Lacks statistical significance unless we were to filter bya ver high min count of pairs\n",
    "* Binomal paired t-test distrib ? \n",
    "\n",
    "##### Get ourperforming pairs (vs white prior win rate)\n",
    "* https://maxhalford.github.io/blog/target-encoding/. - smooth the ratio? "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "R2NQS2m-wqag",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 300
    },
    "id": "R2NQS2m-wqag",
    "outputId": "b0cbea00-a017-427d-a890-81280cc082cf"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pair_prior</th>\n",
       "      <th>pair_count</th>\n",
       "      <th>white_prior</th>\n",
       "      <th>white_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.10</td>\n",
       "      <td>4.95</td>\n",
       "      <td>0.10</td>\n",
       "      <td>1251.69</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.15</td>\n",
       "      <td>1.16</td>\n",
       "      <td>0.05</td>\n",
       "      <td>58.05</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.00</td>\n",
       "      <td>4.00</td>\n",
       "      <td>0.01</td>\n",
       "      <td>26.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.00</td>\n",
       "      <td>4.00</td>\n",
       "      <td>0.06</td>\n",
       "      <td>1228.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.00</td>\n",
       "      <td>5.00</td>\n",
       "      <td>0.09</td>\n",
       "      <td>1252.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.20</td>\n",
       "      <td>6.00</td>\n",
       "      <td>0.13</td>\n",
       "      <td>1276.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.00</td>\n",
       "      <td>15.00</td>\n",
       "      <td>0.27</td>\n",
       "      <td>2455.00</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       pair_prior  pair_count  white_prior  white_count\n",
       "count   288765.00   288765.00    288765.00    288765.00\n",
       "mean         0.10        4.95         0.10      1251.69\n",
       "std          0.15        1.16         0.05        58.05\n",
       "min          0.00        4.00         0.01        26.00\n",
       "25%          0.00        4.00         0.06      1228.00\n",
       "50%          0.00        5.00         0.09      1252.00\n",
       "75%          0.20        6.00         0.13      1276.00\n",
       "max          1.00       15.00         0.27      2455.00"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_grp = df.filter(['black_card_text', 'white_card_text', \"pair_prior\",\"pair_count\",\"white_prior\",'text',\"white_count\"],axis=1).drop_duplicates()\n",
    "df_grp =df_grp.sort_values([\"white_prior\",\"pair_prior\"],ascending=False)\n",
    "\n",
    "# df_grp = df_grp.round(4)\n",
    "df_grp.describe().round(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "q8XJAeEp19uq",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 423
    },
    "id": "q8XJAeEp19uq",
    "outputId": "e60638ea-9cf8-4b4f-f073-7d63496a67c8"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>black_card_text</th>\n",
       "      <th>white_card_text</th>\n",
       "      <th>pair_prior</th>\n",
       "      <th>pair_count</th>\n",
       "      <th>white_prior</th>\n",
       "      <th>text</th>\n",
       "      <th>white_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>568344</th>\n",
       "      <td>What makes life worth living?</td>\n",
       "      <td>Dying</td>\n",
       "      <td>1.00</td>\n",
       "      <td>6</td>\n",
       "      <td>0.1494</td>\n",
       "      <td>What makes life worth living? Dying</td>\n",
       "      <td>1305</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1735731</th>\n",
       "      <td>What makes life worth living?</td>\n",
       "      <td>Masturbating</td>\n",
       "      <td>0.75</td>\n",
       "      <td>4</td>\n",
       "      <td>0.2561</td>\n",
       "      <td>What makes life worth living? Masturbating</td>\n",
       "      <td>1343</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1211070</th>\n",
       "      <td>What makes life worth living?</td>\n",
       "      <td>Eating ass</td>\n",
       "      <td>0.75</td>\n",
       "      <td>4</td>\n",
       "      <td>0.2372</td>\n",
       "      <td>What makes life worth living? Eating ass</td>\n",
       "      <td>1248</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>129149</th>\n",
       "      <td>What makes life worth living?</td>\n",
       "      <td>Blowing my boyfriend so hard he shits</td>\n",
       "      <td>0.75</td>\n",
       "      <td>4</td>\n",
       "      <td>0.1559</td>\n",
       "      <td>What makes life worth living? Blowing my boyfriend so hard he shits</td>\n",
       "      <td>1257</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>327884</th>\n",
       "      <td>What makes life worth living?</td>\n",
       "      <td>Period poops</td>\n",
       "      <td>0.75</td>\n",
       "      <td>4</td>\n",
       "      <td>0.1492</td>\n",
       "      <td>What makes life worth living? Period poops</td>\n",
       "      <td>1280</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>515157</th>\n",
       "      <td>What makes life worth living?</td>\n",
       "      <td>The secret to truly resilient hair</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>0.0260</td>\n",
       "      <td>What makes life worth living? The secret to truly resilient hair</td>\n",
       "      <td>1231</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>23640</th>\n",
       "      <td>What makes life worth living?</td>\n",
       "      <td>Sanding off a man's nose</td>\n",
       "      <td>0.00</td>\n",
       "      <td>7</td>\n",
       "      <td>0.0246</td>\n",
       "      <td>What makes life worth living? Sanding off a man's nose</td>\n",
       "      <td>1303</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1055043</th>\n",
       "      <td>What makes life worth living?</td>\n",
       "      <td>Sharpening a foam broadsword on a foam whetstone</td>\n",
       "      <td>0.00</td>\n",
       "      <td>5</td>\n",
       "      <td>0.0239</td>\n",
       "      <td>What makes life worth living? Sharpening a foam broadsword on a foam whetstone</td>\n",
       "      <td>1213</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>557093</th>\n",
       "      <td>What makes life worth living?</td>\n",
       "      <td>An emergency all-floor meeting on inclusion</td>\n",
       "      <td>0.00</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0210</td>\n",
       "      <td>What makes life worth living? An emergency all-floor meeting on inclusion</td>\n",
       "      <td>1284</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>80175</th>\n",
       "      <td>What makes life worth living?</td>\n",
       "      <td>Charging up all the way</td>\n",
       "      <td>0.00</td>\n",
       "      <td>6</td>\n",
       "      <td>0.0111</td>\n",
       "      <td>What makes life worth living? Charging up all the way</td>\n",
       "      <td>1266</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>701 rows × 7 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "                       black_card_text                                   white_card_text  pair_prior  pair_count  white_prior                                                                            text  white_count\n",
       "568344   What makes life worth living?                                             Dying        1.00           6       0.1494                                             What makes life worth living? Dying         1305\n",
       "1735731  What makes life worth living?                                      Masturbating        0.75           4       0.2561                                      What makes life worth living? Masturbating         1343\n",
       "1211070  What makes life worth living?                                        Eating ass        0.75           4       0.2372                                        What makes life worth living? Eating ass         1248\n",
       "129149   What makes life worth living?             Blowing my boyfriend so hard he shits        0.75           4       0.1559             What makes life worth living? Blowing my boyfriend so hard he shits         1257\n",
       "327884   What makes life worth living?                                      Period poops        0.75           4       0.1492                                      What makes life worth living? Period poops         1280\n",
       "...                                ...                                               ...         ...         ...          ...                                                                             ...          ...\n",
       "515157   What makes life worth living?                The secret to truly resilient hair        0.00           6       0.0260                What makes life worth living? The secret to truly resilient hair         1231\n",
       "23640    What makes life worth living?                          Sanding off a man's nose        0.00           7       0.0246                          What makes life worth living? Sanding off a man's nose         1303\n",
       "1055043  What makes life worth living?  Sharpening a foam broadsword on a foam whetstone        0.00           5       0.0239  What makes life worth living? Sharpening a foam broadsword on a foam whetstone         1213\n",
       "557093   What makes life worth living?       An emergency all-floor meeting on inclusion        0.00           4       0.0210       What makes life worth living? An emergency all-floor meeting on inclusion         1284\n",
       "80175    What makes life worth living?                           Charging up all the way        0.00           6       0.0111                           What makes life worth living? Charging up all the way         1266\n",
       "\n",
       "[701 rows x 7 columns]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# df_grp.loc[df_grp[\"black_card_text\"]==\"What makes life worth living?\"].drop_duplicates(subset=\"white_card_text\").sort_values([\"pair_count\",\"pair_prior\",\"white_prior\"],ascending=False)\n",
    "df_grp.loc[df_grp[\"black_card_text\"]==\"What makes life worth living?\"].drop_duplicates(subset=\"white_card_text\").sort_values([\"pair_prior\",\"white_prior\"],ascending=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "wq76SiJA08UV",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 476
    },
    "id": "wq76SiJA08UV",
    "outputId": "fdbe9d99-486b-4d7d-dfe1-a74e21707f91"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>black_card_text</th>\n",
       "      <th>white_card_text</th>\n",
       "      <th>pair_prior</th>\n",
       "      <th>pair_count</th>\n",
       "      <th>white_prior</th>\n",
       "      <th>text</th>\n",
       "      <th>white_count</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>84899</th>\n",
       "      <td>My country, 'tis of thee, sweet land of __.</td>\n",
       "      <td>Sluts, whores, and bitches</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5</td>\n",
       "      <td>0.2750</td>\n",
       "      <td>My country, 'tis of thee, sweet land of Sluts, whores, and bitches.</td>\n",
       "      <td>1302</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>283889</th>\n",
       "      <td>\"This is madness!\" \\n \"No. THIS IS __!\"</td>\n",
       "      <td>A big black dick</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.2728</td>\n",
       "      <td>\"This is madness!\" \\n \"No. THIS IS A big black dick!\"</td>\n",
       "      <td>1228</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>666097</th>\n",
       "      <td>I love being a mom. But it's tough when my kids come home filthy from __. That's why there's Tide.®</td>\n",
       "      <td>A big black dick</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.2728</td>\n",
       "      <td>I love being a mom. But it's tough when my kids come home filthy from A big black dick. That's why there's Tide.®</td>\n",
       "      <td>1228</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>644015</th>\n",
       "      <td>Unfortunately, no one can be told what __ is. You have to experience it for yourself.</td>\n",
       "      <td>The Asshole Destroyer 3000</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.2692</td>\n",
       "      <td>Unfortunately, no one can be told what The Asshole Destroyer 3000 is. You have to experience it for yourself.</td>\n",
       "      <td>1252</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1227772</th>\n",
       "      <td>One more thing. Watch out for Big Mike. They say he killed a man with __.</td>\n",
       "      <td>Three dicks at the same time</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.2665</td>\n",
       "      <td>One more thing. Watch out for Big Mike. They say he killed a man with Three dicks at the same time.</td>\n",
       "      <td>1246</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>623669</th>\n",
       "      <td>Personal foul: __. 15-yard penalty. Automatic first down.</td>\n",
       "      <td>Incest</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.2662</td>\n",
       "      <td>Personal foul: Incest. 15-yard penalty. Automatic first down.</td>\n",
       "      <td>1247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1299630</th>\n",
       "      <td>Lovin' you is easy 'cause you're __.</td>\n",
       "      <td>Anal</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5</td>\n",
       "      <td>0.2652</td>\n",
       "      <td>Lovin' you is easy 'cause you're Anal.</td>\n",
       "      <td>1312</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>643604</th>\n",
       "      <td>But before I kill you, Mr. Bond, I must show you __.</td>\n",
       "      <td>Dem titties</td>\n",
       "      <td>1.0</td>\n",
       "      <td>5</td>\n",
       "      <td>0.2630</td>\n",
       "      <td>But before I kill you, Mr. Bond, I must show you Dem titties.</td>\n",
       "      <td>1247</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>48201</th>\n",
       "      <td>Don't miss Rachel Ray's hit new show, Cooking with __.</td>\n",
       "      <td>Elf cum</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.2622</td>\n",
       "      <td>Don't miss Rachel Ray's hit new show, Cooking with Elf cum.</td>\n",
       "      <td>1274</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>202107</th>\n",
       "      <td>Goldfinger! He's the man, the man with __.</td>\n",
       "      <td>Elf cum</td>\n",
       "      <td>1.0</td>\n",
       "      <td>4</td>\n",
       "      <td>0.2622</td>\n",
       "      <td>Goldfinger! He's the man, the man with Elf cum.</td>\n",
       "      <td>1274</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                             black_card_text               white_card_text  pair_prior  pair_count  white_prior                                                                                                               text  white_count\n",
       "84899                                                            My country, 'tis of thee, sweet land of __.    Sluts, whores, and bitches         1.0           5       0.2750                                                My country, 'tis of thee, sweet land of Sluts, whores, and bitches.         1302\n",
       "283889                                                               \"This is madness!\" \\n \"No. THIS IS __!\"              A big black dick         1.0           4       0.2728                                                              \"This is madness!\" \\n \"No. THIS IS A big black dick!\"         1228\n",
       "666097   I love being a mom. But it's tough when my kids come home filthy from __. That's why there's Tide.®              A big black dick         1.0           4       0.2728  I love being a mom. But it's tough when my kids come home filthy from A big black dick. That's why there's Tide.®         1228\n",
       "644015                 Unfortunately, no one can be told what __ is. You have to experience it for yourself.    The Asshole Destroyer 3000         1.0           4       0.2692      Unfortunately, no one can be told what The Asshole Destroyer 3000 is. You have to experience it for yourself.         1252\n",
       "1227772                            One more thing. Watch out for Big Mike. They say he killed a man with __.  Three dicks at the same time         1.0           4       0.2665                One more thing. Watch out for Big Mike. They say he killed a man with Three dicks at the same time.         1246\n",
       "623669                                             Personal foul: __. 15-yard penalty. Automatic first down.                        Incest         1.0           4       0.2662                                                      Personal foul: Incest. 15-yard penalty. Automatic first down.         1247\n",
       "1299630                                                                 Lovin' you is easy 'cause you're __.                          Anal         1.0           5       0.2652                                                                             Lovin' you is easy 'cause you're Anal.         1312\n",
       "643604                                                  But before I kill you, Mr. Bond, I must show you __.                   Dem titties         1.0           5       0.2630                                                      But before I kill you, Mr. Bond, I must show you Dem titties.         1247\n",
       "48201                                                 Don't miss Rachel Ray's hit new show, Cooking with __.                       Elf cum         1.0           4       0.2622                                                        Don't miss Rachel Ray's hit new show, Cooking with Elf cum.         1274\n",
       "202107                                                            Goldfinger! He's the man, the man with __.                       Elf cum         1.0           4       0.2622                                                                    Goldfinger! He's the man, the man with Elf cum.         1274"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# df_grp\n",
    "df_grp.sort_values([\"pair_prior\",\"white_prior\"],ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "WSDHv05-08ZM",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 300
    },
    "id": "WSDHv05-08ZM",
    "outputId": "7b624ead-f5d5-42b8-f027-97c6ad703d0e"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pair_prior</th>\n",
       "      <th>pair_count</th>\n",
       "      <th>white_prior</th>\n",
       "      <th>white_count</th>\n",
       "      <th>outperform_sub</th>\n",
       "      <th>outperform_ratio</th>\n",
       "      <th>outperform_ratio_smooth</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.10</td>\n",
       "      <td>4.95</td>\n",
       "      <td>0.10</td>\n",
       "      <td>1251.69</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.53</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.15</td>\n",
       "      <td>1.16</td>\n",
       "      <td>0.05</td>\n",
       "      <td>58.05</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.70</td>\n",
       "      <td>0.71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.00</td>\n",
       "      <td>4.00</td>\n",
       "      <td>0.01</td>\n",
       "      <td>26.00</td>\n",
       "      <td>-0.28</td>\n",
       "      <td>0.27</td>\n",
       "      <td>-0.47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.00</td>\n",
       "      <td>4.00</td>\n",
       "      <td>0.06</td>\n",
       "      <td>1228.00</td>\n",
       "      <td>-0.09</td>\n",
       "      <td>0.52</td>\n",
       "      <td>0.04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.00</td>\n",
       "      <td>5.00</td>\n",
       "      <td>0.09</td>\n",
       "      <td>1252.00</td>\n",
       "      <td>-0.05</td>\n",
       "      <td>0.65</td>\n",
       "      <td>0.30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.20</td>\n",
       "      <td>6.00</td>\n",
       "      <td>0.13</td>\n",
       "      <td>1276.00</td>\n",
       "      <td>0.09</td>\n",
       "      <td>1.41</td>\n",
       "      <td>0.87</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.00</td>\n",
       "      <td>15.00</td>\n",
       "      <td>0.28</td>\n",
       "      <td>2455.00</td>\n",
       "      <td>0.95</td>\n",
       "      <td>7.25</td>\n",
       "      <td>6.91</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       pair_prior  pair_count  white_prior  white_count  outperform_sub  outperform_ratio  outperform_ratio_smooth\n",
       "count   288765.00   288765.00    288765.00    288765.00       288765.00         288765.00                288765.00\n",
       "mean         0.10        4.95         0.10      1251.69            0.00              1.00                     0.53\n",
       "std          0.15        1.16         0.05        58.05            0.14              0.70                     0.71\n",
       "min          0.00        4.00         0.01        26.00           -0.28              0.27                    -0.47\n",
       "25%          0.00        4.00         0.06      1228.00           -0.09              0.52                     0.04\n",
       "50%          0.00        5.00         0.09      1252.00           -0.05              0.65                     0.30\n",
       "75%          0.20        6.00         0.13      1276.00            0.09              1.41                     0.87\n",
       "max          1.00       15.00         0.28      2455.00            0.95              7.25                     6.91"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_grp[\"outperform_sub\"] = df_grp[\"pair_prior\"].sub(df_grp[\"white_prior\"])\n",
    "# df_grp[\"outperform_ratio\"] = (df_grp[\"pair_prior\"].sub(df_grp[\"white_prior\"])).div(df_grp[\"white_prior\"]) #ORIG\n",
    "df_grp[\"outperform_ratio\"] = (0.1 +df_grp[\"pair_prior\"]).div(0.1 +df_grp[\"white_prior\"])\n",
    "df_grp[\"outperform_ratio_smooth\"] = (0.1 +(df_grp[\"pair_prior\"]).sub(df_grp[\"white_prior\"])).div(df_grp[\"white_prior\"]+0.1) ## 0.1 baseline smoothing\n",
    "df_grp = df_grp.round(4)\n",
    "df_grp.describe().round(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "3D_6n1YO6TR5",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 300
    },
    "id": "3D_6n1YO6TR5",
    "outputId": "90c85d07-230c-41ad-bc63-69d3565c8e06"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>pair_prior</th>\n",
       "      <th>pair_count</th>\n",
       "      <th>white_prior</th>\n",
       "      <th>white_count</th>\n",
       "      <th>outperform_sub</th>\n",
       "      <th>outperform_ratio</th>\n",
       "      <th>outperform_ratio_smooth</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "      <td>288765.00</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>0.10</td>\n",
       "      <td>4.95</td>\n",
       "      <td>0.10</td>\n",
       "      <td>1251.69</td>\n",
       "      <td>0.00</td>\n",
       "      <td>1.00</td>\n",
       "      <td>0.53</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>0.15</td>\n",
       "      <td>1.16</td>\n",
       "      <td>0.05</td>\n",
       "      <td>58.05</td>\n",
       "      <td>0.14</td>\n",
       "      <td>0.70</td>\n",
       "      <td>0.71</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>0.00</td>\n",
       "      <td>4.00</td>\n",
       "      <td>0.01</td>\n",
       "      <td>26.00</td>\n",
       "      <td>-0.28</td>\n",
       "      <td>0.27</td>\n",
       "      <td>-0.47</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>0.00</td>\n",
       "      <td>4.00</td>\n",
       "      <td>0.06</td>\n",
       "      <td>1228.00</td>\n",
       "      <td>-0.09</td>\n",
       "      <td>0.52</td>\n",
       "      <td>0.04</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>0.00</td>\n",
       "      <td>5.00</td>\n",
       "      <td>0.09</td>\n",
       "      <td>1252.00</td>\n",
       "      <td>-0.05</td>\n",
       "      <td>0.65</td>\n",
       "      <td>0.30</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>0.20</td>\n",
       "      <td>6.00</td>\n",
       "      <td>0.13</td>\n",
       "      <td>1276.00</td>\n",
       "      <td>0.09</td>\n",
       "      <td>1.41</td>\n",
       "      <td>0.87</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>1.00</td>\n",
       "      <td>15.00</td>\n",
       "      <td>0.28</td>\n",
       "      <td>2455.00</td>\n",
       "      <td>0.95</td>\n",
       "      <td>7.25</td>\n",
       "      <td>6.91</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "       pair_prior  pair_count  white_prior  white_count  outperform_sub  outperform_ratio  outperform_ratio_smooth\n",
       "count   288765.00   288765.00    288765.00    288765.00       288765.00         288765.00                288765.00\n",
       "mean         0.10        4.95         0.10      1251.69            0.00              1.00                     0.53\n",
       "std          0.15        1.16         0.05        58.05            0.14              0.70                     0.71\n",
       "min          0.00        4.00         0.01        26.00           -0.28              0.27                    -0.47\n",
       "25%          0.00        4.00         0.06      1228.00           -0.09              0.52                     0.04\n",
       "50%          0.00        5.00         0.09      1252.00           -0.05              0.65                     0.30\n",
       "75%          0.20        6.00         0.13      1276.00            0.09              1.41                     0.87\n",
       "max          1.00       15.00         0.28      2455.00            0.95              7.25                     6.91"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# df_grp.loc[df_grp[\"pair_prior\"].round(4)==df_grp[\"white_prior\"].round(4)].describe()\n",
    "df_grp.describe().round(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "ZVr6HlG46ezt",
   "metadata": {
    "id": "ZVr6HlG46ezt"
   },
   "outputs": [],
   "source": [
    "# df_grp.loc[df_grp[\"pair_prior\"]==df_grp[\"white_prior\"]]\n",
    "# df_grp.loc[df_grp[\"pair_prior\"].round(4)==df_grp[\"white_prior\"].round(4)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "lvZQ_gbG3QHa",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 265
    },
    "id": "lvZQ_gbG3QHa",
    "outputId": "25abc996-b958-4aab-8a84-afa8aa35c795"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAD4CAYAAADy46FuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAZTElEQVR4nO3df4xd9Z3e8fezOKEEL4Qf2ZFlszURbFTArXc9MlRRoqHOgpOgQCroDqLBNFROEKkSLVID25VIQUjQiqVCabx1YhdDshgKy4IWvKwVGJGV+GUndG0gLEPwLoNd3MReYJLAdsjTP+53nGvvne/8uOO5c5znJV3dcz7nfM98rrHm8fmecw+yTURExER+rdcNRETE/JagiIiIqgRFRERUJSgiIqIqQREREVULet3AbDv55JO9dOnSA+s//elPOfbYY3vX0DQ1qd8m9QrN6rdJvUKz+m1SrzB3/W7fvv3Htj/UcaPtI+q1YsUKt3v88cfdJE3qt0m92s3qt0m92s3qt0m92nPXL7DNE/xezdRTRERUJSgiIqIqQREREVUJioiIqEpQREREVYIiIiKqEhQREVGVoIiIiKoERUREVB1xj/Do1tJrH+7Jz91186d78nMjIiaTM4qIiKhKUERERFWCIiIiqhIUERFRlaCIiIiqBEVERFQlKCIioipBERERVQmKiIiomjQoJG2UtFfSzrbaPZKeK69dkp4r9aWSft627Y/bxqyQtEPSsKTbJanUjy7HG5b0tKSlbWPWSHq5vNbM5gePiIipmcojPO4Avg7cOV6w/Xvjy5JuBd5s2/8V28s7HGcdsBZ4CngEWA1sAa4E9ts+TdIgcAvwe5JOBK4H+gED2yU9ZHv/lD9dRER0bdIzCttPAPs6bStnBf8GuLt2DEmLgONsP2nbtELnorL5QmBTWb4PWFWOez6w1fa+Eg5baYVLRETMoW4fCvgx4A3bL7fVTpX0A+At4A9tfw9YDIy07TNSapT31wBsj0l6Ezipvd5hzEEkraV1tkJfXx9DQ0MHto2Ojh60Pplrlo1Ned/ZNN7jdPvtpSb1Cs3qt0m9QrP6bVKvMD/67TYoLuXgs4k9wG/a/omkFcCfSToTUIexLu8TbauNObhorwfWA/T393tgYODAtqGhIdrXJ3NFr54ee9kAMP1+e6lJvUKz+m1Sr9CsfpvUK8yPfmd815OkBcC/Bu4Zr9l+1/ZPyvJ24BXgt2idDSxpG74E2F2WR4BT2o55PK2prgP1DmMiImKOdHN77CeAH9o+MKUk6UOSjirLHwZOB35kew/wtqRzyvWHy4EHy7CHgPE7mi4GHivXMR4FzpN0gqQTgPNKLSIi5tCkU0+S7gYGgJMljQDX294ADPKPL2J/HLhB0hjwHvBF2+MXwq+idQfVMbTudtpS6huAuyQN0zqTGASwvU/SjcCzZb8b2o4VERFzZNKgsH3pBPUrOtTuB+6fYP9twFkd6u8Al0wwZiOwcbIeIyLi8Mk3syMioipBERERVQmKiIioSlBERERVgiIiIqoSFBERUZWgiIiIqgRFRERUJSgiIqIqQREREVUJioiIqEpQREREVYIiIiKqEhQREVGVoIiIiKoERUREVCUoIiKiKkERERFVkwaFpI2S9kra2Vb7mqTXJT1XXp9q23adpGFJL0k6v62+QtKOsu12SSr1oyXdU+pPS1raNmaNpJfLa82sfeqIiJiyqZxR3AGs7lC/zfby8noEQNIZwCBwZhnzDUlHlf3XAWuB08tr/JhXAvttnwbcBtxSjnUicD1wNrASuF7SCdP+hBER0ZVJg8L2E8C+KR7vQmCz7XdtvwoMAyslLQKOs/2kbQN3Ahe1jdlUlu8DVpWzjfOBrbb32d4PbKVzYEVExGHUzTWKL0n66zI1Nf4v/cXAa237jJTa4rJ8aP2gMbbHgDeBkyrHioiIObRghuPWATcCLu+3Ap8H1GFfV+rMcMxBJK2lNa1FX18fQ0NDB7aNjo4etD6Za5aNTXnf2TTe43T77aUm9QrN6rdJvUKz+m1SrzA/+p1RUNh+Y3xZ0jeBPy+rI8ApbbsuAXaX+pIO9fYxI5IWAMfTmuoaAQYOGTM0QT/rgfUA/f39Hhj45bChoSHa1ydzxbUPT3nf2bTrsgFg+v32UpN6hWb126ReoVn9NqlXmB/9zmjqqVxzGPdZYPyOqIeAwXIn06m0Llo/Y3sP8Lakc8r1h8uBB9vGjN/RdDHwWLmO8ShwnqQTytTWeaUWERFzaNIzCkl30/qX/cmSRmjdiTQgaTmtqaBdwBcAbD8v6V7gBWAMuNr2e+VQV9G6g+oYYEt5AWwA7pI0TOtMYrAca5+kG4Fny3432J7qRfWIiJglkwaF7Us7lDdU9r8JuKlDfRtwVof6O8AlExxrI7Bxsh4jIuLwyTezIyKiKkERERFVCYqIiKhKUERERFWCIiIiqhIUERFRlaCIiIiqBEVERFQlKCIioipBERERVQmKiIioSlBERERVgiIiIqoSFBERUZWgiIiIqgRFRERUJSgiIqIqQREREVUJioiIqJo0KCRtlLRX0s622n+V9ENJfy3pAUkfLPWlkn4u6bny+uO2MSsk7ZA0LOl2SSr1oyXdU+pPS1raNmaNpJfLa81sfvCIiJiaqZxR3AGsPqS2FTjL9j8H/ga4rm3bK7aXl9cX2+rrgLXA6eU1fswrgf22TwNuA24BkHQicD1wNrASuF7SCdP4bBERMQsmDQrbTwD7Dqn9pe2xsvoUsKR2DEmLgONsP2nbwJ3ARWXzhcCmsnwfsKqcbZwPbLW9z/Z+WuF0aGBFRMRhtmAWjvF54J629VMl/QB4C/hD298DFgMjbfuMlBrl/TUA22OS3gROaq93GHMQSWtpna3Q19fH0NDQgW2jo6MHrU/mmmVjk+90GIz3ON1+e6lJvUKz+m1Sr9CsfpvUK8yPfrsKCkn/CRgDvlNKe4DftP0TSSuAP5N0JqAOwz1+mAm21cYcXLTXA+sB+vv7PTAwcGDb0NAQ7euTueLah6e872zaddkAMP1+e6lJvUKz+m1Sr9CsfpvUK8yPfmd811O5uHwBcFmZTsL2u7Z/Upa3A68Av0XrbKB9emoJsLssjwCnlGMuAI6nNdV1oN5hTEREzJEZBYWk1cBXgc/Y/llb/UOSjirLH6Z10fpHtvcAb0s6p1x/uBx4sAx7CBi/o+li4LESPI8C50k6oVzEPq/UIiJiDk069STpbmAAOFnSCK07ka4Djga2lrtcnyp3OH0cuEHSGPAe8EXb4xfCr6J1B9UxwJbyAtgA3CVpmNaZxCCA7X2SbgSeLfvd0HasiIiYI5MGhe1LO5Q3TLDv/cD9E2zbBpzVof4OcMkEYzYCGyfrMSIiDp98MzsiIqoSFBERUZWgiIiIqgRFRERUJSgiIqIqQREREVUJioiIqEpQREREVYIiIiKqEhQREVGVoIiIiKoERUREVCUoIiKiKkERERFVCYqIiKhKUERERFWCIiIiqhIUERFRlaCIiIiqSYNC0kZJeyXtbKudKGmrpJfL+wlt266TNCzpJUnnt9VXSNpRtt0uSaV+tKR7Sv1pSUvbxqwpP+NlSWtm7VNHRMSUTeWM4g5g9SG1a4Hv2j4d+G5ZR9IZwCBwZhnzDUlHlTHrgLXA6eU1fswrgf22TwNuA24pxzoRuB44G1gJXN8eSBERMTcmDQrbTwD7DilfCGwqy5uAi9rqm22/a/tVYBhYKWkRcJztJ20buPOQMePHug9YVc42zge22t5nez+wlX8cWBERcZgtmOG4Ptt7AGzvkfQbpb4YeKptv5FS+39l+dD6+JjXyrHGJL0JnNRe7zDmIJLW0jpboa+vj6GhoQPbRkdHD1qfzDXLxqa872wa73G6/fZSk3qFZvXbpF6hWf02qVeYH/3ONCgmog41V+ozHXNw0V4PrAfo7+/3wMDAgW1DQ0O0r0/mimsfnvK+s2nXZQPA9PvtpSb1Cs3qt0m9QrP6bVKvMD/6neldT2+U6STK+95SHwFOadtvCbC71Jd0qB80RtIC4HhaU10THSsiIubQTIPiIWD8LqQ1wINt9cFyJ9OptC5aP1Omqd6WdE65/nD5IWPGj3Ux8Fi5jvEocJ6kE8pF7PNKLSIi5tCkU0+S7gYGgJMljdC6E+lm4F5JVwJ/B1wCYPt5SfcCLwBjwNW23yuHuorWHVTHAFvKC2ADcJekYVpnEoPlWPsk3Qg8W/a7wfahF9UjIuIwmzQobF86waZVE+x/E3BTh/o24KwO9XcoQdNh20Zg42Q9RkTE4ZNvZkdERFWCIiIiqhIUERFRlaCIiIiqBEVERFQlKCIioipBERERVQmKiIioSlBERERVgiIiIqoSFBERUZWgiIiIqgRFRERUJSgiIqIqQREREVUJioiIqEpQREREVYIiIiKqEhQREVE146CQ9BFJz7W93pL0FUlfk/R6W/1TbWOukzQs6SVJ57fVV0jaUbbdLkmlfrSke0r9aUlLu/q0ERExbTMOCtsv2V5uezmwAvgZ8EDZfNv4NtuPAEg6AxgEzgRWA9+QdFTZfx2wFji9vFaX+pXAftunAbcBt8y034iImJnZmnpaBbxi+28r+1wIbLb9ru1XgWFgpaRFwHG2n7Rt4E7gorYxm8ryfcCq8bONiIiYG2r9bu7yINJG4Pu2vy7pa8AVwFvANuAa2/slfR14yva3y5gNwBZgF3Cz7U+U+seAr9q+QNJOYLXtkbLtFeBs2z8+5OevpXVGQl9f34rNmzcf2DY6OsrChQun/Fl2vP7m9P8AZsGyxccD0++3l5rUKzSr3yb1Cs3qt0m9wtz1e+6552633d9p24JuDy7p/cBngOtKaR1wI+DyfivweaDTmYArdSbZ9suCvR5YD9Df3++BgYED24aGhmhfn8wV1z485X1n067LBoDp99tLTeoVmtVvk3qFZvXbpF5hfvQ7G1NPn6R1NvEGgO03bL9n+xfAN4GVZb8R4JS2cUuA3aW+pEP9oDGSFgDHA/tmoeeIiJii2QiKS4G7x1fKNYdxnwV2luWHgMFyJ9OptC5aP2N7D/C2pHPK9YfLgQfbxqwpyxcDj3k25soiImLKupp6kvQB4HeBL7SV/4uk5bSmiHaNb7P9vKR7gReAMeBq2++VMVcBdwDH0LpusaXUNwB3SRqmdSYx2E2/ERExfV0Fhe2fAScdUvtcZf+bgJs61LcBZ3WovwNc0k2PERHRnXwzOyIiqhIUERFRlaCIiIiqBEVERFQlKCIioipBERERVQmKiIioSlBERERVgiIiIqoSFBERUZWgiIiIqgRFRERUJSgiIqIqQREREVUJioiIqEpQREREVYIiIiKqEhQREVHVVVBI2iVph6TnJG0rtRMlbZX0cnk/oW3/6yQNS3pJ0vlt9RXlOMOSbpekUj9a0j2l/rSkpd30GxER0zcbZxTn2l5uu7+sXwt81/bpwHfLOpLOAAaBM4HVwDckHVXGrAPWAqeX1+pSvxLYb/s04DbgllnoNyIipuFwTD1dCGwqy5uAi9rqm22/a/tVYBhYKWkRcJztJ20buPOQMePHug9YNX62ERERc0Ot380zHCy9CuwHDPwP2+sl/b3tD7bts9/2CZK+Djxl+9ulvgHYAuwCbrb9iVL/GPBV2xdI2gmstj1Str0CnG37x4f0sZbWGQl9fX0rNm/efGDb6OgoCxcunPJn2vH6m9P8U5hdfcfAGz+f25+5bPHxMxo33T/bXmtSv03qFZrVb5N6hbnr99xzz93eNjN0kAVdHvujtndL+g1gq6QfVvbtdCbgSr025uCCvR5YD9Df3++BgYED24aGhmhfn8wV1z485X0Ph2uWjXHrjm7/s0zPrssGZjRuun+2vdakfpvUKzSr3yb1CvOj366mnmzvLu97gQeAlcAbZTqJ8r637D4CnNI2fAmwu9SXdKgfNEbSAuB4YF83PUdExPTMOCgkHSvp18eXgfOAncBDwJqy2xrgwbL8EDBY7mQ6ldZF62ds7wHelnROuf5w+SFjxo91MfCYu5kri4iIaetmjqMPeKBcW14A/Intv5D0LHCvpCuBvwMuAbD9vKR7gReAMeBq2++VY10F3AEcQ+u6xZZS3wDcJWmY1pnEYBf9RkTEDMw4KGz/CPgXHeo/AVZNMOYm4KYO9W3AWR3q71CCJiIieiPfzI6IiKoERUREVCUoIiKiKkERERFVCYqIiKhKUERERFWCIiIiqhIUERFRlaCIiIiqBEVERFTN7fOsY15aOsNHq1+zbKyrx7LvuvnTMx4bEXMnZxQREVGVoIiIiKoERUREVCUoIiKiKkERERFVCYqIiKhKUERERFWCIiIiqmYcFJJOkfS4pBclPS/py6X+NUmvS3quvD7VNuY6ScOSXpJ0flt9haQdZdvtklTqR0u6p9SflrS0i88aEREz0M0ZxRhwje1/BpwDXC3pjLLtNtvLy+sRgLJtEDgTWA18Q9JRZf91wFrg9PJaXepXAvttnwbcBtzSRb8RETEDMw4K23tsf78svw28CCyuDLkQ2Gz7XduvAsPASkmLgONsP2nbwJ3ARW1jNpXl+4BV42cbERExN9T63dzlQVpTQk8AZwG/D1wBvAVso3XWsV/S14GnbH+7jNkAbAF2ATfb/kSpfwz4qu0LJO0EVtseKdteAc62/eNDfv5aWmck9PX1rdi8efOBbaOjoyxcuHDKn2XH629O9+PPqr5j4I2f97SFKeu212WLj5+9ZqZgun8XeqlJvUKz+m1SrzB3/Z577rnbbfd32tb1QwElLQTuB75i+y1J64AbAZf3W4HPA53OBFypM8m2Xxbs9cB6gP7+fg8MDBzYNjQ0RPv6ZLp5yN1suGbZGLfuaMazGrvtdddlA7PXzBRM9+9CLzWpV2hWv03qFeZHv13d9STpfbRC4ju2/xTA9hu237P9C+CbwMqy+whwStvwJcDuUl/SoX7QGEkLgOOBfd30HBER09PNXU8CNgAv2v6jtvqitt0+C+wsyw8Bg+VOplNpXbR+xvYe4G1J55RjXg482DZmTVm+GHjMszFXFhERU9bNHMdHgc8BOyQ9V2p/AFwqaTmtKaJdwBcAbD8v6V7gBVp3TF1t+70y7irgDuAYWtcttpT6BuAuScO0ziQGu+g3IiJmYMZBYfuv6HwN4ZHKmJuAmzrUt9G6EH5o/R3gkpn2GBER3cs3syMioipBERERVQmKiIioSlBERERVgiIiIqoSFBERUZWgiIiIqgRFRERUJSgiIqIqQREREVXNeJ51HJGWzvEj3a9ZNsYV1z7Mrps/Pac/N6LpckYRERFVCYqIiKhKUERERFWCIiIiqhIUERFRlaCIiIiqBEVERFTlexTxK2euv7/RLt/hiCZqxBmFpNWSXpI0LOnaXvcTEfGrZN4HhaSjgP8OfBI4A7hU0hm97Soi4ldHE6aeVgLDtn8EIGkzcCHwQk+7ipiBqU57jT9uZLZkyiu6Idu97qFK0sXAatv/vqx/Djjb9pfa9lkLrC2rHwFeajvEycCP56jd2dCkfpvUKzSr3yb1Cs3qt0m9wtz1+09tf6jThiacUahD7aB0s70eWN9xsLTNdv/haOxwaFK/TeoVmtVvk3qFZvXbpF5hfvQ7769RACPAKW3rS4DdPeolIuJXThOC4lngdEmnSno/MAg81OOeIiJ+Zcz7qSfbY5K+BDwKHAVstP38NA7RcUpqHmtSv03qFZrVb5N6hWb126ReYR70O+8vZkdERG81YeopIiJ6KEERERFVR2xQNO2xH5I2StoraWeve5mMpFMkPS7pRUnPS/pyr3uaiKR/IukZSf+79Pqfe93TVEg6StIPJP15r3uZjKRdknZIek7Stl73UyPpg5Luk/TD8vf3X/a6p4lI+kj5Mx1/vSXpKz3p5Ui8RlEe+/E3wO/Sur32WeBS2/P229ySPg6MAnfaPqvX/dRIWgQssv19Sb8ObAcumo9/vpIEHGt7VNL7gL8Cvmz7qR63ViXp94F+4DjbF/S6nxpJu4B+2/P+S2ySNgHfs/2tchflB2z/fY/bmlT5nfY6rS8b/+1c//wj9YziwGM/bP8DMP7Yj3nL9hPAvl73MRW299j+fll+G3gRWNzbrjpzy2hZfV95zet/HUlaAnwa+FavezmSSDoO+DiwAcD2PzQhJIpVwCu9CAk4coNiMfBa2/oI8/QXWdNJWgr8NvB0j1uZUJnGeQ7YC2y1PW97Lf4b8B+BX/S4j6ky8JeStpfH6cxXHwb+L/A/y7TetyQd2+umpmgQuLtXP/xIDYpJH/sR3ZO0ELgf+Irtt3rdz0Rsv2d7Oa1v9a+UNG+n9iRdAOy1vb3XvUzDR23/Dq0nPF9dplHnowXA7wDrbP828FOgCdcv3w98BvhfverhSA2KPPbjMCvz/fcD37H9p73uZyrKNMMQsLq3nVR9FPhMmfffDPwrSd/ubUt1tneX973AA7SmfuejEWCk7YzyPlrBMd99Evi+7Td61cCRGhR57MdhVC4QbwBetP1Hve6nRtKHJH2wLB8DfAL4YU+bqrB9ne0ltpfS+nv7mO1/2+O2JiTp2HJDA2Ua5zxgXt65Z/v/AK9J+kgpraIZ/7uCS+nhtBM04BEeMzELj/2Yc5LuBgaAkyWNANfb3tDbrib0UeBzwI4y9w/wB7Yf6V1LE1oEbCp3jfwacK/teX/LaYP0AQ+0/u3AAuBPbP9Fb1uq+g/Ad8o/IH8E/Lse91Ml6QO07t78Qk/7OBJvj42IiNlzpE49RUTELElQREREVYIiIiKqEhQREVGVoIiIiKoERUREVCUoIiKi6v8Deojc91mDiUkAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_grp[\"outperform_ratio\"].hist();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "rgeUqBIi4cbq",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 265
    },
    "id": "rgeUqBIi4cbq",
    "outputId": "6050a6f8-115e-4edb-c162-302aa7f69d82"
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYoAAAD4CAYAAADy46FuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAY3UlEQVR4nO3dcXCc9X3n8fenVkIM1GBw2LqW5+Qebq5gXa+xxrjHXEY5p9g9GMwf0IohwfR845Zxc+SqTmK3f3guGd/A3BFaegczntjFEIpxnWTwlLqJz7BDO4MNNklOGMdFDT6s2MWhdoiVKwT5vvfH/nSzWq9+knYfabXl85rZ0bPf5/n99F3J44+e3/NopYjAzMxsPD/T6gbMzGx2c1CYmVmWg8LMzLIcFGZmluWgMDOzrI5WN1C0BQsWRFdXV6vbAOAnP/kJl112WavbyHKPxWmHPt1jMdqhR5han0eOHHk7Ij5ad2dE/JN6LF++PGaL559/vtUtTMg9Fqcd+nSPxWiHHiOm1idwOMb5f9VLT2ZmluWgMDOzLAeFmZllTRgUknZIOiPp1Tr7fl9SSFpQVdssaVDScUmrq+rLJQ2kfQ9LUqpfIunpVD8kqatqzDpJr6fHuqZfrZmZTdlkzigeA9bUFiUtBn4NeLOqdh3QB1yfxjwiaU7a/SiwAViaHqNzrgfORcS1wEPAA2muq4AtwA3ACmCLpPlTe3lmZtasCYMiIl4AztbZ9RDweaD6XQXXArsi4r2IeAMYBFZIWgjMi4gX09X1x4HbqsbsTNt7gFXpbGM1sD8izkbEOWA/dQLLzMymV0O/RyHpVuAHEfHdtII0ahFwsOr5UKq9n7Zr66NjTgJExIikd4Crq+t1xtT2s4HK2QqlUolyudzIyyrc8PDwrOllPO6xOO3Qp3ssRjv0CMX1OeWgkHQp8IfATfV216lFpt7omLHFiG3ANoCenp7o7e2td9iMK5fLzJZexuMei9MOfbrHYrRDj1Bcn43c9fTPgSXAdyWdADqBVyT9HJWf+hdXHdsJnEr1zjp1qsdI6gCuoLLUNd5cZmY2g6Z8RhERA8A1o89TWPRExNuS9gJ/JunLwM9TuWj9UkRckHRe0krgEHA38Cdpir3AOuBF4HbguYgISd8E/kvVBeybgM2NvMip6Nr0bGFz9XePcM8k5ztx/82FfV4zsyJNGBSSngJ6gQWShoAtEbG93rERcVTSbuA1YATYGBEX0u57qdxBNRfYlx4A24EnJA1SOZPoS3OdlfQl4OV03Bcjot5FdTMzm0YTBkVE3DnB/q6a51uBrXWOOwwsq1N/F7hjnLl3ADsm6tHMzKaPfzPbzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCxrwqCQtEPSGUmvVtX+q6TvSfpfkr4h6cqqfZslDUo6Lml1VX25pIG072FJSvVLJD2d6ockdVWNWSfp9fRYV9SLNjOzyZvMGcVjwJqa2n5gWUT8S+Bvgc0Akq4D+oDr05hHJM1JYx4FNgBL02N0zvXAuYi4FngIeCDNdRWwBbgBWAFskTR/6i/RzMyaMWFQRMQLwNma2rciYiQ9PQh0pu21wK6IeC8i3gAGgRWSFgLzIuLFiAjgceC2qjE70/YeYFU621gN7I+IsxFxjko41QaWmZlNs44C5vj3wNNpexGV4Bg1lGrvp+3a+uiYkwARMSLpHeDq6nqdMWNI2kDlbIVSqUS5XG74xfR3j0x80CSV5k5+vmZ6bsbw8HDLPvdktUOP0B59usditEOPUFyfTQWFpD8ERoAnR0t1DotMvdExY4sR24BtAD09PdHb2zt+0xO4Z9OzDY+t1d89woMDk/sSn7irt7DPOxXlcplmvl4zoR16hPbo0z0Wox16hOL6bPiup3Rx+RbgrrScBJWf+hdXHdYJnEr1zjr1MWMkdQBXUFnqGm8uMzObQQ0FhaQ1wBeAWyPi/1Tt2gv0pTuZllC5aP1SRJwGzktama4/3A08UzVm9I6m24HnUvB8E7hJ0vx0EfumVDMzsxk04bqIpKeAXmCBpCEqdyJtBi4B9qe7XA9GxO9ExFFJu4HXqCxJbYyIC2mqe6ncQTUX2JceANuBJyQNUjmT6AOIiLOSvgS8nI77YkSMuahuZmbTb8KgiIg765S3Z47fCmytUz8MLKtTfxe4Y5y5dgA7JurRzMymj38z28zMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsa8KgkLRD0hlJr1bVrpK0X9Lr6eP8qn2bJQ1KOi5pdVV9uaSBtO9hSUr1SyQ9neqHJHVVjVmXPsfrktYV9qrNzGzSJnNG8Riwpqa2CTgQEUuBA+k5kq4D+oDr05hHJM1JYx4FNgBL02N0zvXAuYi4FngIeCDNdRWwBbgBWAFsqQ4kMzObGRMGRUS8AJytKa8FdqbtncBtVfVdEfFeRLwBDAIrJC0E5kXEixERwOM1Y0bn2gOsSmcbq4H9EXE2Is4B+7k4sMzMbJp1NDiuFBGnASLitKRrUn0RcLDquKFUez9t19ZHx5xMc41Iege4urpeZ8wYkjZQOVuhVCpRLpcbfFnQ3z3S8NhapbmTn6+ZnpsxPDzcss89We3QI7RHn+6xGO3QIxTXZ6NBMR7VqUWm3uiYscWIbcA2gJ6enujt7Z2w0fHcs+nZhsfW6u8e4cGByX2JT9zVW9jnnYpyuUwzX6+Z0A49Qnv06R6L0Q49QnF9NnrX01tpOYn08UyqDwGLq47rBE6lemed+pgxkjqAK6gsdY03l5mZzaBGg2IvMHoX0jrgmap6X7qTaQmVi9YvpWWq85JWpusPd9eMGZ3rduC5dB3jm8BNkuani9g3pZqZmc2gCddFJD0F9AILJA1RuRPpfmC3pPXAm8AdABFxVNJu4DVgBNgYERfSVPdSuYNqLrAvPQC2A09IGqRyJtGX5jor6UvAy+m4L0ZE7UV1MzObZhMGRUTcOc6uVeMcvxXYWqd+GFhWp/4uKWjq7NsB7JioRzMzmz7+zWwzM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmllX0Hy6yBnUV+AeTpqK/e4TelnxmM2sXPqMwM7MsB4WZmWU5KMzMLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmltVUUEj6T5KOSnpV0lOSPiLpKkn7Jb2ePs6vOn6zpEFJxyWtrqovlzSQ9j0sSal+iaSnU/2QpK5m+jUzs6lrOCgkLQL+I9ATEcuAOUAfsAk4EBFLgQPpOZKuS/uvB9YAj0iak6Z7FNgALE2PNam+HjgXEdcCDwEPNNqvmZk1ptmlpw5grqQO4FLgFLAW2Jn27wRuS9trgV0R8V5EvAEMAiskLQTmRcSLERHA4zVjRufaA6waPdswM7OZ0fB7PUXEDyT9N+BN4B+Bb0XEtySVIuJ0Oua0pGvSkEXAwaophlLt/bRdWx8dczLNNSLpHeBq4O3qXiRtoHJGQqlUolwuN/qy6O8eaXhsrdLcYuebDqW5NPX1mgnDw8Ozvkdojz7dYzHaoUcors+GgyJde1gLLAF+BPy5pE/nhtSpRaaeGzO2ELEN2AbQ09MTvb29mTby7inwzfn6u0d4cGB2v+9if/cIv9HE12smlMtlmvmezpR26NM9FqMdeoTi+mxm6elTwBsR8cOIeB/4OvCvgbfSchLp45l0/BCwuGp8J5WlqqG0XVsfMyYtb10BnG2iZzMzm6JmguJNYKWkS9N1g1XAMWAvsC4dsw54Jm3vBfrSnUxLqFy0fiktU52XtDLNc3fNmNG5bgeeS9cxzMxshjRzjeKQpD3AK8AI8G0qyz+XA7slracSJnek449K2g28lo7fGBEX0nT3Ao8Bc4F96QGwHXhC0iCVM4m+Rvs1M7PGNLWAHhFbgC015feonF3UO34rsLVO/TCwrE79XVLQmJlZa/g3s83MLMtBYWZmWQ4KMzPLclCYmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMspoKCklXStoj6XuSjkn6VUlXSdov6fX0cX7V8ZslDUo6Lml1VX25pIG072FJSvVLJD2d6ockdTXTr5mZTV2zZxR/DPxVRPwL4JeBY8Am4EBELAUOpOdIug7oA64H1gCPSJqT5nkU2AAsTY81qb4eOBcR1wIPAQ802a+ZmU1Rw0EhaR7wCWA7QET8NCJ+BKwFdqbDdgK3pe21wK6IeC8i3gAGgRWSFgLzIuLFiAjg8Zoxo3PtAVaNnm2YmdnM6Ghi7C8APwT+VNIvA0eA+4BSRJwGiIjTkq5Jxy8CDlaNH0q199N2bX10zMk014ikd4CrgberG5G0gcoZCaVSiXK53PCL6u8eaXhsrdLcYuebDqW5NPX1mgnDw8Ozvkdojz7dYzHaoUcors9mgqID+Djw2Yg4JOmPSctM46h3JhCZem7M2ELENmAbQE9PT/T29mbayLtn07MNj63V3z3CgwPNfImnX3/3CL/RxNdrJpTLZZr5ns6UdujTPRajHXqE4vps5hrFEDAUEYfS8z1UguOttJxE+nim6vjFVeM7gVOp3lmnPmaMpA7gCuBsEz2bmdkUNRwUEfH3wElJH0ulVcBrwF5gXaqtA55J23uBvnQn0xIqF61fSstU5yWtTNcf7q4ZMzrX7cBz6TqGmZnNkGbXRT4LPCnpw8D3gd+iEj67Ja0H3gTuAIiIo5J2UwmTEWBjRFxI89wLPAbMBfalB1QulD8haZDKmURfk/2amdkUNRUUEfEdoKfOrlXjHL8V2FqnfhhYVqf+LilozMysNfyb2WZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWU0HhaQ5kr4t6S/S86sk7Zf0evo4v+rYzZIGJR2XtLqqvlzSQNr3sCSl+iWSnk71Q5K6mu3XzMympqOAOe4DjgHz0vNNwIGIuF/SpvT8C5KuA/qA64GfB/6npF+MiAvAo8AG4CDwl8AaYB+wHjgXEddK6gMeAH6zgJ6tStemZ1vyeU/cf3NLPq+ZTU1TZxSSOoGbga9UldcCO9P2TuC2qvquiHgvIt4ABoEVkhYC8yLixYgI4PGaMaNz7QFWjZ5tmJnZzGj2jOKPgM8DP1tVK0XEaYCIOC3pmlRfROWMYdRQqr2ftmvro2NOprlGJL0DXA28Xd2EpA1UzkgolUqUy+WGX1B/90jDY2uV5hY733RoZY+T/T4NDw839T2dKe3Qp3ssRjv0CMX12XBQSLoFOBMRRyT1TmZInVpk6rkxYwsR24BtAD09PdHbO5l26runwGWY/u4RHhwoYnVv+rSyxxN39U7quHK5TDPf05nSDn26x2K0Q49QXJ/N/A9xI3CrpH8HfASYJ+mrwFuSFqaziYXAmXT8ELC4anwncCrVO+vUq8cMSeoArgDONtGzmZlNUcPXKCJic0R0RkQXlYvUz0XEp4G9wLp02DrgmbS9F+hLdzItAZYCL6VlqvOSVqbrD3fXjBmd6/b0OS46ozAzs+kzHWsO9wO7Ja0H3gTuAIiIo5J2A68BI8DGdMcTwL3AY8BcKnc77Uv17cATkgapnEn0TUO/ZmaWUUhQREQZKKftfwBWjXPcVmBrnfphYFmd+rukoDEzs9bwb2abmVmWg8LMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU1HBSSFkt6XtIxSUcl3ZfqV0naL+n19HF+1ZjNkgYlHZe0uqq+XNJA2vewJKX6JZKeTvVDkrqaeK1mZtaAZs4oRoD+iPglYCWwUdJ1wCbgQEQsBQ6k56R9fcD1wBrgEUlz0lyPAhuApemxJtXXA+ci4lrgIeCBJvo1M7MGNBwUEXE6Il5J2+eBY8AiYC2wMx22E7gtba8FdkXEexHxBjAIrJC0EJgXES9GRACP14wZnWsPsGr0bMPMzGaGKv83NzlJZUnoBWAZ8GZEXFm171xEzJf034GDEfHVVN8O7ANOAPdHxKdS/d8AX4iIWyS9CqyJiKG07++AGyLi7ZrPv4HKGQmlUmn5rl27Gn4tAz94p+GxtUpz4a1/LGy6adHKHrsXXTGp44aHh7n88sunuZvmtUOf7rEY7dAjTK3PT37yk0cioqfevo5mG5F0OfA14HMR8ePMD/z1dkSmnhszthCxDdgG0NPTE729vRN0Pb57Nj3b8Nha/d0jPDjQ9Jd4WrWyxxN39U7quHK5TDPf05nSDn26x2K0Q49QXJ9N3fUk6UNUQuLJiPh6Kr+VlpNIH8+k+hCwuGp4J3Aq1Tvr1MeMkdQBXAGcbaZnMzObmmbuehKwHTgWEV+u2rUXWJe21wHPVNX70p1MS6hctH4pIk4D5yWtTHPeXTNmdK7bgeeiiLUyMzObtGbWHG4EPgMMSPpOqv0BcD+wW9J64E3gDoCIOCppN/AalTumNkbEhTTuXuAxYC6V6xb7Un078ISkQSpnEn1N9GtmZg1oOCgi4m+ofw0BYNU4Y7YCW+vUD1O5EF5bf5cUNGZm1hr+zWwzM8ua3bfk2D9pXZO8w6y/e6TQu9FO3H9zYXOZfRD4jMLMzLIcFGZmluWgMDOzLAeFmZllOSjMzCzLQWFmZlkOCjMzy3JQmJlZloPCzMyyHBRmZpbloDAzsywHhZmZZTkozMwsy0FhZmZZDgozM8tyUJiZWZb/cJF94Ez2DyZN1WT+wJL/aJK1I59RmJlZloPCzMyy2iIoJK2RdFzSoKRNre7HzOyDZNZfo5A0B/gfwK8BQ8DLkvZGxGut7cxs6qbr+shEfG3EmjHrgwJYAQxGxPcBJO0C1gIOCrNJGg2oyVxwL5pDqv0pIlrdQ5ak24E1EfEf0vPPADdExO9WHbMB2JCefgw4PuON1rcAeLvVTUzAPRanHfp0j8Vohx5han3+s4j4aL0d7XBGoTq1MekWEduAbTPTzuRJOhwRPa3uI8c9Fqcd+nSPxWiHHqG4PtvhYvYQsLjqeSdwqkW9mJl94LRDULwMLJW0RNKHgT5gb4t7MjP7wJj1S08RMSLpd4FvAnOAHRFxtMVtTdasWw6rwz0Wpx36dI/FaIceoaA+Z/3FbDMza612WHoyM7MWclCYmVmWg2IatMNbjkjaIemMpFdb3ct4JC2W9LykY5KOSrqv1T3VkvQRSS9J+m7q8T+3uqfxSJoj6duS/qLVvYxH0glJA5K+I+lwq/upR9KVkvZI+l76t/mrre6pmqSPpa/f6OPHkj7X1Jy+RlGs9JYjf0vVW44Ad862txyR9AlgGHg8Ipa1up96JC0EFkbEK5J+FjgC3DabvpaSBFwWEcOSPgT8DXBfRBxscWsXkfR7QA8wLyJuaXU/9Ug6AfRExKz9ZTZJO4G/joivpDsxL42IH7W4rbrS/0c/oPJLyv+70Xl8RlG8//+WIxHxU2D0LUdmlYh4ATjb6j5yIuJ0RLySts8Dx4BFre1qrKgYTk8/lB6z7qcvSZ3AzcBXWt1LO5M0D/gEsB0gIn46W0MiWQX8XTMhAQ6K6bAIOFn1fIhZ9p9bO5LUBfwKcKjFrVwkLel8BzgD7I+IWdcj8EfA54H/2+I+JhLAtyQdSW/NM9v8AvBD4E/TMt5XJF3W6qYy+oCnmp3EQVG8Cd9yxKZG0uXA14DPRcSPW91PrYi4EBH/isq7BqyQNKuW8iTdApyJiCOt7mUSboyIjwO/DmxMS6SzSQfwceDRiPgV4CfAbL0O+WHgVuDPm53LQVE8v+VIgdK6/9eAJyPi663uJyctQZSBNa3t5CI3Arem9f9dwL+V9NXWtlRfRJxKH88A36CylDubDAFDVWeNe6gEx2z068ArEfFWsxM5KIrntxwpSLpQvB04FhFfbnU/9Uj6qKQr0/Zc4FPA91raVI2I2BwRnRHRReXf43MR8ekWt3URSZelmxZIyzk3AbPqrryI+HvgpKSPpdIqZu+fPLiTApadoA3ewqPdtMtbjkh6CugFFkgaArZExPbWdnWRG4HPAAPpGgDAH0TEX7aupYssBHamu0t+BtgdEbP29tNZrgR8o/LzAR3An0XEX7W2pbo+CzyZfhD8PvBbLe7nIpIupXLn5W8XMp9vjzUzsxwvPZmZWZaDwszMshwUZmaW5aAwM7MsB4WZmWU5KMzMLMtBYWZmWf8Pk4d6O2EvPxcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "df_grp[\"outperform_ratio_smooth\"].hist();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "g28Q9Jds1IuP",
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 685
    },
    "id": "g28Q9Jds1IuP",
    "outputId": "33e41c47-a8d9-477e-b7b1-df07b4d7ec4c"
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>black_card_text</th>\n",
       "      <th>white_card_text</th>\n",
       "      <th>pair_prior</th>\n",
       "      <th>pair_count</th>\n",
       "      <th>white_prior</th>\n",
       "      <th>text</th>\n",
       "      <th>white_count</th>\n",
       "      <th>outperform_sub</th>\n",
       "      <th>outperform_ratio</th>\n",
       "      <th>outperform_ratio_smooth</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>361415</th>\n",
       "      <td>What's a girl's best friend?</td>\n",
       "      <td>The hose</td>\n",
       "      <td>1.00</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0517</td>\n",
       "      <td>What's a girl's best friend? The hose</td>\n",
       "      <td>1316</td>\n",
       "      <td>0.9483</td>\n",
       "      <td>7.2525</td>\n",
       "      <td>6.9118</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>660342</th>\n",
       "      <td>Madam President, the asteroid is headed directly for Earth and there's only one thing that can stop it: __.</td>\n",
       "      <td>A time travel paradox</td>\n",
       "      <td>1.00</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0655</td>\n",
       "      <td>Madam President, the asteroid is headed directly for Earth and there's only one thing that can stop it: A time travel paradox.</td>\n",
       "      <td>1298</td>\n",
       "      <td>0.9345</td>\n",
       "      <td>6.6471</td>\n",
       "      <td>6.2514</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>200181</th>\n",
       "      <td>And I would have gotten away with it, too, if it hadn't been for __!</td>\n",
       "      <td>Aaron Burr</td>\n",
       "      <td>1.00</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0677</td>\n",
       "      <td>And I would have gotten away with it, too, if it hadn't been for Aaron Burr!</td>\n",
       "      <td>1241</td>\n",
       "      <td>0.9323</td>\n",
       "      <td>6.5598</td>\n",
       "      <td>6.1562</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>610701</th>\n",
       "      <td>As reparations for slavery, all African Americans will receive __.</td>\n",
       "      <td>A strong horse and enough rations for thirty days</td>\n",
       "      <td>0.75</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0321</td>\n",
       "      <td>As reparations for slavery, all African Americans will receive A strong horse and enough rations for thirty days.</td>\n",
       "      <td>1278</td>\n",
       "      <td>0.7179</td>\n",
       "      <td>6.4354</td>\n",
       "      <td>6.1925</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>334602</th>\n",
       "      <td>When I was a kid, we used to play Cowboys and __.</td>\n",
       "      <td>Hot dog, the game you can eat, but only once</td>\n",
       "      <td>0.75</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0374</td>\n",
       "      <td>When I was a kid, we used to play Cowboys and Hot dog, the game you can eat, but only once.</td>\n",
       "      <td>1230</td>\n",
       "      <td>0.7126</td>\n",
       "      <td>6.1864</td>\n",
       "      <td>5.9142</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>353211</th>\n",
       "      <td>When I am a billionaire, I shall erect a 50-foot statue to commemorate __.</td>\n",
       "      <td>A diverse group of female friends casually discussing the side effects of birth control</td>\n",
       "      <td>1.00</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0788</td>\n",
       "      <td>When I am a billionaire, I shall erect a 50-foot statue to commemorate A diverse group of female friends casually discussing the side effects of birth control.</td>\n",
       "      <td>1244</td>\n",
       "      <td>0.9212</td>\n",
       "      <td>6.1529</td>\n",
       "      <td>5.7122</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>732661</th>\n",
       "      <td>What brought the orgy to a grinding halt?</td>\n",
       "      <td>A diverse group of female friends casually discussing the side effects of birth control</td>\n",
       "      <td>1.00</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0788</td>\n",
       "      <td>What brought the orgy to a grinding halt? A diverse group of female friends casually discussing the side effects of birth control</td>\n",
       "      <td>1244</td>\n",
       "      <td>0.9212</td>\n",
       "      <td>6.1529</td>\n",
       "      <td>5.7122</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>159067</th>\n",
       "      <td>Mom's to-do list: \\n • Buy groceries \\n • Clean up __ \\n • Soccer practice</td>\n",
       "      <td>The economy</td>\n",
       "      <td>1.00</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0794</td>\n",
       "      <td>Mom's to-do list: \\n • Buy groceries \\n • Clean up The economy \\n • Soccer practice</td>\n",
       "      <td>1197</td>\n",
       "      <td>0.9206</td>\n",
       "      <td>6.1327</td>\n",
       "      <td>5.6903</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>408892</th>\n",
       "      <td>Members of New York's social elite are paying thousands of dollars just to experience __.</td>\n",
       "      <td>A 0.7 waist-to-hip ratio</td>\n",
       "      <td>0.75</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0402</td>\n",
       "      <td>Members of New York's social elite are paying thousands of dollars just to experience A 0.7 waist-to-hip ratio.</td>\n",
       "      <td>1243</td>\n",
       "      <td>0.7098</td>\n",
       "      <td>6.0617</td>\n",
       "      <td>5.7748</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>756254</th>\n",
       "      <td>When I pooped, what came out of my butt?</td>\n",
       "      <td>These awesome gadgets that track my every move</td>\n",
       "      <td>0.75</td>\n",
       "      <td>4</td>\n",
       "      <td>0.0419</td>\n",
       "      <td>When I pooped, what came out of my butt? These awesome gadgets that track my every move</td>\n",
       "      <td>1265</td>\n",
       "      <td>0.7081</td>\n",
       "      <td>5.9903</td>\n",
       "      <td>5.6950</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                                                                                                    black_card_text                                                                          white_card_text  pair_prior  pair_count  white_prior                                                                                                                                                             text  white_count  outperform_sub  outperform_ratio  outperform_ratio_smooth\n",
       "361415                                                                                 What's a girl's best friend?                                                                                 The hose        1.00           4       0.0517                                                                                                                            What's a girl's best friend? The hose         1316          0.9483            7.2525                   6.9118\n",
       "660342  Madam President, the asteroid is headed directly for Earth and there's only one thing that can stop it: __.                                                                    A time travel paradox        1.00           4       0.0655                                   Madam President, the asteroid is headed directly for Earth and there's only one thing that can stop it: A time travel paradox.         1298          0.9345            6.6471                   6.2514\n",
       "200181                                         And I would have gotten away with it, too, if it hadn't been for __!                                                                               Aaron Burr        1.00           4       0.0677                                                                                     And I would have gotten away with it, too, if it hadn't been for Aaron Burr!         1241          0.9323            6.5598                   6.1562\n",
       "610701                                           As reparations for slavery, all African Americans will receive __.                                        A strong horse and enough rations for thirty days        0.75           4       0.0321                                                As reparations for slavery, all African Americans will receive A strong horse and enough rations for thirty days.         1278          0.7179            6.4354                   6.1925\n",
       "334602                                                            When I was a kid, we used to play Cowboys and __.                                             Hot dog, the game you can eat, but only once        0.75           4       0.0374                                                                      When I was a kid, we used to play Cowboys and Hot dog, the game you can eat, but only once.         1230          0.7126            6.1864                   5.9142\n",
       "353211                                   When I am a billionaire, I shall erect a 50-foot statue to commemorate __.  A diverse group of female friends casually discussing the side effects of birth control        1.00           4       0.0788  When I am a billionaire, I shall erect a 50-foot statue to commemorate A diverse group of female friends casually discussing the side effects of birth control.         1244          0.9212            6.1529                   5.7122\n",
       "732661                                                                    What brought the orgy to a grinding halt?  A diverse group of female friends casually discussing the side effects of birth control        1.00           4       0.0788                                What brought the orgy to a grinding halt? A diverse group of female friends casually discussing the side effects of birth control         1244          0.9212            6.1529                   5.7122\n",
       "159067                                   Mom's to-do list: \\n • Buy groceries \\n • Clean up __ \\n • Soccer practice                                                                              The economy        1.00           4       0.0794                                                                              Mom's to-do list: \\n • Buy groceries \\n • Clean up The economy \\n • Soccer practice         1197          0.9206            6.1327                   5.6903\n",
       "408892                    Members of New York's social elite are paying thousands of dollars just to experience __.                                                                 A 0.7 waist-to-hip ratio        0.75           4       0.0402                                                  Members of New York's social elite are paying thousands of dollars just to experience A 0.7 waist-to-hip ratio.         1243          0.7098            6.0617                   5.7748\n",
       "756254                                                                     When I pooped, what came out of my butt?                                           These awesome gadgets that track my every move        0.75           4       0.0419                                                                          When I pooped, what came out of my butt? These awesome gadgets that track my every move         1265          0.7081            5.9903                   5.6950"
      ]
     },
     "execution_count": 28,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_grp.sort_values([\"outperform_ratio\",\"pair_prior\",\"white_prior\"],ascending=False).head(10)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "WWit1Jd-0c44",
   "metadata": {
    "id": "WWit1Jd-0c44"
   },
   "source": [
    "#### save output\n",
    "* Keep alt targets/leaks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "ExuBcotA0dWJ",
   "metadata": {
    "id": "ExuBcotA0dWJ"
   },
   "outputs": [],
   "source": [
    "# df_grp.sample(frac=1).to_csv(\"cah_outperform_min6.csv.gz\",index=False,compression=\"gzip\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea520232",
   "metadata": {
    "id": "ea520232"
   },
   "source": [
    "#### Whites prior: \n",
    "* 20.5% accuracy on test set!!;\n",
    "* 23% after min 3 filtering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "2bf2c879",
   "metadata": {
    "id": "2bf2c879"
   },
   "outputs": [
    {
     "ename": "NameError",
     "evalue": "name 'df_test' is not defined",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[1;31mNameError\u001b[0m                                 Traceback (most recent call last)",
      "Input \u001b[1;32mIn [30]\u001b[0m, in \u001b[0;36m<cell line: 2>\u001b[1;34m()\u001b[0m\n\u001b[0;32m      1\u001b[0m df_white_prior \u001b[38;5;241m=\u001b[39m df\u001b[38;5;241m.\u001b[39mgroupby([\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mwhite_card_text\u001b[39m\u001b[38;5;124m\"\u001b[39m], as_index\u001b[38;5;241m=\u001b[39m\u001b[38;5;28;01mFalse\u001b[39;00m)[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mwon\u001b[39m\u001b[38;5;124m\"\u001b[39m]\u001b[38;5;241m.\u001b[39mmean()\u001b[38;5;241m.\u001b[39mrename(columns\u001b[38;5;241m=\u001b[39m{\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mwon\u001b[39m\u001b[38;5;124m\"\u001b[39m:\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mwhite_prior\u001b[39m\u001b[38;5;124m\"\u001b[39m})\u001b[38;5;241m.\u001b[39mset_index(\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mwhite_card_text\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[1;32m----> 2\u001b[0m df_test \u001b[38;5;241m=\u001b[39m \u001b[43mdf_test\u001b[49m\u001b[38;5;241m.\u001b[39mjoin(df_white_prior,on\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mwhite_card_text\u001b[39m\u001b[38;5;124m\"\u001b[39m,how\u001b[38;5;241m=\u001b[39m\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mleft\u001b[39m\u001b[38;5;124m\"\u001b[39m)\n\u001b[0;32m      3\u001b[0m prior \u001b[38;5;241m=\u001b[39m df_test[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mwhite_prior\u001b[39m\u001b[38;5;124m\"\u001b[39m]\u001b[38;5;241m.\u001b[39mmean()\n\u001b[0;32m      4\u001b[0m df_test[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mwhite_prior\u001b[39m\u001b[38;5;124m\"\u001b[39m] \u001b[38;5;241m=\u001b[39m df_test[\u001b[38;5;124m\"\u001b[39m\u001b[38;5;124mwhite_prior\u001b[39m\u001b[38;5;124m\"\u001b[39m]\u001b[38;5;241m.\u001b[39mfillna(prior)\n",
      "\u001b[1;31mNameError\u001b[0m: name 'df_test' is not defined"
     ]
    }
   ],
   "source": [
    "df_white_prior = df.groupby([\"white_card_text\"], as_index=False)[\"won\"].mean().rename(columns={\"won\":\"white_prior\"}).set_index(\"white_card_text\")\n",
    "df_test = df_test.join(df_white_prior,on=\"white_card_text\",how=\"left\")\n",
    "prior = df_test[\"white_prior\"].mean()\n",
    "df_test[\"white_prior\"] = df_test[\"white_prior\"].fillna(prior)\n",
    "print(df_test.shape[0],\"test shape after join\")\n",
    "print(\"Prior Acc:\",df_test.sort_values(\"white_prior\",ascending=False).groupby(\"fake_round_id\").head(1)[\"won\"].mean())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "25c1ce14",
   "metadata": {
    "id": "25c1ce14"
   },
   "source": [
    "### keep 1s (for train) ?\n",
    "* MAY want to filter - e.g. min # cases of it being picked (+- keeping that for use for sample weight)\n",
    "* Keep all data for now, (to allow for grouping by rounds if we want, since this is a ranking problem) \n",
    "\n",
    "* temp"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "collapsed_sections": [
    "a1d8eb98"
   ],
   "name": "1.1-cah_extract_data_outperformers.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}