cuml/ridge_regression_demo.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Ridge Regression\n",
    "\n",
    "**Ridge Regression** extends linear regression by providing L2 regularization of the coefficients. It can reduce the variance of the predictors, and improves the conditioning of the problem.\n",
    "\n",
    "The model can take array-like objects, either in host as NumPy arrays or in device (as Numba or cuda_array_interface-compliant), as well as cuDF DataFrames as the input. \n",
    "\n",
    "For information about cuDF, refer to the [cuDF documentation](https://rapidsai.github.io/projects/cudf/en/latest/).\n",
    "\n",
    "For information about cuML's ridge regression API: https://rapidsai.github.io/projects/cuml/en/0.11.0/api.html#cuml.Ridge."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import cudf\n",
    "from cuml import make_regression, train_test_split\n",
    "from cuml.metrics.regression import r2_score\n",
    "from cuml.linear_model import Ridge as cuRidge\n",
    "from sklearn.linear_model import Ridge as skRidge"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Define Parameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "n_samples = 2**20\n",
    "n_features = 399\n",
    "\n",
    "random_state = 23"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Generate Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "X, y = make_regression(n_samples=n_samples, n_features=n_features, random_state=0)\n",
    "\n",
    "X = cudf.DataFrame.from_gpu_matrix(X)\n",
    "y = cudf.DataFrame.from_gpu_matrix(y)[0]\n",
    "\n",
    "X_cudf, X_cudf_test, y_cudf, y_cudf_test = train_test_split(X, y, test_size = 0.2, random_state=random_state)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Copy dataset from GPU memory to host memory.\n",
    "# This is done to later compare CPU and GPU results.\n",
    "X_train = X_cudf.to_pandas()\n",
    "X_test = X_cudf_test.to_pandas()\n",
    "y_train = y_cudf.to_pandas()\n",
    "y_test = y_cudf_test.to_pandas()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Scikit-learn Model\n",
    "\n",
    "### Fit, predit and evaluate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "ridge_sk = skRidge(fit_intercept=False, normalize=True, alpha=0.1)\n",
    "\n",
    "ridge_sk.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "predict_sk= ridge_sk.predict(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "r2_score_sk = r2_score(y_cudf_test, predict_sk)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## cuML Model\n",
    "\n",
    "### Fit, predit and evaluate"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "%%time\n",
    "# Run the cuml ridge regression model to fit the training dataset.\n",
    "# Eig is the faster algorithm, but svd is more accurate.\n",
    "# In general svd uses significantly more memory and is slower than eig.\n",
    "# If using CUDA 10.1, the memory difference is even bigger than in the other supported CUDA versions\n",
    "ridge_cuml = cuRidge(fit_intercept=False, normalize=True, solver='eig', alpha=0.1)\n",
    "\n",
    "ridge_cuml.fit(X_cudf, y_cudf)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "predict_cuml = ridge_cuml.predict(X_cudf_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "r2_score_cuml = r2_score(y_cudf_test, predict_cuml)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Compare Results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"R^2 score (SKL):  %s\" % r2_score_sk)\n",
    "print(\"R^2 score (cuML): %s\" % r2_score_cuml)"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}