12.mnist_sklearn_wrapper.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 程序说明\n",
    "名称：使用sklearn wrapper做参数搜索\n",
    "\n",
    "时间：2016年11月17日\n",
    "\n",
    "说明：建造一个简单的卷积模型，通过使用sklearn的GridSearchCV去发现最好的模型。\n",
    "\n",
    "数据集：MNIST"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1.加载keras模块"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n",
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/cross_validation.py:44: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. Also note that the interface of the new CV iterators are different from that of this module. This module will be removed in 0.20.\n",
      "  \"This module will be removed in 0.20.\", DeprecationWarning)\n",
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/grid_search.py:43: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.\n",
      "  DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "from __future__ import print_function\n",
    "import numpy as np\n",
    "np.random.seed(1337)  # for reproducibility\n",
    "\n",
    "from keras.datasets import mnist\n",
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Dropout, Activation, Flatten\n",
    "from keras.layers import Convolution2D, MaxPooling2D\n",
    "from keras.utils import np_utils\n",
    "from keras.wrappers.scikit_learn import KerasClassifier\n",
    "from sklearn.grid_search import GridSearchCV"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2.变量初始化"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "nb_classes = 10\n",
    "\n",
    "# input image dimensions\n",
    "img_rows, img_cols = 28, 28"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3.准备数据"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# load training data and do basic data normalization\n",
    "(X_train, y_train), (X_test, y_test) = mnist.load_data()\n",
    "X_train = X_train.reshape(X_train.shape[0], img_rows, img_cols, 1)\n",
    "X_test = X_test.reshape(X_test.shape[0], img_rows, img_cols, 1)\n",
    "X_train = X_train.astype('float32')\n",
    "X_test = X_test.astype('float32')\n",
    "X_train /= 255\n",
    "X_test /= 255"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 转换类标号"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# convert class vectors to binary class matrices\n",
    "y_train = np_utils.to_categorical(y_train, nb_classes)\n",
    "y_test = np_utils.to_categorical(y_test, nb_classes)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4.建立模型\n",
    "### 使用Sequential（）\n",
    "构造一个有两个卷积层和若干个全连接层组成的模型，这里全连接的层数是由参数所决定的。\n",
    "\n",
    "dense_layer_sizes：层尺寸的列表。这个列表中对于每个层都有一组数字。\n",
    "\n",
    "nb_filters：每个卷积层中滤波器的个数\n",
    "\n",
    "nb_conv：卷积核的尺寸\n",
    "\n",
    "nb_pool：用于max pooling的池化面积"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def make_model(dense_layer_sizes, nb_filters, nb_conv, nb_pool):\n",
    "    '''Creates model comprised of 2 convolutional layers followed by dense layers\n",
    "    dense_layer_sizes: List of layer sizes. This list has one number for each layer\n",
    "    nb_filters: Number of convolutional filters in each convolutional layer\n",
    "    nb_conv: Convolutional kernel size\n",
    "    nb_pool: Size of pooling area for max pooling\n",
    "    '''\n",
    "\n",
    "    model = Sequential()\n",
    "\n",
    "    model.add(Convolution2D(nb_filters, nb_conv, nb_conv,\n",
    "                            border_mode='valid',\n",
    "                            input_shape=(img_rows, img_cols, 1)))\n",
    "    model.add(Activation('relu'))\n",
    "    model.add(Convolution2D(nb_filters, nb_conv, nb_conv))\n",
    "    model.add(Activation('relu'))\n",
    "    model.add(MaxPooling2D(pool_size=(nb_pool, nb_pool)))\n",
    "    model.add(Dropout(0.25))\n",
    "\n",
    "    model.add(Flatten())\n",
    "    for layer_size in dense_layer_sizes:\n",
    "        model.add(Dense(layer_size))\n",
    "    model.add(Activation('relu'))\n",
    "    model.add(Dropout(0.5))\n",
    "    model.add(Dense(nb_classes))\n",
    "    model.add(Activation('softmax'))\n",
    "\n",
    "    model.compile(loss='categorical_crossentropy',\n",
    "                  optimizer='adadelta',\n",
    "                  metrics=['accuracy'])\n",
    "\n",
    "    return model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5.sklearn接口\n",
    "\n",
    "KerasClassifier()实现了sklearn的分类器接口\n",
    "\n",
    "`keras.wrappers.scikit_learn.KerasClassifier(build_fn=None, **sk_params）`\n",
    "\n",
    "build_fn:可调用的函数或类对象\n",
    "\n",
    "sk_params:模型参数和训练参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "dense_size_candidates = [[32], [64], [32, 32], [64, 64]]\n",
    "my_classifier = KerasClassifier(make_model, batch_size=32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### sklearn中的GridSearchCV函数\n",
    "说明：对估计器的指定参数值进行穷举搜索。\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "validator = GridSearchCV(my_classifier,\n",
    "                         param_grid={'dense_layer_sizes': dense_size_candidates,\n",
    "                                     # nb_epoch可用于调整，即使不是模型构建函数的参数\n",
    "                                     'nb_epoch': [3, 6],\n",
    "                                     'nb_filters': [8],\n",
    "                                     'nb_conv': [3],\n",
    "                                     'nb_pool': [2]},\n",
    "                         scoring='log_loss',\n",
    "                         n_jobs=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 开始拟合"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.8605 - acc: 0.7147    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 11s - loss: 0.5645 - acc: 0.8208    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.4642 - acc: 0.8525    \n",
      " 1536/20000 [=>............................] - ETA: 2s"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19968/20000 [============================>.] - ETA: 0sEpoch 1/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.8284 - acc: 0.7265    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.5357 - acc: 0.8283    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.4524 - acc: 0.8563    \n",
      " 1280/20000 [>.............................] - ETA: 2s"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19968/20000 [============================>.] - ETA: 0sEpoch 1/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.8130 - acc: 0.7311    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.5159 - acc: 0.8359    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.4416 - acc: 0.8602    \n",
      " 1152/20000 [>.............................] - ETA: 3s"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19968/20000 [============================>.] - ETA: 0sEpoch 1/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.8093 - acc: 0.7304    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.4811 - acc: 0.8459    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.4099 - acc: 0.8723    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.3624 - acc: 0.8859    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.3331 - acc: 0.8956    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.3093 - acc: 0.9030    \n",
      "  928/20000 [>.............................] - ETA: 3s"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19936/20000 [============================>.] - ETA: 0sEpoch 1/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.7886 - acc: 0.7393    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.4860 - acc: 0.8451    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.4136 - acc: 0.8712    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.3739 - acc: 0.8827    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.3499 - acc: 0.8924    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.3297 - acc: 0.8989    \n",
      "  800/20000 [>.............................] - ETA: 4s"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19936/20000 [============================>.] - ETA: 0sEpoch 1/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.9260 - acc: 0.6871    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.6032 - acc: 0.8043    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.5158 - acc: 0.8342    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.4425 - acc: 0.8599    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.4088 - acc: 0.8709    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.3644 - acc: 0.8848    \n",
      "  544/20000 [..............................] - ETA: 6s"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20000/20000 [==============================] - 2s     \n",
      "Epoch 1/3\n",
      "40000/40000 [==============================] - 11s - loss: 0.6009 - acc: 0.8104    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 11s - loss: 0.3410 - acc: 0.8968    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.2770 - acc: 0.9162    \n",
      "  256/20000 [..............................] - ETA: 14s"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19904/20000 [============================>.] - ETA: 0sEpoch 1/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.6185 - acc: 0.8061    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.3376 - acc: 0.8999    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.2741 - acc: 0.9193    \n",
      "   32/20000 [..............................] - ETA: 119s"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19936/20000 [============================>.] - ETA: 0sEpoch 1/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.6259 - acc: 0.7990    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.3257 - acc: 0.9015    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 12s - loss: 0.2599 - acc: 0.9230    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19936/20000 [============================>.] - ETA: 0sEpoch 1/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.6295 - acc: 0.7993    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.3693 - acc: 0.8871    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.2988 - acc: 0.9092    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.2542 - acc: 0.9238    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.2246 - acc: 0.9343    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.2026 - acc: 0.9413    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19968/20000 [============================>.] - ETA: 0sEpoch 1/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.5739 - acc: 0.8182    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.3139 - acc: 0.9077    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.2565 - acc: 0.9245    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.2306 - acc: 0.9316    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.2072 - acc: 0.9398    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.1947 - acc: 0.9416    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20000/20000 [==============================] - 2s     \n",
      "Epoch 1/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.6035 - acc: 0.8089    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.3363 - acc: 0.8993    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.2729 - acc: 0.9181    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.2380 - acc: 0.9298    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.2114 - acc: 0.9376    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.1930 - acc: 0.9442    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19904/20000 [============================>.] - ETA: 0sEpoch 1/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.7216 - acc: 0.7599    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.4140 - acc: 0.8687    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.3545 - acc: 0.8897    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19968/20000 [============================>.] - ETA: 0sEpoch 1/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.8014 - acc: 0.7343    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.4586 - acc: 0.8549    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.3886 - acc: 0.8797    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20000/20000 [==============================] - 2s     \n",
      "Epoch 1/3\n",
      "40000/40000 [==============================] - 14s - loss: 0.8124 - acc: 0.7284    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.4838 - acc: 0.8477    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.4148 - acc: 0.8705    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19936/20000 [============================>.] - ETA: 0sEpoch 1/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.7192 - acc: 0.7608    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.4043 - acc: 0.8712    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.3514 - acc: 0.8902    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.3170 - acc: 0.9009    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.2986 - acc: 0.9079    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.2777 - acc: 0.9138    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20000/20000 [==============================] - 2s     \n",
      "Epoch 1/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.7651 - acc: 0.7428    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.4377 - acc: 0.8626    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.3688 - acc: 0.8846    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.3298 - acc: 0.8983    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.3050 - acc: 0.9052    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.2945 - acc: 0.9091    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19968/20000 [============================>.] - ETA: 0sEpoch 1/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.8654 - acc: 0.7107    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.5192 - acc: 0.8338    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.4300 - acc: 0.8638    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.3788 - acc: 0.8795    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.3477 - acc: 0.8908    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.3197 - acc: 0.8999    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19968/20000 [============================>.] - ETA: 0sEpoch 1/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.5614 - acc: 0.8237    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.2812 - acc: 0.9163    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.2251 - acc: 0.9347    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19904/20000 [============================>.] - ETA: 0sEpoch 1/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.5107 - acc: 0.8401    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.2421 - acc: 0.9307    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.1988 - acc: 0.9424    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19936/20000 [============================>.] - ETA: 0sEpoch 1/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.5245 - acc: 0.8351    \n",
      "Epoch 2/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.2639 - acc: 0.9222    \n",
      "Epoch 3/3\n",
      "40000/40000 [==============================] - 13s - loss: 0.2173 - acc: 0.9356    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19904/20000 [============================>.] - ETA: 0sEpoch 1/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.5514 - acc: 0.8266    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.2738 - acc: 0.9178    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 12s - loss: 0.2165 - acc: 0.9365    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.1909 - acc: 0.9453    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.1734 - acc: 0.9492    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 13s - loss: 0.1621 - acc: 0.9533    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20000/20000 [==============================] - 2s     \n",
      "Epoch 1/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.5373 - acc: 0.8282    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.2628 - acc: 0.9222    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.2104 - acc: 0.9392    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.1844 - acc: 0.9455    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 10s - loss: 0.1657 - acc: 0.9530    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.1482 - acc: 0.9576    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "19936/20000 [============================>.] - ETA: 0sEpoch 1/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.5453 - acc: 0.8316    \n",
      "Epoch 2/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.2769 - acc: 0.9198    \n",
      "Epoch 3/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.2206 - acc: 0.9356    \n",
      "Epoch 4/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.1952 - acc: 0.9447    \n",
      "Epoch 5/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.1756 - acc: 0.9485    \n",
      "Epoch 6/6\n",
      "40000/40000 [==============================] - 11s - loss: 0.1650 - acc: 0.9511    \n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/root/Util/miniconda/lib/python2.7/site-packages/sklearn/metrics/scorer.py:127: DeprecationWarning: Scoring method log_loss was renamed to neg_log_loss in version 0.18 and will be removed in 0.20.\n",
      "  sample_weight=sample_weight)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "20000/20000 [==============================] - 2s     \n",
      "Epoch 1/6\n",
      "60000/60000 [==============================] - 17s - loss: 0.4784 - acc: 0.8494    \n",
      "Epoch 2/6\n",
      "60000/60000 [==============================] - 16s - loss: 0.2399 - acc: 0.9295    \n",
      "Epoch 3/6\n",
      "60000/60000 [==============================] - 16s - loss: 0.1875 - acc: 0.9451    \n",
      "Epoch 4/6\n",
      "60000/60000 [==============================] - 16s - loss: 0.1602 - acc: 0.9521    \n",
      "Epoch 5/6\n",
      "60000/60000 [==============================] - 16s - loss: 0.1445 - acc: 0.9584    \n",
      "Epoch 6/6\n",
      "60000/60000 [==============================] - 16s - loss: 0.1357 - acc: 0.9610    \n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=None, error_score='raise',\n",
       "       estimator=<keras.wrappers.scikit_learn.KerasClassifier object at 0x7f42703d3e10>,\n",
       "       fit_params={}, iid=True, n_jobs=1,\n",
       "       param_grid={'dense_layer_sizes': [[32], [64], [32, 32], [64, 64]], 'nb_epoch': [3, 6], 'nb_pool': [2], 'nb_conv': [3], 'nb_filters': [8]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, scoring='log_loss', verbose=0)"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "validator.fit(X_train, y_train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 打印最好模型的参数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The parameters of the best model are: \n",
      "{'dense_layer_sizes': [64, 64], 'nb_conv': 3, 'nb_pool': 2, 'nb_epoch': 6, 'nb_filters': 8}\n"
     ]
    }
   ],
   "source": [
    "print('The parameters of the best model are: ')\n",
    "print(validator.best_params_)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 返回模型\n",
    "\n",
    "validator.best_estimator_ 返回sklearn-wrapped版本的最好模型\n",
    "\n",
    "validator.best_estimator_.model 返回（unwrapped）keras模型"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10000/10000 [==============================] - 1s     \n",
      "\n",
      "\n",
      "loss :  0.0535527251991\n",
      "acc :  0.9825\n"
     ]
    }
   ],
   "source": [
    "best_model = validator.best_estimator_.model\n",
    "metric_names = best_model.metrics_names\n",
    "metric_values = best_model.evaluate(X_test, y_test)\n",
    "print('\\n')\n",
    "for metric, value in zip(metric_names, metric_values):\n",
    "    print(metric, ': ', value)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  },
  "ssap_exp_config": {
   "error_alert": "Error Occurs!",
   "initial": [],
   "max_iteration": 1000,
   "recv_id": "",
   "running": [],
   "summary": [],
   "version": "1.1.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}