Multi-Input & Multi-Output Models.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The functional API makes it easy to manipulate a large number of intertwined datastreams."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's consider the following model. We seek to predict how many retweets and likes a news headline will receive on Twitter. The main input to the model will be the headline itself, as a sequence of words, but to spice things up, our model will also have an auxiliary input, receiving extra data such as the time of day when the headline was posted, etc. The model will also be supervised via two loss functions. Using the main loss function earlier in a model is a good regularization mechanism for deep models."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The main input will receive the headline, as a sequence of integers (each integer encodes a word). The integers will be between 1 and 10,000 (a vocabulary of 10,000 words) and the sequences will be 100 words long."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![multi-input-multi-output-graph.png](multi-input-multi-output-graph.png)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import Input, Embedding, LSTM, Dense\n",
    "from keras.models import Model\n",
    "import keras"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Headline input: meant to receive sequences of 100 integers, between 1 and 10000.\n",
    "# Note that we can name any layer by passing it a \"name\" argument.\n",
    "main_input = Input(shape=(100,), dtype='int32', name='main_input')\n",
    "\n",
    "# This embedding layer will encode the input sequence\n",
    "# into a sequence of dense 512-dimensional vectors.\n",
    "x = Embedding(output_dim=512, input_dim=1000, input_length=100)(main_input)\n",
    "\n",
    "# A LSTM will transform the vector sequence into a single vector,\n",
    "# containing information about the entire sequence\n",
    "lstm_out = LSTM(32)(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we insert the auxiliary loss, allowing the LSTM and Embedding layer to be trained smoothly even though the main loss will be much higher in the model.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "auxiliary_output = Dense(1, activation='sigmoid', name='aux_output')(lstm_out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "At this point, we feed into the model our auxiliary input data by concatenating it with the LSTM output:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "auxiliary_input = Input(shape=(5,), name='aux_input')\n",
    "x = keras.layers.concatenate([lstm_out, auxiliary_input])\n",
    "\n",
    "# We stack a deep densely-connected network on top\n",
    "x = Dense(64, activation='relu')(x)\n",
    "x = Dense(64, activation='relu')(x)\n",
    "x = Dense(64, activation='relu')(x)\n",
    "\n",
    "# And finally we add the main logistic regression layer\n",
    "main_output = Dense(1, activation='sigmoid', name='main_output')(x)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### This defines a model with two inputs and two outputs:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model = Model(inputs=[main_input, auxiliary_input], outputs=[main_output, auxiliary_output])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### We compile the model and assign a weight of 0.2 to the auxiliary loss."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "model.compile(optimizer='rmsprop', loss='binary_crossentropy', loss_weights=[1.0,0.2])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model.fit([headline_data, additional_data], [labels, labels], epochs=50, batch_size=32)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since our inputs and outputs are named (we passed them a \"name\" argument), we could also have compiled the model via:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "#model.compile(optimizer='rmsprop',\n",
    "#              loss={'main_output': 'binary_crossentropy', 'aux_output': 'binary_crossentropy'},\n",
    "#              loss_weights={'main_output': 1., 'aux_output': 0.2})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# And trained it via:\n",
    "#model.fit({'main_input': headline_data, 'aux_input': additional_data},\n",
    "#          {'main_output': labels, 'aux_output': labels},\n",
    "#          epochs=50, batch_size=32)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}