04_Explicit_Feedback_deep_learning_with_Tensorflow_0.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Explicit Feedback Neural Recommender Systems\n",
    "\n",
    "Goals:\n",
    "- Understand recommendation system \n",
    "- Build different models architectures using Tensorflow\n",
    "- Retrieve Embeddings and visualize them\n",
    "- Add metadata information as input to the model\n",
    "\n",
    "\n",
    "This notebook is inspired by Oliver Grisel Notebook who used Keras\n",
    "https://github.com/ogrisel for building the moels. We will be using Basic Tensorflow APIs instead. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import os\n",
    "import matplotlib.pyplot as plt\n",
    "from sklearn.model_selection import train_test_split\n",
    "import tensorflow as tf\n",
    "from tensorflow.contrib import layers\n",
    "from tensorflow.python.estimator.inputs import numpy_io\n",
    "from tensorflow.contrib.learn import *\n",
    "%matplotlib inline "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('Tensorflow Version : {0}'.format(tf.__version__))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Ratings file\n",
    "\n",
    "Each line contains a rated movie: \n",
    "- a user\n",
    "- an item\n",
    "- a rating from 1 to 5 stars"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Base Path for MovieLens dataset\n",
    "ML_100K_PATH = os.path.join('processed','ml-100k','ml-100k')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_raw_ratings = pd.read_csv(os.path.join(ML_100K_PATH, 'u.data'), sep='\\t',\n",
    "                      names=[\"user_id\", \"item_id\", \"rating\", \"timestamp\"])\n",
    "df_raw_ratings.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Item metadata file\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "m_cols = ['item_id', 'title', 'release_date', 'video_release_date', 'imdb_url']\n",
    "# Loading only 5 columns\n",
    "df_items = pd.read_csv(os.path.join(ML_100K_PATH, 'u.item'), sep='|',\n",
    "                    names=m_cols, usecols=range(5), encoding='latin-1')\n",
    "df_items.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_release_year(x):\n",
    "    splits = str(x).split('-')\n",
    "    if(len(splits) == 3):\n",
    "        return int(splits[2])\n",
    "    else:\n",
    "        return 1920\n",
    "    \n",
    "df_items['release_year'] = df_items['release_date'].map(lambda x : get_release_year(x))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_items.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Merge Rating with Item Metadata"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_all_ratings = pd.merge(df_items, df_raw_ratings)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_all_ratings.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Data Preprocessing\n",
    "\n",
    "To understand well the distribution of the data, the following statistics are computed:\n",
    "- the number of users\n",
    "- the number of items\n",
    "- the rating distribution"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Number of users\n",
    "max_user_id = df_all_ratings['user_id'].max()\n",
    "max_user_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Number of items\n",
    "max_item_id = df_all_ratings['item_id'].max()\n",
    "max_item_id"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_all_ratings.groupby('rating')['rating'].count().plot(kind='bar', rot=0);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# ratings\n",
    "df_all_ratings['rating'].describe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Add Popularity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "popularity = df_all_ratings.groupby('item_id').size().reset_index(name='popularity')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Enrich the ratings data with the popularity as an additional metadata."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_all_ratings = pd.merge(df_all_ratings, popularity)\n",
    "df_all_ratings.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df_all_ratings.nlargest(10, 'popularity')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Later in the analysis we will assume that this popularity does not come from the ratings themselves but from an external metadata, e.g. box office numbers in the month after the release in movie theaters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train Test Validation Split"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Split All ratings into train_val and test\n",
    "ratings_train_val, ratings_test = train_test_split(df_all_ratings, test_size=0.2, random_state=0)\n",
    "# Split train_val into training and validation set\n",
    "ratings_train, ratings_val = train_test_split(ratings_train_val, test_size=0.2, random_state=0)\n",
    "\n",
    "print('Total rating rows count: {0} '.format(len(df_all_ratings)))\n",
    "print('Total training rows count: {0} '.format(len(ratings_train_val)))\n",
    "print('Total validation rows count: {0} '.format(len(ratings_val)))\n",
    "print('Total test rows count: {0} '.format(len(ratings_test)))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratings_train.info()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Explicit feedback: supervised ratings prediction\n",
    "\n",
    "For each pair of (user, item) try to predict the rating the user would give to the item.\n",
    "\n",
    "This is the classical setup for building recommender systems from offline data with explicit supervision signal. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Predictive ratings  as a regression problem\n",
    "\n",
    "The following code implements the following architecture:\n",
    "\n",
    "<img src=\"images/rec_archi_1.svg\" style=\"width: 600px;\" />"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matrix Factorization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_size = 30 # embedding size\n",
    "reg_param = 0.01 # regularization parameter lambda\n",
    "learning_rate = 0.01 # learning rate \n",
    "\n",
    "\n",
    "# create tensorflow graph\n",
    "g = tf.Graph()\n",
    "with g.as_default():\n",
    "    # setting up random seed\n",
    "    tf.set_random_seed(1234)\n",
    "    \n",
    "    # placeholders\n",
    "    users = tf.placeholder(shape=[None], dtype=tf.int64)\n",
    "    items = tf.placeholder(shape=[None], dtype=tf.int64)\n",
    "    ratings = tf.placeholder(shape=[None], dtype=tf.float32)\n",
    "    \n",
    "    # variables\n",
    "    with tf.variable_scope(\"embedding\"):\n",
    "        user_weight = tf.get_variable(\"user_w\"\n",
    "                                      , shape=[max_user_id + 1, embedding_size]\n",
    "                                      , dtype=tf.float32\n",
    "                                      , initializer=layers.xavier_initializer())\n",
    "\n",
    "        item_weight = tf.get_variable(\"item_w\"\n",
    "                                       , shape=[max_item_id + 1, embedding_size]\n",
    "                                       , dtype=tf.float32\n",
    "                                       , initializer=layers.xavier_initializer())\n",
    "    # prediction\n",
    "    with tf.name_scope(\"inference\"):\n",
    "        user_embedding = tf.nn.embedding_lookup(user_weight, users)\n",
    "        item_embedding = tf.nn.embedding_lookup(item_weight, items)\n",
    "        pred = tf.reduce_sum(tf.multiply(user_embedding, item_embedding), 1) \n",
    "        \n",
    "    # loss \n",
    "    with tf.name_scope(\"loss\"):\n",
    "        reg_loss = tf.contrib.layers.apply_regularization(layers.l2_regularizer(scale=reg_param),\n",
    "                                               weights_list=[user_weight, item_weight])\n",
    "        loss = tf.nn.l2_loss(pred - ratings) + reg_loss\n",
    "        train_ops = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(loss)\n",
    "        rmse = tf.sqrt(tf.reduce_mean(tf.pow(pred - ratings, 2)))\n",
    "\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "def train_model():\n",
    "    # Training \n",
    "    epochs = 1000 # number of iterations \n",
    "    losses_train = []\n",
    "    losses_val = []\n",
    "\n",
    "\n",
    "\n",
    "    with tf.Session(graph=g) as sess:\n",
    "        # initializer\n",
    "        sess.run(tf.global_variables_initializer())\n",
    "\n",
    "\n",
    "        train_input_dict = {  users: ratings_train['user_id']\n",
    "                            , items: ratings_train['item_id']\n",
    "                            , ratings: ratings_train['rating']}\n",
    "        val_input_dict =  {  users: ratings_val['user_id']\n",
    "                            , items: ratings_val['item_id']\n",
    "                            , ratings: ratings_val['rating']}\n",
    "\n",
    "        test_input_dict =  {  users: ratings_test['user_id']\n",
    "                            , items: ratings_test['item_id']\n",
    "                            , ratings: ratings_test['rating']}\n",
    "\n",
    "        def check_overfit(validation_loss):\n",
    "            n = len(validation_loss)\n",
    "            if n < 5:\n",
    "                return False\n",
    "            count = 0 \n",
    "            for i in range(n-4, n):\n",
    "                if validation_loss[i] < validation_loss[i-1]:\n",
    "                    count += 1\n",
    "                if count >=2:\n",
    "                    return False\n",
    "            return True\n",
    "\n",
    "        for i in range(epochs):\n",
    "            # run the training operation\n",
    "            sess.run([train_ops], feed_dict=train_input_dict)\n",
    "\n",
    "            # show intermediate results \n",
    "            if i % 5 == 0:\n",
    "                loss_train = sess.run(loss, feed_dict=train_input_dict)\n",
    "                loss_val = sess.run(loss, feed_dict=val_input_dict)\n",
    "                losses_train.append(loss_train)\n",
    "                losses_val.append(loss_val)\n",
    "\n",
    "\n",
    "                # check early stopping \n",
    "                if(check_overfit(losses_val)):\n",
    "                    print('overfit !')\n",
    "                    break\n",
    "\n",
    "                print(\"iteration : {0} train loss: {1:.3f} , valid loss {2:.3f}\".format(i,loss_train, loss_val))\n",
    "\n",
    "        # calculate RMSE on the test dataset\n",
    "        print('RMSE on test dataset : {0:.4f}'.format(sess.run(rmse, feed_dict=test_input_dict)))\n",
    "\n",
    "        plt.plot(losses_train, label='train')\n",
    "        plt.plot(losses_val, label='validation')\n",
    "        #plt.ylim(0, 50000)\n",
    "        plt.legend(loc='best')\n",
    "        plt.title('Loss');"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Matrix Factorization with Biases"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_size = 30 # embedding size\n",
    "reg_param = 0.01 # regularization parameter lambda\n",
    "learning_rate = 0.01 # learning rate \n",
    "\n",
    "\n",
    "# create tensorflow graph\n",
    "g = tf.Graph()\n",
    "with g.as_default():\n",
    "    \n",
    "    tf.set_random_seed(1234)\n",
    "    \n",
    "    # placeholders\n",
    "    users = tf.placeholder(shape=[None], dtype=tf.int64)\n",
    "    items = tf.placeholder(shape=[None], dtype=tf.int64)\n",
    "    ratings = tf.placeholder(shape=[None], dtype=tf.float32)\n",
    "    \n",
    "    # variables\n",
    "    with tf.variable_scope(\"embedding\"):\n",
    "        user_weight = tf.get_variable(\"user_w\"\n",
    "                                      , shape=[max_user_id + 1, embedding_size]\n",
    "                                      , dtype=tf.float32\n",
    "                                      , initializer=layers.xavier_initializer())\n",
    "\n",
    "        item_weight = tf.get_variable(\"item_w\"\n",
    "                                       , shape=[max_item_id + 1, embedding_size]\n",
    "                                       , dtype=tf.float32\n",
    "                                       , initializer=layers.xavier_initializer())\n",
    "        \n",
    "        user_bias = tf.get_variable(\"user_b\"\n",
    "                                , shape=[max_user_id + 1]\n",
    "                                , dtype=tf.float32\n",
    "                                , initializer=tf.zeros_initializer)\n",
    "        \n",
    "        item_bias = tf.get_variable(\"item_b\"\n",
    "                                 , shape=[max_item_id + 1]\n",
    "                                 , dtype=tf.float32\n",
    "                                 , initializer=tf.zeros_initializer)\n",
    "        \n",
    "    # prediction\n",
    "    with tf.name_scope(\"inference\"):\n",
    "        user_embedding = tf.nn.embedding_lookup(user_weight, users)\n",
    "        item_embedding = tf.nn.embedding_lookup(item_weight, items)\n",
    "        user_b = tf.nn.embedding_lookup(user_bias, users)\n",
    "        item_b = tf.nn.embedding_lookup(item_bias, items)\n",
    "        pred = tf.reduce_sum(tf.multiply(user_embedding, item_embedding), 1) + user_b + item_b\n",
    "        \n",
    "    # loss \n",
    "    with tf.name_scope(\"loss\"):\n",
    "        reg_loss = tf.contrib.layers.apply_regularization(layers.l2_regularizer(scale=reg_param),\n",
    "                                               weights_list=[user_weight, item_weight])\n",
    "        loss = tf.nn.l2_loss(pred - ratings) + reg_loss\n",
    "        train_ops = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(loss)\n",
    "        rmse = tf.sqrt(tf.reduce_mean(tf.pow(pred - ratings, 2)))\n",
    "\n",
    "   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_model()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## A Deep recommender model\n",
    "\n",
    "We can use deep learning models with multiple layers ( fully connected and dropout ) for the recommendation system.\n",
    "\n",
    "<img src=\"images/rec_archi_2.svg\" style=\"width: 600px;\" />\n",
    "\n",
    "To build this model we will need a new kind of layer:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_size = 50\n",
    "reg_param = 0.01\n",
    "learning_rate = 0.01\n",
    "n_users = max_user_id + 1\n",
    "n_items = max_item_id + 1\n",
    "\n",
    "g = tf.Graph()\n",
    "with g.as_default():\n",
    "    \n",
    "    tf.set_random_seed(1234)\n",
    "\n",
    "    users = tf.placeholder(shape=[None,1], dtype=tf.int64, name='input_users')\n",
    "    items = tf.placeholder(shape=[None,1], dtype=tf.int64, name='input_items')\n",
    "    ratings = tf.placeholder(shape=[None,1], dtype=tf.float32, name='input_ratings')\n",
    "    \n",
    "    l2_loss = tf.constant(0.0)\n",
    "    \n",
    "    # embeddding layer\n",
    "    with tf.variable_scope(\"embedding\"):\n",
    "        user_weights = tf.get_variable(\"user_w\"\n",
    "                                      , shape=[n_users, embedding_size]\n",
    "                                      , dtype=tf.float32\n",
    "                                      , initializer=layers.xavier_initializer())\n",
    "        \n",
    "        item_weights = tf.get_variable(\"item_w\"\n",
    "                                       , shape=[n_items, embedding_size]\n",
    "                                       , dtype=tf.float32\n",
    "                                       , initializer=layers.xavier_initializer())\n",
    "        \n",
    "        user_embedding = tf.squeeze(tf.nn.embedding_lookup(user_weights, users),axis=1, name='user_embedding')\n",
    "        item_embedding = tf.squeeze(tf.nn.embedding_lookup(item_weights, items),axis=1, name='item_embedding')\n",
    "        \n",
    "        l2_loss += tf.nn.l2_loss(user_weights)\n",
    "        l2_loss += tf.nn.l2_loss(item_weights)\n",
    "        \n",
    "        \n",
    "        print(user_embedding)\n",
    "        print(item_embedding)\n",
    "        \n",
    "    \n",
    "    # combine inputs\n",
    "    with tf.name_scope('concatenation'):\n",
    "        input_vecs = tf.concat([user_embedding, item_embedding], axis=1)\n",
    "        print(input_vecs)\n",
    "        \n",
    "    # fc-1\n",
    "    num_hidden = 64\n",
    "    with tf.name_scope(\"fc_1\"):\n",
    "        W_fc_1 = tf.get_variable(\n",
    "            \"W_hidden\",\n",
    "            shape=[2*embedding_size, num_hidden],\n",
    "            initializer=tf.contrib.layers.xavier_initializer())\n",
    "        b_fc_1 = tf.Variable(tf.constant(0.1, shape=[num_hidden]), name=\"b\")\n",
    "        hidden_output = tf.nn.relu(tf.nn.xw_plus_b(input_vecs, W_fc_1, b_fc_1), name='hidden_output')\n",
    "        l2_loss += tf.nn.l2_loss(W_fc_1)\n",
    "        print(hidden_output)\n",
    "        \n",
    "    # dropout\n",
    "    with tf.name_scope(\"dropout\"):\n",
    "        h_drop = tf.nn.dropout(hidden_output, 0.99, name=\"hidden_output_drop\")\n",
    "        print(h_drop)\n",
    "    \n",
    "    # fc-2\n",
    "    with tf.name_scope(\"fc_2\"):\n",
    "        W_fc_2 = tf.get_variable(\n",
    "            \"W_output\",\n",
    "            shape=[num_hidden,1],\n",
    "            initializer=tf.contrib.layers.xavier_initializer())\n",
    "        b_fc_2 = tf.Variable(tf.constant(0.1, shape=[1]), name=\"b\")\n",
    "        pred = tf.nn.xw_plus_b(h_drop, W_fc_2, b_fc_2, name='pred')\n",
    "        l2_loss += tf.nn.l2_loss(W_fc_2)\n",
    "        print(pred)\n",
    "\n",
    "    # loss\n",
    "    with tf.name_scope(\"loss\"):\n",
    "        loss = tf.nn.l2_loss(pred - ratings) + reg_param * l2_loss\n",
    "        train_ops = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(loss)\n",
    "        rmse = tf.sqrt(tf.reduce_mean(tf.pow(pred - ratings, 2)))\n",
    "\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_model_deep():\n",
    "    losses_train = []\n",
    "    losses_val = []\n",
    "    epochs = 1000\n",
    "\n",
    "    with tf.Session(graph=g) as sess:\n",
    "        sess.run(tf.global_variables_initializer())\n",
    "        train_input_dict = {users: ratings_train['user_id'].values.reshape([-1,1])\n",
    "            , items: ratings_train['item_id'].values.reshape([-1,1])\n",
    "            , ratings: ratings_train['rating'].values.reshape([-1,1])}\n",
    "\n",
    "        val_input_dict = {users: ratings_val['user_id'].values.reshape([-1,1])\n",
    "            , items: ratings_val['item_id'].values.reshape([-1,1])\n",
    "            , ratings: ratings_val['rating'].values.reshape([-1,1])}\n",
    "\n",
    "        test_input_dict = {users: ratings_test['user_id'].values.reshape([-1,1])\n",
    "            , items: ratings_test['item_id'].values.reshape([-1,1])\n",
    "            , ratings: ratings_test['rating'].values.reshape([-1,1])}\n",
    "\n",
    "        def check_overfit(validation_loss):\n",
    "                n = len(validation_loss)\n",
    "                if n < 5:\n",
    "                    return False\n",
    "                count = 0 \n",
    "                for i in range(n-4, n):\n",
    "                    if validation_loss[i] < validation_loss[i-1]:\n",
    "                        count += 1\n",
    "                    if count >=3:\n",
    "                        return False\n",
    "                return True\n",
    "\n",
    "\n",
    "\n",
    "        for i in range(epochs):\n",
    "            sess.run([train_ops], feed_dict=train_input_dict)\n",
    "            if i % 10 == 0:\n",
    "                loss_train = sess.run(loss, feed_dict=train_input_dict)\n",
    "                loss_val = sess.run(loss, feed_dict=val_input_dict)\n",
    "                losses_train.append(loss_train)\n",
    "                losses_val.append(loss_val)\n",
    "\n",
    "                # check early stopping \n",
    "                if(check_overfit(losses_val)):\n",
    "                    print('overfit !')\n",
    "                    break\n",
    "\n",
    "                print(\"iteration : %d train loss: %.3f , valid loss %.3f\" % (i,loss_train, loss_val))\n",
    "\n",
    "         # calculate RMSE on the test dataset\n",
    "        print('RMSE on test dataset : {0:.4f}'.format(sess.run(rmse, feed_dict=test_input_dict)))\n",
    "\n",
    "        # user and item embedding\n",
    "        user_embedding_variable = [v for v in tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES) if v.name.endswith('embedding/user_w:0')][0]\n",
    "        item_embedding_variable = [v for v in tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES) if v.name.endswith('embedding/item_w:0')][0]\n",
    "        user_embedding_weights, item_embedding_weights = sess.run([user_embedding_variable,item_embedding_variable])\n",
    "        \n",
    "        \n",
    "        # plot train and validation loss\n",
    "        plt.plot(losses_train, label='train')\n",
    "        plt.plot(losses_val, label='validation')\n",
    "        plt.legend(loc='best')\n",
    "        plt.title('Loss');\n",
    "        \n",
    "        return user_embedding_weights, item_embedding_weights "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_embedding_weights, item_embedding_weights  = train_model_deep()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Model Embeddings"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"First item name from metadata:\", df_items[\"title\"][1])\n",
    "print(\"Embedding vector for the first item:\")\n",
    "print(item_embedding_weights[1])\n",
    "print(\"shape:\", item_embedding_weights[1].shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Visualizing embeddings using TSNE\n",
    "\n",
    "- we use scikit learn to visualize items embeddings\n",
    "- Try different perplexities, and visualize user embeddings as well\n",
    "- check what is the impact of different perplexity value. Here is a very nice tutorial if you want to know in detail (https://distill.pub/2016/misread-tsne/ )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.manifold import TSNE\n",
    "\n",
    "item_tsne = TSNE(perplexity=50).fit_transform(item_embedding_weights)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "plt.figure(figsize=(10, 10))\n",
    "plt.scatter(item_tsne[:, 0], item_tsne[:, 1]);\n",
    "plt.xticks(()); plt.yticks(());\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Using item metadata in the model\n",
    "\n",
    "Using a similar framework as previously, we will build another deep model that can also leverage additional metadata. The resulting system is therefore an **Hybrid Recommender System** that does both **Collaborative Filtering** and **Content-based recommendations**.\n",
    "\n",
    "<img src=\"images/rec_archi_3.svg\" style=\"width: 600px;\" />\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "embedding_size = 50\n",
    "reg_param = 0.01\n",
    "learning_rate = 0.01\n",
    "n_users = max_user_id + 1\n",
    "n_items = max_item_id + 1\n",
    "meta_size = 2\n",
    "\n",
    "g = tf.Graph()\n",
    "with g.as_default():\n",
    "\n",
    "    tf.set_random_seed(1234)\n",
    "    \n",
    "    users = tf.placeholder(shape=[None,1], dtype=tf.int64, name='input_users')\n",
    "    items = tf.placeholder(shape=[None,1], dtype=tf.int64, name='input_items')\n",
    "    meta = tf.placeholder(shape=[None,2], dtype=tf.float32, name='input_metadata')\n",
    "    ratings = tf.placeholder(shape=[None,1], dtype=tf.float32, name='input_ratings')\n",
    "    \n",
    "    l2_loss = tf.constant(0.0)\n",
    "    \n",
    "    # embeddding layer\n",
    "    with tf.variable_scope(\"embedding\"):\n",
    "        user_weights = tf.get_variable(\"user_w\"\n",
    "                                      , shape=[n_users, embedding_size]\n",
    "                                      , dtype=tf.float32\n",
    "                                      , initializer=layers.xavier_initializer())\n",
    "        \n",
    "        item_weights = tf.get_variable(\"item_w\"\n",
    "                                       , shape=[n_items, embedding_size]\n",
    "                                       , dtype=tf.float32\n",
    "                                       , initializer=layers.xavier_initializer())\n",
    "        \n",
    "        \n",
    "        \n",
    "        user_embedding = tf.squeeze(tf.nn.embedding_lookup(user_weights, users),axis=1, name='user_embedding')\n",
    "        item_embedding = tf.squeeze(tf.nn.embedding_lookup(item_weights, items),axis=1, name='item_embedding')\n",
    "        \n",
    "        l2_loss += tf.nn.l2_loss(user_weights)\n",
    "        l2_loss += tf.nn.l2_loss(item_weights)\n",
    "        \n",
    "        \n",
    "        print(user_embedding)\n",
    "        print(item_embedding)\n",
    "        \n",
    "    \n",
    "    # combine inputs\n",
    "    with tf.name_scope('concatenation'):\n",
    "        input_vecs = tf.concat([user_embedding, item_embedding, meta], axis=1)\n",
    "        print(input_vecs)\n",
    "        \n",
    "    # fc-1\n",
    "    num_hidden = 64\n",
    "    with tf.name_scope(\"fc_1\"):\n",
    "        W_fc_1 = tf.get_variable(\n",
    "            \"W_hidden\",\n",
    "            shape=[2*embedding_size + meta_size, num_hidden],\n",
    "            initializer=tf.contrib.layers.xavier_initializer())\n",
    "        b_fc_1 = tf.Variable(tf.constant(0.1, shape=[num_hidden]), name=\"b\")\n",
    "        hidden_output = tf.nn.relu(tf.nn.xw_plus_b(input_vecs, W_fc_1, b_fc_1), name='hidden_output')\n",
    "        l2_loss += tf.nn.l2_loss(W_fc_1)\n",
    "        print(hidden_output)\n",
    "    \n",
    "    # dropout\n",
    "    with tf.name_scope(\"dropout\"):\n",
    "        h_drop = tf.nn.dropout(hidden_output, 0.99, name=\"hidden_output_drop\")\n",
    "        print(h_drop)\n",
    "    \n",
    "    # fc-2\n",
    "    with tf.name_scope(\"fc_2\"):\n",
    "        W_fc_2 = tf.get_variable(\n",
    "            \"W_output\",\n",
    "            shape=[num_hidden,1],\n",
    "            initializer=tf.contrib.layers.xavier_initializer())\n",
    "        b_fc_2 = tf.Variable(tf.constant(0.1, shape=[1]), name=\"b\")\n",
    "        pred = tf.nn.xw_plus_b(h_drop, W_fc_2, b_fc_2, name='pred')\n",
    "        l2_loss += tf.nn.l2_loss(W_fc_2)\n",
    "        print(pred)\n",
    "\n",
    "    # loss\n",
    "    with tf.name_scope(\"loss\"):\n",
    "        loss = tf.nn.l2_loss(pred - ratings) + reg_param * l2_loss\n",
    "        train_ops = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(loss)\n",
    "        rmse = tf.sqrt(tf.reduce_mean(tf.pow(pred - ratings, 2)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.preprocessing import QuantileTransformer\n",
    "\n",
    "meta_columns = ['popularity', 'release_year']\n",
    "\n",
    "scaler = QuantileTransformer()\n",
    "item_meta_train = scaler.fit_transform(ratings_train[meta_columns])\n",
    "item_meta_val = scaler.transform(ratings_val[meta_columns])\n",
    "item_meta_test = scaler.transform(ratings_test[meta_columns])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def train_model_deep_meta():\n",
    "\n",
    "    losses_train = []\n",
    "    losses_val = []\n",
    "    epochs = 1000\n",
    "\n",
    "    with tf.Session(graph=g) as sess:\n",
    "        sess.run(tf.global_variables_initializer())\n",
    "        train_input_dict = {users: ratings_train['user_id'].values.reshape([-1,1])\n",
    "            , items: ratings_train['item_id'].values.reshape([-1,1])\n",
    "            , ratings: ratings_train['rating'].values.reshape([-1,1])\n",
    "                           ,meta: item_meta_train}\n",
    "\n",
    "        val_input_dict = {users: ratings_val['user_id'].values.reshape([-1,1])\n",
    "            , items: ratings_val['item_id'].values.reshape([-1,1])\n",
    "            , ratings: ratings_val['rating'].values.reshape([-1,1])\n",
    "                         ,meta : item_meta_val}\n",
    "\n",
    "        test_input_dict = {users: ratings_test['user_id'].values.reshape([-1,1])\n",
    "            , items: ratings_test['item_id'].values.reshape([-1,1])\n",
    "            , ratings: ratings_test['rating'].values.reshape([-1,1])\n",
    "                          ,meta : item_meta_test}\n",
    "        def check_overfit(validation_loss):\n",
    "            n = len(validation_loss)\n",
    "            if n < 5:\n",
    "                return False\n",
    "            count = 0 \n",
    "            for i in range(n-4, n):\n",
    "                if validation_loss[i] < validation_loss[i-1]:\n",
    "                    count += 1\n",
    "                if count >=3:\n",
    "                    return False\n",
    "            return True\n",
    "\n",
    "\n",
    "        for i in range(epochs):\n",
    "            sess.run([train_ops], feed_dict=train_input_dict)\n",
    "            if i % 10 == 0:\n",
    "                loss_train = sess.run(loss, feed_dict=train_input_dict)\n",
    "                loss_val = sess.run(loss, feed_dict=val_input_dict)\n",
    "                losses_train.append(loss_train)\n",
    "                losses_val.append(loss_val)\n",
    "\n",
    "                 # check early stopping \n",
    "                if(check_overfit(losses_val)):\n",
    "                    print('overfit !')\n",
    "                    break\n",
    "                print(\"iteration : %d train loss: %.3f , valid loss %.3f\" % (i,loss_train, loss_val))\n",
    "        \n",
    "        # plot train and validation loss\n",
    "        plt.plot(losses_train, label='train')\n",
    "        plt.plot(losses_val, label='validation')\n",
    "        plt.legend(loc='best')\n",
    "        plt.title('Loss');\n",
    "        \n",
    "         # calculate RMSE on the test dataset\n",
    "        print('RMSE on test dataset : {0:.4f}'.format(sess.run(rmse, feed_dict=test_input_dict)))\n",
    "        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_model_deep_meta()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}