conference_notebooks/KDD_2019/plasticc/Part_1-2_RNN_Extract_Bottleneck.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Part 1.2 - Extract Bottleneck Features From RNN \n",
    "In this notebook, we will load a pre-trained RNN model and run both our train & test timeseries samples through it, extracting the bottleneck features (the outputs of the layer just before the fully-connected / classification layer(s)).\n",
    "\n",
    "The bottleneck features will be used to represent the embedded timeseries features for each sample. In the next notebook, we will concatenate them with other features to train our final XGBoost classification model. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings(\"ignore\")\n",
    "import math\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import time\n",
    "import tensorflow as tf\n",
    "from rnn import PlasticcRNN\n",
    "import matplotlib.pyplot as pltß\n",
    "%matplotlib inline\n",
    "\n",
    "print(tf.__version__)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load Train & Test Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = pd.read_pickle('train_rnn.pkl')\n",
    "test = pd.read_pickle('test_rnn.pkl')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load pre-trained RNN model\n",
    "\n",
    "You can find the code for the RNN model in `rnn.py` if you'd like to take a look at the implementation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = PlasticcRNN('weight/rnn.npy')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We use the function `predict_bottleneck()` to feed each training example through the pre-trained RNN model and extract the outputs from the layer just before the final classification layer (the bottleneck layer)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_bn = model.predict_bottleneck(train)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also extract the bottleneck features for the test dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_bn = model.predict_bottleneck(test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's verify that we have embedded each of our timeseries into the 16-dimensional space. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(train_bn.shape)\n",
    "print(test_bn.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Convert Bottleneck Features to DataFrames\n",
    "\n",
    "Since we will be concatenating these bottleneck features with a Dataframe in the next notebook, we can create dataframes for our bottleneck features."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_bn = pd.DataFrame(train_bn,columns=['bottleneck%d'%i for i in range(train_bn.shape[1])])\n",
    "train_bn['object_id'] = train.object_id.unique().astype(\"int32\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_bn.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "test_bn = pd.DataFrame(test_bn,columns=['bottleneck%d'%i for i in range(test_bn.shape[1])])\n",
    "test_bn['object_id'] = test.object_id.unique().astype(\"int32\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Store Features to Disk"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_bn.to_pickle('train_bn.pkl')\n",
    "test_bn.to_pickle('test_bn.pkl')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "rapids",
   "language": "python",
   "name": "rapids"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}