Meetup12-DeepLearningTensorFlowinAzure-Part-5.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<h1 align=\"center\">Deep learning with TensorFlow in Azure PART 5</h1>\n",
    "<h1 align=\"center\">How to put models in production as a Web Service</h1>\n",
    "<h1 align=\"center\">Meetup DFW Data & AI - Microsoft</h1>\n",
    "## Setting Up Environment "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1) Deploy the Linux Deep Learning VM in Azure "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In order to complete this notebook, you must deploy a Linux DLVM (Deep Learning VM) in your azure subscription. Click [HERE](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/microsoft-ads.dsvm-deep-learning?tab=Overview), then click on GET IT NOW and once you are directed to the Azure portal click CREATE.\n",
    "\n",
    "**In Basics blade:**<br>\n",
    "**Name:** meetupdsvmgpu <br>\n",
    "**Select OS:** Linux <br>\n",
    "**Username:** sshuser<br>\n",
    "**Password:** Passw0rd.1!!<br>\n",
    "**Resource Group:** meetupdsvmgpu_rg <br>\n",
    "**Location:** Pick among East US or West US 2<br>\n",
    "\n",
    "**In Settings blade:**\n",
    "Leave as is\n",
    "\n",
    "The Deep Learning Virtual Machine (DLVM) is a specially configured variant of the Data Science Virtual Machine(DSVM) to make it easier to use GPU-based VM instances for training deep learning models. It is supported on Windows 2016, or the Ubuntu Data Science Virtual Machine and shares the same core VM images (and hence all the rich toolset) as the DSVM. We also provide end-to-end AI samples for image and text understanding. The deep learning virtual machine also makes the rich set of tools and samples on the DSVM more easily discoverable. In terms of the tooling, the Deep Learning Virtual Machine provides several popular deep learning frameworks, tools to acquire and pre-process image, textual data. \n",
    "\n",
    "\n",
    "The DLVM contains several tools for AI including popular GPU editions of deep learning frameworks like Microsoft Cognitive Toolkit, TensorFlow, Keras, Caffe2, Chainer; tools to acquire and pre-process image, textual data, tools for data science modeling and development activities such as Microsoft R Server Developer Edition, Anaconda Python, Jupyter notebooks for Python and R, IDEs for Python and R , SQL database and many other data science and ML tools. \n",
    "\n",
    "\n",
    "The DLVM runs on Azure GPU NC-series VM instances. These GPUs use discrete device assignment, resulting in performance close to bare-metal, and are well-suited to deep learning problems.."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2) SSH into the VM and git clone the meetup repo"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "```\n",
    "> ssh sshuser@YOUR.VM.IP.ADDRESS\n",
    "\n",
    "> cd notebooks\n",
    "\n",
    "> git clone https://github.com/pablomarin/Meetups-Data-AI-DFW.git\n",
    "\n",
    "> sudo ln -s /anaconda/envs/py35/bin/pip /usr/bin/pip3\n",
    "\n",
    "> sudo pip3 install tqdm\n",
    "\n",
    "> sudo pip3 install ipython-autotime\n",
    "\n",
    "> az login\n",
    "\n",
    "#make sure you pick the right subscription to work with (IsDefault)\n",
    "\n",
    "> az account list --output table\n",
    "\n",
    "#If the subscription you want to work with is not default, then pick it by doing:\n",
    "\n",
    "> az account set --subscription \"My Demos\"\n",
    "\n",
    "# Register the Azure ML providers needed\n",
    "\n",
    "> az provider register -n Microsoft.MachineLearningCompute\n",
    "> \n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 3) Open the Jupyter notebook from your VM on your local browser"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> https://YOUR.VM.IP.ADDRESS:8000 <br>\n",
    "> Login with your VM username and password<br>\n",
    "> Go to the ***Meetups-Data-AI-DFW folder***<br>\n",
    "> Open the Notebook:***Meetup10-DeepLearningTensorFlowinAzure-Part-5.ipynb***<br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## PART 5 - Azure Machine Learning Model Management"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are couple of ways to put a deep learning model (or any ML model) into production: Manually or using Azure ML Model Management service.\n",
    "- Manual: Create a Python Flask web API like this: https://blog.metaflow.fr/tensorflow-how-to-freeze-a-model-and-serve-it-with-a-python-api-d4f3596b3adc\n",
    "\n",
    "- Azure ML Model Management: This is what we are going to explain now"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1) What is the Azure Model Management Service?\n",
    "Source: https://docs.microsoft.com/en-us/azure/machine-learning/desktop-workbench/model-management-overview<br>\n",
    "\n",
    "Azure Machine Learning Model Management enables you to manage and deploy machine-learning workflows and models. \n",
    "\n",
    "Model Management provides capabilities for::\n",
    "\n",
    "    - Model versioning\n",
    "    - Tracking models in production\n",
    "    - Deploying models to production through AzureML Compute Environment with Azure Container Service and Kubernetes\n",
    "    - Creating Docker containers with the models and testing them locally\n",
    "    - Automated model retraining\n",
    "    - Capturing model telemetry for actionable insights.\n",
    "\n",
    "Azure Machine Learning Model Management provides a registry of model versions. It also provides automated workflows for packaging and deploying Machine Learning containers as REST APIs. The models and their runtime dependencies are packaged in Linux-based Docker container with prediction API. \n",
    "\n",
    "    - Authentication\n",
    "    - Load balancing\n",
    "    - Automatic scale-out\n",
    "    - Encryption\n",
    "\n",
    "Azure Machine Learning Model Management provides these capabilities through the CLI, API, and the Azure portal.\n",
    "\n",
    "\n",
    "\n",
    "<div style=\"width:100%; margin-left:auto; margin-right:auto; margin-bottom:5px; margin-top:17px;\">\n",
    "<img src=\"https://docs.microsoft.com/en-us/azure/machine-learning/desktop-workbench/media/model-management-overview/modelmanagement.png\" alt=\"IMAGE\" /><br>\n",
    "</div>\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 2) Containers? why?\n",
    "\n",
    "You can use the Azure Model Managent Service to build Docker-based container images in their respective environments. The containerized, Docker-based images provide enterprises with the flexibility to run these images on the following compute environments:\n",
    "    - Kubernetes based Azure Container Service\n",
    "    - On-premises container services\n",
    "    - Development environments\n",
    "    - IoT devices\n",
    "\n",
    "These Docker-based containerized images are self-contained with all necessary dependencies required for generating predictions.\n",
    "\n",
    "You ou can deploy Docker-based container images with a single command to Azure Container Service managed by ML Compute Environment. These deployments are created with a front-end server that provides the following features:\n",
    "    - Low latency predictions at scale\n",
    "    - Load balancing\n",
    "    - Automatic scaling of ML endpoints\n",
    "    - API key authorization\n",
    "    - API swagger document\n",
    "    \n",
    "You can control the deployment scale and telemetry through the following configuration settings:\n",
    "\n",
    "    - System logging and model telemetry for each web service level. If enabled, all stdout logs are streamed to Azure Application Insights. Model telemetry is archived in storage that you provide. \n",
    "    - Auto-scale and concurrency limits. These settings automatically increase the number of deployed containers based on the load within the existing cluster. They also control the throughput and consistency of prediction latency.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3) What is the process to create the web service in ML Model management?\n",
    "\n",
    "<div style=\"width:100%; margin-left:auto; margin-right:auto; margin-bottom:5px; margin-top:17px;\">\n",
    "<img src=\"https://docs.microsoft.com/en-us/azure/machine-learning/desktop-workbench/media/model-management-overview/modelmanagementworkflow.png\" alt=\"IMAGE\" /><br>\n",
    "</div>\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok, enough theory, let's run some code.."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true
   },
   "source": [
    "## 4) LAB 1 - Put in production the Stock Demo example from Part 3\n",
    "\n",
    "Before you run the below lab, make sure that you run the Part 3 lab: [Meetup10-Lab-stockdemo.ipynb](Meetup10-Lab-stockdemo.ipynb)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Click [here](Meetup12-Lab1-BuildWebServiceStockDemo.ipynb) to open the notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.5.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}