01_02g_Accommodation_GCE.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89abf6-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89ae44-1a4d-11e7-b31a-0242ac12000d",
     "previous": null
    }
   },
   "source": [
    "# Elastic Stackの構築手順：収容（GCE）\n",
    "\n",
    "----\n",
    "\n",
    "Elasticsearchのインストール先となるGoogle Compute&nbsp;Engineインスタンスを確保します。  \n",
    "大まかな手順は次の通りです。  \n",
    "1. 動作環境の確認\n",
    "2. GCEインスタンスの設定を定義\n",
    "3. GCEインスタンスの生成\n",
    "4. Ansibleとの連携確認\n",
    "5. Inventoryへのホスト追加"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89ae44-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89b056-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89abf6-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "# 動作環境の確認\n",
    "\n",
    "このNotebookは、 [Google Python Client Library](https://github.com/google/google-api-python-client) を使ってマシンの確保を行います。そのため、Libraryがインポート可能であることの確認と、Credentialsが設定されている"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89b056-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89b268-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89ae44-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "## Libraryの確認\n",
    "\n",
    "このNotebook環境にGoogle Python Client Libraryがインストールされている必要があります。インストールされていない場合は、以下のセル実行に失敗し、 ImportError となります。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89b268-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89b45c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89b056-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/cffi/model.py:525: UserWarning: 'point_conversion_form_t' has no values explicitly defined; guessing that it is equivalent to 'unsigned int'\n",
      "  % self._get_c_name())\n"
     ]
    }
   ],
   "source": [
    "from googleapiclient import discovery"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89b45c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89b63c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89b268-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "もし、インストールされていない場合は、[Using the Python Client Library](https://cloud.google.com/compute/docs/tutorials/python-guide)を参考にインストールしてください。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89b63c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89b81c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89b45c-1a4d-11e7-b31a-0242ac12000d"
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "#!pip2 install --upgrade google-api-python-client\n",
    "#from googleapiclient import discovery"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89b81c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89ba1a-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89b63c-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "## Credentialの指定\n",
    "\n",
    "Google Compute EngineにアクセスするためのCredentialを指定してください。\n",
    "\n",
    "- JSONフォーマットのService Account情報\n",
    "- プロジェクトID\n",
    "\n",
    "を用意しておく。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89ba1a-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89bcf4-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89b81c-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [],
   "source": [
    "creds = '~/.keys/testdev-98ad0186d51d.json'\n",
    "target_project_id = 'testdev'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89bcf4-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89bede-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89ba1a-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "computeサービスを取得しておく。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89bede-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89c0b4-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89bcf4-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<googleapiclient.discovery.Resource at 0x7ff521f9d950>"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import os\n",
    "from oauth2client.client import GoogleCredentials\n",
    "\n",
    "credentials = GoogleCredentials.from_stream(os.path.expanduser(creds))\n",
    "compute = discovery.build('compute', 'v1', credentials=credentials)\n",
    "compute"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89c0b4-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89c2a8-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89bede-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "# GCEインスタンスの設定と生成\n",
    "\n",
    "サーバ構成に応じたGCEインスタンスの設定を定義し、インスタンスを生成します。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89c2a8-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89c4b0-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89c0b4-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "## ゾーンの設定\n",
    "\n",
    "どのZoneにインスタンスを確保するかを定義しておく。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89c4b0-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89c69a-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89c2a8-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [],
   "source": [
    "target_zone = 'us-central1-f'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89c69a-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89c866-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89c4b0-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "動作確認として、現在のインスタンス名の一覧を確認する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89c866-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89ca46-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89c69a-1a4d-11e7-b31a-0242ac12000d"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "instances = compute.instances().list(zone=target_zone, project=target_project_id).execute()\n",
    "map(lambda i: i['name'], instances['items'] if 'items' in instances else [])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89ca46-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89cc1c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89c866-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "## インスタンスの設定\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89cc1c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89cdfc-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89ca46-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "### イメージの設定\n",
    "\n",
    "マシンイメージを設定します。\n",
    "\n",
    "まずイメージの一覧を確認する。projectには、利用したいイメージの所属プロジェクトを指定する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89cdfc-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89cfd2-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89cc1c-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(u'centos-6-v20170227',\n",
       "  u'https://www.googleapis.com/compute/v1/projects/centos-cloud/global/images/centos-6-v20170227'),\n",
       " (u'centos-7-v20170227',\n",
       "  u'https://www.googleapis.com/compute/v1/projects/centos-cloud/global/images/centos-7-v20170227')]"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "images = compute.images().list(project='centos-cloud').execute()['items']\n",
    "images = filter(lambda i: i['status'] == 'READY' and 'deprecated' not in i, images)\n",
    "map(lambda i: (i['name'], i['selfLink']), images)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89cfd2-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89d19e-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89cdfc-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "利用したいイメージのURLを設定する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89d19e-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89d36a-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89cfd2-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [],
   "source": [
    "# centos-7-v20170227\n",
    "source_disk_image = 'https://www.googleapis.com/compute/v1/projects/centos-cloud/global/images/centos-7-v20170227'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89d36a-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89d630-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89d19e-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "### マシンタイプの設定\n",
    "\n",
    "[マシンタイプ](https://cloud.google.com/compute/docs/machine-types)のページから適切なタイプを選択してください。\n",
    "\n",
    "例えば60GB程度のメモリを利用するのであれば **n1-standard-16** が選択候補となります。  \n",
    "利用したいサイズに応じてタイプを選んで下さい。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89d630-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89d856-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89d36a-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "このZoneで利用可能なMachine Typeは以下の通りです。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89d856-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89da2c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89d630-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(u'f1-micro', u'1 vCPU (shared physical core) and 0.6 GB RAM'),\n",
       " (u'g1-small', u'1 vCPU (shared physical core) and 1.7 GB RAM'),\n",
       " (u'n1-highcpu-16', u'16 vCPUs, 14.4 GB RAM'),\n",
       " (u'n1-highcpu-2', u'2 vCPUs, 1.8 GB RAM'),\n",
       " (u'n1-highcpu-32', u'32 vCPUs, 28.8 GB RAM'),\n",
       " (u'n1-highcpu-4', u'4 vCPUs, 3.6 GB RAM'),\n",
       " (u'n1-highcpu-8', u'8 vCPUs, 7.2 GB RAM'),\n",
       " (u'n1-highmem-16', u'16 vCPUs, 104 GB RAM'),\n",
       " (u'n1-highmem-2', u'2 vCPUs, 13 GB RAM'),\n",
       " (u'n1-highmem-32', u'32 vCPUs, 208 GB RAM'),\n",
       " (u'n1-highmem-4', u'4 vCPUs, 26 GB RAM'),\n",
       " (u'n1-highmem-8', u'8 vCPUs, 52 GB RAM'),\n",
       " (u'n1-standard-1', u'1 vCPU, 3.75 GB RAM'),\n",
       " (u'n1-standard-16', u'16 vCPUs, 60 GB RAM'),\n",
       " (u'n1-standard-2', u'2 vCPUs, 7.5 GB RAM'),\n",
       " (u'n1-standard-32', u'32 vCPUs, 120 GB RAM'),\n",
       " (u'n1-standard-4', u'4 vCPUs, 15 GB RAM'),\n",
       " (u'n1-standard-8', u'8 vCPUs, 30 GB RAM')]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "machineTypes = compute.machineTypes().list(project=target_project_id, zone=target_zone).execute()['items']\n",
    "map(lambda t: (t['name'], t['description']), machineTypes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89da2c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89dbf8-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89d856-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [],
   "source": [
    "machine_type = 'n1-standard-16'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89dbf8-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89ddf6-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89da2c-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "### キーペアの設定\n",
    "\n",
    "現在のSSHキーの一覧を取得する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89ddf6-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89dfd6-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89dbf8-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'',\n",
       " u'ansible:ssh-rsa XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX ansible@XXXXXXXXXXXX']"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "projectMetadata = compute.projects().get(project=target_project_id).execute()\n",
    "currentSSHKeys = filter(lambda metadata: metadata['key'] == 'sshKeys', projectMetadata['commonInstanceMetadata']['items']) \\\n",
    "                        if 'commonInstanceMetadata' in projectMetadata and 'items' in projectMetadata['commonInstanceMetadata'] else []\n",
    "currentSSHKeys = currentSSHKeys[0]['value'].split('\\n') if currentSSHKeys else []\n",
    "currentSSHKeys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89dfd6-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89e1ac-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89ddf6-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "SSHのキー一覧にこのNotebook環境のキーがなければ、追加する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89e1ac-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89e396-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89dfd6-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [],
   "source": [
    "#pub_key = None\n",
    "#with open(os.path.expanduser('~/.ssh/ansible_id_rsa.pub'), 'r') as f:\n",
    "#    pub_key = f.readlines()[0].strip()\n",
    "#\n",
    "#if not filter(lambda k: k.endswith(pub_key), currentSSHKeys):\n",
    "#    currentSSHKeys.append('ansible:' + pub_key)\n",
    "#currentSSHKeys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89e396-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89e56c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89e1ac-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "Metadataに反映する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89e56c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89e74c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89e396-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [],
   "source": [
    "#compute.projects().setCommonInstanceMetadata(project=target_project_id,\n",
    "#                                             body={'items': [{'key': 'sshKeys',\n",
    "#                                                              'value': '\\n'.join(currentSSHKeys)}]}).execute()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89e74c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89e922-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89e56c-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "Metadataに反映されたことを確認する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89e922-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89eb0c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89e74c-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [],
   "source": [
    "#projectMetadata = compute.projects().get(project=target_project_id).execute()\n",
    "#currentSSHKeys = filter(lambda metadata: metadata['key'] == 'sshKeys', projectMetadata['commonInstanceMetadata']['items']) \\\n",
    "#                        if 'commonInstanceMetadata' in projectMetadata and 'items' in projectMetadata['commonInstanceMetadata'] else []\n",
    "#currentSSHKeys = currentSSHKeys[0]['value'].split('\\n') if currentSSHKeys else []\n",
    "#currentSSHKeys"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89eb0c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89ecd8-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89e922-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "### ディスクサイズの設定\n",
    "\n",
    "ディスクに確保する容量を設定する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89ecd8-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89eeae-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89eb0c-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [],
   "source": [
    "disk_size_gb = 1024"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89eeae-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89f0ac-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89ecd8-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "### インスタンスの生成\n",
    "\n",
    "設定した情報を用いてマシンを確保する。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89f0ac-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89f2a0-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89eeae-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "作成するインスタンス数を設定します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89f2a0-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89f46c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89f0ac-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [],
   "source": [
    "instance_count = 3"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89f46c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89f642-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89f2a0-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "インスタンス群に付与する名前を設定します。  \n",
    "この名前は、後の処理でタグのNameキーに設定されます。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f89f642-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89f80e-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89f46c-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [],
   "source": [
    "instance_name = 'es-cluster-'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89f80e-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89f9e4-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89f642-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "ディスクをインスタンス数だけ準備を行います。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89f9e4-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89fbba-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89f80e-1a4d-11e7-b31a-0242ac12000d"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('es-cluster-disk-01',\n",
       "  {u'id': u'6743525392042058698',\n",
       "   u'insertTime': u'2017-03-23T07:35:17.655-07:00',\n",
       "   u'kind': u'compute#operation',\n",
       "   u'name': u'operation-1490279717064-54b66c926cd41-cb7769be-96374247',\n",
       "   u'operationType': u'insert',\n",
       "   u'progress': 0,\n",
       "   u'selfLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/operations/operation-1490279717064-54b66c926cd41-cb7769be-96374247',\n",
       "   u'status': u'PENDING',\n",
       "   u'targetId': u'8568716669835174858',\n",
       "   u'targetLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/disks/es-cluster-disk-01',\n",
       "   u'user': u'XXXXXXXXXXXX-compute@developer.gserviceaccount.com',\n",
       "   u'zone': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f'}),\n",
       " ('es-cluster-disk-02',\n",
       "  {u'id': u'1674876474185638857',\n",
       "   u'insertTime': u'2017-03-23T07:35:18.541-07:00',\n",
       "   u'kind': u'compute#operation',\n",
       "   u'name': u'operation-1490279717937-54b66c9341f69-9c2b142f-60f9f3d8',\n",
       "   u'operationType': u'insert',\n",
       "   u'progress': 0,\n",
       "   u'selfLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/operations/operation-1490279717937-54b66c9341f69-9c2b142f-60f9f3d8',\n",
       "   u'status': u'PENDING',\n",
       "   u'targetId': u'7245561722679747529',\n",
       "   u'targetLink': u'https://www.googleapis.com/compute/v1/projects/textdev/zones/us-central1-f/disks/es-cluster-disk-02',\n",
       "   u'user': u'XXXXXXXXXXXX-compute@developer.gserviceaccount.com',\n",
       "   u'zone': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f'}),\n",
       " ('es-cluster-disk-03',\n",
       "  {u'id': u'4010348714722747336',\n",
       "   u'insertTime': u'2017-03-23T07:35:19.479-07:00',\n",
       "   u'kind': u'compute#operation',\n",
       "   u'name': u'operation-1490279718823-54b66c941a459-b0cea26b-06615981',\n",
       "   u'operationType': u'insert',\n",
       "   u'progress': 0,\n",
       "   u'selfLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/operations/operation-1490279718823-54b66c941a459-b0cea26b-06615981',\n",
       "   u'status': u'PENDING',\n",
       "   u'targetId': u'5410346553858483144',\n",
       "   u'targetLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/disks/es-cluster-disk-03',\n",
       "   u'user': u'XXXXXXXXXXXX-compute@developer.gserviceaccount.com',\n",
       "   u'zone': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f'})]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_disks = []\n",
    "for i in range(0, instance_count):\n",
    "    name = '{0}disk-{1:02d}'.format(instance_name, i + 1)\n",
    "    config = {\n",
    "            'name': name\n",
    "        }\n",
    "\n",
    "    new_disk = compute.disks().insert(project=target_project_id, zone=target_zone,\n",
    "                                      sourceImage=source_disk_image, body=config).execute()\n",
    "    new_disks.append((name, new_disk))\n",
    "new_disks"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89fbba-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89fd86-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89f9e4-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "要求発行に成功したら、statusの確認を行います。全てREADYとなればOKです。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89fd86-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f89ff5c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89fbba-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'READY', u'READY', u'READY']"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "status = []\n",
    "for name, disk in new_disks:\n",
    "    result = compute.disks().get(project=target_project_id, zone=target_zone,\n",
    "                                 disk=name).execute()\n",
    "    status.append(result['status'])\n",
    "status"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f89ff5c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a0132-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89fd86-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "サイズの変更を行います。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a0132-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a02fe-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f89ff5c-1a4d-11e7-b31a-0242ac12000d"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{u'id': u'7416271034719822807',\n",
       "  u'insertTime': u'2017-03-23T07:35:37.030-07:00',\n",
       "  u'kind': u'compute#operation',\n",
       "  u'name': u'operation-1490279736861-54b66ca54e148-f9fb49c4-e1b1e82b',\n",
       "  u'operationType': u'resizeDisk',\n",
       "  u'progress': 0,\n",
       "  u'selfLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/operations/operation-1490279736861-54b66ca54e148-f9fb49c4-e1b1e82b',\n",
       "  u'status': u'PENDING',\n",
       "  u'targetId': u'8568716669835174858',\n",
       "  u'targetLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/disks/es-cluster-disk-01',\n",
       "  u'user': u'XXXXXXXXXXXX-compute@developer.gserviceaccount.com',\n",
       "  u'zone': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f'},\n",
       " {u'id': u'5833448206003758038',\n",
       "  u'insertTime': u'2017-03-23T07:35:37.539-07:00',\n",
       "  u'kind': u'compute#operation',\n",
       "  u'name': u'operation-1490279737352-54b66ca5c5f41-628c11ba-8a8aa256',\n",
       "  u'operationType': u'resizeDisk',\n",
       "  u'progress': 0,\n",
       "  u'selfLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/operations/operation-1490279737352-54b66ca5c5f41-628c11ba-8a8aa256',\n",
       "  u'status': u'PENDING',\n",
       "  u'targetId': u'7245561722679747529',\n",
       "  u'targetLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/disks/es-cluster-disk-02',\n",
       "  u'user': u'XXXXXXXXXXXX-compute@developer.gserviceaccount.com',\n",
       "  u'zone': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f'},\n",
       " {u'id': u'5695048485527512022',\n",
       "  u'insertTime': u'2017-03-23T07:35:38.017-07:00',\n",
       "  u'kind': u'compute#operation',\n",
       "  u'name': u'operation-1490279737854-54b66ca640832-fb8866bb-5db62be4',\n",
       "  u'operationType': u'resizeDisk',\n",
       "  u'progress': 0,\n",
       "  u'selfLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/operations/operation-1490279737854-54b66ca640832-fb8866bb-5db62be4',\n",
       "  u'status': u'PENDING',\n",
       "  u'targetId': u'5410346553858483144',\n",
       "  u'targetLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/disks/es-cluster-disk-03',\n",
       "  u'user': u'XXXXXXXXXXXX-compute@developer.gserviceaccount.com',\n",
       "  u'zone': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f'}]"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "results = []\n",
    "for name, disk in new_disks:\n",
    "    config = {\n",
    "            'sizeGb': disk_size_gb\n",
    "        }\n",
    "\n",
    "    result = compute.disks().resize(project=target_project_id, zone=target_zone,\n",
    "                                      disk=name,\n",
    "                                      body=config).execute()\n",
    "    results.append(result)\n",
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a02fe-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a04d4-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a0132-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "ディスクサイズが変更されており、statusがREADYであることを確認します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a04d4-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a06aa-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a02fe-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(u'READY', u'1024'), (u'READY', u'1024'), (u'READY', u'1024')]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "status = []\n",
    "for name, disk in new_disks:\n",
    "    result = compute.disks().get(project=target_project_id, zone=target_zone,\n",
    "                                 disk=name).execute()\n",
    "    status.append((result['status'], result['sizeGb']))\n",
    "status"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a06aa-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a0894-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a04d4-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "Configを指示してマシンを起動します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a0894-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a0a60-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a06aa-1a4d-11e7-b31a-0242ac12000d"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('es-cluster-01',\n",
       "  {u'id': u'8250746789074723756',\n",
       "   u'insertTime': u'2017-03-23T07:35:47.882-07:00',\n",
       "   u'kind': u'compute#operation',\n",
       "   u'name': u'operation-1490279747176-54b66caf24641-0400efe4-e25d2ffd',\n",
       "   u'operationType': u'insert',\n",
       "   u'progress': 0,\n",
       "   u'selfLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/operations/operation-1490279747176-54b66caf24641-0400efe4-e25d2ffd',\n",
       "   u'status': u'PENDING',\n",
       "   u'targetId': u'9129921960296211372',\n",
       "   u'targetLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/instances/es-cluster-01',\n",
       "   u'user': u'XXXXXXXXXXXX-compute@developer.gserviceaccount.com',\n",
       "   u'zone': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f'}),\n",
       " ('es-cluster-02',\n",
       "  {u'id': u'1509235558562292651',\n",
       "   u'insertTime': u'2017-03-23T07:35:48.982-07:00',\n",
       "   u'kind': u'compute#operation',\n",
       "   u'name': u'operation-1490279748182-54b66cb019ff1-4e249edb-0878d0d6',\n",
       "   u'operationType': u'insert',\n",
       "   u'progress': 0,\n",
       "   u'selfLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/operations/operation-1490279748182-54b66cb019ff1-4e249edb-0878d0d6',\n",
       "   u'status': u'PENDING',\n",
       "   u'targetId': u'5386367412142498731',\n",
       "   u'targetLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/instances/es-cluster-02',\n",
       "   u'user': u'XXXXXXXXXXXX-compute@developer.gserviceaccount.com',\n",
       "   u'zone': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f'}),\n",
       " ('es-cluster-03',\n",
       "  {u'id': u'214349729666851754',\n",
       "   u'insertTime': u'2017-03-23T07:35:49.996-07:00',\n",
       "   u'kind': u'compute#operation',\n",
       "   u'name': u'operation-1490279749282-54b66cb1268d0-76690433-44f673c6',\n",
       "   u'operationType': u'insert',\n",
       "   u'progress': 0,\n",
       "   u'selfLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/operations/operation-1490279749282-54b66cb1268d0-76690433-44f673c6',\n",
       "   u'status': u'PENDING',\n",
       "   u'targetId': u'8652397038081565610',\n",
       "   u'targetLink': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f/instances/es-cluster-03',\n",
       "   u'user': u'XXXXXXXXXXXX-compute@developer.gserviceaccount.com',\n",
       "   u'zone': u'https://www.googleapis.com/compute/v1/projects/testdev/zones/us-central1-f'})]"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "new_vms = []\n",
    "for i, disk in enumerate(new_disks):\n",
    "    name = '{0}{1:02d}'.format(instance_name, i + 1)\n",
    "    config = {\n",
    "            'name': name,\n",
    "            'machineType': \"zones/{}/machineTypes/{}\".format(target_zone, machine_type),\n",
    "            'disks': [\n",
    "                {\n",
    "                    'boot': True,\n",
    "                    'autoDelete': True,\n",
    "                    'source': disk[1]['targetLink'],\n",
    "                }\n",
    "            ],\n",
    "            'networkInterfaces': [{\n",
    "                'network': 'global/networks/default',\n",
    "                'accessConfigs': [\n",
    "                    {'type': 'ONE_TO_ONE_NAT', 'name': 'External NAT'}\n",
    "                ]\n",
    "            }],\n",
    "            'serviceAccounts': [{\n",
    "                'email': 'default',\n",
    "                'scopes': [\n",
    "                    'https://www.googleapis.com/auth/devstorage.read_write',\n",
    "                    'https://www.googleapis.com/auth/logging.write'\n",
    "                ]\n",
    "            }],\n",
    "            'metadata': {\n",
    "                'items': []\n",
    "            }\n",
    "        }\n",
    "\n",
    "    new_vm = compute.instances().insert(project=target_project_id, zone=target_zone, body=config).execute()\n",
    "    new_vms.append((name, new_vm))\n",
    "new_vms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a0a60-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a0c2c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a0894-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "statusがRUNNINGであることを確認する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a0c2c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a0df8-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a0a60-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[u'RUNNING', u'RUNNING', u'RUNNING']"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "status = []\n",
    "for name, instance in new_vms:\n",
    "    s = compute.instances().get(project=target_project_id, zone=target_zone, instance=name).execute()\n",
    "    status.append(s['status'])\n",
    "status"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a0df8-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a0fd8-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a0c2c-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "対象マシンのIPアドレスを列挙する。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a0fd8-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a11a4-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a0df8-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[('es-cluster-01', u'10.128.0.2', u'130.211.181.251'),\n",
       " ('es-cluster-02', u'10.128.0.3', u'35.184.36.21'),\n",
       " ('es-cluster-03', u'10.128.0.4', u'35.184.148.62')]"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "host_list = []\n",
    "for name, instance in new_vms:\n",
    "    s = compute.instances().get(project=target_project_id, zone=target_zone, instance=name).execute()\n",
    "    host_list.append((name, s['networkInterfaces'][0]['networkIP'],\n",
    "                      s['networkInterfaces'][0]['accessConfigs'][0]['natIP']))\n",
    "host_list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a11a4-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a1370-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a0fd8-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "pingが通ることを確認します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a1370-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a1546-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a11a4-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "['PING 130.211.181.251 (130.211.181.251): 56 data bytes\\n64 bytes from 130.211.181.251: icmp_seq=0 ttl=55 time=127.655 ms\\n64 bytes from 130.211.181.251: icmp_seq=1 ttl=55 time=127.055 ms\\n64 bytes from 130.211.181.251: icmp_seq=2 ttl=55 time=127.052 ms\\n64 bytes from 130.211.181.251: icmp_seq=3 ttl=55 time=127.051 ms\\n--- 130.211.181.251 ping statistics ---\\n4 packets transmitted, 4 packets received, 0% packet loss\\nround-trip min/avg/max/stddev = 127.051/127.203/127.655/0.261 ms',\n",
       " 'PING 35.184.36.21 (35.184.36.21): 56 data bytes\\n64 bytes from 35.184.36.21: icmp_seq=0 ttl=55 time=128.380 ms\\n64 bytes from 35.184.36.21: icmp_seq=1 ttl=55 time=127.199 ms\\n64 bytes from 35.184.36.21: icmp_seq=2 ttl=55 time=127.167 ms\\n64 bytes from 35.184.36.21: icmp_seq=3 ttl=55 time=127.166 ms\\n--- 35.184.36.21 ping statistics ---\\n4 packets transmitted, 4 packets received, 0% packet loss\\nround-trip min/avg/max/stddev = 127.166/127.478/128.380/0.521 ms',\n",
       " 'PING 35.184.148.62 (35.184.148.62): 56 data bytes\\n64 bytes from 35.184.148.62: icmp_seq=0 ttl=55 time=129.412 ms\\n64 bytes from 35.184.148.62: icmp_seq=1 ttl=55 time=128.852 ms\\n64 bytes from 35.184.148.62: icmp_seq=2 ttl=55 time=128.828 ms\\n64 bytes from 35.184.148.62: icmp_seq=3 ttl=55 time=128.784 ms\\n--- 35.184.148.62 ping statistics ---\\n4 packets transmitted, 4 packets received, 0% packet loss\\nround-trip min/avg/max/stddev = 128.784/128.969/129.412/0.257 ms']"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import commands, sys\n",
    "results = map(lambda host:commands.getoutput('ping -c 4 ' + host[2]), host_list)\n",
    "results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a1546-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a171c-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a1370-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "Ansibleによる疎通確認も実施します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a171c-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a1906-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a1546-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "es-cluster-01 ansible_host=130.211.181.251 ansible_ssh_user=ansible ansible_ssh_private_key_file=~/.ssh/ansible_id_rsa\r\n",
      "es-cluster-02 ansible_host=35.184.36.21 ansible_ssh_user=ansible ansible_ssh_private_key_file=~/.ssh/ansible_id_rsa\r\n",
      "es-cluster-03 ansible_host=35.184.148.62 ansible_ssh_user=ansible ansible_ssh_private_key_file=~/.ssh/ansible_id_rsa\r\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "import tempfile\n",
    "temp_dir = tempfile.mkdtemp()\n",
    "\n",
    "with open(os.path.join(temp_dir, 'hosts'), 'w') as f:\n",
    "    for name, priv, pub in host_list:\n",
    "        f.write('{} ansible_host={} ansible_ssh_user=ansible ansible_ssh_private_key_file=~/.ssh/ansible_id_rsa\\n'.format(name, pub))\n",
    "!cat {temp_dir}/hosts"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a1906-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a1b90-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a171c-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/cffi/model.py:525: UserWarning: 'point_conversion_form_t' has no values explicitly defined; guessing that it is equivalent to 'unsigned int'\n",
      "  % self._get_c_name())\n",
      "\u001b[0;32mes-cluster-03 | SUCCESS => {\n",
      "    \"changed\": false, \n",
      "    \"ping\": \"pong\"\n",
      "}\u001b[0m\n",
      "\u001b[0;32mes-cluster-01 | SUCCESS => {\n",
      "    \"changed\": false, \n",
      "    \"ping\": \"pong\"\n",
      "}\u001b[0m\n",
      "\u001b[0;32mes-cluster-02 | SUCCESS => {\n",
      "    \"changed\": false, \n",
      "    \"ping\": \"pong\"\n",
      "}\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!ansible -m ping -i {temp_dir}/hosts all"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a1b90-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a1d7a-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a1906-1a4d-11e7-b31a-0242ac12000d"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/cffi/model.py:525: UserWarning: 'point_conversion_form_t' has no values explicitly defined; guessing that it is equivalent to 'unsigned int'\n",
      "  % self._get_c_name())\n",
      "\u001b[0;32mes-cluster-03 | SUCCESS | rc=0 >>\n",
      "Filesystem      Size  Used Avail Use% Mounted on\n",
      "/dev/sda1       1.0T  1.4G 1023G   1% /\n",
      "devtmpfs         30G     0   30G   0% /dev\n",
      "tmpfs            30G     0   30G   0% /dev/shm\n",
      "tmpfs            30G  8.3M   30G   1% /run\n",
      "tmpfs            30G     0   30G   0% /sys/fs/cgroup\n",
      "tmpfs           5.9G     0  5.9G   0% /run/user/1000\n",
      "\u001b[0m\n",
      "\u001b[0;32mes-cluster-01 | SUCCESS | rc=0 >>\n",
      "Filesystem      Size  Used Avail Use% Mounted on\n",
      "/dev/sda1       1.0T  1.4G 1023G   1% /\n",
      "devtmpfs         30G     0   30G   0% /dev\n",
      "tmpfs            30G     0   30G   0% /dev/shm\n",
      "tmpfs            30G  8.3M   30G   1% /run\n",
      "tmpfs            30G     0   30G   0% /sys/fs/cgroup\n",
      "tmpfs           5.9G     0  5.9G   0% /run/user/1000\n",
      "\u001b[0m\n",
      "\u001b[0;32mes-cluster-02 | SUCCESS | rc=0 >>\n",
      "Filesystem      Size  Used Avail Use% Mounted on\n",
      "/dev/sda1       1.0T  1.4G 1023G   1% /\n",
      "devtmpfs         30G     0   30G   0% /dev\n",
      "tmpfs            30G     0   30G   0% /dev/shm\n",
      "tmpfs            30G  8.3M   30G   1% /run\n",
      "tmpfs            30G     0   30G   0% /sys/fs/cgroup\n",
      "tmpfs           5.9G     0  5.9G   0% /run/user/1000\n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "!ansible -a 'df -h' -i {temp_dir}/hosts all"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a1d7a-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a1f46-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a1b90-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "全てのインスタンスに対しpingが通ればGCEインスタンス生成は完了です。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a1f46-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a2126-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a1d7a-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "### 名前による疎通確認\n",
    "\n",
    "GCEインスタンス中では、(内部的には)インスタンス名により名前解決することができる。このことを念のため確認しておく。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a2126-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a22fc-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a1f46-1a4d-11e7-b31a-0242ac12000d"
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python2.7/dist-packages/cffi/model.py:525: UserWarning: 'point_conversion_form_t' has no values explicitly defined; guessing that it is equivalent to 'unsigned int'\n",
      "  % self._get_c_name())\n",
      "\u001b[0;32mes-cluster-01 | SUCCESS | rc=0 >>\n",
      "PING es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2) 56(84) bytes of data.\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=1 ttl=64 time=0.059 ms\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=2 ttl=64 time=0.066 ms\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=3 ttl=64 time=0.081 ms\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=4 ttl=64 time=0.059 ms\n",
      "\n",
      "--- es-cluster-01.c.testdev.internal ping statistics ---\n",
      "4 packets transmitted, 4 received, 0% packet loss, time 2999ms\n",
      "rtt min/avg/max/mdev = 0.059/0.066/0.081/0.010 ms\n",
      "\u001b[0m\n",
      "\u001b[0;32mes-cluster-03 | SUCCESS | rc=0 >>\n",
      "PING es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2) 56(84) bytes of data.\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=1 ttl=64 time=1.53 ms\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=2 ttl=64 time=0.432 ms\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=3 ttl=64 time=0.388 ms\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=4 ttl=64 time=0.327 ms\n",
      "\n",
      "--- es-cluster-01.c.testdev.internal ping statistics ---\n",
      "4 packets transmitted, 4 received, 0% packet loss, time 3001ms\n",
      "rtt min/avg/max/mdev = 0.327/0.669/1.531/0.499 ms\n",
      "\u001b[0m\n",
      "\u001b[0;32mes-cluster-02 | SUCCESS | rc=0 >>\n",
      "PING es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2) 56(84) bytes of data.\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=1 ttl=64 time=1.37 ms\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=2 ttl=64 time=0.370 ms\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=3 ttl=64 time=0.338 ms\n",
      "64 bytes from es-cluster-01.c.testdev.internal (XXX.XXX.XXX.2): icmp_seq=4 ttl=64 time=0.348 ms\n",
      "\n",
      "--- es-cluster-01.c.testdev.internal ping statistics ---\n",
      "4 packets transmitted, 4 received, 0% packet loss, time 3001ms\n",
      "rtt min/avg/max/mdev = 0.338/0.607/1.372/0.441 ms\n",
      "\u001b[0m\n",
      "/usr/local/lib/python2.7/dist-packages/cffi/model.py:525: UserWarning: 'point_conversion_form_t' has no values explicitly defined; guessing that it is equivalent to 'unsigned int'\n",
      "  % self._get_c_name())\n",
      "\u001b[0;32mes-cluster-01 | SUCCESS | rc=0 >>\n",
      "PING es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3) 56(84) bytes of data.\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=1 ttl=64 time=0.296 ms\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=2 ttl=64 time=0.387 ms\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=3 ttl=64 time=0.366 ms\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=4 ttl=64 time=0.350 ms\n",
      "\n",
      "--- es-cluster-02.c.testdev.internal ping statistics ---\n",
      "4 packets transmitted, 4 received, 0% packet loss, time 3001ms\n",
      "rtt min/avg/max/mdev = 0.296/0.349/0.387/0.040 ms\n",
      "\u001b[0m\n",
      "\u001b[0;32mes-cluster-03 | SUCCESS | rc=0 >>\n",
      "PING es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3) 56(84) bytes of data.\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=1 ttl=64 time=1.41 ms\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=2 ttl=64 time=0.341 ms\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=3 ttl=64 time=0.343 ms\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=4 ttl=64 time=0.380 ms\n",
      "\n",
      "--- es-cluster-02.c.testdev.internal ping statistics ---\n",
      "4 packets transmitted, 4 received, 0% packet loss, time 3001ms\n",
      "rtt min/avg/max/mdev = 0.341/0.619/1.415/0.460 ms\n",
      "\u001b[0m\n",
      "\u001b[0;32mes-cluster-02 | SUCCESS | rc=0 >>\n",
      "PING es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3) 56(84) bytes of data.\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=1 ttl=64 time=0.072 ms\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=2 ttl=64 time=0.052 ms\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=3 ttl=64 time=0.028 ms\n",
      "64 bytes from es-cluster-02.c.testdev.internal (XXX.XXX.XXX.3): icmp_seq=4 ttl=64 time=0.076 ms\n",
      "\n",
      "--- es-cluster-02.c.testdev.internal ping statistics ---\n",
      "4 packets transmitted, 4 received, 0% packet loss, time 2999ms\n",
      "rtt min/avg/max/mdev = 0.028/0.057/0.076/0.019 ms\n",
      "\u001b[0m\n",
      "/usr/local/lib/python2.7/dist-packages/cffi/model.py:525: UserWarning: 'point_conversion_form_t' has no values explicitly defined; guessing that it is equivalent to 'unsigned int'\n",
      "  % self._get_c_name())\n",
      "\u001b[0;32mes-cluster-03 | SUCCESS | rc=0 >>\n",
      "PING es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4) 56(84) bytes of data.\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=1 ttl=64 time=0.058 ms\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=2 ttl=64 time=0.068 ms\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=3 ttl=64 time=0.060 ms\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=4 ttl=64 time=0.059 ms\n",
      "\n",
      "--- es-cluster-03.c.testdev.internal ping statistics ---\n",
      "4 packets transmitted, 4 received, 0% packet loss, time 2999ms\n",
      "rtt min/avg/max/mdev = 0.058/0.061/0.068/0.006 ms\n",
      "\u001b[0m\n",
      "\u001b[0;32mes-cluster-02 | SUCCESS | rc=0 >>\n",
      "PING es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4) 56(84) bytes of data.\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=1 ttl=64 time=0.289 ms\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=2 ttl=64 time=0.381 ms\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=3 ttl=64 time=0.350 ms\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=4 ttl=64 time=0.361 ms\n",
      "\n",
      "--- es-cluster-03.c.testdev.internal ping statistics ---\n",
      "4 packets transmitted, 4 received, 0% packet loss, time 3001ms\n",
      "rtt min/avg/max/mdev = 0.289/0.345/0.381/0.036 ms\n",
      "\u001b[0m\n",
      "\u001b[0;32mes-cluster-01 | SUCCESS | rc=0 >>\n",
      "PING es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4) 56(84) bytes of data.\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=1 ttl=64 time=1.23 ms\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=2 ttl=64 time=0.415 ms\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=3 ttl=64 time=0.382 ms\n",
      "64 bytes from es-cluster-03.c.testdev.internal (XXX.XXX.XXX.4): icmp_seq=4 ttl=64 time=0.407 ms\n",
      "\n",
      "--- es-cluster-03.c.testdev.internal ping statistics ---\n",
      "4 packets transmitted, 4 received, 0% packet loss, time 3000ms\n",
      "rtt min/avg/max/mdev = 0.382/0.610/1.239/0.364 ms\n",
      "\u001b[0m\n"
     ]
    }
   ],
   "source": [
    "for name, instance in new_vms:\n",
    "    !ansible -a 'ping -c 4 {name}' -i {temp_dir}/hosts all"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a22fc-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a24c8-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a2126-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "## Inventory作成用のインスタンスリストを作成"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a24c8-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a2694-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a22fc-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "source": [
    "ホストのパブリックDNS一覧をリスト化します  \n",
    "次のブックの[Inventory作成](01_03_Set_Inventory.ipynb)で使用します。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {
    "lc_cell_meme": {
     "current": "6f8a2694-1a4d-11e7-b31a-0242ac12000d",
     "next": "6f8a286a-1a4d-11e7-b31a-0242ac12000d",
     "previous": "6f8a24c8-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "host_list = {'host1_name':'es-cluster-01','host2_name':'es-cluster-02','host3_name':'es-cluster-03','host1_addr':'130.211.181.251','host2_addr':'35.184.36.21','host3_addr':'35.184.148.62'}\n"
     ]
    }
   ],
   "source": [
    "host_list_pub = ['\\'host{0}_name\\':\\'{1}\\''.format(str(i+1), x[0]) for i,x in enumerate(host_list)] \n",
    "host_list_priv = ['\\'host{0}_addr\\':\\'{1}\\''.format(str(i+1), x[2]) for i,x in enumerate(host_list)] \n",
    "print('host_list = {{{0}}}'.format(','.join(host_list_pub + host_list_priv)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true,
    "lc_cell_meme": {
     "current": "6f8a286a-1a4d-11e7-b31a-0242ac12000d",
     "next": null,
     "previous": "6f8a2694-1a4d-11e7-b31a-0242ac12000d"
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.9"
  },
  "lc_notebook_meme": {
   "current": "6f89a7dc-1a4d-11e7-b31a-0242ac12000d",
   "lc_server_signature": {
    "history": []
   }
  },
  "toc": {
   "colors": {
    "hover_highlight": "#DAA520",
    "navigate_num": "#000000",
    "navigate_text": "#333333",
    "running_highlight": "#FF0000",
    "selected_highlight": "#FFD700",
    "sidebar_border": "#EEEEEE",
    "wrapper_background": "#FFFFFF"
   },
   "moveMenuLeft": true,
   "nav_menu": {
    "height": "30px",
    "width": "252px"
   },
   "navigate_menu": true,
   "number_sections": true,
   "sideBar": true,
   "threshold": 4,
   "toc_cell": false,
   "toc_position": {
    "height": "573px",
    "left": "0px",
    "right": "1654.67px",
    "top": "106px",
    "width": "292px"
   },
   "toc_section_display": "block",
   "toc_window_display": true,
   "widenNotebook": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}