VAE_mshvartsman.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "toc": "true"
   },
   "source": [
    "# Table of Contents\n",
    " <p><div class=\"lev1 toc-item\"><a href=\"#TF-intro\" data-toc-modified-id=\"TF-intro-1\"><span class=\"toc-item-num\">1&nbsp;&nbsp;</span>TF intro</a></div><div class=\"lev1 toc-item\"><a href=\"#MLP-on-MNIST\" data-toc-modified-id=\"MLP-on-MNIST-2\"><span class=\"toc-item-num\">2&nbsp;&nbsp;</span>MLP on MNIST</a></div><div class=\"lev1 toc-item\"><a href=\"#VAEs!\" data-toc-modified-id=\"VAEs!-3\"><span class=\"toc-item-num\">3&nbsp;&nbsp;</span>VAEs!</a></div>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TF intro\n",
    "\n",
    "This guide assumes you can read through basic Python code or use your google skills to catch up on that as needed. We begin by understanding how tensorflow works. The key point to remember is that all the tensorflow computation happens in a graph, and all that you get to do in python is to manipulate and run that graph. This creates a programming paradigm that looks a lot like python but is actually quite different. We begin with a simple logistic regression example. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "import tensorflow as tf\n",
    "import numpy as np\n",
    "\n",
    "n_obs = 1000\n",
    "n_features = 5\n",
    "\n",
    "x_ph = tf.placeholder(shape=(n_obs, n_features), name=\"x_ph\", dtype=tf.float32)\n",
    "beta_init = np.random.normal(size=(n_features, 1))\n",
    "beta_hat = tf.Variable(beta_init, dtype=tf.float32, name=\"beta_hat\")\n",
    "y_hat = tf.nn.sigmoid(tf.matmul(x_ph, beta_hat), name=\"yhat\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can visualize thie graph, and then explain what the different parts are: \n",
    "\n",
    "This code is a modification from the DeepDream notebook. There is more visualization / exploration that can be done using tensorboard, which is the tool this uses. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# TensorFlow Graph visualizer code\n",
    "# https://stackoverflow.com/questions/41388673/visualizing-a-tensorflow-graph-in-jupyter-doesnt-work\n",
    "import numpy as np\n",
    "from IPython.display import clear_output, Image, display, HTML\n",
    "\n",
    "def strip_consts(graph_def, max_const_size=32):\n",
    "    \"\"\"Strip large constant values from graph_def.\"\"\"\n",
    "    strip_def = tf.GraphDef()\n",
    "    for n0 in graph_def.node:\n",
    "        n = strip_def.node.add() \n",
    "        n.MergeFrom(n0)\n",
    "        if n.op == 'Const':\n",
    "            tensor = n.attr['value'].tensor\n",
    "            size = len(tensor.tensor_content)\n",
    "            if size > max_const_size:\n",
    "                tensor.tensor_content = bytes(\"<stripped %d bytes>\"%size, 'utf-8')\n",
    "    return strip_def\n",
    "\n",
    "def show_graph(graph_def, max_const_size=32):\n",
    "    \"\"\"Visualize TensorFlow graph.\"\"\"\n",
    "    if hasattr(graph_def, 'as_graph_def'):\n",
    "        graph_def = graph_def.as_graph_def()\n",
    "    strip_def = strip_consts(graph_def, max_const_size=max_const_size)\n",
    "    code = \"\"\"\n",
    "        <script src=\"//cdnjs.cloudflare.com/ajax/libs/polymer/0.3.3/platform.js\"></script>\n",
    "        <script>\n",
    "          function load() {{\n",
    "            document.getElementById(\"{id}\").pbtxt = {data};\n",
    "          }}\n",
    "        </script>\n",
    "        <link rel=\"import\" href=\"https://tensorboard.appspot.com/tf-graph-basic.build.html\" onload=load()>\n",
    "        <div style=\"height:800px\">\n",
    "          <tf-graph-basic id=\"{id}\"></tf-graph-basic>\n",
    "        </div>\n",
    "    \"\"\".format(data=repr(str(strip_def)), id='graph'+str(np.random.rand()))\n",
    "\n",
    "    iframe = \"\"\"\n",
    "        <iframe seamless style=\"width:1200px;height:800px;border:0\" srcdoc=\"{}\"></iframe>\n",
    "    \"\"\".format(code.replace('\"', '&quot;'))\n",
    "    display(HTML(iframe))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe seamless style=\"width:1200px;height:800px;border:0\" srcdoc=\"\n",
       "        <script src=&quot;//cdnjs.cloudflare.com/ajax/libs/polymer/0.3.3/platform.js&quot;></script>\n",
       "        <script>\n",
       "          function load() {\n",
       "            document.getElementById(&quot;graph0.11813897124247885&quot;).pbtxt = 'node {\\n  name: &quot;x_ph&quot;\\n  op: &quot;Placeholder&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 1000\\n        }\\n        dim {\\n          size: 5\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;beta_hat/initial_value&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 5\\n          }\\n          dim {\\n            size: 1\\n          }\\n        }\\n        tensor_content: &quot;\\\\345\\\\315\\\\245\\\\277\\\\354\\\\242\\\\276\\\\277\\\\201G\\\\343\\\\276\\\\373\\\\003\\\\350?\\\\273\\\\303\\\\225\\\\277&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;beta_hat&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 5\\n        }\\n        dim {\\n          size: 1\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;beta_hat/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;beta_hat&quot;\\n  input: &quot;beta_hat/initial_value&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@beta_hat&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;beta_hat/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;beta_hat&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@beta_hat&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;x_ph&quot;\\n  input: &quot;beta_hat/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;yhat&quot;\\n  op: &quot;Sigmoid&quot;\\n  input: &quot;MatMul&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\n';\n",
       "          }\n",
       "        </script>\n",
       "        <link rel=&quot;import&quot; href=&quot;https://tensorboard.appspot.com/tf-graph-basic.build.html&quot; onload=load()>\n",
       "        <div style=&quot;height:800px&quot;>\n",
       "          <tf-graph-basic id=&quot;graph0.11813897124247885&quot;></tf-graph-basic>\n",
       "        </div>\n",
       "    \"></iframe>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_graph(tf.get_default_graph().as_graph_def())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "What we do in tensorflow is construct graphs like this, and then evaluate nodes. Each graph node is associated with some code. When we evaluate a node like `y_hat`, tensorflow figures out what nodes it depends on, evaluates all of those nodes, and then evaluates `y_hat`. In this graph, there are three types of nodes (`Tensor`s), a `Variable`, a `placeholder`, and a vanilla `Tensor` that is none of the above. Tensors have no state: they are computable from the rest of the graph. Variables have state (they're the only thing we can optimize / save / load). Placeholders are not computable and don't have state: we must feed values into them. We can also feed values into other tensors, but TF will explicitly complain if we fail to feed a value into a placeholder. \n",
    "\n",
    "Our python objects like `beta_hat` are references to the TF graph nodes, not the nodes themselves (i.e. copying the python object does not dupliate the graph node). \n",
    "\n",
    "To evaluate a graph we need to associate it with a session, and then either a tensor's `eval` method or the session's `run` method. The difference is that we can `run` multiple tensors together, which might be useful if they share dependencies. \n",
    "\n",
    "Now we define the loss, generate synth data, and optimize: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter 0, loss 1257.209106\n",
      "Iter 100, loss 425.954163\n",
      "Iter 200, loss 425.796875\n",
      "Iter 300, loss 425.795349\n",
      "Iter 400, loss 425.795319\n",
      "Iter 500, loss 425.795227\n",
      "Iter 600, loss 425.795227\n",
      "Iter 700, loss 425.795227\n",
      "Iter 800, loss 425.795227\n",
      "Iter 900, loss 425.795227\n",
      "Iter 1000, loss 425.795227\n",
      "Iter 1100, loss 425.795227\n",
      "Iter 1200, loss 425.795227\n",
      "Iter 1300, loss 425.795227\n",
      "Iter 1400, loss 425.795227\n",
      "Iter 1500, loss 425.795227\n",
      "Iter 1600, loss 425.795227\n",
      "Iter 1700, loss 425.795227\n",
      "Iter 1800, loss 425.795227\n",
      "Iter 1900, loss 425.795227\n",
      "Iter 2000, loss 425.795227\n",
      "Iter 2100, loss 425.795227\n",
      "Iter 2200, loss 425.795227\n",
      "Iter 2300, loss 425.795227\n",
      "Iter 2400, loss 425.795227\n",
      "0.814\n"
     ]
    }
   ],
   "source": [
    "from scipy.special import expit as logistic\n",
    "\n",
    "true_beta = np.random.normal(size=(n_features, 1))\n",
    "x = np.random.normal(size=(n_obs, n_features))\n",
    "y = np.random.binomial(n=1, p=logistic(x @ true_beta))\n",
    "\n",
    "y_ph = tf.placeholder(shape=(n_obs, 1), name=\"y_ph\", dtype=tf.float32)\n",
    "\n",
    "logistic_loss = -tf.reduce_sum(y_ph * tf.log(1e-10 + y_hat)+ (1-y_ph) * tf.log(1e-10 + 1 - y_hat))\n",
    "\n",
    "# if we needed the gradients for some reason, e.g. to pass to an external optimizier or to plot\n",
    "# grads = tf.gradients(logistic_loss, beta_hat)\n",
    "\n",
    "# create the optimizer\n",
    "optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.0005)\n",
    "\n",
    "# this is the long way of doing it\n",
    "# grads_and_vars = optimizer.compute_gradients(logistic_loss, beta_hat)\n",
    "# optionally, modify gradients here (e.g. threshold)\n",
    "# minimize_op = optimizer.apply_gradients(grads_and_vars)\n",
    "\n",
    "# the short way\n",
    "minimize_op = optimizer.minimize(logistic_loss) # by default, minimize w.r.t. all variables\n",
    "\n",
    "sess = tf.Session()\n",
    "sess.run(tf.global_variables_initializer())\n",
    "for i in range(2500):\n",
    "    if i % 100 == 0:\n",
    "        l = logistic_loss.eval(session=sess, feed_dict={x_ph:x, y_ph:y})\n",
    "        print(\"Iter %i, loss %f\" % (i, l))\n",
    "    sess.run(minimize_op, feed_dict={x_ph:x, y_ph:y})\n",
    "\n",
    "accuracy = np.average(tf.equal(tf.round(y_hat), y_ph).eval(session=sess, feed_dict={x_ph:x, y_ph:y}))\n",
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Iter 0, loss 1257.209106\n",
      "Iter 100, loss 621.341553\n",
      "Iter 200, loss 453.863708\n",
      "Iter 300, loss 430.841217\n",
      "Iter 400, loss 426.745361\n",
      "Iter 500, loss 425.957336\n",
      "Iter 600, loss 425.818573\n",
      "Iter 700, loss 425.798065\n",
      "Iter 800, loss 425.795532\n",
      "Iter 900, loss 425.795319\n",
      "0.814\n"
     ]
    }
   ],
   "source": [
    "# another optimizer\n",
    "optimizer = tf.train.AdamOptimizer(learning_rate=0.01)\n",
    "minimize_op = optimizer.minimize(logistic_loss) # by default, minimize w.r.t. all variables\n",
    "\n",
    "sess.run(tf.global_variables_initializer())\n",
    "for i in range(1000):\n",
    "    if i % 100 == 0:\n",
    "        l = logistic_loss.eval(session=sess, feed_dict={x_ph:x, y_ph:y})\n",
    "        print(\"Iter %i, loss %f\" % (i, l))\n",
    "    sess.run(minimize_op, feed_dict={x_ph:x, y_ph:y})\n",
    "\n",
    "accuracy = np.average(tf.equal(tf.round(y_hat), y_ph).eval(session=sess, feed_dict={x_ph:x, y_ph:y}))\n",
    "print(accuracy)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "One more tutorial point is on how to print things. Since a tensor only has a value when the graph is executed, inspecting things is trickier than usual. The `Print` op returns the same op as its input, but prints as a side-effect. This means we need to inject the op into the graph. Unfortunately, the print happens on the C++ end, so we will see the logging messages in the jupyter server log (or in a shell). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "logistic_loss_with_print = tf.Print(input_=logistic_loss, data=[x, logistic_loss])\n",
    "\n",
    "_ = logistic_loss_with_print.eval(session=sess, feed_dict={x_ph:x, y_ph:y})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# MLP on MNIST\n",
    "\n",
    "Now we build some neural network building blocks we will reuse for VAEs. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Successfully downloaded train-images-idx3-ubyte.gz 9912422 bytes.\n",
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Successfully downloaded train-labels-idx1-ubyte.gz 28881 bytes.\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Successfully downloaded t10k-images-idx3-ubyte.gz 1648877 bytes.\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Successfully downloaded t10k-labels-idx1-ubyte.gz 4542 bytes.\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n"
     ]
    }
   ],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "global_dtype = tf.float32\n",
    "mnist = input_data.read_data_sets('MNIST_data', one_hot=True)\n",
    "input_size = mnist.train.images.shape[1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Define our neural network building blocks. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "def _dense_mlp_layer(x, input_size, out_size, nonlinearity=tf.nn.softmax, name_prefix=\"\"):\n",
    "    w_init = tf.truncated_normal(shape=[input_size, out_size], stddev=0.001)\n",
    "    b_init = tf.ones(shape=[out_size]) * 0.1\n",
    "    W = tf.Variable(w_init, name=\"%s_W\" % name_prefix)\n",
    "    b = tf.Variable(b_init, name=\"%s_b\" % name_prefix)\n",
    "    out = nonlinearity(tf.matmul(x, W) + b)\n",
    "    return out, [W, b]\n",
    "\n",
    "def _mlp(x, n_layers, units_per_layer, input_size, out_size, nonlinearity=tf.tanh):\n",
    "    train_vars = []\n",
    "\n",
    "    x, v = _dense_mlp_layer(x, input_size, units_per_layer, nonlinearity, name_prefix=\"into_hidden\")\n",
    "    train_vars.extend(v)\n",
    "    # exploit the fact that repeatedly calling the same TF function creates multiple ops. \n",
    "    # no need to hang onto the intermediate layer handles (though we can get them back if we need them)\n",
    "    for l in range(n_layers-1):\n",
    "        x, v = _dense_mlp_layer(x, units_per_layer, units_per_layer, nonlinearity, name_prefix=\"hidden\")\n",
    "        train_vars.extend(v)\n",
    "\n",
    "    x, v = _dense_mlp_layer(x, units_per_layer, out_size, nonlinearity, name_prefix=\"readout\")\n",
    "    train_vars.extend(v)\n",
    "    return x, train_vars"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we construct the graph. The graph and scope boilerplate makes our life easier as far as visualization and debugging is concerned. We can visualize/run only this graph and not the graph for logistic regression (above). "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "mlp_graph = tf.Graph()\n",
    "\n",
    "with mlp_graph.as_default():\n",
    "    with tf.name_scope(\"Feedforward_Net\"):\n",
    "        x = tf.placeholder(shape=[None, input_size], dtype=global_dtype, name='x')\n",
    "        y = tf.placeholder(shape=[None, 10], dtype=global_dtype, name='y')\n",
    "        y_hat, mlp_test_vars = _mlp(x, n_layers=2, units_per_layer=30, input_size=784, out_size=10)\n",
    "\n",
    "    with tf.name_scope(\"Opt_and_loss\"):\n",
    "        cross_entropy = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=y_hat))\n",
    "        # learning rates are much smaller for optimizers like Adam and RMSProp\n",
    "        train_step_mlptest = tf.train.AdamOptimizer(0.01).minimize(cross_entropy)\n",
    "\n",
    "    with tf.name_scope(\"Support_stuff\"):\n",
    "        init = tf.global_variables_initializer()\n",
    "        correct_prediction = tf.equal(tf.argmax(y,1), tf.argmax(y_hat,1))\n",
    "        mlp_acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "scrolled": false
   },
   "source": [
    "We visualize our graph: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe seamless style=\"width:1200px;height:800px;border:0\" srcdoc=\"\n",
       "        <script src=&quot;//cdnjs.cloudflare.com/ajax/libs/polymer/0.3.3/platform.js&quot;></script>\n",
       "        <script>\n",
       "          function load() {\n",
       "            document.getElementById(&quot;graph0.9689909770597044&quot;).pbtxt = 'node {\\n  name: &quot;Feedforward_Net/x&quot;\\n  op: &quot;Placeholder&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: -1\\n        }\\n        dim {\\n          size: 784\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/y&quot;\\n  op: &quot;Placeholder&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: -1\\n        }\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;\\\\020\\\\003\\\\000\\\\000\\\\036\\\\000\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0010000000474974513\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Feedforward_Net/truncated_normal/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Feedforward_Net/truncated_normal/TruncatedNormal&quot;\\n  input: &quot;Feedforward_Net/truncated_normal/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Feedforward_Net/truncated_normal/mul&quot;\\n  input: &quot;Feedforward_Net/truncated_normal/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/ones&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 30\\n          }\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/mul/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Feedforward_Net/ones&quot;\\n  input: &quot;Feedforward_Net/mul/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_W&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 784\\n        }\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_W/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W&quot;\\n  input: &quot;Feedforward_Net/truncated_normal&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_W/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_b&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_b/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b&quot;\\n  input: &quot;Feedforward_Net/mul&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_b/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Feedforward_Net/x&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/add&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Feedforward_Net/MatMul&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/Tanh&quot;\\n  op: &quot;Tanh&quot;\\n  input: &quot;Feedforward_Net/add&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_1/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;\\\\036\\\\000\\\\000\\\\000\\\\036\\\\000\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_1/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_1/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0010000000474974513\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_1/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_1/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_1/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_1/TruncatedNormal&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_1/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_1&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_1/mul&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_1/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/ones_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 30\\n          }\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/mul_1/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/mul_1&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Feedforward_Net/ones_1&quot;\\n  input: &quot;Feedforward_Net/mul_1/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_W&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_W/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/hidden_W&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_W/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/hidden_W&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_b&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_b/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/hidden_b&quot;\\n  input: &quot;Feedforward_Net/mul_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_b/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/hidden_b&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Feedforward_Net/Tanh&quot;\\n  input: &quot;Feedforward_Net/hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/add_1&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Feedforward_Net/MatMul_1&quot;\\n  input: &quot;Feedforward_Net/hidden_b/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/Tanh_1&quot;\\n  op: &quot;Tanh&quot;\\n  input: &quot;Feedforward_Net/add_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_2/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;\\\\036\\\\000\\\\000\\\\000\\\\n\\\\000\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_2/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_2/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0010000000474974513\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_2/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_2/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_2/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_2/TruncatedNormal&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_2/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/truncated_normal_2&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_2/mul&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_2/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/ones_2&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/mul_2/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/mul_2&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Feedforward_Net/ones_2&quot;\\n  input: &quot;Feedforward_Net/mul_2/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_W&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_W/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/readout_W&quot;\\n  input: &quot;Feedforward_Net/truncated_normal_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_W/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/readout_W&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_b&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_b/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/readout_b&quot;\\n  input: &quot;Feedforward_Net/mul_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_b/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/readout_b&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/MatMul_2&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Feedforward_Net/Tanh_1&quot;\\n  input: &quot;Feedforward_Net/readout_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/add_2&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Feedforward_Net/MatMul_2&quot;\\n  input: &quot;Feedforward_Net/readout_b/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/Tanh_2&quot;\\n  op: &quot;Tanh&quot;\\n  input: &quot;Feedforward_Net/add_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Rank&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 2\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Feedforward_Net/Tanh_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Rank_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 2\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Shape_1&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Feedforward_Net/Tanh_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Sub/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Sub&quot;\\n  op: &quot;Sub&quot;\\n  input: &quot;Opt_and_loss/Rank_1&quot;\\n  input: &quot;Opt_and_loss/Sub/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Slice/begin&quot;\\n  op: &quot;Pack&quot;\\n  input: &quot;Opt_and_loss/Sub&quot;\\n  attr {\\n    key: &quot;N&quot;\\n    value {\\n      i: 1\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;axis&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Slice/size&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Slice&quot;\\n  op: &quot;Slice&quot;\\n  input: &quot;Opt_and_loss/Shape_1&quot;\\n  input: &quot;Opt_and_loss/Slice/begin&quot;\\n  input: &quot;Opt_and_loss/Slice/size&quot;\\n  attr {\\n    key: &quot;Index&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/concat/values_0&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: -1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/concat/axis&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/concat&quot;\\n  op: &quot;ConcatV2&quot;\\n  input: &quot;Opt_and_loss/concat/values_0&quot;\\n  input: &quot;Opt_and_loss/Slice&quot;\\n  input: &quot;Opt_and_loss/concat/axis&quot;\\n  attr {\\n    key: &quot;N&quot;\\n    value {\\n      i: 2\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Feedforward_Net/Tanh_2&quot;\\n  input: &quot;Opt_and_loss/concat&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Rank_2&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 2\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Shape_2&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Feedforward_Net/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Sub_1/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Sub_1&quot;\\n  op: &quot;Sub&quot;\\n  input: &quot;Opt_and_loss/Rank_2&quot;\\n  input: &quot;Opt_and_loss/Sub_1/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Slice_1/begin&quot;\\n  op: &quot;Pack&quot;\\n  input: &quot;Opt_and_loss/Sub_1&quot;\\n  attr {\\n    key: &quot;N&quot;\\n    value {\\n      i: 1\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;axis&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Slice_1/size&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Slice_1&quot;\\n  op: &quot;Slice&quot;\\n  input: &quot;Opt_and_loss/Shape_2&quot;\\n  input: &quot;Opt_and_loss/Slice_1/begin&quot;\\n  input: &quot;Opt_and_loss/Slice_1/size&quot;\\n  attr {\\n    key: &quot;Index&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/concat_1/values_0&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: -1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/concat_1/axis&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/concat_1&quot;\\n  op: &quot;ConcatV2&quot;\\n  input: &quot;Opt_and_loss/concat_1/values_0&quot;\\n  input: &quot;Opt_and_loss/Slice_1&quot;\\n  input: &quot;Opt_and_loss/concat_1/axis&quot;\\n  attr {\\n    key: &quot;N&quot;\\n    value {\\n      i: 2\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Feedforward_Net/y&quot;\\n  input: &quot;Opt_and_loss/concat_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/SoftmaxCrossEntropyWithLogits&quot;\\n  op: &quot;SoftmaxCrossEntropyWithLogits&quot;\\n  input: &quot;Opt_and_loss/Reshape&quot;\\n  input: &quot;Opt_and_loss/Reshape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Sub_2/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Sub_2&quot;\\n  op: &quot;Sub&quot;\\n  input: &quot;Opt_and_loss/Rank&quot;\\n  input: &quot;Opt_and_loss/Sub_2/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Slice_2/begin&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Slice_2/size&quot;\\n  op: &quot;Pack&quot;\\n  input: &quot;Opt_and_loss/Sub_2&quot;\\n  attr {\\n    key: &quot;N&quot;\\n    value {\\n      i: 1\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;axis&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Slice_2&quot;\\n  op: &quot;Slice&quot;\\n  input: &quot;Opt_and_loss/Shape&quot;\\n  input: &quot;Opt_and_loss/Slice_2/begin&quot;\\n  input: &quot;Opt_and_loss/Slice_2/size&quot;\\n  attr {\\n    key: &quot;Index&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Reshape_2&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/SoftmaxCrossEntropyWithLogits&quot;\\n  input: &quot;Opt_and_loss/Slice_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Const&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Mean&quot;\\n  op: &quot;Mean&quot;\\n  input: &quot;Opt_and_loss/Reshape_2&quot;\\n  input: &quot;Opt_and_loss/Const&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n          }\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Const&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Fill&quot;\\n  op: &quot;Fill&quot;\\n  input: &quot;Opt_and_loss/gradients/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Const&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Reshape/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Fill&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Reshape/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Opt_and_loss/Reshape_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Tile&quot;\\n  op: &quot;Tile&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tmultiples&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Shape_1&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Opt_and_loss/Reshape_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Shape_2&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n          }\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Const&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Prod&quot;\\n  op: &quot;Prod&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Shape_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Const&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Const_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Prod_1&quot;\\n  op: &quot;Prod&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Shape_2&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Const_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Maximum/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Maximum&quot;\\n  op: &quot;Maximum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Prod_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Maximum/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/floordiv&quot;\\n  op: &quot;FloorDiv&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Prod&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Maximum&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Cast&quot;\\n  op: &quot;Cast&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/floordiv&quot;\\n  attr {\\n    key: &quot;DstT&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;SrcT&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/truediv&quot;\\n  op: &quot;RealDiv&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Tile&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Cast&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Reshape_2_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Opt_and_loss/SoftmaxCrossEntropyWithLogits&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Reshape_2_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/truediv&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Reshape_2_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/zeros_like&quot;\\n  op: &quot;ZerosLike&quot;\\n  input: &quot;Opt_and_loss/SoftmaxCrossEntropyWithLogits:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/SoftmaxCrossEntropyWithLogits_grad/ExpandDims/dim&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: -1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/SoftmaxCrossEntropyWithLogits_grad/ExpandDims&quot;\\n  op: &quot;ExpandDims&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Reshape_2_grad/Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/SoftmaxCrossEntropyWithLogits_grad/ExpandDims/dim&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tdim&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/SoftmaxCrossEntropyWithLogits_grad/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/SoftmaxCrossEntropyWithLogits_grad/ExpandDims&quot;\\n  input: &quot;Opt_and_loss/SoftmaxCrossEntropyWithLogits:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Reshape_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Feedforward_Net/Tanh_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Reshape_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/SoftmaxCrossEntropyWithLogits_grad/mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Reshape_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/Tanh_2_grad/TanhGrad&quot;\\n  op: &quot;TanhGrad&quot;\\n  input: &quot;Feedforward_Net/Tanh_2&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Reshape_grad/Reshape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Feedforward_Net/MatMul_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 10\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/Tanh_2_grad/TanhGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/Tanh_2_grad/TanhGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/add_2_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/tuple/control_dependency&quot;\\n  input: &quot;Feedforward_Net/readout_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Feedforward_Net/Tanh_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/MatMul_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/MatMul_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/MatMul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/Tanh_1_grad/TanhGrad&quot;\\n  op: &quot;TanhGrad&quot;\\n  input: &quot;Feedforward_Net/Tanh_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Feedforward_Net/MatMul_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 30\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/Tanh_1_grad/TanhGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/Tanh_1_grad/TanhGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/add_1_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/tuple/control_dependency&quot;\\n  input: &quot;Feedforward_Net/hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Feedforward_Net/Tanh&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/MatMul_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/MatMul_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/MatMul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/Tanh_grad/TanhGrad&quot;\\n  op: &quot;TanhGrad&quot;\\n  input: &quot;Feedforward_Net/Tanh&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Feedforward_Net/MatMul&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 30\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/Tanh_grad/TanhGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/Tanh_grad/TanhGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/add_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/add_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/add_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/tuple/control_dependency&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Feedforward_Net/x&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/MatMul_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/MatMul_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/MatMul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta1_power/initial_value&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.8999999761581421\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta1_power&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta1_power/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Opt_and_loss/beta1_power&quot;\\n  input: &quot;Opt_and_loss/beta1_power/initial_value&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta1_power/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/beta1_power&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta2_power/initial_value&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.9990000128746033\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta2_power&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta2_power/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Opt_and_loss/beta2_power&quot;\\n  input: &quot;Opt_and_loss/beta2_power/initial_value&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta2_power/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/beta2_power&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_W/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 784\\n          }\\n          dim {\\n            size: 30\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_W/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 784\\n        }\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_W/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W/Adam&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_W/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_W/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 784\\n          }\\n          dim {\\n            size: 30\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_W/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 784\\n        }\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_W/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W/Adam_1&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_W/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_b/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 30\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_b/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_b/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b/Adam&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_b/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_b/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 30\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_b/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_b/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b/Adam_1&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/into_hidden_b/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_W/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 30\\n          }\\n          dim {\\n            size: 30\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_W/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_W/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/hidden_W/Adam&quot;\\n  input: &quot;Feedforward_Net/hidden_W/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_W/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/hidden_W/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_W/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 30\\n          }\\n          dim {\\n            size: 30\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_W/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_W/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/hidden_W/Adam_1&quot;\\n  input: &quot;Feedforward_Net/hidden_W/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_W/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/hidden_W/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_b/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 30\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_b/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_b/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/hidden_b/Adam&quot;\\n  input: &quot;Feedforward_Net/hidden_b/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_b/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/hidden_b/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_b/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 30\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_b/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_b/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/hidden_b/Adam_1&quot;\\n  input: &quot;Feedforward_Net/hidden_b/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/hidden_b/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/hidden_b/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_W/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 30\\n          }\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_W/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_W/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/readout_W/Adam&quot;\\n  input: &quot;Feedforward_Net/readout_W/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_W/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/readout_W/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_W/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 30\\n          }\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_W/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 30\\n        }\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_W/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/readout_W/Adam_1&quot;\\n  input: &quot;Feedforward_Net/readout_W/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_W/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/readout_W/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_b/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_b/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_b/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/readout_b/Adam&quot;\\n  input: &quot;Feedforward_Net/readout_b/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_b/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/readout_b/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_b/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_b/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_b/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Feedforward_Net/readout_b/Adam_1&quot;\\n  input: &quot;Feedforward_Net/readout_b/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Feedforward_Net/readout_b/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Feedforward_Net/readout_b/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.009999999776482582\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/beta1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.8999999761581421\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/beta2&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.9990000128746033\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 9.99999993922529e-09\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Feedforward_Net/into_hidden_W/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W/Adam&quot;\\n  input: &quot;Feedforward_Net/into_hidden_W/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Feedforward_Net/into_hidden_b/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b/Adam&quot;\\n  input: &quot;Feedforward_Net/into_hidden_b/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Feedforward_Net/hidden_W/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Feedforward_Net/hidden_W&quot;\\n  input: &quot;Feedforward_Net/hidden_W/Adam&quot;\\n  input: &quot;Feedforward_Net/hidden_W/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_1_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Feedforward_Net/hidden_b/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Feedforward_Net/hidden_b&quot;\\n  input: &quot;Feedforward_Net/hidden_b/Adam&quot;\\n  input: &quot;Feedforward_Net/hidden_b/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_1_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Feedforward_Net/readout_W/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Feedforward_Net/readout_W&quot;\\n  input: &quot;Feedforward_Net/readout_W/Adam&quot;\\n  input: &quot;Feedforward_Net/readout_W/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/MatMul_2_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Feedforward_Net/readout_b/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Feedforward_Net/readout_b&quot;\\n  input: &quot;Feedforward_Net/readout_b/Adam&quot;\\n  input: &quot;Feedforward_Net/readout_b/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Feedforward_Net/add_2_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/into_hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/into_hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/readout_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/readout_b/ApplyAdam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Opt_and_loss/beta1_power&quot;\\n  input: &quot;Opt_and_loss/Adam/mul&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/mul_1&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/into_hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/into_hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/readout_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/readout_b/ApplyAdam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/Assign_1&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Opt_and_loss/beta2_power&quot;\\n  input: &quot;Opt_and_loss/Adam/mul_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Feedforward_Net/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/into_hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/into_hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/readout_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Feedforward_Net/readout_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/Assign&quot;\\n  input: &quot;^Opt_and_loss/Adam/Assign_1&quot;\\n}\\nnode {\\n  name: &quot;Support_stuff/init&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Feedforward_Net/into_hidden_W/Assign&quot;\\n  input: &quot;^Feedforward_Net/into_hidden_b/Assign&quot;\\n  input: &quot;^Feedforward_Net/hidden_W/Assign&quot;\\n  input: &quot;^Feedforward_Net/hidden_b/Assign&quot;\\n  input: &quot;^Feedforward_Net/readout_W/Assign&quot;\\n  input: &quot;^Feedforward_Net/readout_b/Assign&quot;\\n  input: &quot;^Opt_and_loss/beta1_power/Assign&quot;\\n  input: &quot;^Opt_and_loss/beta2_power/Assign&quot;\\n  input: &quot;^Feedforward_Net/into_hidden_W/Adam/Assign&quot;\\n  input: &quot;^Feedforward_Net/into_hidden_W/Adam_1/Assign&quot;\\n  input: &quot;^Feedforward_Net/into_hidden_b/Adam/Assign&quot;\\n  input: &quot;^Feedforward_Net/into_hidden_b/Adam_1/Assign&quot;\\n  input: &quot;^Feedforward_Net/hidden_W/Adam/Assign&quot;\\n  input: &quot;^Feedforward_Net/hidden_W/Adam_1/Assign&quot;\\n  input: &quot;^Feedforward_Net/hidden_b/Adam/Assign&quot;\\n  input: &quot;^Feedforward_Net/hidden_b/Adam_1/Assign&quot;\\n  input: &quot;^Feedforward_Net/readout_W/Adam/Assign&quot;\\n  input: &quot;^Feedforward_Net/readout_W/Adam_1/Assign&quot;\\n  input: &quot;^Feedforward_Net/readout_b/Adam/Assign&quot;\\n  input: &quot;^Feedforward_Net/readout_b/Adam_1/Assign&quot;\\n}\\nnode {\\n  name: &quot;Support_stuff/ArgMax/dimension&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Support_stuff/ArgMax&quot;\\n  op: &quot;ArgMax&quot;\\n  input: &quot;Feedforward_Net/y&quot;\\n  input: &quot;Support_stuff/ArgMax/dimension&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Support_stuff/ArgMax_1/dimension&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Support_stuff/ArgMax_1&quot;\\n  op: &quot;ArgMax&quot;\\n  input: &quot;Feedforward_Net/Tanh_2&quot;\\n  input: &quot;Support_stuff/ArgMax_1/dimension&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Support_stuff/Equal&quot;\\n  op: &quot;Equal&quot;\\n  input: &quot;Support_stuff/ArgMax&quot;\\n  input: &quot;Support_stuff/ArgMax_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT64\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Support_stuff/Cast&quot;\\n  op: &quot;Cast&quot;\\n  input: &quot;Support_stuff/Equal&quot;\\n  attr {\\n    key: &quot;DstT&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;SrcT&quot;\\n    value {\\n      type: DT_BOOL\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Support_stuff/Const&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Support_stuff/Mean&quot;\\n  op: &quot;Mean&quot;\\n  input: &quot;Support_stuff/Cast&quot;\\n  input: &quot;Support_stuff/Const&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\n';\n",
       "          }\n",
       "        </script>\n",
       "        <link rel=&quot;import&quot; href=&quot;https://tensorboard.appspot.com/tf-graph-basic.build.html&quot; onload=load()>\n",
       "        <div style=&quot;height:800px&quot;>\n",
       "          <tf-graph-basic id=&quot;graph0.9689909770597044&quot;></tf-graph-basic>\n",
       "        </div>\n",
       "    \"></iframe>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_graph(mlp_graph.as_graph_def())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we create a session, initialize our variables, and train the network: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test accuracy: 0.935200\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x130c7b208>]"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAD8CAYAAACMwORRAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xl8VNX5P/DPM5MdQtjCGiBhDSB7gICAAqKA+1JFW6m2\nFq11bdVi1Wrdq9Xa1q1ate3X/sRdacEVURFl33fCHrYECAkQsp/fH3PvzJ39TuZOZsnn/XrxYube\nc++cO5N55rnnnnuOKKVARESJxRbtChARkfUY3ImIEhCDOxFRAmJwJyJKQAzuREQJiMGdiCgBMbgT\nESUgBnciogTE4E5ElICSovXC7du3V7m5udF6eSKiuLRy5cojSqnsYOWiFtxzc3OxYsWKaL08EVFc\nEpE9ZsqxWYaIKAExuBMRJSAGdyKiBMTgTkSUgBjciYgSUNDgLiKvi0iJiGzws15E5K8iUiQi60Rk\nuPXVJCKiUJjJ3P8JYGqA9dMA9NH+zQLwUvjVIiKicAQN7kqpbwEcC1DkYgD/Vg5LALQWkc5WVZCI\nYsv64nKsKz4e7WpY6qPV+3Gqui7a1bCUFW3uXQHsMzwv1pZ5EZFZIrJCRFaUlpZa8NJE1NQufP47\nXPT84mhXwzJr9h3HHW+vwQMf+Wx5jltNekFVKfWKUqpAKVWQnR307lmiqNp66ARKT1S7LTteWYON\nB8pNbf/DjqOobwh/AvolO4+irr4BAFB+uhbri829vhUqqmpNZeklFVXYfvgEAGD/8dPYfeSUV12V\nUvi+6AiUUthZehIHjp82XQ/j/v1ZX1yO8tO1ftcv330MX28t8VquZ+wHy6u81h0sP40dpSdN1zOQ\nslM1eH9lMcpO1Viyv2CsCO77AXQzPM/RlhHFtfOe+xaTnvnabdllL32P8//6XdBtFxcdwdWvLsFL\nXxeFVYcVu49hxitL8JcF2wEAM19bigufD/76Vrnhnytw0fOLg/5InfnHrzDlz986Hj/5Fc7+09e4\n1qOu760sxjX/WIoPVu3HpGe+wdgnvzJdj9FPLHDu3xelFC58/jv89PVlfsv86OUfcN0by72aX0Tf\nB7yPccwTX2HyM9+Yrmcgl7y4GL95dy0ueqFpPj8rgvtcADO1XjOFAMqVUgct2C9RyKpq6/HJ+oOo\nrKlD+elaHK/0nSUppbD3aCUAoKFBYd8xx+P9x0/j842HcEjL4k5U1aGkwpXR7Sw95Xx8qLwKVbX1\nPve/9ZAjy9xeYi7rM9ZBt/doJQ5XOM4cvt1+BLX1DVirZcI/7DiK0zX1KD1RjS2HKrDvWCWUUjhU\nXoVtPjLcIyercdJPm3L56Vr8d+0B5zGXnapxZsCr9pYBAGrqGry2Ky6rxLFTNdiwvxy19d6BcZ1W\n152lJ6GUwp6jrvdYd6LKf6ZtfB3l47el9EQ19hw9hcqaOufrr9nnfZZxoqoWR0+6zsBW7ilzL6BF\nd6WAlXuOoajkBCpr6pzvBwBsPFCOjQfKsf3wCRSVnMTRk47X3nP0FFbuKcPKPWVQSmHLoQrn2V5J\nRRVO17j+PvTj33fM/BlLOIIOHCYibwE4G0B7ESkG8CCAZABQSr0MYD6A6QCKAFQCuD5SlSUKZuyT\nX+GYx2nv7ifP9yr37opi3PP+Orxz4xgsLjqCvyzYjrm3nOmzLXnU4wt87qPwiQWY2C8bb1w/ymvd\nw//bBACmm2X+9lUR/vzlNiy862zktW+Br7eW4Lo3lmPqwE4AgLX7juPBuRud5a9+dQly22Vg91HX\nD8K90/LxxCdbAABbHpmKtGS7c13Bo1+iS1Yavr93stdrX/X3H7BF+zHa/eT5GPbIF87HNpsADQrV\ndfVIT7G7bTfujwtNHdukZ77BQxcOcGbGNnGtm/DUQqz+/bl+tz1Yftrv64x87EsAQH6nTLx70xgA\nQIrdO1+d9Mw3bs1rM19fhgW/OQu9slsCAESL7muLj+Pyl37w+VpmztZuGJeHf3y3C4DjvRv1+AIM\n794aH9x8ZtBtIyFocFdKXR1kvQLwK8tqRBSChgaFeqWQrH2pPQM7AJysrkOLFDtEBNV19UhNsmP5\nbkcHsG2HT+D7HUcAuDJNM5SWSi7cWgqlFE7X1qO2XiE92e6WITf4SjkB1NY3oLK6HlkZyaipa8A3\n2xxtwXuPVSKnTTo2H3QEWz1zBoAvNh1224cxsAPAsl2uTm0nq+uQbLfheGWNM8gfKK9CbX0DGpRC\napIdSinU1itnYPd1jA3aj5OeuVfX+T5T0enXBjx9vukw+nbMBABU1brKlFXWorquHil2G+oaFOwi\nKD9di7RkO+w2cZ5d6apq65GaZIOI6xdiy6ETqNQy5JQkm7PuVbUNSEmyeV03AYA9R0+hXYsUZKYl\nO8/ujPVqjLdXuPqV6O/Tqr3H3Z7ryitrkZWRHNbrBRO1IX+JrHDH22swd+0Bn5m17owHP8PI3DaY\nMbI7fvPuWnxx5wS8u7IYgCMI6Jnb/SH0ljA2Q7z49Q48/dlWn+X8Ze597vsEAPDEZYNw7wfrncv1\nNuPfTs0HAJQYApOvphEjQ7xDwaNfBnzdT+8Yj6U7j7mdDQDAriOuZqfffbgedVr9q+sasGF/OS74\nW+AMtre2f0/f7ziK73ccBQA8v9D9OkS/+z/1eh/8yX/gU9w2qTd+fW4/t+WjH18AAEiyO96EZz7f\nhucXFqGf9oPi6Wf/tH648RNVrh/1fvd/6nz8fdERXPOPpW5lX/pmB2ZPy7e8DkYcfoDi2ty1B0yV\nW767DPPWOy4FrTVk6A0KritqIahrcAXa91cV+y0XrFnmw9Xm+x7U+smKG2PdvnKf793WQxXOx28t\nc2WiSsF5thMJ/1tn7nMEgP+3bJ/fdfoZ3JzlewEAW4P0sGkKi4qOeC1LsTfijy5EovycNkZaQUGB\n4mQdFK7c2fMAAGd0bYX/3Tre+dyXCX2z8e02a+6vWPvguRjyh8/D3k9BjzZY4XmBjxqtY6tUVNbU\nu2XRseg3U/ri1sl9GrWtiKxUShUEK8fMnRLChv0VQctYmchYlUXXRym5SlQ1dQ0xH9gBIMnHhV+r\nMbhT3PJ38c6fRdu9T48by1+bdqhW702s2/ijrawyeNfKWFBZE/kfIAZ3ilvx8kUm8rR0V+SuX+gY\n3ClueTazrNwT+S8MkRWSm+CCKoM7xS3P9mp/N6AQxRq7jW3uRACA0zX1+PMX25x9vXeWnky4Ufyo\n+Ui2RT5z501MFBde/LoIf/uqCO1apmDmmFxc/MLiuOgVQeSLvQmCOzN3ahIbD5Rj9d4yVFQ5Bqky\n63RNPT5cXYxT1Y7bt9cVl+PzjYcY2OOEvztEY9Glw3xOQxERyUmRD73M3KlJ6AMvTRnQEV9sOoz+\nnVuhd4eWQbd7bP4mvLlkL/I7OYLEeyuL8d5K/3eEkm99O7ZEuxap6NgqFR+tMf/jGi5bE2SosWxw\nTpbPMYuuGJET8ddm5k6WeXPJHhyu8J7wwGh/mWO4U8+hcrccqvA56Jc+5G2iZ+qBxsaxYp+f33kW\n3ppViOdmDAtpHwM6t3Lb3+aHA02nHJrOWWmmy/58XF5YAbFlahLOH+R79s8uWWnY/eT5zlEoZo7p\n4Xc/ax/0P4KlroVh9My5t4zzWWZoTuug+wkXgztZYu2+47j/ow3OAZxCNfW5RZj6nPdkDPoXzooZ\njZqj/E6Z8EyeO7UyH1Q9O3WkJVsXMmaM7G667IS+2V7HYZWfeATzQDcNJ9nEeRbpT4pHk8v4Pu29\nyngOnxwJDO4J4MjJarfZZRoaFIrLKgNsYT1fw6pW1db7zeTLKmu8JmooOVHt1nddKeWc2OFQkDMC\no4uGdAm57I0TemLn49NNb6dr1yIl5G1C9eRlg7yWJdvFLYsuemyaz23n3zYe2x51X7fwrrNNv3a6\nNlzwLRN7AwBEBNsfm4aix6bhCR/18mT8PK8YkYNVD0wBAKQm2XDb5N5uZbc8MhVbH/V9ZnBW32zn\n6J2eHrhgADY9fF7QeviaaQkAfnlWL8cDfdIOP+UAx4XQ+beNx+0BxoXxDO7/un4U3rlxjPP576bn\nu421HykM7gmg4NEvcf5fFzmf/2XBdoz740LsOXoqwFbW8vV1+OWbK/1m8te+tgyFPta9vdw14t9/\nlu7FxgPBx4zxlNMmPeD6Hu0ynI/P7ueYy3dIt9aNah/2/CJb7Zz+HdHPR6Y47YzObtlfkt2GET3a\nOJ93be14D2w28RrHJJQ6Z6QkOfejS7bbkGS3Ocdn1/U3NOH4MiqvLTK0Op8/uLPbmOwAkJZsR2qS\nd9DLSneMey5+Pp4Uuzjr6Ul/TxT8Z+Se9TCWG53X1m1dkk0c72mAv5XpWvPPuN6OjN1mE7ezjqYI\n7AAvqCYM48QNX21xTPxwuKIaPdq1CLhdTV0DRBx/tPofeUODY/KJ1CSbzwGOauoakGx3lXdM+uA9\nzsvCrd4jMBqHyj1VUw+lFKoN45Sv2luGGaMcp+uNvUU7v3MrfHHnBK85N+8+rx+mDOiIXtkt0et3\n8wEAlw3PwcjctujWNsOt7KjctlhmYohbm0dgOKNrK+cgZt/ePRETnvY/W9GXvz4L5zzrf37Opb+b\njNYZyUhNsmPRPRMx/inHvj6/cwJytc917YPnOn9Z/3PDaJyoqoNNAgcQz254PxqR4xzf3pMejH2F\nshE92uDDm8fi8fmbsXx3Ge45rx/+/u0OLNnp/b69cu0ITBnQESKCJfdORlvtjGfl/edghMc4PT/c\nOwljnnDMr7r6gSnOniX639vsafm4qqAb7n5vLb7cXOIcc96XV2cWYLg2s1Qw+pmBcXf/+tkovLDQ\n0Q33nP4dnd8H/cfu5+PycP2Zuc4fzBS7DZlpybhgcBcM6prl3E+lYbq9pCa4gQlg5p5wSk5UYf1+\nx9V5f5mObt+xSvS9/xP0ue8T/PmLbc7lQx7+HAMf/MznxAslFVXoe/8n+L8le5zL3lmxDzf/Z5Xf\n1zGemm877D6n6O8/3oj8Bz713AQLNh8Oqcukpz4+uuD175yJvh0zvYKbZ2AHgMtHmOsW5/k9NWae\n3dt579coV1uvZ9meOrZKc+7PWMe+HTOdwSQrPdk5o09ash3Zmalo1zIVLVLN523jfLQJ69oGaXYa\n1r0NhnZzXBzMzkzFhL7ZznXd22Y4s+Du7TKcwblTVpqz/u1apnrts3W64zVFgDYtUtBSOxa9d1Xf\nji3RpkUKztJeq3OW/zO1Fql2rZ7BL2Dq35fWhhmS9PfUUe9Ur7LJdhty2mSgQ2YaOmSmoXVGCuw2\nwYgebdzOkFqlu/bZNciZpVWYuSeYA8ddbdPBGhmMEym/vWKfc3abQD1T9miTOH+85gBmjskFEHzC\njEAXQ40/Ekb62YdVnr1yCCbldzRd/sqCbujYKg3XvbE8YDlj5v7G9SPx/FeOWYZe/skIr7LXFvZw\nO94kuw2f3jEeHTLNXeB858YxQWdjMmvebeOQ0zoDmw5WoLBnW9w+Zw0AR3PJvHWu+e19BV9P90zN\nx7kDO+GMrlno37kV+ndqhcMVVZh2Rmdc8fL3AOC3vdwX/S31bPq4fmwuBnZphcKe7QAAPynsgd4d\nMlHY073p5LWfFuDn/3LMFZGaZMd/bxmHvOwWuOudtQAcc81OPaMTznr6a/fX1f7P8zjb1X+gjMeg\nPzY7jPTQbq0xZ1YhGhoUxvb2/2NqJWbuCeSR/22CPUi6XllTh9zZ8/DuCvfZbA5XVCN39jyvyS4q\nqmox5A+fe/UtFzgy9tzZ89xOOQGg/HSt2378Tb0WiJWzDgGO5hdPgdqeRQRn9+sQdL/G93uioXy7\nlt4Zr6+28/xOrYJmx7pReW0DZtmhGNglC1kZyRjTq51bm/MAj3Zz/YKxr+PRJdttGJnrCLB2m2Bi\nfgfMGNXdbY7QYGeRRnrZnu3d74Ow2cQZ2B3lxKv+ADC5v/uP+KCcLLRMTXJeKO3eNiNoc6WRnsnr\nGTzgmuQ7lD5chT3bNVlgB5i5J5TXvtuFyw1BzNcXSu83/sLCIjxwwYCg+zxyohrlp2vx4sIir37G\nf12wHQBQUuHeU8Y4D2djVZyObL/2V2cWoENm8KzUKNkuePbKobj1rdXOZXrb6/3n9wfg/2zp6SsG\n47LhOcjvlIljp2rQMi02v3qzJvTE0G6t8WNtzs+fFPZARord54+jGWaC35e/noBjp1w9p1KT7Hh1\nZoGzucest2cVor32mS66ZyL2eEyu7czAtQ/pszsmuE1m7qqzwv9uHecsf+HgLqitV7h4qKsXls1w\nfSpWxeZfGPnV0KBw6YuLccukPpgyoCPeWLzLbX2SYSjRunqFaX9ZhN9Nz8e1rzkmXtaz1d1HK52n\nroHsKHUEas8/4RV7ypxZnd5dUXfJC4tDOiajd1YU450V4d2BaiZJnDLAfBONblj3NrhwSBf34K69\nWKCM2ibAjwq6AQAKctv6LRcLku02nGnILu02cda9MTK1H7FAY6n07uB9RtOYz2e0Iavv1jbD61qK\nfh1Cn2fV80xKD/pKAWcYLobabOKV2KRpF5pTLez3bzUG9zhTWVuPtcXluGPOamx8eCr+8N9NbuuN\nbcCHKqqw+WCF26zyobbZ/v2bHQBcbYvGJsajPu4ojZbFsyfhzCe/cls295Yz8cGq/W5dBEP19qxC\nHD9di11HTuEybeyR+beNx4NzN+DRSwbh9jmOQO95XUF/n564bBAKTL7+G9ePxPVB2vibytNXDMbA\nLlnBCwbx8k9G4OM1+9GzvflmkEh56KKB6Nm+hVvzmZGzHd3EvmaM7IajJ6tx44ReFtbQWgzuceA/\nS/cgK93RvUrn7w/ww9XWjruiT968+2glnvtyW9A2/Wjp2jod5w/qjHnrXRcDB+e0xuAwb/M2ZoO6\nAV1a4d2bxgJwZaR6D09X9uf4hK4eZf4uTH9BJxrCydaNOrZKw6wYCYBZ6ckBJ6U2Zu7BJNttuOOc\nvhbVLDIY3OPAfR86xi2/YHCXoE0OLyzc4Xzs2Qc7XM99ud3S/VnhmtHdnWOG6P3lk5tg8mHdC9cM\nx6uLdmJAF8eFyGevHIoXv94R1tlCNNw+uY9bc0xzdPd5/VDXoHDZ8KYbHTKSGNzjyLJdxzCwS+C7\nAI0+2eDIYk321opLj1/qugVeH4IhswkvVua2b4HHDHXo1jbD1G35sebOKbGdhTaFdi1T8acfDYl2\nNSwTu1cDyMuVf//B2RxjJmDPX38oovWJNfdM7Yd+HTND7mVh9Pilg5xDEhDFM2buccbsTRNGnr1Z\nEtWw7m3w2Z0TwtrHNaO745rR5tvJiWIVM/c4E8PdaslC0wd1inYVKM4xuEfZlkMVKAulS6EW3E97\nTHbRXERiUotY9OKPvYcvIAoFg3uUTX1uES7423emywcaa5qISMfgHgOMbeKHyqsCjqvSHJtlXp1Z\nEO0qEMUdBvcYcrqmHoVPLHC7o9RTYy6oxjt/A3x1CWEOTqLmxlRwF5GpIrJVRIpEZLaP9Vki8l8R\nWSsiG0Xkeuurmvj0oQE+3+jqwug5MJHxmdUjJ8Yqf7diffHrs5q0Hk1hze+nYN1DwSdhJgomaHAX\nETuAFwBMAzAAwNUi4jmc4K8AbFJKDQFwNoBnRCTyk0smGH3iB+M4JT21GYN0DYbMvU8jhtJNJKFM\nSBEvWmekoFVacvCCREGYydxHAShSSu1UStUAmAPgYo8yCkCmOAZWbgngGIDIjtmawAJNG9bcrqfO\nmVXoHPOjUys2wxCZZSa4dwVgnNmhWFtm9DyA/gAOAFgP4HalVPNoMwjDR6v3uz3X43Z1XQOG/OFz\nn9s0s9iOM7pmOadZG9vbexAvIvLNqvPa8wCsATAJQC8AX4jIIqWU29T1IjILwCwA6N6ddwE+v7DI\n7bnxWmn56Vr40tBEF1RFYmNMmiSbYFj3NnjuqqHOMb7fuXEM2mSw6YIoEDOZ+34AxvE/c7RlRtcD\n+EA5FAHYBSDfc0dKqVeUUgVKqYLs7OY9fkdFVS2KSlyTRa/eW4Ypz34TdLt9x5pmKIHp2kiLZo2L\n0IiC+giPlwzr6mxjH5XX1ucE2ETkYia4LwfQR0TytIukMwDM9SizF8BkABCRjgD6AdhpZUUTzdKd\nx9yePz5/M0pOVPsp7eJvQml/hjRyEK0HLxiA/E6ZmDOr0FT5Z68agkuHNX6o1Nev892XPdAMPkTk\nX9DgrpSqA3ALgM8AbAbwjlJqo4jcJCI3acUeATBWRNYDWADgt0qpI5GqdDw5WV2Hu95di4oqVzOL\nUgpPf7Yl6Lb3f+Td3z3UWPeL8XmhbaBplZ6MT++Y4DYhMQD88/qRPst3yEzDn68a6nd/gfqkD8nJ\nwqT80KdVIyL/TLW5K6XmA5jvsexlw+MDANg514fXv9uF91YWo3NWGn5zbj8AjrtMtx0+6Vaupt67\ngfvNJXu9loU6AUedj/3qzunfAav2HscxH2Pb+MuYaz3299erh+GE4Yfr7vP64enPtnptd+nwrs6J\nRO4/vz8+XnMA6/eXAwD+dvVwt7KtM5Lx4IUDsLM0/Im2iZor3qEaYd8VeZ/A+LrLdO2+46b2F+rk\nSoFudLpiRA5WPTDF5zp/0+nVeezvoiFd8OPRPZzPfzWxt9c2w7q3RkaKI4+4+exeuGF8T1xb2MNZ\nh+7t3CcyfvKyQbh0WI7zx5CIQsfgHmHLdh3zWhbO+DAnqkK7fcCzz/ztk/tgYJdW6NY2HWN6ul8E\nffqKwc7H/n5Eak1UfoxHU47xbEPfXB8AjS3qRJHB4B4F4XRn/GZrqemyvzy7l1emfcc5fTDvtvFY\ndM8kZHl0JzROiix+onttXfDbF97yuAibkWJ3TT6sBXX9LbB6nlcicmBwjyBjV0djCAun/3hNCOPJ\n2AQY18e9y6m/oB3I5PwOzsej8tqGvP3PxuVhSn/HBdMLBnUB4MrgbfwLJIqIxBucI4ZU1ng3odTV\nN2DZbu+mGis8dflg3PP+Oudzmwjy2rfAz8fl4bXvduG+6f0btd/XrvPdQ8asif0cPw7GiTb0s5fG\n/NgQUXDMmyLIV/P08wuL8NPXl0Xk9Yb3cO/TrgfOT9YfBADMWe7d+6YxQrk7dGwv30MG6BeVfYX2\nfp1aNaZaRGTAzD2CjG3rp2rqUXaqBhsPVATYIjy9O2Ri26PT0Pd+x2iRem/Gw9rNUTss6lq4/L5z\n0KAC99wpemwa6pVCsp92F/2d8dXmnte+hQW1JGreGNwjyDgW+2vf7cJr3+2K+GsaJ7YY2CULANCj\nbQZ2Hjll+m7VHh5dEz0l2YOf8CXZbQH/uHp3aAkAGGqo04DOrbDpYOR+/IiaEwZ3C9TVN/gMeKFc\n/IwEfaCt7u0cwf1WH33QPS2/7xxkpNgjXTWM7dUe3949Ed3apjuXvffLMThZzZGiiazANvcwfbOt\nFL3v+wTrit1vQvp0wyFc8+rSqNQp2e67vcTMOC3ZmalNNglG93YZbhdUM1KS0CGTY7YTWYGZe5gW\nbikBAKzYXYbBOa4mBuNUeU1t8exJOF5pHMsmcPnvZ09CVW19hGtFRE2JmbtFTlXXIXf2PMxdewAA\n8MFqz1GRm06HzDT09TUkrp/EvUvrdPTMbhnZShFRk2Jwt8i+skoAwIseE3BY5YZxgUd3/O8t4yLy\nukQUnxjcLbbl0Ankzp5n+X4vHNLF77oWKXYMysnyuz4GJlQioibG4B4m1804kb3TMlCf8nqT4xnw\nXlCi5oPB3SKRvos+0ABb9UFGavQ1xDARJTYG9zDFQtgMFtyfvHwwrizIwZkRmueUiGIPg3ucMGbu\nOW3S3dYFG2K9a+t0PHXFEOdk00SU+Phtt0ikm2WM++cY6EQUDIO7ZZrugipvOCKiYBjcw9RU1yqN\nvXFKtFEedXed27dpKkFEcYPDD1hkRYQm4NApP5dujRNgEBHpmLlbZLthSr1I8BxQSx8yl4jIF2bu\nYVqz73jwQiHI75SJLYdOOJ9vfngqRIC0ZDvumdoPT326FdeNzcUDFwwIa6JtIkpsDO5h+HTDQazf\nX27pPj17wqQbxlZP0obsTbIJ7DaBnfecEpEfbJYJw8o9ZZbvM1A2rq+ymRiXnYiaNwb3MESiVSTQ\n3ab6KoZ2IgqGwT0MkWjxDpi5a68ovImJiIJgcA9DJC5oBtqlvo6xnYiCYXAPg5nYntMmHdMHdTK9\nz8Bt7o51bHInomAY3MNgZoyXYd3b4MUfjzC9z0CTU6ckOT6utCS73zJERACDe1hG5rYJWibUJPuV\nmQWYPS3f57qZY3Jx66Te+MWEniHulYiaGwb3MJhpcQ+1fbxr63TcdFYvn+vSku34zbn9kJbMzJ2I\nAmNwD4OZC6psHieiaDAV3EVkqohsFZEiEZntp8zZIrJGRDaKyDfWVjM2zVt3MGgZvdvi+D6cBYmI\nmk7Q4C4idgAvAJgGYACAq0VkgEeZ1gBeBHCRUmoggB9FoK4x55MNh4KWCZa5XzasqzWVISIyMJO5\njwJQpJTaqZSqATAHwMUeZa4B8IFSai8AKKVKrK1m7Cg/XYtPTQR1s569aqhl+yIi0pkJ7l0B7DM8\nL9aWGfUF0EZEvhaRlSIy09eORGSWiKwQkRWlpaWNq3GU3frWatz05kos3Gry90tL3evq/bfPF/Zs\na0HNiIhcrLqgmgRgBIDzAZwH4AER8ZoeSCn1ilKqQClVkJ2dbdFLN629R08BAI5X1pgqr8+gVFlT\n57fMv382OvyKEREZmAnu+wF0MzzP0ZYZFQP4TCl1Sil1BMC3AIZYU8XYog/e9bevikyV17tCTh/U\nOWgZo46tUp03LRERhcpM9FgOoI+I5IlICoAZAOZ6lPkYwDgRSRKRDACjAWy2tqpN58Dx037X6YN3\n7Sw9FXAfd5zTB4DrguqsCT2x5ZGp2P7YNGeZrY9OdStjtPi3k7DxD+eZrzQRkUHQ4K6UqgNwC4DP\n4AjY7yilNorITSJyk1ZmM4BPAawDsAzAP5RSGyJX7cj5YcdRjH3yK3y8xvPkxMHsWGEFPRzt6MO6\nO+5iFRGkJduRbHe95anaMAK+RnlMstvcyhIRhcLUTExKqfkA5nsse9nj+dMAnrauatGx6WAFAGD1\n3uO4eKhKBU/6AAAOCElEQVR3N0WzwX10z7b4+q6z0aNdRtCyvNGJiKzGafb88DdsgDIZ3ZNsgtz2\nLcJ6LSKixuJ5f4gCTJTk9Mq1I0KaUIOTbxCR1RjcPcxZtheAe7/077YfQe7seSipqHJeUA1kTK92\nEasfEZEZDO4etpecBACcrHb1S//n97sBAKv3HTeVuTMTJ6Joa/bB/asth3Hlyz+gwUfUrqtvwGUv\nLsYPO44AcAwUZubmpcaG9hT2jiEiizT7C6q/fHMVqusaUFPfgDSba5x0AVB6shqr9h53Lpu79oCp\nfQaboekvM4aia+t0t2UPXzwQY9mcQ0QWafbBvbquAQCwam8Zrnl1qWuFAH//Zmej9hmsVcZXF8uZ\nY3Ib9VpERL6wHUDjFtg1els7EVG8YXAnIkpADO7+mLwT1VN+p0xeGCWiqGv2be7+nKj2P0RvIJ/e\nMcHimhARhY4pph9fbDoc7SoQETUag3uYvr7r7GhXgYjIC4N7mFKT+RYSUexhZArD5cNzOOY6EcUk\nRqYwvL+qONpVICLyicGdiCgBMbiHKZWTWBNRDGJkCsMtE3sjMy0Z7/9ybLSrQkTkhsE9DG1bpAAA\nRvRoE+WaEBG54x2qYTB2g/zfreNwurY+irUhInJhcA9DZlqy8/EZXbOiWBMiIndslgnD9DM6RbsK\nREQ+Mbg3UqdWaUjiDUxEFKMYnYiIEhCDeyMFm0qPiCiaGNyJiBJQs+0to5TCf9cdjHY1iIgiotlm\n7gs2l+C2t1Y3evubJ/a2sDZERNZqlsFdKRX2TEvXFvawqDZERNZrlsH9zaV78faKfY3evn3LFAtr\nQ0RkvYQN7ofKq/yu2374RFj7XnTPpLC2JyKKtIQM7qv2lqHwiQV4f6XvyTQalApr/8l29oMkothm\nKriLyFQR2SoiRSIyO0C5kSJSJyJXWFdF85RSUEqh6PBJAMDSXUd9lmswGdv/cNFAn8uFndyJKMYF\n7QopInYALwCYAqAYwHIRmauU2uSj3B8BfB6JigaSd+88DO6ahbXF5Rjfpz0uHtoVAFDf4Lu8Mpm5\nTxvUCQ/O3ei13MbYTkQxzkzmPgpAkVJqp1KqBsAcABf7KHcrgPcBlFhYP1OUAtYWlwMAFm0/4gy+\n/ppfGvwEfU8C31GcmTsRxTozwb0rAGPXkmJtmZOIdAVwKYCXrKuaOWOeWOC1zK5F93o/7S9m29yZ\noRNRvLLqgupzAH6rlAqYE4vILBFZISIrSktLLXnhgz56xdi0zLreX+Zuss3dJoJ3bhyDxy49o9H1\nIyKKBjPBfT+AbobnOdoyowIAc0RkN4ArALwoIpd47kgp9YpSqkApVZCdnd3IKgfnzNzrXVF8cdER\nXPnyD1i77zjeX+W7Fw0APHqJK5DbRDAqry0GduFEHEQUX8wE9+UA+ohInoikAJgBYK6xgFIqTymV\nq5TKBfAegJuVUh9ZXluT9OYUBVdwv+PtNVi2+xjufHtNwG2vLHD9jonHu5OebPfbg4aIKJYE7S2j\nlKoTkVsAfAbADuB1pdRGEblJW/9yhOsYsv8s3QvAdUH01W93ovREtWNZgHb09GQ7UpJcEd3mUbhv\nx5b46dhcaytLRBQBpkaFVErNBzDfY5nPoK6Uui78aoVn0fYjAFyB/LH5m53rPAN2ILygSkTxKiHv\nUA1ke8lJr2U5bdJ9lvX8IQjvvlYioqYT18E92M1I4XZH17fv27ElOmelYfbU/PB2SETUROJ6so4l\nO48FXD9//SHU+btNNQARx41Rept9RkoSfrh3cqPqSEQUDXGduV/96pKgZV77blfI+/2tlqEnsdGd\niOJUXGfuZui9ZALxbL656axeuOmsXhGqERFR5MV15m6VZLvjbbhqZLcgJYmI4kPCZ+5mJNts2PLI\nVKTY+VtHRImBwV2TlmyPdhWIiCyT8KkqR+clouYo4YP7q4uC95bhDwARJZqED+5ERM0RgzsRUQJi\ncCciSkDNOrinaz1k0lPYU4aIEkuz7go5oEsrnN03G1cU5ES7KkRElmrWwT3FbsOtk/tEuxpERJZr\n1s0yyUnN+vCJKIE16+jG4QaIKFHFbXTznKjD3ojheW+d1Nuq6hARxZS4De67j1a6PW/M0OtDurW2\nqDZERLElboP7xD997fZcQhxDYHh3BnYiSlxxG9w92UMM7m/NKoxQTYiIoi9xgnuI7TL6/KhERImo\n+QZ3xnYiSmAJE9xDncyasZ2IElnCBPfQM3eGdyJKXAkT3DPTkpDfKdN0eYZ2IkpkcR/cR+e1BQB0\nykrDnFmF+PDmsaa2Y+JORIks7gcO0ye2tttsaJ2RgmHdU0xtx2YZIkpkcZ+5N2jDEIR6QZWIKJHF\nZXB/8OMNzsclFdUAGje2DBFRoorL4P6vH/Y4H289fAIAM3ciIqO4DO6+MHMnInJJmOBuzNyvG5uL\npy4fHMXaEBFFl6ngLiJTRWSriBSJyGwf638sIutEZL2IfC8iQ6yvamBJhok3HrpoIK4c2a2pq0BE\nFDOCdoUUETuAFwBMAVAMYLmIzFVKbTIU2wXgLKVUmYhMA/AKgNGRqLCn1hnJAICfnZnXFC9HRBQX\nzPRzHwWgSCm1EwBEZA6AiwE4g7tS6ntD+SUAcqysZCAdM9Pw2Z0TmurliIjigplmma4A9hmeF2vL\n/Pk5gE98rRCRWSKyQkRWlJaWmq9lADZeSCUi8mLpBVURmQhHcP+tr/VKqVeUUgVKqYLs7GxLXtPs\nHNeT8ztg95PnW/KaRESxzkxo3A/AeHUyR1vmRkQGA/gHgIuVUketqV5w087o7HddZpqr1WnKgI6O\nZalJSE1KmE5CREQ+mYlyywH0EZE8EUkBMAPAXGMBEekO4AMA1yqltllfTf+uHdPD77qV909xPr5K\n6z2z8oEpWPfQuRGvFxFRNAW9oKqUqhORWwB8BsAO4HWl1EYRuUlb/zKA3wNoB+BFbUCuOqVUQeSq\n7ZJs8//7lGLI0PWBwlKYtRNRM2BqVEil1HwA8z2WvWx4fAOAG6ytmjlJdl5QJSLyFPdpLMeUISLy\nFvfBneOyExF5i/vgTkRE3hjciYgSUFwHd7bIEBH5FtfBPcXs7alERM1MXEZH/Q7T+gYV5ZoQEcUm\nU/3cY02STVANoF4FD+6/mtgLA7tkRb5SREQxJC6Duz4SpInYjrvPy49wbYiIYk9cNsvo11H/9KMm\nn/CJiCguxGVwVwq4/sxcXDGiyeYEISKKK/EZ3AEI2A+SiMifuAvuB46fxsnqOpyqrot2VYiIYlbc\nBfdVe8sAAIcqqqJcEyKi2BV3wV0fBbLBTFcZIqJmKu6Cu93GG5iIiIKJw+Du+J+ZOxGRf3EY3B1V\nbmiIckWIiGJY/AV3bShIjghJRORf3AV3BUdzDOdOJSLyL+6Ce512IVVvniEiIm9xGyHbtUiJdhWI\niGJW3I0KWZjXDj8p7I5fjO8Z7aoQEcWsuAvu6Sl2PHrJoGhXg4gopsVtswwREfnH4E5ElIAY3ImI\nEhCDOxFRAmJwJyJKQAzuREQJiMGdiCgBMbgTESUgUVEaF11ESgHsaeTm7QEcsbA68YDH3DzwmJuH\ncI65h1IqO1ihqAX3cIjICqVUQbTr0ZR4zM0Dj7l5aIpjZrMMEVECYnAnIkpA8RrcX4l2BaKAx9w8\n8Jibh4gfc1y2uRMRUWDxmrkTEVEAcRfcRWSqiGwVkSIRmR3t+lhJRHaLyHoRWSMiK7RlbUXkCxHZ\nrv3fxlD+Xu192Coi50Wv5uaJyOsiUiIiGwzLQj5GERmhvVdFIvJXkdicMt3P8T4kIvu1z3mNiEw3\nrIvr4wUAEekmIgtFZJOIbBSR27Xlifw5+zvm6H3WSqm4+QfADmAHgJ4AUgCsBTAg2vWy8Ph2A2jv\nsewpALO1x7MB/FF7PEA7/lQAedr7Yo/2MZg4xgkAhgPYEM4xAlgGoBCAAPgEwLRoH1sIx/sQgLt8\nlI3749Xq2hnAcO1xJoBt2rEl8ufs75ij9lnHW+Y+CkCRUmqnUqoGwBwAF0e5TpF2MYB/aY//BeAS\nw/I5SqlqpdQuAEVwvD8xTSn1LYBjHotDOkYR6QyglVJqiXJ8G/5t2Cam+Dlef+L+eAFAKXVQKbVK\ne3wCwGYAXZHYn7O/Y/Yn4sccb8G9K4B9hufFCPwGxhsF4EsRWSkis7RlHZVSB7XHhwB01B4n0nsR\n6jF21R57Lo8nt4rIOq3ZRm+eSLjjFZFcAMMALEUz+Zw9jhmI0mcdb8E90Y1TSg0FMA3Ar0RkgnGl\n9kue0N2bmsMxAngJjqbFoQAOAngmutWJDBFpCeB9AHcopSqM6xL1c/ZxzFH7rOMtuO8H0M3wPEdb\nlhCUUvu1/0sAfAhHM8th7VQN2v8lWvFEei9CPcb92mPP5XFBKXVYKVWvlGoA8CpczWkJc7wikgxH\nkPuPUuoDbXFCf86+jjman3W8BfflAPqISJ6IpACYAWBulOtkCRFpISKZ+mMA5wLYAMfx/VQr9lMA\nH2uP5wKYISKpIpIHoA8cF2LiUUjHqJ3aV4hIodaTYKZhm5inBzjNpXB8zkCCHK9Wx9cAbFZKPWtY\nlbCfs79jjupnHe2rzI24Kj0djivROwDcF+36WHhcPeG4er4WwEb92AC0A7AAwHYAXwJoa9jmPu19\n2IoY7UXg4zjfguP0tBaO9sSfN+YYARRoX5QdAJ6HdkNerP3zc7z/B2A9gHXal7xzohyvVtdxcDS5\nrAOwRvs3PcE/Z3/HHLXPmneoEhEloHhrliEiIhMY3ImIEhCDOxFRAmJwJyJKQAzuREQJiMGdiCgB\nMbgTESUgBnciogT0/wH/5GWcQE/w3QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x12b8e1da0>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sess = tf.Session(graph=mlp_graph)\n",
    "sess.run(init)\n",
    "train_steps = 2500\n",
    "\n",
    "acc = np.zeros(train_steps)\n",
    "\n",
    "# create this op outside of the loop so we don't create it 5000 times\n",
    "for i in range(train_steps):\n",
    "    batch_xs, batch_ys = mnist.train.next_batch(100)\n",
    "    acc[i] = mlp_acc.eval(session = sess, feed_dict = {x: batch_xs, y: batch_ys})\n",
    "    sess.run(train_step_mlptest, feed_dict={x: batch_xs, y: batch_ys})\n",
    "\n",
    "test_acc = mlp_acc.eval(session=sess, feed_dict={x: mnist.test.images, y: mnist.test.labels})\n",
    "print(\"Test accuracy: %f\" % test_acc)\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "plt.plot(acc)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# VAEs! \n",
    "$$\n",
    "\\DeclareMathOperator{\\Tr}{Tr}\n",
    "\\newcommand{\\trp}{{^\\top}} % transpose\n",
    "\\newcommand{\\trace}{\\text{Trace}} % trace\n",
    "\\newcommand{\\inv}{^{-1}}\n",
    "\\newcommand{\\mb}{\\mathbf{b}}\n",
    "\\newcommand{\\M}{\\mathbf{M}}\n",
    "\\newcommand{\\G}{\\mathbf{G}}\n",
    "\\newcommand{\\A}{\\mathbf{A}}\n",
    "\\newcommand{\\R}{\\mathbf{R}}\n",
    "\\renewcommand{\\S}{\\mathbf{S}}\n",
    "\\newcommand{\\B}{\\mathbf{B}}\n",
    "\\newcommand{\\Q}{\\mathbf{Q}}\n",
    "\\newcommand{\\mH}{\\mathbf{H}}\n",
    "\\newcommand{\\U}{\\mathbf{U}}\n",
    "\\newcommand{\\mL}{\\mathbf{L}}\n",
    "\\newcommand{\\diag}{\\mathrm{diag}}\n",
    "\\newcommand{\\etr}{\\mathrm{etr}}\n",
    "\\renewcommand{\\H}{\\mathbf{H}}\n",
    "\\newcommand{\\vecop}{\\mathrm{vec}}\n",
    "\\newcommand{\\I}{\\mathbf{I}}\n",
    "\\newcommand{\\X}{\\mathbf{X}_{ij}}\n",
    "\\newcommand{\\Y}{\\mathbf{Y}_{jk}}\n",
    "\\newcommand{\\Z}{\\mathbf{Z}_{ik}}\n",
    "$$\n",
    "\n",
    "In our generative model, we would like to estimate the density of some complicated probability density $\\log P_{\\theta}(X)$. This is slightly odd notation but seems standard in these papers: it says that $X$ is a random variable but $\\theta$ are parameters. To do this, we will write it as follows:\n",
    "\n",
    "$$\n",
    "\\log P_{\\theta}(X) =  \\log P_{\\theta}(X,Z) - \\log P_{\\theta}(Z\\mid X) \n",
    "$$\n",
    "\n",
    "This is just the definition of marginal probability. Next, we add/subtract $\\log Q_{\\phi}(Z\\mid X)$ which sums to 0: \n",
    "\n",
    "$$\n",
    "\\log P_{\\theta}(X) =  \\log P_{\\theta}(X,Z) - \\log P_{\\theta}(Z\\mid X) - \\log Q_{\\phi}(Z\\mid X) + \\log Q_{\\phi}(Z\\mid X)\n",
    "$$\n",
    "\n",
    "We take expectation of both sides. Note that this expectation has to be w.r.t. the conditional distribution $Q(Z\\mid X)$ for the rest of this to work properly: \n",
    "\n",
    "$$\n",
    "\\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X) =  \\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X,Z) - \\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(Z\\mid X) - \\mathbb{E}_{Q(Z\\mid X)}\\log Q_{\\phi}(Z\\mid X) + \\mathbb{E}_{Q(Z\\mid X)}\\log Q_{\\phi}(Z\\mid X)\n",
    "$$\n",
    "\n",
    "Since the LHS is independent of Z, that expectation just goes away. We rearrange the terms and realize we ended up with the evidence lower bound (ELBO) and a KL divergence: \n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "\\log P_{\\theta}(X) =&  \\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X,Z) - \\mathbb{E}_{Q(Z\\mid X)}\\log Q_{\\phi}(Z\\mid X) + \\mathbb{E}_{Q(Z\\mid X)}\\log Q_{\\phi}(Z\\mid X)- \\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(Z\\mid X) \\\\\n",
    "  =&  \\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X,Z) - \\mathbb{E}_{Q(Z\\mid X)}\\log Q_{\\phi}(Z\\mid X) + \\int_Z Q_{\\phi}(Z\\mid X)\\log Q_{\\phi}(Z\\mid X)- \\int_Z Q_{\\phi}(Z\\mid X)\\log P_{\\theta}(Z\\mid X) \\\\\n",
    "=&  \\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X,Z) +\\mathbb{E}_{Q(Z\\mid X)}\\log Q_{\\phi}(Z\\mid X) - \\int_Z Q_{\\phi}(Z\\mid X)\\log \\frac{Q_{\\phi}(Z\\mid X)}{P_{\\theta}(Z\\mid X)} \\\\\n",
    "=&  \\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X,Z) + \\mathbb{E}_{Q(Z\\mid X)}\\log Q_{\\phi}(Z\\mid X) -\\mathcal{D}_{KL}(Q_{\\theta}(Z\\mid X)||  P_{\\phi}(Z\\mid X)) \n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "The ELBO is a lower bound because KL divergence is greater than equal to 0. So if we want to maximize the LHS, we can choose to either minimize the KL divergence, or maximize the ELBO.  We would rather do the former, because the KL divergence contains the posterior $P(Z\\mid X)$ and if we knew how to do that, we wouldn't be going through this hassle. The nice thing is, since this holds for any $Q$ and any $Z$ we can define both distributions to be as nice as we like. So we're going to say that the likelihood $P_{\\theta}(X,Z)$, prior $P_{\\theta}(Z)$ and approximate posterior $Q_{\\theta}(X,Z)$ are all gaussian. We pick the easiest possible marginal distribution over $Q(Z)$, an identity-covariance gaussian: \n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "P_{\\theta}(X\\mid Z) :=& \\mathcal{N}(a(Z,\\theta), b(Z,\\theta)b(Z,\\theta)\\trp) \\\\\n",
    "Q_{\\phi}(Z\\mid X):=&\\mathcal{N}(f(X,\\phi),g(X,\\phi)g(X,\\phi)^{\\intercal})\\\\\n",
    "P_{\\theta}(Z) :=& \\mathcal{N}(0,\\mathbf{I})\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "The distributions $P_{\\theta}(X\\mid Z)$ and $Q_{\\phi}(Z\\mid X)$ are parameterized by mean and covariance-square-root functions $a,b,f,g$ which we leave unspecified for now. In practice for VAEs people use the equivalent of a mean-field assumption, which means that the covariance functions will just return SDs/variances, but I'd like to write the reparameterization trick in general form. We can additionally rewrite the expression to get a second KL divergence: \n",
    "\n",
    "$$\n",
    "\\begin{align}\n",
    "\\log P_{\\theta}(X) =& \\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X\\mid Z) + \\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(Z) + \\mathbb{E}_{Q(Z\\mid X)}\\log Q_{\\phi}(Z\\mid X)-\\mathcal{D}_{KL}(Q_{\\theta}(Z\\mid X)||  P_{\\phi}(Z\\mid X)) \\\\\n",
    "=&\\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X\\mid Z) + \\mathcal{D}_{KL}( Q_{\\phi}(Z\\mid X)||P_{\\theta}(Z)) - \\mathcal{D}_{KL}(Q_{\\theta}(Z\\mid X)||  P_{\\phi}(Z\\mid X))\n",
    "\\end{align}\n",
    "$$\n",
    "\n",
    "Conveniently, the KL divergence between two gaussians (the prior and approximate posterior) is analytic. What remains is the expectation $\\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X\\mid Z)$, which we can compute from its empirical, sample-based mean: \n",
    "\n",
    "$$\n",
    "\\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X\\mid Z) = \\int Q_{\\phi}(Z\\mid X)\\log P_{\\theta}(X|Z)d Q \\approx \\frac{1}{N} \\sum \\log P_{\\theta}(X|Z) Z_i, \\\\\\\n",
    "Z_i\\sim\\mathcal{N}(f(X,\\phi),g(X,\\phi)g(X,\\phi)\\trp)\n",
    "$$\n",
    "\n",
    "The naive gradient estimator here has very high variance according to the VAE paper, though it is used in Blei, Jordan and Paisley 2012 (ICML):\n",
    "\n",
    "$$\n",
    "\\nabla_{\\phi}\\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X\\mid Z) \\approx \\frac{1}{N} \\sum \\nabla_{\\phi}\\log P_{\\theta}(X|Z) Z_i\n",
    "$$\n",
    "\n",
    "What the VAE paper does instead is apply the reparameterization trick:  \n",
    "\n",
    "\\begin{align}\n",
    "\\epsilon&\\sim\\mathcal{N}(0, \\I)\\\\\n",
    "Z &= f(X,\\phi) + g(X,\\phi)\\epsilon\\\\\n",
    "\\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X\\mid Z)  &\\approx \\frac{1}{N} \\sum \\log P_{\\theta}(X|Z)(f(X,\\phi) + g(X,\\phi)\\epsilon)\\\\\n",
    "\\nabla_{\\phi}\\mathbb{E}_{Q(Z\\mid X)}\\log P_{\\theta}(X\\mid Z)  &\\approx \\frac{1}{N} \\sum \\nabla_{\\phi}\\log P_{\\theta}(X|Z)(f(X,\\phi) + g(X,\\phi)\\epsilon)\n",
    "\\end{align}\n",
    "\n",
    "Now let's see if we can implement it using tensorflow. We begin by sanity-checking a basic MLP and then go to VAEs. First, import some things we'll need and download MNIST: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "encoder_depth = 2\n",
    "decoder_depth = 2\n",
    "encoder_units = 500\n",
    "decoder_units = 500\n",
    "latent_size = 10\n",
    "global_dtype = tf.float32\n",
    "minibatch_size = 100\n",
    "input_size = mnist.train.images.shape[1]\n",
    "train_steps = mnist.train.num_examples // minibatch_size\n",
    "encoder_nonlinearity = tf.nn.sigmoid\n",
    "decoder_nonlinearity = tf.nn.sigmoid\n",
    "n_epochs = 10"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Construct the graph: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "vae_graph = tf.Graph()\n",
    "\n",
    "with vae_graph.as_default():\n",
    "\n",
    "    with tf.name_scope(\"Encoder_Q\"):\n",
    "\n",
    "        x = tf.placeholder(shape=[None, input_size], dtype=global_dtype, name='x')\n",
    "        q_network, q_mu_vars = _mlp(x, n_layers=encoder_depth, units_per_layer=encoder_units, input_size=input_size, out_size=encoder_units, nonlinearity=encoder_nonlinearity)\n",
    "        w_mu = tf.Variable(tf.truncated_normal(shape=[encoder_units, latent_size], stddev=0.1), name=\"w_mu\")\n",
    "        w_logsig = tf.Variable(tf.truncated_normal(shape=[encoder_units, latent_size], stddev=0.1), name=\"w_logsig\")\n",
    "        b_mu = tf.Variable(tf.truncated_normal(shape=[latent_size], stddev=0.1), name=\"b_mu\")\n",
    "        b_logsig = tf.Variable(tf.truncated_normal(shape=[latent_size], stddev=0.1), name=\"b_logsig\")\n",
    "        q_mu = tf.matmul(q_network, w_mu) + b_mu\n",
    "        q_logsigma = tf.matmul(q_network, w_logsig) + b_logsig\n",
    "        epsilon = tf.random_normal([minibatch_size, latent_size])\n",
    "        z = q_mu + tf.sqrt(tf.exp(q_logsigma)) * epsilon\n",
    "\n",
    "\n",
    "    with tf.name_scope(\"Decoder_P\"):\n",
    "\n",
    "        p_mu, p_mu_vars = _mlp(z, n_layers=decoder_depth, units_per_layer=decoder_units, input_size=latent_size,  out_size=input_size, nonlinearity=decoder_nonlinearity)\n",
    "\n",
    "    with tf.name_scope(\"Opt_and_loss\"):\n",
    "        kld = 0.5 * tf.reduce_sum(1 + q_logsigma - tf.square(q_mu) - tf.exp(q_logsigma), 1)\n",
    "        ll = tf.reduce_sum(x * tf.log(1e-10 + p_mu)+ (1-x) * tf.log(1e-10 + 1 - p_mu), 1)\n",
    "        elbo = tf.reduce_mean(ll + kld)\n",
    "        minimize_op = tf.train.AdamOptimizer(0.001).minimize(-elbo)\n",
    "\n",
    "    with tf.name_scope(\"Support_stuff\"):\n",
    "        init = tf.global_variables_initializer()\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "        <iframe seamless style=\"width:1200px;height:800px;border:0\" srcdoc=\"\n",
       "        <script src=&quot;//cdnjs.cloudflare.com/ajax/libs/polymer/0.3.3/platform.js&quot;></script>\n",
       "        <script>\n",
       "          function load() {\n",
       "            document.getElementById(&quot;graph0.5321912839271279&quot;).pbtxt = 'node {\\n  name: &quot;Encoder_Q/x&quot;\\n  op: &quot;Placeholder&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: -1\\n        }\\n        dim {\\n          size: 784\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;\\\\020\\\\003\\\\000\\\\000\\\\364\\\\001\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0010000000474974513\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal/TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/truncated_normal/mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/ones&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/mul/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/ones&quot;\\n  input: &quot;Encoder_Q/mul/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_W&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 784\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_W/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/into_hidden_W&quot;\\n  input: &quot;Encoder_Q/truncated_normal&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_W/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/into_hidden_W&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_b&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_b/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/into_hidden_b&quot;\\n  input: &quot;Encoder_Q/mul&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_b/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/into_hidden_b&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/x&quot;\\n  input: &quot;Encoder_Q/into_hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/add&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/MatMul&quot;\\n  input: &quot;Encoder_Q/into_hidden_b/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/Sigmoid&quot;\\n  op: &quot;Sigmoid&quot;\\n  input: &quot;Encoder_Q/add&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_1/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;\\\\364\\\\001\\\\000\\\\000\\\\364\\\\001\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_1/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_1/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0010000000474974513\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_1/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_1/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_1/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_1/TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_1/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_1&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/truncated_normal_1/mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_1/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/ones_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/mul_1/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/mul_1&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/ones_1&quot;\\n  input: &quot;Encoder_Q/mul_1/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_W&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_W/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/hidden_W&quot;\\n  input: &quot;Encoder_Q/truncated_normal_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_W/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/hidden_W&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_b&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_b/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/hidden_b&quot;\\n  input: &quot;Encoder_Q/mul_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_b/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/hidden_b&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/Sigmoid&quot;\\n  input: &quot;Encoder_Q/hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/add_1&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/MatMul_1&quot;\\n  input: &quot;Encoder_Q/hidden_b/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/Sigmoid_1&quot;\\n  op: &quot;Sigmoid&quot;\\n  input: &quot;Encoder_Q/add_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_2/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;\\\\364\\\\001\\\\000\\\\000\\\\364\\\\001\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_2/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_2/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0010000000474974513\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_2/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_2/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_2/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_2/TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_2/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_2&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/truncated_normal_2/mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_2/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/ones_2&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/mul_2/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/mul_2&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/ones_2&quot;\\n  input: &quot;Encoder_Q/mul_2/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_W&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_W/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/readout_W&quot;\\n  input: &quot;Encoder_Q/truncated_normal_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_W/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/readout_W&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_b&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_b/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/readout_b&quot;\\n  input: &quot;Encoder_Q/mul_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_b/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/readout_b&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/MatMul_2&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/Sigmoid_1&quot;\\n  input: &quot;Encoder_Q/readout_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/add_2&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/MatMul_2&quot;\\n  input: &quot;Encoder_Q/readout_b/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/Sigmoid_2&quot;\\n  op: &quot;Sigmoid&quot;\\n  input: &quot;Encoder_Q/add_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_3/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;\\\\364\\\\001\\\\000\\\\000\\\\n\\\\000\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_3/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_3/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_3/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_3/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_3/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_3/TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_3/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_3&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/truncated_normal_3/mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_3/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_mu&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_mu/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/w_mu&quot;\\n  input: &quot;Encoder_Q/truncated_normal_3&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_mu/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/w_mu&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_mu&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_4/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;\\\\364\\\\001\\\\000\\\\000\\\\n\\\\000\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_4/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_4/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_4/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_4/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_4/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_4/TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_4/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_4&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/truncated_normal_4/mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_4/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_logsig&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_logsig/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/w_logsig&quot;\\n  input: &quot;Encoder_Q/truncated_normal_4&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_logsig/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/w_logsig&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_logsig&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_5/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 10\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_5/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_5/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_5/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_5/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_5/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_5/TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_5/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_5&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/truncated_normal_5/mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_5/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_mu&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_mu/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/b_mu&quot;\\n  input: &quot;Encoder_Q/truncated_normal_5&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_mu/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/b_mu&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_mu&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_6/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 10\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_6/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_6/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_6/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_6/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_6/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_6/TruncatedNormal&quot;\\n  input: &quot;Encoder_Q/truncated_normal_6/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/truncated_normal_6&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/truncated_normal_6/mul&quot;\\n  input: &quot;Encoder_Q/truncated_normal_6/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_logsig&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_logsig/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/b_logsig&quot;\\n  input: &quot;Encoder_Q/truncated_normal_6&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_logsig/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/b_logsig&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_logsig&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/MatMul_3&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/Sigmoid_2&quot;\\n  input: &quot;Encoder_Q/w_mu/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/add_3&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/MatMul_3&quot;\\n  input: &quot;Encoder_Q/b_mu/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/MatMul_4&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/Sigmoid_2&quot;\\n  input: &quot;Encoder_Q/w_logsig/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/add_4&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/MatMul_4&quot;\\n  input: &quot;Encoder_Q/b_logsig/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/random_normal/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\n\\\\000\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/random_normal/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/random_normal/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/random_normal/RandomStandardNormal&quot;\\n  op: &quot;RandomStandardNormal&quot;\\n  input: &quot;Encoder_Q/random_normal/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/random_normal/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/random_normal/RandomStandardNormal&quot;\\n  input: &quot;Encoder_Q/random_normal/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/random_normal&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/random_normal/mul&quot;\\n  input: &quot;Encoder_Q/random_normal/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/Exp&quot;\\n  op: &quot;Exp&quot;\\n  input: &quot;Encoder_Q/add_4&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/Sqrt&quot;\\n  op: &quot;Sqrt&quot;\\n  input: &quot;Encoder_Q/Exp&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/mul_3&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/Sqrt&quot;\\n  input: &quot;Encoder_Q/random_normal&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/add_5&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Encoder_Q/add_3&quot;\\n  input: &quot;Encoder_Q/mul_3&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;\\\\n\\\\000\\\\000\\\\000\\\\364\\\\001\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0010000000474974513\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Decoder_P/truncated_normal/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Decoder_P/truncated_normal/TruncatedNormal&quot;\\n  input: &quot;Decoder_P/truncated_normal/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Decoder_P/truncated_normal/mul&quot;\\n  input: &quot;Decoder_P/truncated_normal/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/ones&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/mul/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Decoder_P/ones&quot;\\n  input: &quot;Decoder_P/mul/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_W&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_W/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/into_hidden_W&quot;\\n  input: &quot;Decoder_P/truncated_normal&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_W/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/into_hidden_W&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_b&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_b/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/into_hidden_b&quot;\\n  input: &quot;Decoder_P/mul&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_b/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/into_hidden_b&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/add_5&quot;\\n  input: &quot;Decoder_P/into_hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/add&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Decoder_P/MatMul&quot;\\n  input: &quot;Decoder_P/into_hidden_b/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/Sigmoid&quot;\\n  op: &quot;Sigmoid&quot;\\n  input: &quot;Decoder_P/add&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_1/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;\\\\364\\\\001\\\\000\\\\000\\\\364\\\\001\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_1/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_1/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0010000000474974513\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_1/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Decoder_P/truncated_normal_1/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_1/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Decoder_P/truncated_normal_1/TruncatedNormal&quot;\\n  input: &quot;Decoder_P/truncated_normal_1/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_1&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Decoder_P/truncated_normal_1/mul&quot;\\n  input: &quot;Decoder_P/truncated_normal_1/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/ones_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/mul_1/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/mul_1&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Decoder_P/ones_1&quot;\\n  input: &quot;Decoder_P/mul_1/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_W&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_W/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/hidden_W&quot;\\n  input: &quot;Decoder_P/truncated_normal_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_W/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/hidden_W&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_b&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_b/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/hidden_b&quot;\\n  input: &quot;Decoder_P/mul_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_b/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/hidden_b&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Decoder_P/Sigmoid&quot;\\n  input: &quot;Decoder_P/hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/add_1&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Decoder_P/MatMul_1&quot;\\n  input: &quot;Decoder_P/hidden_b/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/Sigmoid_1&quot;\\n  op: &quot;Sigmoid&quot;\\n  input: &quot;Decoder_P/add_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_2/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;\\\\364\\\\001\\\\000\\\\000\\\\020\\\\003\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_2/mean&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_2/stddev&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0010000000474974513\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_2/TruncatedNormal&quot;\\n  op: &quot;TruncatedNormal&quot;\\n  input: &quot;Decoder_P/truncated_normal_2/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;seed&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n  attr {\\n    key: &quot;seed2&quot;\\n    value {\\n      i: 0\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_2/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Decoder_P/truncated_normal_2/TruncatedNormal&quot;\\n  input: &quot;Decoder_P/truncated_normal_2/stddev&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/truncated_normal_2&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Decoder_P/truncated_normal_2/mul&quot;\\n  input: &quot;Decoder_P/truncated_normal_2/mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/ones_2&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 784\\n          }\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/mul_2/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.10000000149011612\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/mul_2&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Decoder_P/ones_2&quot;\\n  input: &quot;Decoder_P/mul_2/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_W&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 784\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_W/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/readout_W&quot;\\n  input: &quot;Decoder_P/truncated_normal_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_W/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/readout_W&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_b&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 784\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_b/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/readout_b&quot;\\n  input: &quot;Decoder_P/mul_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_b/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/readout_b&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/MatMul_2&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Decoder_P/Sigmoid_1&quot;\\n  input: &quot;Decoder_P/readout_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/add_2&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Decoder_P/MatMul_2&quot;\\n  input: &quot;Decoder_P/readout_b/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/Sigmoid_2&quot;\\n  op: &quot;Sigmoid&quot;\\n  input: &quot;Decoder_P/add_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/add/x&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/add&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Opt_and_loss/add/x&quot;\\n  input: &quot;Encoder_Q/add_4&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Square&quot;\\n  op: &quot;Square&quot;\\n  input: &quot;Encoder_Q/add_3&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/sub&quot;\\n  op: &quot;Sub&quot;\\n  input: &quot;Opt_and_loss/add&quot;\\n  input: &quot;Opt_and_loss/Square&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Exp&quot;\\n  op: &quot;Exp&quot;\\n  input: &quot;Encoder_Q/add_4&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/sub_1&quot;\\n  op: &quot;Sub&quot;\\n  input: &quot;Opt_and_loss/sub&quot;\\n  input: &quot;Opt_and_loss/Exp&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Sum/reduction_indices&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/sub_1&quot;\\n  input: &quot;Opt_and_loss/Sum/reduction_indices&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/mul/x&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.5\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/mul/x&quot;\\n  input: &quot;Opt_and_loss/Sum&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/add_1/x&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 1.000000013351432e-10\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/add_1&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Opt_and_loss/add_1/x&quot;\\n  input: &quot;Decoder_P/Sigmoid_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Log&quot;\\n  op: &quot;Log&quot;\\n  input: &quot;Opt_and_loss/add_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/mul_1&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/x&quot;\\n  input: &quot;Opt_and_loss/Log&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/sub_2/x&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/sub_2&quot;\\n  op: &quot;Sub&quot;\\n  input: &quot;Opt_and_loss/sub_2/x&quot;\\n  input: &quot;Encoder_Q/x&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/sub_3/x&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/sub_3&quot;\\n  op: &quot;Sub&quot;\\n  input: &quot;Opt_and_loss/sub_3/x&quot;\\n  input: &quot;Decoder_P/Sigmoid_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Log_1&quot;\\n  op: &quot;Log&quot;\\n  input: &quot;Opt_and_loss/sub_3&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/mul_2&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/sub_2&quot;\\n  input: &quot;Opt_and_loss/Log_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/add_2&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Opt_and_loss/mul_1&quot;\\n  input: &quot;Opt_and_loss/mul_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Sum_1/reduction_indices&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/add_2&quot;\\n  input: &quot;Opt_and_loss/Sum_1/reduction_indices&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/add_3&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Opt_and_loss/Sum_1&quot;\\n  input: &quot;Opt_and_loss/mul&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Const&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Mean&quot;\\n  op: &quot;Mean&quot;\\n  input: &quot;Opt_and_loss/add_3&quot;\\n  input: &quot;Opt_and_loss/Const&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Neg&quot;\\n  op: &quot;Neg&quot;\\n  input: &quot;Opt_and_loss/Mean&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n          }\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Const&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 1.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Fill&quot;\\n  op: &quot;Fill&quot;\\n  input: &quot;Opt_and_loss/gradients/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Const&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Neg_grad/Neg&quot;\\n  op: &quot;Neg&quot;\\n  input: &quot;Opt_and_loss/gradients/Fill&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Reshape/shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Neg_grad/Neg&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Reshape/shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Tile/multiples&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 100\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Tile&quot;\\n  op: &quot;Tile&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Tile/multiples&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tmultiples&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 100\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n          }\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Const&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Prod&quot;\\n  op: &quot;Prod&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Const&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Const_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Prod_1&quot;\\n  op: &quot;Prod&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Shape_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Const_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Maximum/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Maximum&quot;\\n  op: &quot;Maximum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Prod_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Maximum/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/floordiv&quot;\\n  op: &quot;FloorDiv&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Prod&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Maximum&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Cast&quot;\\n  op: &quot;Cast&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/floordiv&quot;\\n  attr {\\n    key: &quot;DstT&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;SrcT&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/truediv&quot;\\n  op: &quot;RealDiv&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Tile&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/Cast&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 100\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Shape_1&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Opt_and_loss/mul&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/truediv&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Mean_grad/truediv&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_3_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_3_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/add_3_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\020\\\\003\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Size&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 2\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/add&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Opt_and_loss/Sum_1/reduction_indices&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Size&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/mod&quot;\\n  op: &quot;FloorMod&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/add&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Size&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n          }\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/range/start&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/range/delta&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/range&quot;\\n  op: &quot;Range&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/range/start&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Size&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/range/delta&quot;\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Fill/value&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Fill&quot;\\n  op: &quot;Fill&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Shape_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Fill/value&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/DynamicStitch&quot;\\n  op: &quot;DynamicStitch&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/range&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/mod&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Fill&quot;\\n  attr {\\n    key: &quot;N&quot;\\n    value {\\n      i: 2\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Maximum/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Maximum&quot;\\n  op: &quot;Maximum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/DynamicStitch&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Maximum/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/floordiv&quot;\\n  op: &quot;FloorDiv&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Maximum&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/tuple/control_dependency&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/DynamicStitch&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Tile&quot;\\n  op: &quot;Tile&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/floordiv&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tmultiples&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n          }\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Shape_1&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Opt_and_loss/Sum&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/tuple/control_dependency_1&quot;\\n  input: &quot;Opt_and_loss/Sum&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/mul_1&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/mul/x&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_3_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/mul_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/mul_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/mul_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\020\\\\003\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\020\\\\003\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Tile&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_1_grad/Tile&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/add_2_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Opt_and_loss/sub_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Size&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 2\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/add&quot;\\n  op: &quot;Add&quot;\\n  input: &quot;Opt_and_loss/Sum/reduction_indices&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Size&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/mod&quot;\\n  op: &quot;FloorMod&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/add&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Size&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n          }\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/range/start&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/range/delta&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/range&quot;\\n  op: &quot;Range&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/range/start&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Size&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/range/delta&quot;\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Fill/value&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Fill&quot;\\n  op: &quot;Fill&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Shape_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Fill/value&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/DynamicStitch&quot;\\n  op: &quot;DynamicStitch&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/range&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/mod&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Fill&quot;\\n  attr {\\n    key: &quot;N&quot;\\n    value {\\n      i: 2\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Maximum/y&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n        }\\n        int_val: 1\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Maximum&quot;\\n  op: &quot;Maximum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/DynamicStitch&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Maximum/y&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/floordiv&quot;\\n  op: &quot;FloorDiv&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Maximum&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_grad/tuple/control_dependency_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/DynamicStitch&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Tile&quot;\\n  op: &quot;Tile&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/floordiv&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tmultiples&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Encoder_Q/x&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\020\\\\003\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/tuple/control_dependency&quot;\\n  input: &quot;Opt_and_loss/Log&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/mul_1&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/x&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/mul_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Opt_and_loss/sub_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\020\\\\003\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/tuple/control_dependency_1&quot;\\n  input: &quot;Opt_and_loss/Log_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/mul_1&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/sub_2&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_2_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/mul_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Opt_and_loss/sub&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Shape_1&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Opt_and_loss/Exp&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Tile&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Sum_grad/Tile&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Neg&quot;\\n  op: &quot;Neg&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Sum_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Neg&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Log_grad/Reciprocal&quot;\\n  op: &quot;Reciprocal&quot;\\n  input: &quot;Opt_and_loss/add_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Log_grad/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_1_grad/tuple/control_dependency_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Log_grad/Reciprocal&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Log_1_grad/Reciprocal&quot;\\n  op: &quot;Reciprocal&quot;\\n  input: &quot;Opt_and_loss/sub_3&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Log_1_grad/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/mul_2_grad/tuple/control_dependency_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Log_1_grad/Reciprocal&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Opt_and_loss/add&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Shape_1&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Opt_and_loss/Square&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/tuple/control_dependency&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/tuple/control_dependency&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Neg&quot;\\n  op: &quot;Neg&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Sum_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Neg&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/sub_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/sub_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Exp_grad/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_1_grad/tuple/control_dependency_1&quot;\\n  input: &quot;Opt_and_loss/Exp&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n          }\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\020\\\\003\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Log_grad/mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Log_grad/mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n          }\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\020\\\\003\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Log_1_grad/mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Log_1_grad/mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Neg&quot;\\n  op: &quot;Neg&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Sum_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Neg&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n          }\\n        }\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Shape_1&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Encoder_Q/add_4&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/tuple/control_dependency&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/tuple/control_dependency&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/add_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/add_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/add_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Square_grad/mul/x&quot;\\n  op: &quot;Const&quot;\\n  input: &quot;^Opt_and_loss/gradients/Opt_and_loss/sub_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 2.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Square_grad/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Square_grad/mul/x&quot;\\n  input: &quot;Encoder_Q/add_3&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Opt_and_loss/Square_grad/mul_1&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_grad/tuple/control_dependency_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Square_grad/mul&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/AddN&quot;\\n  op: &quot;AddN&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_1_grad/tuple/control_dependency_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/sub_3_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;N&quot;\\n    value {\\n      i: 2\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/add_1_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/Sigmoid_2_grad/SigmoidGrad&quot;\\n  op: &quot;SigmoidGrad&quot;\\n  input: &quot;Decoder_P/Sigmoid_2&quot;\\n  input: &quot;Opt_and_loss/gradients/AddN&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\020\\\\003\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 784\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/Sigmoid_2_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/Sigmoid_2_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_2_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_2_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/add_2_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/add_2_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/tuple/control_dependency&quot;\\n  input: &quot;Decoder_P/readout_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Decoder_P/Sigmoid_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/MatMul_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/MatMul_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/MatMul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/Sigmoid_1_grad/SigmoidGrad&quot;\\n  op: &quot;SigmoidGrad&quot;\\n  input: &quot;Decoder_P/Sigmoid_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\364\\\\001\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 500\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/Sigmoid_1_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/Sigmoid_1_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_1_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/add_1_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/add_1_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/tuple/control_dependency&quot;\\n  input: &quot;Decoder_P/hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Decoder_P/Sigmoid&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/MatMul_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/MatMul_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/MatMul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/Sigmoid_grad/SigmoidGrad&quot;\\n  op: &quot;SigmoidGrad&quot;\\n  input: &quot;Decoder_P/Sigmoid&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Shape&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\364\\\\001\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 500\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/Sigmoid_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/Sigmoid_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/add_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/add_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/add_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_grad/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/tuple/control_dependency&quot;\\n  input: &quot;Decoder_P/into_hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_grad/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/add_5&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_grad/MatMul_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/MatMul_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_grad/MatMul_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Decoder_P/MatMul_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Decoder_P/MatMul_grad/MatMul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Encoder_Q/add_3&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\n\\\\000\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_grad/tuple/control_dependency&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_grad/tuple/control_dependency&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_5_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_5_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_5_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_5_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_5_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_5_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/AddN_1&quot;\\n  op: &quot;AddN&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Square_grad/mul_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;N&quot;\\n    value {\\n      i: 2\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/Square_grad/mul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Encoder_Q/MatMul_3&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 10\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/AddN_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/AddN_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_3_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_3_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_3_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_3_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_3_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_3_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Encoder_Q/Sqrt&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 2\\n          }\\n        }\\n        tensor_content: &quot;d\\\\000\\\\000\\\\000\\\\n\\\\000\\\\000\\\\000&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/tuple/control_dependency_1&quot;\\n  input: &quot;Encoder_Q/random_normal&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/mul_1&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Encoder_Q/Sqrt&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_5_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/mul_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/mul_3_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/mul_3_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/mul_3_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/tuple/control_dependency&quot;\\n  input: &quot;Encoder_Q/w_mu/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/Sigmoid_2&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/MatMul_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/MatMul_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/MatMul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/Sqrt_grad/SqrtGrad&quot;\\n  op: &quot;SqrtGrad&quot;\\n  input: &quot;Encoder_Q/Sqrt&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/mul_3_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/Exp_grad/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/Sqrt_grad/SqrtGrad&quot;\\n  input: &quot;Encoder_Q/Exp&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/AddN_2&quot;\\n  op: &quot;AddN&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/Exp_grad/mul&quot;\\n  input: &quot;Opt_and_loss/gradients/Opt_and_loss/add_grad/tuple/control_dependency_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/Exp_grad/mul&quot;\\n  attr {\\n    key: &quot;N&quot;\\n    value {\\n      i: 3\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Opt_and_loss/Exp_grad/mul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Encoder_Q/MatMul_4&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 10\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/AddN_2&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/AddN_2&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_4_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_4_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_4_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_4_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_4_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_4_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/tuple/control_dependency&quot;\\n  input: &quot;Encoder_Q/w_logsig/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/Sigmoid_2&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/MatMul_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/MatMul_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/MatMul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/AddN_3&quot;\\n  op: &quot;AddN&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/tuple/control_dependency&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;N&quot;\\n    value {\\n      i: 2\\n    }\\n  }\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/Sigmoid_2_grad/SigmoidGrad&quot;\\n  op: &quot;SigmoidGrad&quot;\\n  input: &quot;Encoder_Q/Sigmoid_2&quot;\\n  input: &quot;Opt_and_loss/gradients/AddN_3&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Encoder_Q/MatMul_2&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 500\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/Sigmoid_2_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/Sigmoid_2_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_2_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_2_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_2_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_2_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/tuple/control_dependency&quot;\\n  input: &quot;Encoder_Q/readout_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/Sigmoid_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/MatMul_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/MatMul_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/MatMul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/Sigmoid_1_grad/SigmoidGrad&quot;\\n  op: &quot;SigmoidGrad&quot;\\n  input: &quot;Encoder_Q/Sigmoid_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Encoder_Q/MatMul_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 500\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/Sigmoid_1_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/Sigmoid_1_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_1_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_1_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_1_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/tuple/control_dependency&quot;\\n  input: &quot;Encoder_Q/hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/Sigmoid&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/MatMul_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/MatMul_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/MatMul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/Sigmoid_grad/SigmoidGrad&quot;\\n  op: &quot;SigmoidGrad&quot;\\n  input: &quot;Encoder_Q/Sigmoid&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Shape&quot;\\n  op: &quot;Shape&quot;\\n  input: &quot;Encoder_Q/MatMul&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;out_type&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Shape_1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_INT32\\n        tensor_shape {\\n          dim {\\n            size: 1\\n          }\\n        }\\n        int_val: 500\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/BroadcastGradientArgs&quot;\\n  op: &quot;BroadcastGradientArgs&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Shape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Sum&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/Sigmoid_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/BroadcastGradientArgs&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Reshape&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Shape&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Sum_1&quot;\\n  op: &quot;Sum&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/Sigmoid_grad/SigmoidGrad&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/BroadcastGradientArgs:1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tidx&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n  attr {\\n    key: &quot;keep_dims&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Reshape_1&quot;\\n  op: &quot;Reshape&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Sum_1&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Shape_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;Tshape&quot;\\n    value {\\n      type: DT_INT32\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_grad/Reshape_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Reshape&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_grad/Reshape&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/Reshape_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/add_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/add_grad/Reshape_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_grad/MatMul&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/tuple/control_dependency&quot;\\n  input: &quot;Encoder_Q/into_hidden_W/read&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_grad/MatMul_1&quot;\\n  op: &quot;MatMul&quot;\\n  input: &quot;Encoder_Q/x&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/tuple/control_dependency&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_a&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;transpose_b&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_grad/tuple/group_deps&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_grad/MatMul_1&quot;\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_grad/tuple/control_dependency&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_grad/MatMul&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/MatMul_grad/MatMul&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_grad/tuple/control_dependency_1&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_grad/MatMul_1&quot;\\n  input: &quot;^Opt_and_loss/gradients/Encoder_Q/MatMul_grad/tuple/group_deps&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Opt_and_loss/gradients/Encoder_Q/MatMul_grad/MatMul_1&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta1_power/initial_value&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.8999999761581421\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta1_power&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta1_power/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Opt_and_loss/beta1_power&quot;\\n  input: &quot;Opt_and_loss/beta1_power/initial_value&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta1_power/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/beta1_power&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta2_power/initial_value&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.9990000128746033\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta2_power&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta2_power/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Opt_and_loss/beta2_power&quot;\\n  input: &quot;Opt_and_loss/beta2_power/initial_value&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/beta2_power/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Opt_and_loss/beta2_power&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_W/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 784\\n          }\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_W/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 784\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_W/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/into_hidden_W/Adam&quot;\\n  input: &quot;Encoder_Q/into_hidden_W/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_W/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/into_hidden_W/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_W/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 784\\n          }\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_W/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 784\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_W/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/into_hidden_W/Adam_1&quot;\\n  input: &quot;Encoder_Q/into_hidden_W/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_W/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/into_hidden_W/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_b/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_b/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_b/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/into_hidden_b/Adam&quot;\\n  input: &quot;Encoder_Q/into_hidden_b/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_b/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/into_hidden_b/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_b/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_b/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_b/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/into_hidden_b/Adam_1&quot;\\n  input: &quot;Encoder_Q/into_hidden_b/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/into_hidden_b/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/into_hidden_b/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_W/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_W/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_W/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/hidden_W/Adam&quot;\\n  input: &quot;Encoder_Q/hidden_W/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_W/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/hidden_W/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_W/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_W/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_W/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/hidden_W/Adam_1&quot;\\n  input: &quot;Encoder_Q/hidden_W/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_W/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/hidden_W/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_b/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_b/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_b/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/hidden_b/Adam&quot;\\n  input: &quot;Encoder_Q/hidden_b/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_b/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/hidden_b/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_b/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_b/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_b/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/hidden_b/Adam_1&quot;\\n  input: &quot;Encoder_Q/hidden_b/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/hidden_b/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/hidden_b/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_W/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_W/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_W/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/readout_W/Adam&quot;\\n  input: &quot;Encoder_Q/readout_W/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_W/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/readout_W/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_W/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_W/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_W/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/readout_W/Adam_1&quot;\\n  input: &quot;Encoder_Q/readout_W/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_W/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/readout_W/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_b/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_b/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_b/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/readout_b/Adam&quot;\\n  input: &quot;Encoder_Q/readout_b/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_b/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/readout_b/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_b/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_b/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_b/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/readout_b/Adam_1&quot;\\n  input: &quot;Encoder_Q/readout_b/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/readout_b/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/readout_b/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_mu/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_mu/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_mu/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/w_mu/Adam&quot;\\n  input: &quot;Encoder_Q/w_mu/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_mu/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/w_mu/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_mu&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_mu/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_mu/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_mu/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/w_mu/Adam_1&quot;\\n  input: &quot;Encoder_Q/w_mu/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_mu/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/w_mu/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_mu&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_logsig/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_logsig/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_logsig/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/w_logsig/Adam&quot;\\n  input: &quot;Encoder_Q/w_logsig/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_logsig/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/w_logsig/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_logsig&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_logsig/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_logsig/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_logsig/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/w_logsig/Adam_1&quot;\\n  input: &quot;Encoder_Q/w_logsig/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/w_logsig/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/w_logsig/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_logsig&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_mu/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_mu/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_mu/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/b_mu/Adam&quot;\\n  input: &quot;Encoder_Q/b_mu/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_mu/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/b_mu/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_mu&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_mu/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_mu/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_mu/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/b_mu/Adam_1&quot;\\n  input: &quot;Encoder_Q/b_mu/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_mu/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/b_mu/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_mu&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_logsig/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_logsig/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_logsig/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/b_logsig/Adam&quot;\\n  input: &quot;Encoder_Q/b_logsig/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_logsig/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/b_logsig/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_logsig&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_logsig/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 10\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_logsig/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_logsig/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Encoder_Q/b_logsig/Adam_1&quot;\\n  input: &quot;Encoder_Q/b_logsig/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Encoder_Q/b_logsig/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Encoder_Q/b_logsig/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_logsig&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_W/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 10\\n          }\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_W/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_W/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/into_hidden_W/Adam&quot;\\n  input: &quot;Decoder_P/into_hidden_W/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_W/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/into_hidden_W/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_W/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 10\\n          }\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_W/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 10\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_W/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/into_hidden_W/Adam_1&quot;\\n  input: &quot;Decoder_P/into_hidden_W/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_W/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/into_hidden_W/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_b/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_b/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_b/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/into_hidden_b/Adam&quot;\\n  input: &quot;Decoder_P/into_hidden_b/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_b/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/into_hidden_b/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_b/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_b/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_b/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/into_hidden_b/Adam_1&quot;\\n  input: &quot;Decoder_P/into_hidden_b/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/into_hidden_b/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/into_hidden_b/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_W/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_W/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_W/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/hidden_W/Adam&quot;\\n  input: &quot;Decoder_P/hidden_W/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_W/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/hidden_W/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_W/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_W/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_W/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/hidden_W/Adam_1&quot;\\n  input: &quot;Decoder_P/hidden_W/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_W/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/hidden_W/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_b/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_b/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_b/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/hidden_b/Adam&quot;\\n  input: &quot;Decoder_P/hidden_b/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_b/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/hidden_b/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_b/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_b/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_b/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/hidden_b/Adam_1&quot;\\n  input: &quot;Decoder_P/hidden_b/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/hidden_b/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/hidden_b/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_W/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 784\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_W/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 784\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_W/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/readout_W/Adam&quot;\\n  input: &quot;Decoder_P/readout_W/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_W/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/readout_W/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_W/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 500\\n          }\\n          dim {\\n            size: 784\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_W/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 500\\n        }\\n        dim {\\n          size: 784\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_W/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/readout_W/Adam_1&quot;\\n  input: &quot;Decoder_P/readout_W/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_W/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/readout_W/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_b/Adam/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 784\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_b/Adam&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 784\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_b/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/readout_b/Adam&quot;\\n  input: &quot;Decoder_P/readout_b/Adam/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_b/Adam/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/readout_b/Adam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_b/Adam_1/Initializer/zeros&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n          dim {\\n            size: 784\\n          }\\n        }\\n        float_val: 0.0\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_b/Adam_1&quot;\\n  op: &quot;VariableV2&quot;\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;container&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;shape&quot;\\n    value {\\n      shape {\\n        dim {\\n          size: 784\\n        }\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;shared_name&quot;\\n    value {\\n      s: &quot;&quot;\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_b/Adam_1/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Decoder_P/readout_b/Adam_1&quot;\\n  input: &quot;Decoder_P/readout_b/Adam_1/Initializer/zeros&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Decoder_P/readout_b/Adam_1/read&quot;\\n  op: &quot;Identity&quot;\\n  input: &quot;Decoder_P/readout_b/Adam_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_b&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.0010000000474974513\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/beta1&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.8999999761581421\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/beta2&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 0.9990000128746033\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  op: &quot;Const&quot;\\n  attr {\\n    key: &quot;dtype&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;value&quot;\\n    value {\\n      tensor {\\n        dtype: DT_FLOAT\\n        tensor_shape {\\n        }\\n        float_val: 9.99999993922529e-09\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Encoder_Q/into_hidden_W/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Encoder_Q/into_hidden_W&quot;\\n  input: &quot;Encoder_Q/into_hidden_W/Adam&quot;\\n  input: &quot;Encoder_Q/into_hidden_W/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Encoder_Q/into_hidden_b/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Encoder_Q/into_hidden_b&quot;\\n  input: &quot;Encoder_Q/into_hidden_b/Adam&quot;\\n  input: &quot;Encoder_Q/into_hidden_b/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Encoder_Q/hidden_W/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Encoder_Q/hidden_W&quot;\\n  input: &quot;Encoder_Q/hidden_W/Adam&quot;\\n  input: &quot;Encoder_Q/hidden_W/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_1_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Encoder_Q/hidden_b/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Encoder_Q/hidden_b&quot;\\n  input: &quot;Encoder_Q/hidden_b/Adam&quot;\\n  input: &quot;Encoder_Q/hidden_b/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_1_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Encoder_Q/readout_W/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Encoder_Q/readout_W&quot;\\n  input: &quot;Encoder_Q/readout_W/Adam&quot;\\n  input: &quot;Encoder_Q/readout_W/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_2_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Encoder_Q/readout_b/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Encoder_Q/readout_b&quot;\\n  input: &quot;Encoder_Q/readout_b/Adam&quot;\\n  input: &quot;Encoder_Q/readout_b/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_2_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Encoder_Q/w_mu/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Encoder_Q/w_mu&quot;\\n  input: &quot;Encoder_Q/w_mu/Adam&quot;\\n  input: &quot;Encoder_Q/w_mu/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_3_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Encoder_Q/w_logsig/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Encoder_Q/w_logsig&quot;\\n  input: &quot;Encoder_Q/w_logsig/Adam&quot;\\n  input: &quot;Encoder_Q/w_logsig/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/MatMul_4_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/w_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Encoder_Q/b_mu/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Encoder_Q/b_mu&quot;\\n  input: &quot;Encoder_Q/b_mu/Adam&quot;\\n  input: &quot;Encoder_Q/b_mu/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_3_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_mu&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Encoder_Q/b_logsig/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Encoder_Q/b_logsig&quot;\\n  input: &quot;Encoder_Q/b_logsig/Adam&quot;\\n  input: &quot;Encoder_Q/b_logsig/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Encoder_Q/add_4_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/b_logsig&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Decoder_P/into_hidden_W/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Decoder_P/into_hidden_W&quot;\\n  input: &quot;Decoder_P/into_hidden_W/Adam&quot;\\n  input: &quot;Decoder_P/into_hidden_W/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Decoder_P/into_hidden_b/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Decoder_P/into_hidden_b&quot;\\n  input: &quot;Decoder_P/into_hidden_b/Adam&quot;\\n  input: &quot;Decoder_P/into_hidden_b/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/into_hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Decoder_P/hidden_W/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Decoder_P/hidden_W&quot;\\n  input: &quot;Decoder_P/hidden_W/Adam&quot;\\n  input: &quot;Decoder_P/hidden_W/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_1_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Decoder_P/hidden_b/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Decoder_P/hidden_b&quot;\\n  input: &quot;Decoder_P/hidden_b/Adam&quot;\\n  input: &quot;Decoder_P/hidden_b/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_1_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/hidden_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Decoder_P/readout_W/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Decoder_P/readout_W&quot;\\n  input: &quot;Decoder_P/readout_W/Adam&quot;\\n  input: &quot;Decoder_P/readout_W/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/MatMul_2_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/update_Decoder_P/readout_b/ApplyAdam&quot;\\n  op: &quot;ApplyAdam&quot;\\n  input: &quot;Decoder_P/readout_b&quot;\\n  input: &quot;Decoder_P/readout_b/Adam&quot;\\n  input: &quot;Decoder_P/readout_b/Adam_1&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/learning_rate&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;Opt_and_loss/Adam/epsilon&quot;\\n  input: &quot;Opt_and_loss/gradients/Decoder_P/add_2_grad/tuple/control_dependency_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Decoder_P/readout_b&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;use_nesterov&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/mul&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/beta1_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/beta1&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/into_hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/into_hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/readout_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/readout_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/w_mu/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/w_logsig/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/b_mu/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/b_logsig/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/into_hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/into_hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/readout_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/readout_b/ApplyAdam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/Assign&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Opt_and_loss/beta1_power&quot;\\n  input: &quot;Opt_and_loss/Adam/mul&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/mul_1&quot;\\n  op: &quot;Mul&quot;\\n  input: &quot;Opt_and_loss/beta2_power/read&quot;\\n  input: &quot;Opt_and_loss/Adam/beta2&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/into_hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/into_hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/readout_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/readout_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/w_mu/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/w_logsig/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/b_mu/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/b_logsig/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/into_hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/into_hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/readout_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/readout_b/ApplyAdam&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam/Assign_1&quot;\\n  op: &quot;Assign&quot;\\n  input: &quot;Opt_and_loss/beta2_power&quot;\\n  input: &quot;Opt_and_loss/Adam/mul_1&quot;\\n  attr {\\n    key: &quot;T&quot;\\n    value {\\n      type: DT_FLOAT\\n    }\\n  }\\n  attr {\\n    key: &quot;_class&quot;\\n    value {\\n      list {\\n        s: &quot;loc:@Encoder_Q/into_hidden_W&quot;\\n      }\\n    }\\n  }\\n  attr {\\n    key: &quot;use_locking&quot;\\n    value {\\n      b: false\\n    }\\n  }\\n  attr {\\n    key: &quot;validate_shape&quot;\\n    value {\\n      b: true\\n    }\\n  }\\n}\\nnode {\\n  name: &quot;Opt_and_loss/Adam&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/into_hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/into_hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/readout_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/readout_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/w_mu/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/w_logsig/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/b_mu/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Encoder_Q/b_logsig/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/into_hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/into_hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/hidden_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/hidden_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/readout_W/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/update_Decoder_P/readout_b/ApplyAdam&quot;\\n  input: &quot;^Opt_and_loss/Adam/Assign&quot;\\n  input: &quot;^Opt_and_loss/Adam/Assign_1&quot;\\n}\\nnode {\\n  name: &quot;Support_stuff/init&quot;\\n  op: &quot;NoOp&quot;\\n  input: &quot;^Encoder_Q/into_hidden_W/Assign&quot;\\n  input: &quot;^Encoder_Q/into_hidden_b/Assign&quot;\\n  input: &quot;^Encoder_Q/hidden_W/Assign&quot;\\n  input: &quot;^Encoder_Q/hidden_b/Assign&quot;\\n  input: &quot;^Encoder_Q/readout_W/Assign&quot;\\n  input: &quot;^Encoder_Q/readout_b/Assign&quot;\\n  input: &quot;^Encoder_Q/w_mu/Assign&quot;\\n  input: &quot;^Encoder_Q/w_logsig/Assign&quot;\\n  input: &quot;^Encoder_Q/b_mu/Assign&quot;\\n  input: &quot;^Encoder_Q/b_logsig/Assign&quot;\\n  input: &quot;^Decoder_P/into_hidden_W/Assign&quot;\\n  input: &quot;^Decoder_P/into_hidden_b/Assign&quot;\\n  input: &quot;^Decoder_P/hidden_W/Assign&quot;\\n  input: &quot;^Decoder_P/hidden_b/Assign&quot;\\n  input: &quot;^Decoder_P/readout_W/Assign&quot;\\n  input: &quot;^Decoder_P/readout_b/Assign&quot;\\n  input: &quot;^Opt_and_loss/beta1_power/Assign&quot;\\n  input: &quot;^Opt_and_loss/beta2_power/Assign&quot;\\n  input: &quot;^Encoder_Q/into_hidden_W/Adam/Assign&quot;\\n  input: &quot;^Encoder_Q/into_hidden_W/Adam_1/Assign&quot;\\n  input: &quot;^Encoder_Q/into_hidden_b/Adam/Assign&quot;\\n  input: &quot;^Encoder_Q/into_hidden_b/Adam_1/Assign&quot;\\n  input: &quot;^Encoder_Q/hidden_W/Adam/Assign&quot;\\n  input: &quot;^Encoder_Q/hidden_W/Adam_1/Assign&quot;\\n  input: &quot;^Encoder_Q/hidden_b/Adam/Assign&quot;\\n  input: &quot;^Encoder_Q/hidden_b/Adam_1/Assign&quot;\\n  input: &quot;^Encoder_Q/readout_W/Adam/Assign&quot;\\n  input: &quot;^Encoder_Q/readout_W/Adam_1/Assign&quot;\\n  input: &quot;^Encoder_Q/readout_b/Adam/Assign&quot;\\n  input: &quot;^Encoder_Q/readout_b/Adam_1/Assign&quot;\\n  input: &quot;^Encoder_Q/w_mu/Adam/Assign&quot;\\n  input: &quot;^Encoder_Q/w_mu/Adam_1/Assign&quot;\\n  input: &quot;^Encoder_Q/w_logsig/Adam/Assign&quot;\\n  input: &quot;^Encoder_Q/w_logsig/Adam_1/Assign&quot;\\n  input: &quot;^Encoder_Q/b_mu/Adam/Assign&quot;\\n  input: &quot;^Encoder_Q/b_mu/Adam_1/Assign&quot;\\n  input: &quot;^Encoder_Q/b_logsig/Adam/Assign&quot;\\n  input: &quot;^Encoder_Q/b_logsig/Adam_1/Assign&quot;\\n  input: &quot;^Decoder_P/into_hidden_W/Adam/Assign&quot;\\n  input: &quot;^Decoder_P/into_hidden_W/Adam_1/Assign&quot;\\n  input: &quot;^Decoder_P/into_hidden_b/Adam/Assign&quot;\\n  input: &quot;^Decoder_P/into_hidden_b/Adam_1/Assign&quot;\\n  input: &quot;^Decoder_P/hidden_W/Adam/Assign&quot;\\n  input: &quot;^Decoder_P/hidden_W/Adam_1/Assign&quot;\\n  input: &quot;^Decoder_P/hidden_b/Adam/Assign&quot;\\n  input: &quot;^Decoder_P/hidden_b/Adam_1/Assign&quot;\\n  input: &quot;^Decoder_P/readout_W/Adam/Assign&quot;\\n  input: &quot;^Decoder_P/readout_W/Adam_1/Assign&quot;\\n  input: &quot;^Decoder_P/readout_b/Adam/Assign&quot;\\n  input: &quot;^Decoder_P/readout_b/Adam_1/Assign&quot;\\n}\\n';\n",
       "          }\n",
       "        </script>\n",
       "        <link rel=&quot;import&quot; href=&quot;https://tensorboard.appspot.com/tf-graph-basic.build.html&quot; onload=load()>\n",
       "        <div style=&quot;height:800px&quot;>\n",
       "          <tf-graph-basic id=&quot;graph0.5321912839271279&quot;></tf-graph-basic>\n",
       "        </div>\n",
       "    \"></iframe>\n",
       "    "
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "show_graph(vae_graph.as_graph_def())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we run and visualize: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 0, step 0: average elbo=-498.922028\n",
      "Epoch 0, step 10: average elbo=-214.074295\n",
      "Epoch 0, step 20: average elbo=-214.997269\n",
      "Epoch 0, step 30: average elbo=-212.165344\n",
      "Epoch 0, step 40: average elbo=-210.181015\n",
      "Epoch 0, step 50: average elbo=-213.656189\n",
      "Epoch 0, step 60: average elbo=-207.370346\n",
      "Epoch 0, step 70: average elbo=-203.687546\n",
      "Epoch 0, step 80: average elbo=-210.040588\n",
      "Epoch 0, step 90: average elbo=-207.408844\n",
      "Epoch 0, step 100: average elbo=-200.401810\n",
      "Epoch 0, step 110: average elbo=-211.652115\n",
      "Epoch 0, step 120: average elbo=-203.729889\n",
      "Epoch 0, step 130: average elbo=-198.393951\n",
      "Epoch 0, step 140: average elbo=-206.218460\n",
      "Epoch 0, step 150: average elbo=-207.235901\n",
      "Epoch 0, step 160: average elbo=-209.720154\n",
      "Epoch 0, step 170: average elbo=-217.805222\n",
      "Epoch 0, step 180: average elbo=-207.267563\n",
      "Epoch 0, step 190: average elbo=-205.444199\n",
      "Epoch 0, step 200: average elbo=-203.058441\n",
      "Epoch 0, step 210: average elbo=-211.681366\n",
      "Epoch 0, step 220: average elbo=-213.850220\n",
      "Epoch 0, step 230: average elbo=-210.270935\n",
      "Epoch 0, step 240: average elbo=-204.285355\n",
      "Epoch 0, step 250: average elbo=-202.927597\n",
      "Epoch 0, step 260: average elbo=-205.129791\n",
      "Epoch 0, step 270: average elbo=-197.652893\n",
      "Epoch 0, step 280: average elbo=-205.535339\n",
      "Epoch 0, step 290: average elbo=-192.104752\n",
      "Epoch 0, step 300: average elbo=-202.904068\n",
      "Epoch 0, step 310: average elbo=-202.045944\n",
      "Epoch 0, step 320: average elbo=-206.704285\n",
      "Epoch 0, step 330: average elbo=-219.215546\n",
      "Epoch 0, step 340: average elbo=-215.538666\n",
      "Epoch 0, step 350: average elbo=-205.286438\n",
      "Epoch 0, step 360: average elbo=-201.292419\n",
      "Epoch 0, step 370: average elbo=-206.491440\n",
      "Epoch 0, step 380: average elbo=-207.749573\n",
      "Epoch 0, step 390: average elbo=-206.005707\n",
      "Epoch 0, step 400: average elbo=-203.475937\n",
      "Epoch 0, step 410: average elbo=-195.294220\n",
      "Epoch 0, step 420: average elbo=-200.652267\n",
      "Epoch 0, step 430: average elbo=-208.812149\n",
      "Epoch 0, step 440: average elbo=-200.045319\n",
      "Epoch 0, step 450: average elbo=-199.878784\n",
      "Epoch 0, step 460: average elbo=-206.345490\n",
      "Epoch 0, step 470: average elbo=-207.140121\n",
      "Epoch 0, step 480: average elbo=-204.130676\n",
      "Epoch 0, step 490: average elbo=-203.469315\n",
      "Epoch 0, step 500: average elbo=-205.958557\n",
      "Epoch 0, step 510: average elbo=-203.972565\n",
      "Epoch 0, step 520: average elbo=-201.650574\n",
      "Epoch 0, step 530: average elbo=-204.986053\n",
      "Epoch 0, step 540: average elbo=-198.621933\n",
      "Epoch 1, step 0: average elbo=-202.771149\n",
      "Epoch 1, step 10: average elbo=-201.271362\n",
      "Epoch 1, step 20: average elbo=-195.510757\n",
      "Epoch 1, step 30: average elbo=-209.701096\n",
      "Epoch 1, step 40: average elbo=-198.979568\n",
      "Epoch 1, step 50: average elbo=-198.095337\n",
      "Epoch 1, step 60: average elbo=-210.112228\n",
      "Epoch 1, step 70: average elbo=-201.188522\n",
      "Epoch 1, step 80: average elbo=-205.822937\n",
      "Epoch 1, step 90: average elbo=-198.407654\n",
      "Epoch 1, step 100: average elbo=-193.962448\n",
      "Epoch 1, step 110: average elbo=-194.385345\n",
      "Epoch 1, step 120: average elbo=-204.923477\n",
      "Epoch 1, step 130: average elbo=-208.576111\n",
      "Epoch 1, step 140: average elbo=-204.448181\n",
      "Epoch 1, step 150: average elbo=-204.872742\n",
      "Epoch 1, step 160: average elbo=-200.421188\n",
      "Epoch 1, step 170: average elbo=-198.152557\n",
      "Epoch 1, step 180: average elbo=-200.047562\n",
      "Epoch 1, step 190: average elbo=-206.334961\n",
      "Epoch 1, step 200: average elbo=-202.665192\n",
      "Epoch 1, step 210: average elbo=-204.729080\n",
      "Epoch 1, step 220: average elbo=-197.487152\n",
      "Epoch 1, step 230: average elbo=-196.357513\n",
      "Epoch 1, step 240: average elbo=-207.793549\n",
      "Epoch 1, step 250: average elbo=-209.511246\n",
      "Epoch 1, step 260: average elbo=-202.061935\n",
      "Epoch 1, step 270: average elbo=-197.077194\n",
      "Epoch 1, step 280: average elbo=-195.298264\n",
      "Epoch 1, step 290: average elbo=-194.038712\n",
      "Epoch 1, step 300: average elbo=-199.171677\n",
      "Epoch 1, step 310: average elbo=-188.342758\n",
      "Epoch 1, step 320: average elbo=-203.318146\n",
      "Epoch 1, step 330: average elbo=-196.126175\n",
      "Epoch 1, step 340: average elbo=-191.844864\n",
      "Epoch 1, step 350: average elbo=-193.601959\n",
      "Epoch 1, step 360: average elbo=-201.827423\n",
      "Epoch 1, step 370: average elbo=-188.135818\n",
      "Epoch 1, step 380: average elbo=-197.268051\n",
      "Epoch 1, step 390: average elbo=-192.352890\n",
      "Epoch 1, step 400: average elbo=-202.524338\n",
      "Epoch 1, step 410: average elbo=-195.991409\n",
      "Epoch 1, step 420: average elbo=-198.012756\n",
      "Epoch 1, step 430: average elbo=-197.807190\n",
      "Epoch 1, step 440: average elbo=-203.761795\n",
      "Epoch 1, step 450: average elbo=-195.336060\n",
      "Epoch 1, step 460: average elbo=-195.639923\n",
      "Epoch 1, step 470: average elbo=-197.546234\n",
      "Epoch 1, step 480: average elbo=-194.885941\n",
      "Epoch 1, step 490: average elbo=-186.946213\n",
      "Epoch 1, step 500: average elbo=-200.764420\n",
      "Epoch 1, step 510: average elbo=-201.621597\n",
      "Epoch 1, step 520: average elbo=-197.619278\n",
      "Epoch 1, step 530: average elbo=-190.513000\n",
      "Epoch 1, step 540: average elbo=-202.596405\n",
      "Epoch 2, step 0: average elbo=-196.629211\n",
      "Epoch 2, step 10: average elbo=-201.397049\n",
      "Epoch 2, step 20: average elbo=-196.755905\n",
      "Epoch 2, step 30: average elbo=-190.376953\n",
      "Epoch 2, step 40: average elbo=-191.373810\n",
      "Epoch 2, step 50: average elbo=-197.041595\n",
      "Epoch 2, step 60: average elbo=-196.976074\n",
      "Epoch 2, step 70: average elbo=-196.638824\n",
      "Epoch 2, step 80: average elbo=-193.106171\n",
      "Epoch 2, step 90: average elbo=-194.680786\n",
      "Epoch 2, step 100: average elbo=-188.465073\n",
      "Epoch 2, step 110: average elbo=-194.780670\n",
      "Epoch 2, step 120: average elbo=-191.621445\n",
      "Epoch 2, step 130: average elbo=-188.459686\n",
      "Epoch 2, step 140: average elbo=-205.755707\n",
      "Epoch 2, step 150: average elbo=-189.846405\n",
      "Epoch 2, step 160: average elbo=-200.592209\n",
      "Epoch 2, step 170: average elbo=-195.979645\n",
      "Epoch 2, step 180: average elbo=-187.060059\n",
      "Epoch 2, step 190: average elbo=-190.503784\n",
      "Epoch 2, step 200: average elbo=-188.892029\n",
      "Epoch 2, step 210: average elbo=-194.118698\n",
      "Epoch 2, step 220: average elbo=-193.117111\n",
      "Epoch 2, step 230: average elbo=-188.780701\n",
      "Epoch 2, step 240: average elbo=-195.566757\n",
      "Epoch 2, step 250: average elbo=-188.729858\n",
      "Epoch 2, step 260: average elbo=-197.187729\n",
      "Epoch 2, step 270: average elbo=-188.502106\n",
      "Epoch 2, step 280: average elbo=-194.004059\n",
      "Epoch 2, step 290: average elbo=-191.728455\n",
      "Epoch 2, step 300: average elbo=-195.860443\n",
      "Epoch 2, step 310: average elbo=-197.077072\n",
      "Epoch 2, step 320: average elbo=-188.211090\n",
      "Epoch 2, step 330: average elbo=-184.821442\n",
      "Epoch 2, step 340: average elbo=-184.030563\n",
      "Epoch 2, step 350: average elbo=-194.977005\n",
      "Epoch 2, step 360: average elbo=-202.008591\n",
      "Epoch 2, step 370: average elbo=-191.488541\n",
      "Epoch 2, step 380: average elbo=-188.002853\n",
      "Epoch 2, step 390: average elbo=-193.981567\n",
      "Epoch 2, step 400: average elbo=-184.338333\n",
      "Epoch 2, step 410: average elbo=-184.117584\n",
      "Epoch 2, step 420: average elbo=-190.043304\n",
      "Epoch 2, step 430: average elbo=-192.296600\n",
      "Epoch 2, step 440: average elbo=-189.440826\n",
      "Epoch 2, step 450: average elbo=-194.788101\n",
      "Epoch 2, step 460: average elbo=-193.212387\n",
      "Epoch 2, step 470: average elbo=-189.541321\n",
      "Epoch 2, step 480: average elbo=-189.744705\n",
      "Epoch 2, step 490: average elbo=-181.076370\n",
      "Epoch 2, step 500: average elbo=-192.771469\n",
      "Epoch 2, step 510: average elbo=-191.602188\n",
      "Epoch 2, step 520: average elbo=-189.750305\n",
      "Epoch 2, step 530: average elbo=-195.219528\n",
      "Epoch 2, step 540: average elbo=-186.222946\n",
      "Epoch 3, step 0: average elbo=-199.638092\n",
      "Epoch 3, step 10: average elbo=-188.312546\n",
      "Epoch 3, step 20: average elbo=-192.338470\n",
      "Epoch 3, step 30: average elbo=-185.960022\n",
      "Epoch 3, step 40: average elbo=-192.097534\n",
      "Epoch 3, step 50: average elbo=-192.343826\n",
      "Epoch 3, step 60: average elbo=-183.228088\n",
      "Epoch 3, step 70: average elbo=-200.525116\n",
      "Epoch 3, step 80: average elbo=-183.065903\n",
      "Epoch 3, step 90: average elbo=-192.388733\n",
      "Epoch 3, step 100: average elbo=-197.787506\n",
      "Epoch 3, step 110: average elbo=-183.872620\n",
      "Epoch 3, step 120: average elbo=-183.707382\n",
      "Epoch 3, step 130: average elbo=-186.933151\n",
      "Epoch 3, step 140: average elbo=-184.939850\n",
      "Epoch 3, step 150: average elbo=-190.825287\n",
      "Epoch 3, step 160: average elbo=-184.756760\n",
      "Epoch 3, step 170: average elbo=-188.678238\n",
      "Epoch 3, step 180: average elbo=-191.503235\n",
      "Epoch 3, step 190: average elbo=-178.095306\n",
      "Epoch 3, step 200: average elbo=-183.801208\n",
      "Epoch 3, step 210: average elbo=-179.513519\n",
      "Epoch 3, step 220: average elbo=-177.637024\n",
      "Epoch 3, step 230: average elbo=-186.102478\n",
      "Epoch 3, step 240: average elbo=-187.571564\n",
      "Epoch 3, step 250: average elbo=-183.121918\n",
      "Epoch 3, step 260: average elbo=-185.857697\n",
      "Epoch 3, step 270: average elbo=-172.899399\n",
      "Epoch 3, step 280: average elbo=-172.112228\n",
      "Epoch 3, step 290: average elbo=-179.014175\n",
      "Epoch 3, step 300: average elbo=-180.813889\n",
      "Epoch 3, step 310: average elbo=-173.349045\n",
      "Epoch 3, step 320: average elbo=-182.294907\n",
      "Epoch 3, step 330: average elbo=-180.487030\n",
      "Epoch 3, step 340: average elbo=-176.551819\n",
      "Epoch 3, step 350: average elbo=-177.762909\n",
      "Epoch 3, step 360: average elbo=-174.468552\n",
      "Epoch 3, step 370: average elbo=-177.856094\n",
      "Epoch 3, step 380: average elbo=-175.555847\n",
      "Epoch 3, step 390: average elbo=-182.225510\n",
      "Epoch 3, step 400: average elbo=-173.955200\n",
      "Epoch 3, step 410: average elbo=-178.312073\n",
      "Epoch 3, step 420: average elbo=-169.398071\n",
      "Epoch 3, step 430: average elbo=-173.134094\n",
      "Epoch 3, step 440: average elbo=-181.403122\n",
      "Epoch 3, step 450: average elbo=-184.885941\n",
      "Epoch 3, step 460: average elbo=-173.984009\n",
      "Epoch 3, step 470: average elbo=-175.292831\n",
      "Epoch 3, step 480: average elbo=-171.830017\n",
      "Epoch 3, step 490: average elbo=-168.667892\n",
      "Epoch 3, step 500: average elbo=-179.198853\n",
      "Epoch 3, step 510: average elbo=-175.162155\n",
      "Epoch 3, step 520: average elbo=-183.335052\n",
      "Epoch 3, step 530: average elbo=-174.732819\n",
      "Epoch 3, step 540: average elbo=-172.761948\n",
      "Epoch 4, step 0: average elbo=-169.987076\n",
      "Epoch 4, step 10: average elbo=-179.686371\n",
      "Epoch 4, step 20: average elbo=-181.981567\n",
      "Epoch 4, step 30: average elbo=-170.259186\n",
      "Epoch 4, step 40: average elbo=-175.775681\n",
      "Epoch 4, step 50: average elbo=-167.100113\n",
      "Epoch 4, step 60: average elbo=-173.629333\n",
      "Epoch 4, step 70: average elbo=-172.521881\n",
      "Epoch 4, step 80: average elbo=-165.267502\n",
      "Epoch 4, step 90: average elbo=-171.267242\n",
      "Epoch 4, step 100: average elbo=-174.288712\n",
      "Epoch 4, step 110: average elbo=-166.835739\n",
      "Epoch 4, step 120: average elbo=-166.294037\n",
      "Epoch 4, step 130: average elbo=-169.909088\n",
      "Epoch 4, step 140: average elbo=-177.945663\n",
      "Epoch 4, step 150: average elbo=-176.419724\n",
      "Epoch 4, step 160: average elbo=-167.970230\n",
      "Epoch 4, step 170: average elbo=-175.322250\n",
      "Epoch 4, step 180: average elbo=-174.118790\n",
      "Epoch 4, step 190: average elbo=-182.828949\n",
      "Epoch 4, step 200: average elbo=-165.007339\n",
      "Epoch 4, step 210: average elbo=-178.020233\n",
      "Epoch 4, step 220: average elbo=-170.333313\n",
      "Epoch 4, step 230: average elbo=-170.001465\n",
      "Epoch 4, step 240: average elbo=-171.660828\n",
      "Epoch 4, step 250: average elbo=-165.533676\n",
      "Epoch 4, step 260: average elbo=-175.753433\n",
      "Epoch 4, step 270: average elbo=-159.924728\n",
      "Epoch 4, step 280: average elbo=-167.253693\n",
      "Epoch 4, step 290: average elbo=-168.621872\n",
      "Epoch 4, step 300: average elbo=-166.031387\n",
      "Epoch 4, step 310: average elbo=-166.500565\n",
      "Epoch 4, step 320: average elbo=-167.965042\n",
      "Epoch 4, step 330: average elbo=-168.473206\n",
      "Epoch 4, step 340: average elbo=-169.340530\n",
      "Epoch 4, step 350: average elbo=-169.905426\n",
      "Epoch 4, step 360: average elbo=-169.395905\n",
      "Epoch 4, step 370: average elbo=-172.168900\n",
      "Epoch 4, step 380: average elbo=-164.849762\n",
      "Epoch 4, step 390: average elbo=-166.733353\n",
      "Epoch 4, step 400: average elbo=-166.215134\n",
      "Epoch 4, step 410: average elbo=-162.192719\n",
      "Epoch 4, step 420: average elbo=-162.633209\n",
      "Epoch 4, step 430: average elbo=-164.450668\n",
      "Epoch 4, step 440: average elbo=-171.088303\n",
      "Epoch 4, step 450: average elbo=-153.585678\n",
      "Epoch 4, step 460: average elbo=-174.374100\n",
      "Epoch 4, step 470: average elbo=-167.306763\n",
      "Epoch 4, step 480: average elbo=-164.465836\n",
      "Epoch 4, step 490: average elbo=-164.340775\n",
      "Epoch 4, step 500: average elbo=-166.516174\n",
      "Epoch 4, step 510: average elbo=-165.074493\n",
      "Epoch 4, step 520: average elbo=-161.835327\n",
      "Epoch 4, step 530: average elbo=-159.028549\n",
      "Epoch 4, step 540: average elbo=-158.972260\n",
      "Epoch 5, step 0: average elbo=-161.663269\n",
      "Epoch 5, step 10: average elbo=-155.588303\n",
      "Epoch 5, step 20: average elbo=-158.082321\n",
      "Epoch 5, step 30: average elbo=-155.623718\n",
      "Epoch 5, step 40: average elbo=-156.327423\n",
      "Epoch 5, step 50: average elbo=-171.485825\n",
      "Epoch 5, step 60: average elbo=-161.134216\n",
      "Epoch 5, step 70: average elbo=-161.478607\n",
      "Epoch 5, step 80: average elbo=-154.923416\n",
      "Epoch 5, step 90: average elbo=-166.341919\n",
      "Epoch 5, step 100: average elbo=-161.477417\n",
      "Epoch 5, step 110: average elbo=-157.946991\n",
      "Epoch 5, step 120: average elbo=-164.460999\n",
      "Epoch 5, step 130: average elbo=-156.922211\n",
      "Epoch 5, step 140: average elbo=-155.834213\n",
      "Epoch 5, step 150: average elbo=-159.178268\n",
      "Epoch 5, step 160: average elbo=-163.428238\n",
      "Epoch 5, step 170: average elbo=-157.463638\n",
      "Epoch 5, step 180: average elbo=-154.159225\n",
      "Epoch 5, step 190: average elbo=-153.533615\n",
      "Epoch 5, step 200: average elbo=-154.449234\n",
      "Epoch 5, step 210: average elbo=-154.205353\n",
      "Epoch 5, step 220: average elbo=-157.540344\n",
      "Epoch 5, step 230: average elbo=-151.772644\n",
      "Epoch 5, step 240: average elbo=-155.326782\n",
      "Epoch 5, step 250: average elbo=-168.326431\n",
      "Epoch 5, step 260: average elbo=-158.993423\n",
      "Epoch 5, step 270: average elbo=-155.198044\n",
      "Epoch 5, step 280: average elbo=-160.985458\n",
      "Epoch 5, step 290: average elbo=-153.521072\n",
      "Epoch 5, step 300: average elbo=-149.906235\n",
      "Epoch 5, step 310: average elbo=-154.694489\n",
      "Epoch 5, step 320: average elbo=-147.911850\n",
      "Epoch 5, step 330: average elbo=-157.380417\n",
      "Epoch 5, step 340: average elbo=-154.107620\n",
      "Epoch 5, step 350: average elbo=-151.641586\n",
      "Epoch 5, step 360: average elbo=-152.034393\n",
      "Epoch 5, step 370: average elbo=-162.538422\n",
      "Epoch 5, step 380: average elbo=-150.565140\n",
      "Epoch 5, step 390: average elbo=-155.165543\n",
      "Epoch 5, step 400: average elbo=-156.571915\n",
      "Epoch 5, step 410: average elbo=-161.094864\n",
      "Epoch 5, step 420: average elbo=-170.060822\n",
      "Epoch 5, step 430: average elbo=-151.670639\n",
      "Epoch 5, step 440: average elbo=-155.040726\n",
      "Epoch 5, step 450: average elbo=-157.455261\n",
      "Epoch 5, step 460: average elbo=-154.977341\n",
      "Epoch 5, step 470: average elbo=-160.902618\n",
      "Epoch 5, step 480: average elbo=-153.935730\n",
      "Epoch 5, step 490: average elbo=-153.944031\n",
      "Epoch 5, step 500: average elbo=-149.662338\n",
      "Epoch 5, step 510: average elbo=-157.111954\n",
      "Epoch 5, step 520: average elbo=-162.256409\n",
      "Epoch 5, step 530: average elbo=-150.760132\n",
      "Epoch 5, step 540: average elbo=-152.029526\n",
      "Epoch 6, step 0: average elbo=-155.732666\n",
      "Epoch 6, step 10: average elbo=-156.560745\n",
      "Epoch 6, step 20: average elbo=-152.059937\n",
      "Epoch 6, step 30: average elbo=-152.967682\n",
      "Epoch 6, step 40: average elbo=-158.282028\n",
      "Epoch 6, step 50: average elbo=-145.727631\n",
      "Epoch 6, step 60: average elbo=-152.622086\n",
      "Epoch 6, step 70: average elbo=-149.752701\n",
      "Epoch 6, step 80: average elbo=-152.749710\n",
      "Epoch 6, step 90: average elbo=-153.806732\n",
      "Epoch 6, step 100: average elbo=-147.049759\n",
      "Epoch 6, step 110: average elbo=-150.624130\n",
      "Epoch 6, step 120: average elbo=-152.360565\n",
      "Epoch 6, step 130: average elbo=-149.232956\n",
      "Epoch 6, step 140: average elbo=-162.980728\n",
      "Epoch 6, step 150: average elbo=-154.330383\n",
      "Epoch 6, step 160: average elbo=-155.444077\n",
      "Epoch 6, step 170: average elbo=-150.516769\n",
      "Epoch 6, step 180: average elbo=-151.843826\n",
      "Epoch 6, step 190: average elbo=-148.542816\n",
      "Epoch 6, step 200: average elbo=-151.447815\n",
      "Epoch 6, step 210: average elbo=-155.992691\n",
      "Epoch 6, step 220: average elbo=-151.121979\n",
      "Epoch 6, step 230: average elbo=-149.608627\n",
      "Epoch 6, step 240: average elbo=-152.166275\n",
      "Epoch 6, step 250: average elbo=-153.272659\n",
      "Epoch 6, step 260: average elbo=-148.642929\n",
      "Epoch 6, step 270: average elbo=-147.143890\n",
      "Epoch 6, step 280: average elbo=-155.657715\n",
      "Epoch 6, step 290: average elbo=-139.734222\n",
      "Epoch 6, step 300: average elbo=-148.644791\n",
      "Epoch 6, step 310: average elbo=-142.534683\n",
      "Epoch 6, step 320: average elbo=-156.466125\n",
      "Epoch 6, step 330: average elbo=-150.992905\n",
      "Epoch 6, step 340: average elbo=-157.106628\n",
      "Epoch 6, step 350: average elbo=-153.045273\n",
      "Epoch 6, step 360: average elbo=-150.769318\n",
      "Epoch 6, step 370: average elbo=-147.912750\n",
      "Epoch 6, step 380: average elbo=-151.458710\n",
      "Epoch 6, step 390: average elbo=-152.496918\n",
      "Epoch 6, step 400: average elbo=-152.984879\n",
      "Epoch 6, step 410: average elbo=-151.655121\n",
      "Epoch 6, step 420: average elbo=-151.046295\n",
      "Epoch 6, step 430: average elbo=-144.511719\n",
      "Epoch 6, step 440: average elbo=-152.132370\n",
      "Epoch 6, step 450: average elbo=-149.324203\n",
      "Epoch 6, step 460: average elbo=-145.197098\n",
      "Epoch 6, step 470: average elbo=-151.296692\n",
      "Epoch 6, step 480: average elbo=-158.355743\n",
      "Epoch 6, step 490: average elbo=-149.179382\n",
      "Epoch 6, step 500: average elbo=-158.373520\n",
      "Epoch 6, step 510: average elbo=-153.022156\n",
      "Epoch 6, step 520: average elbo=-152.593201\n",
      "Epoch 6, step 530: average elbo=-153.738419\n",
      "Epoch 6, step 540: average elbo=-146.962357\n",
      "Epoch 7, step 0: average elbo=-151.509415\n",
      "Epoch 7, step 10: average elbo=-149.951126\n",
      "Epoch 7, step 20: average elbo=-160.073105\n",
      "Epoch 7, step 30: average elbo=-152.959381\n",
      "Epoch 7, step 40: average elbo=-149.960999\n",
      "Epoch 7, step 50: average elbo=-152.841812\n",
      "Epoch 7, step 60: average elbo=-143.995834\n",
      "Epoch 7, step 70: average elbo=-148.298065\n",
      "Epoch 7, step 80: average elbo=-161.453522\n",
      "Epoch 7, step 90: average elbo=-144.098984\n",
      "Epoch 7, step 100: average elbo=-148.892197\n",
      "Epoch 7, step 110: average elbo=-152.635895\n",
      "Epoch 7, step 120: average elbo=-150.777634\n",
      "Epoch 7, step 130: average elbo=-146.769760\n",
      "Epoch 7, step 140: average elbo=-150.729996\n",
      "Epoch 7, step 150: average elbo=-154.418198\n",
      "Epoch 7, step 160: average elbo=-140.354202\n",
      "Epoch 7, step 170: average elbo=-145.693726\n",
      "Epoch 7, step 180: average elbo=-149.687698\n",
      "Epoch 7, step 190: average elbo=-143.482346\n",
      "Epoch 7, step 200: average elbo=-149.290619\n",
      "Epoch 7, step 210: average elbo=-150.292618\n",
      "Epoch 7, step 220: average elbo=-145.138580\n",
      "Epoch 7, step 230: average elbo=-137.252075\n",
      "Epoch 7, step 240: average elbo=-148.015137\n",
      "Epoch 7, step 250: average elbo=-147.578522\n",
      "Epoch 7, step 260: average elbo=-142.746353\n",
      "Epoch 7, step 270: average elbo=-146.899185\n",
      "Epoch 7, step 280: average elbo=-149.724579\n",
      "Epoch 7, step 290: average elbo=-147.446030\n",
      "Epoch 7, step 300: average elbo=-148.224670\n",
      "Epoch 7, step 310: average elbo=-146.019089\n",
      "Epoch 7, step 320: average elbo=-151.039597\n",
      "Epoch 7, step 330: average elbo=-143.525162\n",
      "Epoch 7, step 340: average elbo=-149.361511\n",
      "Epoch 7, step 350: average elbo=-149.946106\n",
      "Epoch 7, step 360: average elbo=-145.240402\n",
      "Epoch 7, step 370: average elbo=-138.103790\n",
      "Epoch 7, step 380: average elbo=-141.549301\n",
      "Epoch 7, step 390: average elbo=-142.104797\n",
      "Epoch 7, step 400: average elbo=-149.644714\n",
      "Epoch 7, step 410: average elbo=-142.766296\n",
      "Epoch 7, step 420: average elbo=-151.002380\n",
      "Epoch 7, step 430: average elbo=-145.616943\n",
      "Epoch 7, step 440: average elbo=-135.332092\n",
      "Epoch 7, step 450: average elbo=-144.044830\n",
      "Epoch 7, step 460: average elbo=-141.058121\n",
      "Epoch 7, step 470: average elbo=-148.847992\n",
      "Epoch 7, step 480: average elbo=-137.432938\n",
      "Epoch 7, step 490: average elbo=-144.435593\n",
      "Epoch 7, step 500: average elbo=-149.157562\n",
      "Epoch 7, step 510: average elbo=-143.406174\n",
      "Epoch 7, step 520: average elbo=-153.542984\n",
      "Epoch 7, step 530: average elbo=-149.728409\n",
      "Epoch 7, step 540: average elbo=-148.322662\n",
      "Epoch 8, step 0: average elbo=-152.375198\n",
      "Epoch 8, step 10: average elbo=-140.552338\n",
      "Epoch 8, step 20: average elbo=-144.333649\n",
      "Epoch 8, step 30: average elbo=-143.684143\n",
      "Epoch 8, step 40: average elbo=-143.386276\n",
      "Epoch 8, step 50: average elbo=-140.615875\n",
      "Epoch 8, step 60: average elbo=-147.880035\n",
      "Epoch 8, step 70: average elbo=-151.309311\n",
      "Epoch 8, step 80: average elbo=-147.549103\n",
      "Epoch 8, step 90: average elbo=-147.350769\n",
      "Epoch 8, step 100: average elbo=-146.331955\n",
      "Epoch 8, step 110: average elbo=-151.270279\n",
      "Epoch 8, step 120: average elbo=-146.124786\n",
      "Epoch 8, step 130: average elbo=-144.839905\n",
      "Epoch 8, step 140: average elbo=-149.929428\n",
      "Epoch 8, step 150: average elbo=-149.361938\n",
      "Epoch 8, step 160: average elbo=-142.470749\n",
      "Epoch 8, step 170: average elbo=-142.187073\n",
      "Epoch 8, step 180: average elbo=-140.513519\n",
      "Epoch 8, step 190: average elbo=-144.875626\n",
      "Epoch 8, step 200: average elbo=-149.019836\n",
      "Epoch 8, step 210: average elbo=-140.572418\n",
      "Epoch 8, step 220: average elbo=-143.931229\n",
      "Epoch 8, step 230: average elbo=-150.187775\n",
      "Epoch 8, step 240: average elbo=-146.801804\n",
      "Epoch 8, step 250: average elbo=-142.372375\n",
      "Epoch 8, step 260: average elbo=-152.931595\n",
      "Epoch 8, step 270: average elbo=-135.314545\n",
      "Epoch 8, step 280: average elbo=-148.277603\n",
      "Epoch 8, step 290: average elbo=-142.926910\n",
      "Epoch 8, step 300: average elbo=-144.686462\n",
      "Epoch 8, step 310: average elbo=-149.680130\n",
      "Epoch 8, step 320: average elbo=-146.330063\n",
      "Epoch 8, step 330: average elbo=-151.952103\n",
      "Epoch 8, step 340: average elbo=-151.465546\n",
      "Epoch 8, step 350: average elbo=-149.537109\n",
      "Epoch 8, step 360: average elbo=-140.898880\n",
      "Epoch 8, step 370: average elbo=-147.172318\n",
      "Epoch 8, step 380: average elbo=-138.434204\n",
      "Epoch 8, step 390: average elbo=-146.433090\n",
      "Epoch 8, step 400: average elbo=-150.392761\n",
      "Epoch 8, step 410: average elbo=-140.727234\n",
      "Epoch 8, step 420: average elbo=-141.578613\n",
      "Epoch 8, step 430: average elbo=-148.095291\n",
      "Epoch 8, step 440: average elbo=-145.144806\n",
      "Epoch 8, step 450: average elbo=-143.880737\n",
      "Epoch 8, step 460: average elbo=-137.393417\n",
      "Epoch 8, step 470: average elbo=-140.097153\n",
      "Epoch 8, step 480: average elbo=-146.022690\n",
      "Epoch 8, step 490: average elbo=-144.845795\n",
      "Epoch 8, step 500: average elbo=-143.871811\n",
      "Epoch 8, step 510: average elbo=-149.038940\n",
      "Epoch 8, step 520: average elbo=-139.470383\n",
      "Epoch 8, step 530: average elbo=-148.597366\n",
      "Epoch 8, step 540: average elbo=-144.198151\n",
      "Epoch 9, step 0: average elbo=-140.357361\n",
      "Epoch 9, step 10: average elbo=-141.146988\n",
      "Epoch 9, step 20: average elbo=-141.967148\n",
      "Epoch 9, step 30: average elbo=-146.998611\n",
      "Epoch 9, step 40: average elbo=-144.164032\n",
      "Epoch 9, step 50: average elbo=-148.779556\n",
      "Epoch 9, step 60: average elbo=-151.063293\n",
      "Epoch 9, step 70: average elbo=-137.717789\n",
      "Epoch 9, step 80: average elbo=-152.716034\n",
      "Epoch 9, step 90: average elbo=-144.909103\n",
      "Epoch 9, step 100: average elbo=-143.754822\n",
      "Epoch 9, step 110: average elbo=-140.322479\n",
      "Epoch 9, step 120: average elbo=-134.213715\n",
      "Epoch 9, step 130: average elbo=-147.544861\n",
      "Epoch 9, step 140: average elbo=-143.714844\n",
      "Epoch 9, step 150: average elbo=-138.135941\n",
      "Epoch 9, step 160: average elbo=-153.722473\n",
      "Epoch 9, step 170: average elbo=-150.919556\n",
      "Epoch 9, step 180: average elbo=-140.218307\n",
      "Epoch 9, step 190: average elbo=-142.421478\n",
      "Epoch 9, step 200: average elbo=-134.036087\n",
      "Epoch 9, step 210: average elbo=-149.913147\n",
      "Epoch 9, step 220: average elbo=-145.744049\n",
      "Epoch 9, step 230: average elbo=-136.944687\n",
      "Epoch 9, step 240: average elbo=-137.228867\n",
      "Epoch 9, step 250: average elbo=-135.546371\n",
      "Epoch 9, step 260: average elbo=-141.177231\n",
      "Epoch 9, step 270: average elbo=-142.965271\n",
      "Epoch 9, step 280: average elbo=-138.939972\n",
      "Epoch 9, step 290: average elbo=-136.309570\n",
      "Epoch 9, step 300: average elbo=-138.670303\n",
      "Epoch 9, step 310: average elbo=-148.282288\n",
      "Epoch 9, step 320: average elbo=-131.374817\n",
      "Epoch 9, step 330: average elbo=-144.719391\n",
      "Epoch 9, step 340: average elbo=-141.071304\n",
      "Epoch 9, step 350: average elbo=-144.354645\n",
      "Epoch 9, step 360: average elbo=-144.292847\n",
      "Epoch 9, step 370: average elbo=-139.482513\n",
      "Epoch 9, step 380: average elbo=-143.472778\n",
      "Epoch 9, step 390: average elbo=-146.002213\n",
      "Epoch 9, step 400: average elbo=-144.582214\n",
      "Epoch 9, step 410: average elbo=-148.937836\n",
      "Epoch 9, step 420: average elbo=-136.593323\n",
      "Epoch 9, step 430: average elbo=-146.098541\n",
      "Epoch 9, step 440: average elbo=-137.195114\n",
      "Epoch 9, step 450: average elbo=-150.506149\n",
      "Epoch 9, step 460: average elbo=-136.688766\n",
      "Epoch 9, step 470: average elbo=-140.248764\n",
      "Epoch 9, step 480: average elbo=-130.937668\n",
      "Epoch 9, step 490: average elbo=-141.994659\n",
      "Epoch 9, step 500: average elbo=-145.443604\n",
      "Epoch 9, step 510: average elbo=-140.903854\n",
      "Epoch 9, step 520: average elbo=-138.184250\n",
      "Epoch 9, step 530: average elbo=-141.018204\n",
      "Epoch 9, step 540: average elbo=-134.956665\n"
     ]
    }
   ],
   "source": [
    "sess = tf.Session(graph=vae_graph)\n",
    "sess.run(init)\n",
    "\n",
    "elbo_log = np.zeros(n_epochs * train_steps)\n",
    "\n",
    "for i in range(n_epochs):\n",
    "    for j in range(train_steps):\n",
    "        batch_xs, batch_ys = mnist.train.next_batch(minibatch_size)\n",
    "        sess.run(minimize_op, feed_dict={x: batch_xs})\n",
    "        elbo_log[i*train_steps + j] = elbo.eval(session=sess, feed_dict={x: batch_xs})\n",
    "        if j % 10 == 0:\n",
    "            print(\"Epoch %i, step %i: average elbo=%f\" % (i, j, elbo_log[i*train_steps + j]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[<matplotlib.lines.Line2D at 0x12b9ade48>]"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYMAAAD8CAYAAACVZ8iyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xd8leX9//HXJ5O994gJCMpShIgoKqBUUNri/qFt1epX\nyhet9qfWhXXbUq22Wie11lHr+ImKA1FQVByMoOwhK0BiKGGGlX39/jh3DichyR04CSF33s/H4zy8\nz3WPc10I53Oubc45RESkfoup7QyIiEjtUzAQEREFAxERUTAQEREUDEREBAUDERFBwUBERFAwEBER\nFAxERASIq+0MVFWbNm1ccnJybWdDRKROWbBgwVbnXFu/6+pMMEhOTiYtLa22syEiUqeY2YaqXKdm\nIhERUTAQEREFAxERQcFARERQMBARERQMREQEBQMREUHBQESkVi37cRffb9xR29lQMBARqciOvfms\n3JxTY8/PyS1g9BNfccHT39TYZ1SVgoGIHFWe/WItG7ftO+z79+QVsju3oFryMvqJ2Yz62+wqXZu1\na7/vNXvyCpm9Ops35m8EYMPW0uUsLnZ8tXorzjkApi/N4jevpJFXWHSIOT90dWY5ChEJvi27c5n0\n0UremL+JWbcMK/eazJ37iTHo2LxhuecHPDCD/MJi0ieN5l9fr2fBhh08efkA38/O2LGPBvGxtGmS\nGE77cVdulfL9zdqtXP6PubRoFM87E4aQ0qZx+Ny89dtpnBhLo4Q4hv/l83D6bVOWlHrGnrxCHvxg\nOa/P38TPT+zEE5edxNrsvXy87L9VykO0FAxE5OgR+kHMnrzCck/v3JfPkEmfAZA+aXQ4fV9+Ia/N\n28SvT0smv7A4nH7f+8sBePLy0s+ZvTqbWSuzGT+sG1ty8ujbuTmn/3lWqecWFh14zva9+bRqnMDz\ns9cx4JiW7M8v4hfPz2XmTUOZ8l0GDeNjvfwVMPwvn/P42P6M6d+ZS5/9lnnp2wG4+Sc9Ky1633s+\nDh+/t+hHFmzYQebOUG0j1qzSe6uDgoGI1LqMHfvYuG0fn63cAkD27jyen72Oj5dt5k8XnsCx7ZoA\nsHVPfviex2b8wM9O6IgZ3PTmIhZn7KJrywO1helLN5c6Hv/vBXx123DaN2vAr/45D4AXvl5/UF7S\nt+5lWMQveAjVNhbdcw4PfrgCgNO6twZgxGNflFueG19fyM1vLqKw2IXTHp3xQ5X/PIBwIACIjan5\nYGAlbVNHu9TUVKdVS0VqV15hEfExMcTEGCuycmgQH0vH5g1o4P0yjnTve8sYfnw7zji2Dd3unMYt\n5/Tk+rN6HHRdbkERx/9heqWfe8/PevPrISm8t+hHbnjt+wqv69+1BQs37TwofVSfDkxftpknLz+J\np2atZUVWzXUK14TIWtChMrMFzrlUv+tUMxCph9K37uWal+bzyjWn0KlF6bb35T/m0CQxjvyiIo5t\n17TUuePums6Y/p24a3Rvzn38QMfqpAv7MSilFZ1aNGR5Vg679hfw4jfpvPhNOovuPgcI/ZK//qwe\n7MsvZNXm3bRr1oAbXvueBRv8h1Xe9/5ylv2Yw1sLMiq9rrxAADB9WaiW8NnKLXUuEBwpCgYidcSX\nP2TTu1OzUh2cZfW+ezo92zfl3euGkJNbwHcbdpDSpjEFRS7c1PLWggxu+X+LALj6xfm8Me5Utu/L\nD3d6nvdE6dEzC+/+CS0aJfDh4iwApi78kakLfyx1zT+/Ws/tb5fuEC3x15mh5pFiB5u27+OMh2cd\nRunxDQRV8fZ3mVE/I6jUTCRyFPvta9/z/qIfWffH8+h25zR6tGvCjJuGlrpm3vrtbNi2l9+/tTic\ndv3wY3ly1ppS1618YBQN4mNJvv3Dcj9r/Z/Oo6jYcezEj6q/IBKVI9FMFNU8AzO7xMyWmVmxmaVG\npCeb2X4zW+i9no04N9DMlpjZGjN7wuwIdJOL1AFLM3fR/c5pbI4Yzvj+otAv8CLvR9vqLXtKjZZx\nznHpc9+WCgTAQYEA4Pg/TOexT1ZV+Pkpd0xTIKjHom0mWgpcCDxXzrm1zrn+5aQ/A1wLzAWmAaMA\n/Q2UQNi1v4Afd+6nV8dmVb4nJ7eA6179jhgzioodg//0Kd3aNuaGiM7WP3+0Mnzc866POOv4dtw1\nuhdnPVr+aJaKPPHZwUFCBKKsGTjnVjjnKv6pUYaZdQSaOefmuFD71MvA+dHkQeRImLYkiz53Tye3\n4MBMUOccD36wnDRvHDnA2MlzOPfx2eQWFDF9aRa9757Orv0Hz4bdn1/EnrxCkm//kBPu/YTZq7fy\nxQ/Z4fPrsvfyuzcWht8//1XpIZCfrdxyyIFA6qbZtw4/Ip9Tkx3IKWa2ENgF3OWcmw10BiJ7gTK8\ntHKZ2ThgHEBSUlINZlWkcn+ctoK9+UVsyckjqXUjPlm2maxduTz/1frwF/XbE04Lj1QZ8dgXZOwI\njRMfO3kOvz4tmRe/SefV/zmFG17/ntmrt9ZaWeTIG9mnPb8Z2p3vNuwIz1Woqq6tGtVQrkrzDQZm\nNhPoUM6pic65qRXclgUkOee2mdlA4F0z63OomXPOTQYmQ6gD+VDvF6kuJT1bhcWh9vpxryw46JoL\nIxYbKwkEACuycrh1SqhN/6QHZtRgLuVwPXzRCeQXFXPXu0ujftb/HdGTv878gZeuHkTO/gJ++9r3\nJLduzICklgxIalkqGNz0k57ccHYPPlycxUdLs/jAG7EF0L5ZIhcO6BJ1fqrKNxg450Yc6kOdc3lA\nnne8wMzWAj2BTCCydF28NJGjzh/eXUqfTs0YOyiJTdtDX+6Pf7qax8eeVMs5k84tGoZn6N41uhej\nT+jI3rxCRjz2JQD/vDKVp2at4buN5c87KHH5KUkkxMZw6cldgdDQ1+e+XFfpPX07N+P64ceyPGs3\nT3y6GoCrh6TwwtfrGX5cW24c0YMbR4T6e4qLHbv2F3DxwNJf6r06NuOa01MY078TAKNP6MjoEzry\n6KVFfLg4i8wd+/nfYd2Jiz1ya4lWy9BSM/scuMU5l+a9bwtsd84VmVk3YDbQzzm33czmATdwoAP5\n7865aX6foaGlcqRVNARz9UPn0kOjbg7bP65IJTEuhitemBdOmzCsO09/vhYIDaMc93Ianyw/eIG2\nb+84i5aNEgB47ot1jB/WjcS4A7OfM3bso0WjBJokxvH9xh0VLg19Zs+2fPlDNp/dPJRubZtUmNeS\nvwOf3zKMtdl7uOalNEb2ac9zvwoNnvx0xX9ZnLGLG87uQfq2vSS1akS8zxd4Tm4BCbEx5c7arglV\nHVoaVTAwswuAvwNtgZ3AQufcSDO7CLgfKACKgXucc+9796QCLwINCY0i+q2rQiYUDKSm7c8vIr+o\nmBPv+4S/X3YSv61k2QOpXPqk0RUG05Ix8yXLMifGxZK9O4+TH5oZPv/KnA384d2lHNO6ERu85azb\nN0tk7p2H3FBxUD7SJ42muNiRlZNL5xblr3xaYtP2fWzbm0//ri0oLnb87dPVXHnqMbSuZOLf0eaI\nLEfhnHsHeKec9CnAlAruSQP6RvO5ItHavCuX2BgjPtbof//B7fj1ORDceHYPHveaP1KPaUlKm8b8\nvwUZ4V/TkX548Fx63nWglhS5NtCjl5zIzd5M5/RJo7n//eXMXn3g/shf9E0SQ19FvxnaDYBfnpLE\nyD7teembdJ6atZZHLzmRs3u1i6pcf7/spPAs7JgY8w0EEOq8LenAjYkxbvJZebQu0wxkOaqty95D\nYnxslf7hVmbe+u28Pn8jj15yImYW/rV4wUmdeef7YHVbndqtNd+u21bpNe9eN4Tzn/o6/P7cvh0Y\nOyiJoT3bsi+/kN53f8yVpx7DfWNCv9sKi4qJi41h865c/jNvI8e1b0qRc/z8xE7hP8vXrh3MgGNa\nkFtQTPOG8UBo+G2xq9qqm/vzi0iMCy2CV6KwqJi0DTsY3K31If85lEi+/UMS4mL44cFzD/sZddkR\naSY6khQM6qeSL5rI6fh78wpZnpVDv87NeeCD5Ywf2j386y2vsIivVm+lWcN42jVNZOP2fXy6Ygsv\nfpMOQLumidwy8jhuLTNjty4Z3a8jHy7JKpXWOCGWvflFTBgW+rO4o4J1gkqkTxrN+q17Gf6XzxmU\n3Io3x59a6vyOvfk0axhf5S/x+enbObNn20MvzBGwa38BsTEWrn3UN1q1VOoU5xwbtu0jOWKHqLLn\nU+6YRve2jUlq1YhZqw40N8xbvz28Xs9tby3m3TKLqEXasjuvTgaCP17Qjzfmb2RRxi6e+sUATpq9\njiWZu5i68EdO696aywYl8fDHK7n5nOOIMbhoQBcyd+5n+F8+D9d+endsxvKIFTtT2jRm9q3Dad0k\n4aDPa9n44LSKNEyIPWoDARCupUjlFAyk1j01aw2PfByayP7hDafTu2Mzih18vebAxKwvvUlaa7P3\nsjZ7b6n7V2/ZA8DUhZmVBoLa0K5pIlt255EYF0NexJpCZf16SDL/+jo9/H75/SO5/B9zWbhpJ29P\nOI0BSS25bFDX8Pn/OaMbuQVF7M4t5M7zjufYdk352YmdwucT4oyUNo1JnzSaDxdn8c73mSS3aUTr\nJgnszz8wi/pITWiSo9+RG8QqAvx7zgZG/e3LUmklgQBg9BNfhWoAd04rNfTwyojj8tw9dSk3vr6w\n0mtq0ikprcLHX912YPmARy45EYBPbx7KwxefUOqe0f06hiezDTsu1Dnarmki/77mFBolxBHnNdEU\nFoWacs2MyHUdG8TH8sJVJx+050BlXrnmFN7639MOoWRSX6hmIEdU5AzPvXmF4b1jo/Xytxuq5TmH\n4oPfns6cddt4a0EG3ds1Ye760BpFXVo2YsX9o8gvCnWklvR3XJraiMUZO/n3nI08cH5ffjX4GJb/\nmMPs1dn09ha2e/D8vpzeow0At517PL97fSG9O1V90bvylLT7JxzBCUxS9ygYSK3Yk1dI33s+Dm+o\ncrQqbzhlib6dm9O3c3P+54xuTF2YyX/mbuT3I48DQu3oDTk40N1xbi+aNojn0tTQjNTenZqFv+zL\nrll/cnIrvr79rKjLMKJXO8YP7c5vzuwW9bMkuDSaSI6oiiYiHa0W3XMOz32xlpt+0pOLn/2WhZt2\n8sXvh1FY7OheZubqlt25tGvaoJZyKlI+jSaSapNbUMSslVs4t1/Hw35G+ta9dWJUx7gzuzHZW5tm\n8q8G0rxhPLeOOh6AN39zKvlFxRUOUVQgkLpMwUB83ff+cl6btzE8qgUgc+d+hkz6jMm/Gsg5fUKL\n2k5dmEm7pg1o2iCO4zo0JT42hue+WMufIjZmOVr988pUGibEclr3Ntw26nj25BbSvFHp4JUQF0NC\nnNrdJZgUDAJu0/Z9vDp3I7eOPK7UzE4/mTv38+jHq/jTRf3I2BFaGyZyO8YlGaElB8a9soCnfzGA\nY1o3Omg0T0JsDPlFFQ+nPNJeu3Ywl/1jDhAat3/ZoK6s3LybpZm7OLtX+/B1sTF2UCAQCToFg4DY\nnVtA0wYHf4Fd/9r3LNq0k2e/WMuie84JN9Ws2rybY9s1ITbG+HzVFpZn5TBh2LEAvDF/I7dNCc1g\n7du5OQXeF/qEV7/jX78+mRe/Tmf1f3eHP2PCq9+Vm6ejKRB0admQU7sfWNLg8lNCmyX16tjskLao\nFAkqBYM6aNXm3bRtmkgrb5bo56u2cNW/5tOvc3NeuOpk2jZNpKjY8cb8TeRGTDD62d+/4stbh/PI\nxyt5atba8MYaV/1rPgDLf8yhVeOEUsM07/9geanP/rV3bV0w65ZhpLRpzPOz13FO71BT1nvXD2F+\n+o5azpnI0UejieqA/MJitu7Jo1OLhsxP384lz35L26aJfPDb02nVOKHU2vo92jVhxk1DeXXuBia+\nc/CuTcOPa1tqKYe64N6f9ebe90sHpW5tGzP9xjOJjTFiYw4sPPfVbcP5y8eruHN0L3XoiqDRRHXa\nlt25JMbFsjZ7D+NfWUB8bEx4V6cS2bvzOOWPnx507+ote/jjtBXkV7D0QV0JBI9cfAJvpm1ifvoO\nrhqSwgUDutCsQRxmxrY9eTRMiC23M7dLy0b8TTuRiRyyehMMMnfup7jYVctaLPvyC8krKObWKYvp\n37UF1w0/ttzr0rfuZdhfPuejG88o1S6dW1DE56uyGdW39NbSRcWOgqJiBj108Jf8oZjss23f0eb5\nK1J55ONVrPrvbm76SU86NGvARQO6cF6/jmTvzgNKLzZW3sYiP+ndnhnl7IwlIlVTb4LBkEmfAQfP\n8vRzx9uLadognjvP6xVOG/W32WzcHhphM2P5f8PBYEVWDuu37uW8fh354ofs8Ho6UxZkcNdPe4fv\nf+CD5bw6dyPvTDiNvXmh5X9/3r8T176cxroyi7AF3fDj2nJ2r3Zs25vHbVOWcGLXFgz1VsBsnBhH\n4youO/zsLweGN6sXkUMXVTAws0eAnwH5wFrg1865nd65O4BrgCLgBufcx176QA5sezkNuLEq215W\ntzfmb2TrnvwKf9WXeG3eJiC0ZnthcTEzV2wJ/1otsXVPHk/PWssLX68HYNl9I0strDYvfTtp6dvp\n0b4pV74wL7wT1FOz1jJzRejXbMnOUvVJZGC+NLUrJ3ZtwfEdDm9kT6jv4MjsKSsSRNHWDGYAdzjn\nCs3sz8AdwG1m1hsYC/QBOgEzzaync64IeAa4FphLKBiMIrQX8hFVMnTSLxiUeGVOxQuhpT44s9T7\nRz/5odT7xRm7uPjZbw+6ryQQ1CeXDUqiV8emnNu39GxmMzvsQCAi0Yt2D+RPIt7OAS72jscArzvn\n8oD1ZrYGGGRm6UAz59wcADN7GTifIxgMLnz661KjTL74IZuhPduyO7eA7N15dGvbhF37CmjSII6z\nH/38sD6jpIYgB8TFGIvvPYdGCfWmZVKkTqnOf5lXA294x50JBYcSGV5agXdcNv2I+W7jzlLvr3xh\nHivuH0W/ez+p4A6J1kMX9GVQcisFApGjmO+/TjObCXQo59RE59xU75qJQCHwanVmzszGAeMAkpKS\nqvPRpfS6e3qNPbuu+ev/OZG4mBi+27ij1M5bAN3bNj5ol7FIn908lDnrtnNevw7syStk4jtL+eKH\nbH5xyjE1nGsRiZZvMHDOjajsvJldBfwUODuiIzgT6BpxWRcvLdM7Lpte0WdPBiZDaNKZX14rsjhj\np/9FdcyglFbM8zZTKU/nFg3p3KIh89JD1yS3bsSlJ3fl4emhXcUuGdiFS1K7culzoQls2bvzOLdv\nBy44KfS/52cnduL2c49nyKRZbN0T6jD/9OZh4eGyACclteDigV2INeP8kzrTID6Wbt6yzi0aJfDS\n1YNqqvgiUs2iHU00CrgVGOqc2xdx6j3gP2b2GKEO5B7APOdckZnlmNlgQh3IVwB/jyYPVfHzJ7+u\n6Y844kb0ahcOBhee1Jm3v8+kdeME5t55Njm5heGlKrbtyaOgyNGmSQJxsTE0bxjPxHeWEhcbw6CU\nVvzzylR25xbyuzcO3jIyMS6W964fwmnesFyg1Ib1r107mAbVtFOZiNSuaBtxnwQSgRne3qxznHPj\nnXPLzOxNYDmh5qPrvJFEABM4MLT0I2phJFFd0b9rC969bggAu/YVcOL9oX6N1o0T+OXgY1j2Yw5/\n+Glv2jRJ5PQebTg5uRVxsTHhQAAHT9Aa1acD/5y9nmvPSAHg7F7t+XBxFgBWzqKmnVo05OlfDCi1\nkcvie8+hUXwscdpGUSQwoh1NVOG4TOfcQ8BD5aSnAX2j+dz6IjZiyenmjeJp2iCO3bmFfHbLMBol\nxPF4xLILFw7oUt4jDtK6SSKf3TKsVFrnlg2B0Aql5TmvzKY2zcpZHVVE6jYN7zgKPD62PyP7dOBf\nX6eTV1jEC1+tJye38KDrTujSnK/XbCM+tur7ElRF/64t+OjGMziufdNqfa6I1B2Br+cX1NKa+kvv\nG8kzvxgQfv/S1YP48IbTgVDn7gtXHVhEcEz/UOfr/w7rzu9G9GT2raFN0McP7V7qmc/+ciBvTzit\nRoZo9urY7JA2vxGRYAl8zWDKggz/i3yk3TXioFnGb084jeM7NOXFb9J5ePoqLk3twtdrtjHk2NZ0\nadmIJolxpfYMLllvJ3IJhpeuHhRe4yhS80bx5a6h1LRBfHjbSRGR6hT4YJBbUFTq/UUDuvCHn/ai\nRaOE8Br4JR4f25+73lnK7rzSTTRtIjphLxzQmTZNEunfpQUxMcb4M7tzcnIrTk5uVe7n/3JwEmcf\n377ccyUBQkSktgU+GMSWGfHyi8FJtGgUGm3z1vhTufjZb8N79Z7avTUL7zmH/8zdwKndW7Ngww7a\nNSu9Qcpjl/Yv9T4mxioMBAAPnt+vmkoiIlJzAh8MWkcMswSIbBVPTW7FmofOxQFZO3PDaxb96tRk\nAI5td6BD9dozUnihzIxcEZGgCHwwaBBfeR95yVj5pNaVb3ozcXRvJo7uXek1IiJ1VeBHExUWlV7F\nwsqbWSUiUs8FPxgUlwkGtZQPEZGjWb0JBtcND43ZP8anOUhEpD4KfJ9Bkbcv7iUDu/L7kcfXcm5E\nRI5Oga8Z5BWEgkGsZteKiFQo8MHg9rdDex0rGIiIVCzwwaCEBhGJiFQs8MHgrOPbAdC+aQOfK0VE\n6q/AB4NeHZsSG2NakVNEpBKBDwbOaW6BiIif4AcDIEYdBiIilYoqGJjZI2a20swWm9k7ZtbCS082\ns/1mttB7PRtxz0AzW2Jma8zsCavh9SGKVTUQEfEVbc1gBtDXOXcC8ANwR8S5tc65/t5rfET6M8C1\nQA/vNSrKPFROsUBExFdUwcA594lzrmQnmDlApbuym1lHoJlzbo5zzgEvA+dHkwffPKJmIhERP9XZ\nZ3A18FHE+xSviegLMzvDS+sMRO5DmeGl1ZjiYqc5BiIiPnzXJjKzmUCHck5NdM5N9a6ZCBQCr3rn\nsoAk59w2MxsIvGtmfQ41c2Y2DhgHkJSUdKi3A6GagWKBiEjlfIOBc25EZefN7Crgp8DZXtMPzrk8\nIM87XmBma4GeQCalm5K6eGkVffZkYDJAamqqq+i6yvOvPQxERPxEO5poFHAr8HPn3L6I9LZmFusd\ndyPUUbzOOZcF5JjZYG8U0RXA1GjyUKV81vQHiIjUcdEuYf0kkAjM8H59z/FGDp0J3G9mBUAxMN45\nt927ZwLwItCQUB/DR2UfWp0ch1WhEBGpV6IKBs65YytInwJMqeBcGtA3ms89FE6dBiIivgI/AxkU\nC0RE/NSLYCAiIpWrF8FAo4lERCoX+GDgjXYVEZFKBD4YgHY5ExHxE/hgoHqBiIi/wAcD0GgiERE/\ngQ8G6jIQEfEX+GAAGk0kIuIn8MFAy1GIiPgLfDAA9RmIiPgJfDBQn4GIiL/gBwM0z0BExE/gg0GI\nooGISGUCHwzUTCQi4i/wwQDUTCQi4qceBANVDURE/NSDYKAeAxERP1EFAzN7wMwWm9lCM/vEzDpF\nnLvDzNaY2SozGxmRPtDMlnjnnrAanh6sPgMREX/R1gwecc6d4JzrD3wA3A1gZr2BsUAfYBTwtJnF\nevc8A1wL9PBeo6LMgy/1GYiIVC6qYOCcy4l425gDDfRjgNedc3nOufXAGmCQmXUEmjnn5rjQrjMv\nA+dHkwf/PNbk00VEgiEu2geY2UPAFcAuYLiX3BmYE3FZhpdW4B2XTa/o2eOAcQBJSUmHn0f1GoiI\nVMq3ZmBmM81saTmvMQDOuYnOua7Aq8D11Zk559xk51yqcy61bdu2h/cMjSYSEfHlWzNwzo2o4rNe\nBaYB9wCZQNeIc128tEzvuGx6jXFOfQYiIn6iHU3UI+LtGGCld/weMNbMEs0shVBH8TznXBaQY2aD\nvVFEVwBTo8lDlfJZ0x8gIlLHRdtnMMnMjgOKgQ3AeADn3DIzexNYDhQC1znnirx7JgAvAg2Bj7xX\njVEjkYiIv6iCgXPuokrOPQQ8VE56GtA3ms89VNrpTESkcoGfgayhpSIi/gIfDERExF/gg4GGloqI\n+At8MAANLRUR8RP8YKCKgYiIr+AHA1QzEBHxE/hgoIqBiIi/wAcD0EJ1IiJ+Ah8MnCYaiIj4Cn4w\nQH0GIiJ+Ah8MQAvViYj4CXwwUCuRiIi/wAcD0EJ1IiJ+Ah8MVDEQEfEX+GAA6jMQEfET+GCgoaUi\nIv4CHwwAVQ1ERHxEuwfyA2a22MwWmtknZtbJS082s/1e+kIzezbinoFmtsTM1pjZE1bDvbuqF4iI\n+Iu2ZvCIc+4E51x/4APg7ohza51z/b3X+Ij0Z4BrgR7ea1SUefClioGISOWiCgbOuZyIt43x+SFu\nZh2BZs65OS7UmP8ycH40efClqoGIiK+4aB9gZg8BVwC7gOERp1LMbKGXfpdzbjbQGciIuCbDS6tR\nmmcgIlI535qBmc00s6XlvMYAOOcmOue6Aq8C13u3ZQFJXvPRTcB/zKzZoWbOzMaZWZqZpWVnZx/q\n7YC2vRQRqQrfmoFzbkQVn/UqMA24xzmXB+R59y8ws7VATyAT6BJxTxcvraLPngxMBkhNTT2sb3Xn\n1GcgIuIn2tFEPSLejgFWeultzSzWO+5GqKN4nXMuC8gxs8HeKKIrgKnR5KFq+azpTxARqdui7TOY\nZGbHAcXABqBk1NCZwP1mVuCdG++c2+6dmwC8CDQEPvJeNUZzzkRE/EUVDJxzF1WQPgWYUsG5NKBv\nNJ97qLTTmYhI5QI/A1kdyCIi/gIfDEB9BiIifgIfDNRnICLiL/DBQERE/AU+GKhiICLiL/DBALQc\nhYiIn8AHA/UZiIj4C3wwAC1HISLipx4EA1UNRET8BD4YOKd5BiIifgIfDEDBQETET+CDgRqJRET8\nBT4YgBaqExHxE/hg4DS2VETEV+CDAajPQETET+CDgeoFIiL+Ah8MQJPORET8BD4YqMtARMRftQQD\nM7vZzJyZtYlIu8PM1pjZKjMbGZE+0MyWeOeesCOxipw6DUREKhV1MDCzrsA5wMaItN7AWKAPMAp4\n2sxivdPPANcCPbzXqGjzUBlVDERE/FVHzeCvwK2U/t4dA7zunMtzzq0H1gCDzKwj0Mw5N8eFxny+\nDJxfDXnM+PzgAAAI0UlEQVSokHNOfQYiIj6iCgZmNgbIdM4tKnOqM7Ap4n2Gl9bZOy6bXtHzx5lZ\nmpmlZWdnR5HPw75VRKReiPO7wMxmAh3KOTURuJNQE1GNcM5NBiYDpKamqsVHRKSG+AYD59yI8tLN\nrB+QAizy+oC7AN+Z2SAgE+gacXkXLy3TOy6bXqNUMRARqdxhNxM555Y459o555Kdc8mEmnwGOOc2\nA+8BY80s0cxSCHUUz3POZQE5ZjbYG0V0BTA1+mJUls+afLqISDD41gwOh3NumZm9CSwHCoHrnHNF\n3ukJwItAQ+Aj71WjtAeyiEjlqi0YeLWDyPcPAQ+Vc10a0Le6PteP0+BSERFfgZ+BDOozEBHxE/hg\noD4DERF/gQ8GoHkGIiJ+Ah8MVDMQEfEX+GAA2vZSRMRP4IOBRhOJiPgLfjBwaDiRiIiPwAcDUCwQ\nEfET+GCgRiIREX+BDwagoaUiIn6CHwxUNRAR8RX8YICGloqI+Al8MNDQUhERf4EPBqA+AxERP4EP\nBlqOQkTEX+CDAahmICLiJ/DBQBUDERF/1RIMzOxmM3Nm1sZ7n2xm+81sofd6NuLagWa2xMzWmNkT\ndgT2pNRoIhGRykW97aWZdQXOATaWObXWOde/nFueAa4F5gLTgFHU4D7ITp0GIiK+qqNm8FfgVqrQ\nImNmHYFmzrk5LvQt/TJwfjXkoUIO9RmIiPiJKhiY2Rgg0zm3qJzTKV4T0RdmdoaX1hnIiLgmw0sT\nEZFa5NtMZGYzgQ7lnJoI3EmoiaisLCDJObfNzAYC75pZn0PNnJmNA8YBJCUlHertgIaWiohUhW8w\ncM6NKC/dzPoBKcAirw+4C/CdmQ1yzm0G8rz7F5jZWqAnkOldV6KLl1bRZ08GJgOkpqYe9tf6Eeij\nFhGp0w67mcg5t8Q51845l+ycSybU5DPAObfZzNqaWSyAmXUDegDrnHNZQI6ZDfZGEV0BTI2+GJXk\nsyYfLiISEFGPJqrAmcD9ZlYAFAPjnXPbvXMTgBeBhoRGEdXYSKISqheIiFSu2oKBVzsoOZ4CTKng\nujSgb3V9ri91GoiI+Ar8DGTQ0FIRET+BDwaqF4iI+At8MAD1GYiI+Al8MFCXgYiIv8AHA9A8AxER\nP4EPBtr2UkTEX/CDgVOfgYiIn8AHA9DQUhERP4EPBupAFhHxF/hgEKKqgYhIZQIfDFQxEBHxF/hg\nAOozEBHxE/hgoD2QRUT8BT4YgHoMRET81ItgICIilasXwUB9BiIilQt8MFCXgYiIv+AHAxymXgMR\nkUpFFQzM7F4zyzSzhd7rvIhzd5jZGjNbZWYjI9IHmtkS79wTdgSWFFUzkYhI5aqjZvBX51x/7zUN\nwMx6A2OBPsAo4Gkzi/Wufwa4FujhvUZVQx4qpGYiERF/NdVMNAZ43TmX55xbD6wBBplZR6CZc26O\nC00AeBk4v4byEKaagYhI5aojGPzWzBab2Qtm1tJL6wxsirgmw0vr7B2XTS+XmY0zszQzS8vOzj6s\nzJ3Zsy0Dklr6XygiUo/F+V1gZjOBDuWcmkioyecBQksAPQA8ClxdXZlzzk0GJgOkpqYeVoPPH37a\nu7qyIyISWL7BwDk3oioPMrN/AB94bzOBrhGnu3hpmd5x2XQREalF0Y4m6hjx9gJgqXf8HjDWzBLN\nLIVQR/E851wWkGNmg71RRFcAU6PJg4iIRM+3ZuDjYTPrT6iZKB34DYBzbpmZvQksBwqB65xzRd49\nE4AXgYbAR95LRERqkdWVVT1TU1NdWlpabWdDRKROMbMFzrlUv+sCPwNZRET8KRiIiIiCgYiIKBiI\niAh1qAPZzLKBDYd5extgazVm52gS5LJBsMunstVddal8xzjn2vpdVGeCQTTMLK0qvel1UZDLBsEu\nn8pWdwWxfGomEhERBQMREak/wWBybWegBgW5bBDs8qlsdVfgylcv+gxERKRy9aVmICIilQh0MDCz\nUd4ezGvM7Pbazk9VeRsFbTGzpRFprcxshpmt9v7bMuLcUbPftB8z62pms8xsuZktM7MbvfQ6Xz4z\na2Bm88xskVe2+7z0Ol+2EmYWa2bfm9kH3vsglS3dy9dCM0vz0gJTPl/OuUC+gFhgLdANSAAWAb1r\nO19VzPuZwABgaUTaw8Dt3vHtwJ+9495e2RKBFK/Msd65ecBgwAitDnvuUVC2jsAA77gp8INXhjpf\nPi8fTbzjeGCul786X7aIMt4E/Af4IEh/L718pQNtyqQFpnx+ryDXDAYBa5xz65xz+cDrhPZmPuo5\n574EtpdJHgO85B2/xIG9o4+q/ab9OOeynHPfece7gRWEtj6t8+VzIXu8t/HeyxGAsgGYWRdgNPB8\nRHIgylaJoJcvLMjBoKJ9mOuq9i60ORDAZqC9d1wt+03XBjNLBk4i9As6EOXzmlEWAluAGc65wJQN\n+BtwK1AckRaUskEocM80swVmNs5LC1L5KhXt5jZSC5xzzszq9DAwM2sCTAF+55zLiWxWrcvlc6FN\nnPqbWQvgHTPrW+Z8nSybmf0U2OKcW2Bmw8q7pq6WLcLpzrlMM2sHzDCzlZEnA1C+SgW5ZlDRPsx1\n1X+9KmjJdqNbvPQ6t9+0mcUTCgSvOufe9pIDUz4A59xOYBYwimCUbQjwczNLJ9TkepaZ/ZtglA0A\n51ym998twDuEmpoDUz4/QQ4G84EeZpZiZgnAWEJ7M9dV7wFXesdXcmDv6Dq137SXl38CK5xzj0Wc\nqvPlM7O2Xo0AM2sI/ARYSQDK5py7wznXxTmXTOjf0mfOuV8SgLIBmFljM2tacgycQ2hP90CUr0pq\nuwe7Jl/AeYRGq6wFJtZ2fg4h368BWUABoTbHa4DWwKfAamAm0Cri+oleGVcRMXIBSCX0F3ot8CTe\nJMNaLtvphNpmFwMLvdd5QSgfcALwvVe2pcDdXnqdL1uZcg7jwGiiQJSN0KjDRd5rWcn3RVDKV5WX\nZiCLiEigm4lERKSKFAxERETBQEREFAxERAQFAxERQcFARERQMBARERQMREQE+P+3GA8XwhAMCQAA\nAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x130ea3f60>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(elbo_log)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAagAAAEYCAYAAAAJeGK1AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3XuUXGW5JvDnIQkQrkFyIYRAooAYnANIuCg4g8glKEw4\nLsUgogfByFqAxMWMROTiEXWQQ8AjcoitZEVGBPGImBMjHGBEFgNIwp0QYBrIpXOluSVCgHTyzh97\nN1TX93V3VffeVd/b/fzW6pXeb+2u+qpTT7+19/5qb5oZREREUrNVswcgIiISowYlIiJJUoMSEZEk\nqUGJiEiS1KBERCRJalAiIpIkNSgZ1EjOIbmO5NPd3E6SPyXZSvJJkh9r9BhFUldWjtSgZLCbC2BK\nD7efAGCf/Gs6gOsbMCYRb+aihBypQcmgZmb3AXi1h1WmArjRMg8BGEFybGNGJ+JDWTkaWtQARco2\nZcoUa29vr3n9Rx55ZDGAtytKLWbWUufDjgOwomK5La+trvN+RJJQb46AQrLUpxypQYkb7e3tWLRo\nUc3rk3zbzCaXOCQRd+rNEdC8LKlBiStNOHfkSgDjK5b3yGsibnnJkY5BiStmVvNXQeYB+Eo+C+lw\nAG+YmXbviWv15KigLPUpR9qCEleKfudH8mYARwEYSbINwGUAhuWPNRvAAgCfAdAK4C0AZxQ6AJEm\n8JIjNShxo+Ato877PLWX2w3AOYU+qEgTecqRGpS4ouuXifSflxypQYkrXoIlkjIvOVKDEle8BEsk\nZV5ypAYlrngJlkjKvORIDUrcKOPgrshg4ylHalDiipdgiaTMS47UoMQVL8ESSZmXHKlBiStegiWS\nMi85UoMSV7wESyRlXnKkBiVueDq4K5IqTzlSgxJXvARLJGVecqQGJa54CZZIyrzkSA1KXPESLJGU\necmRGpS44WnfuUiqPOVIDUpc8RIskZR5yZEalLjiJVgiKfOSIzUoccVLsERS5iVHalDiipdgiaTM\nS47UoMQNTwd3RVLlKUdqUOKKl2CJpMxLjtSgxBUvwRJJmZccqUGJK16CJZIyLzlSgxJXvARLJGVe\ncqQGJW54OrgrkipPOVKDEle8BEskZV5ypAYlrngJlkjKvORIDUpc8RIskZR5yZEalLjiJVgiKfOS\nIzUoccPTwV2RVHnKkRqUuOIlWCIp85IjNShxxUuwRFLmJUdqUOKKl2CJpMxLjtSgxBUvwRJJmZcc\nqUGJG54O7oqkylOO1KDEFS/BEkmZlxypQYkrXoIlkjIvOVKDEle8BEskZV5ypAYlrngJlkjKvORI\nDUrc8HRwVyRVnnKkBiWueAmWSMq85GirZg9gsCK5L8nXmz0Obzrf/dXyJVI0kotJHtXscfRXPTlq\nZpYGfYMi+feKry0kN1Ysn9aP+32I5Je7u93MnjezEX29/zrHsobkkY14rLJ5CJV3JJdW5GANybkk\nd2j2uKqR/B7JX5d4/3NJ/qCyZmb7m9m9ZT1mo6hBOWFmO3R+AVgO4KSK2k3NHp90VXSoSE4h+RzJ\nVpIzI7fvTPI/SD6Rv3s+o/AnlaaT8kwcCOAgAN9p8njqxsyg/xsXU0aDKiNL+s/rBckhJC8h+SLJ\ndpI3kRyR37Y9yVtIvkrydZJ/I7kLyVkADgHwy/xd6KzI/e5HsqNi+SGSl+X/rie5gOQuleuSPJvk\napKrSJ5X8bO3kLy4YnkKydb8+98BGA3gP/OxfLOs31XZig4VySEArgNwAoBJAE4lOalqtXMAPGNm\nBwA4CsAsklsX+8zSZWZrANyJrFGB5DYkryK5nORakrNJDu9cn+RUko/nr+EXSE7J67uTnJdnpZXk\n1yt+5nskbyV5I8kN+R+vyRW3X0hyZX7bcyQ/nd/vRQC+mL+un8jXvZfkD0n+XwBvAfhgvkV4TNXj\n/bpi+UiSD+QZXkHyn0hOB3AagG/n9/8f+brv3Vf+u/hJnsdV+ffb5LcdRbKN5AUk1+W5TeLNTb05\namaW1KB69z8AHAfgSAB7ANgE4Jr8trOQTTQZB2AkgHMBvGtmFwBYCOCsfEvsghof60vIQjEWwAgA\n51fcNgTAxwF8EMBnAfwza9htZ2ZfALAOwHH5WH5a41iSVPC7vkMBtJrZi2b2LoBbAEytfkgAO5Ik\ngB0AvAqgA4MEyT2Q/dFpzUtXANgXWcPaG9lr/9J83UMB3AjgfyJ7/f5XAEvzn7sFQBuA3QF8HsCP\nSB5d8VD/PV9nBIB5AH6W3+eHkeXqEDPbEcDxAJaa2R0AfgTgt/nr+oCK+zodwHQAOwJY1svz2wvA\nnwFcC2BU/rweN7MWADcBuDK//5MiP/5dAIfnP3MAstfTxRW37wZg5/x3dCaA6zrfdDZbCVtQpWRJ\nDap3ZwOYaWarzOxtAP+M7F0bkTWrUQA+ZGYdZrbQzN7sx2P9wsxeyO/j35G/a61wmZltNLPHAPwa\nwKn9eCyXCg7VOAArKpbb8lqlnwH4CIBVAJ4CcL6ZbSniuSTudpIbkP1+1gG4LH/NTwfwLTN71cw2\nIGsS0/KfORPAHDO7y8y2mNlKM3uW5HgARwC40MzeNrPHAfwSwFcqHu9+M1tgZpsB/G9kf/ABYDOA\nbQBMIjnMzJaa2Qu9jH2umS3OM7mpl3W/BOBuM7vZzDaZ2Sv5+GpxGoDvm9k6M3sZ2d+G0ytu35Tf\nvsnMFgD4O4AP13jfpSqhQZWSJTWoHuSBHA9gQb75/zqAx5D93nYFcAOAvwL493xz/kf5pm5fran4\n/i1k7zIqVb4AliF7Nzqo1BmqkSQXVXxN78NDHg/gcWS/6wMB/IzkTgU+pVSdnG+xHAVgP2R7CEYB\n2A7AIxV5uCOvA1lWYs1jdwCdDa3TMnT9A1b92t+W5FAzawUwA8D3AKzLd2f39rpf0cvtlbobcy12\nR9cttOpMvmJmlVsIsUw3RR8aVFOypAbVA8v+Z1YCONrMRlR8bWtm7Wb2jpldamb7Idud8QW8/26y\njKkv4yu+3xPZOxEAeBPZH45Ou1X93ICZ0lZnqNrNbHLFV0vV3a1E19/pHnmt0hkAbrNMK4CXkP3B\nHhTM7K8A5gK4CkA7gI0A9q/Iws6WTaYAssbwocjdrALwAZI7VtT2RPi77m4MvzGzIwHshey1/OPO\nm7r7karlnvLR3Zh7uv9Oq/IxdarMZNL60KCakiU1qN7NBnBFvpsCJEeTPCn//hiSk5jNFFqPbH9q\n5ybrWmTHi4p0GcnhJA9Ativht3n9cQAnkhxBchyA86p+royxNFwJuyUWAtiH5MT8YO00ZMc/Ki0H\n8GkAIDkG2S6aFwt8Wh78BMCxAP4LgF8AuIbkaAAgOY7k8fl6NwA4I5/EsFV+235mtgLAAwD+F8lt\nSf4Dst2BvU4RJ/lhkkfnkw/eRtYgKzM2gb3P1HscwDSSw/LJF5+vuO0mAMeQPIXkUJK7kuzctd5b\nbm4GcDHJUSRHIjsWV9q096LUm6NmZkkNqndXArgbwP/J98k/AOBj+W3jAPwRwAYATwNYgPebxjUA\nvkLyNZJXFjCOzQD+huxdxx3I9m3fl982B9lB7OUA5iMLTqUfAvhhvlvm3ALG0jRFhirf/XIusllq\nSwDcamaLmc2WPDtf7XIAnyD5FIB7kB1HaS/p6SXJsuMrNyL7A3whstfaQyTXI8vGh/P1Hkb2Lvka\nAG8g2/3duYVxKoAJyLYw/oDseOrdNTz8NsgmZrQj2w04Gu9Pef9d/u8rJB/t4T4uQbaV9Bqy40S/\nqXhuywF8BsAFyA7aP473j3/dgOzY1+skb4/c7w8ALALwJLJjKo/mteQV3aDKyhJr7I7SRCT3A/C0\nmQ3qU1N95CMfsblz59a8/uGHH/6ImU3ufU2RwaPeHAHNy9Kg/oMn/ugNlUj/ecmRGpS44iVYIinz\nkiM1KAfM7Fno/6qeA7Yi0g1POerXJAn2cu4lkaIVPPMoGcqSNFIJs/hK0ed35RXnXjoW2aeGF5Kc\nZ2bP9PAzvv5qSEOZGWtYpxFDaShlSYrWW5a85Kg/W1C1nHtJpFAe3vX1gbIkDTXgt6AQP/fSYdUr\n5afE6MtpMUQCzhpPrZQlaSgvOSr9wLtlp8RoAbRbQvqn2e/mmk1ZkiJ4ylF/GlQt514SKZSXYNVJ\nWZKG8pKj/jSo9869hCxM05Cdul6kNF6CVSdlSRrKS4763KDMrCM/r9udyC6mN8fMFhc2MpEIL8Gq\nh7IkjeYlR/06BmXZRbgWFDQWkV55CVa9lCVpJC85GvRnJxA/PB3cFUmVpxypQYkrXoIlkjIvOVKD\nEle8BEskZV5ypAYlrngJlkjKvORIDUpc8RIskZR5yZEalLjh6eCuSKo85UgNSlzxEiyRlHnJkRqU\nuOIlWCIp85IjNShxxUuwRFLmJUdqUOKKl2CJpMxLjtSgxA1PB3elGFttFV5TdciQIUFt6NDwT1ls\nvdj9VduyZUtQ6+joCGqbN2+uqRa7v2bylCM1KHHFS7BEUuYlR2pQ4oqXYImkzEuO1KDEFS/BEkmZ\nlxypQYkrXoIlkjIvOepXgyK5FMAGAJsBdJjZ5CIG5cmECROC2gMPPBDUdtttt6BGMqjdddddQe3S\nSy8Nak888USX5Y0bN/Y0zAHB08Hdeg30LMUmJ1RPbBgxYkSwzp577hnUDj744KC27777BrXRo0f3\n+pgA8M4773RZXrlyZbDOokWLgtqSJUuC2iuvvBLU3nrrraD27rvvBrXqiRhlvdY95aiILahPmVl7\nAfcj0isvweojZUkawkuOtItPXPESLJGUeclR7x8K6JkBuJvkIySnFzEgkZ507p6o5csZZUkapp4c\nNTNL/d2COtLMVpIcDeAuks+a2X2VK+RhU+CkEA4bT62UJWkYLznqV4Mys5X5v+tI/gHAoQDuq1qn\nBUALAJD08Vupw8yZM4NabEJETOxFcswxx9RUe/nll7ssz58/P1jnd7/7XVC74447ahpbipr9bq5M\nXrMUm+gTm4iwww47BLUPfehDXZY/9alPBescccQRQS02MWn77bevaWyx10/1eq+99lqwzrhx44La\nzjvvHNSeeeaZoLZixYqg9sYbbwS16jNOxM5AUcTr31OO+ryLj+T2JHfs/B7AcQCeLmpgIjEedkvU\nS1mSRhsMu/jGAPhD/u5jKIDfmJnft+figqfGUwdlSRrKS4763KDM7EUABxQ4FpFeeQlWPZQlaTQv\nOdI0c3HFS7BEUuYlR2pQPRg1alRQqz6Q+7Wvfa2m+2prawtqsbM/7LXXXkFt66237nVsZ5xxRrDO\nl7/85aA2Z86coDZjxoygVv3p+hQ0e3+4hGITEWITIg44INxAnDJlSpflT3ziE8E6Y8aMCWqxszDE\nzv6wYcOGoBabeFB9BovYJI9ddtklqFVP8gCANWvW1FSrhc4koQYlzngJlkjKvORIDUpc8RIskZR5\nyZEalLjiJVgiKfOSIzUoccPTvnORVHnKkRpUbr/99gtqsbMuVJ/+f+nSpcE6zz77bFC78sorg9q9\n994b1Pbee++gdvLJJwe16gPKU6dODdYZNmxYUPvGN74R1FpbW4ParFmzgloKvARrsIhNKNhpp52C\n2vjx44NadZa23XbbYJ3ly5cHtdilL5577rmgFpvoExtb9ZkpJk6cGKwTm5gRm4QRu7TG22+/HdQ2\nb94c1GITOMriJUf9PVmsSEMV/el3klNIPkeylWR43qpsnaNIPk5yMcm/FvqERJqgjDNJlJElbUGJ\nK0W+8yM5BMB1AI4F0AZgIcl5ZvZMxTojAPwbgClmtjw/mauIa0VvQZWVJW1BiSsFv+s7FECrmb1o\nZu8CuAVA9b7SLwG4zcyW54+/rtAnJNIEJWxBlZIlNShxow+hGklyUcVX9aUqxgGoPNV0W16rtC+A\nXUjem1+r6SvlPUOR8tWbo2ZmSbv4cqeeempQqz6IGzN58uSgFjuVfmzCQkxswsJVV13V6889+OCD\nQe2www6r6TF33HHHmtZLQZ27JtrNLPwPqs9QAAcD+DSA4QAeJPmQmT3fz/sdELbaKnyPO3z48KC2\n3Xbb9fqzsbNBPPbYY0Htr38ND12sXbs2qMXOwLLrrrsGteqJHrHxxyY6xM4Q8frrr9f0s7FJEo3U\nh118TcmSGpS4UvC+85UAKqeX7ZHXKrUBeMXM3gTwJsn7kJ3YVQ1K3CphFl8pWdIuPnGl4P3mCwHs\nQ3Iiya0BTAMwr2qdPwI4kuRQktsBOAzAkkKflEiDlXAMqpQsaQtKXCnynZ+ZdZA8F8CdAIYAmGNm\ni0mend8+28yWkLwDwJMAtgD4pZnpYoLiWtFbUGVlqdcGRXIOgBMBrDOzj+a1DwD4LYAJAJYCOMXM\nwuskixSons9k1HGfCwAsqKrNrlr+FwD/0t/HUpYkBWXkKL/fwrNUyxbUXAA/A3BjRW0mgHvM7Ir8\nA1kzAVxY64MOJF/4wheC2s9//vOg1p+DorEDuy0tLV2WDz300Jruq729vdf7SlkZwWqguXCcpdil\nNWK1TZs2BbU333wzqFVPJoplJHYmidhr+O9//3tQq76MBhC/FEj1erHX2KpVq4Laiy++GNRWr14d\n1GqdJNHI17aXHPV6DMrM7gPwalV5KoBf5d//CkB4Lh6REhS837yhlCVJRQnHoErR12NQY8ys863C\nGgDhVcVESpBi4+knZUkazkuO+j1JwsyMZLfPNv9AV/WHukT6xEuw+kJZkkbxkqO+Nqi1JMea2WqS\nYwF0e8oKM2sB0AIAPYWv2a699tqgduKJJwa1gw46qMvy9ddfH6xz8MEHB7Wzzz47qMVeJNOmTQtq\nl1xySVCLnX292uLFi4Na7NLwsf3rKWr27oaSuM5S7FhK7HjQa6+F8z6q14t9sDZ2qfVRo0YFtdiH\nzUePDk/1FstN9ZnWYx+0jx0Li9XWr18f1Do6OoJaM1/HnnLU189BzQPw1fz7ryKb3y5SOg/7zeuk\nLEnDDZhjUCRvBnAUsnMxtQG4DMAVAG4leSaAZQBOKXOQIp0cNZ6AsiSp8JKjXhuUmYUnqct8uuCx\niPTKS7BilCVJhZcc6UwS4oqXYImkzEuO1KBysQ//HXfccUFt/vz5XZZjH5A966yzgtr+++8f1GIf\n9DvttNN6HGd3nnjiiaB2/PHHB7V16/xezqjZ+8MHu9jvPnaZ8tjl0WMTD6onD8QuCx+7RHtsQkTs\n/nffffegFvvwbvV4X3jhhWCd2FUGYn8zYh9SbuSl3GvhKUdqUOKKl2CJpMxLjtSgxBUvwRJJmZcc\nqUGJK16CJZIyLzlSgxJXvARLJGVecqQG1YNXXnklqH384x/vsnz11VcH68yYMaPXn+uuFhM7G/IV\nV1zRZfnyyy8P1vHyIqyVp4O7g0Xs/yM2UWDDhg1BrfoM57WeSSJWi52pInY5+tg4qi/dHpu8VOuE\nCA+vT085UoMSV7wESyRlXnKkBiWueAmWSMq85EgNSlzxEiyRlHnJkRqUuOIlWCIp85IjNah+GjJk\nSOmP8d3vfjeoXXPNNaU/bmo8HdwdLGL/H7VegmPZsmVdlmNnfpg4cWJQGz58eE2P+dZbbwW12Fku\nqidOvP7668E6sQkRMR5en55ypAYlrngJlkjKvORIDUpc8RIskZR5yZEalLjiJVgiKfOSIzUoccVL\nsERS5iVHtVxRdw6AEwGsM7OP5rXvAfg6gJfz1S4yswVlDTIl3/nOd7osn3vuuTX93Nq1a4Pa8uXL\ng9pHP/rRoPb9738/qD344INdlh966KGaxuGZp4O7MYMlS7EJC9VnjQCAl156qcvy6NGjg3W23377\noBY740Rs8kOsFhvbsGHDuixvu+22wTpDh4Z/KknWVEvtNespR+G5QEJzAUyJ1K8xswPzL9eBEj86\nw1XLV4LmQlmSBNSTo2ZmqZZLvt9HckL5QxHpXaKNpybKkqTCS45q2YLqznkknyQ5h2R49sYcyekk\nF5Fc1I/HEgHgfguqO8qSNJSXLai+NqjrAXwQwIEAVgOY1d2KZtZiZpPNbHIfH0vkPR5CVSdlSRrO\nS4Pq0yw+M3vviD/JXwCYX9iIEnL44YcHtYsvvrjLcuygaOyA8Cc/+cmg1traGtSuvfbaoHbOOecE\ntR/84Addlo899thgHUd/pGvS7LCUwXuWav3/6OjoCGrVZ3p49dVXg3WWLl0a1GKTJN55552gts02\n29RU22233bosT5gwIVinra0tqMUu8eGBpxz1qUGRHGtmq/PFfwTwdHFDEumel2DVSlmSZvCSo1qm\nmd8M4CgAI0m2AbgMwFEkDwRgAJYC+EaJYxR5j5dgxShLkgovOaplFt+pkfINJYxFpFdeghWjLEkq\nvORIZ5IQV7wESyRlXnKkBtWDb37zm0Gt+lT/W7ZsCdaZPn16UItNiIj51re+FdRikzWOPvroLsuH\nHHJIsM7DDz9c02N64eng7kBU65kTYrXYxIbqMzZUn9EBiJ+BZeXKlUEtdjmPsWPHBrX9998/qFVf\n5uONN94I1lm4cGFQi70WPbw+PeVIDUpc8RIskZR5yZEalLjiJVgiKfOSIzUoccVLsERS5iVHalC5\nk046Kah98Ytf7PXnqs9uDgA333xzn8cR+0Djt7/97aB2zz33dFk+77zzgnVOP/30Po8jVV6C5V3s\nONJWW4UnnonVYsebdtppp6C26667dlmOfdg2drzphRdeCGqxD8dv3LgxqB1wwAFBrfqDurEP4Ho9\n3tQdL2NXgxI3PB3cFUmVpxypQYkrXoIlkjIvOVKDEle8BEskZV5y1J/LbYg0XNFnYCY5heRzJFtJ\nzuxhvUNIdpD8fGFPRqRJyjibeRlZ0hZUbvLk8AoGsQPF1WJnOS7apk2bglr1B4Rjl6keiIp850dy\nCIDrABwLoA3AQpLzzOyZyHo/BvCfhT14Qmr9sO2QIUOCWmxCxIgRI4La+PHjg9rIkSO7LK9fvz5Y\nZ9myZUFtzZo1QS32uohd8j02tlGjRnVZXr58ebBO7AP5XrZCYooee1lZ0haUuFHCu75DAbSa2Ytm\n9i6AWwBMjax3HoDfA1hX3LMRaY56c9TMLKlBiSt1hmpk5xVo86/qc1CNA7CiYrktr72H5Dhkl8G4\nvsznJdJIfWhQTcmSdvGJK3Xummi3/l999icALjSzLbXs8hXxoA+7+JqSJTUocaXgfecrAVQeHNkj\nr1WaDOCWPFAjAXyGZIeZ3V7kQEQaqYTjZ6VkqZYLFo4HcCOAMcguqtZiZv9K8gMAfgtgArILrZ1i\nZj6vgdwPq1atKv0xqi+NDYQHbT/72c+WPo4UFByshQD2ITkRWZimAfhS1eNN7Pye5FwA8/vanFLN\nUn8mRMTOEDFmzJigFjuzePWEhViWYmeIiL0GYpOE9t1336AWu5x79VnUY2czj2VQkyS6KCVLtRyD\n6gBwgZlNAnA4gHNITgIwE8A9ZrYPgHvyZZHSFH1g18w6AJwL4E4ASwDcamaLSZ5N8uwSnoKyJE1X\nxiSJsrJUyxV1VwNYnX+/geQSZAe/piK7fDUA/ArAvQAu7OtARGpR9Ds/M1sAYEFVbXY36/5TPx9L\nWZIklLH1V0aW6joGRXICgIMA/A3AmDxwALAG2W6L2M9MBxBewU+kDzzvVqmkLEkzeclRzQ2K5A7I\n5q/PMLP1lfutzcxIRp+xmbUAaMnvw8dvRZLlJVg9UZak2bzkqKYGRXIYskDdZGa35eW1JMea2WqS\nYzFIP8R48sknB7XHHnssqMU+JR+z3XbbBbVLLrkkqA0d2vW/7s4776zp/r3zEqzupJClWqb41noZ\njdjrdZdddglq1ZfWAIDRo0d3WY5l5O23367pvqovmQEAxxxzTFCrPnsFALS3t3dZjl3efcOGDUHN\n82vRy9h7nSTB7NV8A4AlZnZ1xU3zAHw1//6rAP5Y/PBE3lfCp98bSlmSFJQxSaIstWxBHQHgdABP\nkXw8r10E4AoAt5I8E8AyAKeUM0SR96XYeOqgLEkSvOSolll89wPobp/Ap4sdjkjPvAQrRlmSVHjJ\nkc4kIa54CZZIyrzkSA0q9/DDDwe1d955J6hts802XZbPP//8YJ1TTgn30Nxxxx1BLXaw96CDDgpq\nsUsVVPvTn/7U6zoDgZdgpaz6dxibEBGbSBFbrzoPQHziROwyF3vuuWeX5b333jscbERsHLVOzIid\nEWLRokVdlmOTJDZu3BjUPL8WvYxdDUrcaPYBW5GBwFOO1KDEFS/BEkmZlxypQYkrXoIlkjIvOVKD\nEle8BEskZV5ypAaVi00y+PGPfxzULr300l7vK3ZpgTPOOKNvA+vGrFmzuiy3tLQUev+p8hIsT2K/\n01onScRUXwoGiJ8RovpsKJMmTQrW2XnnnWsaR+z+V6xYEdTuv//+oDZ//vwuy7HLfmzevDmoeeYl\nR2pQ4oang7siqfKUIzUoccVLsERS5iVHalDiipdgiaTMS47UoMQVL8ESSZmXHKlB9eDyyy8Panff\nfXeX5RkzZgTrfO5zn+vzY951111Bbfbs8KKUt99+e5dlLy+4/vC079y7TZs2BbU333wzqFVPdACA\nl156KajVMsEidrmN2GU0Ytra2oLaX/7yl6D26KOPBrXly5d3WY6dQWYgve485UgNSlzxEiyRlHnJ\nkRqUuOIlWCIp85IjNShxxUuwRFLmJUe9NiiS4wHcCGAMAAPQYmb/SvJ7AL4O4OV81YvMbEFZA22G\n2Ifzqj/oF/vgn5THS7BiUs1S7IO1sVpHR0dQi50dfO3atUHt+eefD2p//vOfuyz358PBsbHV+rw8\nv6b6ystzrmULqgPABWb2KMkdATxCsvNI/jVmdlV5wxN5n6eDu91QlqTpPOWolivqrgawOv9+A8kl\nAMaVPTCRGC/BilGWJBVeclTb9nOO5AQABwH4W146j+STJOeQDK8Ylv3MdJKLSC6K3S5Sj853f7V8\npUxZkmaqJ0fNzFLNDYrkDgB+D2CGma0HcD2ADwI4ENm7wlmxnzOzFjObbGaTCxivDHIeQtUbZUma\nzUuDqmkWH8lhyAJ1k5ndBgBmtrbi9l8AmN/Nj4sUJuXGU4uBlqXY/0dsctFAOxu4d15yVMssPgK4\nAcASM7vqd143AAAHdklEQVS6oj4236cOAP8I4OlyhiiSafa7uf5SliQFnnJUyxbUEQBOB/AUycfz\n2kUATiV5ILLpsksBfKOUEYpU8BKsbihLkgQvOaplFt/9AMIPKAAD6jNP4oOXYMUoS5IKLznSmSTE\nFS/BEkmZlxypQYkrXoIlkjIvOVKDEjc8HdwVSZWnHKlBiStegiWSMi85UoMSV7wESyRlXnKkBiWu\neAmWSMq85KjRDaodwDIAI/PvPfP+HFIb/161rOQlWA0wULLkffxAes+h1yx5yVFDG5SZjQIAkou8\nn0/M+3PwOH5PB3fLNlCy5H38gL/n4ClH2sUnrngJlkjKvORIDUpc8RIskZR5yVGzGlRLkx63SN6f\ng8vxewlWA7n8f6zgffyAw+fgJUdNaVBm5u4/tJr35+B1/F6C1She/x87eR8/4PM5eMlRXVfUFWmm\nMi6yRnIKyedItpKcGbn9tPxKt0+RfIDkAYU/MZEGqjdHzcySjkGJK0W+8yM5BMB1AI4F0AZgIcl5\nZvZMxWovAfhvZvYayROQ7c45rLBBiDRB0VtQZWWp4VtQvXXZ1JCcQ3Idyacrah8geRfJ/5f/u0sz\nx9gTkuNJ/oXkMyQXkzw/r7t5DpUKftd3KIBWM3vRzN4FcAuAqVWP94CZvZYvPgRgj0KfUB95yxGg\nLKWkhC2oUrLU0AZV0WVPADAJ2YXaJjVyDH0wF8CUqtpMAPeY2T4A7smXU9UB4AIzmwTgcADn5L9z\nT8/hPQWHahyAFRXLbXmtO2cC+HM/hl8IpzkClKVklNCgSslSo7egeu2yqTGz+wC8WlWeCuBX+fe/\nAnByQwdVBzNbbWaP5t9vALAE2QvHzXOoVGeoRpJcVPE1va+PS/JTyEJ1YVHPpR/c5QhQllLShwbV\nlCw1+hhUrMt63J8/xsxW59+vATCmmYOpFckJAA4C8Dc4fA71HLDNtVvPn/BfCWB8xfIeea0Lkv8A\n4JcATjCzV+oZQEkGSo4Ah69DwHeW+pAjoElZ0iy+frLsfzr5OZskdwDwewAzzGx95W1engNQ+C6+\nhQD2ITmR5NYApgGYV7kCyT0B3AbgdDN7vvAnJO/x8jocCFkqYRdfKVlq9BZUTV3WgbUkx5rZapJj\nAaxr9oB6QnIYskDdZGa35WVXz6FTH9759XRfHSTPBXAngCEA5pjZYpJn57fPBnApgF0B/BtJAOjo\n5Z1kIwyUHAHOXocDJUtF5ii/v1Ky1OgG9V6XRRaoaQC+1OAxFGEegK8CuCL/94/NHU73mL0SbgCw\nxMyurrjJzXOoVEKwFgBYUFWbXfH9WQDOKvRB+2+g5Ahw9DocSFkqOkf5fRaepUafzTzaZRs5hnqR\nvBnAUcgOErYBuAzZC/FWkmciu+TBKc0bYa+OAHA6gKdIPp7XLoKv5/CeMoLljcccAcpSSrzkiF4G\nKjJ8+HDbe++9a17/6aeffiSB3XEiSak3R0DzsqQzSYgrekMl0n9ecqQGJa54CZZIyrzkSA1KXPES\nLJGUecmRGpS44iVYIinzkiM1KHGjj5+AF5EKnnKkBiWueAmWSMq85EgNSlzxEiyRlHnJkRqUuOIl\nWCIp85IjNShxxUuwRFLmJUdqUOKGp4O7IqnylCM1KHHFS7BEUuYlR2pQ4oqXYImkzEuO1KDEFS/B\nEkmZlxypQYkrXoIlkjIvOVKDEjc8HdwVSZWnHKlBiStegiWSMi85UoMSV7wESyRlXnKkBiWueAmW\nSMq85EgNSlzxEiyRlHnJkRqUuOHp4K5IqjzlSA1KXPESLJGUecmRGpS44iVYIinzkiM1KHHFS7BE\nUuYlR2pQ4oqXYImkzEuO1KDEDU8Hd0VS5SlHalDiipdgiaTMS47UoMQVL8ESSZmXHKlBiStegiWS\nMi85UoMSV7wESyRlXnKkBiVueDq4K5IqTzlSgxJXvARLJGVecqQGJa54CZZIyrzkSA1KXPESLJGU\necmRGpS44iVYIinzkiM1KHHD08FdkVR5ypEalLjiJVgiKfOSIzUoccVLsERS5iVHalDiipdgiaTM\nS47UoMQVL8ESSZmXHKlBiRueDu6KpMpTjtSgxBUvwRJJmZccbdXsAYjUo/PdXy1ftSA5heRzJFtJ\nzozcTpI/zW9/kuTHCn9SIg1WT46amSU1KHGlyFCRHALgOgAnAJgE4FSSk6pWOwHAPvnXdADXF/uM\nRBqv6AZVVpbUoMSVgt/1HQqg1cxeNLN3AdwCYGrVOlMB3GiZhwCMIDm22Gcl0lglbEGVkiUdgxJP\n7gQwso71tyW5qGK5xcxaKpbHAVhRsdwG4LCq+4itMw7A6jrGIZKSenMENClLalDihplNafYYRLzz\nlCPt4pPBbCWA8RXLe+S1etcRGexKyZIalAxmCwHsQ3Iiya0BTAMwr2qdeQC+ks9AOhzAG2am3Xsi\nXZWSJe3ik0HLzDpInotsn/wQAHPMbDHJs/PbZwNYAOAzAFoBvAXgjGaNVyRVZWWJXj6wJSIig4t2\n8YmISJLUoEREJElqUCIikiQ1KBERSZIalIiIJEkNSkREkqQGJSIiSfr/uGXNdw0FjP4AAAAASUVO\nRK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x130f1ed30>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "x_sample = mnist.test.next_batch(minibatch_size)[0]\n",
    "x_reconstruct = p_mu.eval(session=sess, feed_dict={x:x_sample})[0]\n",
    "plt.subplot(1, 2, 1)\n",
    "plt.imshow(x_sample[0].reshape(28, 28), vmin=0, vmax=1, cmap=\"gray\")\n",
    "plt.title(\"Test input\")\n",
    "plt.colorbar()\n",
    "plt.subplot(1, 2, 2)\n",
    "plt.imshow(x_reconstruct.reshape(28, 28), vmin=0, vmax=1, cmap=\"gray\")\n",
    "plt.title(\"Reconstruction\")\n",
    "plt.colorbar()\n",
    "plt.tight_layout()"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.0"
  },
  "toc": {
   "colors": {
    "hover_highlight": "#DAA520",
    "navigate_num": "#000000",
    "navigate_text": "#333333",
    "running_highlight": "#FF0000",
    "selected_highlight": "#FFD700",
    "sidebar_border": "#EEEEEE",
    "wrapper_background": "#FFFFFF"
   },
   "moveMenuLeft": true,
   "nav_menu": {
    "height": "66px",
    "width": "252px"
   },
   "navigate_menu": true,
   "number_sections": true,
   "sideBar": true,
   "threshold": 4,
   "toc_cell": true,
   "toc_section_display": "block",
   "toc_window_display": false,
   "widenNotebook": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}