DL2021_HW1.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "DL2021 HW1",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/miiiiiko/Test/blob/main/DL2021_HW1.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "mz0_QVkxCrX3"
      },
      "source": [
        "# **Homework 1: COVID-19 Cases Prediction (Regression)**"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ZeZnPAiwDRWG"
      },
      "source": [
        "Author: Heng-Jui Chang\n",
        "\n",
        "Slides: https://github.com/ga642381/ML2021-Spring/blob/main/HW01/HW01.pdf  \n",
        "Videos (Mandarin): https://cool.ntu.edu.tw/courses/4793/modules/items/172854  \n",
        "https://cool.ntu.edu.tw/courses/4793/modules/items/172853  \n",
        "Video (English): https://cool.ntu.edu.tw/courses/4793/modules/items/176529\n",
        "\n",
        "\n",
        "Objectives:\n",
        "* Solve a regression problem with deep neural networks (DNN).\n",
        "* Understand basic DNN training tips.\n",
        "* Get familiar with PyTorch.\n",
        "\n",
        "If any questions, please contact the TAs via TA hours, NTU COOL, or email.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Jx3x1nDkG-Uy"
      },
      "source": [
        "# **Download Data**\n",
        "\n",
        "\n",
        "If the Google drive links are dead, you can download data from [kaggle](https://www.kaggle.com/c/ml2021spring-hw1/data), and upload data manually to the workspace."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tMj55YDKG6ch",
        "outputId": "2c3d1b7f-e42f-40ee-efc8-aef1d386d9c4"
      },
      "source": [
        "tr_path = 'covid.train.csv'  # path to training data\n",
        "tt_path = 'covid.test.csv'   # path to testing data\n",
        "\n",
        "!gdown --id '19CCyCgJrUxtvgZF53vnctJiOJ23T5mqF' --output covid.train.csv\n",
        "!gdown --id '1CE240jLm2npU-tdz81-oVKEF3T2yfT1O' --output covid.test.csv"
      ],
      "execution_count": 29,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/gdown/cli.py:131: FutureWarning: Option `--id` was deprecated in version 4.3.1 and will be removed in 5.0. You don't need to pass it anymore to use a file ID.\n",
            "  category=FutureWarning,\n",
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=19CCyCgJrUxtvgZF53vnctJiOJ23T5mqF\n",
            "To: /content/covid.train.csv\n",
            "100% 2.00M/2.00M [00:00<00:00, 224MB/s]\n",
            "/usr/local/lib/python3.7/dist-packages/gdown/cli.py:131: FutureWarning: Option `--id` was deprecated in version 4.3.1 and will be removed in 5.0. You don't need to pass it anymore to use a file ID.\n",
            "  category=FutureWarning,\n",
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1CE240jLm2npU-tdz81-oVKEF3T2yfT1O\n",
            "To: /content/covid.test.csv\n",
            "100% 651k/651k [00:00<00:00, 123MB/s]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "wS_4-77xHk44"
      },
      "source": [
        "# **Import Some Packages**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "k-onQd4JNA5H"
      },
      "source": [
        "# PyTorch\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "from torch.utils.data import Dataset, DataLoader\n",
        "\n",
        "# For data preprocess\n",
        "import numpy as np\n",
        "import csv\n",
        "import os\n",
        "\n",
        "# For plotting\n",
        "import matplotlib.pyplot as plt\n",
        "from matplotlib.pyplot import figure\n",
        "\n",
        "myseed = 42069  # set a random seed for reproducibility\n",
        "torch.backends.cudnn.deterministic = True\n",
        "torch.backends.cudnn.benchmark = False\n",
        "np.random.seed(myseed)\n",
        "torch.manual_seed(myseed)\n",
        "if torch.cuda.is_available():\n",
        "    torch.cuda.manual_seed_all(myseed)"
      ],
      "execution_count": 30,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "BtE3b6JEH7rw"
      },
      "source": [
        "# **Some Utilities**\n",
        "\n",
        "You do not need to modify this part."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FWMT3uf1NGQp"
      },
      "source": [
        "def get_device():\n",
        "    ''' Get device (if GPU is available, use GPU) '''\n",
        "    return 'cuda' if torch.cuda.is_available() else 'cpu'\n",
        "\n",
        "def plot_learning_curve(loss_record, title=''):\n",
        "    ''' Plot learning curve of your DNN (train & dev loss) '''\n",
        "    total_steps = len(loss_record['train'])\n",
        "    x_1 = range(total_steps)\n",
        "    x_2 = x_1[::len(loss_record['train']) // len(loss_record['dev'])]\n",
        "    figure(figsize=(6, 4))\n",
        "    plt.plot(x_1, loss_record['train'], c='tab:red', label='train')\n",
        "    plt.plot(x_2, loss_record['dev'], c='tab:cyan', label='dev')\n",
        "    plt.ylim(0.0, 5.)\n",
        "    plt.xlabel('Training steps')\n",
        "    plt.ylabel('MSE loss')\n",
        "    plt.title('Learning curve of {}'.format(title))\n",
        "    plt.legend()\n",
        "    plt.show()\n",
        "\n",
        "\n",
        "def plot_pred(dv_set, model, device, lim=35., preds=None, targets=None):\n",
        "    ''' Plot prediction of your DNN '''\n",
        "    if preds is None or targets is None:\n",
        "        model.eval()\n",
        "        preds, targets = [], []\n",
        "        for x, y in dv_set:\n",
        "            x, y = x.to(device), y.to(device)\n",
        "            with torch.no_grad():\n",
        "                pred = model(x)\n",
        "                preds.append(pred.detach().cpu())\n",
        "                targets.append(y.detach().cpu())\n",
        "        preds = torch.cat(preds, dim=0).numpy()\n",
        "        targets = torch.cat(targets, dim=0).numpy()\n",
        "\n",
        "    figure(figsize=(5, 5))\n",
        "    plt.scatter(targets, preds, c='r', alpha=0.5)\n",
        "    plt.plot([-0.2, lim], [-0.2, lim], c='b')\n",
        "    plt.xlim(-0.2, lim)\n",
        "    plt.ylim(-0.2, lim)\n",
        "    plt.xlabel('ground truth value')\n",
        "    plt.ylabel('predicted value')\n",
        "    plt.title('Ground Truth v.s. Prediction')\n",
        "    plt.show()"
      ],
      "execution_count": 31,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "39U_XFX6KOoj"
      },
      "source": [
        "# **Preprocess**\n",
        "\n",
        "We have three kinds of datasets:\n",
        "* `train`: for training\n",
        "* `dev`: for validation\n",
        "* `test`: for testing (w/o target value)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "TQ-MdwpLL7Dt"
      },
      "source": [
        "## **Dataset**\n",
        "\n",
        "The `COVID19Dataset` below does:\n",
        "* read `.csv` files\n",
        "* extract features\n",
        "* split `covid.train.csv` into train/dev sets\n",
        "* normalize features\n",
        "\n",
        "Finishing `TODO` below might make you pass medium baseline."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "0zlpIp9ANJRU"
      },
      "source": [
        "class COVID19Dataset(Dataset):\n",
        "    ''' Dataset for loading and preprocessing the COVID19 dataset '''\n",
        "    def __init__(self,\n",
        "                 path,\n",
        "                 mode='train',\n",
        "                 target_only=False):\n",
        "        self.mode = mode\n",
        "\n",
        "        # Read data into numpy arrays\n",
        "        with open(path, 'r') as fp:\n",
        "            data = list(csv.reader(fp))\n",
        "            data = np.array(data[1:])[:, 1:].astype(float)\n",
        "        \n",
        "        if not target_only:\n",
        "            feats = list(range(93))\n",
        "        else:\n",
        "            # TODO: Using 40 states & 2 tested_positive features (indices = 57 & 75)\n",
        "            feats = list(range(40)) + [57, 75] # dim = 42\n",
        "\n",
        "        if mode == 'test':\n",
        "            # Testing data\n",
        "            # data: 893 x 93 (40 states + day 1 (18) + day 2 (18) + day 3 (17))\n",
        "            data = data[:, feats]\n",
        "            self.data = torch.FloatTensor(data)\n",
        "        else:\n",
        "            # Training data (train/dev sets)\n",
        "            # data: 2700 x 94 (40 states + day 1 (18) + day 2 (18) + day 3 (18))\n",
        "            target = data[:, -1]\n",
        "            data = data[:, feats]\n",
        "            \n",
        "            # Splitting training data into train & dev sets\n",
        "            if mode == 'train':\n",
        "                indices = [i for i in range(len(data)) if i % 10 != 0]\n",
        "            elif mode == 'dev':\n",
        "                indices = [i for i in range(len(data)) if i % 10 == 0]\n",
        "            \n",
        "            # Convert data into PyTorch tensors\n",
        "            self.data = torch.FloatTensor(data[indices])\n",
        "            self.target = torch.FloatTensor(target[indices])\n",
        "\n",
        "        # Normalize features (you may remove this part to see what will happen)\n",
        "        self.data[:, 40:] = \\\n",
        "            (self.data[:, 40:] - self.data[:, 40:].mean(dim=0, keepdim=True)) \\\n",
        "            / self.data[:, 40:].std(dim=0, keepdim=True)\n",
        "\n",
        "        self.dim = self.data.shape[1]\n",
        "\n",
        "        print('Finished reading the {} set of COVID19 Dataset ({} samples found, each dim = {})'\n",
        "              .format(mode, len(self.data), self.dim))\n",
        "\n",
        "    def __getitem__(self, index):\n",
        "        # Returns one sample at a time\n",
        "        if self.mode in ['train', 'dev']:\n",
        "            # For training\n",
        "            return self.data[index], self.target[index]\n",
        "        else:\n",
        "            # For testing (no target)\n",
        "            return self.data[index]\n",
        "\n",
        "    def __len__(self):\n",
        "        # Returns the size of the dataset\n",
        "        return len(self.data)"
      ],
      "execution_count": 32,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AlhTlkE7MDo3"
      },
      "source": [
        "## **DataLoader**\n",
        "\n",
        "A `DataLoader` loads data from a given `Dataset` into batches.\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hlhLk5t6MBX3"
      },
      "source": [
        "def prep_dataloader(path, mode, batch_size, n_jobs=0, target_only=False):\n",
        "    ''' Generates a dataset, then is put into a dataloader. '''\n",
        "    dataset = COVID19Dataset(path, mode=mode, target_only=target_only)  # Construct dataset\n",
        "    dataloader = DataLoader(\n",
        "        dataset, batch_size,\n",
        "        shuffle=(mode == 'train'), drop_last=False,\n",
        "        num_workers=n_jobs, pin_memory=True)                            # Construct dataloader\n",
        "    return dataloader"
      ],
      "execution_count": 33,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SGuycwR0MeQB"
      },
      "source": [
        "# **Deep Neural Network**\n",
        "\n",
        "`NeuralNet` is an `nn.Module` designed for regression.\n",
        "The DNN consists of 2 fully-connected layers with ReLU activation.\n",
        "This module also included a function `cal_loss` for calculating loss.\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "DvFWVjZ5Nvga"
      },
      "source": [
        "# **Train/Dev/Test**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "49-uXYovOAI0"
      },
      "source": [
        "class NeuralNet(nn.Module):\n",
        "    ''' A simple fully-connected deep neural network '''\n",
        "    def __init__(self, input_dim):\n",
        "        super(NeuralNet, self).__init__()\n",
        "\n",
        "        # Define your neural network here\n",
        "        # TODO: How to modify this model to achieve better performance?\n",
        "        self.net = nn.Sequential(\n",
        "            nn.Linear(input_dim, 64),\n",
        "            nn.BatchNorm1d(64),\n",
        "            nn.Dropout(p=0.2),\n",
        "            nn.ReLU(),\n",
        "            nn.Linear(64, 1)\n",
        "        )\n",
        "\n",
        "        # Mean squared error loss\n",
        "        self.criterion = nn.MSELoss(reduction='mean')\n",
        "\n",
        "    def forward(self, x):\n",
        "        ''' Given input of size (batch_size x input_dim), compute output of the network '''\n",
        "        return self.net(x).squeeze(1)\n",
        "\n",
        "    def cal_loss(self, pred, target):\n",
        "        ''' Calculate loss '''\n",
        "        # TODO: you may implement L1/L2 regularization here\n",
        "        regularization = 0\n",
        "        for param in model.parameters():\n",
        "          regularization += torch.sum(param**2)\n",
        "        return self.criterion(pred, target) + config['regularization'] * regularization"
      ],
      "execution_count": 55,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "MAM8QecJOyqn"
      },
      "source": [
        "## **Training**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "lOqcmYzMO7jB"
      },
      "source": [
        "def train(tr_set, dv_set, model, config, device):\n",
        "    ''' DNN training '''\n",
        "\n",
        "    n_epochs = config['n_epochs']  # Maximum number of epochs\n",
        "\n",
        "    # Setup optimizer\n",
        "    optimizer = getattr(torch.optim, config['optimizer'])(\n",
        "        model.parameters(), **config['optim_hparas'])\n",
        "\n",
        "    min_mse = 1000.\n",
        "    loss_record = {'train': [], 'dev': []}      # for recording training loss\n",
        "    early_stop_cnt = 0\n",
        "    epoch = 0\n",
        "    while epoch < n_epochs:\n",
        "        model.train()                           # set model to training mode\n",
        "        for x, y in tr_set:                     # iterate through the dataloader\n",
        "            optimizer.zero_grad()               # set gradient to zero\n",
        "            x, y = x.to(device), y.to(device)   # move data to device (cpu/cuda)\n",
        "            pred = model(x)                     # forward pass (compute output)\n",
        "            mse_loss = model.cal_loss(pred, y)  # compute loss\n",
        "            mse_loss.backward()                 # compute gradient (backpropagation)\n",
        "            optimizer.step()                    # update model with optimizer\n",
        "            loss_record['train'].append(mse_loss.detach().cpu().item())\n",
        "\n",
        "        # After each epoch, test your model on the validation (development) set.\n",
        "        dev_mse = dev(dv_set, model, device)\n",
        "        if dev_mse < min_mse:\n",
        "            # Save model if your model improved\n",
        "            min_mse = dev_mse\n",
        "            print('Saving model (epoch = {:4d}, loss = {:.4f})'\n",
        "                .format(epoch + 1, min_mse))\n",
        "            torch.save(model.state_dict(), config['save_path'])  # Save model to specified path\n",
        "            early_stop_cnt = 0\n",
        "        else:\n",
        "            early_stop_cnt += 1\n",
        "\n",
        "        epoch += 1\n",
        "        loss_record['dev'].append(dev_mse)\n",
        "        if early_stop_cnt > config['early_stop']:\n",
        "            print(\"early stop\")\n",
        "            # Stop training if your model stops improving for \"config['early_stop']\" epochs.\n",
        "            break\n",
        "\n",
        "    print('Finished training after {} epochs'.format(epoch))\n",
        "    return min_mse, loss_record"
      ],
      "execution_count": 35,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "0hSd4Bn3O2PL"
      },
      "source": [
        "## **Validation**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "yrxrD3YsN3U2"
      },
      "source": [
        "def dev(dv_set, model, device):\n",
        "    model.eval()                                # set model to evalutation mode\n",
        "    total_loss = 0\n",
        "    for x, y in dv_set:                         # iterate through the dataloader\n",
        "        x, y = x.to(device), y.to(device)       # move data to device (cpu/cuda)\n",
        "        with torch.no_grad():                   # disable gradient calculation\n",
        "            pred = model(x)                     # forward pass (compute output)\n",
        "            mse_loss = model.cal_loss(pred, y)  # compute loss\n",
        "        total_loss += mse_loss.detach().cpu().item() * len(x)  # accumulate loss\n",
        "    total_loss = total_loss / len(dv_set.dataset)              # compute averaged loss\n",
        "\n",
        "    return total_loss"
      ],
      "execution_count": 36,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "g0pdrhQAO41L"
      },
      "source": [
        "## **Testing**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "aSBMRFlYN5tB"
      },
      "source": [
        "def test(tt_set, model, device):\n",
        "    model.eval()                                # set model to evalutation mode\n",
        "    preds = []\n",
        "    for x in tt_set:                            # iterate through the dataloader\n",
        "        x = x.to(device)                        # move data to device (cpu/cuda)\n",
        "        with torch.no_grad():                   # disable gradient calculation\n",
        "            pred = model(x)                     # forward pass (compute output)\n",
        "            preds.append(pred.detach().cpu())   # collect prediction\n",
        "    preds = torch.cat(preds, dim=0).numpy()     # concatenate all predictions and convert to a numpy array\n",
        "    return preds"
      ],
      "execution_count": 37,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "SvckkF5dvf0j"
      },
      "source": [
        "# **Setup Hyper-parameters**\n",
        "\n",
        "`config` contains hyper-parameters for training and the path to save your model."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "NPXpdumwPjE7"
      },
      "source": [
        "device = get_device()                 # get the current available device ('cpu' or 'cuda')\n",
        "os.makedirs('models', exist_ok=True)  # The trained model will be saved to ./models/\n",
        "target_only = False                   # TODO: Using 40 states & 2 tested_positive features\n",
        "\n",
        "# TODO: How to tune these hyper-parameters to improve your model's performance?\n",
        "config = {\n",
        "    'n_epochs': 3000,                # maximum number of epochs\n",
        "    'batch_size': 270,               # mini-batch size for dataloader\n",
        "    'optimizer': 'SGD',              # optimization algorithm (optimizer in torch.optim)\n",
        "    'optim_hparas': {                # hyper-parameters for the optimizer (depends on which optimizer you are using)\n",
        "        'lr': 0.001,                 # learning rate of SGD\n",
        "        'momentum': 0.9              # momentum for SGD\n",
        "    },\n",
        "    'early_stop': 200,               # early stopping epochs (the number epochs since your model's last improvement)\n",
        "    'save_path': 'models/model.pth',\n",
        "    'regularization': 0.00075                       # your model will be saved here\n",
        "}"
      ],
      "execution_count": 51,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "6j1eOV3TOH-j"
      },
      "source": [
        "# **Load data and model**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "eNrYBMmePLKm",
        "outputId": "0583e767-3cff-4ae2-eef2-5f3ed7cdd46f"
      },
      "source": [
        "tr_set = prep_dataloader(tr_path, 'train', config['batch_size'], target_only=target_only)\n",
        "dv_set = prep_dataloader(tr_path, 'dev', config['batch_size'], target_only=target_only)\n",
        "tt_set = prep_dataloader(tt_path, 'test', config['batch_size'], target_only=target_only)"
      ],
      "execution_count": 39,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Finished reading the train set of COVID19 Dataset (2430 samples found, each dim = 93)\n",
            "Finished reading the dev set of COVID19 Dataset (270 samples found, each dim = 93)\n",
            "Finished reading the test set of COVID19 Dataset (893 samples found, each dim = 93)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FHylSirLP9oh"
      },
      "source": [
        "model = NeuralNet(tr_set.dataset.dim).to(device)  # Construct model and move to device"
      ],
      "execution_count": 61,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sX2B_zgSOPTJ"
      },
      "source": [
        "# **Start Training!**"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "model_loss, model_loss_record = train(tr_set, dv_set, model, config, device)"
      ],
      "metadata": {
        "id": "ixwu-L9fAvXR"
      },
      "execution_count": 62,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GrEbUxazQAAZ",
        "outputId": "5b8ee42d-ecb7-4736-fd5d-cef3d5bdb914"
      },
      "source": [
        "model_loss, model_loss_record = train(tr_set, dv_set, model, config, device)"
      ],
      "execution_count": 62,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Saving model (epoch =    1, loss = 49.1548)\n",
            "Saving model (epoch =    3, loss = 13.1899)\n",
            "Saving model (epoch =    4, loss = 10.8707)\n",
            "Saving model (epoch =    5, loss = 8.3246)\n",
            "Saving model (epoch =    6, loss = 5.8061)\n",
            "Saving model (epoch =    7, loss = 5.4883)\n",
            "Saving model (epoch =    8, loss = 4.7650)\n",
            "Saving model (epoch =    9, loss = 4.0655)\n",
            "Saving model (epoch =   10, loss = 3.8010)\n",
            "Saving model (epoch =   11, loss = 3.6722)\n",
            "Saving model (epoch =   12, loss = 3.3060)\n",
            "Saving model (epoch =   13, loss = 3.0589)\n",
            "Saving model (epoch =   14, loss = 2.9512)\n",
            "Saving model (epoch =   15, loss = 2.9056)\n",
            "Saving model (epoch =   16, loss = 2.4979)\n",
            "Saving model (epoch =   17, loss = 2.4285)\n",
            "Saving model (epoch =   19, loss = 2.3508)\n",
            "Saving model (epoch =   20, loss = 2.2366)\n",
            "Saving model (epoch =   22, loss = 2.0815)\n",
            "Saving model (epoch =   23, loss = 1.9689)\n",
            "Saving model (epoch =   24, loss = 1.9365)\n",
            "Saving model (epoch =   28, loss = 1.8741)\n",
            "Saving model (epoch =   29, loss = 1.7867)\n",
            "Saving model (epoch =   31, loss = 1.7270)\n",
            "Saving model (epoch =   36, loss = 1.6912)\n",
            "Saving model (epoch =   38, loss = 1.5725)\n",
            "Saving model (epoch =   42, loss = 1.5449)\n",
            "Saving model (epoch =   44, loss = 1.4558)\n",
            "Saving model (epoch =   48, loss = 1.3427)\n",
            "Saving model (epoch =   56, loss = 1.3323)\n",
            "Saving model (epoch =   58, loss = 1.3031)\n",
            "Saving model (epoch =   61, loss = 1.2681)\n",
            "Saving model (epoch =   68, loss = 1.2112)\n",
            "Saving model (epoch =   72, loss = 1.1835)\n",
            "Saving model (epoch =   73, loss = 1.1750)\n",
            "Saving model (epoch =   77, loss = 1.1518)\n",
            "Saving model (epoch =   89, loss = 1.1383)\n",
            "Saving model (epoch =   94, loss = 1.1218)\n",
            "Saving model (epoch =  101, loss = 1.0869)\n",
            "Saving model (epoch =  108, loss = 1.0693)\n",
            "Saving model (epoch =  114, loss = 1.0583)\n",
            "Saving model (epoch =  121, loss = 1.0553)\n",
            "Saving model (epoch =  125, loss = 1.0352)\n",
            "Saving model (epoch =  147, loss = 1.0072)\n",
            "Saving model (epoch =  192, loss = 1.0034)\n",
            "Saving model (epoch =  201, loss = 0.9831)\n",
            "Saving model (epoch =  233, loss = 0.9826)\n",
            "Saving model (epoch =  244, loss = 0.9699)\n",
            "Saving model (epoch =  289, loss = 0.9677)\n",
            "Saving model (epoch =  342, loss = 0.9551)\n",
            "Saving model (epoch =  364, loss = 0.9480)\n",
            "Saving model (epoch =  381, loss = 0.9468)\n",
            "Saving model (epoch =  413, loss = 0.9399)\n",
            "Saving model (epoch =  433, loss = 0.9351)\n",
            "Saving model (epoch =  456, loss = 0.9271)\n",
            "Saving model (epoch =  565, loss = 0.9267)\n",
            "Saving model (epoch =  579, loss = 0.9139)\n",
            "Saving model (epoch =  637, loss = 0.9041)\n",
            "Saving model (epoch =  754, loss = 0.8990)\n",
            "Saving model (epoch =  851, loss = 0.8930)\n",
            "early stop\n",
            "Finished training after 1052 epochs\n",
            "0.8930309414863586\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 313
        },
        "id": "hsNO9nnXQBvP",
        "outputId": "9cb0eb0c-2918-41de-e7cb-2cc8b82fc2e1"
      },
      "source": [
        "plot_learning_curve(model_loss_record, title='deep model')\n",
        "print(len(model_loss_record['train']))"
      ],
      "execution_count": 64,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXgAAAEWCAYAAABsY4yMAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3hUVfrHv+9MJj2kUUIIJfQICNIUKYquyqJiF3XtvS4qKqir4q593RXLomJZ3Z+K2Ct2QVAEDUgJJBBSSA/pPZl2fn/cO5Mpd2buzNw7M7k5n+fJkzv3nvLe9p5z3/Oe9xBjDBwOh8PRHrpwC8DhcDgcdeAKnsPhcDQKV/AcDoejUbiC53A4HI3CFTyHw+FoFK7gORwOR6NwBc9RBCKaT0QHwi1HpEBEc4mokIjaiehsGenfIKJHQiFbqCCiTUR0rcy0jIjGqi1Tf4MreA1ARKVE9KdwysAY28IYmxBOGSKMvwN4gTGWyBj7JNzCcPonXMFzZEFE+nDLECwhPoeRAPaFsD4Oxw2u4DUMEemIaCURFRFRAxG9R0RpDsffJ6IaImohos1ENMnh2BtE9CIRbSCiDgALxS+Fu4hoj5hnPRHFiulPJKIKh/we04rH7yGiaiKqIqJrvX2iE1EaEf1XTNtERJ+I+68kop9d0trLkTiHu8Tz1TukP4eI9si5XhJyXUdEh4iokYg+I6JMcX8RgNEAPhdNNDESeY8hop1E1EZE6wHEuhw/g4h2EVEzEW0loqMdjmUS0YdEVEdEJUT0V4djq4joA/F6t4l1TPVyDoyIbhbNSW1E9A8iGiPW2Speg2hf5yweO4WICsT7/QIAcqnraiLKF+/hN0Q00pNcHIVgjPG/Pv4HoBTAnyT2LwOwDUAWgBgALwNY53D8agBJ4rHVAHY5HHsDQAuAuRA6ArFiPb8ByASQBiAfwI1i+hMBVLjI5CntIgA1ACYBiAfwFgAGYKyH8/sSwHoAqQAMAE4Q918J4GeXtPZyPJxDEYBTHNK/D2ClnOvlUs9JAOoBTBfTPg9gs697Ih6LBnAYwB3i+ZwPwATgEfH4MQCOADgWgB7AFWJ5MeJ57ADwoFjOaADFAE4T864SyzpfLPsuACUADB5kYQA+BTBAvB89AH4Qy00GsB/AFb7OGcBAAG0O9d4BwAzgWvH4WQAOAcgBEAXgbwC2St03/qegbgi3APxPgZvoWcHnAzjZ4fdQ8eWPkkibIr5kyeLvNwD8T6KeSx1+PwXgJXH7RLgreE9pXwfwuMOxsZ5ecFFmK4BUiWNXwreCdz2HRwC8Lm4nAegAMDKA6/UagKccfieKaUd5uyfisQUAqgCQw76t6FXwLwL4h0ueAwBOgKD0y1yO3Qvgv+L2KgDbHI7pAFQDmO9BFgZgrsPvHQBWOPz+F4DVvs4ZwOUu9RKACvQq+K8AXOMiV6fDtecKXoU/bqLRNiMBfCx+5jdDUGAWAEOISE9ET4jmiFYICgkQemI2yiXKrHHY7oTwknvCU9pMl7Kl6rExHEAjY6zJSxpvuJb9DoBzRbPJuQB2MsYOi8c8Xi+JcjMh9MIBAIyxdgANAIbJkCkTQCUTNZvIYYftkQCW2+QQZRku5hsJINPl2H0uMtrPmTFmhaBoM+GZWoftLonfjvfN0zk73VPx3Byv/UgAzzrI3AihEZBzvTgBEhVuATiqUg7gasbYL64HiOgyCJ/Nf4Kg3JMBNMHZbqpWqNFqCGYQG8O9pC0HkEZEKYyxZpdjHRBMPAAAIsqQyO90Doyx/UR0GMCfAVwCQeE71iV5vSSogqC0bHUnAEgHUCkjbzWAYUREDkp+BATzkU2ORxljj7pmJKI5AEoYY+O8lD/cIb0OwrWukiGXL7ydc7VLvQTn+2o7p7cVkIMjE96D1w4GIop1+IsC8BKAR22DWUQ0iIjOEtMnQbC3NkBQko+FUNb3AFxFRDlEFA/gAU8JGWPVED7v1xBRKhEZiGiBeHg3gElENE0cwF0ls/53INjbF0Cwwdvwdr1cWSeewzTxa+AxANsZY6Uy6v8Vgn36r+L5nAtgtsPxVwDcSETHkkACEZ1OREkQxjXaiGgFEcWJX2KTiWiWQ/4ZRHSu+AzcDuE+b5Mhly+8nfOXEO6Frd6/AnBscF8CcC+JA/lElExEFyggE8cLXMFrhw0QPqdtf6sAPAvgMwDfElEbhJf8WDH9/yB8bldCGEhTQgHIgjH2FYDnAGyEMPBmq7vHQ5bLINh6CyAMPt4ulnMQgr/59wAKAfzsIb8r6yDYs39kjNU77Pd2vVzP4XsIDdOHEHqvYwBcJKdyxpgRgnnoSgimiqUAPnI4ngvgOgAvQPiqOiSmBWPMAuAMANMgDJ7WA3gVwheYjU/FMpsgXLtzGWMmObL5kNvjOYvX8QIAT0DoNIwD8ItD3o8BPAngXdEkmAfhK4qjIuRsBuRwQg8R5UB44WMYY+Zwy9OXIaJVEAYrLw23LJzww3vwnLBAgv95DBGlQujZfc6VO4ejLKoqeBImu+wVJ2zkqlkXp89xAwRzSxEET5WbwisOh6M9VDXREFEpgJkudk4Oh8PhhABuouFwOByNonYPvgTCSD4D8DJjbK1EmusBXA8ACQkJMyZOnBhQXQzAnrZODDGbkFJZ4TM9ABiyhkGfkhJQfRx3eg4cADOZETNhPMhgQE9REVhXN6LHjIYuLk4yj6mqCpbGJhgyh0Kf5jHsC7r37QMYEDt5ksc0kUB3nhBfLNLl5GiHHTt21DPGBkkdU1vBD2OMVRLRYADfAbiNMbbZU/qZM2ey3NzATPVmK0PWT7txW0Mlzv3bXbLyDH38caSc4zNUN0cmhfMXwFxXh7E//QTDkMEoOe98dO/bh1Hvv4+4KZMl81Q/+BCa33sPGatWIfWipR7Lzs85CmAME/fvA+ki98Mzf2IOACCnID/MknD6C0S0gzE2U+qYqm8KY6xS/H8EwMdwnsyhKDpx/qVFrQo4HA6nj6Gaghdn3yXZtgGcCsHXWRV0JGh4K/lI6AifA8DhcDSMmrFohkAI3GSr5x3G2Ncq1gcdACv80fAcdeANJ4cTCaim4BljxQA8LjSgBnoiRGV6C5rHURXijSsn9JhMJlRUVKC7uzvcoqhKbGwssrKyYDAYZOfRVDRJHQG6QZKDyZxQwE1enDBQUVGBpKQkjBo1CqTRTgZjDA0NDaioqEB2drbsfJHrjhAAOgAWf5QMV0gqoc2XTE0sra3hFqHP0t3djfT0dM0qdwAgIqSnp/v9laItBU8EK9fZEYDrTZBzU2TeOA02yl1783Bw9rFo3bAh3KL0WbSs3G0Eco7aUvDw001SIw+FsbwcdS/8B2GPDOp6PeVcX7n3QCP3Soru/P0AgI5ff1WvjoMH0f6znHVMOFpCUwpeT+Sf/0a4FaJClN9wI+pfeAGmSmHRHkt7B9q3yA2NzukPlCw5C+XXXhtuMTRJc3Mz1qxZ43e+xYsXo7nZdZEyZdGUgteRsDpzf4P12NbJEBqs6nvvRfl118FYIWf1OA6HEwyeFLzZ7D369YYNG5CicqgUTSl4Avk3yBokTevfQ5cYeySS6CkpBgCwrs4wS8LhaJ+VK1eiqKgI06ZNw6xZszB//nwsWbIERx11FADg7LPPxowZMzBp0iSsXdsbjmvUqFGor69HaWkpcnJycN1112HSpEk49dRT0dXVpYhsmnKT1FNop9jUPPQQAB53hBMk2rAURgQ1jz2GnvwCRcuMyZmIjPvu83j8iSeeQF5eHnbt2oVNmzbh9NNPR15ent2d8fXXX0daWhq6urowa9YsnHfeeUhPT3cqo7CwEOvWrcMrr7yCCy+8EB9++CEuvTT4Rbk0peB1fvfg+ZvFiSS0O5Dcn5g9e7aTr/pzzz2Hjz/+GABQXl6OwsJCNwWfnZ2NadOmAQBmzJiB0tJSRWTRlILXExRzk7R2dMBYXo7YAMMXczic0OOtpx0qEhIS7NubNm3C999/j19//RXx8fE48cQTJX3ZY2Ji7Nt6vV4xE422bPAEWBTqlVfcdhtKzj4HzGhUpLx+hZrjIBrxfNIyluZmdO2LvLEptUhKSkJbW5vksZaWFqSmpiI+Ph4FBQXYtm1bSGXTlILXgxR7/zt37AQAMGt/9MsJEDV91TXsB681Sv9yKUrPO1/yWMuXX+LgcXPATKYQS6Ue6enpmDt3LiZPnoy7777b6diiRYtgNpuRk5ODlStX4rjjjgupbJoy0fRXN8mIgffc+wXWri4cOGY6Bq9YgfSrrnQ7biwq8pi39pFHYWluhqWtDVEOK3h1bP8NrRs2YOjDq1SQWH3eeecdyf0xMTH46quvJI/Z7OwDBw5EXl5vJPW77pK3YJEcNNWD93+QVaOE+xLw3rZ/9LFn1iJOzml8803Fyiy74go0r1+vWHkcAW0peAUHWTVBuBRtH1NYEQNvGDkKozEFT7D6033likhZuILicCIKTSl4PXgPnsPhcGxoSsH73YPncDgcDaMtBQ/AEoB+796/Hy1ffKm4PBwV4Ga1sNJTVISqCJhMxJGHxtwkCdYAFEDJuecBAJLPOF12npZPP/W7npCjJWXI7fsRQeWdy9Fz4EDwBWnp2ZRg1apVSExMVNTlMRC01YMPoRdN1YqVoakoECJNF8p5mRV64Y2HD6Nr715FyuL4IJB7xhvqkKIpBa+H/zZ4o6+gPhIPce2TT/lVR8iJlM6RrBWdlK2y6LRFKL3gQmUL5fSi8Z53MDz66KMYP3485s2bhwPiV05RUREWLVqEGTNmYP78+SgoKEBLSwtGjhwJqzhLvqOjA8OHD4dJhdm9GjPR+GeDZ4zBWFYmfdBFOXX89hsMmcMQnTUMjf/9bxBShhDuB8/pY1SvWoXBy5dDn5QUUP4HCiuQ165MoC4bkxPj8I9xWV7T7NixA++++y527doFs9mM6dOnY8aMGbj++uvx0ksvYdy4cdi+fTtuvvlm/Pjjj5g2bRp++uknLFy4EF988QVOO+00GAwGReUGNKjgg/Gisfb0wNLQAENmppuSKrv8CgA89rtX+Oc3xxc+Gv/md9dDl5CAIS4xXSKdLVu24JxzzkF8fDwAYMmSJeju7sbWrVtxwQUX2NP1iKuvLV26FOvXr8fChQvx7rvv4uabb1ZFLm0p+CBDFRyYKsRjdlLiXGnJh/fcA0S4bqaKCnT+8QfijzkmzPJ4Qal77O29CqIKXz3tUGK1WpGSkoJdu3a5HVuyZAnuu+8+NDY2YseOHTjppJNUkUFTNnibiSb26KPlZ5J4YPMn5jiscxp6uvbswaE/nQJLe3vYZAgK3igGRMfWrTh88SXhFoMTAAsWLMAnn3yCrq4utLW14fPPP0d8fDyys7Px/vvvAxBMwrt37wYAJCYmYtasWVi2bBnOOOMM6PV6VeTSlIK3DbIm/elP8jKEocdpNRphqq72mqbu2edgqqhA1x/uLX+w9BQVgXk475q//wOH/nRK8JXwqJLahTfekkyfPh1Lly7F1KlT8ec//xmzZs0CALz99tt47bXXMHXqVEyaNAmfOrhXL126FG+99RaWLl2qmlyaMtGQn4Os5poaNL31tnoCSVC1/C60ffcdJu7LQ9euXYidMgW66OiQ1N3x668ou+pqDH3kH0g53z1ed5OHkKey4fHgtY9jAxtMY6vBhvr+++/H/fff77b/66+/lkx//vnne+xsKYW2evC2UAUydUH9mhfRc/CgukK50LZxIwChJ334L5ei9pFHQ1Z3T3ExAGHmrpqUXXU1Dl9xpap1cPooCjTUlvZ2NL71turKUQtoSsHrIHQMkhYuDLcoPrG2tACAMrMCXQnzg28sLUXn9u1hlaFPwRWVX9Q8/HfUPvIIOrf/Fm5RIh5tKXgCLIwhZuzYcIsSHKHwVFADrqj6AQ73WK3ny0extgVHmLHXEaI/9OYDOUdNKXjBRKMw4XxwlH6B+vI70A9eYE5gxMbGoqGhQdNKnjGGhoYGxMbG+pVPW4OsgOJL9tW/9DIG33G7omXKRrUHVq2eFx8I5SiAn499VlYWKioqUFdXp448EUJsbCyysvzz89eUgtcTKd5JbXg5DApeKUXpsYEI/iqZ6+pgqqpC3NSpQZfV12EWC+pWr0balVeqXlfN3/+B5g8+wMQ9u1Wvq69gMBiQnZ0dbjEiEk0peCEefN/6TGNSyjbYc/DUQCjYwS4+62xYGhtDHrqBIfKCZXZs3YqGV15FT0mJ6nUF7cqqJfrYu+6IpaUF0EdBn5igaj2q2+CJSE9EfxDRF2rXpSeCWYV7bm5oUL5QWZEWI02V9WJpbAxthRF8LZjFIvxXIRpgRBMuBRu5j4JsDh57HA6dcILq9YRikHUZgJB082J0BJMKAeE7ft2mXGHh7HX03Q4PJ0LQ8kBmqLF2dKheh6oKnoiyAJwO4FU167Fh0OlgZIr70aijlMPaI428LhBXHBw7kfd49lnU7sGvBnAP4Nl7kYiuJ6JcIsoNdhQ8hgjGUC3pFCz9KV6LN3ki2PTC4fR1VFPwRHQGgCOMsR3e0jHG1jLGZjLGZg4aNCioOg26XgUfNWRIUGV5wtKu0GeVqPTIobvSufMPtHyp4OLf4Vae4a4/VERag9pP4F99vlGzBz8XwBIiKgXwLoCTiOgtFetDtI5gZAyMMQx75t8Kltz7IFkagxxwdVF6jl40hy+5BFXLw7tIb1D0F4XuAQrQtsAVlZ/08+fMH1RT8IyxexljWYyxUQAuAvAjY+xSteoDgGjxxptYJDrTueDHQ8qMxv7nocHpvzi0d20/bkT+pMnOX868QZSNpkIVGHTC6SjuSaPiA2UqPYyOrVtdK3T6dWD2sTg4d55qMihGH3nxrJ2dKL3kL+gOcSRRAOgpLAQzGkNer2IodYtlPit1zz8PWCwwlR12O0a8J++TkCh4xtgmxtgZatcToxNueA+THzI43FhaWlB29TXSB8VzYN3dsLa2hk4oCRhjdn/vMAsSdBGdubno2rkTR/75tAIC9SI5ac0B05EjKD5zCWr+8Q9F6w0bgdwLrpRDirZ68DYTjZUp27r3kZ6pT4I4j6oVK1AwaTLMTU2eE/XXBT9kymZtawMAdO7YqaY0fZ8IvtV9DU0p+GixB29kLOwKwVRVZV/cwwl/lKxqscb8vzatn30OACg65VSlpQk7jDFYROUbYAHBChBcfgfMTU3In5iDpnffDSh/T1ERWr/+xnOCCOrs8MFp32hKwceINvgeqwqTnfyk+JxzUXHTzZ4TeFGyHVt/Da5yXw9+EC+GVa2FwMP4sjb+9w0cnDUbpqqqoMoJ1ItGSUyVwjk0v/e+7DxtGzei41fhmSs+/QxU3h6m6KlecFLmkfw1F2FoSsEn6IXT6bBYETt5sjqVyP0cF1dsCqqcYJ9j1zr4iyFJ2w8/AEDQCj5Q6p5ZHbK6TDU1bvsqbroZZVddLa8Ax2cokOfJ34ZcpUeWWa0ovfRStG/erE4FEYKmoknaFHyb2QLS6xUrV9FPQXGgsvHN/ylXpif60idsKBufCLsuqn0VSWAsK4MhIwMAULdmDaxtftbdV1cbc8Ha0YGu3B2oLFiOCbm/h1UWNdGUgk8UlXqHJfwmGl+0fePFzhksnl4ehV7OkguXKlJO2AmXjomQBqb+uefDLQJHZTRlokmKEk6n3aywO5/D+2hpcXdXLDzhRFi7upStU02C7D1179mjkCB+oIXYPSHstfpy2VSmEvl1MKMRR559Ftbubr/zKlF/f0VTCt7Wg29XsQdfev75bvvMtbUov+561erkOKDmSx2oAo4kPePrFIKVNcDr3/TuejS8+BJYZ2eQAkTGYHZfQVMKPk60wXcqreBlPNTmUC+AEYmo+d71hQHiSJAxkhobB5gpyNm7ap2Xxr8CNKXgbROdzArftO79+32mMRYX48jTgc+MrH38iYDz9ie68/NRfsutysTmsT0mSj0vEaQsPPZy+8i4Q8Vtt6Hqvvuli1JC20dCYwyAmc2qlq9JBW9S+EVrekteEMyGV18LuI7GN9/0Ow+zWsGkfP7F83cNLaD2wxSK3mP1ynvR/sMP6CkqUr7wQF96udkUfC4PHnscKm77q/8ZI6cNstNdUOC2r+2779Hy0Ue9O6SucYQo6WCof/llVcvXlILXk/AcqLFsXyRScNQklJx1tsfjVXfd7fT7yJNPAgAszc2qyhUQ4ej9BqAf2n7ciLr//Ed5WfzE0tKCtu++83g8JIOsClF19z3hFiFsmGuPqFq+phQ8ESFaR4r34MOFnHg6PYWFno95iJaomt+1J3G93I+Iiggo47mpuPlm1D//QmDlh+JcI+hyOqJKWAHuieMTTSl4AIgidRbe7qtUrVgZvpgdEaps5GKqqpI2gbnSnx63EDxLko2+Y6QClR6shtdeQ/7EHFj7cjhnFzSn4KOptwefdGrfDoylhGJu+fRT9wHJSOo1B4rKC6H3FJfg0Ekn+zeuEgnXNYSNjV9mINekMu+f6i6RDvesYe0rAIRZrlpBcwo+ysFEM2Dxn8MsTYSi8c9Sf7D29KDrjz/c9psqKwEAndu3h1okRVBaMXZs3Yr8iTkw1dYqWi4A92UsNfh8WtrawrKeguYUfLSDiWbAokVhliY4iCgsqw75g6NnjhJKpae4pHe2owO2kpUYPHRUILWPPOorcdD1hYNArpM3xdq0/j0hjZ8ztiuW3Y78iTnuByLhaydEMJMJB2fNRs3Dfw953ZpT8FGknUFWAGj7/vvgC1HxerR++SWsRiPMjY2SkQr9gRmNKF68GJV33Ok7cRAKwnzkiL0MY0mJ4uX7RNXZuIFnbRaVuJJ4irlkUWKFMiUuYwh0hc1E2vL556rX5YrmFHy0jmDkg6zeUVB5MZMJFTfdjMLj5wJBxuG3fQ10bNsmI3Hg97jmwYcA9JphepG6Lv7YmX2ljYBeqxcZew4eCJkYxYtP9y+DH/Hgqx9ahZZPP/VRoEQZGvyq0JyCjyJSfCZrpGFpbnby7nBXVD5Q+EHu+OUX2WnNTU0+/fBZTw/KbrgBPYcOuR1T1LasdFA6TmB4ex4DeFab169H1YqVQQikHTSn4A0R1oM3lpUpuqiAua4OB4+bg/o1LwZeSBgbwMI5x+PgcXPQtP495E/M6fXwcRTJakXHT5tR8/dIWJzaDwUjUxkZS0vVmYnriKdbrGDj7ldj68czJ+UmaWlpcVuQpeLmm9VxafRDVnNjI5rWrVOsPKXRnoKPsB580amnofz6G9BTXBxQfteH3VxXB6B3FaK+St1zzwGQDr8cWSj5LPWWVXbNtQqW605Pfr70oCljMNfXw1xfr2r9fuHtfRWPlV97LQ6ddLLbYVOFn1+vrsU7/vCj8WNWKyzt7ai8czlqHv675NemG2EwAWlSwRtZ5C344bfN0QOhcCE78sxqGCsqgi7H0tjk+VhDg/MOgl8vQMk556JTwr1Riq69eahascJ90hKR5zrVfhdDsG5wd16e5P7CefNROG9+0OX75wcfOZ0uwPm2Wzs6UPPoY5LeW56oe2Y1Ds6cBWPZYQAhiPMUIJpT8NE6gjny9LtyiO9JT36+alU0vPwyKm65FQBQ89hjKDzhxIDKMZWXByeIhFJwVCodW36WVUz5zTeh5dPPYK5z6bUG0aPKn5gTETFpvKJkI+JFQVva2mAM5l4r0LM11db69ZXseDr1r76Kpv/7P79cQFs3bAAAWCP8C1RzCj4qQnvwAaHUJ10AvSebbbzpf/8Hs7fJLcHKKFc2iXqYsUdeVgcvere6g+hZ+r+ubnDXSsqfnFmtqH74YVkhrdWi5LzzUXSKcrPGmR8eMzYOnXBiQF/JBGh6sF1zCj5a52yDp9jYMEqjLM0ffoiKW2/1Ox8zm50Ho5S0BSr56S3DFuuI7DACtvP1JatUCBRZ5+eexjEIXOtXX6H4nHOd08m8bi1ffgljaanH441vvInmde+i5Nzz3A/6fZ8D82YxlZXJKNuP5f265JtKQhWbgVmtqLr/fnQfOABzU5O751qEmaBsaGrRbUDswTt40VBMDJgftrWIwuWhqb7/bwHlKz79DMleeP7EHCSddhqynl3ts4xw0Pn77+gpLkHM6OzgCvKi4C0eImsGFOXSIU/xmUuQUyCY0SqX3yWYSwIwmVQtvwtkMGDiXul1cI889ZSLCME03sHd8+6DB6FPSZE8Vrf6WdnlNL3zTlBy+IPcMzZVVKDlw4/Q+dvviJkwvvdAhPvOa07BR+t0mprJqgRuyt3hofQ00zAsSLwsxYsX2xWlkuXa9vdILDbhhEqPks0bSg7MZJKe7i+VNozPfsmSs4CoAFRKGJSk1yo9eB/ZsLb3nWBkmjPRRFH/WfAjYJRUAkG/nL5lcfV/lkP7L7/0KlFPPXilFUukdSz8PL/ufB+NnRwi1JtEMSK8x+6K5hR8NLn04EP80ikaMY4InTvluQI6C+HjnENlN5eVXcjf8tlnHtPI8jF2ofyaa1F68SXCD5t+d234Xe5V4OYNefnsMXBE2n+WPwM4FEhF1fQKg7x4+cEgdU8UV7IynuEgnvNwtvuaU/BRuvBOdPI2w7Ts+uv9Lq9jy5ZgxJGNucnZZ51ZLbLNAr5gjMFUWYnqBx6UqFhQst27pW3MwWASffk9edF4G7xUo6dWdu11Tr/Lr1V3slMoMCkwXyKs+DDHuEFwbs/90DWss1NYUKRHnveXEmhOwUdTeEMVeJuC3rE5NMraZ1RH1/jbVisK5xzvvK9H5hRwuW5sJ/8Jze+/r0hZfiPTi8bTzE9vWFpbUfvEE/LkCMHkJsevkNILLgw6wqcdT9chiM6UySGqZ8hxqtN3/R7HNgKQPZRrImtOwYc92FgE2GGZrx6C60PZF65XMEpARQVy5Jlnet0EI9A+6x6i1v97baqulhfh07UmH/f20IITkD8xJ6hJe1X3rAjpwLLqK0wpjOYUvGu44PRQfwaHUVkeefKpsKwaEwxO9nUFe0mS+Lg3kjZ4X3VrfVARQPEZZ8LqKX57mDs03Xl5aPUyfsMYk5517K/cTnOv/H0eXesKXSOhmoInolgi+o2IdhPRPiJ6WK26HDEQwQrAIt7Agddf5z2D0gT6wCugxNq++w4Fkyb7TNf+44++CwvRi+s4YURWhH+Sb04AACAASURBVExPlgKrFe0//eS2v33Lll47cSDn5Eee9o0b/S9fYZTqzZrr63FwzvHoPnAw4tco7TnkJTKnaBar/88aycOsqwvtmzb5rKNbnIfAwKRn2kbAl7sUavbgewCcxBibCmAagEVEdJyK9QEAEvTCKbU7TD9Ov/Yatat1IMAbHaEPSChpeOWVgPM2vfsuym+40W1/+XW9A9t+eXwo0OBWrVgBq59L3AHCsoX5E3PQ+t13QcvghsznrH3TJliamtD4vzeVlyGUyDhfx1nHnrDFl3cLoCfjOfFqglLZrKfaRCcmNHO2aYIG8U91LTbAoAcAtFqsSDYI+2LGjQMAJJ5wAkxVlegp9N/tTms0/u//7Nttcnr0wdTld8wWz3iaFi9r0RMfT5/VaIS1uxs6x/AWQazS1PLpZzDXN3g87onufUIUyLZvvvU7ry/zgaWlRVY5sr4E+pY52plAFWsA+Q5fdrnngyp37FS1wRORnoh2ATgC4DvGmNsS9UR0PRHlElFunR+z+zyRHCUqeIkAQrrkAaC4+KDr8AZjLDDfdSlUbN1rH3vMvm1tk5iur+CD1/b114qVpSZll1+BA9OOkZ3e2trq0zPIn9WugqV9yxbpmDQOVN5+h3+FensGWehnzlY/tErCFObNrTGAd8irmyQ5N6KBnH8IG0ZVFTxjzMIYmwYgC8BsInIzEDPG1jLGZjLGZg4aNCjoOm0KvsXUq+B1SQMAAFEDB6nvqmax4vAll6hbh9L0AfMQU2TlHv/Ps/O33xSoNjTXt+VTicHGMN7a8hvdTWbB0rx+vX8ZXK6963wPf3HTzbbyI9CDCvBTwRORjogG+FsJY6wZwEYAi/zN6y8pooJvNPV6NyQuPBFDn3gcg25fprqCV3SgLVSKVyq8coQ+sEHh1/XU4Pn7gwLPXsdPyi1VqQQdW7eicM7xwS2h6em9YMLgq99fNCq/Zz4VPBG9Q0QDiCgBQB6A/UR0t4x8g4goRdyOA3AKAAWCXXhnZFwMAKCkq9cXnIiQcvbZ0EVHhzUYU6QSzDVp/vhjBSXxRPji4iuCvy9xoHJK5Qvy0gUXnVIBZNTf8Mqrng86XJPOXbuE/zt3ei3P6/tABOZo/nWcWHbe+SjIOcq7sF7kUwM5PfijGGOtAM4G8BWAbACXycg3FMBGItoD4HcINvgvApZUJklReqRG6VHR7eGTvi8p+BC9XO0/Snx1yLxOXbk7FJZGAi8LuNS/9DK6fUWEtBcj7+utp6gIPcUqL4otByXuv9qPe4S9Tt4GkRVprHQ6Z7OdWGbjW29LLrrSJsMFU03keNEYiMgAQcG/wBgzEZHP28oY2wNA/oiVgsTqvYQMDsF08b6GpFkpghpC5mXFnbrVq1G3ejXSrrxSVln1L73kM03x6WfIFU1dIuge9BU6f//dvm1pa0PtI4+4pWl46WXETZnis6ymdesQN306DBkZHtPYJoC1fPSR5PGKG2+SrN9OBLhJvgygFMBuAJuJaCSAiF6IUE/wHK5AK8v59SNav/7KZ5quPTKClTH/Fp4IBZLmgHCbRQD0HBR8wy2tbWGWxD+6Dx60bze89lrvwLPV6rwOwvc/eC7EbMaRfz+DhrVrAQAJxx/vOW0AOC1oEm4TDWPsOcbYMMbYYiZwGMBCVaUKkigiWDxct7iZM0MrTF8lApSMjdbPXOOpuNPlw64qEKYesZeX2OjHQtGB4HNBEw80vfUWAB8LwkTgF4ZTIC838eQ9080ffWxX7oAwONtbROS8F3KQM8i6TBxkJSJ6jYh2AjgpBLIFTBSRRxNNxr33hlgaTsQQgQpJ0vwkymnx16VP4vxaN2xQOT5RZF3TJocJfP6u8Ws/5MUl11JfH4hYYUPOIOvV4iDrqQBSIQywyoyPGh70RPZYNK5QdHSIpQmcvhY4TKvkT8xBj2o9bc+Kxv9JUtJlyYlPFLEoOWYWib3vcLtJove7ZjGA/2OM7UOEOwlHebPB9yHKrwnfghCuqw/1d1o+/xym6urAMnt5ibv3B7nerMbp3rdPlXJbPvlElXIBwcQjm3Db4AHsIKJvISj4b4goCUBEj1RGEcEc0RJywkHpBRcGnpkxHFoYoGXSSy/UXKvQghwc38jsLXds9z/2vSPV990XVH4lkeNFcw2EaJDFjLFOIkoHcJW6YgVHlBcTDaf/wkymcIvghqVNOS8VtzVnQ0Ffes9k2h3UWD7SI+E20TDGrBBiyfyNiJ4GcLzo4x6xeBtk5XACoWPLz6qU2/ja64qV1VeCuoUKa3vku3g2r18vOUFKKeR40TwBYBmA/eLfX4noMe+5wou3QVYOJxDUsgVLoUxgNY7Nl99G2MMueMBXBNBgkGODXwzgFMbY64yx1yEEDIuQqX7S+BpkTb3k4hBKw+H4R/XfHgi3CJqARZgLZziQG00yxWE7WQ1BlCSKCLmtnZIx4QEg48EHQywRh6M9mNncx2zwkdmDVxM5Cv5xAH8Q0RtE9CaAHQAeVVes4Oi0CF4L1+aVhFkSDke7WGWuDhUuKLK9uUOCTy8axtg6ItoEYJa4awVjLKJ9u2wDrPkd3WGWhMPhhAt3E03/U/geFTwRTXfZJS5Nj0wiymSMyQn+ERZscWhUXa6Kw+HgyOrV4RZBNj1FoQsBzRiLiEFdbz34f3k5xhDB8WhsA6z6CLjAHI6WafcWlTHC8BTSV8t4VPCMsYiOGOkNm4lGx/U7h9NvCcliNJ5gLCIGdeXMZO1z2C6rvh/a3DgcTgRgNsPc0RFuKbRpph4gLrzNw9FwOJxwUPPoYyico+xCIYGgSQWfpBcUfKPJ7DHNxP37kLJ0aahE4nA4/Yjm9evDLQIALwqeiC512J7rcuxWNYUKljtHDQEATEmM85iGdL2nbhg+XHWZOBwOJ9R468Hf6bD9vMuxq1WQRTFmJCdgSmKc3VTjEXEwNn56WNYG53A4HAAe1uZVAG8KnjxsS/2OOKJ1BKPs8KkRfzocDkfDOK0lqyDeFDzzsC31O+KI1hG6fSz3lX7N1YgZNxZJp51m30cxMWqLxuFwOE6oFerZm5vkRCLaA6F7O0bchvh7tCrSKEiiXo+aHu8LPESPHInRn3+Ozj/+6N0ZAb6rHA6HowTeFHxOyKRQgQFRehyUGYsmbtIk+zYZDGDdPIYNh8MJHSG3wTPGDjv+AWgHMB3AQPF3RJMUpUebRTpcsCsUHY1xP2/BkHtXImbcOJUl43A4HBdCreCJ6AsimixuDwWQB8F75v+I6HZVpFEQxhgaTRZUdstbHSdq4ECkXXEFN9FwOJyQ0/r5F6qU622QNZsxliduXwXgO8bYmQCORYS7SQJAlzjA+nW9fzGr42fPcvqdsGC+YjJxOByOFF27dqlSrjcF7zhCeTKADQDAGGtDH4gCsHxUBgDBm8YfBt3qPIdrxNq1isnE4XA4ocTbIGs5Ed0GIQ78dABfAwARxQEwhEC2oEgzCKfWYfavLSK9j8lRHA6H00fw1oO/BsAkAFcCWMoYs3niHwfgvyrLFTTxeuHU2i3+f2zokp2XnR36xOOKyMThcDihxJsXzRHG2I2MsbMYY9867N/IGHs6NOIFjp4IcTodOmR60jgy6u23nH6nnH02kk49VSnROBwOJyR4W7LvM28ZGWNLlBdHWRL0OtQZzegwW5DgKy6NAzFjx2L89m0wlpX17uTeNRwOp4/hzQY/B0A5gHUAtqMPBmxJ0OvwQW0Tfm5qx665k3xncECfnIy4KVPc9g9b/Qwqb79DKRE5HA5HNbzZ4DMA3AdgMoBnAZwCoJ4x9hNj7KdQCBcsMWJI4Bqj95AFckiYcxwAIDo7O+iyOBwOJxR4s8FbGGNfM8augDCwegjAJrmx4IloOBFtJKL9RLSPiJYpJHNYSFm6FOO2bEbshAnhFoXD4XBk4XVFJyKKIaJzAbwF4BYAzwH4WGbZZgDLGWNHQWggbiGio4IRNhg6A/CmcYSIEDVoEABhNSgOh8OJdLwNsv4PgnlmA4CHHWa1yoIxVg2gWtxuI6J8AMMA7A9c3MBpN1vsrpPB4rgaFKKiALPnpQE5HA4nXHjTeJcCGAdgGYCtRNQq/rURUas/lRDRKADHQBisdT12PRHlElFuXV2dP8X6RUeQPXhXDFlZGHLvSlCU0EYmn38eBt3BB185HE7k4LEHzxhTpLtLRIkAPgRwO2PMrWFgjK0FsBYAZs6cqWhINccee6ePxT/8Zez33wEALG3tqH/hBQxdtQrMYkHdM88oWg+Hw+EEijI2Cw8QkQGCcn+bMfaRmnVJsXhQ74zUDrP/E57kMOjWW5BTkG/vyXM4HE6koJqCJyIC8BqAfMbYv9Wqxxu3jRiMh8ZkAug10aw6VImfm9rUqZBPhuJwOBGEmj34uQAuA3ASEe0S/xarWJ8bRIQFaUkAgNJuIzI27sJL5XU4f1eROvUZDIiVmBwVKLqEBKffY3/6CQMWh/QScjicPoxqdgXG2M+IgNmviaIdfkdLh+p1ERGy338PANCZm4vDl14WXHnR0UBHr9yGIYOhS0oKqkwOh9N/UNUGHwmkimGDP6htCmm90WPG2Lcn7NyB2ClToBswQDLt2I0/Yuhjj7ntd+3Bczgcjj9oXsEn6XXQS3xH3Lpf3WVlo1JT7du6+Hhkv/8exv+6VTKtYehQpJx7jvsBhfz2ORxO/0TzGoSIYJFwvgx1j14x/DB6UXS0enJwOJyIR/MKPpyM374N47f9GnD+AUHGoDcMGxZUfg6H07fpFwr+5uGDw1KvPjkZ+pQU+2/S65Gx6iHoEhNl5ZecGeuHK6Ytdg6Hw+mf9AsFnxkbOUvIpl50EaJHjJCV1inmjW2fHwp+2DNhmX7A4XAihH6h4K8aNhCvThoVbjECIvWSiwEAiQsX+p03Kj1daXH6FLFHHx1uETicsNIvFLyeCGcMTvGdMERkPv1PJJ/lvuKhlEmFiTNwE+bPE/cIPfhBy+/EkPvuk+1KmbHqIcn98bNny8ovRdaaNQHnDQW6mJhwi8DhhJV+oeBtXDVsYLhFAADEjB6NzCefRNJppyHrxV4lacjMdEsbN20aACB2Yo7Tfl1cPNIuvwyjv/zCaX/C8cc7/R6w+M+ATofUiy6SlCXxxBMDOQUAQNJJ/n9VcDgcCSTMsYoUq0qpEYohwmLFZD27GkkeTC+G4cMBAMlnn4WxG39E/PRjnBMwwffTkJGBnIJ8jP78M2R//BGGv/qKU7Jh//43crwtUCKWk3bVVRjy4AMek439aZOPs3FIu/FHZD71pOz0nki9PLiZwNFjRgeVP/nss4PKz+HIhikaSNdOv1LwUS4KvtNiRWFHd5ikcWfQHXdAn56O8bm5GPvdtwCEQVXD0KH2NMnnnQsASDzxBKe8MePGITbHuZcvhxhxCcK4KZORdsklSLvySqFeQ+/ANMXFwTBkiGR+17GB7M8+hWHoUCQvWYKsF9dg6OOP248Nuf9+TNi9y6dMcTNnIHbKFAy5+25/T8eJ9Guu8Xhs0O23e807MW8vBq+4R1Y9Qx991G1fzPjxsvJyOAC4gleCKJcO/DV5JZj/WwHMVnUurr8kHHcsxv/yM/SJnu3qcZMmIacgH9FiD98NGV8pg+9ajjFff4VRH3yAxHlzMea7b3uDmIn502+4AQOWnAkAGP6fF5zyJ53yJySfdZZbfTkF+Yh1UGxJCxci5ZzeXnDaZZc62cUNojfRiDffdCp/1FtvIfv990AGA8Zt2ezzfDzh8RoBGHjjDV7zUlQUolJTEX+csNg6RUdjwJlnuqWLmzEDKeedi2HPP+d8QKfrvUYBwgeJfRCCL3KlXI2lxtxCQb9S8PNSnQN1bWwUwgZ3K7wYSDiR40aZfu21iB41CnGTJwFwVoQ2u3riiSdi2FNPIacg382un/X888h88gkAQOrSCwOWdey33wjlHzsbaVdcLpnG2wuWeMIJbvuGr305YHmksDVQ2Z98gsynnsT43N/tx4Y+8g+MevstAHBfD0BHQU00ixk/HqkXX+w5gV4fcNn2IgYNRMzEiUGXAwAJ8+b5TqQ0KvV6HVEqOmz8cXMUKcdf+pWCX5CWhG9mun86d0dIDz4SiJ81CzkF+Xbl7wspJSsH17DHaVddBQCIGix/UtrAm27EiP++bv+dU5CPxAULMPCWWzBo2V8BABP27Maw1asDkhEAks86CxN270LM6GwQEfQOk9RSzj/fYz4ine8xAC+N8fC1L/sVlmL0hi+dfstxqx398cdeZbB9wfki9ZKLkeX6BdNH8DW2lHa5dMdDLjE5OUJn6mjlwoj7Q79S8AAwNSke05LinfbZevBGqxXFnT3hEEtxKDZWcn/0qFGK15V02ml+fYKO+3Wr/QvAHxzt5kMe+Btip05Fwpw5iDnKeexh0G23YuBNNwEAdNHRGLDoNMnyMp9+Wla9Uu6Whqwsr3kS5s3z+jWV/fFHSFywQPLYsOefgyEjQ/LY2M0/Se6PGS1vQHnUBx/Yt6MGDvTaiAy5916f5U3cuwdDHngAurg4jP78MzdzW6Sjd5lVHj9zptPv2MmCSVQKqa/O1EsucfodN3kSxnz9FaId7s+IN94IUFr/6XcKHgCGucxstSn4ew5U4Pjt+Wg2mcMhlmIMuf9+ZH/4gdv+8bm5yP70k4DLHbvxR4zesMFtf9azq5H5pHyvmajUVKdBXF+k33QjKC7OyW6e9pe/2BVo9ocfYmL+flllpV11FcZt/QUAkHzG6bJlcGTsjz8g+2PvK1Amzp9n7x0nLVqELJdxDMcB8aGPPuJ0bMApp3gs1xal1FUxOZJ0yikY9NfbJI/FTZ6EUe+tR8aqVQAA0rmbeoavfRmjxHUN7PmmT5csjwwG+32IGTcOCcd6nlfh7Ysgycs5SxE3cwaGv/aq/bdj+I9Bd94pqwzDsGFu80jSrr7KOZEHM9DQRx9B7CT3r9yMBx9wahCYmJ+IMHrDBiT9eRHipx+DtGuuliVjsPRLBZ/rsvjHvO0FaDdb8EOjsCb4mTsL8VtzezhEU4S0yy5FjEM8ehv6xISgJv8Yhg5FzOjsYETzjJfe7uBlyzDxj50AgOGvvuo0d0DISj7HHuKOEdxME+Ych6i0tKBENWRmQu+68IpL/dGjR9sHkeOPmYakk0/2WJ4+NQ36gb7naCSecALIYMCQ++/HyHfXuR2n2FjoBw1E1vPPSY5d2OSJO/popF60FAAwbPUzSL38MqcvsMQFCxDnYnuOnzXLp3yS6PV2c1HcZM9mCm8mnsErVzj9jps+HaPeeguJc+fa903I/d3egMidwT3m668k9sqzi6Wcd56b8ndc08HeaDuYf2NGZyPrmWdA0dFIOc/ZvJd+rWePr2Dolwp+fqr7qkjTf90Hk3gzCjt7sOSPQwAEV8qyLm2YbcJFxt8fRsrSpV7T6OLiAADxxx7rNV3ivLke5w54w2bekRo0o5gYpFxwgd9lOqIfICzwnnrJxcgpyEdUWhriJk3CmO++RaqLHTdhwXwXAQBdgrPZUIr0G64HIDbg2b0NrS2cxYTc3zFu40bJvOO3b8Noia+36KwsZNx3HzKffBJZL67xOEg98PrrfMrnypivv8K4nzZBF+9+bql/+Yt9O/lcwfV3zPff2/eNeP01AMKzk37llZiwI7c3s65XCWet+Q+G3CeYknQxollSp3MyFY3PzUXmP//pJoPtK9Lpq9ZFv9sGz21jRN5IXHiiQ0ZRtXpw4IgZnY3ha19GgthIxU5Rx2NKtSX7Ipl/TRyOBL0Ob1Y12Pe1mq2Il1hg48q9xdjc1I6ahdNCKaKmSL3Qt6eNfsAAjN7wpWohjhOOnS1pS81a8x/EjB+P6KwsdO7YAWNxcUDlx08/Bln/ecHNm0TKVXPE2rUAAIbe3l3q0otw5KmnJH3qfZHx4IMAJDx5HNAnJ/ssx1PDqU9O9mt1scErV4B0evt4T9y0aWj98ksYRvRei4wH/oamt9+GYcQIZD4mnHN01jCM3fgjzPX1iJsyxcXU0Vu+46zupJNO6q13+Z2g2Fgkn77YaS0EfWICkk7uTedKrDgXRApbx2PwPXdj8PI7UeDwFZIwdy4oNhasW2Iuja0RYp499BIXLEDz+6IpVSWPoH6p4GN0OoyIczdVdFrcb8bmJsFUY2UMugibCas15A4UKomjghiwaBHq16wBSfQ4ZZXlxQzjFSKkX30V0t3sv84/3cxCUDfmv93s5TJeMviee2BpavI4sJ4uTpazkXrpX5A4fx50Lo3MuJ+32BWoDcPQoU4T+6QYco/0BDh9Sgoy7r9P8pguPh45BflgjKFj82a/rxsRAS4NaNTAgZi46w8cPPY4WFpanNOLoQeYhE5xKVjc4ApeUWJ1/inrteV1OGdIKobERE7oYY6yDDjzDNSvWePRu0VpoocNQweErxdvJC1ahJQLzkfMuHFO+8dv3+Zz1S7dgAEY+8P3XtN4zJucjIE334wBpzu7tLo1RD4gIkSPGgVzk/MqalEyxh3cZIqPD2qlMiKSdO01DB8OU3m5z8lTsZMmIcllIR6banYcB4oSvaCis0d5LW/wPfeAmUwBuxv7ot8qeFtv/JT0AfiuoVUyjdHBfraqqApvVzfgw2ljMZgreU0Sk52NrDVrgoqw6Q+DV6xA/LHHId6Dh4oNXUyM04CiDW9mF316OlIuWorUCy+U7PnLgYg8euMEgq23nnKxdOA7r7IYBFXlyZsnWLI//ACW1lYYi4p8pnPDpiccAoYlzJ6NEf97E/EzZngtLzprGIa/qF5U1n6r4IeJSnp2coJHBb+/3dm2VtjZg6O37uP2eA0TygiZupgYDDjN87KMtoFXx1XB5EJEGCq6QkYKuthYjN/2K3QBNDi6mBhkf/Kx1/ATwaAfMAD6AQNgrqvzP7OEggcEJR9u+q2CPyV9APbOnYRB0QY8VlwtaQFbtOOgx/xmK0O9yYwM3pvnqETSKacg46EHkXzOOeEWRTECaaxsxCoUVsEb8cccg2HPPYv4WbNABnmmIEdf90ijX7pJAuLnZ7SgnNdPdfcZ94aVMawqqsS0rfvQ0scnRXEiFyJC6sUXQ+dhVnKoGfXuOgy85ZZwi6E6A049FVGpqV6D/jky7OmnETd1KshlwDgS6Lc9eEcWpCVhcHQUjhjlKet97V34tl4w6+xp68L8tN5PzopuI547XItHx2XB4OdALocTycRNm2ZfgKYvMPKdd2CuqVa9nqSTFkbs4jfEQhCRTS4zZ85kubm5vhOqQG2PCbVGEy7bU4xamYrextKMNNwyYjAeLKzEnvZONJosyIwx4NuZEzAwWpk2tMNsQYxOh6gAG43vG1oxY0A8Ug28TQ8X7T//AmY0Rqwy4PRNiGgHY2ym1LF+a6JxZUiMAUcnxfut3AFgfU0jTss9iE1NbWg0WQAAVT0mPHu4RjH5xmzZixv3lwaUt8VkxqV7inHV3hLF5DFbGVrNFsXK6w8kzpvLlTsnpHAF78LLk0YGlE8qprxe4UGXL+paYLRacdJvBdjUKO35I4VR/Eo7pGCkzBUHyzF+y15YI+gLkMPhOMMVvAtnDU7Fvyf674olpebqjWasr26Ulb/JZPY4YOu44lRltwn7O7qx4kCFbNlMMuPdb2lsQ7vMXvk68by6fM3U43A4YYMreAkuykjDmqMC68k78kFtE5YVlOGb+hY0mcxgjKGkswd1RhMAwRunROxV5/ychwk/50mW0+MQz6JIDHzmzwBujwcFb7RasbetEwBQ3WPEBbuL8Nf8Mlll2r5OOkKs4D+sacSrFQH4Kvdz9rZ14p3qBt8JOZqCj7hJoCPCuUNSMTg6CuPiYzEkxoBnS2vxeElgI/JXiLbvf08cjjsLyhGn06HkhKPxTGkt/llagymJ0u5Vr1XUYURsNGYk97prXbpHCIZ1qLMHywvKkG6IwqKByZjukObHhlZMSIhFnF6HNEMUfvUQ+njET3sAAK9PHoXxCYIrXn5Hl9dz+aa+BeurG6EnwMSATgWXO7QyhuoeE4bFRqPNbEFhR7fTeQHALWIDdG2W77UyXyw7ggaTGX8bk+m0v9NiRbfVirQQDDi/WHYEJ6UPwISE8Lo6npIrzOm4ZKi8ULocbcAVvBcc13BdNmoIbhs5GCsPVmBWcgJuk9nTdeTOgnIAQJfVik2NrXihrBYAsLe9V6lub25HYWcPfmxoxYZ6IYDRH8cfJVne26KZ5LmyIxgSHYV4vQ5bj83BJXt6IyJemzUQr1bUAxDCbDSazPi1ud3JbLOvvQtj4gUFVNJlxF/zD+Op8cMRK0bXbDCaYWYMQ2IM9sbKhq0H/3VdC67ME44VLZiCKCLE6Pz7QHz+8BE8XlKNbcflYHlBOX5pbkfpgqPtcjjSbrYgMcr7uqQPF1UBgJuCX7zjIAo6uhWfkcwYQ4vZghSx4TBZGR4uqsKzh2tRML83CmGr2YIkvc6viTFPFFcjTqfDslFDgpLRaLUi2s/7ojTdFivaLVbFPMw4nuEmGj/QEeGpCcNx1uDAZ+PZuGh3MbokTCdn/XEIdx0otyt3ALL882uNZpR0GdHsYkO3KXdAGBM46uc8XJNXihv3H7bv/1dpLa7LK7X/fq+mCc8cFhqf3JYOTPolD1O37pOs1xaB06bcAWDM5r04c0eh/feXdc34oaEVHRYLdrZ2uJUBCGGZbV9IFd1G5IrpPJmA7jpQLrlfDgUdQgiK3aJ5qqrbiG0KLPCyvqYRE3/Ow4GObhzq7MZJvxcAcD6HBqMZ47fsxWrx+spl9WHPX5C/Nbfjn+Ixo9WKDovncZSWID2fvq5rQcbGXajsNgaU/+mSGozavAeTf5E2R3pj+Kbd9vNUk8eLq7FkZ6HvhArAGIOaruqqKXgiep2IjhCR/3cywonW6fDzsRNx+AQhSH+aIfgV7r1xWq7nkAmuvOLFPu3tMTrY6Rx359nDtWCM4QyHB/1gh3vc6zYPCmNPexem/bIPL2SHxwAAFA1JREFU71Q34Jq8UvxlTzHGbN6LxTsK7QOz+9u7sCy/DBbG8HW9tFdQh8UCxhhO/r0AH9X2RiP85EgzTFaG8m4jNtQ1uzUcNuXtjSv2CI3S4h2FOPuPQ1gn2qhdB44ZY+i0WFHTY8LHogxWxrChrhlGq9U+CL6xsQ2AYO+et70AheL4imM//Yg4/rK+xn3wvbCjG/kOX3PPlta6rT4mxZI/DuFfpcL9OnNnIcZs3usxbbOp935ZGcMh8b6XdPbgsJeFbV4qO4I7C8rwRqXQYdhQJ3RAGGO4aFcRvjjS7FNOAHi6tNd12FWxGa1W/KukRvKZYozBxBj+VSrdMCqpKJ89XIvfHK77oc5uu7dYh8WCVrPwTDYE4FJto8dqxb0HK7x2npRAzR78GwAWqVh+WBkbH4sYnQ41C6dh9/GT8eT4LByaPwVLHHr3aQY9fpw1AXEhnNH6bw8vQCA4vowAsOC3Arc0jxRVIWPjLsn8NUaT3SzlyNV5JcjYuAsn/X4A62sascNFiVV0G+0Dw7mtnVhWUIZ97d242eGrAwCWFZRh1q/7cXVeKRbvKMTFu4VIgPvau5waRccXv7qnt+dp6+nWiEr3joJyvFpRh+zNe1BnNOGHhlZcuOsQVhyswOjNezBt6z7ctP8wPqltwpV7S3B1XilG/LTHHrPItmCMa+NidWhabXMHSruMKOvqwR+tnbAyhh8bWjH/twIs/P2AkIcxPF5S7dTA2stjDBkbd7kNNm9uasfuNqGB2CI2Nq449uD/XVqLedsLsK+9C3O25+PYbb0LbHxb34LJP+fZv9BWFVXhnepGdIljLg8cqsQ1eSUo6OjGpqY2XLuvFI8WVfnlVeX6Zfp2dSP+WVqDteW958XEc/1Hkeeeu4UxDN20G0M37fb49fLC4VpcsbcYHRYL3q1uQL1Yd3Fnj115t5stbkp7X3sX5m0vwEuiTHO3FWD8lr1YW1GHSb/kuTWKW5vanbzePPFtfSv+W1mPRpNF9gz6QFDNCMYY20xEo9QqP5Iw6AhXDBNiW6+dNAqnpjdiVnICMmIMiNHpUHLCVABA5sZdcHz83zp6tH3QNFiWZqRJ9gqDSeupt+TIfolevS82uigf2/KINm53aBRclbojjj16W7k7WzvcJmC9UdWAqm4jLstMx2wHJdZmseIZl0bsb4WVAIDVpbV4Teyt2hZ9sXGji0x57V3Y2NAKs6goXnEwiwGAmQlmoMzYaDQ59KCvyivBvvZuxOoI3Q5KgTGG+0Q5XNnf3mVXgH8rrHQabF66uzfU7b2FFXh/2hgcs3U/jnYYxK/pERqz/PYuewN+stioOPLgoUrUm8y4eHeR02C046I4X9a14Mu6XlPi82VHkG6Iwo0jBgMQFK+FMUTrdOi0WFHiogzP2FmI3+f0ji8Vis9SvF6HIz0m1BhNGCjWvab8iD2dlTE8UFiJy4alY2JCnFOj1WKyIEEvfFFXdRtx9h+H8N60MXikWGggjv5lHzosVhyT1ICT0pPsz/i243Jw1s5Cp4mOjDH7cp1/L6rCzSMG2zsDn9QKXywV3UaMFBcPym3pwLm7DuGOkUOwYvRQvFlZjzkpiajqMaKmx4Q0QxQu31uCHXOOsn8tqo2qoQpEBf8FY2yylzTXA7geAEaMGDHj8GHPL3Rf58OaRhR29sBAhIXpSZg+IAGtZgsOdXZjULQBmxpbce6QVDxbWotrsgbhiZJqbGlqQ5vZKmk7PSYpHnoSerk1C6c59aQPzp+C8Vt6P9XnpiTil+Z2vHTUSMxJScTUrfsQqyPMSUl0U7h9nYuHptn99COJe7IzsK+9y0kpSpEcpXe630clxHpsSDfMGIfFO6TtxeuOHo2L/exATEyIhcnK7O64/nL3qAwsz87ADw2t+ItY9wfTxuD8XdJx1i/LTMdpA5NxQmoS7jxQhvdrmnBv9lCvHmsL05KwsbENI2OjcU3WQDx4qMp+7KKMNMTrdbAwhp+a2lDaJW+sQAfA9dtj46wJ+POOg/bG1/UdA4A3JmcjJzEW3zW0IjVKb/fy+nbmeJzqYlo9beAAfFPfin9PGI47XcaQghnw9xaqIOwK3pFwxqKJdMxWhjnb85EVa8A92UNhIMLM5ASYrIJtMl6vw6/N7Xj4UBUuz0zHJZnpGL5pN0yMofrEqSAi1Ik9IiLCl3XNGBJtwJSkOKwurcUlmek4e2chKsUeHiDEzH9l8ih8VdeC58uOYHZygpNtcs/xk3Coswfn7joEAxFMETir1Zty5CjPmYNSMDs5AQ8ckv4C4Qim20aTc4eNK3iO39T2mNBqtmCcHz7YXxxpRopB7+QiarYyVPUYMSIuBh1mCzosglXZdfnCPW2dODX3IL6eMR5bmtpw/fBB+G9FPWanJNh7mfF6HW7IGoTKHiPeqxFMLOPiY3Dx0HSMjIvGy+V19kakZMHRyN4s+OpfOjQdcXrCKxX1uCgjDe/KMDHtmHMUNjW2YfmBciwZnILv6lskPZdcOWNQMr7w0ct2ZFx8DEbFxaDZZMHv4mDvI+OG2c09kUBWrAEV3SbfCTlhwdYJCwSu4Dlhp7CjG4Ojo5DsYM/d2doBo5VhVnKCU9weo9UKMxMag8+PNGNteR3WTR2NOL0OmxrbMCs5AcsLynH6oGREEWFiYiyGRBuQFKXH8dvyUdzVg6cnDMelmelgjCG/oxs5CbH2Fyi/vQu72jrxwuEjeD5nBF6vrMdDYzPBGOzLMW5rbserFXX454ThOGI048HCSvzUJJiydh8/Cd82tCCvrQstZguenjDc7pNv+4SX+py38di4YU429l3HT8LW5nb7eMOSwSmYnhSPG0cMxrbmdpztMkYhxT8nZCHdEIWXy+uwXcLz5vAJR+P1inr73AAAWDUmE6scfnsiK9aAO0ZmYLkX19Rzh6RiV2snirt6kG6IQkOY10mI1+ucxgscmZeSiJ8VcItVkltGDMYDLvM15BIWBU9E6wCcCGAggFoADzHGXvOWhyt4TrAwxlDRY8Lw2MAXZvaEnJDNTSYzCECKIQpNJjM+rG1Ct8WKW0c6T1Aq6uwGg2Bvty0844mMjbtwbHICThZnxCbodTg+JRErD1bg6/oWfDNzPIbG9J7vJ7VNWFfdiFtGDMbQGAMyYwxIEBugZfllWF/TaO8xVvcYkW6IwoLfClDaZcT/pmTj3oMVqOwxIYqAnIQ4fDtzPGqMJhyzdT8AYHhsNOamJOLdmkasGpOJXW2d9kYuv70L4xNi8VJ5Hc4ZnIKMGAM+r2vG5MQ4RJEwmPxGZT0eGTcMnx1pRk5iLG7dX4aDHd34bPo4JEXpkG6Iwm8tHbhcnFS3Yfo4FHR0484D5Xh8fBa+rW/BksEp+KWpHSPjovFBTRMOO/jl7zp+ElKi9Mht7UBplxHTkuLwYnkdPqxtwrj4GGyePdE+QH+Lw4TFBL3OPmchmgiHTzga/yytgYEIT5bUYPqAeOxsFTykUqL09jknhfOnYJw43vXNzPG4eHcRGk0WZEQbMCg6Ci9PGoUnS6pxQUYacsT7V9ljgg7A3QfKkdvaidtHDsHK0UO9PgeeCFsP3l+4gudw3KkzmpCo1yNOYkavv5itDD1Wq13he6LbYnWbQfxtfQvmpCQiKUoPK2PY2dqJmcnyVj3yl26LFQ8XVeHCjDQcMyAejDHktnZi5oB4j6aMw109GBEbLXncbGXY2dqB2SmJTvuNVisKO3uQEqVHRowBBzq6MSTagAS9zun8LYxBB8DCgHaLBclRejxfdgTnDUnFsNho7G/vwqDoKJ+NtSsmK0O1aP4MFK7gORwOR6PwBT84HA6nH8IVPIfD4WgUruA5HA5Ho3AFz+FwOBqFK3gOh8PRKFzBczgcjkbhCp7D4XA0ClfwHA6Ho1G4gudwOByNwhU8h8PhaBSu4DkcDkejcAXP4XA4GoUreA6Hw9EoXMFzOByORuEKnsPhcDQKV/AcDoejUbiC53A4HI3CFTyHw+FoFK7gORwOR6NwBc/hcDgahSt4DofD0ShcwXM4HI5G4Qqew+FwNApX8BwOh6NRuILncDgcjcIVPIfD4WgUruA5HA5Ho3AFz+FwOBqFK3gOh8PRKFzBczgcjkbhCp7D4XA0ClfwHA6Ho1G4gudwOByNwhU8h8PhaBRVFTwRLSKiA0R0iIhWqlkXh8PhcJxRTcETkR7AfwD8GcBRAC4moqPUqo/D4XA4zqjZg58N4BBjrJgxZgTwLoCzVKyPw+FwOA5EqVj2MADlDr8rABzrmoiIrgdwvfiznYgOBFjfQAD1AebVAvz8+/f5A/wa9NfzH+npgJoKXhaMsbUA1gZbDhHlMsZmKiBSn4Sff/8+f4Bfg/5+/lKoaaKpBDDc4XeWuI/D4XA4IUBNBf87gHFElE1E0QAuAvCZivVxOBwOxwHVTDSMMTMR3QrgGwB6AK8zxvapVR8UMPP0cfj5c/r7Nejv5+8GMcbCLQOHw+FwVIDPZOVwOByNwhU8h8PhaJQ+r+C1Gg6BiIYT0UYi2k9E+4hombg/jYi+I6JC8X+quJ+I6DnxOuwhoukOZV0hpi8koivCdU6BQER6IvqDiL4Qf2cT0XbxPNeLA/ggohjx9yHx+CiHMu4V9x8gotPCcyaBQUQpRPQBERUQUT4RzelPzwAR3SE+/3lEtI6IYvvbMxAUjLE++wdh8LYIwGgA0QB2Azgq3HIpdG5DAUwXt5MAHIQQ8uEpACvF/SsBPCluLwbwFQACcByA7eL+NADF4v9UcTs13Ofnx3W4E8A7AL4Qf78H4CJx+yUAN4nbNwN4Sdy+CMB6cfso8bmIAZAtPi/6cJ+XH+f/JoBrxe1oACn95RmAMFmyBECcw72/sr89A8H89fUevGbDITDGqhljO8XtNgD5EB74/2/vXEOtKqI4/vuTvUy6WpBUUmqJQVIaET2lLOytvago6EkQBlEfKqovSVBh0gvCgqyorECTEhPNSj9kpmWKWWZZRk97d8ve2b8Ps473YOfmrevt3HPO+sHm7j2zzuxZc9dZZ2b27DXjKV964u9pcT4eeMSFV4D+knYHjgfm2/7G9rfAfOCE/1GV/4ykQcDJwANxLWAMMCNENte/0i4zgGNDfjzwpO1fba8D1lLsptcjqQ0YDUwFsP2b7e9oIRugrPTbUVIfoC/wGS1kA92l0R18rXAIe9apLj1GDDVHAUuAgbY/i6z1wMA476wtGrmN7gKuBf6M612B72z/EdfVumzSM/LbQ76R9R8CfAk8FNNUD0jaiRaxAdufAJOBDymOvR1YRmvZQLdodAff9EjqBzwFXGX7++o8l/FnU65zlXQK8IXtZfWuSx3pAxwETLE9CviRMiWziSa3gQGU3vcQYA9gJxpn5NEraHQH39ThECRtS3Hu02zPjOTPY9hN/P0i0jtri0ZtoyOAcZI+oEy9jQHupkw7VF7Qq9Zlk56R3wZ8TePqD6Wn+bHtJXE9g+LwW8UGjgPW2f7S9u/ATIpdtJINdItGd/BNGw4h5g6nAqtt31GVNQuorIK4EHimKv2CWElxKNAew/h5wFhJA6JHNDbSejW2r7c9yPZgyv/1RdvnAwuAs0Jsc/0r7XJWyDvSz40VFkOAYcDS/0mNbmF7PfCRpOGRdCzwFi1iA5SpmUMl9Y3vQ0X/lrGBblPvp7zdPSgrB96hPBm/sd712Yp6HUkZeq8EVsRxEmVO8QXgXeB5YJeQF2WDlfeAN4CDq8q6hPJgaS1wcb11+w9tcTQdq2iGUr6ca4HpwPaRvkNcr438oVWfvzHaZQ1wYr31+Ze6jwReCzt4mrIKpmVsAJgIvA2sAh6lrIRpKRvozpGhCpIkSZqURp+iSZIkSTohHXySJEmTkg4+SZKkSUkHnyRJ0qSkg0+SJGlS0sEnvRZJu0paEcd6SZ9UXW+3hc8eLOmeLtzj5a1X47+V3V/ShJ4qP0m2RC6TTBoCSTcBG2xPrkrr446YJL2OiCE02/aIOlclaVGyB580FJIelnSfpCXAJEmHSFocwbherrz1KelodcSQv0nSg5IWSnpf0pVV5W2okl+ojtjr0+LtSSSdFGnLIt767Br12l/S0hhdrJQ0DLgN2CfSbg+5ayS9GjITI21w1T1XRx36Rt5tKnsCrJQ0efP7Jsk/0WObbidJDzIIONz2Rkk7A0e5bPJ+HHALcGaNz+wHHEOJrb9G0hSX+CbVjAL2Bz4FFgFHSHoNuB8YbXudpCc6qdPlwN22p8X00TaUwGAjbI8EkDSW8pr8IZS3TmdJGk15JX84cKntRZIeBCZIegg4HdjPtiX1//dNlbQy2YNPGpHptjfGeRswXdIq4E6Kg67Fsy7xwL+iBOcaWENmqe2Pbf9JCQ0xmPLD8L5LHHGAzhz8YuAGSdcBe9v+uYbM2DiWA69H2cMi7yPbi+L8MUqoinbgF2CqpDOAnzq5d5LUJB180oj8WHV+M7Ag5rlPpcQjqcWvVecbqT167YpMTWw/DowDfgbmSBpTQ0zArbZHxrGv7amVIv5epP+g9PZnAKcAc7tanySBdPBJ49NGR+jXi3qg/DXAUHXs73lOLSFJQyk9/Xso0Q0PAH6gTAlVmAdcEjH+kbSnpN0iby9Jh8X5ecBLIddmew5wNXDgVtMqaQnSwSeNziTgVknL6YFnSjHVMgGYK2kZxWm31xA9G1glaQUwgrJ13tfAIpUNo2+3/Rxlf9nFkt6g9MwrPwBrgCskraZEjJwSebMlrQReouxPmyRdJpdJJskWkNTP9oZYVXMv8K7tO7di+YPJ5ZRJD5A9+CTZMpdFz/xNypTQ/XWuT5J0iezBJ0mSNCnZg0+SJGlS0sEnSZI0KengkyRJmpR08EmSJE1KOvgkSZIm5S9J0+VBHTTYFAAAAABJRU5ErkJggg==\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "9468\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 350
        },
        "id": "3iZTVn5WQFpX",
        "outputId": "2ca1e261-4681-403a-c325-d4868ad97e6f"
      },
      "source": [
        "del model\n",
        "model = NeuralNet(tr_set.dataset.dim).to(device)\n",
        "ckpt = torch.load(config['save_path'], map_location='cpu')  # Load your best model\n",
        "model.load_state_dict(ckpt)\n",
        "plot_pred(dv_set, model, device)  # Show prediction on the validation set"
      ],
      "execution_count": 59,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 360x360 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAU0AAAFNCAYAAACE8D3EAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOydd3iUZdaH75NeIAVI6AGkqoDBRUV0pYmAq4is2Hvv+im6q66Kuq664lpWxV6wK4uCnQ67gh2MdBAhtFCTENLL8/1xZpwhJGGCmdRzX1euybzzlmei/jznOU2ccxiGYRiBEVLXCzAMw2hImGgahmFUAxNNwzCMamCiaRiGUQ1MNA3DMKqBiaZhGEY1MNE0AkJEOouIE5GwOnj2ehE5sbafW9uU/xuLyOcictFB3CdFRPaKSGjNr9Iw0axHiMjZIvKNiOSKyHbP79eKiNT12qrC8x+o96dMRPL93p9XzXu9JiJ/D9Zafy8icrGIlHq+2x4RWSIipwTjWc65Uc651wNY0z7/U3HOpTvnmjnnSoOxrqaOiWY9QURuBZ4EHgXaAK2Bq4HjgIhKrqkXloTnP9BmzrlmQDpwqt+xt7zn1YWVGiQWeb5rAvAy8L6IJJY/qRF9X8MPE816gIjEA/cD1zrnpjjncpyy2Dl3nnOu0HPeayIySUQ+E5FcYIiIHCoi80QkS0SWichov/vOE5HL/d5fLCL/83vvRORqEVnjuf4Zr1UrIqEiMlFEdorIOuBPB/G9BovIJhH5i4hkAK+WX4PfOrqJyJXAecDtHkvuY7/TUkUkTUSyReQ9EYmq4HmRnu/R2+9YksfyTS53bjcRme+5304Rea+63885Vwa8AkQDXUVkgohMEZE3RWQPcLGIxIvIyyKyVUQ2i8jfvf+zO9DfuIJ/fleIyAoRyRGR5SJypIi8AaQAH3v+ZrdX4Oa3E5HpIrJbRNaKyBV+95wgIu+LyGTPfZeJSP/q/i2aEiaa9YNjgUhgWgDnngs8CDQHvgE+BmYAycANwFsi0rMazz4FOAroC5wJjPAcv8LzWT+gP3BGNe7pTxugBdAJuLKqE51zLwBvAf/0WKmn+n18JjAS6OJZ68UVXF8ITAXOKXfdfOfc9nKnP4D+3RKBDsC/A/9KikeULgf2Ams8h08DpqBW6FvAa0AJ0A39W57kuQaq8TcWkXHABOBCIA4YDexyzl3Avtb9Pyu4/F1gE9DO84x/iMhQv89He85JAKYDTwf4J2iSmGjWD1oBO51zJd4DIrLQYzXli8gJfudOc8595bFyUoFmwMPOuSLn3BzgE/YVjQPxsHMuyzmXDsz13BNUbJ5wzm10zu0GHjrI71YG3OucK3TO5R/kPQCecs5t8azlY791ludt4Gy/9+d6jpWnGBXyds65Aufc/yo4pzIGiEgWkIH+rU93zmV7PlvknPvI888nDjgZuNk5l+sR7sf91ledv/Hl6P9MvvN4IWudcxsOtFAR6Yhu8fzF8z2XAC+h4uvlf865zzx7oG8ARwT4d2iSmGjWD3YBrfz3wJxzA51zCZ7P/P85bfT7vR2w0fMfqJcNQPtqPDvD7/c8VIR/u3e5+x4MO5xzBQd5rT+VrbM8c4EYETlGRDqj4vphBefdDgjwrcclvbQaa/naOZfgnGvlnBvgnJvl95n/36wTEA5s9fwPMAt4HvUKoHp/447AL9VYo5d2wG7nXE655/j/O1L+bxtl+7GVY3+Y+sEioBB17f5zgHP921JtATqKSIifcKYAqz2/5wIxfue3qcaatqL/oXpJqca1/pRvo7XPmkSk/Jp+V9st51ypiLyPWoDbgE/KCYb3vAzUPUZEjgdmicgC59za3/N89l3/RvSfayt/L8KP6vyNNwJdA3hmebYALUSkud/fIQXYXMU1RhWYpVkPcM5lAfcBz4rIGSLSXERCRCQViK3i0m9Qy+B2EQkXkcHAqej+FMASYKyIxIhIN+CyaizrfeBGEengiQz/tZpfqzJ+Ag4XkVRPMGdCuc+3AYf8zme8DZyFBpUqcs0RkXEi0sHzNhMVnrKKzj1YnHNb0X3Tx0QkzvPPtKuIDPKcUp2/8UvAeBH5gyjdRKST57NK/2bOuY3AQuAhEYkSkb7ovwdv1sBXbJKYaNYTPBv4t6Bu4zbPz/PAX9B/6Su6pggVyVHATuBZ4ELn3ErPKY8DRZ57vY4GJgLlReBLVOR+RAMsvxvn3Go0U2AWGjwpv5f4MnCYx5396CCf8Q1q0bYDPvce90SX/+h5exTwjYjsRYMfNznn1nnOWybVzC+tggvRlLHlqDhPAdp6Pgv4b+yc+wANAL4N5AAfoQE20L3Qv3n+ZuMruPwcoDNqdX6I7jHPquA8IwDEmhAbhmEEjlmahmEY1SBoounZP/lWRH7yuDv3eY6/JiK/ipafLfHs2xmGYTQIghk9LwSGOuf2ikg48D8R8e4v3eacmxLEZxuGYQSFoImm083SvZ634Z4f20A1DKNBE9Q9TU9t7RJgOzDTE9UEeFC0jvhxEYkM5hoMwzBqklqJnotIAprqcANa4ZKBpmG8APzinLu/gmuuxFOrHBsb+4devXoFfZ2GYTRStmyB0lII1cZg67MS2JUfQwI/lGY6Vy2Pu9ZSjkTkHiDPOTfR79hgYLxzrsp+hP3793fff/99kFdoGEajJS0NJk6kOK4lF867hHeX9eX+I/7Dkz+d8ctO57pV51bBjJ4neSxMRCQaGA6sFJG2nmMCjAGWBmsNhmEYAPTtS9FNt3H2nCt4d1lfHjlxJndP7s4uyKrurYIZPW8LvO7pHRgCvO+c+0RE5ohIEtosYQnaaNcwDCNoFBbCuPv68PEKePxxuPnm4Qd9r2BGz9PQPoHljw+t4HTDMIygkJ8PY8fCF1/As8/CNdf8vvtZlyPDMBotublw2mkwZw689BJcVp2WNZVgZZSGYTQ+0tLIueMfnNx1FXPnlPHaA+k1IphgomkYRmMjLY3sfzzDiNfO4avt3XhrxGQuXPU3jaDXACaahmE0KjLf+ozh8+7ku+0pvHfGFM4+Zj0kJsLUGuluaHuahmE0HnbuhOEvncPy7PZMPfM9Tu3pGWIQHw/p6TXyDBNNwzAaBdu3w4knwurs9kz704uM7LnN92F2NqQc7MSWfTH33DCMBs/WrTB4MKxdC59OSmdk/CLIzISyMn3NzNS8oxrARNMwjAbNpk0waJB6359/DsOuOATGj9d9zE2b9HX8eOjbt0aeZ+65YRgNlvXrYehQ2LULZsyAgQM9H/TtW7FIpqVpQCg9HVJSiIHo6j7TRNMwjAbJL7+oYO7ZAzNnwtFHH+ACT9MOEhOhQwfIzKQttK7uc000DcNocKxaBcOGaYnk7Nlw5JEBXDR1qgpmYqK+T0ykBEqr+2wTTcMw6pZyLjNjx1a5/7h8uVqYZWUwbx706RPgc9LT1cL0o+wgRNMCQYZh1B1elzkz8zeXmYkTK63eSUvTKLlINQUTVJCzs/c5FAKh1V2yiaZhGHWHv8scEuL7vYLqnR9/hCFDICIC5s+Hww6r5rPGjvWlH3lSkcJMNA3DaFCkp2u1jj8VVO98+63uYTZrBgsWQI8eB/Gsvn33S0XaCtsOfOG+2J6mYRh1R0qKWn7e4Az4qnc8e51ffRfBqFm3kpQMcxZE0qnT73heuVSkvPvuy6/uLczSNAyj7qjAZSYzE3r3hokTmfdzS0bMvJW2MVnMP/p2OmXXTKei34OJpmEYdUcFLjPjx8PSpczKOYaTp19Np4Rs5l36hga+a6hT0e/B3HPDMOqWCqp3Ph8/m9PnXEmPlruZdeFkkmNzoazmOhX9Hkw0DcOoV0yfDuPm3MDhLbYy86K3aRnj2XaswU5Fvwdzzw3DqDdMmQJ//jOkHlrE7CEP0rJwS1A6Ff0eTDQNw6gXvP02nH221pDP/CqGxLuuDVqnot+DueeGYdQ5r78Ol14Kf/wjfPKJ5mNW2qmojjFL0zCMOuWll+CSS7Se/LPPPIJZjzHRNAyjznjmGbjiChgxQgNAMTF1vaIDY6JpGEad8PjjcP31MHo0fPQRRFe7HXDdYKJpGEat8/DDcMstGin/4AOIjKzrFQWOBYIMw6hV7r8f7r0XzjkHJk+GsMpUqJp9NmsLszQNw6gVnIO//U0F88IL4Y03DiCY1eizWZuYaBqGEXScg9tvhwcfhMsvh1dfhdCqOllWo89mbRM091xEooAFQKTnOVOcc/eKSBfgXaAl8ANwgXOuKFjrMAyjbnEObr4ZnnoKrr0W/v1v1cF9KO+KL1myvyteQZ/NuiCYlmYhMNQ5dwSQCowUkQHAI8DjzrluQCZwWRDXYBhGHVJWpkL51FPwf/8HTz9diWCWd8V//RXWrt33vMZee+6UvZ634Z4fBwwFpniOvw6MCdYaDMOoO0pLNQfzuefgL3+Bxx7T2T77UZEr3rs3LFu2f5/Nxl57LiKhIrIE2A7MBH4BspxzJZ5TNgHtg7kGwzBqn5ISuPhieOUVuOceeOihSgQTKh550bUrdOnS9GrPnXOlQKqIJAAfAr0CvVZErgSuBEipBya5YTR5AkwBKi6G88+H99+Hv/8d7rrrAPetbORFaipMmFCjX6EmqJXouXMuC5gLHAskiIhXrDsAmyu55gXnXH/nXP+kpKTaWKZhGKDiOGGCdtCYMEHfB5gCVFQEZ52lgvnoowEIJlQ+8qIeuOIVETTRFJEkj4WJiEQDw4EVqHie4TntImBasNZgGEY1qUwcJ006YApQQYHq3IcfwpNPqjcdEJWNvKgHrnhFBNM9bwu8LiKhqDi/75z7RESWA++KyN+BxcDLQVyDYRjVwT8oA77XBQvg1FP3PdcvBSg/H8aMgRkzVF+vvrqaz62nbeAqImii6ZxLA/pVcHwdcHSwnmsYxu8gPV0tTH/i4zXZMju7wlG7ubmqp/Pmwcsvq1ffmLGKIMMwfKSkqBj6k50NAwZUuO+YM+IMRo2C+fO1jryxCyaYaBqG4U9lQZlrr91v3zH7qts56ZbeLFyooyrOP7+uF187WJcjwzB8eIMy/qlFl13m22/0vO7+7zJGnJnATxklfHDGFE4/9DCgYexJ/l5MNA3D2JcDBGV2zl/G8NObsTy7NVPHvcsprb+DiZ/V64h3TWLuuWEYgZGWxrbxjzJ4ZCQrs1ozfeSznNJrbb3qQFQbmGgahnFg0tLYcv9LDH7lAn4tas+nPW5hxKaXISNDP68nHYhqAxNNwzAOyMZXZzFo1t1syk3kiz63MzT+B4iKgpUr9YR60oGoNjDRNAyjStavh0EvXcD2gubMOP8N/nhsiZb/OAdZWfW+7LGmMdE0DKNS1q6FE06AzJJmzBrzDMd23AStW8PAgb62RfW87LGmsei5YTQlqjGsbOVKGDYMCgth7usbSf3kJ8hM1P3LiAjo2bNJiaUXE03DaCp4m3EkJkJ4OHz+udY9tm6tpZOpqb+J6NKlcOKJ6oHPmwe9e/eAXlXkbzYhTDQNo6ngbcZRWAhff60VP3l5sHGj9nSLiYGJE/np1L9x4rU9CA+HOXOgl7cLbgNqqhFMbE/TMJoK3g7pK1dq5DsnR19FIDoatmzhh+I+DLmwI1FRWk/eK+C24U0HszQNo7Hj3cf88Uedu5OdrS55QYHO0Y2Kgqgovt6SwsivrychPJe5C6Lp0qWuF14/MdE0jMZAZQEe/33MY47Rvpg7d0JYmApmYSHEx/O/ZYmM2v0srSN2Mmf0s6R0eaiuv1G9xdxzw2joVDWKwr+pcNu2MGgQtGkDW7ZAixYQHc289Z0Zsest2ssW5kePImXTwv3GWBg+TDQNo6FT0Qhcby14+UmPbdqoFXrCCfDnPzOzaBAn50+hc0g689qeS/sOAtu3a/t1o0JMNA2joVPRCFxvLXhlTYVTU/ns6AmcuuNlukdvZt4xf6VNt2bQsiU0b67RdaNCbE/TMBo6KSmwZg1s3qyCGB8P7dtD9+7Quzc88IDO1U1K0uNhYXzU5f84cwz0ifqFGb1uomVE8b73dK5uvksDwCxNw2jo9O4NixZpHXjz5vq6aJH+Pn26fp6UBDt2wLJlfJB0DeNu68yRR8Lss16kZf4mnYzmnL7m5Oh4C6NCzNI0jIbO0qUqclu2qKWZkACHHQYffwxHHKH7m927A/DWokO48PFjGHgcfPopxK2/BO5co4KanQ2RkdC1q463MCrERNMwGjrp6dCtG/To4TtWVgbffacBHw+vLUnl0hmjGdRmFR9/fijNmqFpSf/4R8D16IaJpmE0bNLSYN06mDtX3evISE0tio7WfcwpUyA5mRfKLueq/45heMeVfDT8WWImttxXJCdMqOtv0mAw0TSMhoo3PzMmRvcxndMa8o0boaQEDj8cSkt5eu0Ibth8Pie3W8x/Dr+fqO0l0Cxk35zOJtit6GCxQJBhNFS8+Zl5edCqlVqWzqlrHhMDW7bwr5BbuWHzHZzWbDZTu95OVOc2cMghFed0GgFhlqZhNBS8FT5Llqhl+csvKoDbtqmFGRurZZG5uVBUxEN7b+DO7RcxLmkub502hfDSjvp5UtK+921C831qAhNNw2gIeF3x0lLdwwwJUaHculWj3vn5ammWlOBKS7k//zYmcB/nRnzA68kPEvbhJi2bbNdOhdMTTQea1HyfmsDcc8NoCHhd8c2bNciTkADJyb5k9rw8KCzElTnu4h9M4D4u5lUmh1xCWMYmFdvmzVU0Fy3SZPiysiY336cmMEvTMBoCS5aouC1dquIXE6NueEGBimhpKS40jPE8yr+4mStDXmKSXEtIUSm0iNea87IyX1rS5s0aaW/CHdgPlqCJpoh0BCYDrQEHvOCce1JEJgBXADs8p97pnPssWOswjAZPWhr8+qs2C46I8O1hhoSo8O3Zg5MQbox6nqdzL+X6sOd4KuwWRMKgsFTbwBUVaV05aE5nVBS88krdfq8GSjAtzRLgVufcjyLSHPhBRGZ6PnvcOTcxiM82jIbDgYadTZ2qpZDffacljoWFvii5c5SFR3JN2Eu8kHsJt7R8jYmtnkTWlWjakXOwaZMKpzfR3X8PsxqD1gwlaHuazrmtzrkfPb/nACuA9sF6nmE0SKrqhen9fNo07bi+e7cvrchDqQvhstyneKH4Eu6Qh5lYcAOydYue47VMQ0JUYH/4AVav9u1hHujZRoXUSiBIRDoD/YBvPIeuF5E0EXlFRBJrYw2GUS+pqhemV9QiIvRn714VTc+88RJCubDsNV4rPo8JsY/yYNITSHSU7nOGhak73ratphhFRmqwaMsWXyJ7Vc82KiXooikizYD/ADc75/YAk4CuQCqwFXiskuuuFJHvReT7HTt2VHSKYTR8quqF6RW1fv3UJfe620AxYZzL27zNufyDO7k3diKSnKRVQHFxvmFpoIIbH68R90MO8bnfVT3bqJSgiqaIhKOC+ZZzbiqAc26bc67UOVcGvAgcXdG1zrkXnHP9nXP9k8on4xpGY6GyJsEpKT5Ra9MGjj32Nwuz0EUwTv7DB5zJY9zCHTyk6UchITo+MiVF3fGCAhXZ4mIV3Vat9s3HrOrZRqUETTRFRICXgRXOuX/5HW/rd9rpwNJgrcEw6j1jx/pyJcvnTaakaNXPvHnwzTcQFkZBWDPGyodMc6P5d/gt3BLypLriOTkqkitXqsgmJ2tuZl6ePqd5c+jced98zKqebVSKuCB1aBaR44H/Aj8DZZ7DdwLnoK65A9YDVznntlZ1r/79+7vvv/8+KOs0jDrHP4IdEaEWZWGh7mF+9ZXuScbFkffzL4zJeo2ZDOf55uO5svk7el5RkbrdISGaf5mXp/01ExJg7Vq1NgcM0B6Z5SPjTTx6LiI/OOf6V+eaoKUcOef+B0gFH1lOpmH407fv/uN2k5Lgyy9VRMPC2Ls1h1Pz3mc+x/JK5DVc0mY2JHoi3t266TC0det0HzMlBfbs0SDQ669XLYLeZxsBY2WUhlFfKB/NLiqCpCT2lMYycssrLCgawBtx13OJe0U7raekaIf2/v2hWTOtJ+/eXfcui4osEh4krIzSMGqbylzi9HTNl/QSH0/WnhBGLnuU74v78G7cVYyL/gQS2qggrl6tAaLsbP2Ji9PrCgo0gGSR8KBgomkYtYm/C16+CXBKigqhZ9bPbpfISSueIK24F1NaXs2YsE90fzIpSdvA7dih+5+ZmerG5+fr+4ICTVOySHhQMPfcMGqTqhLKe/fWeeNZWeyI7MCQVZNYWtSDDzvcyJjiDzRBvWNHdcULClQ8CwtVcI88UiuGQF30xYt1clpGhlX41DBmaRpGMCnvis+fr+k9e/ao+9yrl1qPH32kP7GxZLjWDPvpCdYVdWD6kMc5qafAws5qRcbGqkVZUKBTI1NS1LWfNEmfNWkSzJyp1UDDhqnQ2jiLGsVE0zBqGv8O67/+qhZk167qei9ZojmUbdqo+M2Zo8nnyclQWMjm0BSGrniGTWVt+ez8dxjSKQ82FcLdd8MDD6hLnpSk9wsL2zensm9faN0a/vQntV79mTrVRLOGMNE0jJpkyhQVt+JizZeMjNQemHFxuleZnKyjKhIStD3bnj1aHjlyJOmLdzF0yWNsK2nBl0nncXzaL7AyQl3vM87QXpheqzUyUi3UJ56oOpgEFhCqYWxP0zBqirQ0FUwRtQZzc33VNitWaGCmdWu1AqOjVTABWrbk14ieDFr+LDuL45nZbCzHl8yD8HA9Z+NGvXffvjpq9+ab9d6Rkft3J7LSyKBjomkYNcXUqWphxsf79h9FVPiysyE0VF30rCw9/5hjoFMn1kT14YTXLiG7OJbZPa5lQPRP6nrHxMCgQeqK++dbVhVMstLIoGOiaRg1RXq6WpgFBfq+VSt9zclRcdu1yzdud+dOmDKFlWvDGLTsGQoKhbkXvMIf5Ed15Tt0UKFduVLv5+9eext5bNumdenTpule6ZIlao2OH68iummTvloQqEaxPU3DqClSUjQFaKmnB01srIpbdra608nJcNRRWu64bh1LpQ/D9kxFwkOZl3Ixh2/3zCvftUutUm+kfMECtTj9n7NmjT4nKkpFNjtbLVivG28iGTTM0jSMmmLsWBW73r1VzHbs0L3LZ56BgQNhxAgte2zWjCXtTmbwnmmEUsa80Y9zeMJm2LCh8nuL7PucpUv1WFSUrwXc4Ydb2WQtYJamYdQUXtd46lQN0gwZ4otqL12qe4uJiXy/tT0nrXuWZiF5zOl+Nd1W/uqLhuflaQCotNSXy5maqhas/3O6dNH7ec/p108tWYuSBx0TTcOoSSpzjceOhYkTWbS1MyN/eZ4WodnMPeRyOkftgNAoPSchQV+zsrTqZ/BgfZ+ZqR2L/ElN/U2EfyMz06LktYCJpmHUJP6J7d58zNRUGDuW/w69h5Ov6kibqF3M6XgJHfu1guVbNUhUWKj5mKA9NLdv1+h3draK4WWX7fscjwgDvn3Tis4zahwTTcOoKbzNOEpLdXrknj3aom3tWuZ8WcypSx6gY5cQ5sxpR7udE1Vcly/XvcmBAzWHE6BPH20mvGmTWo6XXba/9eq/FeAt0azoPKPGCUg0RaQT0N05N0tEooEwz1hewzC8TJ2q1T3z5mnzjPBwiIriy619GbP8bromZDB7fjvVxnYeN95rMUZE+CzL0FB46qkDC6BFyeuEA0bPReQKYArwvOdQB+CjYC7KMBokS5ZowCcrS5PTRfgk5wRG736VnlHpzG1//m/G5G9YXmWDIxBL8zp0YuQ3AM65NSKSHNRVGUZDJCtL9ydDQkCED0tHc1bhKxwhaXwZfQ4tduT48ij9MYuxQRFInmahc67I+0ZEwtChaIZheElLg61bNdfSOd7LH824vNf4Az8wK2wULdit1ULeGnGjwRKIaM4XkTuBaBEZDnwAfBzcZRlGA8IbAEpIgPbteTP0Is4tncyxLGKGjCTeeWrNe/WyuT2NgEDc878Cl6GjeK9Cp0m+FMxFGUa9o3wz4d69df8yPV3LIouLIT2dV3acyuWlTzM4fCEfl5xMbHgRtGil9earV0OLFpaA3sA5oGg658qAFz0/htH0KD/X57vv4Mkn1bJMSdG0oexsngu/gWtKH+OkkFl8WDKGmPBi7YHZsqXeJz9fg0WjRtXt9zF+FwcUTRH5lQr2MJ1zhwRlRYZRF1Q2IRL2bcW2bRt8/72WPO7Yoe/z83mKG7mp+DH+FD2bKUnXEbWtBELDNCiUn6814s5pKpK1aWvQBLKn2R84yvPzR+Ap4M1gLsowahWvJZmZuX9TX/C1YgNtJlxUpE0yCgogNJSJ3MpNPMnpfMjU5hcTVbLXF0UfONDXcFgEhg+3SHkDJxD3fFe5Q0+IyA/APcFZkmHUEl7rcto0TS7v18/X1Bd8c3VSUnx13tnZun9ZpAklD+bfwt+4jzN5jzflQsJzw3X/0jspMiICTjjBV+Z4zTV1+IWNmiAQ9/xIv7chqOVp5ZdGw8Z/n9I5/Vm0CI49VoeeeefqpKVpSeTs2Vqp4xwUFOCcYwL3cX/pPZzPm7zKxYSFhWijjYQEtSxvu00bEFuZY6MiEPF7zO/3EmA9cGZQVmMYtYX/PmVCgm/fceVKFc3sbLUS77oL1q7VLkNZWbBjB67McYc8wiPudi6R13gx5CpCIyP1vt75PH366HWFhfvvkRoNmgPuaTrnhvj9DHfOXeGcW1UbizOMoOG/T3noob5GvllZvrk6ItptKC5O9yWdw5WUcqt7lEfc7VzNc7wUeR2hiXF6bmIinH46nHaalkTOm6f15+X3SI0GTaWWpojcUtWFzrl/1fxyDKOW8N+nbN1aAzY//qifJSaqK/3EE2ophobC+vWU5eRyo3uSZ7ieG3mKJ0JvRcpCtElHXJzuXfbooWIZF6f3WrXK1xfTZo83CqqyNJsf4KdKRKSjiMwVkeUiskxEbvIcbyEiM0Vkjec18UD3Mowap/zUxogI7Xw+YIBaod7u65GRkJFBWVEJV5U+wzNcz3gm8kTYeCQ6yperefTR0K2b3js7W139qCifu26zxxsNlVqazrn7fue9S4BbnXM/ikhz4AcRmQlcDMx2zj0sIn9FK47+8jufZRjVo3w/Su+4ichIFc/MTJ03HhVF6d58LnY7sXIAACAASURBVMt7mtfLLuAu+QcPhN2HRESohSmioyf8O6nHx+seKfi2AGz2eKMhkOh5FFpGeTgQ5T3unLu0quucc1uBrZ7fc0RkBdAeOA0Y7DntdWAeJppGXeDfXWjCBLU2vQnsK1bA9u2URDfnwtLXeKdsNPdzD3eHPgTxCXouaF6npzP7b53Ue/bUCZLgE1Prqt5oCCS5/Q2gDTACmI/206xWA2IR6Qz0Q9vLtfYIKkAGUL7DoGHUPv6zxBcuhPx8ilu05pzlf+Od3NE8FH4Pd0c8oufm5OhI3ogIzcf0Rsa9fTGLi3Xk7uDB+rv1yGxUBJJy1M05N05ETnPOvS4ibwP/DfQBItIM+A9ws3Nuj/iNInXOORGpsM2ciFwJXAmQYm6NEWy8gaEVKyAqisLIOM5ccgfTC4byr/Db+b/Qf4MTdcnLytQt798f7r7bJ4bWF7NJEIilWex5zRKR3kA8EFATYhEJRwXzLeectx/WNhFp6/m8LbC9omudcy845/o75/onJSUF8jjDOHi8gaHt28kPa87pafcxPWcoT4fcyP9FTtL9zpAQ3fNMSNAk9sWL4dlnLZWoiRGIaL7giXDfDUwHlgOPHOgiUZPyZWBFufSk6cBFnt8vAqZVa8WGEQw87nVey46M/ul+vsgewAsxN3NdzKsqmKCpRyEhmssZEaFt3hYvthzMJkYg7vmrzrlSdD+zOp2NjgMuAH4WkSWeY3cCDwPvi8hlwAasusioJ+w9pC+n7H6T/+a24NVmN3JRwfPqhoOvAUd+vh5r29bXiMPbWNhc8yZBIKL5q4h8AbwHzHHOBTTqwjn3P0Aq+XhYgOszjFphzx44eVAuX69owRutbuHciA+hOFQbc0RHq1AWFanV2aKFuuf5+Wp9LlkCW7bojaxcstETiHveC5iFDlhbLyJPi8jxwV2WYdQemZnase2btCje7fo3zm0zR8sfY2N9o3UjI/V98+aax5mfrxfu2aM5mO3aWblkEyGQ2vM859z7zrmxQCoQh7rqhtHg2bULhg3Trckpg5/hjJCpmnZUXKzWZPPmal3GxsK4caquhYV6cfPmEBOjnx92mK8BiM0AatQE1OJNRAYBZwEjge+xfUijEbB9O5x4IqxeVca0cW8zatkr6mZHROjc8uxsdclDQ9W6nDRJL/T24XzrLbUwDzuM3waaW7lkoyeQiqD1wGLgfeA251xusBdlGMFm61a1MNf/WsYnJzzKia22wDHHwJo1mry+d6+vf2ZEhFqf5WeWN2vm647kxcolGz2BWJp9nXN7gr4Sw6glNm+GoUP19fOzXmdQ/BZft/ZDD9Upk6Wluq8ZF6eBn9hYn9vtbV589NFaLjl/vnY4ioqycskmQCDjLkwwjYZBVcPRPGzYoIK5Ywd8+SUc9/J/Ib6D74SjjoJfflH3/LDDfLOAUlN93Y+8e5egpZI//gjffANjxlh39iZAINFzw6j/HGg4GrDus5Wc0HsXu9bnMLPNBRz35jUaFfe2bwPdm+zQQV3yPXs03WjgQLUiU1L2bV7sPX/ECDjySG36YYLZ6LFZP0bjoLwFWG442pqPVzLkzFbkF4Uyp9MlHJm7CN7JVQHs1EmHqsXHq4B27qzHunb1HfO63VOn+lrAebF9zCaFdW43Ggfp6WohAmRk6KyfrCwAVqSew9AL2lNSUsLcLpfSd+9Czb0sKdEAz969ajHm5Kj4Pfig3sff1fd3u70t4MoLqtEkqMrS9HZn74nOPJ/ueX8q8G0wF2UYFVLVnqW3S1FhoU6VjIqCiAh+zuzAsDOSCHH5zEscx+G5q1Uwc3O1LNIbIV+9Gt54Y1/3uiJXu3zzYpsy2eQ4YOd2EVkAHOmcy/G8nwB8WiurMwwv/iN3vXuWd90F7dtrLmVEhIbDt2/XfUpgcUZbhm9+mciQYua0PpueJSshM9vXeAP0tXlzTWYPtH7cWsA1aQIJBLUGivzeF2GNg43aJC0NbrwRvv8efvpJQ99FRToid/FiFVHvuApPQvq3xf0YuuUNYsMKWNDzCnoWpmnpY3GxRsNB04oiIjStKCnJktKNgAgkEDQZ+FZEPvS8H4OOqTCM4OO1MLdvh1atVPgWLtQcyubNVTxDQtQC7doV9u5lYfJpjJx2La1CdzC3+1V0yvhGRbVTJ00n8lb6tGih9wwJUYvVgjlGAASSp/mgiHwO/NFz6BLn3OLgLsswPHij4jExKnilpepeFxZCmzYaxJk2TYMyPXuyoGQgJ394De2aZTOn5810+PV7vU/btlrB07OnFpzn5Oh9W7RQwQwL0z1SwzgAgaYcxQB7nHOvikiSiHRxzv0azIUZBqAuc3i45kwWFu7rhhcUaHpQXBzk5zP78yJO3TqBTh3LmHP6ZNr+UgKrirQ+PDZWrdSCAq2f3LbNl7BeSSK8YVREILXn9wL90Sj6q0A48CbaZNgwgktKCkyZotFu5/RVRIU0NFTFEPgiawCnb3qEbok7mLWoPa1bjwfGw9VX677nnj1qjfbrp8KbmqrJ6IZRTQKxNE9HJ0n+COCc2+KZY24Ywad3b3jsMXXPY2PV2iwq0sBNVBRER/Nx+hGcsf5RDmuRwcyj7qLVhGbw9dcqst2767kVJaobxkEQiGgW+U+NFJHYIK/JaMqUz8Xctk1d8N27fUGf8HCNoCcmMjXibM7acAX92mbw5bEPkJj2DawP1SARaPONpCQV27Q0TXhPSPA13zCX3KgmgaQcvS8izwMJInIF2sX9peAuy2iSVFQ//sknGvzJyVEXe/duddGBd4vGcubMyzkqaT0zT32KxDXfqusdF6cJ7Nu3w86dGkDavVuPH3GECqV1WTcOkkA6t08EpqCjeHsC9zjnngr2wowmiH/9eEiIWof5+VoWGR6uOZaeueOTQy7mvJ1PclzzNL5MuoD4NtHQpYvuc5aUwMaNen50tFqos2ap+HrvbV3WjYPkgKIpIo8452Y6525zzo13zs0UkQOO8DWMalO+g9DKlZpWlJ+vKUGRkRAVxctlF3Nx7tMMTljMZ9d+SvOjemlQJzVVhXH1arVMc3M1Wh4RoWK6adO+z7Mu68ZBEIh7PryCY6NqeiGGQUqKBmoyMmDePPj5Z3WrPWIJMKn0Ci4vmsSIxO/4JOU6YvN3+pLSe/f21Z+HhKilme0pm0xJ0X1Qf6w7kXEQVCqaInKNiPwM9BKRNL+fX4Gfa2+JRpNh7Fjdf5w/H/LyVCi9Y3Lj4ngycQLXFj7BqZEz+OiQW4iOcrBunYrspZfC009Dnz5a5VNWpqlJcXE636dnT3XxMzP1s8xM/bGEdqOaVBU9fxv4HHgI+Kvf8Rzn3O6grspoXATQUR3QYx07+mrL27bV4A/wz1/H8Zfc8YyN/ZJ3Ov2ViKwcbfybl6eWaHKyphmFhcFxx6mLHhWln+3cqcfvvluj6dadyPgdiPMfClXRCSIDgGV+XY7igEOdc9/UwvoA6N+/v/v+++9r63FGTeLfncg/T3L8+IoF69JLNXLu7UKUkcEDXxzFPRsu4+yk2Uwe8CzhR/ZR4fU2BC4s1P3PlSvVJY+I0GcVFOh9UlLgqadMII39EJEfnHP9q3NNIHuak4C9fu/3eo4ZxoEpHxE/UNTau68JuK0Z3P35QO7ZcBkXtP6SN79MInz6f3xjJdLTVRgXLVI3vkULrUXfvVuT4RMS9J7XX2+CadQYgYimOD9z1DlXho3JMAKlfEQcfFHrtDQVwEsv1de0NLUgMzNxq1bz16lH8/f0C7kscSqvDnmD0MfL5VWmpMCSJb9VBpGXp005IiN1fnlCAgwYoC65YdQQgYjmOhG5UUTCPT83AeuCvTCjkeBnOf5GdrYKW0WD0AB363j+779j+efOS7mm3Ue8cOYsQnt2299CHTtWOxY556tLDwvT0sm2bXVSZLdullZk1CiBiObVwEBgM7AJOAa4MpiLMhoRHstxv6i1cxW67WXPTOK6ywt5cvMZ3BTzIs8UXE7IR1O1MqigYF8B7NsXhg/XKPmePVqbnpyswum1bi2tyKhhAqkI2u6cO9s5l+yca+2cO9c5t702Fmc0ArwzdRITNbk8MVHfFxXt57aX5hVy5TtDmPT9Udwe9RSPF1+P5OVqwvr69TBjhgZ5/LnmGk0nOuEEHaVbUqIC2rOnpRUZQaGqaZS3O+f+KSL/BvYLsTvnbqzqxiLyCnAKsN0519tzbAJwBeDNMr7TOffZQa7dqK9UlGJUvg2bdxCaZxRuaZlwyYejeSPnZO5OeZ379kxA9pZBsafuPDZWX0X2f86ePfqshAR1yZ3zpSxZWpFRw1QV0FnheT3YXJ/XgKfRcRn+PO6pZzcaI1OmwAMPaOpPUpJ2KfrwQ60LT0315WiOHfvbHmZxXhEXTj2dd3efxP1tJ3F3yD8hP0+typIS/Sko8HUrgn1Tmfr29aUyXXONiaQRVKqaRvmx5/Wg5gE55xaISOeDW5bRIElLU8EUUYHbvVuPJSf7XOWJE305muPHU/Tv5znnndOYmnsSj7R7ktuTXoU12/UeISG+uvHISBVi7/6kfyoT+F4DnShpGAdJVe75x1Tglntxzo0+yGdeLyIXohbsrc65zEqefyWegFOKbeQ3DKZO9XUW2rBBK3FA3eeYmH2FDSh8fxrj3h/Hx7mDeXzgB9zcZwUsciqSpaV6LxFNKfJOjvTuT6ana9TdH2vAYdQCVbnnXhd6LNAGHXEBcA6w7SCfNwl4ABXjB4DHgEsrOtE59wLwAmhF0EE+z6hN0tN1P3GFZ2enuFjrvXfvVsGcN0+bABcUkP9NGqcvf5Av9xzKs50e4Zq9b4MMhGOPVVd7506tGy8q0nuK6L6mN+Wo3J4oYJFyo1aoyj2fDyAij5UrM/pYRA5qn9M595vYisiLwCcHcx+jnhIRoUnlZWW+/peFhZoClJGh3dQjIsjdtpfRC25jbl5PXurxTy5LmApEqdgOHgwDB8KyZfp7QQEsWKD3P+EEn4s/ejRMn67HbYyFUYsEkqcZKyKHeN+ISBfgoEZeiEhbv7enA1aq0ZgQ0Z9mzXQ/MjRUrcSSEq3W2bSJnN3FnJz9NvPyjub1Xg9x2ZB1KozOqRWamelrrpGYCN9+qxbnoEEaDffuYy5dWnEqk+1nGkEmkHLI/wPmicg6QIBOwFUHukhE3gEGA61EZBNwLzBYRFJR93x9IPcxGhCFhTrAbONGdau9rrVzUFxMdkEko3a/zLcFfXnrkHs4O+ozaHOauuSLF6vgJib60oTOOMO3dxni9/93795l374mkkatc0DRdM59ISLdgV6eQyudc4UBXHdOBYdfrub6jIZESoq64WvWqHXpR6a0YET2FBaX9uG9Q+7kzxEfQ4Rnlk9kpCajjx6tFuQTT/jyO23v0qhnBDLuIga4DbjeOfcTkCIipwR9ZUbDY+xY2LxZLcayMrUwgZ1hbRhaOpOfSg9naqur+HPkJ1oT3q+fz7X27lGWr0X3dmO35sFGPSEQ9/xV4AfgWM/7zcAHWBDHKE/fvhoAys1VgQO2S2tOLJnBarozLfIsRhbPg+RUePDBfV3rCRMqzrv07l36VxhZlY9RhwQiml2dc2eJyDkAzrk8Ef9aNqPRE2jn9bQ0tRw9lTxbS5MZVjaD9XTm09DTGBa/BNp0qrghcFV5l7Z3adQjAhHNIhGJxpPoLiJdgQPuaRqNBP9yRX+3efx4/XzSJJg7VyPf3hSj8HA2lbRhaNkXbKEdnzOKQXwF8YdoVLwiAbS9S6OBEIho3gt8AXQUkbeA44CLg7koox5RWbnis89qXuVPP/lG7HoqeNaXpTC09DN2kciM0JMZ6L6C+AQYOhR69Kj4OX616JZ3adRnqgwEiUgIkIhWBV0MvAP0d87NC/rKjPpBZZ3X583TBHTvTJ6QECgr45eijgzK/5zMsnhmxZ/BwOjFKrRnnOFrPOzffd1LZS3kzC036hlVWprOuTJPi7j3gU9raU1GfaIytzkrS3/3NtYoK2NVaTeGMYt8opkjJ9Jvz2K1QAcM0MR0L5U11bC9S6MBEEhF0CwRGS8iHUWkhfcn6Csz6geVdV5PSFDL0ZNetLywK4OYRxERzGMI/VjsqxnP9OvJYk01jAZOIHuaZ3ler/M75oBDKjjXaIiUj4737r3vfHBv0rl/ys+zz8KXX0JoKGn53Tmx+FNCKWUegzksZBVERmmCe3GxJrtnZECbNhbcMRo8gVQEdamNhRh1RPno+OrVMHmyljZ27apW4vTp++8vXnstbNrEjytjGL7uOaLJYw5D6SFrISpa8zVFtKVbaakGjSIjLbhjNHgOKJoiEgVcCxyPWpj/BZ5zzhUEeW1GbeAfHc/IgIULtf/lV19po4zWrX3n9e27j1X6bcTxjNh4A3Ex+cw97XkOmZUNWeG6x1lSou58s2a6r7llCwwZYonpRoMnEPd8MpAD/Nvz/lzgDWBcsBZl1CLepPKMDFi0SKt5mjXT14ULtU1bUpJvTrnHKv1qVy9GfXIdSSG7mHPac3Q6/UhY1VHd74ICFcrERGjRQvtgjhq1/5wgw2iABCKavZ1zh/m9nysiy4O1IKOW8UbHV67UDumxsZp3GRur71es0JSiyEi48UbYsIF5e47klN0P0j50K7Obj6XDp+nwmeheaMeO2kA4NFRFs7RURddqxY1GQiCi+aOIDHDOfQ0gIsdw8MPWjLqkonJIb1L59u3QqpVambt2aeR7wwYVvbIyddU3bGDWzlRG732DLvzKrLKTaJu3y9dDc/Vq7Xu5bp1eu2sXjBmj+5/mkhuNhEBSjv4ALBSR9SKyHlgEHCUiP4tIBVnKRr3E61qX7yIEGuRJTlYLMSpKXerwcN2XjIhQ1z0ujs93Hc0pe9+hG2uZyxDaui0aHS8p0blAZWUqvv37w2GHaaf2Nm3q9nsbRg0TiKU5MuirMIJPVdMbJ0zQJhoTJ8KqVRr88eZXxsXB2rVM/zyccTmvcjjLmMlJtGSXXu9p/0ZhoQ5P27pVrxOBdu32n0BpGA2cQFKONtTGQowgE8j0xthYWLtWrcwWnvqF0FCmlI7hnL0vciQ/8gUjSQzNgVJ++5zQUO3SnpCg4hkdrWJ62GE2WtdodATinhuNgZQUjWz7400097ruEREazGnTRidIRkTw9uZBnL33JY7mW2YynESydJ8zNNTXbDghQfM6o6L0fXy8Rt296UpWBWQ0Ikw0mwqVlUOOHbuv637YYWol5uXx+i/Hc/7mhzme//Fl+KnEhebpvbwD1BITVTAHDICjj4Y33oBLL4XUVJ9gglUBGY0KE82mQlVdhPw7GbVuDT168GLhBVxS8CzDmMNnUX+mWWi+WpfR0RopF1F3//nn4aOPdF+0b9+qxdkwGgGBBIKMxkJlXYTKdTJ6ZslxXF90M6MiZjE19EyiinN8AZ+4OHXP27VTy7L8/bzibOMpjEaKiWZTorKxFX4NgB9fMZJb1o5idPO5vH/040R+VwqFZWpZhoVpelFsrJ5fmRBaizejEWOi2djxCuWSJfDrrxro8Tbi8B9bERPDw+8fwh0Zo/hz1Ce83Wo8EWsLtMKnrExd+uJiTSs69lhtKmwYTRATzcZIRULpzZ1cunTfRhyTJkFuLvevPpt7M07mnLhPmJx4M2GhAll7tKQyKQm6dFGxTE5WATWMJoqJZmMjLQ3uuksrczZs0L3I/HztPNS6tTbTWLFCf4+Px03/mLsjHubB70ZwYesveCXlQUKlpe5bFhdrx6O9e2HkSE1Fysy0SLjRpDHRbOiU36dctkwT1OPi9PPQ0N9yLgkPVxHMyQHAtWvP7VtuZuKOEVze7weeL7uTkOjmet2ePdqZ6KuvdB8zOdkXCbd+mEYTxlKOGjIV1ZPPnq1WZXS0JpuLaIei3Fx1q/PyICICt3wFN085jok7LuLa7jN5/tRPCEmIU0u0oEBTkFq3hj59fC65DTszDLM0GzQV1ZOHhurQs6Qk7Vq0aZO66CUl0L497NpFWWEx15b+m+eLL+D/4l/mse3jkTc899m9W8U2NVVFODRU69JNKA0DMEuzYVPReN2UFLUqvT0xW7XS/cmwMGjenNLwKC4vepbn8y7gry1e4DG5DUlO0gh5fr668H366H6mWZaGsR9BszRF5BXgFGC7c66351gL4D2gM7AeONM5l1nZPYwDUNF43V69ftuzJDtbhXPAAIiOpuTnFVyS9ThvFp7JPc3+xYS8vyHNm+l43T174LTTfPezLuuGUSHBtDRfY/+2cn8FZjvnugOzPe+Ng6WiksXQUPjHPzSI07mz7m2GhVG8Lp3zNj7Mm4Vn8vfmj3Bf3GNIaYnucS5frtdu22bNNQzjAARNNJ1zC4Dd5Q6fBrzu+f11YEywnt8kqKye/IwzVFDj4uCIIyhq1Y6zlt7N+3mn8GjU37ir8B5tOFxaqi3dQkK0YfDChfDLL5ZSZBhVUNuBoNbOua2e3zOA1lWdbARARSWLaWk6z2fNGgqyCzkj93U+5U88KTdxY/EzGk33tnUrK1PBbNFC3fmlS+Evf6mb72IYDYA6i54755yIuMo+F5ErgSsBUszyqbxuvKLzJk6ENWvI35nLmKL3mMEIJnE1V7vntXmwN3E9PFzd97w83dOMj1dr1QI/hlEptR093yYibQE8r9srO9E594Jzrr9zrn9SUlKtLbBeUtl8n7S0/c875xyYMoXcLVn8qWgqMxnOy1zK1TzvO6+wUC3MkBCNqhcUwDHHaJpRamrtfjfDaGDUtmhOBy7y/H4RMK2Wn98w8c/HDAnx/T51qu+ctDS44QZYtYqcgnBG8TnzGcRkLuRSXt3/nmEeJyMnR4NHixdb30vDCICgiaaIvINOruwpIptE5DLgYWC4iKwBTvS8Nw5ERfmY5aPcU6fCihVkuXhOcl+wkIG8zbmcz1sV3zMyUpsJh4T4AkKWk2kYByRoe5rOuXMq+WhYsJ7ZaKkoH/OHH7TOvHt3rfQRYXd2KCPcZ/xEXz5gHKfzUcX3CwlR0YyJ0dZv+fmao2mCaRgHxMoo6zOV9cL84QeYP1+7p7dpA1lZ7Pw1h+HFn7PcHcpUxnIKn1Z8z/h4TUVq21bLJbOzNSBkbrlhBISJZn3FG/zxRrNjYjQdKDdXLcx27fQH2BbThWEFj/CL68h0RjOCGfvfLzxcrdIuXVQ0N2+GHTv0+N13m5VpGAFiolnf8FqXH32kLvSRR6o7HRenluGGDZoe1LkzAFsKWzIs7V+klyTzacRYhrq5UCK+mT7eLkcdO8I77+ixqVP1XkOGVJ66ZBhGhZho1if8rUtQ4Vu4EHr0gNWrVfyc09zKX39lY8eBDF37JBlFLfiiy7X8ce9iOH40zJ2rjYO9aUUREVpa6RVHE0nDOGhMNOsT/qlFCQkaoImKgu++842nSEiAHj1Y/9lyhiybxG5JYEaXqzm29H9wwgnqbrdsqdeVluo1cXEqvIZh/G5MNOsT6emavA7arWjRIhW+bdvUcgwNhUGDWNvmeIaWPUuOC2N2uwvo3z4Trv+nCuMFF+j+Z9u2vobChx+ugmwWpmH8bkw06xP+qUVt2qgIzp2rLrmnscbK2ZsZlnsuhaVhzO1+FannHLHvvmSXLnoPb1lkv37aed06FxlGjWBNiOsT5Vu9padrM43kZIiIYGnEkQzOnEpJYSnzks8idXDC/iWV3lLI006DwYNVfLOzrXORYdQQJpr1ifKt3goLtftQ27b8lDycIbs+IIQy5kecRO+2u9QFL19SWVGPTSuPNIwaw0SzvtG3r3ZNv/lmDfqsXcsP6a0Ysvo5osJLmZ8whl5tsnxBHti3pLKyHpu2n2kYNYLtadYn/CuAli+H0lK+zj+CkXveIiEkm7mdrqRL9gaIjNu3Fr28+11Rj03DMGoEE836gn+O5saNsGMH/ysbyKiS92gt25gTOoKUPbnQrZtWBJWUaJCofXvtWGSzyA2jVjD3vL7gn6O5eTNzQ09kRM4HtA/dxvze15OSlK/NgnNytAa9tFQt0jlz4IgjzLI0jFrCRLO+4Nf+bUbhIE7eNZnOYZuYlzCG9hE79JyiIg3ubNumUfHUVH2dPHn/hsSGYQQFE836QkoKZGfz2ZrujM6eTI/QX5gXP4Y2YTth/Xq1MmNiYNcu2L1bLU0RFdri4n0bEhuGETRsT7MuSUuDZ5+Fr7+G/Hw+yhzEmTvPpE/EKmZEnErL4kwoC9U9yxYttBxywwatQd+5U5sIFxRAUpIlrxtGLWGiGSwONAgtLQ3uvFNH5jZvzge7hnLujkf4Q+gSvmh/NQkhYbA7VEWxVSt1xVu00DSikhL9yc/Xz7t2teR1w6glTDSDgX8k3H8Q2ujR2hMzPR3WrVNXOyyMt9Ydy4W7HmVgyDd8mnA+cYktoDgGYmMhK0vzNVetgoED4Y9/1Ki5iDbl6NpVLVFLXjeMWsFEMxj4R8JBX3fuhAce0NLGDh3UJd+0iddKL+DSvMcZFPJfPo4+i2Z7s2D9HhXDqCi1Jp1TkVy+XC3OY4/VVKOioqrH+RqGUeOYaAYD/25FXjZt0oCNV0iTk3lhzRCuKnmG4SGz+IjTicnP005GubkqmAUFWirZqxesWAFbtmjj4AcfNJE0jDrCRLMm8e5jLlyo+ZTNm2uwBmDNGhXMjAxo04ansy/ghpLLOJlP+Q9nEkWBphOFhmpkfOtWbTbcr5+mFUVGqmBOmFCnX9EwmjommjWFdx+ztFR/cnJ0Bk9xsXZPDwtTN3vRIh6LuZvxS87gtPDPeK/sTCKlCCRUO6x7x1NkZ0P//trhyNt0w6p+DKPOMdGsKbz7mD/9pK61iEa4RVQ0y8pg504e2nE5d+afwbiIabyVdDPhOWHQLEFF1RsRHzdOE9h79PBF3y+7zFxyw6gHmGjWFN59zOxsDfp4AzgAZWU4hPsLbmdC2d2cG/EBr7e8lbDwUA3oZGdrG7jQUDjkEBXd1FRzxQ2jHmIVe3E4DQAAEBlJREFUQTWFp6KH0FCt2Ckr++0jV+a4q+wBJpTdy8XhbzF5wCTCRp+s4hgSojN92rTR1549rf+lYdRjzNKsLpUlrY8dq3uahYUqhEVFADhgPBP5F7dwpbzApPBbCTn0PBXJE06Ab77RcbzefMzu3S2FyDDqMU1bNCsTwKqOV5S07m3yO348XHihTo7cuJGyMrhJnuJpdx3X8zRPhd2KJLVVwczI0C5FIuqKm1AaRoOg6brnXgHMzNxXAKdMqfi4V0i9SeshIfuPmujbF8aMgdNOoyz1SK6JepWn3XXcGvYET7W8D2nfTgV161aYP1+Hnx199P5zfgzDqLc0XUuzoqodgKef1uDMTz/pHmV8vL73Wp7+SesZGb6kc9A+l9u2UTpjNpfv/ievFYzhjuSXeTD5FaTPcE1wz8uDDz7QHMyBAzV53X9NZm0aRr2m6YpmRVU78fFaE757t4paXJxGwX/+Wat0UlN9I3YzMnQuuQi0a6fJ65MnU3L0QC4KeYO3s45hQtxj3NPhHaTjIXov70jdkhLNyVy92hcE8p/zYxhGvaVORFNE1gM5QClQ4pzrX+uL8J8x7iU7W93ukBAVTdDXwkK1Etu2hZkzVei8OZjOwWGHwYoVFDdvwXlzLuODHcfwj6GzuKN3OiSeoqlDEyZo0npiogZ88vM1er5ypY3ZNYwGRF3uaQ5xzqXWiWBC5aNuu3TR9948y3zPmIlt29Q6HDZMr1+zRtOLBg6E1q0pzMxj3IZH+WDHEB5r9xh3lP5da8e91qNfZ3YOPVQ/c06j5jZm1zAaDE03EFTZqNtBg3RvMjpaAzXR0SqW7drpOW3bwqhRKnxhYdC6NQUlYYxd/y+mZQ/h3/F3cUu36Sq2CxaodQm+PE7QYNDAgWqpgo3ZNYwGRF3taTpghog44Hnn3At1soqKRt2uXg0ffqg140lJKpabNsFxx+17XmoqzJ5N3va9jPnyambu6cbzUTdxZYfZQEvfed6qIG8eJ6jFGRGhiewmlobRoKgrS/N459yRwCjgOhE5ofwJInKliHwvIt/v2LGjdlaVlgbTp2uAKDfXN+3RW9roT1QUewefwp8+v45Z6w7hlY73cuWfNmuXda+FesIJvyW5V2rZmmAaRoOiTixN59xmz+t2EfkQOBpYUO6cF4AXAPr37+9qZWFTp2qHok2bNDjTubO61OvXq5t9yCFqJWZns2dbPievepxF6bG88Sact0Ygs7269l4yM/dNKarIsjUMo0FR65amiMSKSHPv78BJwNLaXkeFpKerYEZFqaXonfYYFqa5mh4rMSu6LSctf5xvlsby7rtw3nlUHliy4I5hNCrqwtJsDXwoGgQJA952zn1RB+vYn5QUHUORlAR792q3otxcndWTkQHPPcfu3XDSSZC2UnPUx4zxXOt1v/3LL62dm2E0OmpdNJ1z64Ajavu5ATF2rAaBMjK0gXBhoa+p8PLl7Ji3jBNvOpxVq/S0P/2p3PXmfhtGo6fpphxVRN++cPfdPgszJERHVoSEkJEfz+CxiaxerbGi/QTTMIwmQdMto/RSUUejVq10X7O0FKKi2Bx3KEPXPMemwgQ+u+A1hry7ABbaFEjDaIo0bdGcMkXH6npzMgsKNJfSOQ38xMSQXpDM0J8eZ1tRAl/G/pnjE3pAfAVt4QzDaBI0XdGcMgWuukrd8NBQdcm3btVhZnFxkJPDr0XtGLrySTJLmjGz2VgGpGyBxGP0em/NunUmMowmRdMUzbQ0uPNOjZBHROixvXs1EX3VKujYkTXZyQxd+AC5pVHM7nU9f8heDgNP3vc+1pnIMJocTVM0p05V9zo8XN+HhuprSQmkp7NiwCUMW3gVxWFFzB31T45I7QLbTvEJrBfrTGQYTY6mKZrp6SqU0dHaWKO0VCPlhYUsLezOsHevQsLDmfdtOIcffp9e4+30Dr9VBdkscsNoejTNlKOUFI2QOwcxMb8J5hJSGVz4BWFR4cyfD4cf7neN1Y4bhkFTtTTHjoUfftAgUFEROMf3chQn5U6lWaso5syHbt0quM6S1w2jydM0Lc2+feHBB2HkSEhOZlHCKIblf0x8chQLFkVULJiGYRg0VUsTVDgnTWLBAq3uaZOiXeA6dqzrhRmGUZ9pmpamhzlztAl7hw46UdcE0zCMA9FkRfPLL9XC7NIF5s3TBu2GYRgHokmK5iefwOjROm1i7lwd2WMYhhEITU40P/zQ12djzhwtOTcMwwiUJiWa770H48bBH/4As2ZBixZ1vSLDMBoaTUY033wTzj0Xjj0WZszwjSA3DMOoDk1CNF95BS68UEeaf/GF9hU2DMM4GBq9aD73nJaHDx+uAaDY2LpekWEYDZlGLZpPPQXXXKOpRdOmaZm5YRjG76HRiubEiXDTTXD66doJLiqqrldkGEZjoFGK5oMPwm23wZlnasS8fBtMwzCMg6VRiaZzcO+98Le/wfnnw1tv+foMG4Zh1ASNpmGHc3DHHfDII3DJJfDii76G7Mb/t3fuMVJddRz/fEt51NKmIoTgk4KGFg1SXJpqHz5Qg9gHTTeKJIZaI6HhUUkxxRAMbWKCGKGNWMwWeYgKSLHpihXbIrSpTVooZZdHC6UtRgmCTS1CLSvs/vzjnIHLMrO7s+vOPVN+n2Qy5557z5zv/nbmN+fcO/d7HMf5f/GuGGmawd13h4Q5ZQosXeoJ03Gc7qHqk2ZLC0yfDosWwYwZ8OCDwYjdcRynO6jq6XlLS1iFd+nSsPLEggUg5a3KcZx3M1U7JmtuhjvuCAlzzhxPmI7jVIaqHGmeOhVui1y9Gu67D+bOzVuR4zjnC7mMNCWNlbRX0n5Js8tpe/IkTJgQEub8+Z4wHcepLBVPmpJ6AD8DvgIMB74haXhH2jY1QW0trF8PCxfCPfd0p1LHcZxzyWOkeTWw38xeM7P/AmuAW9pr9M474ZbI+npYvBhmzux2nY7jOOeQR9L8APC3zPbfY11JWlrC8hQbN0JdHUyd2q36HMdxSpLshSBJk4HJAL17j+DkSVi+HCZNylmY4zjnNXmMNA8C2cVyPxjrzsLM6sysxsxqmpp6smqVJ0zHcfJHZlbZDqULgX3AGEKy3ApMNLPdbbT5J/BXoD/wRiV0doKUtUHa+lLWBq6vK6SsDWCYmZW1lkPFp+dmdkrSNOBPQA9gWVsJM7YZACBpm5nVVEBm2aSsDdLWl7I2cH1dIWVtEPSV2yaXc5pm9hjwWB59O47jdIWqvY3ScRwnD6otadblLaANUtYGaetLWRu4vq6QsjbohL6KXwhyHMepZqptpOk4jpMrVZE0u2LwUQkkHZC0U9KOzlyN6wY9yyQdkbQrU9dP0hOSXonP701I2zxJB2P8dkgal5O2D0naLGmPpN2S7or1qcSulL5U4tdH0vOSGqK+e2P95ZKei5/ftZIqvtRhG9pWSHo9E7uR7b6YmSX9IPws6VVgCNALaACG562rlcYDQP+8dWT03ACMAnZl6hYAs2N5NvCjhLTNA2YlELdBwKhYvoTwe+LhCcWulL5U4iegbyz3BJ4DrgF+C0yI9T8H7kxI2wqgtpzXqoaRZqcMPs5nzOxp4M1W1bcAK2N5JTC+oqIiJbQlgZkdMrPtsXwMeIngi5BK7ErpSwILHI+bPePDgC8AD8f6XOLXhrayqYakWbbBRw4Y8LikF+I98yky0MwOxfI/gIF5iinCNEmNcfqey/Q3i6TBwFWEEUlysWulDxKJn6QeknYAR4AnCLPEt8zsVDwkt89va21mVojdD2PsFknq3d7rVEPSrAauM7NRBI/QqZJuyFtQW1iYo6T0s4klwFBgJHAI+EmeYiT1BdYD3zWzf2f3pRC7IvqSiZ+ZNZvZSIKnxNXAFXlpaU1rbZI+AXyfoHE00A9o16W3GpJmhww+8sTMDsbnI8AjhDdLahyWNAggPh/JWc9pzOxwfEO3AA+RY/wk9SQkpF+b2e9idTKxK6YvpfgVMLO3gM3Ap4HLoucEJPD5zWgbG095mJk1AcvpQOyqIWluBT4Wr8D1AiYA9TlrOo2kiyVdUigDXwZ2td0qF+qBgk/UJODRHLWcRSEhRW4lp/hJEvAL4CUzW5jZlUTsSulLKH4DJF0WyxcBXyKcd90M1MbDcolfCW0vZ74MRTjX2n7s8rzaVsaVr3GEK4WvAnPy1tNK2xDCFf0GYHcK+oDVhGnaScI5pG8D7wM2Aa8ATwL9EtK2CtgJNBIS1KCctF1HmHo3AjviY1xCsSulL5X4jQBejDp2AT+I9UOA54H9wDqgd0La/hxjtwv4FfEKe1sPvyPIcRynDKpheu44jpMMnjQdx3HKwJOm4zhOGXjSdBzHKQNPmo7jOGXgSdNJlujeM6tI/XhJwzvxeoMlTcxs3y5pcVd1Fulni6Rk18VxuoYnTadLZO70qCTjCe4+59COnsHAxDb2O067eNJ0SiJpbvQxfUbS6sKoL46k7o/eoXdJGiPpRQVP0WUF0wMFn9H+sVwjaUssz4vHbZH0mqQZmT7nSNon6RlgWBFNnwFuBn4c/Q+HFtGzQlJtpk3B3WY+cH1sNzPWvV/SRgWvzAVF+hsraV1m+3OSNsTyEknbsv6MRdofz5RrJa2I5QGS1kvaGh/Xtv3fcFIhl9UonfSRNBq4DfgkwUZrO/BC5pBeZlYjqQ/hTpkxZrZP0i+BO4H72+niCuDzBF/IvZKWEO7amEAwnriwSJ+Y2bOS6oENZvZw1HpaT9xeUaLP2QTfyRvjcbfHvq4CmqKOn5pZ1lXrSaBO0sVm9jbwdYI9IYS7v96U1APYJGmEmTW283cXeABYZGbPSPowYUnrKzvY1skRH2k6pbgWeNTMTljwbvx9q/1r4/Mw4HUz2xe3VxKMhtvjD2bWZGZvEAwwBgLXA4+Y2X8suPeU4zGwtv1DirLJzI6a2QlgD/CR7E4LlmYbgZvi1P+rnLl3+muSthNuz/s4JU4ZlOCLwOJoVVYPXBrdi5zE8ZGm01ne7sAxpzjzxdyn1b6mTLmZrr8Xs3pO9yvpAoLjfyk6omMNMI1gnrzNzI5JuhyYBYw2s3/F0W3rvxHOtpHL7r8AuCYma6eK8JGmU4q/EEZXfeII6MYSx+0FBkv6aNz+JvBULB8APhXLt3Wgz6eB8ZIuis5RN5U47hhhWl+KbL83E04vdKRdKZ4iLNHxHc5MzS8lJOqjkgYSvFSLcVjSlTF535qpfxyYXthQR9amcZLAk6ZTFDPbSpg2NgJ/JDjBHC1y3AngW8A6STuBFsI6MAD3Ag/ECzTNHehzO2Ga3RD73Fri0DXA9+LFp6FF9j8EfFZSA8HPsTAKbQSaFRbXmlmkXSldzcAGQmLcEOsaCNPyl4HfEL5kijE7tnmW4O5UYAZQo+AYvgeY0lE9Tr64y5FTEkl9zey4pPcQRoGTY2JznPMWP6fptEVd/BF5H2ClJ0zH8ZGm4zhOWfg5TcdxnDLwpOk4jlMGnjQdx3HKwJOm4zhOGXjSdBzHKQNPmo7jOGXwP8BUMt9gX8//AAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "aQikz3IPiyPf"
      },
      "source": [
        "# **Testing**\n",
        "The predictions of your model on testing set will be stored at `pred.csv`."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "O8cTuQjQQOon",
        "outputId": "58757e60-5685-4a9b-a6bf-da023eab09a4"
      },
      "source": [
        "def save_pred(preds, file):\n",
        "    ''' Save predictions to specified file '''\n",
        "    print('Saving results to {}'.format(file))\n",
        "    with open(file, 'w') as fp:\n",
        "        writer = csv.writer(fp)\n",
        "        writer.writerow(['id', 'tested_positive'])\n",
        "        for i, p in enumerate(preds):\n",
        "            writer.writerow([i, p])\n",
        "\n",
        "preds = test(tt_set, model, device)  # predict COVID-19 cases with your model\n",
        "save_pred(preds, 'pred.csv')         # save prediction file to pred.csv"
      ],
      "execution_count": 65,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Saving results to pred.csv\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nfrVxqJanGpE"
      },
      "source": [
        "# **Hints**\n",
        "\n",
        "## **Simple Baseline**\n",
        "* Run sample code\n",
        "\n",
        "## **Medium Baseline**\n",
        "* Feature selection: 40 states + 2 `tested_positive` (`TODO` in dataset)\n",
        "\n",
        "## **Strong Baseline**\n",
        "* Feature selection (what other features are useful?)\n",
        "* DNN architecture (layers? dimension? activation function?)\n",
        "* Training (mini-batch? optimizer? learning rate?)\n",
        "* L2 regularization\n",
        "* There are some mistakes in the sample code, can you find them?"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "9tmCwXgpot3t"
      },
      "source": [
        "# **Reference**\n",
        "This code is completely written by Heng-Jui Chang @ NTUEE.  \n",
        "Copying or reusing this code is required to specify the original author. \n",
        "\n",
        "E.g.  \n",
        "Source: Heng-Jui Chang @ NTUEE (https://github.com/ga642381/ML2021-Spring/blob/main/HW01/HW01.ipynb)\n"
      ]
    }
  ]
}