DQL_Episode3_Quadric_Layers.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyMNOMevPPyf05jzExhtbnoO",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/diro5t/deep_quadric_learning/blob/main/DQL_Episode3_Quadric_Layers.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Deep Quadric Learning - A tale in 3 episodes\n",
        "## Episode 3: Quadric Layers\n",
        "### (C) Dirk Roeckmann 2023\n",
        "\n",
        "https://scholar.google.com/citations?hl=en&user=hamuGh4AAAAJ"
      ],
      "metadata": {
        "id": "w3qE1JwTJGgk"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Introduction\n",
        "In the last episode I established that classification problems where layers of linear units are necessary can be solved with a single quadric neuron which is proof of a significant reduction of model size and parameters. In this episode I will introduce quadric layers in PyTorch exemplified in a custom layer class and apply it to the IRIS dataset for a little demonstration in multi-class classification.\n",
        "\n"
      ],
      "metadata": {
        "id": "RxbaHI7YJMPI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# get stuff needed for this episode\n",
        "import copy\n",
        "import matplotlib.pyplot as plt\n",
        "import numpy as np\n",
        "import pandas as pd\n",
        "import torch\n",
        "import torch.nn as nn\n",
        "import torch.optim as optim\n",
        "import tqdm\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.preprocessing import OneHotEncoder\n",
        "import math\n",
        "from typing import Any\n",
        "from torch import Tensor\n",
        "from torch.nn.parameter import Parameter, UninitializedParameter"
      ],
      "metadata": {
        "id": "xWqKV8jxc0W1"
      },
      "execution_count": 7,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Custom Quadric Layer\n",
        "\n",
        "The quadric layer can be modeled straight-forward as a custom layer in PyTorch:"
      ],
      "metadata": {
        "id": "pezlBJzmKtgW"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class Quadric(nn.Module):\n",
        "    __constants__ = ['in_features', 'out_features']\n",
        "    in_features: int\n",
        "    out_features: int\n",
        "    qweight: Tensor\n",
        "    lweight: Tensor\n",
        "\n",
        "    def __init__(self, in_features: int, out_features: int, bias: bool = True,\n",
        "                 device=None, dtype=None) -> None:\n",
        "        factory_kwargs = {'device': device, 'dtype': dtype}\n",
        "        super().__init__()\n",
        "        self.in_features = in_features\n",
        "        self.out_features = out_features\n",
        "        # quadratic weights\n",
        "        self.qweight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))\n",
        "        # linear weights\n",
        "        self.lweight = Parameter(torch.empty((out_features, in_features), **factory_kwargs))\n",
        "        # bias\n",
        "        if bias:\n",
        "            self.bias = Parameter(torch.empty(out_features, **factory_kwargs))\n",
        "        else:\n",
        "            self.register_parameter('bias', None)\n",
        "        self.reset_parameters()\n",
        "\n",
        "    def reset_parameters(self) -> None:\n",
        "        nn.init.kaiming_uniform_(self.qweight, a=math.sqrt(5))\n",
        "        nn.init.kaiming_uniform_(self.lweight, a=math.sqrt(5))\n",
        "        if self.bias is not None:\n",
        "            fan_in_q, _ = nn.init._calculate_fan_in_and_fan_out(self.qweight)\n",
        "            fan_in_l, _ = nn.init._calculate_fan_in_and_fan_out(self.lweight)\n",
        "            bound = 1 / math.sqrt(fan_in_l) if fan_in_l > 0 else 0\n",
        "            nn.init.uniform_(self.bias, -bound, bound)\n",
        "\n",
        "    def forward(self, input: Tensor) -> Tensor:\n",
        "        input_sqr = torch.mul(input, input)\n",
        "        qi = nn.functional.linear(input_sqr, self.qweight, None)\n",
        "        wib = nn.functional.linear(input, self.lweight, self.bias)\n",
        "        return torch.add(qi, wib)\n",
        "\n",
        "    def extra_repr(self) -> str:\n",
        "        return 'in_features={}, out_features={}, bias={}'.format(\n",
        "            self.in_features, self.out_features, self.bias is not None\n",
        "        )"
      ],
      "metadata": {
        "id": "jdPjyN_lLM3M"
      },
      "execution_count": 8,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# IRIS Multi-Class Classification"
      ],
      "metadata": {
        "id": "ylQ15gsjf5t6"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Linear Layers\n",
        "\n",
        "First let's classify the IRIS dataset with linear units:\n",
        "4 inputs, a 8-unit hidden layer and a 3-unit output layer\n",
        "\n",
        "This example code has been taken verbatim from:\n",
        "\n",
        "https://machinelearningmastery.com/building-a-multiclass-classification-model-in-pytorch/\n",
        "\n",
        "Training will be 200 epochs with a batch size of 5."
      ],
      "metadata": {
        "id": "9a4aFvdagQkN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# read data and apply one-hot encoding\n",
        "data = pd.read_csv(\"sample_data/iris.csv\", header=None)\n",
        "X = data.iloc[:, 0:4]\n",
        "y = data.iloc[:, 4:]\n",
        "ohe = OneHotEncoder(handle_unknown='ignore', sparse_output=False).fit(y)\n",
        "y = ohe.transform(y)\n",
        "\n",
        "# convert pandas DataFrame (X) and numpy array (y) into PyTorch tensors\n",
        "X = torch.tensor(X.values, dtype=torch.float32)\n",
        "y = torch.tensor(y, dtype=torch.float32)\n",
        "\n",
        "# split\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, train_size=0.7, shuffle=True)\n",
        "\n",
        "class Multiclass(nn.Module):\n",
        "    def __init__(self):\n",
        "        super().__init__()\n",
        "        self.hidden = nn.Linear(4, 8)\n",
        "        self.act = nn.ReLU()\n",
        "        self.output = nn.Linear(8, 3)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.act(self.hidden(x))\n",
        "        x = self.output(x)\n",
        "        return x\n",
        "\n",
        "# loss metric and optimizer\n",
        "model = Multiclass()\n",
        "loss_fn = nn.CrossEntropyLoss()\n",
        "optimizer = optim.Adam(model.parameters(), lr=0.001)\n",
        "\n",
        "# prepare model and training parameters\n",
        "n_epochs = 200\n",
        "batch_size = 5\n",
        "batches_per_epoch = len(X_train) // batch_size\n",
        "\n",
        "best_acc = - np.inf   # init to negative infinity\n",
        "best_weights = None\n",
        "train_loss_hist = []\n",
        "train_acc_hist = []\n",
        "test_loss_hist = []\n",
        "test_acc_hist = []\n",
        "\n",
        "# training loop\n",
        "for epoch in range(n_epochs):\n",
        "    epoch_loss = []\n",
        "    epoch_acc = []\n",
        "    # set model in training mode and run through each batch\n",
        "    model.train()\n",
        "    with tqdm.trange(batches_per_epoch, unit=\"batch\", mininterval=0) as bar:\n",
        "        bar.set_description(f\"Epoch {epoch}\")\n",
        "        for i in bar:\n",
        "            # take a batch\n",
        "            start = i * batch_size\n",
        "            X_batch = X_train[start:start+batch_size]\n",
        "            y_batch = y_train[start:start+batch_size]\n",
        "            # forward pass\n",
        "            y_pred = model(X_batch)\n",
        "            loss = loss_fn(y_pred, y_batch)\n",
        "            # backward pass\n",
        "            optimizer.zero_grad()\n",
        "            loss.backward()\n",
        "            # update weights\n",
        "            optimizer.step()\n",
        "            # compute and store metrics\n",
        "            acc = (torch.argmax(y_pred, 1) == torch.argmax(y_batch, 1)).float().mean()\n",
        "            epoch_loss.append(float(loss))\n",
        "            epoch_acc.append(float(acc))\n",
        "            bar.set_postfix(\n",
        "                loss=float(loss),\n",
        "                acc=float(acc)\n",
        "            )\n",
        "    # set model in evaluation mode and run through the test set\n",
        "    model.eval()\n",
        "    y_pred = model(X_test)\n",
        "    ce = loss_fn(y_pred, y_test)\n",
        "    acc = (torch.argmax(y_pred, 1) == torch.argmax(y_test, 1)).float().mean()\n",
        "    ce = float(ce)\n",
        "    acc = float(acc)\n",
        "    train_loss_hist.append(np.mean(epoch_loss))\n",
        "    train_acc_hist.append(np.mean(epoch_acc))\n",
        "    test_loss_hist.append(ce)\n",
        "    test_acc_hist.append(acc)\n",
        "    if acc > best_acc:\n",
        "        best_acc = acc\n",
        "        best_weights = copy.deepcopy(model.state_dict())\n",
        "    print(f\"Epoch {epoch} validation: Cross-entropy={ce:.2f}, Accuracy={acc*100:.1f}%\")\n",
        "\n",
        "# Restore best model\n",
        "model.load_state_dict(best_weights)\n",
        "\n",
        "# Plot the loss and accuracy\n",
        "plt.plot(train_loss_hist, label=\"train\")\n",
        "plt.plot(test_loss_hist, label=\"test\")\n",
        "plt.xlabel(\"epochs\")\n",
        "plt.ylabel(\"cross entropy\")\n",
        "plt.legend()\n",
        "plt.show()\n",
        "\n",
        "plt.plot(train_acc_hist, label=\"train\")\n",
        "plt.plot(test_acc_hist, label=\"test\")\n",
        "plt.xlabel(\"epochs\")\n",
        "plt.ylabel(\"accuracy\")\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "VC66t32wcLyc",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "outputId": "998672e2-f713-4cda-a26c-26e72ed17d49"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 0: 100%|██████████| 21/21 [00:00<00:00, 186.00batch/s, acc=0.2, loss=1.74]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0 validation: Cross-entropy=1.53, Accuracy=31.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 1: 100%|██████████| 21/21 [00:00<00:00, 244.51batch/s, acc=0.2, loss=1.56]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1 validation: Cross-entropy=1.38, Accuracy=31.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 2: 100%|██████████| 21/21 [00:00<00:00, 200.31batch/s, acc=0.2, loss=1.43]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2 validation: Cross-entropy=1.28, Accuracy=31.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 3: 100%|██████████| 21/21 [00:00<00:00, 204.29batch/s, acc=0.2, loss=1.33]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3 validation: Cross-entropy=1.22, Accuracy=31.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 4: 100%|██████████| 21/21 [00:00<00:00, 184.25batch/s, acc=0.2, loss=1.26]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4 validation: Cross-entropy=1.17, Accuracy=31.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 5: 100%|██████████| 21/21 [00:00<00:00, 200.57batch/s, acc=0.2, loss=1.21]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5 validation: Cross-entropy=1.14, Accuracy=31.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 6: 100%|██████████| 21/21 [00:00<00:00, 197.12batch/s, acc=0.2, loss=1.17]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6 validation: Cross-entropy=1.11, Accuracy=31.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 7: 100%|██████████| 21/21 [00:00<00:00, 152.38batch/s, acc=0.2, loss=1.14]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7 validation: Cross-entropy=1.09, Accuracy=31.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 8: 100%|██████████| 21/21 [00:00<00:00, 43.53batch/s, acc=0.2, loss=1.12]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8 validation: Cross-entropy=1.07, Accuracy=31.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 9: 100%|██████████| 21/21 [00:00<00:00, 135.35batch/s, acc=0.2, loss=1.1]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9 validation: Cross-entropy=1.06, Accuracy=31.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 10: 100%|██████████| 21/21 [00:00<00:00, 124.74batch/s, acc=0.2, loss=1.08]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 10 validation: Cross-entropy=1.05, Accuracy=31.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 11: 100%|██████████| 21/21 [00:00<00:00, 147.23batch/s, acc=0.4, loss=1.06]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 11 validation: Cross-entropy=1.04, Accuracy=28.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 12: 100%|██████████| 21/21 [00:00<00:00, 139.12batch/s, acc=0.2, loss=1.05]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 12 validation: Cross-entropy=1.03, Accuracy=28.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 13: 100%|██████████| 21/21 [00:00<00:00, 156.49batch/s, acc=0.2, loss=1.04]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 13 validation: Cross-entropy=1.02, Accuracy=37.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 14: 100%|██████████| 21/21 [00:00<00:00, 143.96batch/s, acc=0.2, loss=1.03]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 14 validation: Cross-entropy=1.01, Accuracy=35.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 15: 100%|██████████| 21/21 [00:00<00:00, 146.40batch/s, acc=0.4, loss=1.02]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 15 validation: Cross-entropy=1.00, Accuracy=33.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 16: 100%|██████████| 21/21 [00:00<00:00, 143.41batch/s, acc=0.4, loss=1]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 16 validation: Cross-entropy=0.99, Accuracy=33.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 17: 100%|██████████| 21/21 [00:00<00:00, 148.73batch/s, acc=0.4, loss=0.993]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 17 validation: Cross-entropy=0.99, Accuracy=35.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 18: 100%|██████████| 21/21 [00:00<00:00, 116.00batch/s, acc=0.4, loss=0.982]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 18 validation: Cross-entropy=0.98, Accuracy=35.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 19: 100%|██████████| 21/21 [00:00<00:00, 198.23batch/s, acc=0.4, loss=0.971]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 19 validation: Cross-entropy=0.96, Accuracy=42.2%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 20: 100%|██████████| 21/21 [00:00<00:00, 192.44batch/s, acc=0.4, loss=0.959]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 20 validation: Cross-entropy=0.95, Accuracy=53.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 21: 100%|██████████| 21/21 [00:00<00:00, 204.32batch/s, acc=0.4, loss=0.947]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 21 validation: Cross-entropy=0.94, Accuracy=57.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 22: 100%|██████████| 21/21 [00:00<00:00, 214.14batch/s, acc=0.8, loss=0.934]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 22 validation: Cross-entropy=0.93, Accuracy=64.4%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 23: 100%|██████████| 21/21 [00:00<00:00, 205.03batch/s, acc=0.8, loss=0.921]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 23 validation: Cross-entropy=0.92, Accuracy=68.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 24: 100%|██████████| 21/21 [00:00<00:00, 202.74batch/s, acc=0.8, loss=0.907]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 24 validation: Cross-entropy=0.91, Accuracy=68.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 25: 100%|██████████| 21/21 [00:00<00:00, 224.32batch/s, acc=0.8, loss=0.893]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 25 validation: Cross-entropy=0.89, Accuracy=68.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 26: 100%|██████████| 21/21 [00:00<00:00, 218.68batch/s, acc=0.8, loss=0.879]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 26 validation: Cross-entropy=0.88, Accuracy=68.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 27: 100%|██████████| 21/21 [00:00<00:00, 226.08batch/s, acc=0.8, loss=0.864]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 27 validation: Cross-entropy=0.86, Accuracy=68.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 28: 100%|██████████| 21/21 [00:00<00:00, 215.17batch/s, acc=0.8, loss=0.848]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 28 validation: Cross-entropy=0.85, Accuracy=68.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 29: 100%|██████████| 21/21 [00:00<00:00, 80.61batch/s, acc=0.8, loss=0.833] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 29 validation: Cross-entropy=0.84, Accuracy=68.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 30: 100%|██████████| 21/21 [00:00<00:00, 155.74batch/s, acc=0.8, loss=0.817]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 30 validation: Cross-entropy=0.82, Accuracy=71.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 31: 100%|██████████| 21/21 [00:00<00:00, 229.56batch/s, acc=0.8, loss=0.801]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 31 validation: Cross-entropy=0.81, Accuracy=71.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 32: 100%|██████████| 21/21 [00:00<00:00, 228.19batch/s, acc=0.8, loss=0.786]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 32 validation: Cross-entropy=0.79, Accuracy=71.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 33: 100%|██████████| 21/21 [00:00<00:00, 218.63batch/s, acc=0.8, loss=0.771]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 33 validation: Cross-entropy=0.78, Accuracy=71.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 34: 100%|██████████| 21/21 [00:00<00:00, 80.79batch/s, acc=0.8, loss=0.756]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 34 validation: Cross-entropy=0.77, Accuracy=71.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 35: 100%|██████████| 21/21 [00:00<00:00, 197.03batch/s, acc=0.8, loss=0.741]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 35 validation: Cross-entropy=0.75, Accuracy=71.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 36: 100%|██████████| 21/21 [00:00<00:00, 201.40batch/s, acc=0.8, loss=0.728]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 36 validation: Cross-entropy=0.74, Accuracy=71.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 37: 100%|██████████| 21/21 [00:00<00:00, 213.52batch/s, acc=0.8, loss=0.715]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 37 validation: Cross-entropy=0.73, Accuracy=71.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 38: 100%|██████████| 21/21 [00:00<00:00, 201.40batch/s, acc=0.8, loss=0.702]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 38 validation: Cross-entropy=0.72, Accuracy=71.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 39: 100%|██████████| 21/21 [00:00<00:00, 190.74batch/s, acc=0.8, loss=0.69]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 39 validation: Cross-entropy=0.71, Accuracy=73.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 40: 100%|██████████| 21/21 [00:00<00:00, 65.91batch/s, acc=0.8, loss=0.678] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 40 validation: Cross-entropy=0.69, Accuracy=73.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 41: 100%|██████████| 21/21 [00:00<00:00, 224.36batch/s, acc=0.8, loss=0.666]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 41 validation: Cross-entropy=0.68, Accuracy=73.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 42: 100%|██████████| 21/21 [00:00<00:00, 217.58batch/s, acc=0.8, loss=0.654]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 42 validation: Cross-entropy=0.67, Accuracy=73.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 43: 100%|██████████| 21/21 [00:00<00:00, 213.39batch/s, acc=0.8, loss=0.643]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 43 validation: Cross-entropy=0.66, Accuracy=73.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 44: 100%|██████████| 21/21 [00:00<00:00, 199.38batch/s, acc=0.8, loss=0.632]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 44 validation: Cross-entropy=0.65, Accuracy=73.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 45: 100%|██████████| 21/21 [00:00<00:00, 78.11batch/s, acc=0.8, loss=0.621] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 45 validation: Cross-entropy=0.64, Accuracy=73.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 46: 100%|██████████| 21/21 [00:00<00:00, 238.98batch/s, acc=0.8, loss=0.61]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 46 validation: Cross-entropy=0.63, Accuracy=73.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 47: 100%|██████████| 21/21 [00:00<00:00, 244.04batch/s, acc=0.8, loss=0.6]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 47 validation: Cross-entropy=0.62, Accuracy=73.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 48: 100%|██████████| 21/21 [00:00<00:00, 237.48batch/s, acc=0.8, loss=0.59]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 48 validation: Cross-entropy=0.61, Accuracy=73.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 49: 100%|██████████| 21/21 [00:00<00:00, 230.04batch/s, acc=0.8, loss=0.58]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 49 validation: Cross-entropy=0.60, Accuracy=75.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 50: 100%|██████████| 21/21 [00:00<00:00, 80.11batch/s, acc=0.8, loss=0.571] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 50 validation: Cross-entropy=0.59, Accuracy=75.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 51: 100%|██████████| 21/21 [00:00<00:00, 215.92batch/s, acc=0.8, loss=0.562]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 51 validation: Cross-entropy=0.59, Accuracy=75.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 52: 100%|██████████| 21/21 [00:00<00:00, 228.28batch/s, acc=0.8, loss=0.554]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 52 validation: Cross-entropy=0.58, Accuracy=75.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 53: 100%|██████████| 21/21 [00:00<00:00, 223.32batch/s, acc=0.8, loss=0.546]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 53 validation: Cross-entropy=0.57, Accuracy=75.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 54: 100%|██████████| 21/21 [00:00<00:00, 230.95batch/s, acc=0.8, loss=0.538]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 54 validation: Cross-entropy=0.56, Accuracy=77.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 55: 100%|██████████| 21/21 [00:00<00:00, 82.80batch/s, acc=0.8, loss=0.53]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 55 validation: Cross-entropy=0.55, Accuracy=82.2%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 56: 100%|██████████| 21/21 [00:00<00:00, 237.84batch/s, acc=0.8, loss=0.523]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 56 validation: Cross-entropy=0.55, Accuracy=82.2%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 57: 100%|██████████| 21/21 [00:00<00:00, 260.34batch/s, acc=0.8, loss=0.515]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 57 validation: Cross-entropy=0.54, Accuracy=82.2%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 58: 100%|██████████| 21/21 [00:00<00:00, 214.83batch/s, acc=0.8, loss=0.509]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 58 validation: Cross-entropy=0.53, Accuracy=84.4%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 59: 100%|██████████| 21/21 [00:00<00:00, 186.19batch/s, acc=0.8, loss=0.502]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 59 validation: Cross-entropy=0.53, Accuracy=84.4%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 60: 100%|██████████| 21/21 [00:00<00:00, 230.27batch/s, acc=0.8, loss=0.496]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 60 validation: Cross-entropy=0.52, Accuracy=86.7%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 61: 100%|██████████| 21/21 [00:00<00:00, 83.50batch/s, acc=0.8, loss=0.49] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 61 validation: Cross-entropy=0.52, Accuracy=86.7%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 62: 100%|██████████| 21/21 [00:00<00:00, 252.93batch/s, acc=0.8, loss=0.484]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 62 validation: Cross-entropy=0.51, Accuracy=86.7%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 63: 100%|██████████| 21/21 [00:00<00:00, 225.16batch/s, acc=0.8, loss=0.478]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 63 validation: Cross-entropy=0.50, Accuracy=86.7%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 64: 100%|██████████| 21/21 [00:00<00:00, 235.22batch/s, acc=0.8, loss=0.473]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 64 validation: Cross-entropy=0.50, Accuracy=86.7%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 65: 100%|██████████| 21/21 [00:00<00:00, 240.81batch/s, acc=0.8, loss=0.468]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 65 validation: Cross-entropy=0.49, Accuracy=86.7%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 66: 100%|██████████| 21/21 [00:00<00:00, 80.65batch/s, acc=0.8, loss=0.463] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 66 validation: Cross-entropy=0.49, Accuracy=86.7%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 67: 100%|██████████| 21/21 [00:00<00:00, 215.73batch/s, acc=0.8, loss=0.458]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 67 validation: Cross-entropy=0.48, Accuracy=88.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 68: 100%|██████████| 21/21 [00:00<00:00, 187.14batch/s, acc=0.8, loss=0.453]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 68 validation: Cross-entropy=0.48, Accuracy=88.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 69: 100%|██████████| 21/21 [00:00<00:00, 217.18batch/s, acc=0.8, loss=0.449]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 69 validation: Cross-entropy=0.48, Accuracy=91.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 70: 100%|██████████| 21/21 [00:00<00:00, 232.54batch/s, acc=0.8, loss=0.444]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 70 validation: Cross-entropy=0.47, Accuracy=91.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 71: 100%|██████████| 21/21 [00:00<00:00, 82.07batch/s, acc=0.8, loss=0.44]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 71 validation: Cross-entropy=0.47, Accuracy=91.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 72: 100%|██████████| 21/21 [00:00<00:00, 210.27batch/s, acc=0.8, loss=0.436]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 72 validation: Cross-entropy=0.46, Accuracy=91.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 73:  38%|███▊      | 8/21 [00:00<00:00, 193.12batch/s, acc=0.8, loss=0.566]"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 80 validation: Cross-entropy=0.43, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 81: 100%|██████████| 21/21 [00:00<00:00, 203.56batch/s, acc=1, loss=0.406]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 81 validation: Cross-entropy=0.43, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 82: 100%|██████████| 21/21 [00:00<00:00, 106.73batch/s, acc=1, loss=0.403]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 82 validation: Cross-entropy=0.43, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 83: 100%|██████████| 21/21 [00:00<00:00, 144.06batch/s, acc=1, loss=0.4]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 83 validation: Cross-entropy=0.43, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 84: 100%|██████████| 21/21 [00:00<00:00, 206.00batch/s, acc=1, loss=0.397]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 84 validation: Cross-entropy=0.42, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 85: 100%|██████████| 21/21 [00:00<00:00, 181.40batch/s, acc=1, loss=0.394]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 85 validation: Cross-entropy=0.42, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 86: 100%|██████████| 21/21 [00:00<00:00, 147.82batch/s, acc=1, loss=0.392]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 86 validation: Cross-entropy=0.42, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 87: 100%|██████████| 21/21 [00:00<00:00, 145.87batch/s, acc=1, loss=0.389]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 87 validation: Cross-entropy=0.41, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 88: 100%|██████████| 21/21 [00:00<00:00, 177.65batch/s, acc=1, loss=0.386]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 88 validation: Cross-entropy=0.41, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 89: 100%|██████████| 21/21 [00:00<00:00, 200.00batch/s, acc=1, loss=0.384]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 89 validation: Cross-entropy=0.41, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 90: 100%|██████████| 21/21 [00:00<00:00, 157.85batch/s, acc=1, loss=0.381]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 90 validation: Cross-entropy=0.41, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 99: 100%|██████████| 21/21 [00:00<00:00, 164.24batch/s, acc=1, loss=0.36]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 99 validation: Cross-entropy=0.38, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 100: 100%|██████████| 21/21 [00:00<00:00, 207.97batch/s, acc=1, loss=0.358]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 100 validation: Cross-entropy=0.38, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 101: 100%|██████████| 21/21 [00:00<00:00, 203.79batch/s, acc=1, loss=0.356]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 101 validation: Cross-entropy=0.38, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 102: 100%|██████████| 21/21 [00:00<00:00, 195.95batch/s, acc=1, loss=0.354]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 102 validation: Cross-entropy=0.38, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 103: 100%|██████████| 21/21 [00:00<00:00, 210.40batch/s, acc=1, loss=0.352]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 103 validation: Cross-entropy=0.37, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 104: 100%|██████████| 21/21 [00:00<00:00, 183.99batch/s, acc=1, loss=0.35]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 104 validation: Cross-entropy=0.37, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 105: 100%|██████████| 21/21 [00:00<00:00, 195.99batch/s, acc=1, loss=0.348]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 105 validation: Cross-entropy=0.37, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 106: 100%|██████████| 21/21 [00:00<00:00, 170.12batch/s, acc=1, loss=0.346]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 106 validation: Cross-entropy=0.37, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 107: 100%|██████████| 21/21 [00:00<00:00, 203.14batch/s, acc=1, loss=0.344]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 107 validation: Cross-entropy=0.37, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 108: 100%|██████████| 21/21 [00:00<00:00, 82.50batch/s, acc=1, loss=0.342] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 108 validation: Cross-entropy=0.36, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 125: 100%|██████████| 21/21 [00:00<00:00, 135.92batch/s, acc=1, loss=0.312]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 125 validation: Cross-entropy=0.33, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 126: 100%|██████████| 21/21 [00:00<00:00, 176.26batch/s, acc=1, loss=0.31]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 126 validation: Cross-entropy=0.33, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 127: 100%|██████████| 21/21 [00:00<00:00, 164.93batch/s, acc=1, loss=0.308]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 127 validation: Cross-entropy=0.33, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 128: 100%|██████████| 21/21 [00:00<00:00, 216.49batch/s, acc=1, loss=0.307]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 128 validation: Cross-entropy=0.32, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 129: 100%|██████████| 21/21 [00:00<00:00, 150.61batch/s, acc=1, loss=0.305]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 129 validation: Cross-entropy=0.32, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 130: 100%|██████████| 21/21 [00:00<00:00, 74.81batch/s, acc=1, loss=0.303] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 130 validation: Cross-entropy=0.32, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 131: 100%|██████████| 21/21 [00:00<00:00, 189.72batch/s, acc=1, loss=0.301]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 131 validation: Cross-entropy=0.32, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 132: 100%|██████████| 21/21 [00:00<00:00, 186.42batch/s, acc=1, loss=0.3]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 132 validation: Cross-entropy=0.32, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 133: 100%|██████████| 21/21 [00:00<00:00, 138.09batch/s, acc=1, loss=0.298]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 133 validation: Cross-entropy=0.32, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 134: 100%|██████████| 21/21 [00:00<00:00, 168.72batch/s, acc=1, loss=0.296]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 134 validation: Cross-entropy=0.31, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 135: 100%|██████████| 21/21 [00:00<00:00, 178.35batch/s, acc=1, loss=0.295]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 135 validation: Cross-entropy=0.31, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 142: 100%|██████████| 21/21 [00:00<00:00, 169.87batch/s, acc=1, loss=0.283]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 142 validation: Cross-entropy=0.30, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 143: 100%|██████████| 21/21 [00:00<00:00, 176.36batch/s, acc=1, loss=0.282]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 143 validation: Cross-entropy=0.30, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 144: 100%|██████████| 21/21 [00:00<00:00, 230.46batch/s, acc=1, loss=0.28]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 144 validation: Cross-entropy=0.30, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 145: 100%|██████████| 21/21 [00:00<00:00, 207.96batch/s, acc=1, loss=0.278]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 145 validation: Cross-entropy=0.30, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 146: 100%|██████████| 21/21 [00:00<00:00, 206.47batch/s, acc=1, loss=0.277]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 146 validation: Cross-entropy=0.29, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 147: 100%|██████████| 21/21 [00:00<00:00, 188.94batch/s, acc=1, loss=0.275]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 147 validation: Cross-entropy=0.29, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 148: 100%|██████████| 21/21 [00:00<00:00, 193.82batch/s, acc=1, loss=0.274]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 148 validation: Cross-entropy=0.29, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 149: 100%|██████████| 21/21 [00:00<00:00, 184.33batch/s, acc=1, loss=0.272]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 149 validation: Cross-entropy=0.29, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 150: 100%|██████████| 21/21 [00:00<00:00, 117.09batch/s, acc=1, loss=0.27]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 150 validation: Cross-entropy=0.29, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 151: 100%|██████████| 21/21 [00:00<00:00, 198.34batch/s, acc=1, loss=0.269]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 151 validation: Cross-entropy=0.29, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 166: 100%|██████████| 21/21 [00:00<00:00, 59.22batch/s, acc=1, loss=0.246] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 166 validation: Cross-entropy=0.26, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 167: 100%|██████████| 21/21 [00:00<00:00, 68.51batch/s, acc=1, loss=0.244]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 167 validation: Cross-entropy=0.26, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 168: 100%|██████████| 21/21 [00:00<00:00, 137.07batch/s, acc=1, loss=0.243]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 168 validation: Cross-entropy=0.26, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 169: 100%|██████████| 21/21 [00:00<00:00, 141.41batch/s, acc=1, loss=0.242]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 169 validation: Cross-entropy=0.26, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 170: 100%|██████████| 21/21 [00:00<00:00, 119.20batch/s, acc=1, loss=0.24]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 170 validation: Cross-entropy=0.26, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 171: 100%|██████████| 21/21 [00:00<00:00, 143.52batch/s, acc=1, loss=0.239]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 171 validation: Cross-entropy=0.26, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 172: 100%|██████████| 21/21 [00:00<00:00, 125.52batch/s, acc=1, loss=0.237]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 172 validation: Cross-entropy=0.26, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 173: 100%|██████████| 21/21 [00:00<00:00, 140.64batch/s, acc=1, loss=0.236]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 173 validation: Cross-entropy=0.25, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 174: 100%|██████████| 21/21 [00:00<00:00, 159.21batch/s, acc=1, loss=0.234]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 174 validation: Cross-entropy=0.25, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 175: 100%|██████████| 21/21 [00:00<00:00, 177.68batch/s, acc=1, loss=0.233]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 175 validation: Cross-entropy=0.25, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 176: 100%|██████████| 21/21 [00:00<00:00, 127.75batch/s, acc=1, loss=0.231]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 176 validation: Cross-entropy=0.25, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 182: 100%|██████████| 21/21 [00:00<00:00, 66.44batch/s, acc=1, loss=0.223]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 182 validation: Cross-entropy=0.24, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 183: 100%|██████████| 21/21 [00:00<00:00, 194.79batch/s, acc=1, loss=0.221]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 183 validation: Cross-entropy=0.24, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 184: 100%|██████████| 21/21 [00:00<00:00, 73.62batch/s, acc=1, loss=0.22] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 184 validation: Cross-entropy=0.24, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 185: 100%|██████████| 21/21 [00:00<00:00, 158.81batch/s, acc=1, loss=0.219]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 185 validation: Cross-entropy=0.24, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 186: 100%|██████████| 21/21 [00:00<00:00, 200.16batch/s, acc=1, loss=0.217]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 186 validation: Cross-entropy=0.24, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 187: 100%|██████████| 21/21 [00:00<00:00, 89.83batch/s, acc=1, loss=0.216]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 187 validation: Cross-entropy=0.24, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 188: 100%|██████████| 21/21 [00:00<00:00, 147.96batch/s, acc=1, loss=0.215]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 188 validation: Cross-entropy=0.24, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 189: 100%|██████████| 21/21 [00:00<00:00, 146.35batch/s, acc=1, loss=0.213]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 189 validation: Cross-entropy=0.23, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 190: 100%|██████████| 21/21 [00:00<00:00, 143.02batch/s, acc=1, loss=0.212]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 190 validation: Cross-entropy=0.23, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 191: 100%|██████████| 21/21 [00:00<00:00, 176.85batch/s, acc=1, loss=0.211]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 191 validation: Cross-entropy=0.23, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 192: 100%|██████████| 21/21 [00:00<00:00, 201.58batch/s, acc=1, loss=0.209]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 192 validation: Cross-entropy=0.23, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 193:  29%|██▊       | 6/21 [00:00<00:00, 199.94batch/s, acc=1, loss=0.287]"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAGwCAYAAABVdURTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABSc0lEQVR4nO3deVhU9f4H8PfMwAwgm8iOCLhkkogKwSVbtEg0f1a2mXlzKe1WWiYt5i2tvPdK5dW0sux2JevWTVtsuVmWYloquaBYbrihILKIygz7DDPf3x/DjIzsw8BhZt6v55kHOHPOmc9x1Hnz3Y5MCCFARERE5CDkUhdAREREZEsMN0RERORQGG6IiIjIoTDcEBERkUNhuCEiIiKHwnBDREREDoXhhoiIiByKi9QFdDWDwYBz587By8sLMplM6nKIiIioDYQQKC8vR2hoKOTylttmnC7cnDt3DuHh4VKXQURERFbIz89H7969W9zH6cKNl5cXAOMfjre3t8TVEBERUVtoNBqEh4ebP8db4nThxtQV5e3tzXBDRERkZ9oypIQDiomIiMihMNwQERGRQ2G4ISIiIofidGNu2kqv10On00ldhl1ydXWFQqGQugwiInJSDDdXEEKgqKgIZWVlUpdi13x9fREcHMy1hIiIqMsx3FzBFGwCAwPh4eHBD+d2EkKgqqoKJSUlAICQkBCJKyIiImcjabj55ZdfsGTJEmRlZaGwsBBfffUV7rzzzhaP2bp1K1JTU3Ho0CGEh4fjxRdfxLRp02xSj16vNwebXr162eSczsjd3R0AUFJSgsDAQHZRERFRl5J0QHFlZSViY2OxcuXKNu2fm5uLcePGYdSoUcjOzsZTTz2FGTNm4Mcff7RJPaYxNh4eHjY5nzMz/Rly3BIREXU1SVtuxo4di7Fjx7Z5/1WrViEqKgpLly4FAAwaNAjbt2/HG2+8gZSUlCaPqa2tRW1trflnjUbT6uuwK6rj+GdIRERSsaup4JmZmUhOTrbYlpKSgszMzGaPSUtLg4+Pj/nB+0oRERE5NrsKN0VFRQgKCrLYFhQUBI1Gg+rq6iaPmT9/PtRqtfmRn5/fFaUSERGRROwq3FhDpVKZ7yPF+0m1TWRkJJYvXy51GURERFaxq6ngwcHBKC4utthWXFwMb29v8wwdZzVy5EgMHTrUJqFkz5496NGjR8eLIiKyEYNBoFBTAyGE1KVQGyhd5Aj0cpPs9e0q3CQlJeH777+32LZp0yYkJSVJVJH9EEJAr9fDxaX1tzwgIKALKiIiarvHP9mHjYeKpC6D2mh4H1+sf3yEZK8vabdURUUFsrOzkZ2dDcA41Ts7Oxt5eXkAjONlpkyZYt7/0UcfxalTp/Dcc8/h6NGjeOedd/DZZ59h7ty5nVajEAJV2jpJHm39DWXatGnYtm0bVqxYAZlMBplMhjVr1kAmk+GHH35AXFwcVCoVtm/fjpMnT+KOO+5AUFAQPD09ce2112Lz5s0W57uyW0omk+Hf//43JkyYAA8PDwwYMADffvutLf+YiYialXXmkjnYqFzkfNjBw1Uh7agXSVtu9u7di1GjRpl/Tk1NBQBMnToVa9asQWFhoTnoAEBUVBQ2bNiAuXPnYsWKFejduzf+/e9/NzsN3BaqdXpEL7TNOjrtdXhRCjyUrb9FK1aswLFjxzB48GAsWrQIAHDo0CEAwPPPP49//vOf6Nu3L3r27In8/Hzcdttt+Mc//gGVSoWPPvoI48ePR05ODvr06dPsa7zyyit4/fXXsWTJErz11luYPHkyzpw5Az8/P9tcLBFRM1ZkHAcATIwPx2v3DJG4GrIHkoabkSNHttg6sWbNmiaP2b9/fydWZX98fHygVCrh4eGB4OBgAMDRo0cBAIsWLcKtt95q3tfPzw+xsbHmn//2t7/hq6++wrfffovZs2c3+xrTpk3DpEmTAACLFy/Gm2++id27d2PMmDGdcUlERACMrTa/HDsPhVyGWaP6S10O2Qm7GnMjBXdXBQ4v6ryWodZeu6Pi4+Mtfq6oqMDLL7+MDRs2oLCwEHV1daiurrZoIWvKkCGXf1vq0aMHvL29zfePIqLOU1Jeg9W/5qJap5e6FEnszr0IALh7eBj69OLq8dQ2DDetkMlkbeoa6q6unPX0zDPPYNOmTfjnP/+J/v37w93dHffccw+0Wm2L53F1dbX4WSaTwWAw2LxeIrL0yv8OY8PvhVKXISmFXIbZowZIXQbZEfv91CYLSqUSen3rv9nt2LED06ZNw4QJEwAYW3JOnz7dydURkTWOFZfj+z+MweYvN/WFysU5b0IbH9GTrTbULgw3DiIyMhK7du3C6dOn4enp2WyryoABA7B+/XqMHz8eMpkMCxYsYAsMUTf1ZsZxCAGMHRyM+WMHSV0Okd1guHEQzzzzDKZOnYro6GhUV1fjgw8+aHK/ZcuW4aGHHsJ1110Hf39/zJs3r003EyWSgt4goK1zzvB9qrQCG+pbbZ68hV0yRO0hE0623KNGo4GPjw/UanWjWzHU1NQgNzcXUVFRcHOTbmVFR8A/S+qogrJq3PH2dpRWtDwezNGNHRyMd/8cJ3UZRJJr6fP7Sg5/bykisk9vbznh9MHGS+WC1FuvkroMIrvDbiki6nbOXqrC53vzAQCfzEjE0HBfaQuSiLIbrPRKZI8Yboio21n580nUGQRG9O+FEf39pS6HiOwMfyUgom7l7KUqfJFlbLWZcwu7ZIio/RhuiKhbeWfrSej0xlabhCjeu4yI2o/hhoi6jYZjbdhqQ0TW4pgbIuo61ZeAA2sBXVWTTx86VIyZKEOfQA8knD0KnO3i+ojINrxCgaGTJHt5hhsi6jq/LgV2vtXs0ykAUlwBaABkdFVRRGRzvRMYbojISZzJNH7tOwrwCYNBAL8eL0VpeS0MEDAYgEBvFUYNDJC2TiLqmJ5Rkr48w42DGDlyJIYOHYrly5fb5HzTpk1DWVkZvv76a5ucjwh1WqDod+P3/7cM8OuLb/cX4Knfss27uMhlWDcxCYjoKU2NROQQGG6IqGsU/wHotYC7H9AzCnqDwJtbjgMAZo/qj4nXhsPbzRU+Hq4SF0pE9o6zpRzAtGnTsG3bNqxYsQIymQwymQynT5/GwYMHMXbsWHh6eiIoKAgPPvggSktLzcd98cUXiImJgbu7O3r16oXk5GRUVlbi5ZdfxocffohvvvnGfL6tW7dKd4HkGM5mGb+GxQEyGb77/RxOna+Er4cr/nJTX4T7eTDYEJFNsOWmNUI0O7Oj07l6ADJZq7utWLECx44dw+DBg7Fo0SLjoa6uSEhIwIwZM/DGG2+guroa8+bNw3333YctW7agsLAQkyZNwuuvv44JEyagvLwcv/76K4QQeOaZZ3DkyBFoNBrz3cX9/LjeCLVMXaXD9wcLodM3fRfvEb9vRT8A2aI/fs88jfTtuQCAmTf0hZcbQw0R2Q7DTWt0VcDiUGle+6/nAGWPVnfz8fGBUqmEh4cHgoODAQB///vfMWzYMCxevNi8X3p6OsLDw3Hs2DFUVFSgrq4Od911FyIiIgAAMTEx5n3d3d1RW1trPh9Ra+Z9+Ts2Hipq9vkM5T5ADrxxxBvbDh0CAPh6uGJKUkRXlUhEToLhxkEdOHAAP//8Mzw9PRs9d/LkSYwePRq33HILYmJikJKSgtGjR+Oee+5Bz54cyEntd+icGhsPFUEmA1KigyG/osPbXV+OfqcKAQCBV/8Jtyl8IJPJcE9cb7baEJHNMdy0xtXD2IIi1WtbqaKiAuPHj8drr73W6LmQkBAoFAps2rQJO3fuxE8//YS33noLL7zwAnbt2oWoKGmn8JH9eTPDODB4/JBQvDlpWOMdTmwGTgHw64slU27u2uKIyOkw3LRGJmtT15DUlEol9Hq9+efhw4fjyy+/RGRkJFxcmn6bZTIZRowYgREjRmDhwoWIiIjAV199hdTU1EbnI2rO4XMa/HioGDIZ8OQt/ZveyTyYOL7rCiMip8Vw4yAiIyOxa9cunD59Gp6enpg1axbef/99TJo0Cc899xz8/Pxw4sQJrF27Fv/+97+xd+9eZGRkYPTo0QgMDMSuXbtw/vx5DBo0yHy+H3/8ETk5OejVqxd8fHzg6sruA7vz27vGhxCd9hJBlbXYrtLD3VWBXp+omt6p6oLxa2+GGyLqfAw3DuKZZ57B1KlTER0djerqauTm5mLHjh2YN28eRo8ejdraWkRERGDMmDGQy+Xw9vbGL7/8guXLl0Oj0SAiIgJLly7F2LFjAQAzZ87E1q1bER8fj4qKCvz8888YOXKktBdJ7SME8OsyoLKkU1+mFwDIANQBULewo0wB9GOXFBF1PpkQnfgrXTek0Wjg4+MDtVoNb29vi+dqamqQm5uLqKgouLm5SVShY+CfZTdQlgcsjwHkLsD0HwC5wuYvkfb9UWSeuoDrB/jjuZSBLe/sGQz4hNm8BiJyDi19fl+JLTdEjursXuPXoMFAeILNT3/4nAbvnSyCTOaLf467EQjysvlrEBFZg+GGyFEV1A/i7cA4l7OXqpCdX9bkc2t35wMAxsWE4CoGGyLqRhhuiBxVQYPbHVjBYBC4b1Umzqlrmt3HOENqgFXnJyLqLAw3RI5IrwPOZRu/t3L6dU5xOc6pa6B0kWN4H98m90keFMRWGyLqdhhumuBkY6w7Bf8MJVZyGKirBlQ+QK9m1p5pxW+njNO3/9S3Fz56yPZjdoiIOgvvCt6AaR2XqiqJbpTpQEx/hlwbRyLmLqnhaHQvhDa6HG5401Qisi9suWlAoVDA19cXJSXGdUE8PDwga8NduekyIQSqqqpQUlICX19fKBS2n35MbXC24+NtduVeBGBsuSEisicMN1cw3QXbFHDIOr6+vryjeGerugic2wc01QOYt9P41cqZUjnF5Sir0sFDqUBMmI/1NRIRSYDh5goymQwhISEIDAyETqeTuhy75OrqyhabrvDJPZe7n5pjZcuNqUsqPtIPrgr2XhORfWG4aYZCoeAHNHVf1ZcuB5vgIcY52VfqdwvgGWjV6U3hJjGK422IyP5IHm5WrlyJJUuWoKioCLGxsXjrrbeQkND0zAydToe0tDR8+OGHKCgowMCBA/Haa69hzJgxXVw1kcQK9hm/9owEHv3VqlOUlNfgRElFk89xvA0R2TNJw826deuQmpqKVatWITExEcuXL0dKSgpycnIQGNj4N84XX3wRH3/8Md5//31cffXV+PHHHzFhwgTs3LkTw4YNk+AKiCRing1l3ZgadbUOY5f/iguV2mb3cXdVYEhvjrchIvsj6Y0zExMTce211+Ltt98GABgMBoSHh+OJJ57A888/32j/0NBQvPDCC5g1a5Z529133w13d3d8/PHHbXrN9tx4i6jb+uQ+4PiPwJhXgT891u7DV2w+jjc2H4OXmwtCfBrf2FQGGe6N740ZN/S1RbVERB1mFzfO1Gq1yMrKwvz5883b5HI5kpOTkZmZ2eQxtbW1je4w7e7uju3btzf7OrW1taitrTX/rNFoOlg5kcSE6FDLjbpah9XbTwEAFk+IwfjYUFtWR0QkOcmmQZSWlkKv1yMoKMhie1BQEIqKipo8JiUlBcuWLcPx48dhMBiwadMmrF+/HoWFhc2+TlpaGnx8fMyP8PBwm14HUZcrOwNUlQJyVyA4pt2Hr9lxGpqaOgwI9MRtMSGdUCARkbTsao7nihUrMGDAAFx99dVQKpWYPXs2pk+fDnkLK7DOnz8farXa/MjPz+/Ciok6wdm9xq/BgwHXxl1KLdHUXG61efKWAVDIuUglETkeycKNv78/FAoFiouLLbYXFxc3u/hbQEAAvv76a1RWVuLMmTM4evQoPD090bdv8+MCVCoVvL29LR5Edq0DXVJbjpRAU1OHvv492GpDRA5LsnCjVCoRFxeHjIwM8zaDwYCMjAwkJSW1eKybmxvCwsJQV1eHL7/8EnfccUdnl0vUfZjCjRWrD5vWr0mODmKrDRE5LEmngqempmLq1KmIj49HQkICli9fjsrKSkyfPh0AMGXKFISFhSEtLQ0AsGvXLhQUFGDo0KEoKCjAyy+/DIPBgOeee07KyyDqfFUXgZ8WGBfvO7ffuM2K1Ycvr1/DxfmIyHFJGm4mTpyI8+fPY+HChSgqKsLQoUOxceNG8yDjvLw8i/E0NTU1ePHFF3Hq1Cl4enritttuw3/+8x/4+vpKdAVEXWTfR0B2g+UOPIMBv37tOkWRuga5pZWQy4y3VSAiclSSrnMjBa5zQ3Zp7WTg6HfA4HuAqBuAPtcBAVe16xTfZBdgztpsDOntg29nX99JhRIRdQ67WOeGiNrBdLuF+IeAyBFWncI03oa3VCAiR2dXU8GJnJLmHFB+DpDJgZBYq0/z2ymOtyEi58BwQ9Tdmda1CYwGVJ5WnYLjbYjImTDcEHV3BfXhxorZUSa7co1dUteE+sDbzdUWVRERdVsMN0TdnWm8jRXr2pis31cAAEjqx/E2ROT4GG6IujODvkPr2gDA/rxL2HbsPBRyGSYn9rFhcURE3RPDDVF3dv4ooK0AlJ5AwNVWnWJFxnEAwF3DwhDRq4ctqyMi6pY4FZyoq+iqgRpN+445tc34NXQYIFe0+yX3513C1hxjq83sm/u3+3giInvEcEPUFcrygHeuA7Tl1h1vZZeUqdVmAlttiMiJMNwQdYXjPzUINu28YaV7T+CaCe1+yez8ssutNqPYakNEzoPhhqgrnK2/k/eNzwE3v9AlL7li8zEAxlabSH+22hCR8+CAYqKuUFAfbjownbs9svPL8DNbbYjISTHcEHW2GjVQamxF6chCfO1harW5cyhbbYjI+TDcEHW2gn0ABOAbAfTw7/SXa9hq8wRnSBGRE2K4IepsptsndFGXFFttiMjZMdwQdTbT7RPCOj/cHGCrDRERww1RpxLi8l29u6DlxrSuzR1DQ9lqQ0ROi+GGqDOp84HKEkDuAgTHdOpLHcgvw5ajJZDLgCduHtCpr0VE1J1xnRuizlByFDjyLVBqbElB0GDA1b1TX9LUanPnsDBEsdWGiJwYww1RZ/j60ct38waA8IROfTm22hARXcZwQ2Rr2kqg8IDx++FTAJU3kDS7U1/yTbbaEBGZMdwQ2dq5bEAYAK9Q4Pa3bH76nw4V4XhJhfnnKm0dMthqQ0RkxnBDZGvmdW1svxrx3tMX8ch/spp87s6hbLUhIgIYbohsz3QfqU5Y12b5ZmP3U2xvHwwM9jJv91C6YBbvIUVEBIDhhsj2THcAt/F9pPaevojtJ0rhIpfh7QeGI9zPw6bnJyJyFFznhsiWyosAzVlAJgdCh9n01Kap3vfE9WawISJqAVtuiGzJtBpxwCBA5dnh0326Ow9Lf8qBts4ATU0dXOQydj8REbWC4YbIlkzjbWwwmFhTo0Pa90egqakzb5uc2IetNkRErWC4IbIl00wpGwwmXrPjNDQ1dRgQ6Il3/xwHV4UM4T0ZbIiIWsNwQ2StE5uB71KBuprL2yrPG792cDCxpkaHf/96CgDwxC0D0D+w411cRETOguGGyFoH1gJlZxpv9+0DBFxt1Sl3nChF/sUq7D59EZqaOvQP9MS4mJAOFkpE5FwYboispSk0fr11EdDvlsvb/aIARfv/aWWevIDJ/95lse3JWwZAIZd1pEoiIqfDcENkrfJzxq9h8UDw4A6fbvnmYwCAq4O90LunO/qx1YaIyCoMN0TWEALQ1Icb79AOny7z5AXsyr0IpUKO9GnXItTXvcPnJCJyVlzEj8ga1ZcuDyT26njryooMY6vNxGvDGWyIiDqILTdE1iivH2/j0QtwdbPqFJ/tyce/fj2FOr0Bpy9UQamQ47GR/WxYJBGRc2K4IbKGqUvKy7ouqbIqLRZ9dxgVtZcX6PvznyLYakNEZAOSd0utXLkSkZGRcHNzQ2JiInbv3t3i/suXL8fAgQPh7u6O8PBwzJ07FzU1NS0eQ2Rz5vE21nVJrd6ei4raOlwd7IXPH03C17NG4K+3WTd9nIiILEnacrNu3TqkpqZi1apVSExMxPLly5GSkoKcnBwEBgY22v+///0vnn/+eaSnp+O6667DsWPHMG3aNMhkMixbtkyCKyCn1YHBxGVVWnyw4zQA4KnkAbg20s+GhRERkaQtN8uWLcPMmTMxffp0REdHY9WqVfDw8EB6enqT++/cuRMjRozAAw88gMjISIwePRqTJk1qtbWHyObKre+WathqMzo62MaFERGRZOFGq9UiKysLycnJl4uRy5GcnIzMzMwmj7nuuuuQlZVlDjOnTp3C999/j9tuu63Z16mtrYVGo7F4EHWYaQG/drbclFVpsaZBq42cC/QREdmcZN1SpaWl0Ov1CAoKstgeFBSEo0ePNnnMAw88gNLSUlx//fUQQqCurg6PPvoo/vrXvzb7OmlpaXjllVdsWjuRtWNu0rfnopytNkREnUryAcXtsXXrVixevBjvvPMO9u3bh/Xr12PDhg3429/+1uwx8+fPh1qtNj/y8/O7sGJyWFZ0S6mrdOaxNnNuYasNEVFnkazlxt/fHwqFAsXFxRbbi4uLERzc9G+0CxYswIMPPogZM2YAAGJiYlBZWYlHHnkEL7zwAuTyxllNpVJBpVLZ/gLIeemqjYv4Ae3qllq943KrTco1bLUhIuoskrXcKJVKxMXFISMjw7zNYDAgIyMDSUlJTR5TVVXVKMAoFAoAgBCi84olasjUJeXqAbj5tOkQdZUOH2zPBcBWGyKizibpVPDU1FRMnToV8fHxSEhIwPLly1FZWYnp06cDAKZMmYKwsDCkpaUBAMaPH49ly5Zh2LBhSExMxIkTJ7BgwQKMHz/eHHKIOl15g8HEsraFFFOrzcAgttoQEXU2ScPNxIkTcf78eSxcuBBFRUUYOnQoNm7caB5knJeXZ9FS8+KLL0Imk+HFF19EQUEBAgICMH78ePzjH/+Q6hLIGZlmSrXxnlIWrTacIUVE1Olkwsn6czQaDXx8fKBWq+Ht7S11OWSPti8HNr8EDJkI3PWvVnd/Y9MxrMg4joFBXvhhzg0MN0REVmjP57ddzZYi6hbK277Gjbpah/QdbLUhIupKDDdE7dWOm2amb89FeY1xrM0YjrUhIuoSDDdE7dXG+0o1bLV5kjOkiIi6DMMNUXsIAVw8Zfzep3eLu36ww9hqc1WQJ8YOZqsNEVFXYbghao9LuUD1RUChBAIHNbubulqH1eZ1ba5iqw0RURdiuCFqj7NZxq/BQwCX5le+ZqsNEZF0GG6I2qNgr/Fr7/gWd/vud+OMqlmj+rPVhoioizHcELXH2fpwE9Z8uKmt0yO3tBIAkBDl1xVVERFRAww3RG1VVwsU/W78Pmx4s7udLq2C3iDgpXJBsLdbFxVHREQmDDdEbVV0ENBrAXc/wK9vs7sdKy4HAPQP8oSsjfeeIiIi22G4IWqrgvrBxGFxLd4w83hJBQDgqkCvrqiKiIiuwHBD1FZtHEx8vL7lZkCQZ2dXRERETZD0ruBE3Ya2CqgqbXmf/N3Gr2FxLe5markZEMSWGyIiKTDcEFWcB96OB2rK2rZ/C+FGW2fA6fqZUlex5YaISBIMN0Snf6kPNrIWF+YDAETfAXg0P707t7QSdZwpRUQkKYYbItOqw9c+DIxb2qFTHS/hTCkiIqlxQDFRQesL87XVsWLOlCIikhrDDTk3vQ4oPGD8vpVZUG1xooQzpYiIpMZwQ86t+BBQVwO4+QB+/Tp8OlPLDWdKERFJh+GGnJupSyp0OCDv2D+HKm0dZ0oREXUDDDfk3EyDiW3QJfXJb3moMwhE9PLgTCkiIgkx3JBzs9Fg4iptHd775SQAYNao/pwpRUQkIYYbcl41aqD0mPH7DrbcfPJbHkortAj3c8eEYWE2KI6IiKzFdW7IfhTsA45+Bwhhm/OVFxm/+kYAPfzbfXiJpgaf7s5HbZ0e6/bkAwCeGDUArgr+zkBEJCWGG7IfXz4MXDxl+/OGJ1h12NKfjmHd3vzLp/Fzx4ThbLUhIpIaww3Zh4qS+mAjAxIfBWw1pkWhBOIfsurQHSeNN9ocHxuKYG8V7hgaxlYbIqJugOGG7ENB/aymgIHA2FelrQVA/sUqnL1UDRe5DK/eFYMeKv5TIiLqLvhrJtmHs6ZZTc3fkbsr7cq9CAAY0tuHwYaIqJthuCH7YGq56Sbh5rdTFwAAiX17SVwJERFdieGGuj+DwThTCrDJYnu2YAo3f2K4ISLqdhhuqPu7cAKoVQMu7kBgtNTVmMfbKOQyxEf0lLocIiK6AsMNdX+mLqmQWEDhKm0t4HgbIqLujv8zU/dnukWCxF1SW3NKcKKkAj8eMi7+xy4pIqLuieGGur9uMFPq97NlmPbBHottDDdERN0Tww11jupLwIe3A2V5HT9XTZnxq4QtNys2HwcADArxxsAgT/Tx88AN/dt/ywYiIup8DDfUOY5vBop+t935gmIAn3Dbna8dfj9bhoyjJZDLgHcmD0eUfw9J6iAiorZhuKHOYRonM3QycP3cjp/PN8J2t1xoJ1OrzZ3DwhhsiIjsQLeYLbVy5UpERkbCzc0NiYmJ2L17d7P7jhw5EjKZrNFj3LhxXVgxtco0TqbvKMB/QMcfLkpJLuNggdrcavPEzQMkqYGIiNpH8nCzbt06pKam4qWXXsK+ffsQGxuLlJQUlJSUNLn/+vXrUVhYaH4cPHgQCoUC9957bxdXTs2q017ukurdPVYUttYPBwsBAGMHh7DVhojITkgebpYtW4aZM2di+vTpiI6OxqpVq+Dh4YH09PQm9/fz80NwcLD5sWnTJnh4eDQbbmpra6HRaCwe1MmK/wD0WsDdD+gZJXU1HfLbKeOaNjddFSBxJURE1FaShhutVousrCwkJyebt8nlciQnJyMzM7NN51i9ejXuv/9+9OjR9G/VaWlp8PHxMT/Cw6UZlOpUzja4D5RE42RsoUpbh9/PlgHgtG8iIntiVbj5+eefbfLipaWl0Ov1CAoKstgeFBSEoqKiVo/fvXs3Dh48iBkzZjS7z/z586FWq82P/Pz8DtdNregmi+511L4zZdDpBUJ93BDu5y51OURE1EZWhZsxY8agX79++Pvf/y5pWFi9ejViYmKQkJDQ7D4qlQre3t4WD+pk5jt423e4aXhzTJkdt0ARETkbq8JNQUEBZs+ejS+++AJ9+/ZFSkoKPvvsM2i12nadx9/fHwqFAsXFxRbbi4uLERwc3OKxlZWVWLt2LR5++OF210+dqPqS8UaXABA2XNpaOoh3/iYisk9WrXPj7++PuXPnYu7cudi3bx8++OADPP7443j88cfxwAMP4OGHH0ZsbGyr51EqlYiLi0NGRgbuvPNOAIDBYEBGRgZmz57d4rGff/45amtr8ec//9maS6C2KDkKlBxu3zGlx4xf/foCHn62r6mLVGv1OMDxNkREdqnDi/gNHz4cwcHB6NWrF1599VWkp6fjnXfeQVJSElatWoVrrrmmxeNTU1MxdepUxMfHIyEhAcuXL0dlZSWmT58OAJgyZQrCwsKQlpZmcdzq1atx5513olcvfvB0iuoy4F8jgbpq64638y6pfXmXoNMLhHC8DRGR3bE63Oh0OnzzzTdIT0/Hpk2bEB8fj7fffhuTJk3C+fPn8eKLL+Lee+/F4cMt/+Y/ceJEnD9/HgsXLkRRURGGDh2KjRs3mgcZ5+XlQS637D3LycnB9u3b8dNPP1lbPrXmUq4x2Li4Ab2vbd+xLm7AdU90Tl2dbNlPOfjpcDEuVRm7WDnehojI/siEEKK9Bz3xxBP49NNPIYTAgw8+iBkzZmDw4MEW+xQVFSE0NBQGg8FmxdqCRqOBj48P1Go1Bxe35Oj3wNpJQOgw4JGtUlfTJQ4WqPF/b2232LbygeEYNyREooqIiMikPZ/fVrXcHD58GG+99RbuuusuqFSqJvfx9/e32ZRxkkD5OeNXr1Bp6+hCy+vvIXVrdBCmXRcJbzdXDA5jACYisjdWhZuMjIzWT+zigptuusma01N3oKkPN97OEW4OFqix+Ugx5DJg3pir0T/QU+qSiIjISlaFm7S0NAQFBeGhhx6y2J6eno7z589j3rx5NimOJKQx3lMJ3o7XJVNZW4d9eZegN1zukV29PRcAcHtsKIMNEZGdsyrcvPfee/jvf//baPs111yD+++/n+HGEWgKjF8dsFvqyU/3I+No4xuzymXAbN75m4jI7lkVboqKihAS0vg3+oCAABQWFna4KOoGyk0tN44VbvblXULG0RIo5DIMCvGyeO7/hrDVhojIEVgVbsLDw7Fjxw5ERVne8XnHjh0IDXWsD0OnpXHMcLOiftDw3cPD8Po9rS80SURE9seqcDNz5kw89dRT0Ol0uPnmmwEYBxk/99xzePrpp21aIEmgRgNoy43feznOmJv9eZew7dh5KOQyzB7F7iciIkdlVbh59tlnceHCBTz++OPm+0m5ublh3rx5mD9/vk0LJAmYuqRUPoDKcbppVmQYW23uGhaGPr08JK6GiIg6i1XhRiaT4bXXXsOCBQtw5MgRuLu7Y8CAAc2ueUN2xjSY2IFmSu3Pu4StOfWtNjf3l7ocIiLqRB26t5SnpyeuvbadS/NT9+eA420attpE9OohcTVERNSZrA43e/fuxWeffYa8vDxz15TJ+vXrO1wYScjBVifOzi9jqw0RkROxKtysXbsWU6ZMQUpKCn766SeMHj0ax44dQ3FxMSZMmGDrGqmrmVcnNnZL1ej0OFFSgfbfhax7WPpTDgC22hAROQurws3ixYvxxhtvYNasWfDy8sKKFSsQFRWFv/zlL02uf0N2pkG3lBACD67ehT2nL0lbUwex1YaIyHlYFW5OnjyJcePGAQCUSiUqKyshk8kwd+5c3HzzzXjllVdsWiR1sQarE2/NOY89py/BRS5DoJd9DhiXyWSYlBDOVhsiIidhVbjp2bMnysuN66CEhYXh4MGDiImJQVlZGaqqqmxaIEmgfiq48ArG8vXHAAAPXx+F+bcNkrIqIiKiNrEq3Nx4443YtGkTYmJicO+992LOnDnYsmULNm3ahFtuucXWNVJXqtMClecBADtKVDhwtgDurgrMvLGvxIURERG1jVXh5u2330ZNTQ0A4IUXXoCrqyt27tyJu+++Gy+++KJNC6QuZmq1USixZPsFAMCDSRHw97TPLikiInI+7Q43dXV1+O6775CSkgIAkMvleP75521eGEmkPtzoewThwFk1lAo5HmGrDRER2RF5ew9wcXHBo48+am65IQdTPw283DUAANA/0JOtNkREZFfaHW4AICEhAdnZ2TYuhbqFKmNX1AVZTwDAVUGOc28pIiJyDlaNuXn88ceRmpqK/Px8xMXFoUcPyym2Q4YMsUlxJIGqiwCAEp07AGBAkJeU1RAREbWbVeHm/vvvBwA8+eST5m0ymQxCCMhkMuj1ettUR12v2rhYX0GtGwBgQCBbboiIyL5YFW5yc3NtXQd1F/Xh5nSlEgBbboiIyP5YFW4iIiJsXQd1F/Xh5ryhB1QucvTx85C4ICIiovaxKtx89NFHLT4/ZcoUq4qhbqDaOOZGLTzRL9ATCrlM4oKIiIjax6pwM2fOHIufdTodqqqqoFQq4eHhwXBjz+pbbsqEJwZwphQREdkhq6aCX7p0yeJRUVGBnJwcXH/99fj0009tXSN1JVO4QQ9cxfE2RERkh6wKN00ZMGAAXn311UatOmRHDAZzuLkkvNCfM6WIiMgO2SzcAMbVi8+dO2fLU1JXqtUAwgAAULPlhoiI7JRVY26+/fZbi5+FECgsLMTbb7+NESNG2KQwkkB9q021UEK4uHGmFBER2SWrws2dd95p8bNMJkNAQABuvvlmLF261BZ1kRTM42080S+AM6WIiMg+WRVuDAaDreug7qDBTKkwXzeJiyEiIrKOTcfckJ1rEG7clVblXiIiIslZFW7uvvtuvPbaa422v/7667j33ns7XBRJpME0cA9XhcTFEBERWceqcPPLL7/gtttua7R97Nix+OWXXzpcFEnEouWG4YaIiOyTVeGmoqICSqWy0XZXV1doNJoOF0USaTCgmOGGiIjslVXhJiYmBuvWrWu0fe3atYiOjm7XuVauXInIyEi4ubkhMTERu3fvbnH/srIyzJo1CyEhIVCpVLjqqqvw/ffft+s1qRlVxvtKlQlPdksREZHdsmrU6IIFC3DXXXfh5MmTuPnmmwEAGRkZ+PTTT/H555+3+Tzr1q1DamoqVq1ahcTERCxfvhwpKSnIyclBYGBgo/21Wi1uvfVWBAYG4osvvkBYWBjOnDkDX19fay6DrtSg5SaALTdERGSnrAo348ePx9dff43Fixfjiy++gLu7O4YMGYLNmzfjpptuavN5li1bhpkzZ2L69OkAgFWrVmHDhg1IT0/H888/32j/9PR0XLx4ETt37oSrqysAIDIy0ppLoKaYx9z0YLcUERHZLavn+44bNw7jxo2z+oW1Wi2ysrIwf/588za5XI7k5GRkZmY2ecy3336LpKQkzJo1C9988w0CAgLwwAMPYN68eVAomv4wrq2tRW1trflnjglqgTnceMGD4YaIiOyUVWNu9uzZg127djXavmvXLuzdu7dN5ygtLYVer0dQUJDF9qCgIBQVFTV5zKlTp/DFF19Ar9fj+++/x4IFC7B06VL8/e9/b/Z10tLS4OPjY36Eh4e3qT6nVF0/5gY94M4xN0REZKesCjezZs1Cfn5+o+0FBQWYNWtWh4tqjsFgQGBgIP71r38hLi4OEydOxAsvvIBVq1Y1e8z8+fOhVqvNj6bqJljcEZyL+BERkT2z6hPs8OHDGD58eKPtw4YNw+HDh9t0Dn9/fygUChQXF1tsLy4uRnBwcJPHhISEwNXV1aILatCgQSgqKoJWq21yerpKpYJKpWpTTU5NW26+I3gZPNlyQ0REdsuqlhuVStUolABAYWEhXFzalpeUSiXi4uKQkZFh3mYwGJCRkYGkpKQmjxkxYgROnDhhcW+rY8eOISQkpMlgQ+1Q32pTAyVqoeSYGyIisltWhZvRo0ebu3tMysrK8Ne//hW33nprm8+TmpqK999/Hx9++CGOHDmCxx57DJWVlebZU1OmTLEYcPzYY4/h4sWLmDNnDo4dO4YNGzZg8eLFndoV5jTq17hRwxMAOFuKiIjsllXdUv/85z9x4403IiIiAsOGDQMAZGdnIygoCP/5z3/afJ6JEyfi/PnzWLhwIYqKijB06FBs3LjRPMg4Ly8Pcvnl/BUeHo4ff/wRc+fOxZAhQxAWFoY5c+Zg3rx51lwGNVTfcnNJ9AAAdksREZHdkgkhhDUHVlZW4pNPPsGBAwfM69xMmjTJvP5Md6XRaODj4wO1Wg1vb2+py+k+/vgC+PJh/GYYhPu1C5C98Fb4erCrj4iIuof2fH5bPSWmR48euP7669GnTx9otVoAwA8//AAAuP322609LUnF3HLDbikiIrJvVoWbU6dOYcKECfjjjz8gk8kghIBMJjM/r9frbVYgdZEG08DlMkCpsGo4FhERkeSs+gSbM2cOoqKiUFJSAg8PDxw8eBDbtm1DfHw8tm7dauMSqUvUhxs1POGhdLEIq0RERPbEqpabzMxMbNmyBf7+/pDL5VAoFLj++uuRlpaGJ598Evv377d1ndTZzHcE532liIjIvlnVcqPX6+Hl5QXAuBjfuXPnAAARERHIycmxXXXUdcoLAQAlwpczpYiIyK5Z1XIzePBgHDhwAFFRUUhMTMTrr78OpVKJf/3rX+jbt6+ta6SuUB9uCtGLC/gREZFdsyrcvPjii6isrAQALFq0CP/3f/+HG264Ab169cK6detsWiB1EY2x9a1Y9IQPww0REdkxq8JNSkqK+fv+/fvj6NGjuHjxInr27MmBqPaoRgNoKwAARcIPweyWIiIiO2azWz/7+fnZ6lTU1eq7pLQuXqiCG7uliIjIrnExEwI0BQCASlUAAMBdabPMS0RE1OUYbgjQGFtuypWBAAB3V/61ICIi+8VPMQLKjYOJ1S7GlhsPttwQEZEdY7gh80ypS4peAAA3DigmIiI7xnBD5m6pCwp/AOCAYiIismsMN2TuljovM854Y7ghIiJ7xnBDDRbwM4YbdksREZE9Y7hxdnVaoPI8AKBQsOWGiIjsH8ONs6soMn5VKFFS5wmA4YaIiOwbw42zq++SglcIqnQGAOyWIiIi+8Zw4+xM4cY7FDU6PQCuc0NERPaN4cbZ1d9XCl4hqNKawg1bboiIyH4x3Di7Bi03Vdo6AOyWIiIi+8Zw4+wsuqWMY27YckNERPaM4cbZ1YcbvWcItHqGGyIisn8MN86ufnXiGvcg8yZ2SxERkT1juHF2NWrjFxdvAIBcBqhc+NeCiIjsFz/FnJ22CgBQJVQAjNPAZTKZlBURERF1CMONM9PrAIMOAFAFY7hhlxQREdk7hhtnpq00f1tpbrlhuCEiIvvGcOPMdMYuKcgUqKozhhqGGyIisncMN85MV238quyB6jreV4qIiBwDw40zM3VLuXqYVydmyw0REdk7hhtnZuqWUnqguv6+Uu5suSEiIjvHcOPMzC03PVBdf0dwd7bcEBGRnWO4cWYNWm54R3AiInIUDDfOrH4BP7iyW4qIiBwHw40z09V3Sykbdku5SFgQERFRx3WLcLNy5UpERkbCzc0NiYmJ2L17d7P7rlmzBjKZzOLh5ubWhdU6kAYtN+yWIiIiRyF5uFm3bh1SU1Px0ksvYd++fYiNjUVKSgpKSkqaPcbb2xuFhYXmx5kzZ7qwYgdiGnPj6o7q+qng7JYiIiJ7J3m4WbZsGWbOnInp06cjOjoaq1atgoeHB9LT05s9RiaTITg42PwICgrqwoodiLapbimGGyIism+ShhutVousrCwkJyebt8nlciQnJyMzM7PZ4yoqKhAREYHw8HDccccdOHToULP71tbWQqPRWDyono7dUkRE5HgkDTelpaXQ6/WNWl6CgoJQVFTU5DEDBw5Eeno6vvnmG3z88ccwGAy47rrrcPbs2Sb3T0tLg4+Pj/kRHh5u8+uwW9rLU8FrdJwtRUREjkHybqn2SkpKwpQpUzB06FDcdNNNWL9+PQICAvDee+81uf/8+fOhVqvNj/z8/C6uuBvTXV7Ez9Ryw24pIiKyd5LO+/X394dCoUBxcbHF9uLiYgQHB7fpHK6urhg2bBhOnDjR5PMqlQoqlarDtTokbePbL3hwKjgREdk5SVtulEol4uLikJGRYd5mMBiQkZGBpKSkNp1Dr9fjjz/+QEhISGeV6bgajLmpZrcUERE5CMl/TU9NTcXUqVMRHx+PhIQELF++HJWVlZg+fToAYMqUKQgLC0NaWhoAYNGiRfjTn/6E/v37o6ysDEuWLMGZM2cwY8YMKS/DPjWYLcVuKSIichSSh5uJEyfi/PnzWLhwIYqKijB06FBs3LjRPMg4Ly8PcvnlBqZLly5h5syZKCoqQs+ePREXF4edO3ciOjpaqkuwXw1bbuq7qDhbioiI7J1MCCGkLqIraTQa+Pj4QK1Ww9vbW+pypPVGDKDOQ91Dm9D/nfMAgP0LbkXPHkqJCyMiIrLUns9vu5stRTZU33JTK7t8+wp2SxERkb1juHFm9eGmGsbZZHIZoHLhXwkiIrJv/CRzVgZDg3BjbLlxd1VAJpNJWRUREVGHMdw4q7pq87eVwthy4841boiIyAEw3Dgr0wJ+ACoNrgAAdyX/OhARkf3jp5mzMt96wQM1dcYJcx6ubLkhIiL7x3DjrLSN7wjOmVJEROQIGG6cle7yfaWqtHUAeOsFIiJyDAw3zkp7+Y7gNTrTTTMZboiIyP4x3Dgri5YbdksREZHjYLhxVtrLA4p5R3AiInIkDDfOyuKmmeyWIiIix8Fw46y0TXVLcSo4ERHZP4YbZ6W7PKCY3VJERORIGG6cVYOWG3ZLERGRI2G4cVa6hov41a9zw3BDREQOgOHGWZlmSyl7oFpnAMBuKSIicgwMN87KYraUseWG3VJEROQIGG6cla7a+JWL+BERkYNhuHFWWs6WIiIix8Rw46x0Tc2W4jo3RERk/xhunJW24WwpU7cU/zoQEZH946eZs9I1nC3FFYqJiMhxMNw4q/qWG72LO7R1xqngHhxzQ0REDoDhxlnVj7mphsq8ibOliIjIETDcOCMhzLOlamRuAACZDFC58K8DERHZP36aOSO9FhDGcTZVwthy4+GqgEwmk7IqIiIim2C4cUamNW4AVNV3S7FLioiIHAXDjTMyrXGjUKKqzthaw3BDRESOguHGGWkb3leqfgE/V04DJyIix8Bw44xq1Mavbt7mcOPGlhsiInIQDDfOqPqS8au7H6p0ppYbhhsiInIMDDfOyBxueqJaWwcA8GDLDREROQiGG2dkEW7YLUVERI6F4cYZVV80fnXvyW4pIiJyOAw3zqiJlht2SxERkaNguHFGpnDj4cduKSIicjjdItysXLkSkZGRcHNzQ2JiInbv3t2m49auXQuZTIY777yzcwt0NA1abi53S3GdGyIicgySh5t169YhNTUVL730Evbt24fY2FikpKSgpKSkxeNOnz6NZ555BjfccEMXVepAqi6PuWG3FBERORrJw82yZcswc+ZMTJ8+HdHR0Vi1ahU8PDyQnp7e7DF6vR6TJ0/GK6+8gr59+7Z4/traWmg0GouH0+NsKSIicmCShhutVousrCwkJyebt8nlciQnJyMzM7PZ4xYtWoTAwEA8/PDDrb5GWloafHx8zI/w8HCb1G7XuIgfERE5MEnDTWlpKfR6PYKCgiy2BwUFoaioqMljtm/fjtWrV+P9999v02vMnz8farXa/MjPz+9w3XbNoL98+4UGi/jxxplEROQo7GoUaXl5OR588EG8//778Pf3b9MxKpUKKpWqkyuzIzVqAML4vbsvqutbbhhuiIjIUUgabvz9/aFQKFBcXGyxvbi4GMHBwY32P3nyJE6fPo3x48ebtxkMBgCAi4sLcnJy0K9fv84t2t6ZuqSUXoDCFVVadksREZFjkbRbSqlUIi4uDhkZGeZtBoMBGRkZSEpKarT/1VdfjT/++APZ2dnmx+23345Ro0YhOzub42nawrzGTU8AQI2WLTdERORYJO+WSk1NxdSpUxEfH4+EhAQsX74clZWVmD59OgBgypQpCAsLQ1paGtzc3DB48GCL4319fQGg0XZqRoOZUgCgrtYBALzdXKWqiIiIyKYkDzcTJ07E+fPnsXDhQhQVFWHo0KHYuHGjeZBxXl4e5HLJZ6w7jgZr3NTo9Kisb7nx81RKWBQREZHtSB5uAGD27NmYPXt2k89t3bq1xWPXrFlj+4IcWYOWmwuVWgCAq0IGL1W3+KtARETUYWwScTYN1ri5WGEMN716qCCTySQsioiIyHYYbhyBEG3ft0HLTWllLQDArwe7pIiIyHEw3Ni7bUuApQOBS6fbtn/15TE35pYbjrchIiIHwnBj73I2ABXFwJHv2rZ/g5abi5WmbimGGyIichwMN/autsL4tSCrbfub17nxMw8o9uvBFZyJiMhxMNzYkSJ1DW54fQtWbD5+eWNtufFrwd62naThbKkK45gbdksREZEjYbixI9uOlSD/YjW+zi64vNEUbsrygIrzrZ+kwTo3F80tNww3RETkOBhu7MixYmMXVMGlahgMwniHb13l5R1a65q64o7gFzjmhoiIHBDDjR05XmIMN1q9AecragFtheUOrXVNWdwRvMGAYnZLERGRA2G4sSPHi8vN35+9VHW5S8q8sZVwc8UdwU1jbjigmIiIHAnDjZ0or9GhUF1j/vnsperG4aZgH2AwNH+SBoOJLe4rxW4pIiJyIAw3dsLUJWViDDf127zDABd3oFYNXDzZ/EnM4cbX3CXlqpDB2433lSIiIsfBcGMnThRfGW6qgFqN8Qf3nkDo0PonmuiaqjgP7P0AOPS18WcPP1youDxTiveVIiIiR8JwYyeO1Y+38XF3BXBFt5TKCwiLM37f1KDi758BvnsKyP7Y+HOPAFyo5HgbIiJyTAw3duJYfbfUTVcFAKgPN6bZUkrPy+HmypYbIYDT243f97sFiLkPGPGUuVvKnzOliIjIwXCwhZ04Ud9yM+rqAHx74JxxrZsajTGdqryA3vHGHYsPAroawNXN+HNZHlBVCshdgfv/a95+8fgpABxMTEREjoctN3agvEaHc/UzpW4cEACFXAat3oCq8jLjDiovwCcc6BEIGOqAot8vH2zqpgoefDnwACit4OrERETkmBhu7MCJ+i6pQC8VenmqEOxtDCkVmjLjDiovQCZrumvqbP2qxWHxFue8WD/mhqsTExGRo2G3lI1cqKjFtmNtuLeTFfblGadwDwjyBACE9XRHQVk1qivqp3arvIxfe8cBx37ApeOZ0F3zEAK93S633PS+MtyYVifmgGIiInIsDDc2cuZiFVI/O9Cpr3FVkDHE9O7pjt25gK6q/j5RpnBT3zqjOfkbpv3rN/z0ZBJcC+trMrXq1GO3FBEROSqGGxvxdnPBjfUzmTpDD6UCU5IiAQC9e3oAAPTVDaaCA0DYcABAhKwE6tJC/PzLVoyuqwHcfAC/fhbnu8ibZhIRkYNiuLGR/oFe+OihhC55rd493QEAMm19uFEau6vg5oNzLuEIrctHrPwk/ti1H6MBY6uN3HJ4lSncsOWGiIgcDQcU2yFTuFHo6te5UXkDAHR6A3bp+gIA4l1OIqLmiPH5K7qkanR6VNTWAeCYGyIicjxsubFD4fXdUkp9FSCDuVvqjwI1sur6YoLrNsxQ/gS9TgcAWJjlht2//2I+vs4gAPC+UkRE5Jj4yWaHgn3cIJMBPVBt3KAydkv9duoCdhgGwwA5VHUVgAyoEG745kI41ChvdJ6BwV68rxQRETkchhs75KqQo1cPJbx0pnBjbLnZdeoickUIvrruK9zdHyir0uG4Pgjv9Aht8jwxvX26qmQiIqIuw3Bjp0I9FVCVGbudoPKCTm/A3tMXAQDRMcOBEG/4ArhWsgqJiIikwQHFdirc03D5B6UXDhaoUanVw9fDFQPr18MhIiJyRgw3dircwzjbSSdXAQoX7KlvtUmI9INcznE0RETkvBhu7FSIuzHc1Mh7AAByS6sAAINCvCWriYiIqDtguLFTQW7G8TZVMK55c/aSMdyY1sAhIiJyVgw3dirA1bjCcHl9uCm4ZJw5Zbo1AxERkbNiuLFTverDjVrvBoNB4GyZKdyw5YaIiJwbp4LbSm05UHy4886v9ACCBgP1i+75KmoAAJf0KpSU10JbZ4BCLkOIj1vn1UBERGQHGG5speQokD66c19j3FLg2hkAAK/61YnLhRt+P1sGAAj2doOLgo1xRETk3BhubMVFBfj17Zxz11YAlSXAiQxzuFHoKgEAFcIdR/MuAWCXFBEREdBNws3KlSuxZMkSFBUVITY2Fm+99RYSEhKa3Hf9+vVYvHgxTpw4AZ1OhwEDBuDpp5/Ggw8+2MVVXyFkCPDk/s45d94uY6vQ2b2AEMauqVrjvaIq4I79Z8oAAGEMN0RERNIPKF63bh1SU1Px0ksvYd++fYiNjUVKSgpKSkqa3N/Pzw8vvPACMjMz8fvvv2P69OmYPn06fvzxxy6uvAuFDAHkLsbWG3W+cVutBoCx5eZAfbcUZ0oRERF1g3CzbNkyzJw5E9OnT0d0dDRWrVoFDw8PpKenN7n/yJEjMWHCBAwaNAj9+vXDnDlzMGTIEGzfvr2LK+9Cru7GwcSAsfUGALQVAIwtN7V1xlsxsFuKiIhI4nCj1WqRlZWF5ORk8za5XI7k5GRkZma2erwQAhkZGcjJycGNN97Y5D61tbXQaDQWD7vUO974tSDL+NXULSUuBxqGGyIiIonDTWlpKfR6PYKCgiy2BwUFoaioqNnj1Go1PD09oVQqMW7cOLz11lu49dZbm9w3LS0NPj4+5kd4eLhNr6HLhMUZv14ZbnB56nc4u6WIiIik75ayhpeXF7Kzs7Fnzx784x//QGpqKrZu3drkvvPnz4darTY/8vPzu7ZYWwmrb7k5lw3odcYZVDB2SwGAXAYEc40bIiIiaWdL+fv7Q6FQoLi42GJ7cXExgoODmz1OLpejf//+AIChQ4fiyJEjSEtLw8iRIxvtq1KpoFKpbFq3JHr1B1Q+QK0aKDncqFsqxMcdrlzjhoiISNqWG6VSibi4OGRkZJi3GQwGZGRkICkpqc3nMRgMqK2t7YwSuw+5HAgbbvy+IOvybKn6lhtOAyciIjKSfJ2b1NRUTJ06FfHx8UhISMDy5ctRWVmJ6dOnAwCmTJmCsLAwpKWlATCOoYmPj0e/fv1QW1uL77//Hv/5z3/w7rvvSnkZXaN3PHDqZ+C3VRZTwQEOJiYiIjKRPNxMnDgR58+fx8KFC1FUVIShQ4di48aN5kHGeXl5kMsvNzBVVlbi8ccfx9mzZ+Hu7o6rr74aH3/8MSZOnCjVJXSdPvWtWaU5xq8u7lB49ASquMYNERGRiUwIIaQuoitpNBr4+PhArVbD29tb6nLaRwjg4JdAeaHx57B4jP26DkcKNXj9niG4L95OZ4IRERG1oj2f35K33FA7yGRAzD0WmyYMO4kanR43DPCXqCgiIqLuhS03RERE1O215/Obc4eJiIjIoTDcEBERkUNhuCEiIiKHwnBDREREDoXhhoiIiBwKww0RERE5FIYbIiIicigMN0RERORQGG6IiIjIoTDcEBERkUNhuCEiIiKHwnBDREREDoXhhoiIiBwKww0RERE5FBepC+hqQggAxlunExERkX0wfW6bPsdb4nThpry8HAAQHh4ucSVERETUXuXl5fDx8WlxH5loSwRyIAaDAefOnYOXlxdkMplNz63RaBAeHo78/Hx4e3vb9NzdgaNfH8BrdASOfn0Ar9EROPr1Aba/RiEEysvLERoaCrm85VE1TtdyI5fL0bt37059DW9vb4f9ywo4/vUBvEZH4OjXB/AaHYGjXx9g22tsrcXGhAOKiYiIyKEw3BAREZFDYbixIZVKhZdeegkqlUrqUjqFo18fwGt0BI5+fQCv0RE4+vUB0l6j0w0oJiIiIsfGlhsiIiJyKAw3RERE5FAYboiIiMihMNwQERGRQ2G4sZGVK1ciMjISbm5uSExMxO7du6UuyWppaWm49tpr4eXlhcDAQNx5553Iycmx2GfkyJGQyWQWj0cffVSiitvn5ZdfblT71VdfbX6+pqYGs2bNQq9eveDp6Ym7774bxcXFElbcfpGRkY2uUSaTYdasWQDs8/375ZdfMH78eISGhkImk+Hrr7+2eF4IgYULFyIkJATu7u5ITk7G8ePHLfa5ePEiJk+eDG9vb/j6+uLhhx9GRUVFF15F81q6Pp1Oh3nz5iEmJgY9evRAaGgopkyZgnPnzlmco6n3/dVXX+3iK2lea+/htGnTGtU/ZswYi32683sItH6NTf27lMlkWLJkiXmf7vw+tuXzoS3/h+bl5WHcuHHw8PBAYGAgnn32WdTV1dmsToYbG1i3bh1SU1Px0ksvYd++fYiNjUVKSgpKSkqkLs0q27Ztw6xZs/Dbb79h06ZN0Ol0GD16NCorKy32mzlzJgoLC82P119/XaKK2++aa66xqH379u3m5+bOnYv//e9/+Pzzz7Ft2zacO3cOd911l4TVtt+ePXssrm/Tpk0AgHvvvde8j729f5WVlYiNjcXKlSubfP7111/Hm2++iVWrVmHXrl3o0aMHUlJSUFNTY95n8uTJOHToEDZt2oTvvvsOv/zyCx555JGuuoQWtXR9VVVV2LdvHxYsWIB9+/Zh/fr1yMnJwe23395o30WLFlm8r0888URXlN8mrb2HADBmzBiL+j/99FOL57vzewi0fo0Nr62wsBDp6emQyWS4++67Lfbrru9jWz4fWvs/VK/XY9y4cdBqtdi5cyc+/PBDrFmzBgsXLrRdoYI6LCEhQcyaNcv8s16vF6GhoSItLU3CqmynpKREABDbtm0zb7vpppvEnDlzpCuqA1566SURGxvb5HNlZWXC1dVVfP755+ZtR44cEQBEZmZmF1Voe3PmzBH9+vUTBoNBCGHf758QQgAQX331lflng8EggoODxZIlS8zbysrKhEqlEp9++qkQQojDhw8LAGLPnj3mfX744Qchk8lEQUFBl9XeFldeX1N2794tAIgzZ86Yt0VERIg33nijc4uzkaaucerUqeKOO+5o9hh7eg+FaNv7eMcdd4ibb77ZYps9vY9Xfj605f/Q77//XsjlclFUVGTe59133xXe3t6itrbWJnWx5aaDtFotsrKykJycbN4ml8uRnJyMzMxMCSuzHbVaDQDw8/Oz2P7JJ5/A398fgwcPxvz581FVVSVFeVY5fvw4QkND0bdvX0yePBl5eXkAgKysLOh0Oov38+qrr0afPn3s9v3UarX4+OOP8dBDD1ncLNae378r5ebmoqioyOJ98/HxQWJiovl9y8zMhK+vL+Lj4837JCcnQy6XY9euXV1ec0ep1WrIZDL4+vpabH/11VfRq1cvDBs2DEuWLLFpU39X2Lp1KwIDAzFw4EA89thjuHDhgvk5R3sPi4uLsWHDBjz88MONnrOX9/HKz4e2/B+amZmJmJgYBAUFmfdJSUmBRqPBoUOHbFKX090409ZKS0uh1+st3iQACAoKwtGjRyWqynYMBgOeeuopjBgxAoMHDzZvf+CBBxAREYHQ0FD8/vvvmDdvHnJycrB+/XoJq22bxMRErFmzBgMHDkRhYSFeeeUV3HDDDTh48CCKioqgVCobfWAEBQWhqKhImoI76Ouvv0ZZWRmmTZtm3mbP719TTO9NU/8OTc8VFRUhMDDQ4nkXFxf4+fnZ3XtbU1ODefPmYdKkSRY3JHzyyScxfPhw+Pn5YefOnZg/fz4KCwuxbNkyCattuzFjxuCuu+5CVFQUTp48ib/+9a8YO3YsMjMzoVAoHOo9BIAPP/wQXl5ejbq97eV9bOrzoS3/hxYVFTX5b9X0nC0w3FCLZs2ahYMHD1qMSQFg0ccdExODkJAQ3HLLLTh58iT69evX1WW2y9ixY83fDxkyBImJiYiIiMBnn30Gd3d3CSvrHKtXr8bYsWMRGhpq3mbP75+z0+l0uO+++yCEwLvvvmvxXGpqqvn7IUOGQKlU4i9/+QvS0tLsYpn/+++/3/x9TEwMhgwZgn79+mHr1q245ZZbJKysc6Snp2Py5Mlwc3Oz2G4v72Nznw/dAbulOsjf3x8KhaLRSPDi4mIEBwdLVJVtzJ49G9999x1+/vln9O7du8V9ExMTAQAnTpzoitJsytfXF1dddRVOnDiB4OBgaLValJWVWexjr+/nmTNnsHnzZsyYMaPF/ez5/QNgfm9a+ncYHBzcaJB/XV0dLl68aDfvrSnYnDlzBps2bbJotWlKYmIi6urqcPr06a4p0Mb69u0Lf39/899LR3gPTX799Vfk5OS0+m8T6J7vY3OfD235PzQ4OLjJf6um52yB4aaDlEol4uLikJGRYd5mMBiQkZGBpKQkCSuznhACs2fPxldffYUtW7YgKiqq1WOys7MBACEhIZ1cne1VVFTg5MmTCAkJQVxcHFxdXS3ez5ycHOTl5dnl+/nBBx8gMDAQ48aNa3E/e37/ACAqKgrBwcEW75tGo8GuXbvM71tSUhLKysqQlZVl3mfLli0wGAzmcNedmYLN8ePHsXnzZvTq1avVY7KzsyGXyxt15diLs2fP4sKFC+a/l/b+Hja0evVqxMXFITY2ttV9u9P72NrnQ1v+D01KSsIff/xhEVRNYT06OtpmhVIHrV27VqhUKrFmzRpx+PBh8cgjjwhfX1+LkeD25LHHHhM+Pj5i69atorCw0PyoqqoSQghx4sQJsWjRIrF3716Rm5srvvnmG9G3b19x4403Slx52zz99NNi69atIjc3V+zYsUMkJycLf39/UVJSIoQQ4tFHHxV9+vQRW7ZsEXv37hVJSUkiKSlJ4qrbT6/Xiz59+oh58+ZZbLfX96+8vFzs379f7N+/XwAQy5YtE/v37zfPFnr11VeFr6+v+Oabb8Tvv/8u7rjjDhEVFSWqq6vN5xgzZowYNmyY2LVrl9i+fbsYMGCAmDRpklSXZKGl69NqteL2228XvXv3FtnZ2Rb/Lk2zS3bu3CneeOMNkZ2dLU6ePCk+/vhjERAQIKZMmSLxlV3W0jWWl5eLZ555RmRmZorc3FyxefNmMXz4cDFgwABRU1NjPkd3fg+FaP3vqRBCqNVq4eHhId59991Gx3f397G1zwchWv8/tK6uTgwePFiMHj1aZGdni40bN4qAgAAxf/58m9XJcGMjb731lujTp49QKpUiISFB/Pbbb1KXZDUATT4++OADIYQQeXl54sYbbxR+fn5CpVKJ/v37i2effVao1WppC2+jiRMnipCQEKFUKkVYWJiYOHGiOHHihPn56upq8fjjj4uePXsKDw8PMWHCBFFYWChhxdb58ccfBQCRk5Njsd1e37+ff/65yb+XU6dOFUIYp4MvWLBABAUFCZVKJW655ZZG137hwgUxadIk4enpKby9vcX06dNFeXm5BFfTWEvXl5ub2+y/y59//lkIIURWVpZITEwUPj4+ws3NTQwaNEgsXrzYIhhIraVrrKqqEqNHjxYBAQHC1dVVREREiJkzZzb6JbE7v4dCtP73VAgh3nvvPeHu7i7KysoaHd/d38fWPh+EaNv/oadPnxZjx44V7u7uwt/fXzz99NNCp9PZrE5ZfbFEREREDoFjboiIiMihMNwQERGRQ2G4ISIiIofCcENEREQOheGGiIiIHArDDRERETkUhhsiIiJyKAw3RERE5FAYbojI6WzduhUymazRzf2IyDEw3BAREZFDYbghIiIih8JwQ0RdzmAwIC0tDVFRUXB3d0dsbCy++OILAJe7jDZs2IAhQ4bAzc0Nf/rTn3Dw4EGLc3z55Ze45pproFKpEBkZiaVLl1o8X1tbi3nz5iE8PBwqlQr9+/fH6tWrLfbJyspCfHw8PDw8cN111yEnJ8f83IEDBzBq1Ch4eXnB29sbcXFx2Lt3byf9iRCRLTHcEFGXS0tLw0cffYRVq1bh0KFDmDt3Lv785z9j27Zt5n2effZZLF26FHv27EFAQADGjx8PnU4HwBhK7rvvPtx///34448/8PLLL2PBggVYs2aN+fgpU6bg008/xZtvvokjR47gvffeg6enp0UdL7zwApYuXYq9e/fCxcUFDz30kPm5yZMno3fv3tizZw+ysrLw/PPPw9XVtXP/YIjINmx2f3EiojaoqakRHh4eYufOnRbbH374YTFp0iTx888/CwBi7dq15ucuXLgg3N3dxbp164QQQjzwwAPi1ltvtTj+2WefFdHR0UIIIXJycgQAsWnTpiZrML3G5s2bzds2bNggAIjq6mohhBBeXl5izZo1Hb9gIupybLkhoi514sQJVFVV4dZbb4Wnp6f58dFHH+HkyZPm/ZKSkszf+/n5YeDAgThy5AgA4MiRIxgxYoTFeUeMGIHjx49Dr9cjOzsbCoUCN910U4u1DBkyxPx9SEgIAKCkpAQAkJqaihkzZiA5ORmvvvqqRW1E1L0x3BBRl6qoqAAAbNiwAdnZ2ebH4cOHzeNuOsrd3b1N+zXsZpLJZACM44EA4OWXX8ahQ4cwbtw4bNmyBdHR0fjqq69sUh8RdS6GGyLqUtHR0VCpVMjLy0P//v0tHuHh4eb9fvvtN/P3ly5dwrFjxzBo0CAAwKBBg7Bjxw6L8+7YsQNXXXUVFAoFYmJiYDAYLMbwWOOqq67C3Llz8dNPP+Guu+7CBx980KHzEVHXcJG6ACJyLl5eXnjmmWcwd+5cGAwGXH/99VCr1dixYwe8vb0REREBAFi0aBF69eqFoKAgvPDCC/D398edd94JAHj66adx7bXX4m9/+xsmTpyIzMxMvP3223jnnXcAAJGRkZg6dSoeeughvPnmm4iNjcWZM2dQUlKC++67r9Uaq6ur8eyzz+Kee+5BVFQUzp49iz179uDuu+/utD8XIrIhqQf9EJHzMRgMYvny5WLgwIHC1dVVBAQEiJSUFLFt2zbzYN///e9/4pprrhFKpVIkJCSIAwcOWJzjiy++ENHR0cLV1VX06dNHLFmyxOL56upqMXfuXBESEiKUSqXo37+/SE9PF0JcHlB86dIl8/779+8XAERubq6ora0V999/vwgPDxdKpVKEhoaK2bNnmwcbE1H3JhNCCInzFRGR2datWzFq1ChcunQJvr6+UpdDRHaIY26IiIjIoTDcEBERkUNhtxQRERE5FLbcEBERkUNhuCEiIiKHwnBDREREDoXhhoiIiBwKww0RERE5FIYbIiIicigMN0RERORQGG6IiIjIofw/v9iErc86mF0AAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Quadric Layers\n",
        "We classify the IRIS dataset now with a hybrid model consisting of 4 inputs, a 4-unit quadric hidden layer and a 3-unit linear hiddeb layer and achieve similar accuracy."
      ],
      "metadata": {
        "id": "QxifrPS6hp1N"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "class Multiclass(nn.Module):\n",
        "    def __init__(self):\n",
        "        super().__init__()\n",
        "        self.hidden = Quadric(4, 3)\n",
        "        self.act = nn.ReLU()\n",
        "        self.output = nn.Linear(3, 3)\n",
        "\n",
        "    def forward(self, x):\n",
        "        x = self.hidden(x)\n",
        "        x = self.act(x)\n",
        "        x = self.output(x)\n",
        "        return x\n",
        "\n",
        "# loss metric and optimizer\n",
        "model = Multiclass()\n",
        "loss_fn = nn.CrossEntropyLoss()\n",
        "optimizer = optim.Adam(model.parameters(), lr=0.001)\n",
        "\n",
        "# prepare model and training parameters\n",
        "n_epochs = 200\n",
        "batch_size = 5\n",
        "batches_per_epoch = len(X_train) // batch_size\n",
        "\n",
        "best_acc = - np.inf   # init to negative infinity\n",
        "best_weights = None\n",
        "train_loss_hist = []\n",
        "train_acc_hist = []\n",
        "test_loss_hist = []\n",
        "test_acc_hist = []\n",
        "\n",
        "# training loop\n",
        "for epoch in range(n_epochs):\n",
        "    epoch_loss = []\n",
        "    epoch_acc = []\n",
        "    # set model in training mode and run through each batch\n",
        "    model.train()\n",
        "    with tqdm.trange(batches_per_epoch, unit=\"batch\", mininterval=0) as bar:\n",
        "        bar.set_description(f\"Epoch {epoch}\")\n",
        "        for i in bar:\n",
        "            # take a batch\n",
        "            start = i * batch_size\n",
        "            X_batch = X_train[start:start+batch_size]\n",
        "            y_batch = y_train[start:start+batch_size]\n",
        "            # forward pass\n",
        "            y_pred = model(X_batch)\n",
        "            loss = loss_fn(y_pred, y_batch)\n",
        "            # backward pass\n",
        "            optimizer.zero_grad()\n",
        "            loss.backward()\n",
        "            # update weights\n",
        "            optimizer.step()\n",
        "            # compute and store metrics\n",
        "            acc = (torch.argmax(y_pred, 1) == torch.argmax(y_batch, 1)).float().mean()\n",
        "            epoch_loss.append(float(loss))\n",
        "            epoch_acc.append(float(acc))\n",
        "            bar.set_postfix(\n",
        "                loss=float(loss),\n",
        "                acc=float(acc)\n",
        "            )\n",
        "    # set model in evaluation mode and run through the test set\n",
        "    model.eval()\n",
        "    y_pred = model(X_test)\n",
        "    ce = loss_fn(y_pred, y_test)\n",
        "    acc = (torch.argmax(y_pred, 1) == torch.argmax(y_test, 1)).float().mean()\n",
        "    ce = float(ce)\n",
        "    acc = float(acc)\n",
        "    train_loss_hist.append(np.mean(epoch_loss))\n",
        "    train_acc_hist.append(np.mean(epoch_acc))\n",
        "    test_loss_hist.append(ce)\n",
        "    test_acc_hist.append(acc)\n",
        "    if acc > best_acc:\n",
        "        best_acc = acc\n",
        "        best_weights = copy.deepcopy(model.state_dict())\n",
        "    print(f\"Epoch {epoch} validation: Cross-entropy={ce:.2f}, Accuracy={acc*100:.1f}%\")\n",
        "\n",
        "# Restore best model\n",
        "model.load_state_dict(best_weights)\n",
        "\n",
        "# Plot the loss and accuracy\n",
        "plt.plot(train_loss_hist, label=\"train\")\n",
        "plt.plot(test_loss_hist, label=\"test\")\n",
        "plt.xlabel(\"epochs\")\n",
        "plt.ylabel(\"cross entropy\")\n",
        "plt.legend()\n",
        "plt.show()\n",
        "\n",
        "plt.plot(train_acc_hist, label=\"train\")\n",
        "plt.plot(test_acc_hist, label=\"test\")\n",
        "plt.xlabel(\"epochs\")\n",
        "plt.ylabel(\"accuracy\")\n",
        "plt.legend()\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "K5EU9glXh3ui",
        "outputId": "8440fdae-7dcd-4282-d273-5e92badbd8ee"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 0: 100%|██████████| 21/21 [00:00<00:00, 124.40batch/s, acc=0.4, loss=7.99]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 0 validation: Cross-entropy=7.88, Accuracy=35.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 1: 100%|██████████| 21/21 [00:00<00:00, 147.90batch/s, acc=0.4, loss=6.13]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1 validation: Cross-entropy=5.92, Accuracy=35.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 2: 100%|██████████| 21/21 [00:00<00:00, 159.03batch/s, acc=0.4, loss=4.48]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 2 validation: Cross-entropy=4.19, Accuracy=35.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 3: 100%|██████████| 21/21 [00:00<00:00, 160.79batch/s, acc=0, loss=3.24]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 3 validation: Cross-entropy=2.94, Accuracy=0.0%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 4: 100%|██████████| 21/21 [00:00<00:00, 208.06batch/s, acc=0, loss=2.34]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 4 validation: Cross-entropy=2.10, Accuracy=24.4%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 5: 100%|██████████| 21/21 [00:00<00:00, 197.49batch/s, acc=0.2, loss=1.65]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 5 validation: Cross-entropy=1.52, Accuracy=28.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 6: 100%|██████████| 21/21 [00:00<00:00, 188.33batch/s, acc=0.4, loss=1.37]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 6 validation: Cross-entropy=1.31, Accuracy=40.0%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 7: 100%|██████████| 21/21 [00:00<00:00, 197.25batch/s, acc=0.2, loss=1.28]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 7 validation: Cross-entropy=1.24, Accuracy=40.0%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 8: 100%|██████████| 21/21 [00:00<00:00, 206.20batch/s, acc=0.2, loss=1.22]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 8 validation: Cross-entropy=1.18, Accuracy=44.4%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 9: 100%|██████████| 21/21 [00:00<00:00, 206.70batch/s, acc=0.2, loss=1.16]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 9 validation: Cross-entropy=1.12, Accuracy=48.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 10: 100%|██████████| 21/21 [00:00<00:00, 177.50batch/s, acc=0.4, loss=1.1]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 10 validation: Cross-entropy=1.07, Accuracy=51.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 11: 100%|██████████| 21/21 [00:00<00:00, 193.38batch/s, acc=0.4, loss=1.05]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 11 validation: Cross-entropy=1.02, Accuracy=51.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 12: 100%|██████████| 21/21 [00:00<00:00, 76.71batch/s, acc=0.4, loss=1]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 12 validation: Cross-entropy=0.98, Accuracy=51.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 13: 100%|██████████| 21/21 [00:00<00:00, 179.42batch/s, acc=0.4, loss=0.96]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 13 validation: Cross-entropy=0.94, Accuracy=53.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 14: 100%|██████████| 21/21 [00:00<00:00, 201.67batch/s, acc=0.4, loss=0.922]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 14 validation: Cross-entropy=0.90, Accuracy=60.0%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 15: 100%|██████████| 21/21 [00:00<00:00, 200.60batch/s, acc=0.4, loss=0.888]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 15 validation: Cross-entropy=0.87, Accuracy=64.4%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 16: 100%|██████████| 21/21 [00:00<00:00, 210.28batch/s, acc=0.4, loss=0.856]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 16 validation: Cross-entropy=0.84, Accuracy=68.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 17: 100%|██████████| 21/21 [00:00<00:00, 195.73batch/s, acc=0.4, loss=0.827]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 17 validation: Cross-entropy=0.82, Accuracy=75.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 18: 100%|██████████| 21/21 [00:00<00:00, 206.16batch/s, acc=0.6, loss=0.8]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 18 validation: Cross-entropy=0.79, Accuracy=82.2%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 19: 100%|██████████| 21/21 [00:00<00:00, 168.87batch/s, acc=0.8, loss=0.775]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 19 validation: Cross-entropy=0.77, Accuracy=86.7%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 20: 100%|██████████| 21/21 [00:00<00:00, 147.51batch/s, acc=0.8, loss=0.752]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 20 validation: Cross-entropy=0.75, Accuracy=86.7%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 21: 100%|██████████| 21/21 [00:00<00:00, 198.98batch/s, acc=0.8, loss=0.73]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 21 validation: Cross-entropy=0.73, Accuracy=88.9%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 22: 100%|██████████| 21/21 [00:00<00:00, 205.27batch/s, acc=0.8, loss=0.71]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 22 validation: Cross-entropy=0.71, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 23: 100%|██████████| 21/21 [00:00<00:00, 94.60batch/s, acc=0.8, loss=0.69] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 23 validation: Cross-entropy=0.69, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 24: 100%|██████████| 21/21 [00:00<00:00, 195.42batch/s, acc=0.8, loss=0.672]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 24 validation: Cross-entropy=0.67, Accuracy=91.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 25: 100%|██████████| 21/21 [00:00<00:00, 211.69batch/s, acc=0.8, loss=0.655]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 25 validation: Cross-entropy=0.66, Accuracy=91.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 26: 100%|██████████| 21/21 [00:00<00:00, 189.89batch/s, acc=0.8, loss=0.638]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 26 validation: Cross-entropy=0.64, Accuracy=91.1%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 27: 100%|██████████| 21/21 [00:00<00:00, 208.99batch/s, acc=0.8, loss=0.622]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 27 validation: Cross-entropy=0.63, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 28: 100%|██████████| 21/21 [00:00<00:00, 81.01batch/s, acc=0.8, loss=0.607] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 28 validation: Cross-entropy=0.61, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 29: 100%|██████████| 21/21 [00:00<00:00, 205.52batch/s, acc=0.8, loss=0.592]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 29 validation: Cross-entropy=0.60, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 30: 100%|██████████| 21/21 [00:00<00:00, 201.03batch/s, acc=0.8, loss=0.578]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 30 validation: Cross-entropy=0.58, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 31: 100%|██████████| 21/21 [00:00<00:00, 191.51batch/s, acc=0.8, loss=0.564]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 31 validation: Cross-entropy=0.57, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 32: 100%|██████████| 21/21 [00:00<00:00, 193.02batch/s, acc=1, loss=0.551]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 32 validation: Cross-entropy=0.56, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 33: 100%|██████████| 21/21 [00:00<00:00, 75.60batch/s, acc=1, loss=0.538]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 33 validation: Cross-entropy=0.55, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 34: 100%|██████████| 21/21 [00:00<00:00, 207.11batch/s, acc=1, loss=0.525]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 34 validation: Cross-entropy=0.53, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 35: 100%|██████████| 21/21 [00:00<00:00, 177.52batch/s, acc=1, loss=0.513]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 35 validation: Cross-entropy=0.52, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 36: 100%|██████████| 21/21 [00:00<00:00, 200.83batch/s, acc=1, loss=0.502]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 36 validation: Cross-entropy=0.51, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 37: 100%|██████████| 21/21 [00:00<00:00, 200.52batch/s, acc=1, loss=0.49]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 37 validation: Cross-entropy=0.50, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 38: 100%|██████████| 21/21 [00:00<00:00, 195.88batch/s, acc=1, loss=0.479]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 38 validation: Cross-entropy=0.49, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 39: 100%|██████████| 21/21 [00:00<00:00, 76.04batch/s, acc=1, loss=0.469] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 39 validation: Cross-entropy=0.48, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 40: 100%|██████████| 21/21 [00:00<00:00, 210.62batch/s, acc=1, loss=0.458]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 40 validation: Cross-entropy=0.47, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 41: 100%|██████████| 21/21 [00:00<00:00, 188.05batch/s, acc=1, loss=0.448]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 41 validation: Cross-entropy=0.46, Accuracy=93.3%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 42: 100%|██████████| 21/21 [00:00<00:00, 186.46batch/s, acc=1, loss=0.439]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 42 validation: Cross-entropy=0.45, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 43: 100%|██████████| 21/21 [00:00<00:00, 171.36batch/s, acc=1, loss=0.429]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 43 validation: Cross-entropy=0.44, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 44: 100%|██████████| 21/21 [00:00<00:00, 78.12batch/s, acc=1, loss=0.42] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 44 validation: Cross-entropy=0.43, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 45: 100%|██████████| 21/21 [00:00<00:00, 198.47batch/s, acc=1, loss=0.411]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 45 validation: Cross-entropy=0.42, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 46: 100%|██████████| 21/21 [00:00<00:00, 187.66batch/s, acc=1, loss=0.402]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 46 validation: Cross-entropy=0.41, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 47: 100%|██████████| 21/21 [00:00<00:00, 206.27batch/s, acc=1, loss=0.394]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 47 validation: Cross-entropy=0.41, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 48: 100%|██████████| 21/21 [00:00<00:00, 209.82batch/s, acc=1, loss=0.386]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 48 validation: Cross-entropy=0.40, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 49: 100%|██████████| 21/21 [00:00<00:00, 79.37batch/s, acc=1, loss=0.378] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 49 validation: Cross-entropy=0.39, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 50: 100%|██████████| 21/21 [00:00<00:00, 205.07batch/s, acc=1, loss=0.37]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 50 validation: Cross-entropy=0.38, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 51: 100%|██████████| 21/21 [00:00<00:00, 212.38batch/s, acc=1, loss=0.363]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 51 validation: Cross-entropy=0.38, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 52: 100%|██████████| 21/21 [00:00<00:00, 204.28batch/s, acc=1, loss=0.356]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 52 validation: Cross-entropy=0.37, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 53: 100%|██████████| 21/21 [00:00<00:00, 200.48batch/s, acc=1, loss=0.349]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 53 validation: Cross-entropy=0.36, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 54: 100%|██████████| 21/21 [00:00<00:00, 77.36batch/s, acc=1, loss=0.342]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 54 validation: Cross-entropy=0.36, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 55: 100%|██████████| 21/21 [00:00<00:00, 53.33batch/s, acc=1, loss=0.335] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 55 validation: Cross-entropy=0.35, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 56: 100%|██████████| 21/21 [00:00<00:00, 156.60batch/s, acc=1, loss=0.329]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 56 validation: Cross-entropy=0.34, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 57: 100%|██████████| 21/21 [00:00<00:00, 168.81batch/s, acc=1, loss=0.322]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 57 validation: Cross-entropy=0.34, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 58: 100%|██████████| 21/21 [00:00<00:00, 82.50batch/s, acc=1, loss=0.316]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 58 validation: Cross-entropy=0.33, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 59: 100%|██████████| 21/21 [00:00<00:00, 144.77batch/s, acc=1, loss=0.31]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 59 validation: Cross-entropy=0.33, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 60: 100%|██████████| 21/21 [00:00<00:00, 135.39batch/s, acc=1, loss=0.305]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 60 validation: Cross-entropy=0.32, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 61: 100%|██████████| 21/21 [00:00<00:00, 137.18batch/s, acc=1, loss=0.299]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 61 validation: Cross-entropy=0.32, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 62: 100%|██████████| 21/21 [00:00<00:00, 122.20batch/s, acc=1, loss=0.293]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 62 validation: Cross-entropy=0.31, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 63: 100%|██████████| 21/21 [00:00<00:00, 137.68batch/s, acc=1, loss=0.288]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 63 validation: Cross-entropy=0.31, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 64: 100%|██████████| 21/21 [00:00<00:00, 131.07batch/s, acc=1, loss=0.283]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 64 validation: Cross-entropy=0.30, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 65: 100%|██████████| 21/21 [00:00<00:00, 133.30batch/s, acc=1, loss=0.278]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 65 validation: Cross-entropy=0.30, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 66: 100%|██████████| 21/21 [00:00<00:00, 157.14batch/s, acc=1, loss=0.273]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 66 validation: Cross-entropy=0.29, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 67: 100%|██████████| 21/21 [00:00<00:00, 151.65batch/s, acc=1, loss=0.268]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 67 validation: Cross-entropy=0.29, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 68: 100%|██████████| 21/21 [00:00<00:00, 147.39batch/s, acc=1, loss=0.263]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 68 validation: Cross-entropy=0.28, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 69: 100%|██████████| 21/21 [00:00<00:00, 146.59batch/s, acc=1, loss=0.259]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 69 validation: Cross-entropy=0.28, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 70: 100%|██████████| 21/21 [00:00<00:00, 158.81batch/s, acc=1, loss=0.254]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 70 validation: Cross-entropy=0.28, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 71: 100%|██████████| 21/21 [00:00<00:00, 220.09batch/s, acc=1, loss=0.25]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 71 validation: Cross-entropy=0.27, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 72: 100%|██████████| 21/21 [00:00<00:00, 225.56batch/s, acc=1, loss=0.245]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 72 validation: Cross-entropy=0.27, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 73: 100%|██████████| 21/21 [00:00<00:00, 229.66batch/s, acc=1, loss=0.241]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 73 validation: Cross-entropy=0.26, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 74: 100%|██████████| 21/21 [00:00<00:00, 225.58batch/s, acc=1, loss=0.237]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 74 validation: Cross-entropy=0.26, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 75: 100%|██████████| 21/21 [00:00<00:00, 202.01batch/s, acc=1, loss=0.233]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 75 validation: Cross-entropy=0.26, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 76: 100%|██████████| 21/21 [00:00<00:00, 173.73batch/s, acc=1, loss=0.229]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 76 validation: Cross-entropy=0.25, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 77: 100%|██████████| 21/21 [00:00<00:00, 211.51batch/s, acc=1, loss=0.225]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 77 validation: Cross-entropy=0.25, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 78: 100%|██████████| 21/21 [00:00<00:00, 226.31batch/s, acc=1, loss=0.222]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 78 validation: Cross-entropy=0.25, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 79: 100%|██████████| 21/21 [00:00<00:00, 212.44batch/s, acc=1, loss=0.218]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 79 validation: Cross-entropy=0.25, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 80: 100%|██████████| 21/21 [00:00<00:00, 218.45batch/s, acc=1, loss=0.214]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 80 validation: Cross-entropy=0.24, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 81: 100%|██████████| 21/21 [00:00<00:00, 102.20batch/s, acc=1, loss=0.211]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 81 validation: Cross-entropy=0.24, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 82: 100%|██████████| 21/21 [00:00<00:00, 209.11batch/s, acc=1, loss=0.207]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 82 validation: Cross-entropy=0.24, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 83: 100%|██████████| 21/21 [00:00<00:00, 221.80batch/s, acc=1, loss=0.204]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 83 validation: Cross-entropy=0.23, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 84: 100%|██████████| 21/21 [00:00<00:00, 226.06batch/s, acc=1, loss=0.201]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 84 validation: Cross-entropy=0.23, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 85: 100%|██████████| 21/21 [00:00<00:00, 204.89batch/s, acc=1, loss=0.197]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 85 validation: Cross-entropy=0.23, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 86: 100%|██████████| 21/21 [00:00<00:00, 85.46batch/s, acc=1, loss=0.194] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 86 validation: Cross-entropy=0.23, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 87: 100%|██████████| 21/21 [00:00<00:00, 207.15batch/s, acc=1, loss=0.191]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 87 validation: Cross-entropy=0.22, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 88: 100%|██████████| 21/21 [00:00<00:00, 223.77batch/s, acc=1, loss=0.188]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 88 validation: Cross-entropy=0.22, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 89: 100%|██████████| 21/21 [00:00<00:00, 222.72batch/s, acc=1, loss=0.185]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 89 validation: Cross-entropy=0.22, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 90: 100%|██████████| 21/21 [00:00<00:00, 206.25batch/s, acc=1, loss=0.182]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 90 validation: Cross-entropy=0.22, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 91: 100%|██████████| 21/21 [00:00<00:00, 86.98batch/s, acc=1, loss=0.179]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 91 validation: Cross-entropy=0.21, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 92: 100%|██████████| 21/21 [00:00<00:00, 51.75batch/s, acc=1, loss=0.176]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 92 validation: Cross-entropy=0.21, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 93: 100%|██████████| 21/21 [00:00<00:00, 153.65batch/s, acc=1, loss=0.173]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 93 validation: Cross-entropy=0.21, Accuracy=95.6%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 104: 100%|██████████| 21/21 [00:00<00:00, 119.97batch/s, acc=1, loss=0.146]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 104 validation: Cross-entropy=0.19, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 105: 100%|██████████| 21/21 [00:00<00:00, 183.61batch/s, acc=1, loss=0.144]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 105 validation: Cross-entropy=0.19, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 106: 100%|██████████| 21/21 [00:00<00:00, 195.97batch/s, acc=1, loss=0.141]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 106 validation: Cross-entropy=0.19, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 107: 100%|██████████| 21/21 [00:00<00:00, 196.43batch/s, acc=1, loss=0.139]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 107 validation: Cross-entropy=0.19, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 108: 100%|██████████| 21/21 [00:00<00:00, 188.73batch/s, acc=1, loss=0.137]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 108 validation: Cross-entropy=0.19, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 109: 100%|██████████| 21/21 [00:00<00:00, 194.25batch/s, acc=1, loss=0.135]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 109 validation: Cross-entropy=0.18, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 110: 100%|██████████| 21/21 [00:00<00:00, 183.42batch/s, acc=1, loss=0.133]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 110 validation: Cross-entropy=0.18, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 111: 100%|██████████| 21/21 [00:00<00:00, 189.57batch/s, acc=1, loss=0.131]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 111 validation: Cross-entropy=0.18, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 112: 100%|██████████| 21/21 [00:00<00:00, 109.37batch/s, acc=1, loss=0.129]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 112 validation: Cross-entropy=0.18, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 113: 100%|██████████| 21/21 [00:00<00:00, 188.23batch/s, acc=1, loss=0.127]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 113 validation: Cross-entropy=0.18, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 114: 100%|██████████| 21/21 [00:00<00:00, 180.09batch/s, acc=1, loss=0.126]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 114 validation: Cross-entropy=0.18, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 123: 100%|██████████| 21/21 [00:00<00:00, 85.56batch/s, acc=1, loss=0.11] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 123 validation: Cross-entropy=0.17, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 124: 100%|██████████| 21/21 [00:00<00:00, 191.28batch/s, acc=1, loss=0.109]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 124 validation: Cross-entropy=0.17, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 125: 100%|██████████| 21/21 [00:00<00:00, 196.45batch/s, acc=1, loss=0.107]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 125 validation: Cross-entropy=0.17, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 126: 100%|██████████| 21/21 [00:00<00:00, 189.91batch/s, acc=1, loss=0.105]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 126 validation: Cross-entropy=0.17, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 127: 100%|██████████| 21/21 [00:00<00:00, 188.90batch/s, acc=1, loss=0.104]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 127 validation: Cross-entropy=0.16, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 128: 100%|██████████| 21/21 [00:00<00:00, 85.34batch/s, acc=1, loss=0.103] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 128 validation: Cross-entropy=0.16, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 129: 100%|██████████| 21/21 [00:00<00:00, 220.75batch/s, acc=1, loss=0.101]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 129 validation: Cross-entropy=0.16, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 130: 100%|██████████| 21/21 [00:00<00:00, 194.42batch/s, acc=1, loss=0.0997]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 130 validation: Cross-entropy=0.16, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 131: 100%|██████████| 21/21 [00:00<00:00, 208.77batch/s, acc=1, loss=0.0984]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 131 validation: Cross-entropy=0.16, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 132: 100%|██████████| 21/21 [00:00<00:00, 209.29batch/s, acc=1, loss=0.097]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 132 validation: Cross-entropy=0.16, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 144: 100%|██████████| 21/21 [00:00<00:00, 93.36batch/s, acc=1, loss=0.0827] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 144 validation: Cross-entropy=0.15, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 145: 100%|██████████| 21/21 [00:00<00:00, 206.83batch/s, acc=1, loss=0.0816]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 145 validation: Cross-entropy=0.15, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 146: 100%|██████████| 21/21 [00:00<00:00, 194.27batch/s, acc=1, loss=0.0806]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 146 validation: Cross-entropy=0.15, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 147: 100%|██████████| 21/21 [00:00<00:00, 208.71batch/s, acc=1, loss=0.0796]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 147 validation: Cross-entropy=0.15, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 148: 100%|██████████| 21/21 [00:00<00:00, 194.58batch/s, acc=1, loss=0.0786]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 148 validation: Cross-entropy=0.15, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 149: 100%|██████████| 21/21 [00:00<00:00, 87.69batch/s, acc=1, loss=0.0776] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 149 validation: Cross-entropy=0.15, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 150: 100%|██████████| 21/21 [00:00<00:00, 199.03batch/s, acc=1, loss=0.0766]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 150 validation: Cross-entropy=0.15, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 151: 100%|██████████| 21/21 [00:00<00:00, 203.04batch/s, acc=1, loss=0.0756]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 151 validation: Cross-entropy=0.15, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 152: 100%|██████████| 21/21 [00:00<00:00, 211.15batch/s, acc=1, loss=0.0747]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 152 validation: Cross-entropy=0.15, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 153: 100%|██████████| 21/21 [00:00<00:00, 194.24batch/s, acc=1, loss=0.0738]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 153 validation: Cross-entropy=0.15, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 165: 100%|██████████| 21/21 [00:00<00:00, 98.75batch/s, acc=1, loss=0.0637] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 165 validation: Cross-entropy=0.14, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 166: 100%|██████████| 21/21 [00:00<00:00, 188.60batch/s, acc=1, loss=0.063]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 166 validation: Cross-entropy=0.14, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 167: 100%|██████████| 21/21 [00:00<00:00, 202.36batch/s, acc=1, loss=0.0623]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 167 validation: Cross-entropy=0.14, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 168: 100%|██████████| 21/21 [00:00<00:00, 189.18batch/s, acc=1, loss=0.0615]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 168 validation: Cross-entropy=0.14, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 169: 100%|██████████| 21/21 [00:00<00:00, 192.88batch/s, acc=1, loss=0.0608]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 169 validation: Cross-entropy=0.14, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 170: 100%|██████████| 21/21 [00:00<00:00, 73.38batch/s, acc=1, loss=0.0601] \n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 170 validation: Cross-entropy=0.14, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 171: 100%|██████████| 21/21 [00:00<00:00, 202.21batch/s, acc=1, loss=0.0594]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 171 validation: Cross-entropy=0.14, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 172: 100%|██████████| 21/21 [00:00<00:00, 181.92batch/s, acc=1, loss=0.0588]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 172 validation: Cross-entropy=0.14, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 173: 100%|██████████| 21/21 [00:00<00:00, 211.63batch/s, acc=1, loss=0.0581]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 173 validation: Cross-entropy=0.14, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 174: 100%|██████████| 21/21 [00:00<00:00, 190.56batch/s, acc=1, loss=0.0574]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 174 validation: Cross-entropy=0.14, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 175: 100%|██████████| 21/21 [00:00<00:00, 109.77batch/s, acc=1, loss=0.0568]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 175 validation: Cross-entropy=0.14, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Epoch 176:  19%|█▉        | 4/21 [00:00<00:00, 242.94batch/s, acc=1, loss=0.164]\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 199 validation: Cross-entropy=0.13, Accuracy=97.8%\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjIAAAGwCAYAAACzXI8XAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABI9klEQVR4nO3deXxU9b3/8feZmcxkX4CQRcIii4osFVSKtNVWClKLWysVaQWx2la8CqgXqRdUrOJSFZde6aKiv962tlZr1WrFBayICCiuCIhsCmHPnkxm+f7+mJOBmACZzJYJr+fjcR6ZnHPm5HNygHnz/X7P91jGGCMAAIAU5Eh2AQAAAO1FkAEAACmLIAMAAFIWQQYAAKQsggwAAEhZBBkAAJCyCDIAACBluZJdQLwFg0Ft375dOTk5siwr2eUAAIA2MMaourpapaWlcjgO3e7S6YPM9u3bVVZWluwyAABAO2zbtk09evQ45PZOH2RycnIkhX4Rubm5Sa4GAAC0RVVVlcrKysKf44fS6YNMU3dSbm4uQQYAgBRzpGEhDPYFAAApiyADAABSFkEGAACkrE4/RgYAgHgJBALy+XzJLiMlpaWlyel0Rn0cggwAABEyxqi8vFwVFRXJLiWl5efnq7i4OKp53ggyAABEqCnEdO/eXZmZmUy4GiFjjOrq6rRr1y5JUklJSbuPRZABACACgUAgHGK6du2a7HJSVkZGhiRp165d6t69e7u7mRjsCwBABJrGxGRmZia5ktTX9DuMZpxRUoPMG2+8ofHjx6u0tFSWZekf//hHs+3GGM2dO1clJSXKyMjQ6NGjtWHDhuQUCwDAQehOil4sfodJDTK1tbUaOnSofvOb37S6/a677tIDDzyghQsXasWKFcrKytLYsWPV0NCQ4EoBAEBHlNQxMuPGjdO4ceNa3WaM0YIFC/Q///M/OvfccyVJTzzxhIqKivSPf/xDF110USJLBQAAHVCHHSOzadMmlZeXa/To0eF1eXl5GjFihJYvX37I93m9XlVVVTVbAABAbPXu3VsLFixIdhkdN8iUl5dLkoqKipqtLyoqCm9rzfz585WXlxdeysrK4lJfZb1P2/bVqbKOiZAAAKnhjDPO0PTp02NyrJUrV+qKK66IybGi0WGDTHvNnj1blZWV4WXbtm1x+Tm3v7BW37zrdf1xxZa4HB8AgEQzxsjv97dp38LCwg5x51aHDTLFxcWSpJ07dzZbv3PnzvC21ng8HuXm5jZb4sHtCv3qvP5gXI4PAEgdxhjVNfqTshhj2lTjlClTtHTpUt1///2yLEuWZWnRokWyLEsvvviihg8fLo/HozfffFMbN27Uueeeq6KiImVnZ+uUU07RK6+80ux4X+1asixLf/jDH3T++ecrMzNT/fv31z//+c9Y/ppb1WEnxOvTp4+Ki4v16quv6mtf+5okqaqqSitWrNAvfvGL5BYnyRMOMoEkVwIASLZ6X0AD5/47KT/7k3ljlek+8sf5/fffr/Xr12vQoEGaN2+eJOnjjz+WJN1www369a9/rWOPPVYFBQXatm2bvve97+m2226Tx+PRE088ofHjx2vdunXq2bPnIX/GLbfcorvuukt33323HnzwQU2aNElbtmxRly5dYnOyrUhqi0xNTY3WrFmjNWvWSAoN8F2zZo22bt0qy7I0ffp0/epXv9I///lPffjhh7rkkktUWlqq8847L5llSzrQItNIiwwAIAXk5eXJ7XYrMzNTxcXFKi4uDs+mO2/ePH33u99V37591aVLFw0dOlQ/+9nPNGjQIPXv31+33nqr+vbte8QWlilTpmjixInq16+fbr/9dtXU1Oidd96J63kltUVm1apV+va3vx3+fubMmZKkyZMna9GiRfrv//5v1dbW6oorrlBFRYW+8Y1v6KWXXlJ6enqySg7zuEIXn64lAEBGmlOfzBubtJ8drZNPPrnZ9zU1Nbr55pv1wgsvaMeOHfL7/aqvr9fWrVsPe5whQ4aEX2dlZSk3Nzf8PKV4SWqQOeOMMw7bt2dZlubNmxduAutIPGl215KPIAMARzvLstrUvdNRZWVlNfv+uuuu0+LFi/XrX/9a/fr1U0ZGhn74wx+qsbHxsMdJS0tr9r1lWQoG4/s5mbq/9SRzO+2upQBBBgCQGtxutwKBI4/tXLZsmaZMmaLzzz9fUqiFZvPmzXGurn067F1LHd2BFhkG+wIAUkPv3r21YsUKbd68WXv27Dlka0n//v319NNPa82aNXr//fd18cUXx71lpb0IMu1EiwwAINVcd911cjqdGjhwoAoLCw855uXee+9VQUGBTjvtNI0fP15jx47VsGHDElxt29C11E4ee3AVY2QAAKliwIABLR7zM2XKlBb79e7dW6+99lqzddOmTWv2/Ve7mlob81pRUdGuOiNBi0w7Nc0jQ4sMAADJQ5BpJzcT4gEAkHQEmXYKz+xL1xIAAElDkGknupYAAEg+gkw7hWf2pUUGAICkIci0k5sWGQAAko4g004Hxsgw2BcAgGQhyLRTU9cSLTIAACQPQaadmrqWfAGjQPDQD74EAADxQ5Bpp6auJUlq9NMqAwDo+M444wxNnz49ZsebMmWKzjvvvJgdrz0IMu3kJsgAAJB0BJl2cjksOazQa2b3BQB0dFOmTNHSpUt1//33y7IsWZalzZs366OPPtK4ceOUnZ2toqIi/eQnP9GePXvC73vqqac0ePBgZWRkqGvXrho9erRqa2t188036/HHH9ezzz4bPt6SJUsSfl48NLKdLMuS2+VQgy8oLy0yAHB0M0by1SXnZ6dlSpZ1xN3uv/9+rV+/XoMGDdK8efNCb01L06mnnqqf/vSnuu+++1RfX69Zs2ZpwoQJeu2117Rjxw5NnDhRd911l84//3xVV1frP//5j4wxuu6667R27VpVVVXpsccekyR16dIlrqfaGoJMFDwuJ0EGABAKMbeXJudn/3K75M464m55eXlyu93KzMxUcXGxJOlXv/qVTjrpJN1+++3h/R599FGVlZVp/fr1qqmpkd/v1wUXXKBevXpJkgYPHhzeNyMjQ16vN3y8ZCDIRCH8mAKCDAAgBb3//vt6/fXXlZ2d3WLbxo0bNWbMGJ155pkaPHiwxo4dqzFjxuiHP/yhCgoKklBt6wgyUeAJ2AAASaHunV9uT97PbqeamhqNHz9ed955Z4ttJSUlcjqdWrx4sd566y29/PLLevDBB3XjjTdqxYoV6tOnTzRVxwxBJgrh2X1pkQGAo5tltal7J9ncbrcCgQP/+R42bJj+/ve/q3fv3nK5Wo8ElmVp1KhRGjVqlObOnatevXrpmWee0cyZM1scLxm4aykK7qbZfQkyAIAU0Lt3b61YsUKbN2/Wnj17NG3aNO3bt08TJ07UypUrtXHjRv373//WpZdeqkAgoBUrVuj222/XqlWrtHXrVj399NPavXu3TjjhhPDxPvjgA61bt0579uyRz+dL+DkRZKJAiwwAIJVcd911cjqdGjhwoAoLC9XY2Khly5YpEAhozJgxGjx4sKZPn678/Hw5HA7l5ubqjTfe0Pe+9z0NGDBA//M//6N77rlH48aNkyRdfvnlOu6443TyySersLBQy5YtS/g50bUUBTeDfQEAKWTAgAFavnx5i/VPP/10q/ufcMIJeumllw55vMLCQr388ssxq689aJGJgofBvgAAJBVBJgpNT8CmawkAgOQgyESBeWQAAEgugkwU6FoCACC5CDJRYLAvABy9jDHJLiHlxeJ3SJCJArdfA8DRJy0tTZJUV5ekh0R2Ik2/w6bfaXtw+3UUaJEBgKOP0+lUfn6+du3aJUnKzMyU1YanT+MAY4zq6uq0a9cu5efny+l0tvtYBJkocNcSABydmp723BRm0D75+flRPzmbIBMFBvsCwNHJsiyVlJSoe/fuSZmWvzNIS0uLqiWmCUEmCm7GyADAUc3pdMbkwxjtx2DfKDDYFwCA5CLIRIGnXwMAkFwEmSjQIgMAQHIRZKLgSWu6/ZrBvgAAJANBJgpuJy0yAAAkE0EmCp40ex4ZH0EGAIBkIMhEoalFpjFAkAEAIBkIMlFoGiPDhHgAACQHQSYK4RYZxsgAAJAUBJkopKcx2BcAgGQiyETBw4R4AAAkFUEmCjxrCQCA5CLIRKFpZt9A0MjPnUsAACQcQSYKTS0yErdgAwCQDASZKDTdtSQxKR4AAMlAkImCy+mQ02FJokUGAIBkIMhEKfwEbFpkAABIOIJMlMJBhtl9AQBIOIJMlLgFGwCA5CHIRKlpUjyCDAAAiUeQiVJTiwyz+wIAkHgEmSgxRgYAgOQhyESJFhkAAJKnQweZQCCgOXPmqE+fPsrIyFDfvn116623yhiT7NLCPAz2BQAgaVzJLuBw7rzzTj388MN6/PHHdeKJJ2rVqlW69NJLlZeXp6uvvjrZ5UlisC8AAMnUoYPMW2+9pXPPPVdnn322JKl3797685//rHfeeSfJlR1A1xIAAMnTobuWTjvtNL366qtav369JOn999/Xm2++qXHjxh3yPV6vV1VVVc2WeGKwLwAAydOhW2RuuOEGVVVV6fjjj5fT6VQgENBtt92mSZMmHfI98+fP1y233JKwGmmRAQAgeTp0i8xf//pX/d///Z/+9Kc/6d1339Xjjz+uX//613r88ccP+Z7Zs2ersrIyvGzbti2uNTJGBgCA5OnQLTLXX3+9brjhBl100UWSpMGDB2vLli2aP3++Jk+e3Op7PB6PPB5P/Ivz1kjeKuVadZJokQEAIBk6dItMXV2dHI7mJTqdTgWDHSA0vDRLuvcEjdz7jCTGyAAAkAwdukVm/Pjxuu2229SzZ0+deOKJeu+993Tvvfdq6tSpyS5NcmVIktLVKImuJQAAkqFDB5kHH3xQc+bM0ZVXXqldu3aptLRUP/vZzzR37txklya5Qt1XbjvI0LUEAEDideggk5OTowULFmjBggXJLqWltFCLjIcWGQAAkqZDj5Hp0FzpkqQ0Q4sMAADJQpBpL7tFxm28khjsCwBAMhBk2qtpjIyhawkAgGQhyLSXfdeSK2gHGR9BBgCARCPItFdaaIyMK2h3LQUIMgAAJBpBpr1cXwkyPsbIAACQaASZ9vpKkOGuJQAAEo8g0172XUtOO8g00CIDAEDCEWTay26RcQQaJHHXEgAAyUCQaa+mIOMnyAAAkCwEmfay71qyAkyIBwBAshBk2sueR8ayW2R8AaNA0CSzIgAAjjoEmfayZ/a1TEAu+SVx5xIAAIlGkGkv+64lSfLIJ4nuJQAAEo0g0172YF9JynKEgkwDjykAACChCDLtZVnhMJPrCnUt0SIDAEBiEWSiYY+TyXWFAgy3YAMAkFgEmWjYdy5lO+0WGbqWAABIKIJMNOy5ZHLoWgIAICkIMtGwx8hkO5ruWqJFBgCARCLIRMMOMll21xIPjgQAILEIMtGw55LJcjR1LdEiAwBAIhFkomHftZTpaJTEGBkAABKNIBMN+66lTMseI8NdSwAAJBRBJhr2XUsZdC0BAJAUBJlo2C0yGRZdSwAAJANBJhr2GJkMupYAAEgKgkw07LuWmp5+3UCLDAAACUWQiYY9j0x6U9cSLTIAACQUQSYadpDxmKYxMgQZAAASiSATDfuuJbcY7AsAQDIQZKJh37XkpkUGAICkIMhEo6lFxnglMUYGAIBEI8hEwx4jk2a3yHDXEgAAiUWQiYYdZFxB7loCACAZCDLRsOeRcQXtriVaZAAASCiCTDTsmX1dwQZJDPYFACDRCDLRsO9acga5awkAgGQgyETDvmvJGWhqkaFrCQCARCLIRMNukXEEuP0aAIBkIMhEwx4j4/CHWmQafLTIAACQSASZaNh3LVnBRlkKMkYGAIAEI8hEw55HRpI88snrD8oYk8SCAAA4uhBkonFQkEm3HxzZGKBVBgCARCHIRMPpkhwuSaEWGYlbsAEASCSCTLTsO5cyLB5TAABAohFkomXfuZTr8kviziUAABKJIBMt+86lHGcoyNC1BABA4hBkomUP+M0OBxlaZAAASBSCTLTsxxRku2iRAQAg0SIOMjfddJO2bNkSj1pSk90ik+WwgwyDfQEASJiIg8yzzz6rvn376swzz9Sf/vQneb3eeNSVOsJdS023X9O1BABAokQcZNasWaOVK1fqxBNP1DXXXKPi4mL94he/0MqVK+NRX8dnD/bNdNC1BABAorVrjMxJJ52kBx54QNu3b9cjjzyiL774QqNGjdKQIUN0//33q7KyMtZ1dlz27deZVqhFhtuvAQBInKgG+xpj5PP51NjYKGOMCgoK9NBDD6msrExPPvlkrGrs2FxNLTL2hHi0yAAAkDDtCjKrV6/WVVddpZKSEs2YMUMnnXSS1q5dq6VLl2rDhg267bbbdPXVV8e61o7Jvmspw6JrCQCARIs4yAwePFhf//rXtWnTJj3yyCPatm2b7rjjDvXr1y+8z8SJE7V79+6YFtph2S0y6eFHFNC1BABAokQcZCZMmKDNmzfrhRde0HnnnSen09lin27duikYjE3LxJdffqkf//jH6tq1qzIyMjR48GCtWrUqJseOCXuMTIboWgIAINFckb5hzpw54dfGGEmSZVmxq+gg+/fv16hRo/Ttb39bL774ogoLC7VhwwYVFBTE5ee1i33XkoeuJQAAEq5dY2QeeeQRDRo0SOnp6UpPT9egQYP0hz/8Ida16c4771RZWZkee+wxnXrqqerTp4/GjBmjvn37xvxntZs9j0y6QvPp0LUEAEDiRBxk5s6dq2uuuUbjx4/X3/72N/3tb3/T+PHjNWPGDM2dOzemxf3zn//UySefrAsvvFDdu3fXSSedpN///veHfY/X61VVVVWzJa7sIOM2TRPi0SIDAECiRNy19PDDD+v3v/+9Jk6cGF53zjnnaMiQIfqv//ovzZs3L2bFff7553r44Yc1c+ZM/fKXv9TKlSt19dVXy+12a/Lkya2+Z/78+brllltiVsMR2XctucNjZGiRAQAgUSJukfH5fDr55JNbrB8+fLj8fn9MimoSDAY1bNgw3X777TrppJN0xRVX6PLLL9fChQsP+Z7Zs2ersrIyvGzbti2mNbVg37XkNk1dS7TIAACQKBEHmZ/85Cd6+OGHW6z/3e9+p0mTJsWkqCYlJSUaOHBgs3UnnHCCtm7desj3eDwe5ebmNlviym6RSTPctQQAQKJF3LUkhQb7vvzyy/r6178uSVqxYoW2bt2qSy65RDNnzgzvd++990ZV3KhRo7Ru3bpm69avX69evXpFddyYSssMfQnaLTJ0LQEAkDARB5mPPvpIw4YNkyRt3LhRUmjemG7duumjjz4K7xeLW7JnzJih0047TbfffrsmTJigd955R7/73e/0u9/9Lupjx4w92Dct2CCJFhkAABIp4iDz+uuvx6OOVp1yyil65plnNHv2bM2bN099+vTRggULYt6FFRW7RcYVCAUZHhoJAEDitKtrqckXX3whSerRo0dMimnN97//fX3/+9+P2/GjZk+I56RFBgCAhIt4sG8wGNS8efOUl5enXr16qVevXsrPz9ett94as8cSpJSmIGO3yHDXEgAAiRNxi8yNN96oRx55RHfccYdGjRolSXrzzTd18803q6GhQbfddlvMi+zQ7K4lp79ekmGwLwAACRRxkHn88cf1hz/8Qeecc0543ZAhQ3TMMcfoyiuvPAqDTKhFxjIBpSlA1xIAAAkUcdfSvn37dPzxx7dYf/zxx2vfvn0xKSql2C0ykpQhL0EGAIAEijjIDB06VA899FCL9Q899JCGDh0ak6JSijNNspySpHQ1ctcSAAAJFHHX0l133aWzzz5br7zyikaOHClJWr58ubZt26Z//etfMS+ww7OsUKtMY7XSrUbtpUUGAICEibhF5vTTT9f69et1/vnnq6KiQhUVFbrgggu0bt06ffOb34xHjR2f/ZiCDHkVCBr5AoQZAAASIaIWGZ/Pp7POOksLFy48+gb1Ho494DfDfgJ2gy+gNGfEGREAAEQook/btLQ0ffDBB/GqJXXZA34zrNDzlhqYSwYAgISIuNngxz/+sR555JF41JK67BaZPJdfEo8pAAAgUSIe7Ov3+/Xoo4/qlVde0fDhw5WVldVse7RPvE5JdotMnssnNUr1BBkAABIiqqdfr1+/PuYFpSS7RSbH6ZMk1TcSZAAASIQO/fTrlGEHmWyHHWRokQEAICEiHiMzdepUVVdXt1hfW1urqVOnxqSolGN3LeU4Q3ctEWQAAEiMiIPM448/rvr6+hbr6+vr9cQTT8SkqJRjt8hkOuzBvnQtAQCQEG3uWqqqqpIxRsYYVVdXKz09PbwtEAjoX//6l7p37x6XIjs8VyjIZFm0yAAAkEhtDjL5+fmyLEuWZWnAgAEttluWpVtuuSWmxaWMphYZggwAAAnV5iDz+uuvyxij73znO/r73/+uLl26hLe53W716tVLpaWlcSmyw7PHyISDDF1LAAAkRJuDzOmnny5J2rRpk8rKyuRwMAV/WNMjCsIz+xJkAABIhIhvv+7Vq5cqKir0zjvvaNeuXQoGm0/Hf8kll8SsuJRhB5l0Q9cSAACJFHGQee655zRp0iTV1NQoNzdXlmWFt1mWdZQGmVDXkkehFpn6Rp61BABAIkTcP3Tttddq6tSpqqmpUUVFhfbv3x9e9u3bF48aOz67RcZj7CBDiwwAAAkRcZD58ssvdfXVVyszMzMe9aQmu0XGbRokSV6CDAAACRFxkBk7dqxWrVoVj1pSV1poTp00xsgAAJBQEY+ROfvss3X99dfrk08+0eDBg5WWltZs+znnnBOz4lKG3SKTFgi1yBBkAABIjIiDzOWXXy5JmjdvXottlmUpEDgKP8TtMTKuoB1kmEcGAICEiDjIfPV2aygcZJx2iwzzyAAAkBhRzWrX0NAQqzpSm9215AzUSzJ0LQEAkCARB5lAIKBbb71VxxxzjLKzs/X5559LkubMmaNHHnkk5gWmBLtFxjJBueUnyAAAkCARB5nbbrtNixYt0l133SW32x1eP2jQIP3hD3+IaXEpI+3Arejp8jIhHgAACRJxkHniiSf0u9/9TpMmTZLT6QyvHzp0qD799NOYFpcynGmSIzTcKEONjJEBACBB2jUhXr9+/VqsDwaD8vl8MSkqJdmtMhmWV/W+gIwxSS4IAIDOL+IgM3DgQP3nP/9psf6pp57SSSedFJOiUpIrNCleunwKBI18AYIMAADxFvHt13PnztXkyZP15ZdfKhgM6umnn9a6dev0xBNP6Pnnn49HjanBHvCboQPPW3K7oropDAAAHEHEn7TnnnuunnvuOb3yyivKysrS3LlztXbtWj333HP67ne/G48aU4PdtZTlCD2mgHEyAADEX8QtMpL0zW9+U4sXL451LanNbpHJc/mlALP7AgCQCPR9xIrdIpPjDA14bvATZAAAiDeCTKzYLTK5rlCQoUUGAID4I8jEih1kchx2kGGMDAAAcUeQiRW7aymbwb4AACRM1EEmEAhozZo12r9/fyzqSV1poXlksppaZHhMAQAAcRdxkJk+fXr44ZCBQECnn366hg0bprKyMi1ZsiTW9aUOu0Umk64lAAASJuIg89RTT2no0KGSpOeee06bNm3Sp59+qhkzZujGG2+MeYEpwx4jk2mFupYIMgAAxF/EQWbPnj0qLi6WJP3rX//ShRdeqAEDBmjq1Kn68MMPY15gyggHmdDMvg3ctQQAQNxFHGSKior0ySefKBAI6KWXXgrP5ltXV9fsadhHHbtrKV20yAAAkCgRz+x76aWXasKECSopKZFlWRo9erQkacWKFTr++ONjXmDKsFtk0g961hIAAIiviIPMzTffrEGDBmnbtm268MIL5fF4JElOp1M33HBDzAtMGXaLjMfYQYauJQAA4q5dz1r64Q9/2Oz7iooKTZ48OSYFpSy7RaYpyDCPDAAA8RfxGJk777xTTz75ZPj7CRMmqGvXrurRo4c++OCDmBaXUuwWGbehawkAgESJOMgsXLhQZWVlkqTFixdr8eLFevHFF3XWWWfpuuuui3mBKcMVmhAvLdggia4lAAASIeKupfLy8nCQef755zVhwgSNGTNGvXv31ogRI2JeYMqwW2RcQVpkAABIlIhbZAoKCrRt2zZJ0ksvvRS+a8kYo0DgKP7wtsfIuOwWGa+PRxQAABBvEbfIXHDBBbr44ovVv39/7d27V+PGjZMkvffee+rXr1/MC0wZdpBxBuyuJVpkAACIu4iDzH333afevXtr27Ztuuuuu5SdnS1J2rFjh6688sqYF5gy7K4lp79ekiHIAACQABEHmbS0tFYH9c6YMSMmBaUsdyjIWDJKVyODfQEASIB2zSOzceNGLViwQGvXrpUkDRw4UNOnT9exxx4b0+JSSlpW+GW2GphHBgCABIh4sO+///1vDRw4UO+8846GDBmiIUOGaMWKFRo4cKAWL14cjxpTg8MhuXMkSdlWHV1LAAAkQMRB5oYbbtCMGTO0YsUK3Xvvvbr33nu1YsUKTZ8+XbNmzYpHjWF33HGHLMvS9OnT4/pz2s0TGi+UpQbV+wIyxiS5IAAAOreIg8zatWt12WWXtVg/depUffLJJzEpqjUrV67Ub3/7Ww0ZMiRuPyNqnlCLTI5VL2Mkr59bsAEAiKeIg0xhYaHWrFnTYv2aNWvUvXv3WNTUQk1NjSZNmqTf//73KigoiMvPiAk7yGSpXhLPWwIAIN4iHux7+eWX64orrtDnn3+u0047TZK0bNky3XnnnZo5c2bMC5SkadOm6eyzz9bo0aP1q1/96rD7er1eeb3e8PdVVVVxqalV7lDXUr7DKwVDc8nkJ+6nAwBw1Ik4yMyZM0c5OTm65557NHv2bElSaWmpbr75Zl199dUxL/Avf/mL3n33Xa1cubJN+8+fP1+33HJLzOtoE7tFpsDVIPl53hIAAPEWUZDx+/3605/+pIsvvlgzZsxQdXW1JCknJycuxW3btk3XXHONFi9erPT09Da9Z/bs2c1ahqqqqsLPhoo7T64kKc/J85YAAEiEiIKMy+XSz3/+8/D8MfEKME1Wr16tXbt2adiwYeF1gUBAb7zxhh566CF5vV45nc5m7/F4PPJ4PHGt65A8TV1LoccUMEYGAID4irhr6dRTT9V7772nXr16xaOeZs4880x9+OGHzdZdeumlOv744zVr1qwWISbpwnct2c9bauSuJQAA4iniIHPllVfq2muv1RdffKHhw4crKyur2fZY3h6dk5OjQYMGNVuXlZWlrl27tljfIdiDfXMdobuWahv9yawGAIBOL+Igc9FFF0lSs4G9lmXJGCPLshQIHMXdKV9pkaluIMgAABBPEQeZTZs2xaOONluyZElSf/5h2YN9s8NBxpfMagAA6PQiDjKJGBuTssKPKKiTJNXQIgMAQFxFPLPv/Pnz9eijj7ZY/+ijj+rOO++MSVEpy+5ayjChMTLVXoIMAADxFHGQ+e1vf6vjjz++xfoTTzxRCxcujElRKcse7JseCLXI0LUEAEB8RRxkysvLVVJS0mJ9YWGhduzYEZOiUpY9RsYTrJXEYF8AAOIt4iBTVlamZcuWtVi/bNkylZaWxqSolGV3LaX5ayUZggwAAHHWrodGTp8+XT6fT9/5znckSa+++qr++7//W9dee23MC0wp9mBfS0aZ8tK1BABAnEUcZK6//nrt3btXV155pRobGyVJ6enpmjVrVvghkkettEzJckgmqCzVq4bBvgAAxFXEQcayLN15552aM2eO1q5dq4yMDPXv3z95zzfqSCxLcudI3krlWPV0LQEAEGcRB5km2dnZOuWUU2JZS+fgCQWZbNVrF0EGAIC4iniwL47AHvCbZTWoxutXMGiSXBAAAJ0XQSbW7AG/OU2z+/LgSAAA4oYgE2t2i0yewyuJuWQAAIgngkys2bP7dk1rCjLcgg0AQLwQZGLNnt23iyt0azoPjgQAIH4IMrFmdy3luxok0bUEAEA8EWRizR7sm2eFgkwVXUsAAMQNQSbW7BaZHEcoyDC7LwAA8UOQiTV7sG+OVS+JriUAAOKJIBNr9mDfLDWNkaFrCQCAeCHIxJrdtZRh7AnxaJEBACBuCDKxZg/2zQiGggxdSwAAxA9BJtbsFhl3IBRkqggyAADEDUEm1uzBvu5AjSSpxssYGQAA4oUgE2v2YF+Xv06WgnQtAQAQRwSZWLO7lqTQnUsEGQAA4ocgE2suj+RwSZKyVc/t1wAAxBFBJtYsK9wqk2U1qMbrlzEmyUUBANA5EWTioekxBaqXL2Dk9QeTXBAAAJ0TQSYe3HaQsR9TwIMjAQCID4JMPGTkS5KK3KEgw+y+AADEB0EmHjK7SpKKXbWSmN0XAIB4IcjEgx1kujsJMgAAxBNBJh6yukmSCh3VkpjdFwCAeCHIxIPdItPFCgUZnrcEAEB8EGTiwQ4y+aqSRNcSAADxQpCJBzvI5AYrJXHXEgAA8UKQiQc7yGQHQi0yzCMDAEB8EGTiwR7sm+nfL8lof21jcusBAKCTIsjEQ0YXSZLT+JWteu0lyAAAEBcEmXhwZ0ppmZKkAqta+wgyAADEBUEmXjJD3UtdRZABACBeCDLxkhnqXiqwqrW31pvkYgAA6JwIMvHSNCmeqtXgC6qukVuwAQCINYJMvDQ9psAZmt13bw3dSwAAxBpBJl7sFplSd50kMU4GAIA4IMjEix1kilyhJ2ATZAAAiD2CTLzYQaab/eBI5pIBACD2CDLxYgeZAjvI7OPOJQAAYo4gEy/2YN88+8GRtMgAABB7BJl4sVtksgKhILOPu5YAAIg5gky82DP7pvur5FSAwb4AAMQBQSZeMvIlWZKkAtXQtQQAQBwQZOLF4ZQyCiTx4EgAAOKFIBNP9oDfLjw4EgCAuCDIxFPT85asKtV4/fL6A0kuCACAzoUgE09Nk+I5muaSoVUGAIBYIsjEU/h5S/WSeHAkAACx1qGDzPz583XKKacoJydH3bt313nnnad169Ylu6y2yyqUJB3jqpJEiwwAALHWoYPM0qVLNW3aNL399ttavHixfD6fxowZo9ra2mSX1ja5pZKkUsdeSQQZAABizZXsAg7npZdeavb9okWL1L17d61evVrf+ta3klRVBPJ6SJKKTCjIMJcMAACx1aGDzFdVVoam++/Spcsh9/F6vfJ6DzygsaqqKu51HVLuMZKkLoHdknhwJAAAsdahu5YOFgwGNX36dI0aNUqDBg065H7z589XXl5eeCkrK0tglV9ht8hk+SuULi9dSwAAxFjKBJlp06bpo48+0l/+8pfD7jd79mxVVlaGl23btiWowlak50nubElSibWPu5YAAIixlOhauuqqq/T888/rjTfeUI8ePQ67r8fjkcfjSVBlR2BZoe6lPetUYu2lRQYAgBjr0C0yxhhdddVVeuaZZ/Taa6+pT58+yS4pcnb30jHWHu2obEhyMQAAdC4dukVm2rRp+tOf/qRnn31WOTk5Ki8vlyTl5eUpIyMjydW1UV5owG+J9ml7Zb0afAGlpzmTXBQAAJ1Dh26Refjhh1VZWakzzjhDJSUl4eXJJ59Mdmltlxtqkenp2idjpK376pJcEAAAnUeHbpExxiS7hOjZLTLHuiskr7RpT60GFOUktyYAADqJDt0i0ynYY2RKrdCkeJv2pMisxAAApACCTLzZXUtd/LslGW3aTZABACBWCDLxZj9vyR2sU65qtWkvQQYAgFghyMSbO1PK7CpJKrX20bUEAEAMEWQSwX7mUom1V7urvarx+pNcEAAAnQNBJhHsAb8DPBWSpM20ygAAEBMEmUSwg8xxGaEncdO9BABAbBBkEsHuWuqVtl8SQQYAgFghyCSC3SJTot2S6FoCACBWCDKJ0G2AJKl7zadKl1efE2QAAIgJgkwiFA+W8nvKFajX6Y736VoCACBGCDKJYFnSwHMlSWc7V6iy3qfPdlUnuSgAAFIfQSZRBp4vSRrjWiOPGvXga58luSAAAFIfQSZRjhkm5fZQuqnXtxwf6J/vb6dVBgCAKBFkEuWg7qXLunwgY6QFr2xIclEAAKQ2gkwi2UHmFO/bOka79cKHO/T+tork1gQAQAojyCRSj1Ok7ifK6avRszl3qNjs1RX/b5XKKxuSXRkAACmJIJNIDof046ekgj7q5tuhpzJuk6p26LLHV6qWB0kCABAxgkyi5ZZKU56X8nvpGFOuv6bfpl3bt+hn/2+1GnyBZFcHAEBKIcgkQ16PUJjJ66le2qEnPbdp3Wef6aePryLMAAAQAYJMsuT3lKY8J+X20LHWdj3tuVnbN36gSx9bqaoGX7KrAwAgJRBkkqmgdyjMFPRRmbVLT7tvlm/TMk1YuFw7KuuTXR0AAB0eQSbZuhwrXbZYOuZk5Vs1+rPndp28+2md/9AyffBFRbKrAwCgQyPIdATZhdLk56SB5ylNfv0q7TFdW3+/frLwdf199RfJrg4AgA6LINNRuDOlCxdJ350nYzl0oesNPe2YrceeekaznvpAdY3cng0AwFcRZDoSy5JGXSPrkmdlckrV17FDT7tvUsma+3T+/a8zCzAAAF9hGWNMsouIp6qqKuXl5amyslK5ubnJLqft6vZJz10jrf2nJGlD8Bjd5J+iQd8YrxmjByjD7UxygQAAxE9bP79pkemoMrtIE56QfviYgpmF6u/4Un9y36ZTl/9CP/31H/Xsmi/VyTMoAABHRJDpyCxLGnSBHFe9I53yUwUtp0Y739MT3mtU+9RVmvLQ81q5eV+yqwQAIGnoWkolezYo8PJNcq5/QZJUZzz6a+B0bTh2sn487ls6oSTFzw8AAFtbP78JMqloy1vyvXSj0na8K0kKGEsvBkdoZemPNXbMOI08tqssy0pykQAAtB9BxtYpg4wkGSNtWqra1+9V1ral4dWrg/21LGec+p7xE535tb5KT2NQMAAg9RBkbJ02yBys/CPVvn6f0tc9I6dCD52sNR4ttk7TvgETNPKM7+mE0vzk1ggAQAQIMrajIsg0qd6pupV/lHflIhXUbw2v3mG6aGXGN+UYdL5O/sZYFednJrFIAACOjCBjO6qCTBNjFNjytnYt/b3yN7+oDFMX3rTDdNEHWafJddwYDfrG91XUtWsSCwUAoHUEGdtRGWQO5mtQ1cf/1q63n1TpzteUaQ48Vdtr0vRx2iBV9jhdhSd9XycMGi6nkzvyAQDJR5CxHfVB5mC+Bu398N/aufo5dd3xhoqCO5tt/lKF2pZ3itx9v6k+w8eo4Jh+SSoUAHC0I8jYCDKHYIz2bflIX7zzrNybX9Oxde/LreYPptxpFWpH/nA5+nxDZSeNVkGP40OT9AEAEGcEGRtBpm389VXauOpl7f9kifJ2vaP+/g1yWcFm+1RZOdqZfaICpcPU9bjT1O24kbKyuiWpYgBAZ0aQsRFk2mff/n3a+O5rqt/whgp2r9IA/zp5LH+L/Xa5SlXRZbDSep6q4hNGKaNsiOTOSkLFAIDOhCBjI8jExr6qGn265i3t3/C2Mne9p54Nn6qvtb3FfkFZ2uspU12XgfL0GKqufYcp7ZivSdlFdEsBANqMIGMjyMRHgy+gTzdt045P3pJ/20oV7P9AxwU2qNCqbHX/ameBqvKPl7NksAp6D5WnZKDUbYDkyU5w5QCAVECQsRFkEqe8skGfbFivXetXy+z8SPmVa9UvuFnHWtvltFr/Y1bpKZW3oL/cJScot2ywHN1PkAoHSJ6cBFcPAOhICDI2gkzyGGO0vbJBn2wp166Na+T/8n1l7v9Ux/i2qL/jCxVaVYd8b627UI15feQu6q/M4gGyuvaVuvaTCvpIaekJPAsAQDIQZGwEmY5nT41XH2+v0uatW1S97RM59nyqnOqNOtZsU3/Hl+puVRzyvUaWGjJLFCw4Vp6iAXIV2uEmv2doSecaA0BnQJCxEWRSQyBo9MX+Oq0rr9aWL75U5Zfr5N/9mbJqNqmnytXbKlcfa4dyrfrDHsfnzpfJK5Ora285CnpJ+b0OhJz8nozJAYAUQZCxEWRSW6M/qE17arVxd40+31WtXeVfqnH3BrkqNqnE/4V6W+Uqs3arh7VbXayaIx7P7ymQlXeMnHmlUm6plHuMlFNy4HVuieTJ5Q4rAEgygoyNINM5GWO0r7ZRn++p1ee7a/T5nlp9uXO3vLs3yVG5VcUmFG4OLHtU0IagI0nBtCxZecfIyikJ3Tae3V3KKrRfF0pZ3UPrMrtJTleczxQAjk4EGRtB5ugTCBrtrGrQ1n112rqvTl/YX/fs2S1/xTZ56naqyNqnEu1TsXVgKbH2Kd+qbfPPMbJkZXY9KOh0D4WcrG5SZhcpo0vLry53HM8cADqPtn5+899JdDpOh6XS/AyV5mfo68d2bbG9rtGvL/bXa5sdcN6qqNeOygbtqGzQvv0VctTsUHftVZH2q9CqUDerUt2sShWqUoX26y6qCt1SXrcntLSVO9sONgUtg056vpSeFxqw7Mk98Do9P/Q9rT8A0AL/MuKok+l2aUBRjgYUtT5XjT8Q1O4ar7ZXNGhHZb12VDToI/vrjsp6ba9s0L6aehWY6oNCzoHA082qUp5qVGDVKF81KrCqla9aOSwjNdaElsqtkReelmUHm7zmQafptSc7FJTc2aHHRHhyQl/dWQfWe7KltEzGAAHoNAgywFe4nA6V5GWoJC9DUkGr+/gCQe2rbdTuaq92VTdod7VXu6u92l7t1fs1Xnu9V7uqvKpvDMhSULmqU4FVrQLVKN+qUYGqQ2HHfp1r1SlHdcq16pSrOuU56pWjOmWowf6htaGlekeUZ2gdCDvurJYByJ0puTKktIxQ6EnLOGixv3elH7QtMzS3T3hbhuRwRFkjALQNQQZohzSnQ0W56SrKTZeUd9h9a73+cLDZV+vVvlqf9tc1al9taPmstrHZ93WNgWbvd8mvbNUrxw44TUEnpyn42K+z1KAsq0G5jgblOBqVY4W+z1KD0k29PMF6WTKSjNRYHVrixelpPQg5PZLroMXpCY0bcqVLTvtrNNucbntJkxzO+J0fgA6DIAPEWZbHpSyPS727te2p4A2+QDjY7K/1aW+tV1UNflXV+1RZ7wt/3V7vU1VD0zq/qhp8OvzQfaMMeZUlr7Ks+lDw0YGwk2k1KFv1ylCj0q1G5bp8ynH4lOP0KdPhU6bVqEyrUekKLW7TIHfQq7Rgg5xBr1xB74EfFfCGloaKaH51UbIOhBpnmuRI+8prd2jc0cGvnW77+4Nft/L+Fvu5QsHJ4ZIs++vB67769bD72K+PtI/loIsQEEEG6HDS05wHdW21XTBoVO1tHniagk7TUusNqMbrV02DX7WNflU3+PWl1x9aZy/hMOSPrG5LQaWrMRSE1KgMy2t/71W65VOGvMp0BpTtDCjbFVC2069MR0CZ9td0h08Zll/pll/plk9u+ZUmn9JMY+hrsFEu45Mz2CinaZQz2ChHoFGOYKMcAa+soO8rFZkDgaqzajUUNYUgpx12DlrC65wHgtBX1zkO2tZiXWvHOvi9jlbWtfG9ltX8q6yvvG4Kbk3rD7XtUMdQG/dr2qY27nfw92pHTfY+B7+OaJ2abz843B5uv2h+XgcL0AQZoJNwOCzlZaQpLyNNZe08hjFG9b6Aahqah5um4BNaH1CNNxSKar1+1fkCqm8MLaHXftX7AqpsDGhHY0D1voBM0P4BQUlfzRsxYikot/xyKaA0+ZWmgLJcQWW6gspyGWW5gspwmlB4chllOALyOIPyWAGlW0G5HQF5LL/cVlDupq/yK80KyGUFmh3bJb9cJvTVafxyKiiHAnKaQPi1ZQJyhJegLOOXZULrrWBACvrtJSCZr3x/8FcTOPRJB/2S/J07rKEDOyjcfP8+afiUpFRBkAEQZlmWMt0uZbpd6h6jYxpj5PUHVWeHmvpGf+h1OPh8JQQ1BlXn88vrC8rrD9hfg2rwBeT1h9Y1NG07eL0vqAa/Q96Dutd225/zHY3TYcnpsJTW9NXpkMtpyeVwyOVq2mavsyS3U0pzBJXuMEpzGHmsgNwOYy9Buayg3FbodZoVVJrDyGUZ+6vksoL2VyOXvc2p0FeHFZTTCsohI6eCclrGfm3kkL3eCoZeW0YOE2y2LbQEwtss2fsaE35tycgyTa+D9n7GDnHBUFgzQYWGcAUlGft70/x1eFvTa7W+3yHf09b9ml5HUs9hjt3qew46tnTQfiaCdTqwPikOqiOJU9KlRJD5zW9+o7vvvlvl5eUaOnSoHnzwQZ166qnJLgtAG1iWpfQ0p9LT4j/41hgjX8AcIuQEwuGoKQj5AkaN/qB8gdDSGAge9H1oW2MgKJ//4O3mwP7h9UaN9vGatnkPOk4g2Pwf+UAwtK4x5r8Bh72kBqfDktOywsHOYR0IeU3bHOFtlixLoXX2a4dlyeGQ/b0lZ9O6VraHFntfx4HXDofs9x603Wq+/eD3Wwcfy9H8uEfabh2034FtTe8LvdfSgX2b9peav9+Sva/dmxVaZ+Swe4BCHVgmFC5l93g1HU9NxzWyjGQdtE+oDhMKnrJ7Ck3T+4wkK/Q+Hfz+0Hvy8rqobaMAY6/DB5knn3xSM2fO1MKFCzVixAgtWLBAY8eO1bp169S9e6z+zwigM7AsS26XJbfLodZnCUqOQNCEg5DPHwx9HzTyB4LyB438ASN/MGh/PWj9Ifc5sM4XCB3v4H1C4elAiDr42E37+wJGQRPa3vQ1vBijoP01EJQCwaC9nw6z34HlwHGP/HsJyEiH6T1Darjt/EGaNOLwd3DGS4cPMvfee68uv/xyXXrppZKkhQsX6oUXXtCjjz6qG264IcnVAcCRhVoYEtMq1ZEYc3Dg0SEDT2vByP+VoBU0oQHtQRM6btDIXm9k7IDVtN9Xtwftn9+0b/Cr24MHvm+xvdm2g993pO0HfuaBmg61PfRzjZrXEPodNl9n7N9r8+/tdc3ef2A/o9b3Dx607avrgqb5ccPbWqnVyMjlSN4A4A4dZBobG7V69WrNnj07vM7hcGj06NFavnx5q+/xer3yeg8MfKuqqop7nQCAlizLCo3zSXYh6NQ6dGfqnj17FAgEVFRU1Gx9UVGRysvLW33P/PnzlZeXF17Kytp7/wYAAOjoOnSQaY/Zs2ersrIyvGzbti3ZJQEAgDjp0C1+3bp1k9Pp1M6dO5ut37lzp4qLi1t9j8fjkcfjSUR5AAAgyTp0i4zb7dbw4cP16quvhtcFg0G9+uqrGjlyZBIrAwAAHUGHbpGRpJkzZ2ry5Mk6+eSTdeqpp2rBggWqra0N38UEAACOXh0+yPzoRz/S7t27NXfuXJWXl+trX/uaXnrppRYDgAEAwNHHMiaJ8wonQFVVlfLy8lRZWanc3NxklwMAANqgrZ/fHXqMDAAAwOEQZAAAQMoiyAAAgJRFkAEAACmLIAMAAFIWQQYAAKQsggwAAEhZHX5CvGg1TZNTVVWV5EoAAEBbNX1uH2m6u04fZKqrqyVJZWVlSa4EAABEqrq6Wnl5eYfc3uln9g0Gg9q+fbtycnJkWVbMjltVVaWysjJt27at084YzDmmvs5+fhLn2Bl09vOTOMf2MMaourpapaWlcjgOPRKm07fIOBwO9ejRI27Hz83N7bR/KJtwjqmvs5+fxDl2Bp39/CTOMVKHa4lpwmBfAACQsggyAAAgZRFk2snj8eimm26Sx+NJdilxwzmmvs5+fhLn2Bl09vOTOMd46vSDfQEAQOdFiwwAAEhZBBkAAJCyCDIAACBlEWQAAEDKIsi0029+8xv17t1b6enpGjFihN55551kl9Qu8+fP1ymnnKKcnBx1795d5513ntatW9dsnzPOOEOWZTVbfv7znyep4sjdfPPNLeo//vjjw9sbGho0bdo0de3aVdnZ2frBD36gnTt3JrHiyPXu3bvFOVqWpWnTpklKvWv4xhtvaPz48SotLZVlWfrHP/7RbLsxRnPnzlVJSYkyMjI0evRobdiwodk++/bt06RJk5Sbm6v8/HxddtllqqmpSeBZHN7hztHn82nWrFkaPHiwsrKyVFpaqksuuUTbt29vdozWrvsdd9yR4DM5tCNdxylTprSo/6yzzmq2T0e+jkc6v9b+TlqWpbvvvju8T0e+hm35fGjLv59bt27V2WefrczMTHXv3l3XX3+9/H5/zOokyLTDk08+qZkzZ+qmm27Su+++q6FDh2rs2LHatWtXskuL2NKlSzVt2jS9/fbbWrx4sXw+n8aMGaPa2tpm+11++eXasWNHeLnrrruSVHH7nHjiic3qf/PNN8PbZsyYoeeee05/+9vftHTpUm3fvl0XXHBBEquN3MqVK5ud3+LFiyVJF154YXifVLqGtbW1Gjp0qH7zm9+0uv2uu+7SAw88oIULF2rFihXKysrS2LFj1dDQEN5n0qRJ+vjjj7V48WI9//zzeuONN3TFFVck6hSO6HDnWFdXp3fffVdz5szRu+++q6efflrr1q3TOeec02LfefPmNbuu//Vf/5WI8tvkSNdRks4666xm9f/5z39utr0jX8cjnd/B57Vjxw49+uijsixLP/jBD5rt11GvYVs+H47072cgENDZZ5+txsZGvfXWW3r88ce1aNEizZ07N3aFGkTs1FNPNdOmTQt/HwgETGlpqZk/f34Sq4qNXbt2GUlm6dKl4XWnn366ueaaa5JXVJRuuukmM3To0Fa3VVRUmLS0NPO3v/0tvG7t2rVGklm+fHmCKoy9a665xvTt29cEg0FjTGpfQ0nmmWeeCX8fDAZNcXGxufvuu8PrKioqjMfjMX/+85+NMcZ88sknRpJZuXJleJ8XX3zRWJZlvvzyy4TV3lZfPcfWvPPOO0aS2bJlS3hdr169zH333Rff4mKktXOcPHmyOffccw/5nlS6jm25hueee675zne+02xdKl3Dr34+tOXfz3/961/G4XCY8vLy8D4PP/ywyc3NNV6vNyZ10SITocbGRq1evVqjR48Or3M4HBo9erSWL1+exMpio7KyUpLUpUuXZuv/7//+T926ddOgQYM0e/Zs1dXVJaO8dtuwYYNKS0t17LHHatKkSdq6daskafXq1fL5fM2u5/HHH6+ePXum7PVsbGzUH//4R02dOrXZg1JT/Ro22bRpk8rLy5tds7y8PI0YMSJ8zZYvX678/HydfPLJ4X1Gjx4th8OhFStWJLzmWKisrJRlWcrPz2+2/o477lDXrl110kkn6e67745pk30iLFmyRN27d9dxxx2nX/ziF9q7d294W2e6jjt37tQLL7ygyy67rMW2VLmGX/18aMu/n8uXL9fgwYNVVFQU3mfs2LGqqqrSxx9/HJO6Ov1DI2Ntz549CgQCzS6KJBUVFenTTz9NUlWxEQwGNX36dI0aNUqDBg0Kr7/44ovVq1cvlZaW6oMPPtCsWbO0bt06Pf3000mstu1GjBihRYsW6bjjjtOOHTt0yy236Jvf/KY++ugjlZeXy+12t/hwKCoqUnl5eXIKjtI//vEPVVRUaMqUKeF1qX4ND9Z0XVr7O9i0rby8XN27d2+23eVyqUuXLil5XRsaGjRr1ixNnDix2cP4rr76ag0bNkxdunTRW2+9pdmzZ2vHjh269957k1ht25111lm64IIL1KdPH23cuFG//OUvNW7cOC1fvlxOp7NTXcfHH39cOTk5LbqtU+Uatvb50JZ/P8vLy1v9u9q0LRYIMgibNm2aPvroo2bjRyQ1648ePHiwSkpKdOaZZ2rjxo3q27dvosuM2Lhx48KvhwwZohEjRqhXr17661//qoyMjCRWFh+PPPKIxo0bp9LS0vC6VL+GRzOfz6cJEybIGKOHH3642baZM2eGXw8ZMkRut1s/+9nPNH/+/JSYCv+iiy4Kvx48eLCGDBmivn37asmSJTrzzDOTWFnsPfroo5o0aZLS09ObrU+Va3ioz4eOgK6lCHXr1k1Op7PFqOydO3equLg4SVVF76qrrtLzzz+v119/XT169DjsviNGjJAkffbZZ4koLeby8/M1YMAAffbZZyouLlZjY6MqKiqa7ZOq13PLli165ZVX9NOf/vSw+6XyNWy6Lof7O1hcXNxi8L3f79e+fftS6ro2hZgtW7Zo8eLFzVpjWjNixAj5/X5t3rw5MQXG2LHHHqtu3bqF/1x2luv4n//8R+vWrTvi30upY17DQ30+tOXfz+Li4lb/rjZtiwWCTITcbreGDx+uV199NbwuGAzq1Vdf1ciRI5NYWfsYY3TVVVfpmWee0WuvvaY+ffoc8T1r1qyRJJWUlMS5uvioqanRxo0bVVJSouHDhystLa3Z9Vy3bp22bt2aktfzscceU/fu3XX22Wcfdr9UvoZ9+vRRcXFxs2tWVVWlFStWhK/ZyJEjVVFRodWrV4f3ee211xQMBsMhrqNrCjEbNmzQK6+8oq5dux7xPWvWrJHD4WjRHZMqvvjiC+3duzf857IzXEcp1Eo6fPhwDR069Ij7dqRreKTPh7b8+zly5Eh9+OGHzQJpUygfOHBgzApFhP7yl78Yj8djFi1aZD755BNzxRVXmPz8/GajslPFL37xC5OXl2eWLFliduzYEV7q6uqMMcZ89tlnZt68eWbVqlVm06ZN5tlnnzXHHnus+da3vpXkytvu2muvNUuWLDGbNm0yy5YtM6NHjzbdunUzu3btMsYY8/Of/9z07NnTvPbaa2bVqlVm5MiRZuTIkUmuOnKBQMD07NnTzJo1q9n6VLyG1dXV5r333jPvvfeekWTuvfde895774Xv2LnjjjtMfn6+efbZZ80HH3xgzj33XNOnTx9TX18fPsZZZ51lTjrpJLNixQrz5ptvmv79+5uJEycm65RaONw5NjY2mnPOOcf06NHDrFmzptnfzaY7Pd566y1z3333mTVr1piNGzeaP/7xj6awsNBccsklST6zAw53jtXV1ea6664zy5cvN5s2bTKvvPKKGTZsmOnfv79paGgIH6MjX8cj/Tk1xpjKykqTmZlpHn744Rbv7+jX8EifD8Yc+d9Pv99vBg0aZMaMGWPWrFljXnrpJVNYWGhmz54dszoJMu304IMPmp49exq3221OPfVU8/bbbye7pHaR1Ory2GOPGWOM2bp1q/nWt75lunTpYjwej+nXr5+5/vrrTWVlZXILj8CPfvQjU1JSYtxutznmmGPMj370I/PZZ5+Ft9fX15srr7zSFBQUmMzMTHP++eebHTt2JLHi9vn3v/9tJJl169Y1W5+K1/D1119v9c/l5MmTjTGhW7DnzJljioqKjMfjMWeeeWaL8967d6+ZOHGiyc7ONrm5uebSSy811dXVSTib1h3uHDdt2nTIv5uvv/66McaY1atXmxEjRpi8vDyTnp5uTjjhBHP77bc3CwHJdrhzrKurM2PGjDGFhYUmLS3N9OrVy1x++eUt/kPYka/jkf6cGmPMb3/7W5ORkWEqKipavL+jX8MjfT4Y07Z/Pzdv3mzGjRtnMjIyTLdu3cy1115rfD5fzOq07GIBAABSDmNkAABAyiLIAACAlEWQAQAAKYsgAwAAUhZBBgAApCyCDAAASFkEGQAAkLIIMgAAIGURZAB0ekuWLJFlWS0ebgcg9RFkAABAyiLIAACAlEWQARB3wWBQ8+fPV58+fZSRkaGhQ4fqqaeeknSg2+eFF17QkCFDlJ6erq9//ev66KOPmh3j73//u0488UR5PB717t1b99xzT7PtXq9Xs2bNUllZmTwej/r166dHHnmk2T6rV6/WySefrMzMTJ122mlat25deNv777+vb3/728rJyVFubq6GDx+uVatWxek3AiBWCDIA4m7+/Pl64okntHDhQn388ceaMWOGfvzjH2vp0qXhfa6//nrdc889WrlypQoLCzV+/Hj5fD5JoQAyYcIEXXTRRfrwww918803a86cOVq0aFH4/Zdccon+/Oc/64EHHtDatWv129/+VtnZ2c3quPHGG3XPPfdo1apVcrlcmjp1anjbpEmT1KNHD61cuVKrV6/WDTfcoLS0tPj+YgBEL2bP0QaAVjQ0NJjMzEzz1ltvNVt/2WWXmYkTJ5rXX3/dSDJ/+ctfwtv27t1rMjIyzJNPPmmMMebiiy823/3ud5u9//rrrzcDBw40xhizbt06I8ksXry41RqafsYrr7wSXvfCCy8YSaa+vt4YY0xOTo5ZtGhR9CcMIKFokQEQV5999pnq6ur03e9+V9nZ2eHliSee0MaNG8P7jRw5Mvy6S5cuOu6447R27VpJ0tq1azVq1Khmxx01apQ2bNigQCCgNWvWyOl06vTTTz9sLUOGDAm/LikpkSTt2rVLkjRz5kz99Kc/1ejRo3XHHXc0qw1Ax0WQARBXNTU1kqQXXnhBa9asCS+ffPJJeJxMtDIyMtq038FdRZZlSQqN35Gkm2++WR9//LHOPvtsvfbaaxo4cKCeeeaZmNQHIH4IMgDiauDAgfJ4PNq6dav69evXbCkrKwvv9/bbb4df79+/X+vXr9cJJ5wgSTrhhBO0bNmyZsddtmyZBgwYIKfTqcGDBysYDDYbc9MeAwYM0IwZM/Tyyy/rggsu0GOPPRbV8QDEnyvZBQDo3HJycnTddddpxowZCgaD+sY3vqHKykotW7ZMubm56tWrlyRp3rx56tq1q4qKinTjjTeqW7duOu+88yRJ1157rU455RTdeuut+tGPfqTly5froYce0v/+7/9Kknr37q3Jkydr6tSpeuCBBzR06FBt2bJFu3bt0oQJE45YY319va6//nr98Ic/VJ8+ffTFF19o5cqV+sEPfhC33wuAGEn2IB0AnV8wGDQLFiwwxx13nElLSzOFhYVm7NixZunSpeGBuM8995w58cQTjdvtNqeeeqp5//33mx3jqaeeMgMHDjRpaWmmZ8+e5u677262vb6+3syYMcOUlJQYt9tt+vXrZx599FFjzIHBvvv37w/v/9577xlJZtOmTcbr9ZqLLrrIlJWVGbfbbUpLS81VV10VHggMoOOyjDEmyVkKwFFsyZIl+va3v639+/crPz8/2eUASDGMkQEAACmLIAMAAFIWXUsAACBl0SIDAABSFkEGAACkLIIMAABIWQQZAACQsggyAAAgZRFkAABAyiLIAACAlEWQAQAAKev/A4ntVDBqIMmMAAAAAElFTkSuQmCC\n"
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjcAAAGwCAYAAABVdURTAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABKR0lEQVR4nO3deXhTVf4/8PdN2qR7S+leSssOlb1Ip+KGVgryxV0RHUFUZhQYkQqDqIDiDEUUxIWRcUH0Ny4ogzoOiMOuQmUvyla2liLdaEv3JWlyfn+kSQltoU2T3Obm/XqePE1ObpLP7QXy5txzzpWEEAJERERECqGSuwAiIiIie2K4ISIiIkVhuCEiIiJFYbghIiIiRWG4ISIiIkVhuCEiIiJFYbghIiIiRfGQuwBnMxqNyM3Nhb+/PyRJkrscIiIiagUhBCoqKhAVFQWV6sp9M24XbnJzcxETEyN3GURERGSDc+fOoUuXLlfcxu3Cjb+/PwDTLycgIEDmaoiIiKg1ysvLERMTY/kevxK3CzfmU1EBAQEMN0RERC6mNUNKOKCYiIiIFIXhhoiIiBSF4YaIiIgUheGGiIiIFIXhhoiIiBSF4YaIiIgUheGGiIiIFEXWcPPjjz9i3LhxiIqKgiRJ+Oabb676mu3bt2Po0KHQarXo2bMnVq9e7fA6iYiIyHXIGm6qqqowaNAgrFixolXbZ2VlYezYsRg5ciQyMjLwzDPP4IknnsAPP/zg4EqJiIjIVci6QvGYMWMwZsyYVm+/cuVKdOvWDUuXLgUA9OvXDz///DPeeOMNpKSkOKpMIiIiciEuNeYmPT0dycnJVm0pKSlIT09v8TV1dXUoLy+3uhEREZFyuVS4yc/PR3h4uFVbeHg4ysvLUVNT0+xr0tLSEBgYaLnxiuBERETKpvgLZ86dOxepqamWx+arihK5q/JaPcpr9HKXQUQKpvFQIczfS7bPd6lwExERgYKCAqu2goICBAQEwNvbu9nXaLVaaLVaZ5RH1OHtP1uCCe/vhq7eKHcpRKRgQ7sGYd3UEbJ9vkuFm6SkJGzYsMGqbdOmTUhKSpKpIiLXUVGrxzNrMqCrN8JTLUElSXKXREQK5amWd9SLrOGmsrISp06dsjzOyspCRkYGgoOD0bVrV8ydOxfnz5/HJ598AgB48skn8c477+Cvf/0rHnvsMWzduhVffvkl1q9fL9cuELmMhd8dxbmSGkQHeeP7Z25AgJen3CURETmErOFm3759GDlypOWxeWzMpEmTsHr1auTl5SEnJ8fyfLdu3bB+/XrMnDkTb775Jrp06YIPPviA08DdxOLvj+PbjPMQQu5KXI+AQEF5HSQJeGP8YAYbIlI0SQj3+qooLy9HYGAgysrKEBAQIHc51EobfsvD1E8PyF2Gy5s+sidmpfSRuwz57P0A2L8acKt/9YhkENEfuHulXd+yLd/fLjXmhtxTflktnv/6NwDAYyO64Z6h0TJX5Jo0Hir0CvOTuwz51OuATS8Bugq5KyFSPs/mJ/k4C8MNdWhGo8DstYdQWq3HgOhAPDemLzQeLrU8E3UUZ3eago1vKHD3P+WuhkjZvAJl/XiGG+rQPk7Pxk8ni+DlqcIb4wcz2JDtTmw0/eyVAvS8Vd5aiMih+E1BHdaJggqkfX8cAPDC7f3Q051PqVD7CAFkfm+632e0vLUQkcOx54acJr+sFpuO5sNgbN1ozi/2noOu3oib+4Tij3+IdXB1pGgXjgOlZwG1Bug+8urbE5FLY7ghp6jVG/DIh7txsrCyTa8L9tVgyX0DIXHBOWoPc69NtxsBLXsAiZSO4YacYvH3x3GysBKdfTVI6tG5Va9RqyT88Q+xrbs+SdnvwIXMdlZJinX0G9PP3jwlReQOGG7I4X48cQGrd2UDAJY+MAg39wmz7wfUVQLvjgBqS+37vqQ8DDdEboHhhhzqYpUOs746BACYmBRr/2ADAKe3moKNpy/Qubv935+UoedtQFCM3FUQkRMw3JDDCCHw/Ne/obCiDj1CfTF3TD/HfJB5im/CJGB0mmM+g4iIXAangpPD/PvAeXx/OB8eKgnLxw+Bt0Zt/w8xGoATP5ju85QDERGBPTdkZ/87ko/Za39Frd4AncEIAJh5W28M6OKg1SrP7weqiwBtIBB7nWM+g4iIXArDDdlNXlkNZn11COW19Za2pO6d8eRNPRz3oeYpvj1vBdS80jURETHckJ0YjQLPfmkKNoO6BOKdh4ZCrZIQHuAFtcqBa9SYx9v0GeO4zyAiIpfCcEPWqkuADbOAqiLT416jgOumt7z99leBszuRV1qDqUVV+ItWwiCvIPh854DxNZcTRqDwKCCpgZ7Jjv88IiJyCQw3ZO3IOuDwvxsfZ+0ABtwP+Ic33bboFLB9EQAgGkC0Oc/87vAqrXW/CfAJdvKHEhFRR8VwQ9aKTpp+9hkLlJw2XZPn5A/A0IlNtz1hGu9yQt0T79SMxjVRAfjTjd2de6kESQV0u8l5n0dERB0eww1ZKz5l+tl7FFBRYOqZydzYfLjJNI13+az2Ouz0vhnzJt0IyV/rxGKJiIia4jo3ZM0cbjr3BPo0rBtzZhugr7XeruYiRE46AGCzcShevXcgQhlsiIioA2C4oUb1dUBpjul+515AxEAgIBrQVwNZP1ptWn1kIyRhQKaxC264dhiS45sZk0NERCQDhhtqVJJlmoGk8Qf8wgBJAnqnmJ5rGF9jdmzHlwCAfdpEvDjWQZdVICIisgHDDTUqbhhM3LmHKdgAjZc0yNwI5GYAuRnYvv1/6Fn+CwDg2lET4Kvl0C0iIuo4+K1EjczjbUJ6NbZ1uxHw8AYqcoH3TLOSbgYACaj2CELvhFucXSUREdEVseeGGhVdMpjYzNMbuGk2jAHRMPpHoVgVglwRjEJVGLS3zgVUTlisj4iIqA3Yc0ONipsJNwBeKBqFTwv7Wh77aNTYMPUGhIX4OrM6IiKiVmHPDTWyjLlpDDdCCPwnI9fyWK2S8Mqd/RHHYENERB0Ue27IpLoEqC423b8k3OSW1aKirh6eagn7590GLw81NB7MxERE1HEx3JBJ8WnTT/9IQOtnac7MLwcA9Aj1Q4CXpxyVERERtQn/C04mLYy3OZ5fAQDoHe7v7IqIiIhswnBDJi2Em8yGcNMnguGGiIhcA8MNmeQeNP0M7WvVbA43fRluiIjIRTDcEFBXAWT/ZLrfo3FRPr3BiNMXKgGw54aIiFwHww0Bp7cBBh0Q3N1qdeIzF6qgNwj4az0QHeQtY4FEREStx3BDwImNpp+9xzReUwrA8YaZUr0j/CFd0k5ERNSRMdy4O6MBOPGD6X6f0VZPcTAxERG5IoYbd3d+P1BdBGgDga5JVk+dKGgIN5wGTkRELoSL+LkrfQ1QdQE4vM70uFcyoLZepO84e26IiMgFMdy4o5qLwNsJjZdbAIDe1qekcktr8PvFGgCcBk5ERK6F4cYd5R5sCDYS4KEFOveyCjdGo8Csrw4BAIbFdkKQj0amQomIiNqO4cYdma8j1WcMMOHzJk+v2pmFXaeL4e2pxpL7Bjq5OCIiovbhgGJ3VHTS9LNzjyZPnblQiSUbMwEAL/5fP3QP9WuyDRERUUfGcOOOLNeR6tXkqc3HCqAzGDG8WzAeGt7VyYURERG1H8ONOyo299z0bPJUZr7pcgvX9ejMhfuIiMglMdy4G30tUHrOdD+kac9NZoFpVWLOkCIiIlfFcONuSs4AEIA2APANtXrKYBQ4WWC+UGaADMURERG1H8ONu7GMt+lpdR0pAMgurkJdvRFenip0DfaRoTgiIqL2Y7hxN1ccb2Nakbh3uD/UKo63ISIi18Rw427Ma9w0M97GcrkFXkuKiIhcGMONu7nCGjcneC0pIiJSAIYbd3OFNW4yG64C3peDiYmIyIUx3LiT6hKgpsR0/7KemxqdAdnFVQDYc0NERK6N4cadmHttAqIBja/VUycLKyAE0NlXg1B/rQzFERER2QfDjTuxnJJqOt7mOMfbEBGRQjDcuBPzysRBsU2eymS4ISIihWC4cSeV+aafAVFNnvr191IAQHwkBxMTEZFrY7hxJxUN4cYv3Kq5Vm/AoXNlAIBr44KdXRUREZFdMdy4k4o800//SKvmw+fLoDMYEeKnRWxnXnaBiIhcm+zhZsWKFYiLi4OXlxcSExOxZ8+eK26/fPly9OnTB97e3oiJicHMmTNRW1vrpGpdXEWB6ad/hFXz3uyLAIBr4zpBknjZBSIicm2yhps1a9YgNTUVCxYswIEDBzBo0CCkpKSgsLCw2e0/++wzPPfcc1iwYAGOHTuGDz/8EGvWrMHzzz/v5MpdkNEAVDYfbvZlm9a+SYjt5OyqiIiI7E7WcLNs2TJMmTIFkydPRnx8PFauXAkfHx+sWrWq2e137dqFESNG4KGHHkJcXBxGjRqFCRMmXLW3hwBUFQHCAEACfMMszUajwL6z5p4bjrchIiLXJ1u40el02L9/P5KTkxuLUamQnJyM9PT0Zl9z3XXXYf/+/ZYwc+bMGWzYsAG33357i59TV1eH8vJyq5tbMs+U8gsD1B6W5lMXKlFWo4e3pxrxUZwpRURErs/j6ps4RlFREQwGA8LDrWfuhIeH4/jx482+5qGHHkJRURGuv/56CCFQX1+PJ5988oqnpdLS0vDyyy/btXaX1MJMqb0Np6SGdA2Cp1r2IVhERETt5lLfZtu3b8eiRYvwj3/8AwcOHMC6deuwfv16vPLKKy2+Zu7cuSgrK7Pczp0758SKO5AWZkrtaxhMPIynpIiISCFk67kJCQmBWq1GQUGBVXtBQQEiIiKafc28efPwyCOP4IknngAADBgwAFVVVfjTn/6EF154ASpV06ym1Wqh1fJaSS3NlDqQ0xBuOJiYiIgUQraeG41Gg4SEBGzZssXSZjQasWXLFiQlJTX7murq6iYBRq1WAwCEEI4rVgksPTeN4UYIgbxS0zT67qG+zb2KiIjI5cjWcwMAqampmDRpEoYNG4bhw4dj+fLlqKqqwuTJkwEAEydORHR0NNLS0gAA48aNw7JlyzBkyBAkJibi1KlTmDdvHsaNG2cJOdQC85ibS8JNeW09dAYjACDEj71bRESkDLKGm/Hjx+PChQuYP38+8vPzMXjwYGzcuNEyyDgnJ8eqp+bFF1+EJEl48cUXcf78eYSGhmLcuHH4+9//LtcuuA7zbKlLxtwUV9YBAPy0HvDyZDgkIiJlkISbnc8pLy9HYGAgysrKEBDgRlOfl/Y1nZqasg2IHgrANFPq/pXpiO3sgx2zR8pcIBERUcva8v3tUrOlyEZWqxM39twUVZh6bnhKioiIlIThxh1UFQHCCEgqwDfU0lxUpQMAdPbVyFUZERGR3THcuAPzTCnfUKvViS09N/7suSEiIuVguHEHLVwws7iqIdyw54aIiBSE4cYdtLA6cXGl6bQUe26IiEhJGG7cQQvXlSpqmAre2ZfhhoiIlIPhxh1UNF3jBrik58aPp6WIiEg5GG7cQTOrEwPABXPPDaeCExGRgjDcuINmritVV29ARW09APbcEBGRsjDcuINmZkuZT0l5qCQEenvKURUREZFDMNwoXQurE5vDTWc/DSRJkqMyIiIih2C4UbqqCy2sTsxLLxARkTIx3CideTCxbxigarzyt3l1Yg4mJiIipWG4UboWZkoVV3EaOBERKRPDjdI1M1MK4BXBiYhIuRhulK7F60qx54aIiJSJ4UbpWriuFC+9QERESsVwo3QVDT03Ta4rxYtmEhGRMjHcKF2LVwQ399zwtBQRESkLw43SNTNbymgUl4y5Yc8NEREpC8ONkhkNQFWh6f4l4aasRg+DUQAAgtlzQ0RECsNwo2QtrU7ccEoq0NsTGg/+ESAiImXhN5uSmcfb+IVbrU6cW1YLAIgM9JKjKiIiIodiuFGyFmZK5ZbWAACig7ydXREREZHDMdwoWQszpczhJorhhoiIFIjhRslauK7UeYYbIiJSMIYbJatsPtw09txwzA0RESkPw42StdBzk1tqGlDMnhsiIlIihhsla2bMjdEokFfG01JERKRcDDdK1sxsqaLKOugNAioJCOd1pYiISIEYbpRKXwtUNoSbgGhLs3kwcUSAFzzUPPxERKQ8/HZTqpIzAASgDQR8QyzNHG9DRERKx3CjVMWnTD9DegKSZGnmGjdERKR0DDdKZQ43nXtaNXONGyIiUjqGG6WyhJteVs2Nl17gGjdERKRMDDdKZQk3PayaczkNnIiIFI7hRqmKTpp+hlzec8MBxUREpGwMN0pUXQLUlJjuB3e3NNfoDCip0gFguCEiIuViuFEi8ympgGhA42tpNp+S8tN6IMDLQ47KiIiIHI7hRolamCl16QUzpUumhxMRESkJw40SmcfbtBhueEqKiIiUi+FGiSwL+FkPJj5XYp4GznBDRETKxXCjRC2clsourgIAdAvxvfwVREREisFwozRGI1B82nT/snBztrgaANA12MfZVRERETkNw43SlJ0DDHWAWgMEdbU0CyEsPTdx7LkhIiIFY7hRGvMpqeDugEptab5YrUdFbT0A9twQEZGyMdwozVXG20QGesHLU335q4iIiBSD4UZpWgg3ZxvCTWxn9toQEZGyMdwoTQtr3GQXmQYTx3XmeBsiIlI2hhulMc+UumyNm8aeG4YbIiJSNoYbJdHXmGZLAU1PS5WYe254WoqIiJSN4UZJSs4AEIBXEODT2eop8xo37LkhIiKlY7hRkksHE19yYcyyGj1KqnQAOKCYiIiUj+FGScyDiS8bb5PT0GsT6q+Fr9bD2VURERE5FcONklguu9DDqtm8xk0sF+8jIiI3wHCjJMXNTwPnTCkiInInDDdKYhlzY31aKruYM6WIiMh9yB5uVqxYgbi4OHh5eSExMRF79uy54valpaWYNm0aIiMjodVq0bt3b2zYsMFJ1XZgVcVAzUXT/eDuVk+da5gG3pXhhoiI3ICso0vXrFmD1NRUrFy5EomJiVi+fDlSUlKQmZmJsLCwJtvrdDrcdtttCAsLw9q1axEdHY2zZ88iKCjI+cV3NOZem8AYQGMdYnLLagAA0UHezq6KiIjI6WQNN8uWLcOUKVMwefJkAMDKlSuxfv16rFq1Cs8991yT7VetWoWSkhLs2rULnp6eAIC4uLgrfkZdXR3q6uosj8vLy+23Ax2JZbyN9WBig1Egv6wWABDFcENERG5AttNSOp0O+/fvR3JycmMxKhWSk5ORnp7e7Gv+85//ICkpCdOmTUN4eDj69++PRYsWwWAwtPg5aWlpCAwMtNxiYmLsvi8dQkmW6WewdbgpqqyD3iCgVkkI89fKUBgREZFzyRZuioqKYDAYEB4ebtUeHh6O/Pz8Zl9z5swZrF27FgaDARs2bMC8efOwdOlS/O1vf2vxc+bOnYuysjLL7dy5c3bdjw6jouF3FhBl1ZxbajolFe6vhYda9iFWREREDudSK7oZjUaEhYXhvffeg1qtRkJCAs6fP4/XXnsNCxYsaPY1Wq0WWq0b9FhUNoQb/wir5txSnpIiIiL3Ilu4CQkJgVqtRkFBgVV7QUEBIiIimn1NZGQkPD09oVarLW39+vVDfn4+dDodNBqNQ2vu0CpaCjemnhuGGyIicheynafQaDRISEjAli1bLG1GoxFbtmxBUlJSs68ZMWIETp06BaPRaGk7ceIEIiMj3TvYAEBFnumnf6RV83mGGyIicjOyDsJITU3F+++/j48//hjHjh3DU089haqqKsvsqYkTJ2Lu3LmW7Z966imUlJRgxowZOHHiBNavX49FixZh2rRpcu1Cx1CvA6qLTff9mu+5iQ7ycnZVREREspB1zM348eNx4cIFzJ8/H/n5+Rg8eDA2btxoGWSck5MDlaoxf8XExOCHH37AzJkzMXDgQERHR2PGjBmYM2eOXLvQMVQ2nNpTeQI+wVZPmde4Yc8NERG5C5vCzbZt2zBy5Ei7FDB9+nRMnz692ee2b9/epC0pKQm//PKLXT5bMczhxj8CkCSrpzigmIiI3I1Np6VGjx6NHj164G9/+5typ1a7Est4G+tTUjU6A0qqdAAYboiIyH3YFG7Onz+P6dOnY+3atejevTtSUlLw5ZdfQqfT2bs+ao2WZko1nJLy1agR4OVSs/6JiIhsZlO4CQkJwcyZM5GRkYHdu3ejd+/emDp1KqKiovD000/j0KFD9q6TrsQcbi4bTJx3ySkp6bLTVURERErV7tlSQ4cOxdy5czF9+nRUVlZi1apVSEhIwA033IAjR47Yo0a6Gq5xQ0REZGFzuNHr9Vi7di1uv/12xMbG4ocffsA777yDgoICnDp1CrGxsbj//vvtWSu1hGvcEBERWdg0EOMvf/kLPv/8cwgh8Mgjj2DJkiXo37+/5XlfX1+8/vrriIqKusK7kN1YZktZX6eLa9wQEZE7sincHD16FG+//TbuueeeFq/bFBISgm3btrWrOGqlFnpuuMYNERG5I5vCzaWXTGjxjT08cNNNN9ny9tQWV1ydmGvcEBGR+7FpzE1aWhpWrVrVpH3VqlV49dVX210UtUELqxPr6o04f9F8WorhhoiI3IdN4eaf//wn+vbt26T9mmuuwcqVK9tdFLWBZaZUpNXqxIdzy6AzGBHsq0GXTgw3RETkPmwKN/n5+YiMjGzSHhoairy8vHYXRW1gGW9jPZh4X3YJAGBYbCeucUNERG7FpnATExODnTt3NmnfuXMnZ0g526XXlbrE3uyLAIBr44IvfwUREZGi2TSgeMqUKXjmmWeg1+txyy23ADANMv7rX/+KZ5991q4F0lU0M1NKCIH9Z03hJiGukxxVERERycamcDN79mwUFxdj6tSplutJeXl5Yc6cOZg7d65dC6SrqGjoufFrPC11pqgKJVU6aD1U6B8VKFNhRERE8rAp3EiShFdffRXz5s3DsWPH4O3tjV69erW45g05UEWu6eclPTfm8TaDY4Kg8Wj3FTaIiIhcSrsuFe3n54drr73WXrVQW9XrgN/3me6H9rE0c7wNERG5M5vDzb59+/Dll18iJyfHcmrKbN26de0ujFohZxdQVw74hgJRQy3NlplSHG9DRERuyKZzFl988QWuu+46HDt2DF9//TX0ej2OHDmCrVu3IjCQYzycJnOj6WevFEBlOpQF5bXILq6GJAFDYxluiIjI/dgUbhYtWoQ33ngD3333HTQaDd58800cP34cDzzwALp27WrvGqk5QgAnvjfd7zO6oUnglf8eBQAM7BKEAC9PuaojIiKSjU3h5vTp0xg7diwAQKPRoKqqCpIkYebMmXjvvffsWiC14EImcDEbUGuA7iMBAN9m5OK/v+ZBrZLw8h3XyFsfERGRTGwKN506dUJFRQUAIDo6GocPHwYAlJaWorq62n7VubmDORfx0c4s6A3Gpk+ae2263Qho/fD7xWrM+9Z0HJ6+pRcGxwQ5r1AiIqIOxKYBxTfeeCM2bdqEAQMG4P7778eMGTOwdetWbNq0Cbfeequ9a3RbL35zGEdyy1F9sRDTAncC9bWNTx7+t+ln79EwGAWe/fIQKmrrMaRrEKaN7CFPwURERB2ATeHmnXfeQW2t6Yv2hRdegKenJ3bt2oV7770XL774ol0LdGf5ZabfceDuJYB6czNbSEDv0fjgpzPYnVUCH40ay8cPhoeaa9sQEZH7anO4qa+vx3//+1+kpKQAAFQqFZ577jm7F+bujEaBi9U6AALJqv0AgN+jRkOnbVy75mLwIBzPNOL1/2UCABaMi0dsZ185yiUiIuow2hxuPDw88OSTT+LYsWOOqIcalNfqYRRAfykLEdJFVAktbj3zIOqguWxL0zibUfHheGBYjPMLJSIi6mBsOi01fPhwZGRkIDY21t71UIOSKtPCiLdrMgAAx32GYVhMZLPbdvLRYOGd/SFJkrPKIyIi6rBsCjdTp05Famoqzp07h4SEBPj6Wp8KGThwoF2Kc2emU1LAraoDgAASRj2ET4f8QeaqiIiIOj6bws2DDz4IAHj66actbZIkQQgBSZJgMBjsU50bK6nSIxwl6CPOAJBMqxATERHRVdkUbrKysuxdB13mYpUOt6oPmh50GQb4hcpbEBERkYuwKdxwrI3jlVTrcIvqgOlB79HyFkNERORCbAo3n3zyyRWfnzhxok3FUKOL1ToMVDX0kHW/WdZaiIiIXIlN4WbGjBlWj/V6Paqrq6HRaODj48NwYwfV5RcRJpWaHoT0lrUWIiIiV2LTUrYXL160ulVWViIzMxPXX389Pv/8c3vX6JY0ZaZemxptCOAVIHM1RERErsNu6/T36tULixcvbtKrQ7bxq8wGANT4x8laBxERkaux60WIPDw8kJuba8+3dFvBNWcBAPWdespcCRERkWuxaczNf/7zH6vHQgjk5eXhnXfewYgRI+xSmLsL0/8OAJBCGG6IiIjawqZwc9ddd1k9liQJoaGhuOWWW7B06VJ71OXW6g1GxBjPAypAG9FH7nKIiIhcik3hxmg02rsOukRZtQ7dpDwAgA/DDRERUZvYdcwN2Uf5hRz4SnWohwoeId3lLoeIiMil2BRu7r33Xrz66qtN2pcsWYL777+/3UW5u9r8kwCAPCkCUHvKXA0REZFrsSnc/Pjjj7j99tubtI8ZMwY//vhju4tyd8YLJwAABZ7RMldCRETkemwKN5WVldBoNE3aPT09UV5e3u6i3J364mkAQIkXr+FFRETUVjaFmwEDBmDNmjVN2r/44gvEx8e3uyh351V+BgBQ4ctwQ0RE1FY2zZaaN28e7rnnHpw+fRq33HILAGDLli34/PPP8dVXX9m1QHcUUGVawK8mkIOJiYiI2sqmcDNu3Dh88803WLRoEdauXQtvb28MHDgQmzdvxk033WTvGt1LvQ6BdaZp4AauTkxERNRmNoUbABg7dizGjh1rz1oIAC5mQw0DKoUXvDpFyl0NERGRy7FpzM3evXuxe/fuJu27d+/Gvn372l2UWys2TQPPEhHo5KuVuRgiIiLXY1O4mTZtGs6dO9ek/fz585g2bVq7i3JrxacAAGdEFIJ9m85IIyIioiuzKdwcPXoUQ4cObdI+ZMgQHD16tN1FubWiS3tuGG6IiIjayqZwo9VqUVBQ0KQ9Ly8PHh42D+MhAMaihp4bYxSCfRhuiIiI2sqmcDNq1CjMnTsXZWVllrbS0lI8//zzuO222+xWnDsSDaelshGBAG9eeoGIiKitbOpmef3113HjjTciNjYWQ4YMAQBkZGQgPDwc/+///T+7FuhWasugrr4AACj1joVaJclcEBERkeuxKdxER0fj119/xaeffopDhw7B29sbkydPxoQJE+Dpyd4GmzX02hSIIHj7BclbCxERkYuyeYCMr68vrr/+enTt2hU6nQ4A8P333wMA7rjjDvtU524uGW8T4s/xNkRERLawKdycOXMGd999N3777TdIkgQhBCSp8RSKwWCwW4FupaHnJktEoDPXuCEiIrKJTQOKZ8yYgW7duqGwsBA+Pj44fPgwduzYgWHDhmH79u12LtGNNCzgd0ZEIsSP4YaIiMgWNvXcpKenY+vWrQgJCYFKpYJarcb111+PtLQ0PP300zh48KC963QPlgX8IpHgx9NSREREtrCp58ZgMMDf3x8AEBISgtzcXABAbGwsMjMz2/x+K1asQFxcHLy8vJCYmIg9e/a06nVffPEFJEnCXXfd1ebP7HCMRqD4NAAgS0QilD03RERENrEp3PTv3x+HDh0CACQmJmLJkiXYuXMnFi5ciO7du7fpvdasWYPU1FQsWLAABw4cwKBBg5CSkoLCwsIrvi47OxuzZs3CDTfcYMsudDwVeYC+GvVQ45wIRWf23BAREdnEpnDz4osvwmg0AgAWLlyIrKws3HDDDdiwYQPeeuutNr3XsmXLMGXKFEyePBnx8fFYuXIlfHx8sGrVqhZfYzAY8PDDD+Pll1++apiqq6tDeXm51a1Dyd4JvH8r8PE4AMB5KRz18EBn9twQERHZxKYxNykpKZb7PXv2xPHjx1FSUoJOnTpZzZq6Gp1Oh/3792Pu3LmWNpVKheTkZKSnp7f4uoULFyIsLAyPP/44fvrppyt+RlpaGl5++eVW1+R0Bz4GzjdeSX2foRcAIIQ9N0RERDaxqeemOcHBwW0KNgBQVFQEg8GA8PBwq/bw8HDk5+c3+5qff/4ZH374Id5///1WfYb5MhHmW3NXM5dVTanpZ9J0VD64Ds/rJgMAZ0sRERHZyKWucllRUYFHHnkE77//PkJCQlr1Gq1WC622AweF2obrc8UMR2HnRNRhB/y0HvDyVMtbFxERkYuSNdyEhIRArVY3ucJ4QUEBIiIimmx/+vRpZGdnY9y4cZY289gfDw8PZGZmokePHo4t2t7qGsYAaQNQVGla6ZmDiYmIiGxnt9NSttBoNEhISMCWLVssbUajEVu2bEFSUlKT7fv27YvffvsNGRkZltsdd9yBkSNHIiMjAzExMc4s3z5qG8KNVyCKK+sA8JQUERFRe8h+Wio1NRWTJk3CsGHDMHz4cCxfvhxVVVWYPNk09mTixImIjo5GWloavLy80L9/f6vXBwUFAUCTdpdhPi3lFYiiqoaeG1/23BAREdlK9nAzfvx4XLhwAfPnz0d+fj4GDx6MjRs3WgYZ5+TkQKWStYPJcYwGQFdhuu8ViKKKEgDgNHAiIqJ2kD3cAMD06dMxffr0Zp+72rWqVq9ebf+CnKXukjV3tAEorsoDAIRyzA0REZHNFNol4iLM4208vAAPDYoqzAOK2XNDRERkK4YbOdU1DiYGgOIqDigmIiJqL4YbOZkHE2sDAIBTwYmIiOyA4UZOtdY9N0WcCk5ERNRuDDdyspyWCkBdvQEVtfUAeF0pIiKi9mC4kdMlp6WKG05JeagkBHp7ylgUERGRa+sQU8EVofA48P1fm7b7hgBjlph+Xs5qdeLG8TZtvQApERERNWK4sZe6CiBrR/PPdR8JDH2kaXttqemnV4BlvE1nX463ISIiag+GG3sJ7g7c+6F12y/vAuf3AfW1zb/mkqngx/JN9yMDvRxYJBERkfIx3NiLb2dgwH3WbZnfm8KNQd/8a8ynpbSB+O4X0+rEyfHhDiySiIhI+Tig2JFUDdnRWN/88w0DivPqPHEsrxyeaglj+kc4qTgiIiJlYrhxJLU53LTQc9NwWir9vAEAcFPvUAT5cBo4ERFRezDcOJKqYUq34co9N9uyTWNy7hgc7YyqiIiIFI3hxpHUDeGmmZ4bXb0R+mpTuDlVroKPRo3kfmHOrI6IiEiRGG4cydJz0zTcTF69B4bqUgBAufDBbfHh8NFwfDcREVF7Mdw4kkpt+tnMgOLTuSXwkkyhJyQ0FFNu6O7MyoiIiBSLXQWOZDkt1TTceNRXACpAQMK3M8cAKuZMIiIie2C4sZODORfx4Hu/WLVNV2XjLyqgrLIagZe0CyGgqa8ANIDQ+EFisCEiIrIbhhs7EQDq6o1WbbVqNaACCksrrcKNzmCEn6g2PfAKcFqNRERE7oDhxk6uiQrAzudusWr75ZNtQAkgLhtQXKs3wl8yh5tAEBERkf0w3NiJ1kON6CBv6zat6SKYxsvWuanVGxAAU7iRGG6IiIjsioM9HEjt0fw6N7V6g6XnhuGGiIjIvhhuHEjlYbqUwuWnpWr0Bvg39NxAyzE3RERE9sRw40At99wYESBxQDEREZEjMNw4kLqh50a6wpgbDigmIiKyL4YbB1J7Nlzh29jMaSmpxvSAp6WIiIjsiuHGgTw8ml+huE5vQACqTA/Yc0NERGRXDDcO5NHQc6O6LNyYBhQ39NxwzA0REZFdMdw4kGdDuJHE5WNujAiQGnputOy5ISIisieGGwfy0LTQc6O7ZCo4e26IiIjsiuHGgTSephWKVZf13NTp6hAhlZge+Ec4uywiIiJFY7hxIE9P04BilTBYtWsrfodGMkAvaYCALnKURkREpFgMNw7kqTH13Khh3XPjW3UWAHDRKwZQ8RAQERHZE79ZHUjTMOZGfdlpKf/KLABAqU+cs0siIiJSPIYbB9I0XBXcAwYYjcLS3qkmBwBQ7hsnR1lERESKxnDjQFpNY7ipqzda2oNrTeGm2j9OjrKIiIgUjeHGgbSWMTcG1OobBxWH6s4BAGoDu8tSFxERkZIx3DiQ+argnjCgxhxu6ioRbCgCAOgZboiIiOyO4caR1KZw43Fpz03JaQBAkQiA2reTXJUREREpFsONI6kaem4kA2p0DTOmik4CALJEBLw81XJVRkREpFgMN46k9rDcrdXpTHeKTT03Z4xRDDdEREQOwHDjSKrGcKOrqzPdKWbPDRERkSMx3DhSw2kpAKiz9NycAgCcEZHwZrghIiKyO4YbR1I3hhtdXR0gBFDUGG68PPnrJyIisjd+uzqSSg0jJACmK4GjshDQVcAgJOSIcPbcEBEROQDDjYMZYBp3o9fpgGrT+jYX4Q8dPKFluCEiIrI7hhsHM0qmAKPX1wH1pkHFuobAw54bIiIi+2O4cTCjdEnPjcE0qFgnPKGSAE+1JGdpREREisRw42DGhungOr3OqufG21MNSWK4ISIisjePq29C7WHuuanX6wCDafaUDp5c44aIiMhBGG4cTDT03Oj1OqC+oRcHHgw3REREDsJw42DmcGPQ6wCDKdCYem54RpCIiMgRGG4czXxaql4H1DeEG8GeGyIiIkdhuHEw0bBKsUGvBwym3po6eHIaOBERkYMw3Dia+bRUvQ6oN82O4pgbIiIix2G4cTDJ3HNTrwcMpjbOliIiInKcDjGqdcWKFYiLi4OXlxcSExOxZ8+eFrd9//33ccMNN6BTp07o1KkTkpOTr7i97BrCjbFe37jOjeCAYiIiIkeR/Rt2zZo1SE1NxYIFC3DgwAEMGjQIKSkpKCwsbHb77du3Y8KECdi2bRvS09MRExODUaNG4fz5806uvHWkhtNSov6SFYobFvEjIiIi+5M93CxbtgxTpkzB5MmTER8fj5UrV8LHxwerVq1qdvtPP/0UU6dOxeDBg9G3b1988MEHMBqN2LJli5Mrbx3LaSnDJT03PC1FRETkMLKGG51Oh/379yM5OdnSplKpkJycjPT09Fa9R3V1NfR6PYKDg5t9vq6uDuXl5VY3Z1J5mMKNMOite240DDdERESOIGu4KSoqgsFgQHh4uFV7eHg48vPzW/Uec+bMQVRUlFVAulRaWhoCAwMtt5iYmHbX3RYqtQZAQ7i5tOfGQ/ZOMyIiIkVy6W/YxYsX44svvsDXX38NLy+vZreZO3cuysrKLLdz5845tcbGnpt6iIZwUyc84MWeGyIiIoeQdSp4SEgI1Go1CgoKrNoLCgoQERFxxde+/vrrWLx4MTZv3oyBAwe2uJ1Wq4VWq7VLvbYwhxtP1MNYXwc1zD03DDdERESOIGvPjUajQUJCgtVgYPPg4KSkpBZft2TJErzyyivYuHEjhg0b5oxSbaZuCDdqGGHQ1QLgIn5ERESOJPsifqmpqZg0aRKGDRuG4cOHY/ny5aiqqsLkyZMBABMnTkR0dDTS0tIAAK+++irmz5+Pzz77DHFxcZaxOX5+fvDz85NtP1piHnPjCQOMl4y58da49BlBIiKiDkv2cDN+/HhcuHAB8+fPR35+PgYPHoyNGzdaBhnn5ORApWoMAu+++y50Oh3uu+8+q/dZsGABXnrpJWeW3jpq06/YA/Uw6s2L+HnwtBQREZGDyB5uAGD69OmYPn16s89t377d6nF2drbjC7Inlem0lIdksAwo1sGTA4qJiIgchOdGHK1hhWJPXBpu2HNDRETkKAw3jtZwWkoNA1BvXsTPk4v4EREROQjDjaOpzFPBDYDRFG7qwAtnEhEROQq/YR2t4dpSHjBAMvfcCF44k4iIyFEYbhzNPKAY9ZAMvHAmERGRozHcOJq6cUCxZGy8cCbDDRERkWMw3Dhaw2wptWSEZGgcUMwxN0RERI7Bb1hHUzVeW0rd0HOjhwc0av7qiYiIHIHfsI5mWaHYALXQAwBUnl6QJEnOqoiIiBSL4cbRGnpuvKCDCgIAEBLY8a6BRUREpBQMN47WMBXcR6qzNKWOGSBXNURERIrHcONoDQOKfVFrabq+T7Rc1RARESkew42jNYQbn4ZwIySVZRwOERER2R/DjaM1nJaK8DYCACS1Vs5qiIiIFI/hxtEaBhSbe27goZGxGCIiIuVjuHE08ykoXWXDY/bcEBERORLDjaM1jLmBMJ2WggfDDRERkSMx3Dhaw2kpCzVPSxERETkSw42jqS8LN+y5ISIiciiGG0dTXTbtmz03REREDsVw42jsuSEiInIqhhtHY88NERGRUzHcONrlA4rZc0NERORQDDeOdvmlFrjODRERkUPxIkeO1qTnhqeliIiUzGAwQK/Xy12GS9JoNFCp2t/vwnDjaJcPKGbPDRGRIgkhkJ+fj9LSUrlLcVkqlQrdunWDRtO+jgCGG0e7fEAxe26IiBTJHGzCwsLg4+MDSZLkLsmlGI1G5ObmIi8vD127dm3X74/hxtGazJZizw0RkdIYDAZLsOncubPc5bis0NBQ5Obmor6+Hp6enld/QQs4oNjRuM4NEZHimcfY+Pj4yFyJazOfjjIYDO16H4YbR+O1pYiI3AZPRbWPvX5/DDeOplJbP2bPDRERkUMx3DiaJFn33rDnhoiIFCouLg7Lly+XuwwOKHYKlQdgbFjzgD03RETUgdx8880YPHiwXULJ3r174evr2/6i2onhxhnUnkB9TcN99twQEZHrEELAYDDAw+PqkSE0NNQJFV0dT0s5w6XTwdlzQ0SkeEIIVOvqZbkJIVpd56OPPoodO3bgzTffhCRJkCQJq1evhiRJ+P7775GQkACtVouff/4Zp0+fxp133onw8HD4+fnh2muvxebNm63e7/LTUpIk4YMPPsDdd98NHx8f9OrVC//5z3/s9WtuEXtunOHS6eBc54aISPFq9AbEz/9Bls8+ujAFPprWfb2/+eabOHHiBPr374+FCxcCAI4cOQIAeO655/D666+je/fu6NSpE86dO4fbb78df//736HVavHJJ59g3LhxyMzMRNeuXVv8jJdffhlLlizBa6+9hrfffhsPP/wwzp49i+Dg4PbvbAvYc+MMlw4o5grFRETUQQQGBkKj0cDHxwcRERGIiIiAWm2a5btw4ULcdttt6NGjB4KDgzFo0CD8+c9/Rv/+/dGrVy+88sor6NGjx1V7Yh599FFMmDABPXv2xKJFi1BZWYk9e/Y4dL/Yc+MMl04HZ88NEZHieXuqcXRhimyfbQ/Dhg2zelxZWYmXXnoJ69evR15eHurr61FTU4OcnJwrvs/AgQMt9319fREQEIDCwkK71NgShhtnULPnhojInUiS1OpTQx3V5bOeZs2ahU2bNuH1119Hz5494e3tjfvuuw86ne6K73P5ZRQkSYLRaLR7vZdy7d+8q1BxzA0REXVMGo2mVZc72LlzJx599FHcfffdAEw9OdnZ2Q6uzjYcc+MMas6WIiKijikuLg67d+9GdnY2ioqKWuxV6dWrF9atW4eMjAwcOnQIDz30kMN7YGzFcOMMXKGYiIg6qFmzZkGtViM+Ph6hoaEtjqFZtmwZOnXqhOuuuw7jxo1DSkoKhg4d6uRqW0cSbZkQrwDl5eUIDAxEWVkZAgICnPOhH9wG/N4wMnzqL0BYP+d8LhEROUVtbS2ysrLQrVs3eHl5yV2Oy7rS77Et39/suXEGNXtuiIiInIXhxhm4QjEREZHTMNw4A1coJiIichqGG2fgCsVEREROw3DjDFyhmIiIyGkYbpyBA4qJiIichuHGGcynpVSegIq/ciIiIkfiN60zmHtuOFOKiIjI4RhunME8FZynpIiIiByO4cYZ2HNDRETkNAw3zsCeGyIi6qBuvvlmPPPMM3Z7v0cffRR33XWX3d7PFgw3zmAON+y5ISIicjiGG2cwn5biGjdERO5BCEBXJc+tDdfDfvTRR7Fjxw68+eabkCQJkiQhOzsbhw8fxpgxY+Dn54fw8HA88sgjKCoqsrxu7dq1GDBgALy9vdG5c2ckJyejqqoKL730Ej7++GN8++23lvfbvn27A37BV+Zx9U2o3cxTwbk6MRGRe9BXA4ui5Pns53MBjW+rNn3zzTdx4sQJ9O/fHwsXLgQAeHp6Yvjw4XjiiSfwxhtvoKamBnPmzMEDDzyArVu3Ii8vDxMmTMCSJUtw9913o6KiAj/99BOEEJg1axaOHTuG8vJyfPTRRwCA4OBgh+1qSzpEz82KFSsQFxcHLy8vJCYmYs+ePVfc/quvvkLfvn3h5eWFAQMGYMOGDU6q1EbsuSEiog4oMDAQGo0GPj4+iIiIQEREBN59910MGTIEixYtQt++fTFkyBCsWrUK27Ztw4kTJ5CXl4f6+nrcc889iIuLw4ABAzB16lT4+fnBz88P3t7e0Gq1lvfTaJz/H3vZe27WrFmD1NRUrFy5EomJiVi+fDlSUlKQmZmJsLCwJtvv2rULEyZMQFpaGv7v//4Pn332Ge666y4cOHAA/fv3l2EPWsF8+QX23BARuQdPH1MPilyf3Q6HDh3Ctm3b4Ofn1+S506dPY9SoUbj11lsxYMAApKSkYNSoUbjvvvvQqVOndn2uPcnec7Ns2TJMmTIFkydPRnx8PFauXAkfHx+sWrWq2e3ffPNNjB49GrNnz0a/fv3wyiuvYOjQoXjnnXecXHkbqNhzQ0TkViTJdGpIjpsktav0yspKjBs3DhkZGVa3kydP4sYbb4RarcamTZvw/fffIz4+Hm+//Tb69OmDrKwsO/3y2k/WcKPT6bB//34kJydb2lQqFZKTk5Gent7sa9LT0622B4CUlJQWt6+rq0N5ebnVzenUHHNDREQdk0ajgcFgsDweOnQojhw5gri4OPTs2dPq5utrGssjSRJGjBiBl19+GQcPHoRGo8HXX3/d7PvJQdZwU1RUBIPBgPDwcKv28PBw5OfnN/ua/Pz8Nm2flpaGwMBAyy0mJsY+xbdFt5uA4O5Avzuc/9lERERXEBcXh927dyM7OxtFRUWYNm0aSkpKMGHCBOzduxenT5/GDz/8gMmTJ8NgMGD37t1YtGgR9u3bh5ycHKxbtw4XLlxAv379LO/366+/IjMzE0VFRdDr9U7fJ9lPSzna3LlzUVZWZrmdO3fO+UWExwNPHwQGPuD8zyYiIrqCWbNmQa1WIz4+HqGhodDpdNi5cycMBgNGjRqFAQMG4JlnnkFQUBBUKhUCAgLw448/4vbbb0fv3r3x4osvYunSpRgzZgwAYMqUKejTpw+GDRuG0NBQ7Ny50+n7JOuA4pCQEKjVahQUFFi1FxQUICIiotnXREREtGl7rVYLrZZjXYiIiJrTu3fvZod2rFu3rtnt+/Xrh40bN7b4fqGhofjf//5nt/psIWvPjUajQUJCArZs2WJpMxqN2LJlC5KSkpp9TVJSktX2ALBp06YWtyciIiL3IvtU8NTUVEyaNAnDhg3D8OHDsXz5clRVVWHy5MkAgIkTJyI6OhppaWkAgBkzZuCmm27C0qVLMXbsWHzxxRfYt28f3nvvPTl3g4iIiDoI2cPN+PHjceHCBcyfPx/5+fkYPHgwNm7caBk0nJOTA5WqsYPpuuuuw2effYYXX3wRzz//PHr16oVvvvmm465xQ0RERE4lCdGGi1AoQHl5OQIDA1FWVoaAgAC5yyEiIgWora1FVlYWunXrBi8vL7nLcVlX+j225ftb8bOliIiInMXN+gvszl6/P4YbIiKidvL0NC3WWl1dLXMlrk2n0wEA1Gp1u95H9jE3RERErk6tViMoKAiFhYUAAB8fH0jtvAyCuzEajbhw4QJ8fHzg4dG+eMJwQ0REZAfm9dbMAYfaTqVSoWvXru0Ohgw3REREdiBJEiIjIxEWFibLJQeUQKPRWM2QthXDDRERkR2p1ep2jxmh9uGAYiIiIlIUhhsiIiJSFIYbIiIiUhS3G3NjXiCovLxc5kqIiIiotczf261Z6M/twk1FRQUAICYmRuZKiIiIqK0qKioQGBh4xW3c7tpSRqMRubm58Pf3t/sCS+Xl5YiJicG5c+cUed0qpe8fwH1UAqXvH8B9VAKl7x9g/30UQqCiogJRUVFXnS7udj03KpUKXbp0cehnBAQEKPYPK6D8/QO4j0qg9P0DuI9KoPT9A+y7j1frsTHjgGIiIiJSFIYbIiIiUhSGGzvSarVYsGABtFqt3KU4hNL3D+A+KoHS9w/gPiqB0vcPkHcf3W5AMRERESkbe26IiIhIURhuiIiISFEYboiIiEhRGG6IiIhIURhu7GTFihWIi4uDl5cXEhMTsWfPHrlLsllaWhquvfZa+Pv7IywsDHfddRcyMzOttrn55pshSZLV7cknn5Sp4rZ56aWXmtTet29fy/O1tbWYNm0aOnfuDD8/P9x7770oKCiQseK2i4uLa7KPkiRh2rRpAFzz+P34448YN24coqKiIEkSvvnmG6vnhRCYP38+IiMj4e3tjeTkZJw8edJqm5KSEjz88MMICAhAUFAQHn/8cVRWVjpxL1p2pf3T6/WYM2cOBgwYAF9fX0RFRWHixInIzc21eo/mjvvixYudvCctu9oxfPTRR5vUP3r0aKttOvIxBK6+j839vZQkCa+99pplm458HFvz/dCaf0NzcnIwduxY+Pj4ICwsDLNnz0Z9fb3d6mS4sYM1a9YgNTUVCxYswIEDBzBo0CCkpKSgsLBQ7tJssmPHDkybNg2//PILNm3aBL1ej1GjRqGqqspquylTpiAvL89yW7JkiUwVt90111xjVfvPP/9seW7mzJn47rvv8NVXX2HHjh3Izc3FPffcI2O1bbd3716r/du0aRMA4P7777ds42rHr6qqCoMGDcKKFSuafX7JkiV46623sHLlSuzevRu+vr5ISUlBbW2tZZuHH34YR44cwaZNm/Df//4XP/74I/70pz85axeu6Er7V11djQMHDmDevHk4cOAA1q1bh8zMTNxxxx1Ntl24cKHVcf3LX/7ijPJb5WrHEABGjx5tVf/nn39u9XxHPobA1ffx0n3Ly8vDqlWrIEkS7r33XqvtOupxbM33w9X+DTUYDBg7dix0Oh127dqFjz/+GKtXr8b8+fPtV6igdhs+fLiYNm2a5bHBYBBRUVEiLS1Nxqrsp7CwUAAQO3bssLTddNNNYsaMGfIV1Q4LFiwQgwYNava50tJS4enpKb766itL27FjxwQAkZ6e7qQK7W/GjBmiR48ewmg0CiFc+/gJIQQA8fXXX1seG41GERERIV577TVLW2lpqdBqteLzzz8XQghx9OhRAUDs3bvXss33338vJEkS58+fd1rtrXH5/jVnz549AoA4e/aspS02Nla88cYbji3OTprbx0mTJok777yzxde40jEUonXH8c477xS33HKLVZsrHcfLvx9a82/ohg0bhEqlEvn5+ZZt3n33XREQECDq6ursUhd7btpJp9Nh//79SE5OtrSpVCokJycjPT1dxsrsp6ysDAAQHBxs1f7pp58iJCQE/fv3x9y5c1FdXS1HeTY5efIkoqKi0L17dzz88MPIyckBAOzfvx96vd7qePbt2xddu3Z12eOp0+nwr3/9C4899pjVxWJd+fhdLisrC/n5+VbHLTAwEImJiZbjlp6ejqCgIAwbNsyyTXJyMlQqFXbv3u30mturrKwMkiQhKCjIqn3x4sXo3LkzhgwZgtdee82uXf3OsH37doSFhaFPnz546qmnUFxcbHlOacewoKAA69evx+OPP97kOVc5jpd/P7Tm39D09HQMGDAA4eHhlm1SUlJQXl6OI0eO2KUut7twpr0VFRXBYDBYHSQACA8Px/Hjx2Wqyn6MRiOeeeYZjBgxAv3797e0P/TQQ4iNjUVUVBR+/fVXzJkzB5mZmVi3bp2M1bZOYmIiVq9ejT59+iAvLw8vv/wybrjhBhw+fBj5+fnQaDRNvjDCw8ORn58vT8Ht9M0336C0tBSPPvqopc2Vj19zzMemub+H5ufy8/MRFhZm9byHhweCg4Nd7tjW1tZizpw5mDBhgtUFCZ9++mkMHToUwcHB2LVrF+bOnYu8vDwsW7ZMxmpbb/To0bjnnnvQrVs3nD59Gs8//zzGjBmD9PR0qNVqRR1DAPj444/h7+/f5LS3qxzH5r4fWvNvaH5+frN/V83P2QPDDV3RtGnTcPjwYasxKQCsznEPGDAAkZGRuPXWW3H69Gn06NHD2WW2yZgxYyz3Bw4ciMTERMTGxuLLL7+Et7e3jJU5xocffogxY8YgKirK0ubKx8/d6fV6PPDAAxBC4N1337V6LjU11XJ/4MCB0Gg0+POf/4y0tDSXWOb/wQcftNwfMGAABg4ciB49emD79u249dZbZazMMVatWoWHH34YXl5eVu2uchxb+n7oCHhaqp1CQkKgVqubjAQvKChARESETFXZx/Tp0/Hf//4X27ZtQ5cuXa64bWJiIgDg1KlTzijNroKCgtC7d2+cOnUKERER0Ol0KC0ttdrGVY/n2bNnsXnzZjzxxBNX3M6Vjx8Ay7G50t/DiIiIJoP86+vrUVJS4jLH1hxszp49i02bNln12jQnMTER9fX1yM7Odk6Bdta9e3eEhIRY/lwq4Ria/fTTT8jMzLzq302gYx7Hlr4fWvNvaERERLN/V83P2QPDTTtpNBokJCRgy5Ytljaj0YgtW7YgKSlJxspsJ4TA9OnT8fXXX2Pr1q3o1q3bVV+TkZEBAIiMjHRwdfZXWVmJ06dPIzIyEgkJCfD09LQ6npmZmcjJyXHJ4/nRRx8hLCwMY8eOveJ2rnz8AKBbt26IiIiwOm7l5eXYvXu35bglJSWhtLQU+/fvt2yzdetWGI1GS7jryMzB5uTJk9i8eTM6d+581ddkZGRApVI1OZXjKn7//XcUFxdb/ly6+jG81IcffoiEhAQMGjToqtt2pON4te+H1vwbmpSUhN9++80qqJrDenx8vN0KpXb64osvhFarFatXrxZHjx4Vf/rTn0RQUJDVSHBX8tRTT4nAwECxfft2kZeXZ7lVV1cLIYQ4deqUWLhwodi3b5/IysoS3377rejevbu48cYbZa68dZ599lmxfft2kZWVJXbu3CmSk5NFSEiIKCwsFEII8eSTT4quXbuKrVu3in379omkpCSRlJQkc9VtZzAYRNeuXcWcOXOs2l31+FVUVIiDBw+KgwcPCgBi2bJl4uDBg5bZQosXLxZBQUHi22+/Fb/++qu48847Rbdu3URNTY3lPUaPHi2GDBkidu/eLX7++WfRq1cvMWHCBLl2ycqV9k+n04k77rhDdOnSRWRkZFj9vTTPLtm1a5d44403REZGhjh9+rT417/+JUJDQ8XEiRNl3rNGV9rHiooKMWvWLJGeni6ysrLE5s2bxdChQ0WvXr1EbW2t5T068jEU4up/ToUQoqysTPj4+Ih33323yes7+nG82veDEFf/N7S+vl70799fjBo1SmRkZIiNGzeK0NBQMXfuXLvVyXBjJ2+//bbo2rWr0Gg0Yvjw4eKXX36RuySbAWj29tFHHwkhhMjJyRE33nijCA4OFlqtVvTs2VPMnj1blJWVyVt4K40fP15ERkYKjUYjoqOjxfjx48WpU6csz9fU1IipU6eKTp06CR8fH3H33XeLvLw8GSu2zQ8//CAAiMzMTKt2Vz1+27Zta/bP5aRJk4QQpung8+bNE+Hh4UKr1Ypbb721yb4XFxeLCRMmCD8/PxEQECAmT54sKioqZNibpq60f1lZWS3+vdy2bZsQQoj9+/eLxMREERgYKLy8vES/fv3EokWLrIKB3K60j9XV1WLUqFEiNDRUeHp6itjYWDFlypQm/0nsyMdQiKv/ORVCiH/+85/C29tblJaWNnl9Rz+OV/t+EKJ1/4ZmZ2eLMWPGCG9vbxESEiKeffZZodfr7Van1FAsERERkSJwzA0REREpCsMNERERKQrDDRERESkKww0REREpCsMNERERKQrDDRERESkKww0REREpCsMNERERKQrDDRG5ne3bt0OSpCYX9yMiZWC4ISIiIkVhuCEiIiJFYbghIqczGo1IS0tDt27d4O3tjUGDBmHt2rUAGk8ZrV+/HgMHDoSXlxf+8Ic/4PDhw1bv8e9//xvXXHMNtFot4uLisHTpUqvn6+rqMGfOHMTExECr1aJnz5748MMPrbbZv38/hg0bBh8fH1x33XXIzMy0PHfo0CGMHDkS/v7+CAgIQEJCAvbt2+eg3wgR2RPDDRE5XVpaGj755BOsXLkSR44cwcyZM/HHP/4RO3bssGwze/ZsLF26FHv37kVoaCjGjRsHvV4PwBRKHnjgATz44IP47bff8NJLL2HevHlYvXq15fUTJ07E559/jrfeegvHjh3DP//5T/j5+VnV8cILL2Dp0qXYt28fPDw88Nhjj1mee/jhh9GlSxfs3bsX+/fvx3PPPQdPT0/H/mKIyD7sdn1xIqJWqK2tFT4+PmLXrl1W7Y8//riYMGGC2LZtmwAgvvjiC8tzxcXFwtvbW6xZs0YIIcRDDz0kbrvtNqvXz549W8THxwshhMjMzBQAxKZNm5qtwfwZmzdvtrStX79eABA1NTVCCCH8/f3F6tWr27/DROR07LkhIqc6deoUqqurcdttt8HPz89y++STT3D69GnLdklJSZb7wcHB6NOnD44dOwYAOHbsGEaMGGH1viNGjMDJkydhMBiQkZEBtVqNm2666Yq1DBw40HI/MjISAFBYWAgASE1NxRNPPIHk5GQsXrzYqjYi6tgYbojIqSorKwEA69evR0ZGhuV29OhRy7ib9vL29m7VdpeeZpIkCYBpPBAAvPTSSzhy5AjGjh2LrVu3Ij4+Hl9//bVd6iMix2K4ISKnio+Ph1arRU5ODnr27Gl1i4mJsWz3yy+/WO5fvHgRJ06cQL9+/QAA/fr1w86dO63ed+fOnejduzfUajUGDBgAo9FoNYbHFr1798bMmTPxv//9D/fccw8++uijdr0fETmHh9wFEJF78ff3x6xZszBz5kwYjUZcf/31KCsrw86dOxEQEIDY2FgAwMKFC9G5c2eEh4fjhRdeQEhICO666y4AwLPPPotrr70Wr7zyCsaPH4/09HS88847+Mc//gEAiIuLw6RJk/DYY4/hrbfewqBBg3D27FkUFhbigQceuGqNNTU1mD17Nu677z5069YNv//+O/bu3Yt7773XYb8XIrIjuQf9EJH7MRqNYvny5aJPnz7C09NThIaGipSUFLFjxw7LYN/vvvtOXHPNNUKj0Yjhw4eLQ4cOWb3H2rVrRXx8vPD09BRdu3YVr732mtXzNTU1YubMmSIyMlJoNBrRs2dPsWrVKiFE44DiixcvWrY/ePCgACCysrJEXV2dePDBB0VMTIzQaDQiKipKTJ8+3TLYmIg6NkkIIWTOV0REFtu3b8fIkSNx8eJFBAUFyV0OEbkgjrkhIiIiRWG4ISIiIkXhaSkiIiJSFPbcEBERkaIw3BAREZGiMNwQERGRojDcEBERkaIw3BAREZGiMNwQERGRojDcEBERkaIw3BAREZGi/H87fKTb/eyYLAAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Summary\n",
        "\n",
        "Let's define some things:\n",
        "\n",
        "$i$ = number of inputs\n",
        "\n",
        "Quadric neuron: $2i+1$ parameters, $3i$ multiplications, $2i+1$ additions\n",
        "\n",
        "Linear neuron: $i+1$ parameters, $i$ multiplications, $i+1$ additions\n",
        "\n",
        "$n$ = number of units in layer\n",
        "\n",
        "$p$ = number of parameters (weights and bias)\n",
        "\n",
        "$m$ = number of multiplications\n",
        "\n",
        "$a$ = number of additions\n",
        "\n",
        "Quadric layer: $p = n(2i+1)$, $m = n(3i)$, $a = n(2i+1)$\n",
        "\n",
        "Linear layer: $p = n(i+1)$, $m = ni$, $a = n(i+1)$\n",
        "\n",
        "Now let's compare the complexity of above models:\n",
        "\n",
        "Linear model:\n",
        " - linear hidden layer with 4 inputs and 8 units: $p_h = 40$, $m_h = 32$, $a_h = 40$\n",
        " - linear output layer with 8 inputs and 3 units: $p_h = 27$, $m_h = 24$, $a_h =27$\n",
        " - total: $p = 67$, $m = 56$, $a = 67$\n",
        "\n",
        "Quadric (hybrid) model:\n",
        "- quadric hidden layer with 4 inputs and 3 units: $p_h = 27$, $m_h = 36$, $a_h = 27$\n",
        "- linear output layer with 3 inputs and 3 units: $p_h = 12$, $m_h = 9$, $a_h = 12$\n",
        "- total: $p = 39$, $m = 45$, $a = 39$\n",
        "\n",
        "The results are by no means a scientific study but even in this small example we can see the following reductions:\n",
        "\n",
        "- reduction in model size (parameters): ~42%\n",
        "- reduction in multiplications: ~20%\n",
        "- reduction in additions: ~42%\n",
        "\n",
        "This is very encouraging for further studies in all kinds of applications like vision, transformers, GAN, LLMs etc. where we encounter many more layers and many more units per layer and will be able to significantly reduce model size by introducing quadric layers and hybrid models or even purely quadric models.\n",
        "\n",
        "In my humble opinion deep quadric learning is a valuable contribution towards much smaller models of the same level of performance with all the advantages mentioned in Episode 1 like democratization of access to AI, reduction and therefore decentralization of compute power, smaller carbon foot print etc.\n",
        "\n"
      ],
      "metadata": {
        "id": "36qo997EYKwP"
      }
    }
  ]
}