demo08.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "demo8.ipynb",
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "accelerator": "GPU"
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "NQ727lB7Bd-Q",
        "colab_type": "text"
      },
      "source": [
        "OK, thus far we have been talking about linear models. All these can be viewed as a single-layer neural net. The next step is to move on to multi-layer nets. Training these is a bit more involved, and implementing from scratch requires time and effort. Instead, we just use well-established libraries. I prefer PyTorch, which is based on an earlier library called Torch (designed for training neural nets via backprop)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "8zHrM6ihB1uA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "import numpy as np\n",
        "import torch\n",
        "import torchvision"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "M5vzZmhnCBDv",
        "colab_type": "text"
      },
      "source": [
        "Torch handles data types a bit differently. Everything in torch is a *tensor*."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "MvAolsJ6CJJs",
        "colab_type": "code",
        "outputId": "3fa32faf-beda-4415-ae6d-3901dc515eca",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 85
        }
      },
      "source": [
        "a = np.random.rand(2,3)\n",
        "print(a)\n",
        "\n",
        "b = torch.from_numpy(a)\n",
        "print(b)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "[[0.97329184 0.8184028  0.53794621]\n",
            " [0.32927566 0.48143315 0.27906962]]\n",
            "tensor([[0.9733, 0.8184, 0.5379],\n",
            "        [0.3293, 0.4814, 0.2791]], dtype=torch.float64)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "cuRIMzC-CYx-",
        "colab_type": "text"
      },
      "source": [
        "The idea in Torch is that tensors allow for easy forward (function evaluations) and backward (gradient) passes."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "gb2LWPSACqiS",
        "colab_type": "code",
        "outputId": "4e3c5277-c7db-4094-f54e-cc55481eef38",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 136
        }
      },
      "source": [
        "A = torch.rand(2,2)\n",
        "b = torch.rand(2,1)\n",
        "x = torch.rand(2,1, requires_grad=True)\n",
        "\n",
        "y = torch.matmul(A,x) + b\n",
        "\n",
        "print(y)\n",
        "z = y.sum()\n",
        "print(z)\n",
        "z.backward()\n",
        "print(x.grad)\n",
        "print(x)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "tensor([[0.9972],\n",
            "        [0.7688]], grad_fn=<AddBackward0>)\n",
            "tensor(1.7660, grad_fn=<SumBackward0>)\n",
            "tensor([[0.8651],\n",
            "        [1.3324]])\n",
            "tensor([[0.0328],\n",
            "        [0.7595]], requires_grad=True)\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "J4ptw5BlEHVL",
        "colab_type": "text"
      },
      "source": [
        "Notice how the backward pass computed the gradients using autograd. OK, enough background. Time to train some networks."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "zSTq5wKoEPlA",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "trainingdata = torchvision.datasets.FashionMNIST('./FashionMNIST/',train=True,download=True,transform=torchvision.transforms.ToTensor())\n",
        "testdata = torchvision.datasets.FashionMNIST('./FashionMNIST/',train=False,download=True,transform=torchvision.transforms.ToTensor())\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qMb0WQthE2vR",
        "colab_type": "code",
        "outputId": "7b47f0ba-8801-45c3-fc8a-77b889a98867",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "print(len(trainingdata),len(testdata))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "60000 10000\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Tty2F0rgE7lw",
        "colab_type": "text"
      },
      "source": [
        "OK, same size as MNIST. Let's display a few images."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rhjYbu_fE-Dz",
        "colab_type": "code",
        "outputId": "48753ba7-9d64-420a-e699-b41279baf66c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 34
        }
      },
      "source": [
        "image, label = trainingdata[0]\n",
        "print(image.shape, label)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "torch.Size([1, 28, 28]) 9\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "beaahX9xFQf2",
        "colab_type": "text"
      },
      "source": [
        "Hmm, it is a tensor, not an array. We need to postprocess to use matplotlib."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "iaEVjo9BFYQo",
        "colab_type": "code",
        "outputId": "235d0e8b-e81c-41d1-c13f-6607b415d087",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        }
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "plt.imshow(image.squeeze().numpy())\n",
        "plt.show()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPsAAAD4CAYAAAAq5pAIAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAAUFElEQVR4nO3da2yc1ZkH8P8z4/ElzjiJk+CE4BIu\noZDCEqhJuIlSKDREVQOli4gQCxLaoF3otl0+gGhXZb+sEFpAaNntroEsYVWoWhUERREFzCULlDQm\npOS2ITeHxDi2ExPbcTz2XJ794Bdqgs/zmnnnRs7/J1kezzNn5njGf78zc+acI6oKIjr+xcrdASIq\nDYadyBMMO5EnGHYiTzDsRJ6oKuWNVUuN1qK+lDdJ5JUUhjCqIzJRLVLYRWQpgEcAxAE8rqr3W5ev\nRT2WyJVRbpKIDOu0zVnL+2m8iMQB/DuAawAsBLBCRBbme31EVFxRXrMvBrBTVXer6iiAXwNYXphu\nEVGhRQn7PAD7xv28Pzjvc0RkpYi0i0h7GiMRbo6Ioij6u/Gq2qqqLarakkBNsW+OiByihL0TQPO4\nn08KziOiChQl7OsBLBCRU0SkGsCNAF4oTLeIqNDyHnpT1YyI3AngDxgbelulqlsK1jMiKqhI4+yq\nugbAmgL1hYiKiB+XJfIEw07kCYadyBMMO5EnGHYiTzDsRJ5g2Ik8wbATeYJhJ/IEw07kCYadyBMM\nO5EnGHYiT5R0KWkqA5lwVeG/iLixZ3xmo1n/5LtnOGsNT78b6bbDfjepSjhrmh6NdttRhT0uljwf\nMx7ZiTzBsBN5gmEn8gTDTuQJhp3IEww7kScYdiJPcJz9OCfxuFnXTMasxxbZe3Vuu32q3X7YXUsM\nLTbbVg3nzHri5XazHmksPWwMP+R+hdjH0Sh9kyojtsbDySM7kScYdiJPMOxEnmDYiTzBsBN5gmEn\n8gTDTuQJjrMf58wxWYSPs+/77nSzftNF/2vW3+491VnbWzPHbKt1ZhlV37nIrJ/xH53OWqbjI/vK\nQ+aMh91vYeIzZriL2azZNjsw4C4a3Y4UdhHpADAIIAsgo6otUa6PiIqnEEf2b6vqwQJcDxEVEV+z\nE3kiatgVwMsi8p6IrJzoAiKyUkTaRaQ9jZGIN0dE+Yr6NP5SVe0UkRMAvCIi/6eqa8dfQFVbAbQC\nQIM0RlvdkIjyFunIrqqdwfceAM8BsKcxEVHZ5B12EakXkeSnpwFcDWBzoTpGRIUV5Wl8E4DnZGze\nbxWAp1X1pYL0igoml0pFaj963hGz/sNp9pzy2ljaWXszZs9X73yt2axn/8ru296Hks5a7v2LzbYz\nN9tj3Q3vd5n1g5fNM+u933S/om0KWU5/xqu7nDXpc0c677Cr6m4A5+bbnohKi0NvRJ5g2Ik8wbAT\neYJhJ/IEw07kCdGIW/Z+GQ3SqEvkypLdnjesZY9DHt8jN1xo1q/5+Rtm/azaj836YK7WWRvVaB/g\nfHT7t8z60O5pzlpsNGTL5JBytsleClrT9nF0xgb37163vNtsK4/NdtY+aHsER/r2Tdh7HtmJPMGw\nE3mCYSfyBMNO5AmGncgTDDuRJxh2Ik9wnL0ShGwPHEnI43v2e/b/+x/MsKewhokbaxsPabXZ9nC2\nPtJt92bcU1zTIWP8j++wp8AeMcbwASCWsR/Tq779vrN2feN6s+0Dp53jrK3TNgxoH8fZiXzGsBN5\ngmEn8gTDTuQJhp3IEww7kScYdiJPcMvmSlDCzzoca8eRE8z6oYapZv1Axt7SeWbcvdxzMjZstp2f\nsPcL7c26x9EBIJ5wL1U9qnGz7T9/4/dmPXVWwqwnxF6K+mJjHYC/3vo3Ztt67DbrLjyyE3mCYSfy\nBMNO5AmGncgTDDuRJxh2Ik8w7ESe4Di752bX2Nse14p7y2UAqJaMWf84PcNZ2zH8dbPthwP2ZwCW\nNm0x62ljLN2aZw+Ej5OfmPjErKfUHoe37tVLmuxx9I1m1S30yC4iq0SkR0Q2jzuvUUReEZEdwXf3\nI0pEFWEyT+OfBLD0mPPuAdCmqgsAtAU/E1EFCw27qq4F0HfM2csBrA5OrwZwbYH7RUQFlu9r9iZV\n7QpOHwDQ5LqgiKwEsBIAajElz5sjoqgivxuvYytWOt/tUNVWVW1R1ZYEaqLeHBHlKd+wd4vIXAAI\nvvcUrktEVAz5hv0FALcEp28B8HxhukNExRL6ml1EngFwOYBZIrIfwC8A3A/gNyJyG4C9AG4oZieP\neyHrxkvcnnutGfdYd3yGPSr6rembzHpvtsGsH87a78NMjx911gYz7r3bAaBv2L7uM2u6zPqGo/Od\ntdnV9ji51W8A6BidZdYX1Bww6w90u/dPaK499v3wz8tceZmzpuv+6KyFhl1VVzhK3O2B6CuEH5cl\n8gTDTuQJhp3IEww7kScYdiJPcIprJQhZSlqq7IfJGnrbd9tZZtsrpthLJr+TmmfWZ1cNmnVrmunc\nmn6zbbIpZdbDhv0aq9zTdwezdWbbKbERsx72e59fbS+D/dNXz3fWkmcfMts2JIxjtDGKyyM7kScY\ndiJPMOxEnmDYiTzBsBN5gmEn8gTDTuQJjrNXAElUm/Vcyh5vtszaNGrWD2btJY+nx+ypntUhSy5b\nWyNf3LjHbNsbMha+YfgUs56Mu7eEnh2zx8mbE/ZY96ZUs1lfM3S6Wb/te686a8+0XmW2rX7pHWdN\n1P148chO5AmGncgTDDuRJxh2Ik8w7ESeYNiJPMGwE3niqzXObiy5LFX2eLHEQ/6vxex6LmXMb87Z\nY81hNG2PhUfxyH89atb3Zaab9QNpux625HLWmGD97vA0s21tzN4uenbVgFkfyNnj9JbBnL3MtTVP\nHwjv+90zdzhrz/Z/x2ybLx7ZiTzBsBN5gmEn8gTDTuQJhp3IEww7kScYdiJPVNQ4e5T10cPGqtUe\n9iyr4eWLzfq+a+1x/JvO+5OzdiCTNNu+b2xrDADTjDnhAFAfsr56St2ff/h41N5OOmys2loXHgBO\nMMbhs2of5zrTdt/ChH3+YH/GWNP++/Zc++lP5dWl8CO7iKwSkR4R2TzuvPtEpFNENgZfy/K7eSIq\nlck8jX8SwNIJzn9YVRcFX2sK2y0iKrTQsKvqWgB9JegLERVRlDfo7hSRD4Kn+c4XOCKyUkTaRaQ9\nDfv1HREVT75h/yWA0wAsAtAF4EHXBVW1VVVbVLUlgZo8b46Iosor7KrarapZVc0BeAyA/XYyEZVd\nXmEXkbnjfrwOwGbXZYmoMoSOs4vIMwAuBzBLRPYD+AWAy0VkEQAF0AHg9kJ0xhpHj6pq7hyznj6l\nyaz3neXeC/zoHGNTbACLlm0z67c2/bdZ7802mPWEGPuzp2eabc+b0mHWX+tfaNYPVk0169Y4/cX1\n7jndAHA4Z++/fmLVJ2b97p0/dNaapthj2Y+fbA8wpTVn1ren7Zes/Tn3fPh/WPi62fY5zDbrLqFh\nV9UVE5z9RF63RkRlw4/LEnmCYSfyBMNO5AmGncgTDDuRJypqiuvINReY9RN+tttZW9Sw32y7sO4t\ns57K2UtRW9Mttw7PM9sezdlbMu8YtYcF+zP2EFRc3MNAPaP2FNcH99jLFrct/k+z/vOPJ5oj9Rex\nOnXWDmXtYbvrp9pLRQP2Y3b719Y6a6dW95htXxyaa9Y/DpkC25ToN+vzE73O2g+SH5pt8x1645Gd\nyBMMO5EnGHYiTzDsRJ5g2Ik8wbATeYJhJ/JEacfZxV4uesm/rDebX5nc4qwdVXtKYdg4eti4qWVa\nlb1s8Ejavpt70vYU1jBn1Bxw1q5r2Gi2XfvoErN+aepHZn3XFfb03LZh91TO3oz9e9+45wqzvuGj\nZrN+4fw9zto5yU6zbdhnG5LxlFm3ph0DwFDO/ff6bsr+/EG+eGQn8gTDTuQJhp3IEww7kScYdiJP\nMOxEnmDYiTwhqu75xoVWN6dZT7v5H5311jv+zWz/dN+Fzlpzrb0d3cnVB836zLi9/a8lGbPHXL+e\nsMdcXxw6yay/cfhMs/7NZIezlhB7u+fLp+w067f+9C6znqm1l9EemO8+nmTq7b+9hnMPmfUfnf6a\nWa82fvfDWXscPex+C9uSOYy1BkEyZm+T/eCy65y1P3Y8if7hrgkfFB7ZiTzBsBN5gmEn8gTDTuQJ\nhp3IEww7kScYdiJPlHQ+eywNTOl2jy++OLDIbH9qnXut7YNpe330Pxw5x6yfVGdv/2ttPXy6MZ8c\nADamppv1l3q/YdZPrLPXT+9OT3PWDqXrzbZHjXnVAPDEww+Z9Qe77XXnr2vc4KydW22Pox/O2cei\nrSHr7Q/map21lNrrG/SHjMMnjb8HAEirHa24seXz9Jg9hj9wjnsb7my3+3ZDj+wi0iwir4vIVhHZ\nIiI/Ds5vFJFXRGRH8D3/1R+IqOgm8zQ+A+AuVV0I4EIAd4jIQgD3AGhT1QUA2oKfiahChYZdVbtU\ndUNwehDANgDzACwHsDq42GoA1xark0QU3Zd6g05E5gM4D8A6AE2q2hWUDgBocrRZKSLtItKeGRmK\n0FUiimLSYReRqQB+B+Anqvq5d4x0bDbNhLMaVLVVVVtUtaWqxn6ziIiKZ1JhF5EExoL+K1V9Nji7\nW0TmBvW5AOxtMYmorEKH3kREADwBYJuqjh+HeQHALQDuD74/H3Zd8dEckvtGnPWc2tMlXzvonurZ\nVDtotl2U3GfWtx+1h3E2DZ/orG2o+prZti7u3u4ZAKZV21Nk66vc9xkAzEq4f/dTauz/wdY0UABY\nn7J/t7+b/YZZ/yjjHqT5/dAZZtutR933OQDMCFnCe9OAu/3RjL2N9kjWjkYqYw/lTquxH9MLGvc6\na9thbxfde64xbfhtd7vJjLNfAuBmAJtE5NNFyO/FWMh/IyK3AdgL4IZJXBcRlUlo2FX1LQCuQ+6V\nhe0OERULPy5L5AmGncgTDDuRJxh2Ik8w7ESeKO2WzUeGEXvzfWf5ty9fYjb/p+W/ddbeDFlu+cUD\n9rjowKg91XP2FPdHfRuMcW4AaEzYHxMO2/K5NmT7308y7k8mjsTsqZxZ50DLmAMj7umzAPB2boFZ\nT+fcWzaPGDUg/PMJfaOzzPqJdf3O2mDGPf0VADoGG836wX57W+XUFDtab2VPc9aWznFvTQ4AdT3u\nxyxm/KnwyE7kCYadyBMMO5EnGHYiTzDsRJ5g2Ik8wbATeaKkWzY3SKMukfwnyvXf5N6y+dS/3262\nXTx9j1nfMGDP2/7IGHdNhyx5nIi5lw0GgCmJUbNeGzLeXB13z0mPTbyA0GdyIePs9XG7b2Fz7Ruq\n3PO6k3F7znfM2NZ4MuLG7/6n/vmRrjsZ8ntn1P6buGjaLmdt1Z6LzbbTlrm32V6nbRjQPm7ZTOQz\nhp3IEww7kScYdiJPMOxEnmDYiTzBsBN5ovTj7PGr3RfI2WuYRzF0/RKzvuTe9XY96R4XPbO622yb\ngD1eXBsynlwfs8fCU8ZjGPbf/K3hZrOeDbmG1z45y6ynjfHm7qMNZtuE8fmBybD2IRjOhGzZPGzP\nd4/H7Nyk3rDn2s/c6v7sRM0a+2/RwnF2ImLYiXzBsBN5gmEn8gTDTuQJhp3IEww7kSdCx9lFpBnA\nUwCaACiAVlV9RETuA/C3AHqDi96rqmus64o6n71SyQX2mvTDc+rMes0he2704Ml2+4Zd7nXpYyP2\nmvO5P28z6/TVYo2zT2aTiAyAu1R1g4gkAbwnIq8EtYdV9V8L1VEiKp7J7M/eBaArOD0oItsAzCt2\nx4iosL7Ua3YRmQ/gPADrgrPuFJEPRGSViMxwtFkpIu0i0p6G/XSViIpn0mEXkakAfgfgJ6o6AOCX\nAE4DsAhjR/4HJ2qnqq2q2qKqLQnY+6kRUfFMKuwiksBY0H+lqs8CgKp2q2pWVXMAHgOwuHjdJKKo\nQsMuIgLgCQDbVPWhcefPHXex6wBsLnz3iKhQJvNu/CUAbgawSUQ2BufdC2CFiCzC2HBcB4Dbi9LD\nrwBdv8ms25MlwzW8k3/baIsx0/FkMu/GvwVMuLi4OaZORJWFn6Aj8gTDTuQJhp3IEww7kScYdiJP\nMOxEnmDYiTzBsBN5gmEn8gTDTuQJhp3IEww7kScYdiJPMOxEnijpls0i0gtg77izZgE4WLIOfDmV\n2rdK7RfAvuWrkH07WVVnT1Qoadi/cOMi7araUrYOGCq1b5XaL4B9y1ep+san8USeYNiJPFHusLeW\n+fYtldq3Su0XwL7lqyR9K+trdiIqnXIf2YmoRBh2Ik+UJewislREtovIThG5pxx9cBGRDhHZJCIb\nRaS9zH1ZJSI9IrJ53HmNIvKKiOwIvk+4x16Z+nafiHQG991GEVlWpr41i8jrIrJVRLaIyI+D88t6\n3xn9Ksn9VvLX7CISB/AhgKsA7AewHsAKVd1a0o44iEgHgBZVLfsHMETkMgBHADylqmcH5z0AoE9V\n7w/+Uc5Q1bsrpG/3AThS7m28g92K5o7fZhzAtQBuRRnvO6NfN6AE91s5juyLAexU1d2qOgrg1wCW\nl6EfFU9V1wLoO+bs5QBWB6dXY+yPpeQcfasIqtqlqhuC04MAPt1mvKz3ndGvkihH2OcB2Dfu5/2o\nrP3eFcDLIvKeiKwsd2cm0KSqXcHpAwCaytmZCYRu411Kx2wzXjH3XT7bn0fFN+i+6FJVPR/ANQDu\nCJ6uViQdew1WSWOnk9rGu1Qm2Gb8M+W87/Ld/jyqcoS9E0DzuJ9PCs6rCKraGXzvAfAcKm8r6u5P\nd9ANvveUuT+fqaRtvCfaZhwVcN+Vc/vzcoR9PYAFInKKiFQDuBHAC2XoxxeISH3wxglEpB7A1ai8\nrahfAHBLcPoWAM+XsS+fUynbeLu2GUeZ77uyb3+uqiX/ArAMY+/I7wLws3L0wdGvUwH8OfjaUu6+\nAXgGY0/r0hh7b+M2ADMBtAHYAeBVAI0V1Lf/AbAJwAcYC9bcMvXtUow9Rf8AwMbga1m57zujXyW5\n3/hxWSJP8A06Ik8w7ESeYNiJPMGwE3mCYSfyBMNO5AmGncgT/w8K8iUImXY9pQAAAABJRU5ErkJg\ngg==\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "C-Yg31snFlNa",
        "colab_type": "text"
      },
      "source": [
        "Cool! Let's try plotting a few images. (Later)."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "rypRoXGKHG2a",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "trainDataLoader = torch.utils.data.DataLoader(trainingdata, batch_size=64, shuffle=True)\n",
        "testDataLoader = torch.utils.data.DataLoader(testdata, batch_size=64, shuffle=False)"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qsEo4adVIXsy",
        "colab_type": "code",
        "outputId": "8429ddcc-284f-4a9a-db28-65d43de0df0c",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 68
        }
      },
      "source": [
        "images, labels = iter(trainDataLoader).next()\n",
        "print(images.size(), labels)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "torch.Size([64, 1, 28, 28]) tensor([2, 7, 3, 6, 3, 7, 4, 9, 2, 6, 4, 9, 5, 2, 1, 4, 2, 5, 0, 4, 4, 4, 6, 2,\n",
            "        5, 0, 4, 0, 5, 3, 9, 2, 3, 0, 6, 6, 8, 5, 9, 9, 4, 1, 2, 8, 5, 5, 6, 6,\n",
            "        6, 7, 7, 4, 3, 3, 4, 0, 3, 8, 3, 8, 3, 4, 3, 3])\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hq0HcYLCIq1B",
        "colab_type": "code",
        "outputId": "8fc26e83-a832-4b07-8e35-f26dfcb2c4aa",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 146
        }
      },
      "source": [
        "plt.figure(figsize=(10,4))\n",
        "for index in np.arange(0,5):\n",
        "  plt.subplot(1,5,index+1)\n",
        "  plt.imshow(images[index].squeeze().numpy(),cmap=plt.cm.gray)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlAAAACBCAYAAAAPH4TmAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAAgAElEQVR4nO2de7AV1bXuvxmjiUZBAUVAEjDIMyIi\nJl4uQZSnogWlxDoWIlhSVCIxHsvSA7lJJfkjhkrFRyitKFGj11jexDomUqKeBNREBBXiC9iAm5eA\nvIT4IBqTkPT9w8X0myP0ZPXa69Gr9/erohi9Rq/uuXr0nN17jjHHcEmSQAghhBBClM+nGt0AIYQQ\nQohmQy9QQgghhBAZ0QuUEEIIIURG9AIlhBBCCJERvUAJIYQQQmREL1BCCCGEEBlp0wuUc26Cc269\nc26Dc25OtRolGoPsWRxky2IhexYH2bI4uErzQDnnjgDwBoCxALYDWAHg8iRJWqrXPFEvZM/iIFsW\nC9mzOMiWxeLTbfjulwFsSJJkEwA45/4fgEkAUm8E51xusnYee+yxXv7Xv/4V6D788MM2H/+zn/2s\nl4866qhA9/7777f5+JWSJIlLUWWyZ55s2V6pli1L+zSFPbt27erlI444wsv//Oc/g/3ee+89L3/0\n0Ue1b1gVKELfdC78Ce01UXN77JtdunTx8nHHHedl7qdA2B+3b99e+4ZVgTR7tuUFqgeAbbS9HcBX\n7E7OuVkAZrXhPAGf+lTodbQvP+UyZMgQL9sBduXKlRUdk+nVq5eXe/fuHeiefPLJso5R58HosPas\nti1FzWhI34xRrXt5+vTpXuY/gv7yl78E+z3++ONebmlJ/+O+SR74de+b9rrwth1zWffpT4ePFL6e\nBw4cqFbzmpnc9c1qMWnSJC+PGTPGy9xPAeCNN97w8g033FD7htWQtrxAlUWSJAsALADKf5OODWqx\nFyb7pjtt2jQvX3vttYHuhBNO8PIHH3wQ6Dp27OjlY445xsuf+9zngv3279/v5U2bNgW6448/3st2\nUOEZrltvvTXQ3X///V7O22BeiS1FfqmnPSu9l6dOnRpsn3feeV5ev369l48++uhgPx7AN2zYEOj+\n/ve/t7ldeaPatrRjMI+79o9Y3vcf//hH2ee4++67vXzZZZd5ed++fcF+PK7bcZxflC3cTmvnvNu9\nnn0zyx8RPXr08PJNN90U6M4991wvDxo0yMv2+cd/0Lz77ruB7vbbb/cyP1/b0uZa0pYg8rcA9KTt\nU0qfieZE9iwOsmWxkD2Lg2xZINryArUCwGnOud7OuaMA/AeAhdVplmgAsmdxkC2LhexZHGTLAlGx\nCy9JkgPOuW8C+B8ARwC4L0mSNVVrmagrsmdxkC2LhexZHGTLYlFxGoOKTlaFGCjL3LlzvcwxT0AY\nv/S3v/0t0P31r39N1fH3eL8dO3YE+5144ole/sxnPpPaZhu3xcdnGQjjNayP+bXXXkMa7GeOBWtG\nVodkQjFQjadatgQaa8/hw4cH2xy/NHLkyED3pz/9yct//vOfvWzjLDh+YtWqVYGOYxB5tR4ArFu3\nrtxmV508983YiscYHGu6aNGiQHf66acfUmdjTY888kgvn3POOYFuypQpXl68eHHZ7ar095RL3vsm\nP2Pt85VX0P3mN78JdP369fOyjXfj51OfPn28bIPIuT9yrBQAvPXWJ95MfvYCwPjx45FGo+ypTORC\nCCGEEBnRC5QQQgghREZy6cKLcc899wTb48aN8/LevXsDXey38ZSfTWPAU8ics8Lmhxo9erSXTzrp\npEDH04i2Hayz7kpOwMlTqQAwb948Ly9YsACVkGc3gchG3t0E7Hqx0/0XXHCBl6+44opAt2fPHi/b\naXzuO5z7yeZyY7e8HRe4z335y18OdHfeeaeXt2zZEuiaxe1TDVvalDCx3zthwgQvz58/P9Cx+4aT\noALAU0895eWJEyemHn/mzJle/tGPfpS6n70HOMfQr3/969TvVSu3IJP3vhmDn7HsTgfC1CEdOnQI\ndN27d/cyP1OtS5b7LbvhgdCG/fv3D3Rsw3rnj5ILTwghhBCiSugFSgghhBAiI3qBEkIIIYTISFPE\nQJ111llefvTRRwPd7t27U7/H8UTWh88xGdYHzjr239rlmLxU2vrfY7WjODbExkDxuXk/AOjUqZOX\nOe4AKH/5dZ7iLETbyHucRWyp9Jw5c7zM/RsIS3ns3Lkz0HEhbi7JYmOsePudd94JdFu3bvWyjfFY\ntmyZl5cuXRroeJyoRpyMpdF9s9zft3z58mB76NChXraF2LlUhz3mqaee6uUVK1Z4+Y477gj247gq\nW4id7wGbSobLaW3evDnQDR48GGlUw85575uMvW5r1671so1B5Otvn098rZ577jkvDxgwINiPY6We\nfvrpQMexcJzSAAjHEy7pVA8UAyWEEEIIUSX0AiWEEEIIkZGKS7nUk+9973tetlOqXIXdTuPHXAgx\njjrqKC/zFLF1J3D2Y5sJmaeBbWbwWKZwzkxu9+OloT/72c8CXb2nNIU4HLE+17lzZy9zRn8A6Nix\no5e3bdsW6NhVzn3MLrtnHbtyAOCll17yciyFSXsjZq8bb7zRyzYbOC9tt+4gxo6R27dv9/LZZ5/t\n5V/84hfBfhwqERs7bUUJTkPRt2/fQHfXXXd5+etf/3qgq2dYSx6wlQD4mbdr165AxxnGOd0IEGYi\nnzp1qpc5bQEQ9rlLL7000PG9ZPtiz56f1GC2KTFioTy1RDNQQgghhBAZ0QuUEEIIIURG9AIlhBBC\nCJGRpoiB4orevMQZCH3u1sceW+bM+9oUB7zN+1n/Psdj8bmAMAaD47SA0CdsYzcYm+KAK8dznJYQ\nzQanGBk4cGCga21t9bKNeeH7Pharwv3R9j+OpeB2tHdi15PLqdgxmK9hrASMjV/l823cuNHL1uYc\nC2N1PD7b9rPd33777UA3Y8YML8dioOwYXMT4KFsyjJ+3Ng6Jn2scwwYAI0eO9HIsFUla7BsQxjbZ\n+Cu+l77yla8EuoULF6IRaAZKCCGEECIjeoESQgghhMhILl14tkL6SSed5GVbvZmzg9tlrOwqs9P4\nnEXVuuZ4mta65hiezrRZWXm5tT0GT5naZdQ8RW2z+vJ0Mi8DB4ARI0Z42WZQFtm49tprg23OvLxy\n5cp6N6eQcH+xaQw4e7XN/s99iftKLGO0XUbNaQ2GDRsW6GyWbaZo7hsb8hBLEcBpAKxrpdyqDjbs\ngM/H4zPb37bThmKwy9C62/iesJUieGzl5wsQLs+343rsedCszJ49O1Vn+w5fD3st2L78XOvWrVuw\nH99LLS0tga5fv35e7tKlS6BjW1944YWBTi48IYQQQogmQS9QQgghhBAZ0QuUEEIIIURGchkDZZdV\nsq/V+rnZJ2vjILgsil2uzP5auyyXS0mwzCUF7Lmt75/9tzZNQiyWgqtf29gsvg72GOPHj/dye46B\nii075nvg5JNPDvbjkgIXXHBBoBs0aJCXK42BsvcH36t2qfAjjzzi5blz5wY6LpfQTNj4F7bFpk2b\nAh2XZbBlWPbu3VvW+ThuxvZbjnmxMVb2/mGKFgMV+z1DhgwJtjm+1MZK8TWLXT8Lx7Rwf7DjP8cv\n2fuI+5WNj+J+9f777wc6jrmaPHlyoFuwYIGXY3FhRcH2MRtLzPCzbNy4cYHu9ddf9/KKFSu8PH36\n9GA/TlPSo0ePQMepemxsJNueS8o0Es1ACSGEEEJkRC9QQgghhBAZyaUL7/Of/3ywzW4t6wrhtAZ2\nySlPAf7ud78LdLNmzfLyl770pUDHU/48pXjbbbcF+/3gBz/w8mmnnRboVq1a5WU7Vf6tb33Ly9/4\nxjcC3bp167xsfyu7Jeyy3NGjR3v5u9/9LtorsWzEU6ZM8bJduszT2D/+8Y8D3ZgxY9rcrtgyezsd\nzfff9ddfH+g4g3IzMXjw4GCb3etbt24NdOymsy4bvo7sLoq5o2LZ/q2bfMCAAV5+/vnnU79XBGL3\n5FVXXZWqs+kPYqkEKnHv8XgPhO5eay/e1/4e3rZuKXbNTZw4MdCxCy92jZoZ7lc2xQ9nIt+xY0eg\n4+etDU3h5zbLsXQ8Fg6nsc8/bpd1/TUKzUAJIYQQQmREL1BCCCGEEBk57AuUc+4+59we59xq+qyT\nc+73zrnW0v8nxI4h8oPsWRxky2IhexYH2bJ9UE4M1P0A7gDwf+mzOQCWJEkyzzk3p7T9X9VqlC0V\nwD5365NmnfWPc/ySrd587733etnGT3B8Bi+N5XIpAPCd73zHyzbl/Xnnnedl+3vYfxtbJmuX5bJP\n2PqHucL8YbgfdbZnPbnooouCbY5v42XNq1evDvbjmICzzjor0L3zzjtetikOuPSHLUERg+MA7JJx\nPg77/YFP4hU++ugjJElyP5rElrbqOsd5cQV2IEzXYeNhOF7D9g+GxwkbG8n2tKWUvvCFL3jZ9qlt\n27alnq9K3I8a27PcuLEJEyYE22kldIDQDjaehrHnSyuZZUuE8Bhvj2Fj5Bi+d2zcDY/X559/fuox\nLOVeP+R8nOVnpU3pwikfbrnllkB38803e9nGkf70pz/1Mo/D/BwGwtIutv+xPX/5y18GOo4HtePi\n0KFDvfzyyy+jXhx2BipJkj8C+LP5eBKAB0ryAwAmQzQFsmdxkC2LhexZHGTL9kGlq/C6JkmysyTv\nAtA1bUfn3CwAs9L0IheUZU/ZsilQ3ywW6pvFQX2zYLQ5jUGSJIlzLnUuM0mSBQAWAEBsP8amBODp\nRjvlx0sp7dQyL/W37j1emm7dYXwcnrK1WZA5dYCdImY3oHU78vTx22+/Heh4ya6dIuapcnvM2NR5\nFmL2rMSWWTITl5vp2aa5YHeDdbvwslvOcG1TV3AaA+uK42t74403Bjqu3G6zlPPyfLuU98orr/Ry\n//79A92aNWu8bKfXhw8fDgB46aWXcDhq0Tcr5Ytf/GKwzX3TZp7me8ZmEWdb8DFiGeht39i5c6eX\nrRuwc+fOXuYM9EBdXHhRqtE32R0Wy9zdt2/fQMfjlE1jwHawYzBv2+8x7Ga19wNjxz22eyyDth3j\n2b1n+xiP3bbfcsqGtmQpb3TfnDdvnpdtxm++jps3bw50Mff31772NS9zhn+bcoDH5E6dOgU67n8t\nLS2Bjl2GNvXJNddc4+WZM2eiXlS6Cm+3c64bAJT+33OY/UW+kT2Lg2xZLGTP4iBbFoxKX6AWAjhY\n4GY6gMeq0xzRIGTP4iBbFgvZszjIlgWjnDQGDwNYDqCfc267c+5qAPMAjHXOtQIYU9oWTYDsWRxk\ny2IhexYH2bJ9cNgYqCRJLk9RjU75vCLY/26r09tlrUwsBor95fYYsTiIWLoAZvv27ak6jsGw7WIf\nP8c82XbaNvP3rG+eY7zssnCO/amXPQ9ir59N/58G+8KBsGK6XT7L9rOlB/i6cPzCq6++GuzH/nxb\nPZyX8to4J05rYJdDc9yIXXbL57PLdZ999lkv2yX4B2OCDhw4UHdbZoXjyjgmAgjvgy1btgQ67js2\nLo77C8e/2PuK4zhsH+P+aPsRjz2xNAm1oB72jJUm4ZhOXsoOhNfT3pPcr6y9uP9bHcdEsU1sLBMv\nbbft5zHSjjV8f9hyPmxbm4Jm7NixXn7sscomifLeN2fPnu1lG4fL/dHGI3MKgldeeSXQjR8/3sv3\n3Xefl3v16hXsx2WdFi5cGOjGjRuXeu7bb7/dy2+++Wagmz9/PhqBMpELIYQQQmREL1BCCCGEEBlp\ncxqDasEuDeuC4ilc69Zinc1Ky1PGdjqep4/t9G7acnr7ObuHYlPXsSrh9nvlVjaPwa4TIHTh1Zty\nXXZAuNR9ypQpgY6Xore2tgY6dunZ6Xiejuap5N69ewf7LV261Mt//OMfAx0vpbfLZzkzr3XTceZ6\nTk0AhC5kew/w0t6OHTsGuoNZfctN+dBITjnlFC/z0nAAWLVqlZdtpmJO62DdRezq4T5m+zCPBfZa\ndejQwcuLFy8OdBdeeKGXrduxCMT649SpU70cq/iQZTyLjVlp4RHWhRdL7RI7XmxfTmNgfw+75a0L\nry2pC/LK97///VTdD3/4w2Cbr5V1sT3//PNePueccw75Hbttx2FOifHVr3410F1yySWp7WwUmoES\nQgghhMiIXqCEEEIIITKSGxceT9PazKU85WenaXm612YK56zUnP3UHieWxTg2DRzTcZvtFGbMTceu\nBztdzK6O2EqYLl26BLr169entrPW2Cy/vNLHZhTnfX/+858Huk2bNnnZFnVm96wtEnzZZZd5mYtM\nDhgwINjv3HPP9bJdgcSuP5u198wzz/SynY7euHGjl20W6z59+niZXUpAeK9a19TBVTB2FUoe4cK8\ntq+88cYbXrZT8+yGtX2A+zv34ViWa+u24na98MILgY5XAbELsj3AGf1tH+BwBetii62047Ev5s7j\n79ns1Hw/2BXabNssbu1YRvGRI0eWdYzYcyPv8O+PuXW5YgcQ2t72OX7OcD+yq2Bff/11L8cqFFi3\nfx7RDJQQQgghREb0AiWEEEIIkRG9QAkhhBBCZCQ3MVDs+1y7dm2g46roNvaHq0Pb5e28HNpmW+Xj\nxNIFlOvDt/C+Nm6Lt63/mX+PPR/7hHkZrj1OvX3HfJ0B4Morr/TyqaeeGug4Xundd98NdBz/ZdMR\ncPoKjocCwnvHxmfwEl2OfRk6dGiwH39vyJAhge7yyz9JKmztxXEyNm7k4Ycf9rJd8svH2bMnrCnK\n9xynbwA+SaMQy86fFzgG0d6vHHdiY8A4Pspe77R4jVjftDGIfE1tHAf3P45va1bs2MPXwmb7ZzvY\n1Ces45QzQOWxP2lpDOy4wPFXlixjMsN25/EDCMczG7/D+zZzDFS56WWsLTi+dt++fYGO0z/w8e1Y\nxc93W1WCnwF2zIzBtqinHTQDJYQQQgiREb1ACSGEEEJkJDcuPM6efdVVVwU6nk5Oc2kAYXZnIJx+\ntdPAsaW3TKxQZizdAW/Hpku5QDAAbN682cvWjXXGGWd4OebCsxmxa0GHDh0wfPhwAP++FPXFF1/0\n8jPPPBPo2EViMz1z1m27fJavk3XxcvqKfv36BTpOQcBuO5vGgKeqn3zyyUDH7gvrkuRitC0tLYGO\np5WtndmFbK9DLPPyI488gmaB01TYNA78m23f5MzvMRcU903rpmNXcawguM30zu5U249i1QXySszF\nNWrUqGCb3VN2zIqFPDCxsS42zrJL16YV4JAEe3y2if1ebOzm32BdTOyasq7+ZcuWHfoHNBmxe5nd\naNbWPN7Z1ED8PR4zrV1mzJjhZWvP5cuXe9mmrIghF54QQgghRJOgFyghhBBCiIzoBUoIIYQQIiO5\niYHiGBQbL8ExEnbZ8VtvveVlGzPEsSR2eTv7V8v1mcaqfceOEYvxsDr2K9vSNFzh2i4hZT9zPdIY\nJEni/de2dMzpp5/uZXvNeKm/XfbP1yVmS74OQHgN58yZE+i4vAC3k335QFg+gssQAMD27du9bFMO\ncKyNvcc4ls+m3+B97bLwDRs2pB7zYCkZjiXLK3wfrFy5MtBx7JHtA/ybY7GLsXibWNwT6+y9y/a1\n90Hfvn29vG7dutTj54nYNZo2bVqwzWOI7X9MLC4tFhtm42nSUsTY9Aoc7xkrYWVjmWL3QAw+5uTJ\nkwMdx0A1SxzcoYg9r/g62v7Bz02bZoCv24MPPujlq6++Otjv2WefTW0Hjxk2vQTHkfKzAmhcfKJm\noIQQQgghMqIXKCGEEEKIjOTGhcduGXbLAeHScTuFu2XLFi9b9x7vW+tpvdj0tJ2m5Hbx0k8gdGXa\nzOp2KXza+W323Fqwf/9+LFmyBMC/ZwbnKWBeyg6E2antVD1PF9vs1OzWsctbOc3FNddcE+jY1cnu\nPAu74qwrgKemrbuNj2/TEfA9YJfyxpbdsnvLplvo3bv3IduRRxYtWuTlxYsXB7qxY8d62dqz3Cz+\nLLP9gHh/Z9cAZ5IHwntk4cKFqecuAhdffHGwzdUa7LhUbsoWC9uyXBeezX7N45k9hu1XacfnPgyE\nrk3bfh5rrBv3pptuSj1fM1Husn/7zOGx3l77YcOGeZmrN9iQEg6XsH2fj2nTJPCzw6aMaVTfLNaI\nIIQQQghRB/QCJYQQQgiREb1ACSGEEEJkJDcxUFySxfqrOWbI+ms5XsXGzTDV8JHG4ipicQE2pib2\nPfYP21QFHJeQpaxMrTh4PTh2AgAGDhzoZRvPtnr1ai/bZap8nWysQ6xkQ6wsD+vYn2+Pz/eY9dlz\nu2ycHS9ttykH+Dj2mNxOW5GcYz527NgR6A5eP3vt8shdd92Vqps0aZKXbQkmtpntO3zdOFbMLtfn\npe/W1nzthgwZEuj+8Ic/ePnxxx9PbX+z0q1bNy/bWCO+LvY+T4tXstt2jGRbxmKgmFjZFUtsXI+1\nORYfxffcoEGDUo/fzMSuKY+F9hnUvXt3L9tSR0888YSXOV7J3md8D9r7bM2aNV7u379/oIs9R2Pp\nOmqJZqCEEEIIITJy2Bco51xP59wzzrkW59wa59x1pc87Oed+75xrLf1f+wq2os3IlsVBfbNYyJbF\nQX2zfVCOC+8AgBuSJHnZOXccgD85534PYAaAJUmSzHPOzQEwB8B/VdoQdlVwRWYA6Nevn5d5CtF+\nz04pxlxusTQDlbjDYvvZaWZ29VhXDC/lZ7cm8O9TzQwv446lO0ANbGkzir/wwgtetpmke/Xq5eWY\nq/boo48OdLwdm+K3y9l5iph1dsqX3YL2GJVWmOdszjbzObuYbJZu6wqMUJe+WSmx7MDstrQ6vqYx\n9xuPBfaaxfoKH9/uV26qkBq5yWtuS04XY/sYu1rsdWcXtF1ezv0jli4gloKAr6dNBxJLOcDni40L\ndgxmO9vv8flseMIZZ5zh5ddeew0Rct03Yy48DoWxFQ9YZ68Nu8P53uKwFCB8ZltdrEIIjxlcraGR\nHHYGKkmSnUmSvFyS9wNYC6AHgEkAHijt9gCAyYc+gsgTsmVxUN8sFrJlcVDfbB9kCiJ3zvUCcCaA\nFwF0TZLkYPTnLgBdU74zC8CsypsoaoFsWSxkz+IgWxYL2bO4lB1E7pw7FsB/A/jPJEkCn03y8Vzb\nIee0kyRZkCTJsCRJhh1KL+qPbFksZM/iIFsWC9mz2JQ1A+WcOxIf3wQPJUnyaOnj3c65bkmS7HTO\ndQOwJ/0Ih2f+/PleHjVqVKBj37wtMbB27Vovn3322YGOY1liJSFsDEZadXHr3y+3hIH9nl26yXDM\nkPXz2t/O8O+x5W5Mu2puS8bGpsTKqYjs1NueWYjFIJ544oleji2Lt32F+xyne7DxPNwfbJwh72tT\nKNj4t7Rj1mLZdD1syaVJYrGfNhaM41asXdlGsTQG9ny8b1qsIhCOn7F4uVjqBXt/cAkwW26K7z/7\nvREjRnj5MDFQue6bMfi6DR48ONBxXCenHACAiy66yMtPP/20l23/5pIvvB8A9OnTx8v2eZfHUkrl\nrMJzAO4FsDZJkltJtRDA9JI8HcBj1W+eqAGyZUFQ3ywcsmVBUN9sH5QzA/W/AUwDsMo592rps28D\nmAfg1865qwG8CeCy2jRRVBnZsjiobxYL2bI4qG+2Aw77ApUkyVIAaWseR1erIXv37vWydUHFUg7s\n2fPJDOjJJ5+c+r2Ym6AWxFIhxJZY85Txe++9F+hilcd37drl5dhUZ5IkNbelqA/16pu1gFOO2AzD\nsWXx7N7hccJWIWCXU8yF/sEHH5Td5lpn+K9H35wwYYKXrXudx41TTjkl0H3729/28vXXXx/o2EVq\nXZtso5hrjsdjO87xmMjpP4Aw5MG69/h8Ntv/T37yEy/PmDEj0PG9ZH/PxRdf7OU777wTaeS9b8bc\n62wzfr4CYQqLkSNHBrrW1lYv8/W2/ZvtbvtmLMXP8ccfn9rmRpE/p6IQQgghRM7RC5QQQgghREb0\nAiWEEEIIkZFMiTRrSSzOiZczWp80f8/GTln/LcN+dhszlLYsN0uV8Bj8vf379wc6W/qE4XQOdr88\nLvEUIg0u/xNLMRKD4ye4BITV2X7KfdqOJ7GK77WOgaoHXErJpnDgtBA2/vKBBx7w8s033xzoNm/e\n7GW79JzjkOz1S7NzzP6xlDOxkiy2XVyiZPXq1YHu0ksv9bItVzJ06NDUtjUTsdQQ/By15dH4eWXj\nl5YuXeplvk5c4gUIY6VGjw7DwdatW+dljosGFAMlhBBCCFEI9AIlhBBCCJGR3LjwuAK3nVbnqULO\nhGphFxcQn6pnKnXFpWUst9ipTj6fPXfMhcdTmvaY5bZFiDzQsWNHL1s3NrsQYik/2NVuXTR8DLv0\nnY9pl8xzhvQi0K1bt2Cb053Y69K5c2cvP/fcc4GO00JY2E1nxzMed9lFCIQuN3bb2TEwFqYRS1XD\n+9oxkX/rr371q0B3xRVXeNk+b2zW8iLC1826tLmf7du3L9Bxige+Jzj1AQAsWbLEy2PGjEk9fteu\nYZlAzh6fFzQDJYQQQgiREb1ACSGEEEJkRC9QQgghhBAZyU0MFPtTrb+afduvvPJK6jF69OgRbHfq\n1MnLNtbBVu6uBG6XjdWILcXltthq3z179kz9Hsch2N8aixMQIm9wf7ElPjhuxvZT3pfLPhx33HGp\nx4+lKrB95Zhjjjls25uJWbNmBdtc7srGlPC1WLZsWaCzsSoMl+2w4yyX47ClYzgGi8dBux+PbTaO\nKhbvyfva8XjEiBFevu2221KPYcuJ8Bg8fPjwQGevWZ6JXTcub2Tjythmu3fvDnTcdwYOHOjlDz/8\nMNhv8uTJqedesWKFl/v06RPorC0Ytm8s3rnaaAZKCCGEECIjeoESQgghhMhIblx47JqbPXt26n6x\n5bS33HJLsL1o0SIv2+nGmJsrzf1ml0qze4HTMNht64bgKU27bNq65piWlhYvc0ZhIJw637p1a+ox\nhMgD7F5hVzsQZi62LiHuV5yZ+Mwzzwz24+X61k3O/c9mWi4aNsv2tm3bvGyzbHNqCZulfPHixV6+\n5JJLAt2oUaO8PHHixEDH41QsJUUtYDs/8cQTge66664r6xjdu3cPtjnj/cyZMwNdM7nwYm5yvi/u\nueeeQHfDDTd42aZ4YDc63yIIWC0AAARUSURBVGf2GcfuQ3uM888/38vLly8PdCtXrkQadpyoF5qB\nEkIIIYTIiF6ghBBCCCEyohcoIYQQQoiMuHpWGHfOpZ6Ml0DOnTs30LFvnpc5AsCDDz5Yreblnm9+\n85tenjZtWqBjv/Idd9wR6B566CEvJ0lSWd0aQ8yWoj5Uy5ZA/e05ZcoUL9t4xFiajzQGDx4cbK9Z\ns8bLtlQML8W2cTkcY2ljLnhJdy3GzUb3zS5duniZy0ZVi759+wbbHO/J5VtiaS3ssnpOebBx48ZA\nZ+Ney+Xuu+/2sr0/XnzxRS//9re/DXQcO9RMfdNe09i9zf2RS94AYR9sbW318oABA4L9nnrqKS/b\nUjEcf8wxzI0mzZ6agRJCCCGEyIheoIQQQgghMlJvF97bAN4E0AVA9eeIs9Pe2vGFJEmqUm5etoxS\nj7ZUzZaAt+cHaF/XsBzUN9tOXtoBqG9Wg7zYs+F9s64vUP6kzq1MkmRY3U+sdlSdvLQ9L+0A8tWW\nLOSp3XlpS17aUQl5aXte2gHkqy1ZyFO789KWPLRDLjwhhBBCiIzoBUoIIYQQIiONeoFa0KDzWtSO\ntpOXtuelHUC+2pKFPLU7L23JSzsqIS9tz0s7gHy1JQt5ande2tLwdjQkBkoIIYQQopmRC08IIYQQ\nIiN6gRJCCCGEyEhdX6CccxOcc+udcxucc3PqfO77nHN7nHOr6bNOzrnfO+daS/+fUId29HTOPeOc\na3HOrXHOXdeotrQF2bI4tgRkz9I5C2FP2bI4tgRkzzzbsm4vUM65IwDcCeACAAMBXO6cG1iv8wO4\nH8AE89kcAEuSJDkNwJLSdq05AOCGJEkGAjgHwOzSdWhEWypCtvQ0vS0B2ZNoenvKlp6mtyUge5bI\nry2TJKnLPwD/C8D/0PZcAHPrdf7SOXsBWE3b6wF0K8ndAKyvZ3tK530MwNg8tEW2bH+2lD2LZU/Z\nsji2lD3zb8t6uvB6ANhG29tLnzWSrkmS7CzJuwB0refJnXO9AJwJ4MVGtyUjsqWhiW0JyJ7/RhPb\nU7Y0NLEtAdkzIG+2VBB5ieTj19i65XRwzh0L4L8B/GeSJO83si1FQ7YsFrJncZAti0U9r2EebVnP\nF6i3APSk7VNKnzWS3c65bgBQ+n9PPU7qnDsSH98IDyVJ8mgj21IhsmWJAtgSkD09BbCnbFmiALYE\nZE+UzpNLW9bzBWoFgNOcc72dc0cB+A8AC+t4/kOxEMD0kjwdH/tWa4pzzgG4F8DaJElubWRb2oBs\nicLYEpA9ARTGnrIlCmNLQPbMty3rHPx1IYA3AGwE8H/qfO6HAewE8A987Ee+GkBnfBy93wpgMYBO\ndWjHCHw81fg6gFdL/y5sRFtkS9lS9iyePWXL4thS9sy3LVXKRQghhBAiIwoiF0IIIYTIiF6ghBBC\nCCEyohcoIYQQQoiM6AVKCCGEECIjeoESQgghhMiIXqCEEEIIITKiFyghhBBCiIz8fwVy18Dt4JCj\nAAAAAElFTkSuQmCC\n",
            "text/plain": [
              "<Figure size 720x288 with 5 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "qUI2KGrDJpRx",
        "colab_type": "code",
        "colab": {}
      },
      "source": [
        "class LinearReg(torch.nn.Module):\n",
        "  def __init__(self):\n",
        "    super(LinearReg, self).__init__()\n",
        "    self.linear = torch.nn.Linear(28*28,10)\n",
        "\n",
        "  def forward(self, x):\n",
        "    x = x.view(-1,28*28)\n",
        "    transformed_x = self.linear(x)\n",
        "    return transformed_x\n",
        "\n",
        "net = LinearReg().cuda()\n",
        "Loss = torch.nn.CrossEntropyLoss()\n",
        "optimizer = torch.optim.SGD(net.parameters(), lr=0.01)\n",
        "\n"
      ],
      "execution_count": 0,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "YfCQuKX7LAqV",
        "colab_type": "text"
      },
      "source": [
        "Cool! Now we have set everything up. Let's try to train the network."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "72TiXpC8LEMX",
        "colab_type": "code",
        "outputId": "c140fbfa-2df2-4659-ab65-32bf6e95e63e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 357
        }
      },
      "source": [
        "train_loss_history = []\n",
        "test_loss_history = []\n",
        "\n",
        "for epoch in range(20):\n",
        "  train_loss = 0.0\n",
        "  test_loss = 0.0\n",
        "  for i, data in enumerate(trainDataLoader):\n",
        "    images, labels = data\n",
        "    images = images.cuda()\n",
        "    labels = labels.cuda()\n",
        "    optimizer.zero_grad()\n",
        "    predicted_output = net(images)\n",
        "    fit = Loss(predicted_output,labels)\n",
        "    fit.backward()\n",
        "    optimizer.step()\n",
        "    train_loss += fit.item()\n",
        "  for i, data in enumerate(testDataLoader):\n",
        "    with torch.no_grad():\n",
        "      images, labels = data\n",
        "      images = images.cuda()\n",
        "      labels = labels.cuda()\n",
        "      predicted_output = net(images)\n",
        "      fit = Loss(predicted_output,labels)\n",
        "      test_loss += fit.item()\n",
        "  train_loss = train_loss/len(trainDataLoader)\n",
        "  test_loss = test_loss/len(testDataLoader)\n",
        "  train_loss_history.append(train_loss)\n",
        "  test_loss_history.append(test_loss)\n",
        "  print('Epoch %s, Train loss %s, Test loss %s'%(epoch, train_loss, test_loss))"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "Epoch 0, Train loss 0.962718647044859, Test loss 0.7394164491231274\n",
            "Epoch 1, Train loss 0.6678871326863385, Test loss 0.6467844312358054\n",
            "Epoch 2, Train loss 0.6037305348209226, Test loss 0.6036576053519158\n",
            "Epoch 3, Train loss 0.5682870054136969, Test loss 0.579158300996586\n",
            "Epoch 4, Train loss 0.5456478984307632, Test loss 0.5595327822645758\n",
            "Epoch 5, Train loss 0.5288768831346589, Test loss 0.5468165380939557\n",
            "Epoch 6, Train loss 0.5162919589769103, Test loss 0.5357637635082196\n",
            "Epoch 7, Train loss 0.5059161315213389, Test loss 0.52770537024091\n",
            "Epoch 8, Train loss 0.4975833669781431, Test loss 0.5209156887926114\n",
            "Epoch 9, Train loss 0.490400868263453, Test loss 0.515774294828913\n",
            "Epoch 10, Train loss 0.4842610348230486, Test loss 0.5096913139531567\n",
            "Epoch 11, Train loss 0.4788224052455125, Test loss 0.506357757528876\n",
            "Epoch 12, Train loss 0.47393937836260175, Test loss 0.5024205670235263\n",
            "Epoch 13, Train loss 0.4699769388796932, Test loss 0.5000300154944134\n",
            "Epoch 14, Train loss 0.46620280413167564, Test loss 0.4947607781097388\n",
            "Epoch 15, Train loss 0.46279092250602333, Test loss 0.49202916937269225\n",
            "Epoch 16, Train loss 0.45980159040770807, Test loss 0.4886204714228393\n",
            "Epoch 17, Train loss 0.4566683091366215, Test loss 0.4868291844228271\n",
            "Epoch 18, Train loss 0.4541553847952438, Test loss 0.48535446642310753\n",
            "Epoch 19, Train loss 0.451550436648987, Test loss 0.483275003588883\n"
          ],
          "name": "stdout"
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "7pY0PZU2NWWf",
        "colab_type": "code",
        "outputId": "d7d3b186-4cd8-4872-e1dc-a51c6000ab43",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 296
        }
      },
      "source": [
        "plt.plot(range(20),train_loss_history,'-',linewidth=3,label='Train error')\n",
        "plt.plot(range(20),test_loss_history,'-',linewidth=3,label='Test error')\n",
        "plt.xlabel('epoch')\n",
        "plt.ylabel('loss')\n",
        "plt.grid(True)\n",
        "plt.legend()"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<matplotlib.legend.Legend at 0x7f2fef2c37f0>"
            ]
          },
          "metadata": {
            "tags": []
          },
          "execution_count": 29
        },
        {
          "output_type": "display_data",
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEGCAYAAABo25JHAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjAsIGh0\ndHA6Ly9tYXRwbG90bGliLm9yZy8GearUAAAgAElEQVR4nO3dd3yb1b348c9XsmzZlrcTJ8RJHDKA\nQJxJBmE4jDLaQtpCCw2r0JtLy2jvbW8ZLVwuv/IrtIX+LqMF2lJomxJadguU1ZgZMgnZO3HiDMfb\nlqdkn98fj2xJtmzLQ5Zjfd+v1/PSM84jfS3L+vqc8zzniDEGpZRSscsW7QCUUkpFlyYCpZSKcZoI\nlFIqxmkiUEqpGKeJQCmlYlxctAPorezsbJOXl9enc+vq6khOTh7YgAaQxtc/Gl//DfUYNb6+W7du\nXZkxZkTIg8aY42qZPXu26asVK1b0+dzBoPH1j8bXf0M9Ro2v74C1povvVW0aUkqpGKeJQCmlYpwm\nAqWUinHHXWexUmp48Hg8FBcX09jYGPY5aWlpbNu2LYJR9c9QiM/pdJKbm4vD4Qj7HE0ESqmoKC4u\nJiUlhby8PEQkrHNqa2tJSUmJcGR9F+34jDGUl5dTXFzMhAkTwj5Pm4aUUlHR2NhIVlZW2ElA9UxE\nyMrK6lUtC2IkETR5W9h9rJbPjnlZuac82uEopXw0CQy8vrynMdE09N62Y3x32XoANtXvZcHErChH\npJRSQ0dM1AjGZyW1r+8vr4tiJEqpoaK8vJwZM2YwY8YMRo0axZgxY9q3m5ubuz137dq13HbbbYMU\naeTFRI1gfJb/lu+DFfW0tBrsNq2SKhXLsrKy2LBhAwD33nsvLpeLH/7wh+3HvV4vcXGhvyLnzJnD\nnDlzBjymlpYW7HZ7l9vhntdbMVEjcCXEke1KAMDTYjhc1RDliJRSQ9H111/PTTfdxLx58/jRj37E\n6tWrWbBgATNnzuSMM85gx44dABQWFvKlL30JsJLIDTfcQEFBAfn5+TzyyCMhn/vtt99mwYIFzJo1\niyuuuAK32w1AXl4et99+O7NmzeJvf/tbp+3nnnuOadOmcdppp3H77be3P5/L5eIHP/gB06dPZ+XK\nlf36uWOiRgCQl5VEmbsJgKLyesZmJvVwhlJqsOTd8XrEnnv/A1/sVfni4mI++eQT7HY7NTU1fPjh\nh8TFxfHuu+9y11138eKLL3Y6Z/v27axYsYIjR44we/ZsvvOd7wRdx19WVsZPf/pT3n33XZKTk3nw\nwQd5+OGHueeeewCrdrJ+vdWPeccdd7RvHz58mPnz57Nu3ToyMjL4whe+wCuvvMLixYupq6tj3rx5\nPPTQQ/14dywxkwjGZSWxtqgSgKKKOs4kO8oRKaWGoiuuuKK9maW6uprrrruOXbt2ISJ4PJ6Q53zx\ni18kISGBrKwsRo4cSUlJCbm5ue3HP/30U7Zu3crChQsBaG5uZsGCBe3Hv/GNbwQ9X9v2mjVrKCgo\nYMQIa9DQJUuW8MEHH7B48WLsdjtf+9rXBuRnjplEkBfQT1BUXh/FSJRSQ1ngMNJ33303ixYt4uWX\nX2b//v0UFBSEPCchIaF93W634/V6g44bY7jgggt47rnnenzNUNuhOJ3OfvULBIqZRBB05VCZXjmk\n1FASbvPNYN+5W11dzZgxYwB45pln+vw88+fP5+abb2b37t1MmjSJuro6Dh06xJQpU7o9b+7cudx2\n222UlZWRkZHBc889x6233trnOLoSE53FoDUCpVTv/ehHP+LOO+9k5syZnf7L740RI0bwzDPPcNVV\nV5Gfn8+CBQvYvn17j+eNHj2aBx54gEWLFjF9+nRmz57NZZdd1uc4uhIzNYKgRFBRR2urwaaXkCql\nsK78CWXBggXs3LmzffunP/0pAAUFBe3NRB3P3bx5c8jnOvfcc1mzZk2n/fv37+92+6qrruKqq67q\ndF7bVUcDIWZqBGlJDpJ9nfiNnlaO1TZFNyCllBoiYiYRAOQk+X9cvcNYKaUsMZUIRib5m4KKNBEo\npRQQY4kguEagHcZKKQUxlgi0RqCUUp3FVCIIqhGUaY1AKaUghi4fBRiZ7E8EReV1GGN0YgylYlR5\neTnnnXceAEePHsVut7cP5bB69Wri4+O7Pb+wsJD4+HjOOOOMiMcaaTGVCFIckJIQR22Tl7rmFsrc\nzYxISej5RKXUsNPTMNQ9KSwsxOVy9TkRdBzmurthr7s7byDEVNOQiDA+2z/UhPYTKKUCrVu3jnPO\nOYfZs2dz4YUXcuTIEQAeeeQRpk6dSn5+PldeeSX79+/niSee4Fe/+hUzZszgww8/DHqeuro6brjh\nBubOncvMmTN59dVXAWuYiksvvZRzzz2X8847r9N2RUUFixcvJj8/n/nz57Nx40bASlTXXHMNCxcu\n5JprrhnwnzumagRgTVKz+VANYF05NCcvM8oRKaW4Ny2sYn0aZeje6rCKGWO49dZbefXVVxkxYgTP\nP/88P/7xj3n66ad54IEH2LdvHwkJCVRVVZGens5NN93UZS3i/vvv59xzz+Xpp5+mqqqKuXPncv75\n5wOwfv16Nm7cSGZmJs8880zQ9q233srMmTN55ZVX+Ne//sW1117bXmvZunUrH330EYmJiX15F7oV\nc4kgL0trBEqpzpqamti8eTMXXHABYM36NXr0aADy8/NZsmQJixcvZvHixT0+19tvv81rr73GL3/5\nSwAaGxs5cOAAABdccAGZmf5/QAO3P/roo/b5Ds4991zKy8upqbH+cb300ksjkgQgBhNB4LSVei+B\nUqqNMYZTTz015Gxfr7/+Oh988AF///vfuf/++9m0aVOPz/Xiiy9y0kknBe1ftWpVn4ac7k25voho\nIhCRi4D/BezA74wxD3Q4Ph54GhgBVABXG2OKIxlT8CikWiNQakgIs/kmksNQJyQkUFpaysqVK1mw\nYAEej4edO3dyyimncPDgQRYtWsSZZ57J8uXLcbvdpKSktP+33tGFF17Io48+yqOPPoqI8NlnnzFz\n5sweYzjrrLNYtmwZd999N4WFhWRnZ5OamjrQP2onEessFhE78DhwMTAVuEpEpnYo9kvgj8aYfOA+\n4GeRiqdNYNPQvjLrElKllLLZbLzwwgvcfvvtTJ8+nRkzZvDJJ5/Q0tLC1VdfzbRp05g5cya33XYb\n6enpfPnLX+bll18O2Vl899134/F4yM/P59RTT+Xuu+8OK4Z7772XdevWkZ+fzx133MGzzz4biR+1\nk0jWCOYCu40xewFEZDlwGbA1oMxU4D996yuAVyIYDwAjUhJIdNhp8LRQ2+ilqt5DRnL31wsrpYa3\nwKGkP/jgg07HP/roo077pkyZ0n5VT0eJiYk8+eSTnfZff/31XH/99V1uZ2Zm8sornb8Guxome6BE\nMhGMAQ4GbBcD8zqU+Rz4Klbz0VeAFBHJMsaUBxYSkaXAUoCcnBwKCwv7FJDb7eb9998n22k46Jt6\n9KV3PmRi+sBM99Zfbre7zz/bYND4+meoxweDG2NaWhq1tbW9OqelpaXX5wymoRJfY2Njr36P0e4s\n/iHwmIhcD3wAHAJaOhYyxjwFPAUwZ84c09W8oT0pLCykoKCAUw+u4+CWowBkjT+Zgplj+vR8A60t\nvqFK4+ufoR4fDG6M27Zt63V7/2BPVdlbQyU+p9MZVp9Em0gmgkPA2IDtXN++dsaYw1g1AkTEBXzN\nGFMVwZgAgm4q03kJlIoeHeZl4PWl3zOSdxavASaLyAQRiQeuBF4LLCAi2SLSFsOdWFcQRZzOX6xU\n9DmdTsrLy/WCjQFkjKG8vByn09mr8yJWIzDGeEXkFuAtrMtHnzbGbBGR+4C1xpjXgALgZyJisJqG\nbo5UPIHGZ2mNQKloy83Npbi4mNLS0rDPaWxs7PWX3GAaCvE5nU5yc3N7dU5E+wiMMW8Ab3TYd0/A\n+gvAC5GMIRStESgVfQ6HgwkTJvTqnMLCwl61fQ+2oR5fV2Jq0Lk2o1KdxMdZP3pFXTM1jZ4oR6SU\nUtETk4nAZhPGZ/qbhw5orUApFcNiMhFAxzGHtJ9AKRW7YjYRBI9CqjUCpVTsitlEMD47oEZQpjUC\npVTsitlEoDUCpZSyxHAi0D4CpZSCGE4Eo9OcOOzWre3Hapuob/ZGOSKllIqOmE0EcXYbYzO0eUgp\npWI2EUDwUBM6W5lSKlbFeCLQ+YuVUiqmE0Ge1giUUiq2E0HwvQRaI1BKxaaYTgTBo5BqjUApFZti\nOhGMSU/E5psc6XB1I42eTrNkKqXUsBfTiSA+zsaYjMT27YMV2jyklIo9MZ0IoOMdxpoIlFKxJ+YT\ngd5LoJSKdTGfCHTMIaVUrIv5RDBe5y9WSsW4mE8EgTeVaY1AKRWLYj4RjM1MQnyXkB6qbKDZ2xrd\ngJRSapDFfCJwOuyMTnUC0GqguFKbh5RSsSXmEwFoP4FSKrZpIgDysrWfQCkVuzQRoDUCpVRs00SA\nXjmklIptmgjQGoFSKrZpIiB4mImDFfV4W/QSUqVU7NBEACTFxzEyJQEAb6vhSHVjlCNSSqnBE9FE\nICIXicgOEdktIneEOD5ORFaIyGcislFELolkPN3RMYeUUrEqYolAROzA48DFwFTgKhGZ2qHYT4C/\nGmNmAlcCv45UPD0ZH9RhrP0ESqnYEckawVxgtzFmrzGmGVgOXNahjAFSfetpwOEIxtOtvID5i4vK\ntEaglIodYoyJzBOLXA5cZIz5tm/7GmCeMeaWgDKjgbeBDCAZON8Ysy7Ecy0FlgLk5OTMXr58eZ9i\ncrvduFyukMdWH/Hy68+bAJg50s73Zjn79Br90V18Q4HG1z9DPT4Y+jFqfH23aNGidcaYOaGOxQ12\nMB1cBTxjjHlIRBYAfxKR04wxQZftGGOeAp4CmDNnjikoKOjTixUWFtLVudmHqvn15x8B4CaRgoJz\n+vQa/dFdfEOBxtc/Qz0+GPoxanyREcmmoUPA2IDtXN++QDcCfwUwxqwEnEB2BGPq0rjAmcoq6mlt\njUxNSSmlhppIJoI1wGQRmSAi8Vidwa91KHMAOA9ARE7BSgSlEYypS6lOB1nJ8QA0e1s5WqOXkCql\nYkPEEoExxgvcArwFbMO6OmiLiNwnIpf6iv0A+DcR+Rx4DrjeRKrTIgzjdagJpVQMimgfgTHmDeCN\nDvvuCVjfCiyMZAy9kZeVzPoDVYA11MQZE6MckFJKDQK9szjAeL2pTCkVgzQRBAicl6CoTG8qU0rF\nBk0EAbRGoJSKRZoIAgTOS1BUXk8U+62VUmrQaCIIkJ4UT1qiA4AGTwultU1RjkgppSJPE0EHeTr4\nnFIqxmgi6ED7CZRSsUYTQQfB/QSaCJRSw58mgg6CawTaNKSUGv40EXQQdC+B1giUUjFAE0EHgTWC\nojK9hFQpNfxpIuggKzkeV4I1BFNtk5eKuuYoR6SUUpGliaADEdH5i5VSMUUTQQh5gc1D2k+glBrm\nNBGEoDUCpVQsialEkFhfDI01PZYbr/cSKKViSGwkguJ18Jcrmbf6Zlj/xx6L670ESqlYEhuJoGQT\n7HzTWl/1BLR4uy0e2EdwQGsESqlhLjYSQf43ICnbWq8+CNte7bb4yJQEnA7rrams91Bd74l0hEop\nFTWxkQgciXD6t/3bnzwG3dwoZrMJ4zMDrhyq0FqBUmr4io1EAHD6t2kVa64BDq+HA592W1yvHFJK\nxYrYSQSuERwdVeDfXvlYt8XzsgOHmtAagVJq+IqdRAAU517q39j+OpTv6bKs1giUUrEiphJBffI4\nmHSBb8vAp7/psqzeXayUihVhJQIR+Z6IpIrl9yKyXkS+EOngImLBzf71DcugviJkMa0RKKViRbg1\nghuMMTXAF4AM4BrggYhFFUknFkDOada6px7WPROy2Oi0ROLt1ttT5m7C3dT9vQdKKXW8CjcRiO/x\nEuBPxpgtAfuOLyLBtYLVT4G381DTdpswNjOxfVubh5RSw1W4iWCdiLyNlQjeEpEUoDVyYUXYaV8D\nV461XnsEtrwUslhwP4E2DymlhqdwE8GNwB3A6caYesABfCtiUUVaXALMXerf7uIGs+Axh7RGoJQa\nnsJNBAuAHcaYKhG5GvgJUB25sAbBnBsgztf0U7IJ9n3QqUjQ/MVlWiNQSg1P4SaC3wD1IjId+AGw\nB+hxGE8RuUhEdojIbhG5I8TxX4nIBt+yU0SqehV9fyRlwswl/u0QN5hpjUApFQvCTQReY83ifhnw\nmDHmcSCluxNExA48DlwMTAWuEpGpgWWMMf9hjJlhjJkBPAqEbqyPlPnfpb3Pe9fbULoj6HBe0LwE\nWiNQSg1P4SaCWhG5E+uy0ddFxIbVT9CducBuY8xeY0wzsBwrkXTlKuC5MOMZGFkT4aRL/NsrHw86\nPCY9kTiblSiO1jTS0NwymNEppdSgENPNKJzthURGAd8E1hhjPhSRcUCBMabL5iERuRy4yBjzbd/2\nNcA8Y8wtIcqOBz4Fco0xnb5tRWQpsBQgJydn9vLly8P64Tpyu924XK6gfWlVW5i54S4AWsXBygW/\nxxOf1n789g/qKam33qOfLkwkNyVyN2OHim8o0fj6Z6jHB0M/Ro2v7xYtWrTOGDMn1LG4cJ7AGHNU\nRJYBp4vIl4DV3SWBPrgSeCFUEvC9/lPAUwBz5swxBQUFfXqRwsJCOp1rzoFjf4PDn2EzHhbGb4MC\nf3fGyXtXU7KzFIARJ06l4NRRfXrtPsc3hGh8/TPU44OhH6PGFxnhDjHxdWA1cAXwdWCV7z/+7hwC\nxgZs5/r2hXIlg90s1EYEFgRUUlb/FjyN7Zt5On+xUmqYC7ed48dY9xBcZ4y5Fqv9/+4ezlkDTBaR\nCSISj/Vl/1rHQiJyMtawFSvDD3uATb0MUnOt9foy2Ph8+yGdv1gpNdyFmwhsxphjAdvlPZ1rjPEC\ntwBvAduAvxpjtojIfSISMB40VwLLTTidFZFid8C8f/dvr3y8/QazoHsJtEaglBqGwuojAP4pIm/h\nb775BvBGTycZY97oWM4Yc0+H7XvDjCGyZl8H7z8IzW4o2wG734XJFwTXCPSmMqXUMBRWjcAY819Y\nnbX5vuUpY8ztkQxs0DnTYNa1/m3fDWa5GYn4riDlcHUDTV69hFQpNbyEfS2kMeZFY8x/+paXIxlU\n1Mz7dxDfW7K3EI5uIiHOzgnp1lAUxsDBioboxaeUUhHQbSIQkVoRqQmx1IpIzWAFOWgy8uCUgO6L\nlb8GdLYypdTw1lOHb4oxJjXEkmKMSR2sIAdV4KWkm/4GtUd1tjKl1LAWU3MWh2Xs6TB2nrXe6oHV\nT2mNQCk1rGkiCCVwBrO1T3Oif8QJdh9zD348SikVQZoIQjn5S5A+3lpvqGRGxZvthz7ZU84nu8ui\nFJhSSg08TQSh2Oy+IaotWZt+x8VTR7Zv3/HSJh2JVCk1bGgi6MrMqyHB1yZUsZefnVZMqtO6/+5A\nRT2/endnFINTSqmBo4mgKwkumHN9+2b657/lJ1/0z6vzuw/3srF48CZUU0qpSNFE0J25/w423ygc\nRR9zxQmlLJyUBUCrgR+9sBFPS2sUA1RKqf7TRNCdtDFw6lfbN2Xl4/zsK/k4Hdbbtv1oLU++vyda\n0Sml1IDQRNCTwEtJt7zMOHs5P/zCSe27Hnlvt15SqpQ6rmki6MkJMyDvLGvdtMCHD/GthROYnmt1\nJDe3tHLHixtpbY3eKNpKKdUfmgjCEVgrWPcH7B8/zANfy2+f2H5tUSV/XlUUpeCUUqp/NBGEY/KF\nMOVi//Z793FK0TK+WzCxfdeDb27nUJWOTKqUOv5oIgiHzQZX/AEmnO3f9887uDX9IyaNdAFQ19zC\nj1/eRDQnWlNKqb7QRBAuRyJc+RyMne/f9cYP+O2M3Yhv4prCHaW8uuFwlAJUSqm+0UTQGwkuWPJX\nOGGmb4dhwoc/5MGT97YX+Z+/b6Hc3RSd+JRSqg80EfSWMw2ufglyTrO2TStXFN3L5SlbAKis93Df\nP7ZGMUCllOodTQR9kZQJ17wC2VMAkFYvP2/5BQttmwB4dcNh/rW9JJoRKqVU2DQR9JVrBFz7qjW9\nJWBrbebphF9xumwH4Mcvb6a20RPFAJVSKjyaCPoj9QS49jVIzQUgwTTyh4RfMF12c6S6kZ//c0eU\nA1RKqZ5pIuivjPFw3WvgygHARQN/jH+AqbKfP31axOp9FVEOUCmluqeJYCBkTbSaiRIzAUiTev4U\n/zMmyiHueHEjjR6dxEYpNXRpIhgoI0+Ba19pn8wmS2r5S/z9tJTv4dF/7YpycEop1TVNBANp9HS4\n+kWIt+42zpEqlsX/X159fzVbDldHOTillApNE8FAG3s6fPN5TFwiALlSxp/jfsqDf1uBVyexUUoN\nQZoIIiHvTOTKZRhbvLVpK+Hu8rtY9q/1UQ5MKaU6i2giEJGLRGSHiOwWkTu6KPN1EdkqIltE5C+R\njGdQTToP+cYfaRE7AJNth5j70Q0UFRdHOTCllAoWsUQgInbgceBiYCpwlYhM7VBmMnAnsNAYcyrw\n/UjFExUnXYz5ym9p8b3Np0gRrt+fSdnHz4KOUqqUGiIiWSOYC+w2xuw1xjQDy4HLOpT5N+BxY0wl\ngDHmWATjiYq4/K9x5Jxftm9nmUqy37kN92/OgyOfRzEypZSyRDIRjAEOBmwX+/YFmgJMEZGPReRT\nEbkogvFETe6iG1k771FKTEb7PtexdZgnz4F//AfU601nSqnokUhNpCIilwMXGWO+7du+BphnjLkl\noMw/AA/wdSAX+ACYZoyp6vBcS4GlADk5ObOXL1/ep5jcbjcul6tP5w6E/WVuWjYt5xreIF78N5l5\n4lLYN2EJO1MX4kpJjVp8PYn2+9cTja//hnqMGl/fLVq0aJ0xZk6oY3ERfN1DwNiA7VzfvkDFwCpj\njAfYJyI7gcnAmsBCxpingKcA5syZYwoKCvoUUGFhIX09d6AcWHQu//b0y3yr5kkK7FbTkMNby5Rd\nTzDa9RYpX38Cxs3v4VmiYyi8f93R+PpvqMeo8UVGJJuG1gCTRWSCiMQDVwKvdSjzClAAICLZWE1F\nexnGxmUl8cgtX+epsQ/y7eYfcKB1RPuxFPc+ePpCeGkp1B6NYpRKqVgSsURgjPECtwBvAduAvxpj\ntojIfSJyqa/YW0C5iGwFVgD/ZYwpj1RMQ0VaooNnb5xH1qzFXND8Cx7yXE6DifcX2Pg8PDobPn4E\nvM3RC1QpFRMi2TSEMeYN4I0O++4JWDfAf/qWmOKw23jga9PIy07mwX/G81LLWdzlWMYX7autAs1u\neOduWP9HuPhBmHRedANWSg1bemdxFIkI3ymYyK+XzKIsLoebPd/nm813sdsEXFxVvgv+/FVYvgQq\ni6IXrFJq2NJEMARcMm00y5fOJzUePmk9jYuafsb/8VxNkz3ZX2j7P+DxufDWj6HmcPSCVUoNO5oI\nhoiZ4zK4e34ik0e68BLH71su4cy6X7A6LeDWCm8jrHwM/nc6vHYblO+JXsBKqWFDE8EQMiLJxgvf\nOYMzJ2UDUEo6Xy+5lp9kPow3J99fsKUZ1j8Lj82BF26Ao5uiFLFSajjQRDDEpCU6+MO3Tuequf5b\nMP58eBQX1N7L0Uv+ALmn+wubVtj8IjxxJiy7AopWRiFipdTxThPBEOSw2/i/X5nGXZecjIi1b19F\nIxe+6eKf8/6Eue7vMPHc4JN2vQ1/uAievhh2vaOD2imlwqaJYIgSEZaePZHfLJmF02H9mqobPNy0\nbD1ffyuO9ec8DUsL4ZRLAfGfeOATWHY5PHkWbH4JWnW+ZKVU9zQRDHEXnTaa55cuYERKQvu+Nfsr\n+eqvP+G7K1rZd94TcPNqmHE12AJuCzm6CV74Fjx2unUvgt6YppTqgiaC48D0sem89f2z+dbCPBx2\n/3//b2w6ygUPv889nzRTdv7DcNsGmHcT+KbJBKBiD7x2q3Wl0crHoW7Y37itlOolTQTHiczkeP77\ny6fy7n+ew5fyR7fv97Ya/riyiHN+voJH1zVSf9798B+b4ez/goQ0/xPUHoa37oJfTrL6ET5+RC8/\nVUoBmgiOO+Ozknnsm7N45eaFzJuQ2b6/rrmFh97ZScEvClm+pR7vOXdZCeH8/4Hkkf4nMK1WP8I7\nd8Ojs+CxufDOf8PB1dDaGoWfSCkVbZoIjlMzxqazfOl8fn/dHCaP9I9/fqy2iTte2sTF//sh7+1r\nwCz8Hnx/E3zxYRg7j6COZYCyHfDx/4PfXwAPTYFXb4Htb0Bz/eD+QEqpqInooHMqskSE807J4Zwp\nI3hhXTEPv7OTY7VNAOw65ubGZ9cyb0Imd11yCtNPvxFOvxHcpbDzn7DjDdizArwN/iesK4XP/mQt\ncYnWJaonXQxTLgLXiC6iUEod7zQRDANxdhtXzh3HpTNO4Pcf7uPJD/bibvICsGpfBZc9/jFfyh/N\nf114EuOzRsCsa6yluR72FlpJYec/rUTQxtsAO163FgTGzmWs42Q4mgUjTwWbViaVGi40EQwjSfFx\n3HreZK6aN45H39vFslUH8LZaN5b9Y+MR3tpylC/nn8CS+eOYNS4DiU+Cky+xltYWOLQOtr9uJYay\nnQHPbODgKiayCp54FpJHwIRz4MQCa0kf2zkYpdRxQxPBMJTtSuB/LjuN6xdO4BdvbeeNTdZsZ54W\nw0ufHeKlzw5xUk4KS+aPY/HMMaQ6HWCzw9i51nLB/0DZbish7HgTDn5qdTK3qSuFzS9YC0DWJH9S\nyDsLEtMH+SdWSvWHJoJhbEJ2Mr9eMpt1RZU8+OZ2Vu+vaD+2o6SWe17dws/e2M6l061aQn5uwBd4\n9iTIvg0W3gZ1ZbDzLY598hdGurdCQ0XwC5XvtpY1vwOxwQmz/Ilh7FyIS0ApNXRpIogBs8dn8Neb\nFrCpuJq/rC7i1Q2HqW+2hp5o8LTw/NqDPL/2IKeNSWXJvPFcOv0EkhMCPhrJ2TBzCVurxzDy7LOh\nZJPVt7BnBRxYaQ2P3ca0wqG11vLhL8GRBOPP8CeGkVOt2odSasjQRBBDpuWm8bPcfO665BRe2XCY\nZZ8Wsf1obfvxzYdquPOlTdz/+jYWzzyBb84dz9QTUoOfxGaD0dOtZeH3wNMIB1fB3hVWcji8AQgY\n8M5TD7vftRawEkPOab7nyKXJC6AAABdJSURBVLceR5wCcfEopaJDE0EMSnE6uGb+eK6eN471B6r4\ny6oD/GPjYZq8Vj+Au8nLnz89wJ8/PcDMceksmTc+6G7mIA4nnHiOtQDUV8D+D63awt5CqNwXXN5T\nD8WrraWNzQE5U62kMCofRs+AnFMhPmngf3ilVCeaCGKYiDB7fAazx2dw95dO4cX1h/jLqiL2lNa1\nl/nsQBWfHajivr9vYV4OpJ5YyYzcdGw2Cf2kSZkw9TJrAajcD3vft2oMRSvBfbTzOa0eOPK5tbQH\nZ4Psk4JrDqOmgTOt8/lKqX7RRKAASE+K58YzJ3DDwjxW7atg2aoD/HPzETwtVjNPTaOXd4rgnV9/\nQrYrnnNPHsn5p+Rw5uRskuK7+Rhl5MHsPJh9nbVdWwJHN/q/+I98DlVFnc8zrVC6zVo2LvfvT82F\nESfBiJODH/VKJaX6TBOBCiIizD8xi/knZlHmnsoL64r5y6oDHKjwDzlR5m7mr2uL+evaYhLibJw5\nKZvzp+Zw3skjGZnq7P4FUnIg5QKYfIF/X0MlHNkYnCDKdhHU19Cmptha9rwXvN81iulxI6F+QUCC\nOBmSs/r+ZigVIzQRqC5luxK46ZyJLD3rRD7eU8ZT/1zP1iob5XX+uQ2avK28t/0Y720/BsD03DTO\nPyWH86fmcPKoFES6aEIKlJgR3M8A0OSGki3BNYfSbdDqDf0c7qNkcBRWbwzen5QdUHs4yaqhpI+D\ntLGQ4Ar5VErFGk0Eqkc2m3DW5BG0HErgrLPPYcPBKt7dVsK7W0vYdcwdVPbz4mo+L67moXd2MiY9\nkfNPGcn5U3OYNyGL+LheDEuR4IJx86ylTYsHKvZC6XYo3eF/LNsJLV1MvFNfBkUfWUtHSVlWQkgf\n51vG+x59+xJSwo9XqeOYJgLVK3abv4P59otOpqi8jne3HePdrSWs3l9BS6u/OedQVQPPrizi2ZVF\nuBLiOGfKCM6cnM38E7PIy0oKr7YQ9OIO/3/2gVq8UFXEphUvMC0nLiBJ7AweVK+j+nJrObIh9PHE\nDH/toS1JZE2EzBOtbbv++ajhQT/Jql/GZyVz45kTuPHMCVTXeyjceYx3tx2jcPsxapv8zTjuJi+v\nbzrC65uOAJCTmtDeF7HgxCzG9yUxtLHHQdZEyrPnwVkF/v2trVB9wJ8YynZC1UGoOgDVxdbVSt1p\nqPT1X3ze+ZgtzmpmyppkLZkn+tYnQsoJOiifOq5oIlADJi3JwWUzxnDZjDE0e1tZs7+Cd7aW8O62\nEoorg/8zL6lp4tUNh3l1w2EARqU6mX9iZnty6FdiaGOzWV/WGXkw5cLgY60t4C6xkkKopfpg181N\nYPVVtA2t0VFcor/m0JYcsibhaK6ykpMmCTXEaCJQEREfZ2PhpGwWTsrmv788lZ0lbt7feYxVeytY\nva8iqLYAcLSmkVc2HOaVDolhwUQrMYzLHIDEEMhmh9QTrGXc/M7HW1uh7lhwcqjcb/VRlO+G2iNd\nP7e3AUo2W0uAhQArbZCYaQ3bkZRlLe3rvsfkwPVsHatJRZwmAhVxIsJJo1I4aVQKS8+eSEurYevh\nGlbuLePTvRWsCSMxjE5zMm9CJjPHZXDamDSmjk4lMT6CYxbZbJAyylrGzu18vMntTwoVe6z5n8t3\nW48dB+ULZFqtDuz6svBjiXf5k4Yrx7oE19VhScmxpiR19HD5rlIhRDQRiMhFwP8CduB3xpgHOhy/\nHvgFcMi36zFjzO8iGZOKPrtNmJabxrTcNJaePRFvSytbj9Tw6d5yPvXVGNwdEsOR6uDEYLcJk0e6\nmDbGep5pY9Jobglx30GkJLh8dzzndz5WX2ElhIqA5FC+G2/pHuJa6jqX70mz21pC3XjXkTMNXKPA\nNdJKYq4ca901ypplzplulUlIgYRUTRwKiGAiEBE78DhwAVAMrBGR14wxWzsUfd4Yc0uk4lBDX5zd\nRn5uOvm56WEnhpZWw/ajtWw/Wsvf1hUDYBM4afOHTBuT6ksQ6Zw8KgWnY5BHO03KtJaxpwft/qiw\nkIIzz/BfrVRfZj3WBWzXlQUc9y1d3TsRSmO1tZTtCK+8Pd5KCM5USEhhekMrHB1v7UtI8e1P9T+m\n5VpXT7lyYCCb6lRURbJGMBfYbYzZCyAiy4HLgI6JQKkgXSWGNfsr2Xyomk2HqtlT6sZ0qAC0Gth2\npIZtR2r461orOcTZhCk5KUwbk8ZpuWlMHZ3C5JwUazKeaIiLh9TR1hIOY6wv9vpya0Ig9zGrk7tt\nqQ1Ydx8D09K7eFqag5qqMgCqNoXxczg733+R0XYfRp6VCDVRHDfEdPxrGqgnFrkcuMgY823f9jXA\nvMD//n1NQz8DSoGdwH8YYw6GeK6lwFKAnJyc2cuXL+9YJCxutxuXa+jeTarxha/BazhQ08r+mlb2\nVbewv6aVo3Xhf5aznMKYFBu5Lhu5KTbGuITRyTbi7ZH78or4+2dacXhqiW+u7LQkNFXi8FQR5633\nLXXYW+qx9TZxhKnF5qQhMYdG50jfYq17HKl445LxxrnwOFy02hJ6lTCG0mcwlKEc36JFi9YZY+aE\nOhbtzuK/A88ZY5pE5N+BZ4FzOxYyxjwFPAUwZ84cU1BQ0KcXKywspK/nDgaNr3/efHcFGSfms6nY\nqjVsOlTNvrLQbfLljYbyxhY2lvq/CO02IS8ryerYzknlpFEupuSkMD4rGXtXo632wpB7/4yxJhVq\nrIEma/l81YdMP3mCb1+ttb+xBpqqoaHKfwVVY1W3T21vbcRVV4Srrod+DZvDGjDQme5/dKZ13ud7\nXLOlnNPnzLWuvBqCN/QNud9xmCL5Th4CAmc1z8XfKQyAMaY8YPN3wM8jGI8a5hLj/APmtalp9LD5\nULVvqWFnSS17St3to6oGamk17CmtY09pXfs8zwAJcTYm51hJYeIIFydmJzNhRDJ5WcmD3/8wkETA\nkWgtKTkAVGbWwtSCns9trPZdUlvkSw5FAdtFVud2OFo9VpNXXWlYxU8HWAsgVnJIyvZffps8wrce\nuC/bfymuTn7UpUgmgjXAZBGZgJUArgS+GVhAREYbY9ouyL4U2BbBeFQMSnU6OGNiNmdMzG7f52lp\nZV9ZHTuO1rKzxOpw3llSy4GK+k79DmANrLf5UA2bD9V0OjYmPZEJ2cn+ZUQyE7KSyc1IJM4+jG8c\nc6ZZ80OMmtb5mDHWHdlVRcGJorrYd7d2lVWjaKiClqY+BmD8d36X7wrvlIQ062ovR5I16ZEj2feY\nBPHJ1tK23qlM22OidcOgwxn8aHcc130iEUsExhiviNwCvIV1+ejTxpgtInIfsNYY8xpwm4hcCniB\nCuD6SMWjVBuH3caUnBSm5AQPKlff7GVXiZsdR2vZUeJPEqW1XX9ZHapq4FBVAx/tDr4vwGEXxmYm\nWbWH7GQmZLuoLm9hSlUDOanOAWlqGrJE/FdOnTCz+7KeBn9iaKwOThIdHxsqqS87SBL1VgLoraZq\na4kEsUFcImdgh8/SrM70jsnC4bTuCYl3WQkp3mVdmRW4HXTMBfEpg9IEFtFXMMa8AbzRYd89Aet3\nAndGMgalwpUUH8f0selMHxs8yU1FXTM7S2rZVVLL3rI69vmWgxX1tHbRP+1pMewtrWNvaXAfxYNr\n/oXDLuRmJJGbkci4zCTGZSYxNuAxLTFKVzRFQ1vTVJhXUa1ua4Nv8Vo37tWVBV92275d6rss13es\nocK6mS9STCt46ogHqO5cc+yXOGdwYvja72DkyQP7EgP6bEoNQ5nJ8Z36HgCava0cqKj3JQZ3e4LY\nV1ZHSU3XtQhPi2kvF0qqM45xWb7EkGElh7ZEMTrNeXz3SwwUe5zvRrmR4ZVvbbFqHM1uaK4HT53v\nsR6a66ylbd1TH7qMx7fuabQ62T0N/scIXX0FWK/hbezd3ei9pIlAqT6Kj7MxaaSLSSNdQE7QMXeT\nl/0BiWFfWR2f7z1CtTcuaGKfUGoavV32SYA1YdCYdCcnpCcyJj2RE3yLte4kMzl+YMdlGg5sdn9z\nVSS0eMDTwMcf/IuFp8/wJYuG4Mf2pOO2hihpu2O8bb2pNmC7zr+v40x9EZhQSROBUhHgSojjtDFp\nnDYmrX1fYWEVBQUF1DV5OVhZz4Hyeg5WNnCwop6DFfUcqKjnYGU9jZ7umzDK3E2UuZv4vDh0e3dC\nnK09QfgThZU4clITGJHiJNUZp8liINkdYHfgiU+3RrsdKMZYCSQwcbhyej6vlzQRKDXIkhPiOHlU\nKiePSu10zBhDqbvJlxwaONCWIHxLSW1T0OQ/oTR5W9lbVsfeLpqewEoWI1ISGJGSwMj2RycVhz20\nbCtp385yxeMYzlc/DXUi/iuaGPgE0EYTgVJDiIgwMsXJyBQns8d3Pu5taaWktonDVQ0c9l2xZK03\nWtuVDZ1Gcg2lydtKcWVDp3kiAJ7ZsjYgHshMim9PGiNc1mO2K4HslHiyA7YzkuKH99VQw5gmAqWO\nI3F2q9lnTHpil2VqGj0BicKfII5UN3CstoljNU00eMLr3DQGyuuaKa9rZvvR2m7L2gSyXL4k4Ypv\nTxxtSWOEy+q/yHLFk5EU37s5rFVEaSJQaphJdTpIHeUI2fTUxt3kpbS2iWM1jZS6reRQ6m5i484i\n7K4MjtU0UuZuoryuOeRNdqG0Giitber2vovgOOPIciWQmRxvJQjfY1uyyEpOaF/PTI4nIU6vlooU\nTQRKxSBXQhyuhDgmZCcH7S90HqWgwD8Rj6ellYq6Zl+iaKSstplSt/VlX9bhsaaxF8NlY10dVdPo\n7fIy2lAxO20tjN70EelJDtKT4klPdJCR5CAtKZ6MJEfQ/vSkeNISHdpcFQZNBEqpLjnsNnJSneSk\nOoG0bss2eVsodze3JwdrsbbbkkdFXTMVdc1U1odf02jjbvLiBsoO9e7u4FRnHBnJ/uSQmRxPepKD\nzKR40pOtBJKZFE96UjwZyQ4ykuJj7l4NTQRKqQGREGdvv6ehJy2thqp6KymU1wU8upupqGvy73Nb\n+yvrm3u8WqorbTWPMOZ3a5fosJOR5CAj2erPyPAljFSng9TEOFKc1nqKM47URN+j0zG4s+QNIE0E\nSqlBZ7cJWa4EslwJTA6jfGuroabRw1srPuKk/FlU1jdTXe+hsr6ZqnoPVfXNVDV4qKz3UF3fTKVv\nX2+bq9o0eFpoqG7hcHVjr8+NX/Emqb7E0DFRpCY6rGOJ/qTStj/Nt8/psA36PR6aCJRSQ57NJqQn\nxZOTbGNGh7GguuNtaaWm0UtVQHKoqLOSR2W9VdOorAtYr/dQWdeMt4+1D7CGHilzN1Pm7v4O8q44\n7NJt0rj2jLxurxrrC00ESqlhK85ua78SKVzGGNxNXqrqPe39GW3rtY1eaho91DZ6qGnwUtvke2z0\nUNPopbq+mf62DnlaTPslu6FcNmOMJgKllIokESHF6SDF6WBsZlKvzl2xYgXzF57tSwxWcqhp8FDb\n6KXa91jT6KGmwX+sptFDdYOVUGoaPDS3dD/ESGriwH9tayJQSqkBIiIkxttJjLczMtXZp+do9LT4\nkkXnpFHd4CErOWGAo9ZEoJRSQ4rTYcfpsDMypeeyA0Xv8VZKqRiniUAppWKcJgKllIpxmgiUUirG\naSJQSqkYp4lAKaVinCYCpZSKcWJ6OxZslIlIKfRqIMFA2UDZAIYz0DS+/tH4+m+ox6jx9d14Y8yI\nUAeOu0TQHyKy1hgzJ9pxdEXj6x+Nr/+GeowaX2Ro05BSSsU4TQRKKRXjYi0RPBXtAHqg8fWPxtd/\nQz1GjS8CYqqPQCmlVGexViNQSinVgSYCpZSKccMyEYjIRSKyQ0R2i8gdIY4niMjzvuOrRCRvEGMb\nKyIrRGSriGwRke+FKFMgItUissG33DNY8flef7+IbPK99toQx0VEHvG9fxtFZNYgxnZSwPuyQURq\nROT7HcoM+vsnIk+LyDER2RywL1NE3hGRXb7HjC7Ovc5XZpeIXDdIsf1CRLb7fn8vi0jIiYB7+ixE\nOMZ7ReRQwO/xki7O7fbvPYLxPR8Q234R2dDFuYPyHvaLMWZYLYAd2AOcCMQDnwNTO5T5LvCEb/1K\n4PlBjG80MMu3ngLsDBFfAfCPKL6H+4Hsbo5fArwJCDAfWBXF3/VRrBtlovr+AWcDs4DNAft+Dtzh\nW78DeDDEeZnAXt9jhm89YxBi+wIQ51t/MFRs4XwWIhzjvcAPw/gMdPv3Hqn4Ohx/CLgnmu9hf5bh\nWCOYC+w2xuw1xjQDy4HLOpS5DHjWt/4CcJ6IyGAEZ4w5YoxZ71uvBbYBYwbjtQfQZcAfjeVTIF1E\nRkchjvOAPcaYvt5pPmCMMR8AFR12B37OngUWhzj1QuAdY0yFMaYSeAe4KNKxGWPeNsZ4fZufArkD\n+Zq91cX7F45w/t77rbv4fN8dXweeG+jXHSzDMRGMAQ4GbBfT+Yu2vYzvj6EayBqU6AL4mqRmAqtC\nHF4gIp+LyJsicuqgBgYGeFtE1onI0hDHw3mPB8OVdP3HF833r02OMeaIb/0okBOizFB4L2/AquGF\n0tNnIdJu8TVfPd1F09pQeP/OAkqMMbu6OB7t97BHwzERHBdExAW8CHzfGFPT4fB6rOaO6cCjwCuD\nHN6ZxphZwMXAzSJy9iC/fo9EJB64FPhbiMPRfv86MVYbwZC7VltEfgx4gWVdFInmZ+E3wERgBnAE\nq/llKLqK7msDQ/7vaTgmgkPA2IDtXN++kGVEJA5IA8oHJTrrNR1YSWCZMealjseNMTXGGLdv/Q3A\nISLZgxWfMeaQ7/EY8DJW9TtQOO9xpF0MrDfGlHQ8EO33L0BJW5OZ7/FYiDJRey9F5HrgS8ASX6Lq\nJIzPQsQYY0qMMS3GmFbgt128dlQ/i77vj68Cz3dVJprvYbiGYyJYA0wWkQm+/xqvBF7rUOY1oO3q\njMuBf3X1hzDQfO2Jvwe2GWMe7qLMqLY+CxGZi/V7GpREJSLJIpLSto7Vqbi5Q7HXgGt9Vw/NB6oD\nmkAGS5f/hUXz/esg8HN2HfBqiDJvAV8QkQxf08cXfPsiSkQuAn4EXGqMqe+iTDifhUjGGNjv9JUu\nXjucv/dIOh/YbowpDnUw2u9h2KLdWx2JBeuqlp1YVxP82LfvPqwPPYATq0lhN7AaOHEQYzsTq4lg\nI7DBt1wC3ATc5CtzC7AF6wqIT4EzBjG+E32v+7kvhrb3LzA+AR73vb+bgDmD/PtNxvpiTwvYF9X3\nDyspHQE8WO3UN2L1O70H7ALeBTJ9ZecAvws49wbfZ3E38K1Bim03Vtt622ew7Sq6E4A3uvssDOL7\n9yff52sj1pf76I4x+rY7/b0PRny+/c+0fe4CykblPezPokNMKKVUjBuOTUNKKaV6QROBUkrFOE0E\nSikV4zQRKKVUjNNEoJRSMU4TgVKDyDcy6j+iHYdSgTQRKKVUjNNEoFQIInK1iKz2jSH/pIjYRcQt\nIr8Sax6J90RkhK/sDBH5NGBs/wzf/kki8q5v8Lv1IjLR9/QuEXnBNx/AssEa+VaprmgiUKoDETkF\n+Aaw0BgzA2gBlmDd0bzWGHMq8D7w375T/gjcbozJx7oTtm3/MuBxYw1+dwbWnalgjTj7fWAq1p2n\nCyP+QynVjbhoB6DUEHQeMBtY4/tnPRFrwLhW/IOL/Rl4SUTSgHRjzPu+/c8Cf/ONLzPGGPMygDGm\nEcD3fKuNb2wa36xWecBHkf+xlApNE4FSnQnwrDHmzqCdInd3KNfX8VmaAtZb0L9DFWXaNKRUZ+8B\nl4vISGife3g81t/L5b4y3wQ+MsZUA5UicpZv/zXA+8aafa5YRBb7niNBRJIG9adQKkz6n4hSHRhj\ntorIT7BmlbJhjTh5M1AHzPUdO4bVjwDWENNP+L7o9wLf8u2/BnhSRO7zPccVg/hjKBU2HX1UqTCJ\niNsY44p2HEoNNG0aUkqpGKc1AqWUinFaI1BKqRiniUAppWKcJgKllIpxmgiUUirGaSJQSqkY9/8B\nhwTKclQFLS8AAAAASUVORK5CYII=\n",
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ]
          },
          "metadata": {
            "tags": []
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "7e1AOrb7PCQ2",
        "colab_type": "text"
      },
      "source": [
        "Evaluate on the entire dataset!"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BpbhEoBGPDs6",
        "colab_type": "code",
        "outputId": "0ef4f1ae-4feb-44ca-89eb-25f767fdcbc5",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 119
        }
      },
      "source": [
        "predicted_output = net(images)\n",
        "print(torch.max(predicted_output, 1))\n",
        "fit = Loss(predicted_output, labels)\n",
        "print(labels)"
      ],
      "execution_count": 0,
      "outputs": [
        {
          "output_type": "stream",
          "text": [
            "torch.return_types.max(\n",
            "values=tensor([ 6.0837,  3.1317,  9.1140,  7.7421,  6.6457,  6.1580, 10.1397,  4.2204,\n",
            "         6.8977, 11.5810, 10.6048, 10.2698,  6.7915,  4.7407,  9.4972,  4.4949],\n",
            "       device='cuda:0', grad_fn=<MaxBackward0>),\n",
            "indices=tensor([3, 1, 7, 5, 8, 2, 5, 6, 8, 9, 1, 9, 1, 8, 1, 5], device='cuda:0'))\n",
            "tensor([3, 2, 7, 5, 8, 4, 5, 6, 8, 9, 1, 9, 1, 8, 1, 5], device='cuda:0')\n"
          ],
          "name": "stdout"
        }
      ]
    }
  ]
}