Autoencoder.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "qFoq-khqMAh7"
   },
   "source": [
    "### Autoencoder\n",
    "\n",
    "AutoEncoder 은 아래의 그림과 같이 단순히 입력을 출력으로 복사하는 신경 망(비지도 학습) 이다. \n",
    "<div><img src=\"https://img1.daumcdn.net/thumb/R1280x0/?scode=mtistory2&fname=http%3A%2F%2Fcfile2.uf.tistory.com%2Fimage%2F996C93475BDC97C00AE328\" height=\"100%\" width=\"100%\" /></div><br>\n",
    "\n",
    "아래 링크는 AutoEncoder에 관한 개념 설명이 나와있다.  \n",
    "<a href=\"https://wjddyd66.github.io/tensorflow/2019/08/28/Tensorflow-AutoEncoder.html\">Auto Encoder</a>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "XXpqf4nGMAh8"
   },
   "source": [
    "### 1. Settings\n",
    "#### 1) Import required libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "gOFx3VQpMAh-"
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import torch\n",
    "import torch.nn as nn\n",
    "import torch.optim as optim\n",
    "import torch.nn.init as init\n",
    "import torchvision.datasets as dset\n",
    "import torchvision.transforms as transforms\n",
    "from torch.utils.data import DataLoader\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "OBDG968zMAiC"
   },
   "source": [
    "#### 2) Set hyperparameters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "oxPTgvcjMAiD"
   },
   "outputs": [],
   "source": [
    "batch_size = 256\n",
    "learning_rate = 0.0002\n",
    "num_epoch = 5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "7uaV_HtJMAiG"
   },
   "source": [
    "### 2. Data\n",
    "\n",
    "#### 1) Download Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "he-GrlguMAiG"
   },
   "outputs": [],
   "source": [
    "mnist_train = dset.MNIST(\"./\", train=True, transform=transforms.ToTensor(), target_transform=None, download=True)\n",
    "mnist_test = dset.MNIST(\"./\", train=False, transform=transforms.ToTensor(), target_transform=None, download=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "rKz97cYDMAiJ"
   },
   "source": [
    "#### 2) Set DataLoader"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "aHAHmwe-MAiJ"
   },
   "outputs": [],
   "source": [
    "train_loader = torch.utils.data.DataLoader(mnist_train,batch_size=batch_size, shuffle=True,num_workers=2,drop_last=True)\n",
    "test_loader = torch.utils.data.DataLoader(mnist_test,batch_size=batch_size, shuffle=False,num_workers=2,drop_last=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "gWlU96lLMAiM"
   },
   "source": [
    "### Stacked AutoEncoder\n",
    "\n",
    "Stacked AutoEncoder는 여러개의 히든 레이어를 가지는 Auto Encoder이며, 레이어를 추가할수록 AutoEncoder가 더 복잡한 코딩을 학습할 수 있다.  \n",
    "Stacked AutoEncoder는 아래의 그림과 같이 가운데 히든레이어를 기준으로 대칭인 구조를 가진다.  \n",
    "\n",
    "<div><img src=\"https://img1.daumcdn.net/thumb/R1280x0/?scode=mtistory2&fname=http%3A%2F%2Fcfile9.uf.tistory.com%2Fimage%2F99F1DE4A5BDC983D09BAED\" height=\"100%\" width=\"100%\" /></div><br>\n",
    "\n",
    "이번 Model은 MNIST를 분류하는 Stacked AutoEncoder를 구현하는 것 이다.  \n",
    "<br>\n",
    "**Encoder:** [batch_size,1,28,28] -> x.view(batch_size,-1) -> [batch_size,784] -> nn.Linear(28 * 28, 20) -> [batch_size,20]  \n",
    "<br>\n",
    "**Decoder:** [batch_size,20] -> nn.Linear(20 , 28 * 28) -> [batch_size,784] -> self.decoder(encoded).view(batch_size,1,28,28) -> [batch_size,1,28,28]  \n",
    "\n",
    "\n",
    "#### 1) Model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "mTd6rSIFMAiN"
   },
   "outputs": [],
   "source": [
    "class Autoencoder(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Autoencoder,self).__init__()\n",
    "        self.encoder = nn.Linear(28*28,20)\n",
    "        self.decoder = nn.Linear(20,28*28)   \n",
    "                \n",
    "    def forward(self,x):\n",
    "        x = x.view(batch_size,-1)\n",
    "        encoded = self.encoder(x)\n",
    "        out = self.decoder(encoded).view(batch_size,1,28,28)\n",
    "        return out"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "yLiqw-rpMAiP"
   },
   "source": [
    "#### 2) Loss func & Optimizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 34
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 22418,
     "status": "ok",
     "timestamp": 1559565570677,
     "user": {
      "displayName": "Choi Gunho",
      "photoUrl": "",
      "userId": "04388737836176863066"
     },
     "user_tz": -540
    },
    "id": "ZKc1UKNgMAiQ",
    "outputId": "cf912acf-d66e-4d9d-eefc-e64a24699806"
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda:0\n"
     ]
    }
   ],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "print(device)\n",
    "\n",
    "model = Autoencoder().to(device)\n",
    "loss_func = nn.MSELoss()\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "_W2UI2F_MAiT"
   },
   "source": [
    "#### 3) Train "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "a7NQzW7mMAiU"
   },
   "outputs": [],
   "source": [
    "loss_arr =[]\n",
    "for i in range(num_epoch):\n",
    "    for j,[image,label] in enumerate(train_loader):\n",
    "        x = image.to(device)\n",
    "        \n",
    "        optimizer.zero_grad()\n",
    "        output = model.forward(x)\n",
    "        loss = loss_func(output,x)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        \n",
    "    if j % 1000 == 0:\n",
    "        print(loss)\n",
    "        loss_arr.append(loss.cpu().data.numpy()[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab_type": "text",
    "id": "Pmru4P0_MAiX"
   },
   "source": [
    "#### 4) Check with Train Image\n",
    "\n",
    "AutoEncoder의 Stacked AutoEncoder는 입력과 출력이 같기 때문에 Input Data만으로도 Model의 성능을 판단할 수 있다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 5074
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 6094,
     "status": "ok",
     "timestamp": 1559565717017,
     "user": {
      "displayName": "Choi Gunho",
      "photoUrl": "",
      "userId": "04388737836176863066"
     },
     "user_tz": -540
    },
    "id": "OvIYXu3lMAiY",
    "outputId": "aece479b-8ff9-4392-d497-61dc93f5f9f7",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([256, 28, 28])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAC4CAYAAAD61bdSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAXSUlEQVR4nO3dXWycVXoH8P9jxwkkDiEfJJjESWBJwveHZEVA9oJqS0VXK8FeEMLFiotVsxeLtEYrVYibXVWqhNSy0ItqpaxApBKFrATbRRVquwor0YgGCIivQKgjcIiDsZ0PEicmgJOnF55Uzpz/g+f1zDv2mfx/EsJ+cjxz3nmPTybzPOccc3eIiEh+2ma6AyIiMj2awEVEMqUJXEQkU5rARUQypQlcRCRTmsBFRDJV1wRuZneb2cdmtt/MHmlUp0Rmmsa25MCmWwduZu0A/hfAXQAGALwJ4AF3//A7fkZF51Iqd7d6H2M6Y3vOnDk+d+7c6sehbc+cOZPEOjo6aNuzZ8/W2GuA/S63tfH3aFGcYf0tMm+w1yG6riL9KoI9X/RcRV7HIq9DkXtZ/Xxff/01xsfHkxdyTs2PmNoIYL+7fwIAZvY8gHsAhINcJBOFx/bcuXOxYcOG82Jz5vBfrxMnTiSxrq4u2nZ0dDSJRY/7zTffJLGLLrqItr344otpvNY+fPvtt7Qtm6zb29uT2OnTp2vuV5GJL/pLkz1f9Bqwa4vaFunb2NhYzW2r3wzs27ePtqvnr7uVAA5O+n6gEjuPmW01sz1mtqeO5xJppsJje3x8vGmdEzmn9CSmu29z9x537yn7uUSaafLYjt4Vi5SpnlF3CED3pO9XVWIiuSs8tt0dtb4Lv+SSS5LY119/Tduyj1uij0VYnH18AQDHjx9PYuyz7uhxo49AWFv2l9u8efPoz7PrjT5/Zq/jl19+Sduy51u8eDFtOzIyksSij0rmz5+fxKJxUOReVj9G9Fl7Pe/A3wSwzsyuNLO5ALYAeKmOxxOZLTS2JQvTfgfu7uNm9hCA/wTQDuBpd9/bsJ6JzBCNbclFXR/cufvLAF5uUF9EZg2NbcmBVmKKiGRKE7iISKZU+yTSAG1tbUlFQVSNwKoyojLEzs7OJBZVRLAFKFEf2OKcFStW0LanTp1KYlEVCYuz6pYiVSwR9hhF+hVV/rCqF1ZBAvDFOdWLcM659NJLk1h0f6r7G66opVEREZn1NIGLiGRKE7iISKY0gYuIZEpJTJEGqU40nTx5krZjiTq2kyBQbGc8lhCLltKzZeRRcpTt8Bft+seW6C9YsIC2ZVjS9quvvqJt2WsTJYOPHj1ac7/YzpCDg4O0bZE9cFjieNGiRbRttK1BNb0DFxHJlCZwEZFMaQIXEcmUJnARkUxpAhcRyZSqUEQawN2TSpLLL7+ctmUVEWyZNcArTqLKhyIVEUUqONgydHaQAQAsXbo0ibFl4FdddRX9+SJnTLLDG6KqG7YdwLFjx2hbtjw+WsrO2i5btoy2ZdcWnZNZXVEUVaXoHbiISKY0gYuIZEoTuIhIpjSBi4hkqq4kppn1AxgFcAbAuLv3NKJTF6Lu7u4k9vzzz9O2d9xxRxKLkj8s+RK13bRpUxLbvXs3bdvqio7t9vb2ZBl4lCRjoiQZWx4f3T+WEIv2x+7o6EhibBl7JEq6su0D2HiNTllnJ81HyVWWxIz2+D506FASi7Y6YIljtgwe4HuoR/eHbZcQ7R1e/fpEWxc0ogrlL9z9cAMeR2S20diWWU0foYiIZKreCdwB/JeZvWVmWxvRIZFZQmNbZr16P0L5vrsfMrPlAP5kZvvc/dXJDSqDX78AkptCYzv6rFmkTHW9A3f3Q5X/DwP4A4CNpM02d+9RglNyUnRss6SgSNmm/Q7czBYAaHP30crXfwXg7xrWswsMqzjZuDGZMwDwLHeRJchR297e3iS2ZcuWmh+3VUxnbI+PjydVEQsXLqRtWcVJdGgBq1yIlmqz5dZRW1b9MDw8TNuyypBoyfqaNWtqahtVsbBKmFWrVtG27733XhKLTrU/cuRIEtuwYQNtOzIyksQuu+wy2padVh/9Zc76xl5bID1AIqraqecjlBUA/lApb5kD4F/d/T/qeDyR2UJjW7Iw7Qnc3T8BcHMD+yIyK2hsSy5URigikilN4CIimdJ+4LPE7bffnsSixAVbVhstxS7S9vXXX/+uLsoUqpe9FzmVnu3PDfCEM1teD/DxMjQ0RNuyZGO0lJ4tF4+SmCxxy/YOjxJ9LGnLlrZHcZZUBHjS9vPPP6dt2e9HdL3scaO9u9l96+/vp22r95KPthPQO3ARkUxpAhcRyZQmcBGRTGkCFxHJlCZwEZFMWVTpUMqTmTXvyTLDMteNOKShSNtW2M/D3fnO9yWbP3++r1+//rxYtAk/q2g4fvw4bcs2yYruE6tkiaonWN+iKhQ2Nu+77z7ali0NZ5UhUXUMq8aKNgr77LPPktgHH3xA27IDGU6fPk3bsutlh0dEoiqvo0eP1tyH6sfo6+vD2NhYctP0DlxEJFOawEVEMqUJXEQkU5rARUQypaX0M2DHjh1JrKzl8ew07s2bN0/VRSnI3ZMT0aO9qdmy6Gjfbra0PLrXzLFjx2h8wYIFSSxKpLKk965du2hbts/3zTenGztec8019OfZ9gNF9lW/9tpraVuW3GSvLQAcOHAgiUXFHmwf9yjJzJKjUZK5euxE91zvwEVEMqUJXEQkU5rARUQypQlcRCRTmsBFRDI1ZRWKmT0N4EcAht39hkpsCYAdANYC6Aew2d15ulsSLKPNYo1YHs8qTnbv3j1VFy8IjRzbbW1tSWVHdPBCdbUKEFesLFq0KIkdPHiQtmWPsXjxYtr23XffTWLRyevXX399EotOlWeVFqyKJKrUYAck7N27l7ZlFSAsBgD79u1LYtGJ8ExUocNe8+i+L1myJIlFB1DUqpZ34M8AuLsq9giAne6+DsDOyvciuXkGGtuSsSkncHd/FUD1Liz3ANhe+Xo7gHsb3C+R0mlsS+6mu5BnhbsPVr7+AkB6aF6FmW0FsHWazyPSbNMa2+yf/iJlqzuJ6RMf3obbxLr7Nnfvcfeeep9LpJmKjO3o0FmRMk131A2ZWZe7D5pZF4DhRnaqVTz++OM0zvZSLmspvRKWhU1rbJtZsm91dNI82986+gvgyJEjSSxadv/xxx8nsSihxpZwR0k9dir94OAgaQls2rQpibHT45cvX05/nj3uunXraNs33ngjiUX7jK9atSqJRclRliReunQpbct+7+bPn0/bsmuLig+qk6NhQQONTu0lAA9Wvn4QwB+n+Tgis43GtmRjygnczJ4D8D8ANpjZgJn9FMBjAO4ysz4Af1n5XiQrGtuSuyk/QnH3B4I/+kGD+yLSVBrbkjutxBQRyZQmcBGRTKn2qUS9vb00zjLKZZ00L81x5syZZFk0WwYP8OqU6NRztsVCVN3ClsJ/8skntC3DlswDvBLmrrvuom3Z2GSVMFFVB6vQYVUwAPDOO+8ksYGBAdqWVYZErzlb5j8yMkLbsuqWsbEx2pYdVrFy5Ura9tSpU+d9Hx0ooXfgIiKZ0gQuIpIpTeAiIpnSBC4ikiklMUvElrwDPNFT71J6LZmfWe3t7ejs7DwvFi1jZ6ehV//sOSzxxWIAcPRo9caKcR/YHt3RKe0siRiddr969eoktmHDhiTG9g0HgBtvvDGJsWQlAIyOjiax6uTfOWwZO9uXHQDWrl1b8+Oyx4i2RWCPGyVSo/tWTe/ARUQypQlcRCRTmsBFRDKlCVxEJFNKYpYoWj1VxkrMJ598smDvpJHMLDmVZ3iYbyXOEpbR6ko2hqIDhdk+1mzfb4AfsNvX10fbsiRmtOf11VdfncTYwcrRHt+ffvppEoteR5bMjZLB7PWNVsqy1assAQnwQ5Sj4gPW3+gkp+rHCAsiaFRERGY9TeAiIpnSBC4ikilN4CIimdIELiKSqSmrUMzsaQA/AjDs7jdUYr8G8DcAzm2S+6i7v1xWJ3OwY8eOJNbMpfTRc0mskWP7zJkzOH78+Hmx6JR3dq+iZd1syfuhQ4doW7aEe8GCBbQtq9Zgp8cDvIIjqrBiS8CrXxcAOH36NP356j3VAaC/v5+27erqSmLR9b711ltJLNrju/pEeCB+za+88sokVmRv92j7gurriOaCWt6BPwPgbhJ/wt1vqfx3QU/ekq1noLEtGZtyAnf3VwGku+SIZE5jW3JXz2fgD5nZe2b2tJmllfoVZrbVzPaY2Z46nkukmQqP7Vp3jxNppOlO4L8F8D0AtwAYBPB41NDdt7l7j7v3TPO5RJppWmM72kJUpEzTGnXuPnTuazP7HYB/b1iPMsUSFM1cSh89lxRTz9iu9R6whGV0r9k7+yhhzfbYjpJ6rC1LIAI8OVokac6Wm7/yyiu0LdvTvLu7m7bdtWtXEosSnuw1Z0lQgG9JsHz5ctqWJSyjQ5hZMjd6zavve0MPNTazyVf+YwAfTOdxRGYbjW3JSS1lhM8BuBPAMjMbAPArAHea2S0AHEA/gJ+V2EeRUmhsS+6mnMDd/QESfqqEvog0lca25E4rMUVEMqUJXEQkU6p9KijKiLN4WUvp2bLeaKmvNE/1/ero6KDt2AnnUVsmOkyBVVXs3buXtl25cmUSu+mmm2hbtsQ+OlTiiy++SGLr169PYgMDA/Tnx8bGkhirTAF4BUh02j072CJ63DVr1iSxaMk72+pgcHCQtmXVPPPmzaNtq8dIWJFGoyIiMutpAhcRyZQmcBGRTGkCFxHJlJKYBd122200vnHjxiRW1lL61157LYnt3r2btpXmaGtrSxJSLFkJ8L24o/3Aa/356PluuOEG2pYlz6KkO9sf+9ixY7QtS24ePnw4ibGT6qOfj7Dfmc8//5y2ZUnM6HqLnB7Pks/RvjhsPli0aBFtW72Uni3vB/QOXEQkW5rARUQypQlcRCRTmsBFRDKlCVxEJFOqQmmQWpfHF2lb5FR6mVnunizjju4fO+U9WqrNsKoQgC8Bj5bos4qI6NCCVatWJbHo2tjydFbBER16cODAgSS2b98+2pYtpY+2Gfj000+TGDtRHuCVLNH1sgM3ogobdrDF0NAQaZluX6Cl9CIiLUYTuIhIpjSBi4hkShO4iEimajkTsxvAvwBYgYlzAre5+z+Z2RIAOwCsxcTZgZvdna+vvQDUujy+SNsocfHEE08U7J0wjRzb7p4ktKL7x5J6UbKRJfuixBdbjr969WraliUA2R7hAF/eHp12PzIyksRYYnF0dJT+PEsKRklbtnd49JqzttEydpbEXLJkCW3Lks+nT5+mbdm1Rfe9kafSjwP4pbtfB+A2AD83s+sAPAJgp7uvA7Cz8r1ITjS2JWtTTuDuPujub1e+HgXwEYCVAO4BsL3SbDuAe8vqpEgZNLYld4XqwM1sLYBbAbwOYIW7nzs76AtM/DOU/cxWAFun30WR8tU7tosciSbSKDUnMc2sE8ALAHrd/cTkP/OJD2johzTuvs3de9y9p66eipSkEWM72kJUpEw1TeBm1oGJAf6su79YCQ+ZWVflz7sADJfTRZHyaGxLzizKbv5/g4l129sBHHX33knxfwBwxN0fM7NHACxx97+d4rG++8kytmPHjiR233330bbsNWfL46N7wzanlwnuXvM+A40c252dnX7zzTefF4uqEVgVCqsKAXj1A6uoAIDu7u4kFlVCXXHFFUns2muvpW1Zf0+cOEFaAj096T+02Unx0Ynw+/fvT2J9fX20LavqiJbds3vBfh4Ali1blsSKVJRFr009+vr6MDY2loztWv7dtwnATwC8b2bvVGKPAngMwO/N7KcADgDY3KjOijSJxrZkbcoJ3N13AYje1fygsd0RaR6NbcmdVmKKiGRKE7iISKZU+9QgLOFY1qn0MvuYWXIPo8QXGxfRaewsYRmdkM4Sg+vXr6dt2fJ0thQf4EvOo2Xox48fp/Fa27E+RAnPgYGBJNbf30/bXn755TX3gSU3o73S2Qn20e8te83Z3vBAWqgQni1AoyIiMutpAhcRyZQmcBGRTGkCFxHJlCZwEZFMqQqlQbZs2ZLEoswxq0LYvDld7BdVMcjsc/bs2eTUcXYKOQB0dnYmsagChFUuRFUo7DHYAQsAX0ofLQFnS+Gj6glWTTNv3rwkxk6fB4C9e/cmMXZoAsCrU6KDF9hWBdGp9Kzy59SpU7QtU+QACjYWgPR3X1UoIiItRhO4iEimNIGLiGRKE7iISKaUxCzR/fffX3PbgwcPJrHe3l7SEnj44YeTmE6qn1lmliTrov2m2d7UUWKyyD7WLHk2ODhIWvIl4FFCjYlOsGeJULbHNzupHuCJyaGhoZqfK0ogsuc7fPgwbcsSsVFBQZGTmNg9jh63+tqi5fl6By4ikilN4CIimdIELiKSKU3gIiKZmnICN7NuM/uzmX1oZnvN7BeV+K/N7JCZvVP574fld1ekcTS2JXe1nErfBaDL3d82s4UA3gJwLyYOej3p7v9Y85O18Kn0MjsUPJW+YWP7oosu8jVr1pwXiyoUFi5cmMSipfRMdJgCW94eVaywioio7WWXXZbEWBVL1AdWaRFVixw5ciSJRQcvXHLJJUmMVe0AxQ5HYfctOnCDLfMvUrESbbfQ0dFx3vfvv/8+Tp48WfxUencfBDBY+XrUzD4CwGuIRDKisS25K/QZuJmtBXArgNcroYfM7D0ze9rMFgc/s9XM9pjZnrp6KlKiesc22/BJpGw1T+Bm1gngBQC97n4CwG8BfA/ALZh4F/M4+zl33+buPe7e04D+ijRcI8Z29RmGIs1Q0wRuZh2YGODPuvuLAODuQ+5+xt3PAvgdgI3ldVOkHBrbkrMpPwO3iY1onwLwkbv/ZlK8q/IZIgD8GMAH5XRRpByNHNvt7e1JcjJKTLKPW9ie2QDfh/rYsWO0LVsuHiXUhoeHk9jixfSTIrp3d7Q/NUvUVSfkgPikefbzLFkJ8ARitCUB24s7SqSya4v+hcXuT7RXOrsXUX+ri0uiYpNaFvJvAvATAO+b2TuV2KMAHjCzWwA4gH4AP6vhsURmE41tyVotVSi7ALC/bl9ufHdEmkdjW3KnlZgiIpnSBC4ikilN4CIimdKBDiIN4O5JdUmtJ44DcRUKEx1wwKpb2LJ9ALj44ouTWLQMnVVKsKoOgFdLsGqcqAKE/Xy0SIq9ZlHlD6vQie7P6OhoEoteG7bMf+nSpbQte76ooqj69Y1eA70DFxHJlCZwEZFMaQIXEcmUJnARkUxNuR94Q5/MbATAuXW5ywDwY6HzpuuaOWvcPd28ugkmje0cXqfpatVry+G66Nhu6gR+3hOb7WnFHQp1XRe2Vn6dWvXacr4ufYQiIpIpTeAiIpmayQl82ww+d5l0XRe2Vn6dWvXasr2uGfsMXERE6qOPUEREMqUJXEQkU02fwM3sbjP72Mz2m9kjzX7+RqqcWD5sZh9Mii0xsz+ZWV/l//ycqlnMzLrN7M9m9qGZ7TWzX1Ti2V9bmVplbGtc53NtTZ3AzawdwD8D+GsA12Hi6KrrmtmHBnsGwN1VsUcA7HT3dQB2Vr7PzTiAX7r7dQBuA/Dzyn1qhWsrRYuN7WegcZ2FZr8D3whgv7t/4u7fAHgewD1N7kPDuPurAKpPZ70HwPbK19sB3NvUTjWAuw+6+9uVr0cBfARgJVrg2krUMmNb4zqfa2v2BL4SwMFJ3w9UYq1kxaQTzb8AsGImO1MvM1sL4FYAr6PFrq3BWn1st9S9b5VxrSRmiXyiRjPbOk0z6wTwAoBedz8x+c9yvzaZvtzvfSuN62ZP4IcAdE/6flUl1kqGzKwLACr/H57h/kyLmXVgYpA/6+4vVsItcW0lafWx3RL3vtXGdbMn8DcBrDOzK81sLoAtAF5qch/K9hKABytfPwjgjzPYl2kxMwPwFICP3P03k/4o+2srUauP7ezvfSuO66avxDSzHwJ4EkA7gKfd/e+b2oEGMrPnANyJie0ohwD8CsC/Afg9gNWY2F50s7tXJ4RmNTP7PoD/BvA+gLOV8KOY+Lww62srU6uMbY3rfK5NS+lFRDKlJKaISKY0gYuIZEoTuIhIpjSBi4hkShO4iEimNIGLiGRKE7iISKb+D9ZHOYLiv/MNAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAC4CAYAAAD61bdSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAW9UlEQVR4nO3da4icVZoH8P+TTjTmYmKutrlqjErwkmgrAzNgllkXdxDifNERGSIMmxFGHEFBEWFkZWE+7OiusIxmUIyQzcxAnDEsumuQQXdkvWREEzXrGpPOzU46iUm6c23T/eyHrux26vwfu6r6reo6lf8PxO6nT7113vc9fVL9nuecY+4OERHJz5jRroCIiNRGHbiISKbUgYuIZEoduIhIptSBi4hkSh24iEimRtSBm9ntZva5mW0zs8eKqpTIaFPblhxYrXngZtYG4H8A3AZgD4APANzj7p99y2uUdC515e420mPU0rbHjRvn48ePPyc2Zkzln48GBgZonP1+Rsdlx6imbNQXtLW1JTEzfpnZcVnZ6L1Y2ei92DGi41ZzL9gxovtTzXHZeUTHLS976tQp9PX1JQcYW/G7p24BsM3dt5fe8LcAVgAIG7lIJqpu2+PHj0dHR8c5sQsvvJCWZb/Ix44do2XZL3h03L6+viR20UUX0bLs/c6cOUPLTp06NYmxTh0Y7GjKsU6umn8sLrjgAlqWne/p06dp2QkTJtA488033ySxkydP0rLs+o4dy7tVdt+j+pZfs02bNvFyNFqZOQB2D/l+Tyl2DjNbZWabzIzXQKT5VN222S+9SL3VfRDT3Ve7e4e7dwxfWiQfQ9v2uHHjRrs6ch4aySOUvQDmDfl+bikmkruq27aZJX8iR38es8cE0Z/dEydOTGLHjx+nZS+++OIkduLECVqW/ekfPc9lj2z6+/tp2UsvvTSJdXV1JbHoEQx71BE9XqrmUQc7h+g6tre3J7HokQ97jBP9NXbJJZckMfbICUjvW/gMnkYr8wGAxWZ2uZldAOBHADaM4HgizUJtW7JQ8ydwdz9jZg8A+A8AbQBedPdPC6uZyChR25ZcjOQRCtz9NQCvFVQXkaahti050ExMEZFMqQMXEcnUiB6hiMggd08yM1gGCcAzD1g2Q1R2ypQpYR3KRemNLLMjmiDEMk4OHjxY8XFZxks0aYi9PsrqKJ/5CgCTJk2iZdn7RZkwPT09SSzKAmHnFmXoHDhwIIlF17xS+gQuIpIpdeAiIplSBy4ikil14CIimdIgpkgBzCwZFOvu7qZl2cBitOIei/f29tKybCp9NdPj2aAgwAf7osE3NhWeTVlndQX4oO2MGTMqLhsNju7bty+JzZ49m5Zlg8+HDh2iZdkAa7TyITvn6Ljl1zdaUlefwEVEMqUOXEQkU+rARUQypQ5cRCRT6sBFRDKlLJQW8dRTT9H4o48+WlEMAJ555plC63Q+MbMkY2Ty5Mm0LNu8oZpNgqNp3SxjJSpbzQbI7LizZs2iZVl2yoIFC5JYtNFENZk07ByOHDlCy7LrG2WLsKnw0bII7HwPHz5My7Lso+j+lGe3hJs106iIiDQ9deAiIplSBy4ikil14CIimRrRIKaZdQLoBdAP4Iy7dxRRKaneokWLaJxNg77//vtp2eeffz6JRYNNra7att3f359McZ8+fToty6Z7Hz16lJZlA6HR4CgbfIvWpmbtIhqoY/WN1t1m62OzqfDRoNzMmTOTWDQwyY5bzfIF0a70bMCTrREO8OUHokHXaKkCpvz6RIPcRWSh/JW789XdRfKmti1NTY9QREQyNdIO3AG8YWZ/MbNVRVRIpEmobUvTG+kjlO+5+14zmwVgo5n9t7u/PbRAqfHrF0ByU1XbjpaDFamnEX0Cd/e9pf93A/gDgFtImdXu3qEBTslJtW072jxYpJ5q/gRuZhMBjHH33tLXfwPg7wurmVTlueeeo/G77747iUWdTTTSfb6ptW2XX79oSjWbPh1tkMA2LTh9+jQty7JTTp48ScsuXrw4iUW7v/f19SWxaJMFlnkzZcqUJBZllixcuDCJRZlQ+/fvT2KsrgC/5mxJA4D/frDsmqhuUR1YdsqxY8do2fJrVo8slNkA/lA68FgA/+ru/z6C44k0C7VtyULNHbi7bwdwQ4F1EWkKatuSC6URiohkSh24iEimtB54i7jtttsqLrtlyxYaj6YWy/Da2tqS6eXRYBYbLIwGltlU+GgaOpsKz9biBvjgWZQKyY5x8CCfoMrqe+mllyaxaBBz165dSeyqq66iZdl1iAY8Wbyzs5OWrWaQmR03uj9sQDpaFqH8dzFaN1yfwEVEMqUOXEQkU+rARUQypQ5cRCRT6sBFRDKlLJQWcfPNN492Fc575dkHUUYEm5bNNlgA+BTqajJW9uzZQ8uyLBA2jR0Atm7dmsQ6OvjSRiyT6auvvqro/QHg8ssvT2LRtZk1a1YS27lzJy3LdruPptKz40YbOrCMomi3+2qm80fnXE6fwEVEMqUOXEQkU+rARUQypQ5cRCRTGsTM0H333ZfEli9fXvHrP/vss+IqIwAGBzDLBzGrGcyK1g6vZvd4NnU/WjucDaRu27aNlp06dWoS+/LLL2lZtk44O99ly5bR18+dOzeJRWuas/ONlg7YsGFDEjt69Cgtu3fv3iQ2c+bMiusQ3fdqlkUovz/RTvf6BC4ikil14CIimVIHLiKSKXXgIiKZUgcuIpKpYbNQzOxFAHcA6Hb3a0uxaQB+B2AhgE4Ad7k7H0aXwq1cuTKJRdOrWRbCs88+W3idclRk23b3JCOBTckGgK+//jqJRZtplG8SAcQZCWx6epSxwtrFmTNnaFmWPRFt6MDqxo67ZMkS+nq20/y+ffto2fb29iS2fft2WpZdXza9PtLb20vjbCp9lPnDps1HU+bLr3mUrVLJJ/CXANxeFnsMwJvuvhjAm6XvRXLzEtS2JWPDduDu/jaA8o8MKwCsKX29BsCdBddLpO7UtiV3tU7kme3uXaWv9wGYHRU0s1UAVtX4PiKNVlPbjvZMFKmnEQ9i+uDDGf6AZvDnq929w935+pMiTaqath2NQYjUU62fwPebWbu7d5lZO4DuIislg6LBxltvvTWJRYMcL7/8chLr6uoiJaWkprbNdqWPBvrY9OtoMIsNALId5QG+63k0MMnWCY92pWdrdEdtiE2Fv+aaa5IY26ke4GuH33TTTbTs559/nsSiwcbx48dXXJbdt2jAkw3wsmsQHTeazj99+vRzvi96Kv0GAGdTIVYCeLXG44g0G7VtycawHbiZrQPwXwCuNrM9ZvYTAL8EcJuZfQHgr0vfi2RFbVtyN+wjFHe/J/jR9wuui0hDqW1L7jQTU0QkU+rARUQypQ0dmlg03ZhhU3oBYN26dUVVR76FuycbF0SbNEyZMiWJTZ48mZZl0+OjzBKWhcKyLwCecRJtWsBy3G+44QZalmVgzJs3L4nNns3T69lGD2xDCADYtWtXxWXZNPYok4YtdcDuWXRc9noAOHDgQBKLllsoz1QayVR6ERFpQurARUQypQ5cRCRT6sBFRDKlQcwmwQaQli5dWvHrN27cSONvvfVWzXWSyrl7MrjIdnMHgBMnTiSxaOd1NoU7Ghxla39Ha2mz94umt7P6Rmu/sN3uFy5cmMSiwVU2mPvBBx/QsmztbzZQCACbN29OYmwQFOADi1GSALsObGATAK677rokxgapGTOjcX0CFxHJlDpwEZFMqQMXEcmUOnARkUxpELNJrFmzJolNmzaNlmUDGq+88krhdZLKuXsyMBjdv2i2IMMGz6K1w3fu3JnEojWv2cBktCHwFVdckcTYZssAH6i75JJLklj5etdnsfXAoxmTbBCyu5sv386uY1QHtkZ3tDn0ZZddlsSiAWm2YXO0k1O0/ndSrqJSIiLSdNSBi4hkSh24iEim1IGLiGRKHbiISKaGzUIxsxcB3AGg292vLcWeBPB3AM7OW33c3V+rVyVbDZtufPXVVyexaA3g48ePJ7HXX3995BU7zxTZts0smVbd09MTli0XTb9mGStRpgVbjiGaAs7KRtktu3fvTmJXXnklLcuyXlg7ZmuXA3zn9mjKO1tmIFp+gmW3REsSVHN/2HGjtc5Zxkl03Oi+lavkE/hLAG4n8WfcfWnpP3XekqOXoLYtGRu2A3f3twHwLSZEMqa2LbkbyTPwB8xss5m9aGZppn6Jma0ys01mtmkE7yXSSFW37fItsEQaodYO/NcAFgFYCqALwK+igu6+2t073L2jxvcSaaSa2jbbY1Kk3mqaSu/u/zcn1Mx+A+DfCqvReeCJJ55IYmzN5MiTTz6ZxKJ1n6U6RbbtaM1stilxtKkxm9YdbYTLpmpH07rZVG02LRzgg+7Hjh2jZdnyAWzAcseOHfT1bAAxGlxlZd944w1altWXTfGPjhtNbWf/cEfJB+y+RxtUlw9iRssv1PQJ3Mzah3z7QwCf1HIckWajti05qSSNcB2A5QBmmNkeAL8AsNzMlgJwAJ0AflrHOorUhdq25G7YDtzd7yHhF+pQF5GGUtuW3GkmpohIptSBi4hkShs6jIIHH3xwRK9fv359QTWReoo2U2AZJ9Hu5Gw6fjQNvb29PYlFmTDMhAkTaHzOnDlJLNrBntWXbRQxdepU+vp33nmn4rJsA4toajo7xvjx42lZltMfZeiw6xvdHyZaFqF8mQDtSi8i0mLUgYuIZEoduIhIptSBi4hkSoOYdRStTVzpuhmPPPIIjXd2dtZaJakTth54tKZzNQtfsYG2aPCNrRMfrU3NBhY7OvhyRYsWLUpiCxYsoGXZtHsWi5Z+YOuMb9rE18Fjg7ZsfW6AXzN2DQBg/vz5SSy6Z/39/UksGryeMmVKRe8FpFPno6n8+gQuIpIpdeAiIplSBy4ikil14CIimVIHLiKSKWWh1NGKFStoPJruW+6jjz4qsjpSR2aWZFucPn2almX3P9q0gE0BZ5kPADB9+vQkFm1awKaGsyyJKB5N0X/33XeT2Ny5c5MY21Ee4Bs9RLvHs6ybaMr7F198kcSirJtdu3YlsSibh02bjzZpYGWje1m+EUdUTp/ARUQypQ5cRCRT6sBFRDKlDlxEJFOV7Ik5D8DLAGZjcJ/A1e7+z2Y2DcDvACzE4N6Bd7k7H204D1x44YVJ7I477qj49QcPHkxiW7ZsGVGd5NsV2bYHBgaSnc/DgScyLToaFDx06FDFZdngaLQ2NVv7m61TDgAzZ85MYmygDwCuueaaJMbWso6mprNlJqLruHnz5iTGBnIBPmhaPlB41lVXXZXEomvOBlKjgWN2ztFxy895JOuBnwHwsLsvAfAdAD8zsyUAHgPwprsvBvBm6XuRnKhtS9aG7cDdvcvdPyx93QtgK4A5AFYAWFMqtgbAnfWqpEg9qG1L7qrKAzezhQCWAXgPwGx37yr9aB8G/wxlr1kFYFXtVRSpv5G27WiFQJF6qngQ08wmAVgP4CF3P2fjO3d3DD5DTLj7anfvcHeeNS8yyopo29XsPSlSlIo6cDMbh8EGvtbdXymF95tZe+nn7QD47pwiTUxtW3JWSRaKAXgBwFZ3f3rIjzYAWAngl6X/v1qXGmZi0qRJSezGG2+s+PVsRJxlpkhximzbAwMDyT1kGxlEounX7NFMlC3CMjhYuwR4FsrgHxspljVTnnFzFsvAOHHiRBLbv38/fX01mRrLli1LYtHyE+yaRRurHD16NImx6wXwDJmovqw9RNk40b0oV8kz8O8C+DGALWZ29uo8jsHG/Xsz+wmAnQDuqugdRZqH2rZkbdgO3N3/DIAnIQLfL7Y6Io2jti2500xMEZFMqQMXEcmU1gMXKYCZJYN90TR2JhpQY9Pjox3Ke3p6kti0adMqfj+29njk+uuvp3G2KzwbFIzeq7s7Tfg5cOAALbt3794kNnHiRFqWrc0e7vRexaAtWyudnS/ABzejAc/yAfFwgJlGRUSk6akDFxHJlDpwEZFMqQMXEcmUOnARkUwpC6UgLANg7dq1tOy9996bxL788svC6ySNM2bMmGTaepS5EE3LZtg09ChjpZoVEVmmRDU7ukdZJOzc2DIRu3fvpq//+OOPk1hbWxsty6amDwwM0LIsi+Obb76hZZkou6Wzs7PismzTF3Z/WTw6L30CFxHJlDpwEZFMqQMXEcmUOnARkUxZpevOFvJmZo17MzkvuXu0umBdTZw40a+99tpzYmwndIAPnkW7jrOByeh3lh0jmqrNBkKjOsyYMSOJRQOA0fT0cr29vTTOBixZggDAp8dH9WIDqWwaPMAHDKMd7KPBRYZdG7ZUApBeh/fffx89PT3JDdIncBGRTKkDFxHJlDpwEZFMqQMXEcnUsB24mc0zsz+Z2Wdm9qmZ/bwUf9LM9prZR6X/flD/6ooUR21bclfJVPozAB529w/NbDKAv5jZxtLPnnH3f6xf9UTqqrC2PXbs2GRH9mineZbtwaZZAzz7gWVfAHxqeVQ22q2e2bFjRxKLMlZYnGVaRNeGZaFEGSvs2rBrEInqwOKHDx+mZcvv+bdhWSjRMgHl5xZlu1SyqXEXgK7S171mthXAnOFeJ9Ls1LYld1U9AzezhQCWAXivFHrAzDab2YtmRv8pMrNVZrbJzDaNqKYidTTStt3X19egmor8v4o7cDObBGA9gIfcvQfArwEsArAUg59ifsVe5+6r3b3D3TsKqK9I4Ypo29EKgSL1VFEHbmbjMNjA17r7KwDg7vvdvd/dBwD8BsAt9aumSH2obUvOhn0GboOjEi8A2OruTw+Jt5eeIQLADwF8Up8qitRHkW17YGAgGXiK1udm072jnczZutvRAGQ1u6mzwbNTp07Rsuz9qpnezv46ia4NO+6RI0do2enTpyex/v5+WpYNEkd1YGt0z5o1i5ZlSxVEA7ysbLT0QPn9iQY7K8lC+S6AHwPYYmYflWKPA7jHzJYCcACdAH5awbFEmonatmStkiyUPwNg/6S8Vnx1RBpHbVtyp5mYIiKZUgcuIpIpdeAiIpnSrvQiBTCzJFMgygBhmRbR5gLsGFEWCstuiSYYsUyLgwcP0rJsY4poowg25ZtNTY82Xqhmp3lmwoQJNM6ueXRcVrcoW4Rl3US70rMlCebPn0/LRlknSb0qKiUiIk1HHbiISKbUgYuIZEoduIhIphq9K/0BADtL384AwEdN8qbzGj0L3H3maLzxkLadw3WqVaueWw7nRdt2Qzvwc97YbFMrrlCo8zq/tfJ1atVzy/m89AhFRCRT6sBFRDI1mh346lF873rSeZ3fWvk6teq5ZXteo/YMXERERkaPUEREMqUOXEQkUw3vwM3sdjP73My2mdljjX7/IpV2LO82s0+GxKaZ2UYz+6L0f7qjeTMzs3lm9icz+8zMPjWzn5fi2Z9bPbVK21a7zufcGtqBm1kbgH8B8LcAlmBw66oljaxDwV4CcHtZ7DEAb7r7YgBvlr7PzRkAD7v7EgDfAfCz0n1qhXOrixZr2y9B7ToLjf4EfguAbe6+3d37APwWwIoG16Ew7v42gK/LwisArCl9vQbAnQ2tVAHcvcvdPyx93QtgK4A5aIFzq6OWadtq1/mcW6M78DkAdg/5fk8p1kpmD9nRfB+A2aNZmZEys4UAlgF4Dy12bgVr9bbdUve+Vdq1BjHryAdzNLPN0zSzSQDWA3jI3c9ZET/3c5Pa5X7vW6ldN7oD3wtg3pDv55ZirWS/mbUDQOn/3aNcn5qY2TgMNvK17v5KKdwS51Ynrd62W+Let1q7bnQH/gGAxWZ2uZldAOBHADY0uA71tgHAytLXKwG8Oop1qYmZGYAXAGx196eH/Cj7c6ujVm/b2d/7VmzXDZ+JaWY/APBPANoAvOju/9DQChTIzNYBWI7B5Sj3A/gFgD8C+D2A+RhcXvQudy8fEGpqZvY9AP8JYAuAsxsHPo7B54VZn1s9tUrbVrvO59w0lV5EJFMaxBQRyZQ6cBGRTKkDFxHJlDpwEZFMqQMXEcmUOnARkUypAxcRydT/AoHsyqgdcYrmAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "out_img = torch.squeeze(output.cpu().data)\n",
    "print(out_img.size())\n",
    "\n",
    "for i in range(3):\n",
    "    plt.subplot(1,2,1)\n",
    "    plt.imshow(torch.squeeze(image[i]).numpy(),cmap='gray')\n",
    "    plt.subplot(1,2,2)\n",
    "    plt.imshow(out_img[i].numpy(),cmap='gray')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5) Check with Test Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "XCKx34u5MAic"
   },
   "outputs": [],
   "source": [
    "with torch.no_grad():\n",
    "  for i in range(1):\n",
    "      for j,[image,label] in enumerate(test_loader):\n",
    "          x = image.to(device)\n",
    "\n",
    "          optimizer.zero_grad()\n",
    "          output = model.forward(x)\n",
    "\n",
    "      if j % 1000 == 0:\n",
    "          print(loss)        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 5074
    },
    "colab_type": "code",
    "executionInfo": {
     "elapsed": 4373,
     "status": "ok",
     "timestamp": 1559565742608,
     "user": {
      "displayName": "Choi Gunho",
      "photoUrl": "",
      "userId": "04388737836176863066"
     },
     "user_tz": -540
    },
    "id": "rvww5H9GMAie",
    "outputId": "a5177d7e-d9e8-4b2b-a2c9-b26e974f14ef",
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([256, 28, 28])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAC4CAYAAAD61bdSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAZbElEQVR4nO3da4yV1bkH8P8DDNIyXAQKDHdQA7FYxI7WxCbFnlPltCTaRK18sKYXsUlNL2lMiGnQNLGt6ann+OG0DQYKNp7ebD01xp5jS5qi0SojpVwEuQk63IabcscZ5zkfZtPM7PV/nD2z956Ztfn/EuPMM2vv9333XrNm8671rMfcHSIikp9B/X0CIiLSOxrARUQypQFcRCRTGsBFRDKlAVxEJFMawEVEMlXWAG5mC83sDTPbaWZLK3VSIv1NfVtyYL1dB25mgwFsB/AZAM0A1gFY7O6vf8BjtOhcqsrdrdzn6E3frqur82HDhpX6/CWfy+DBg5NYe3s7bfvee++V/LzsXNva2mhbdr7RuFHq+UbXwI71/vvvl3ys6Lx6cg3Ven96cqzi5zh37hxaW1uTxkNKPlLqOgA73X134UR+BeAWAGEnF8lEj/v2sGHD8PGPf7ykJx86dGgSiwaTESNGJLFz587Rtm+//XYSiwblOXPmJLEjR47QtnV1dUmstbWVth0zZkwSO3v2bBI7efIkffyQIemQdOrUKdp21KhRSez8+fO0LXvNoz8MbFBlAzUA1NfXl3wO7HjRH/0zZ850+f61116j7cq5hTIZQOce01yIdWFmS8ysycyayjiWSF/qcd+OBjSRaqr6JKa7L3f3RndvrPaxRPpS577NPqWKVFs5t1D2AZja6fsphZhI7nrct909uV3xoQ99iLZltzWGDx9O2+7atSuJffjDH6ZtZ86cmcQOHz5M2x48eDCJsdsMAL8FEt2a2b59exKbPn06bcuwP4TR9bK2PbmvPXr0aNqW3d6J3st33303iUW3uC699NIkFv3hL74NE11XOZ/A1wG4wsxmmtlQAHcCeKaM5xMZKNS3JQu9/gTu7m1mdh+A/wMwGMBKd99SsTMT6Sfq25KLcm6hwN2fA/Bchc5FZMBQ35YcKBNTRCRTGsBFRDJV1i0UEekwaNCgZBUHS0oBgBMnTtDHM1OnTk1iR48epW3ZWnSWCATw1RNRBiF7jijrk61kYYkt0aobtrrlkksuoW1ZYkyUnMNWskSvI1udEj0ve81YghEAHD9+PImxFT5AumIlSiTSJ3ARkUxpABcRyZQGcBGRTGkAFxHJlCYxRSrAzJLJtmirUDYBGO3Oxya5ognAlpaWJBalobN4NFl46NChJMZ2MwT45CibtO1Jujnb4RDgr2OU8s5S/KOdAN95550k1tDQQNuyCU/2PgB8kjm6tuJJ7agv6RO4iEimNICLiGRKA7iISKY0gIuIZEoDuIhIprQKRaQC2trakpqSkyZNom0nTpyYxKK0e1anMqq5yEQp+mxVA1t9AfCVIWxlSnS8sWPHJrHTp0/Tx7MCB+zx0bFYXVAAaGxMC4JF58BS3qNVMywerYRh6fhRKb7ighvR1gX6BC4ikikN4CIimdIALiKSKQ3gIiKZKmsS08z2ADgJ4H0Abe6ezhRI1kaOHEnjX/7yl5PYo48+Stt+6UtfSmKrV68u78SqrKd9u66uDhMmTOgSO3bsGG3L9ryOKsL3BJuYjNK6x40bV9LjAeCtt95KYlE1dZYazvpQlOLPUtPZnugA32aATRADfMIymmxkE4bRNgOsWnz0vAcOHEhiUSp98esTTUZXYhXKje6eTpWL5E99WwY03UIREclUuQO4A3jezF4zsyWVOCGRAUJ9Wwa8cm+hfNLd95nZeAB/MrNt7r62c4NC59cvgOSmR307uu8pUk1lfQJ3932F/7cAeBrAdaTNcndv1ASn5KSnfbsSk5AiPdXrT+BmNhzAIHc/Wfj6JgDfq9iZSbjh/J133pnEfvGLX9C2UTXtUk2bNo3Gly1blsTYjDwAvPHGG2WdQ1/rTd9ubW3F4cOHu8Si176+vj6JRas62HNEadUMW9UB8BT98ePH07Y33nhjEotWt7DzZRXVb7rpJvr4nqwWYat8osIYbPuBqO3HPvaxJMZWkAB8NU20JcGUKVOSGFuRBKS/+1FV+nJuoUwA8HRh6dEQAP/t7v9bxvOJDBTq25KFXg/g7r4bwLwKnovIgKC+LbnQMkIRkUxpABcRyZT2Ax/AVqxYQeOLFy9OYmxSCgCeffbZss4hevyoUaOS2G9/+1vadv369WWdQw4GDRqUpMNHk7oMq9wO8BT7nlSlZynzANDe3l5SDAA2bdqUxD71qU/RtiNGjEhizc3NSewjH/kIfTzrx7NmzaJtWSo9iwH8dYz2a3/99deT2NatW2lbdr3Fk9kXFO/xDSDZfuGC4m0NovdGn8BFRDKlAVxEJFMawEVEMqUBXEQkUxrARUQypVUoAwRLY549ezZty9J6N2/eXNaxAOB730uzxaNU+i1btiSx7373u7RtT1K/c2VmSTp8VHGcFU6IChywVRVRajl7X7dt20bbstUpQ4bw4YCle0fbPLBVFddee20S+/vf/04fz4o/RK/j5MmTk9j27dtpW6apqYnGWX+NVsKwFTaROXPmJLEo7b64L0XFNvQJXEQkUxrARUQypQFcRCRTGsBFRDKlScwB4mc/+1kSmz9/Pm371a9+NYnt2bOn5GOxyRQAuOeee0p+jttuuy2J7dy5s+TH1xp3L3nvdZYKz9KsAb5P+LvvvkvbsgrnUco6mxiMqr+zCdboHNiEJauoHvXt/fv3J7FXX32VtmVp7Ndccw1te+rUqSQWbTOwbt26JBb1bTahHO3x3ZP3vfjcov3A9QlcRCRTGsBFRDKlAVxEJFMawEVEMqUBXEQkU92uQjGzlQAWAWhx97mF2BgAvwYwA8AeAHe4+/HqnWZtWbJkSRK79dZbk9ihQ4fo43/+85+XfKybb745iT399NO0LSsIEK1Mya3SPFPJvs1S6aMtC1g19CiN/dy5c0ksWuVw5syZJMZS0wG+KoM9HgAuv/zyJMYqrAPAjh07ktill16axI4ePUofz1Z7jB07lrZl57B7927atr6+PolFafBshU20CoRtaxEV3GBt2Uoadg7RCqdSPoGvArCwKLYUwBp3vwLAmsL3IrlZBfVtyVi3A7i7rwVwrCh8C4DVha9XA0g/PooMcOrbkrveJvJMcPcL/x44CIAXdgNgZksApPcMRAamXvXtaHc+kWoqexLTOyq3htVb3X25uze6e2O5xxLpSz3p2yxjUqTaevsJ/JCZNbj7ATNrAJDOfgkaGhpo/Ec/+lESYxNTd911V8nHivaT/sY3vpHELrnkEtr2qaeeSmLlVrXPUK/6dltbW1KN/Pz587QtSy2P3r/Tp08nsSg9nh0vqnbP9prvyR+ht99+m8avuuqqJHbsWPFdqngfbFbRPZrgZW3ZHuEA//2Ktp/YuHFjEov+hcUmNzv+7qfYxHFUlb74fY+es7efwJ8BcHfh67sB/KGXzyMy0KhvSza6HcDN7JcAXgYw28yazewrAH4I4DNmtgPAvxa+F8mK+rbkrttbKO6+OPjRv1T4XET6lPq25E6ZmCIimdIALiKSKRV0qBCW7vv973+ftmVpva+88koS27RpU8nHf+SRR2h84cLiREPgpZdeom0ffPDBko8nXdXV1WHixIldYtHKAZZKH1WaZ6sfohUgbMVKlKrNUtmjVU+sSnuUos/S8VlqerQKha3c+sQnPkHbspU7bOsBANi7d28Si1aWTJo0KYlF1e7Z732U9s5+748f57s0FK9uqfQqFBER6WcawEVEMqUBXEQkUxrARUQypUnMCrnvvvuS2Be/+EXalk1irVixIolF+4HffvvtSezee+8t+VirVq2ibVnKs5SmtbU1eb+KJzUvYBNtbM9sgKeLz5gxo+Tziib15syZk8SiSvMLFixIYuvXr6dtR48eXdJ5RRXhWVX5aCKW9Vf2egF8wpP9bgB8YjHavoBtSxFNYrLn2Lx5M21bvCWAmdF2+gQuIpIpDeAiIpnSAC4ikikN4CIimdIkZg8tWrSIxlkWY5Q9tWvXriTGMtteeOEF+vgbbrih5GP98Y9/TGJPPvkkbSu9N3jw4GQCr729nbZlE31RQWFWIDcqanzw4MEkFu0zzgoCz5s3j7ZlxXijyUo2ccuuISqKzPbXjvav70lm4759+5IY25cd4L9L06ZNo21ZVmz02rBJ4qjwdfG5aRJTRKTGaAAXEcmUBnARkUxpABcRyZQGcBGRTHW7CsXMVgJYBKDF3ecWYg8BuAfAhbzVB9z9uWqdZH9hKcsPP/wwbRvNEjPz588vKRbpybG+8IUvJLEohXjJkiUlP28tqHbfbm1tLTk+dOhQ2palkUd7ac+cOTOJRasnWOo+W20C8JUhn/70p2lbdm2s8nqUts/SzV988UXalq0iYStTAL6lwI4dO2hb9vvFVrEAwJAh6RDKqs8D/HWMVgkVr6aJ0vNL+QS+CkBaFQD4D3e/uvBfzQ3eclFYBfVtyVi3A7i7rwWgXY6k5qhvS+7KuQd+n5ltNLOVZsa3UgNgZkvMrMnMmso4lkhf6nHfjm6XiFRTbwfwnwK4DMDVAA4A+HHU0N2Xu3ujuzf28lgifalXfbuurq6vzk/kn3qVSu/u/9z42MweB/Bsxc6oH0Sput/+9reT2Ny5c2lbln4bpbeXi33ae+ihh2jbZcuWJbFoj2gpr28XT35F/YoVCWYTXAB/r9nEGcALFUd7krMJyyitm02ORmnoLI2cFQ/eunUrffyePXuSWFTwme2X/+abb9K2rOBztMc3e32jvfLZJOTu3btp22j7AGbMmDFdvo/6R68+gZtZ59LRnwfAdyUXyYz6tuSklGWEvwSwAMA4M2sG8CCABWZ2NQAHsAcALwcjMoCpb0vuuh3A3X0xCaf1v0Qyo74tuVMmpohIpjSAi4hk6qIr6MBWBjzxxBO07W233VbWsZ57jifxPf744yWdQ1SN+6mnnkpiP/jBD2jbNWvWJLENGzbQttJ77e3tyRYF0SoUlhYdrepg8eKK5RewQg9R2j1bLRJt0XD27NkkFlVTZ0UhmpubkxgrYAIAGzduTGJHjhyhbYtXagDxipVx48YlsZdeeom2ZdsMROfAtiqYOnUqbcv6Q/T+jB07tsv3Yf+gURERGfA0gIuIZEoDuIhIpjSAi4hk6qKbxPzrX/+axK699tqSHx9NOnzuc59LYuvWraNtb7/99iTGKndHx4omLJlXX3215LbSe3V1dWhoaOgSi/aFZunXLS0ttC2bsIzS0C+//PIkFu39ziYAWWo6wNPIZ82aRdvu2rUribEU/Zdffpk+nk1uRtdQPNEHAPv376dtWfX4w4cPk5ZAU1O6715jI9/KiaXoR4sP2tvbk1i03UbxdgtRO30CFxHJlAZwEZFMaQAXEcmUBnARkUxpABcRydRFtwpl5cqVSSxahfKTn/wkiT322GO07c6dO5MY28geAO6///4kxlJlH3nkEfr4KI1Z+k9ra2uyAiJK62YrLaJq6qwa+kc/+lHatriSORCndbPnjVZPsArye/fupW3ZNbPfjeh3jq1iGTp0KG37j3/8I4lFBSzWrl2bxKKtA4pXEwHx7zLbviBazcNS6c+cOUPbFq9YKacqvYiIDEAawEVEMqUBXEQkUxrARUQyZd1VTjezqQCeADABHXUCl7v7Y2Y2BsCvAcxAR+3AO9w9nUXp+lzVKdM+ALA05iiVftSoUUmMTbIsWLCg7PO62Lg7n5kiKtm3R44c6cXp1qz6PMAnxKJ0cdZXzp8/H51DEpsxYwZty9LIJ02aRNuytPmoLUu7Z9sBsAlXgE+CbtmyhbZlk4XRBOLf/va3JHbZZZfRtmyrgmhvd1bZnu0RDvCJ32jyurgKfVNTE06cOJH07VI+gbcB+I67XwngegBfN7MrASwFsMbdrwCwpvC9SE7UtyVr3Q7g7n7A3dcXvj4JYCuAyQBuAbC60Gw1gFurdZIi1aC+Lbnr0TpwM5sBYD6AVwBMcPcDhR8dRMc/Q9ljlgBY0vtTFKm+cvt29E9skWoqeRLTzOoB/A7At9z9ROefeceNdHp/292Xu3uju/P9GEX6WSX6dpRsIlJNJQ3gZlaHjg7+pLv/vhA+ZGYNhZ83AOAbGosMYOrbkrNub6FYR77pCgBb3f3RTj96BsDdAH5Y+P8fqnKGA0yUUrto0aIkFlXe3rRpUxJbvHhxeScmPVbJvt3a2poUZWDVzQG+UoIVWIhEK0BY0QK2SgLgq1MOHjxI27KK7n/+859p24ULFyaxt956K4kdPXqUPp4Vu2ArWwC+HUDUlq3miQo6zJ07N4mx7QQA/vpGz8u2y4j+5Xb27Nku37NiEEBp98BvAHAXgE1mtqEQewAdnfs3ZvYVAHsB3FHCc4kMJOrbkrVuB3B3fxFAtLb2Xyp7OiJ9R31bcqdMTBGRTGkAFxHJ1EW3H3i5li7lSXnLli1LYtHEx80335zEogkkyUNdXV2yF3WU1s32rC6etLqArS+Ptr+46qqrSm5bV1eXxNh2EABw6tSpJHb99dfTtmxLAPa80Wtz4sSJJBZtScEmbaM9zc+dO5fExo8fT9uyvbfZJCjAJxfZZCXAJ2jZfuLROTD6BC4ikikN4CIimdIALiKSKQ3gIiKZ0gAuIpIprUL5ACy9PVqFsmfPniT2ta99jbbVipPaY2YYMqTrr1OUxt7c3JzEpkyZQtuyVQoshRzgadlROv/o0aOTWFQoovi6AJ4eHz0vK1YSFbBgBR3Y8QHgmmuuSWIHDhwgLYEJE9INJU+fPk3bskrxUco7W80T7UzZ2tqaxIoLN1xQ/L5F7fQJXEQkUxrARUQypQFcRCRTGsBFRDKlScwPMHz48CTGJi0A4P77709izz//fMXPSQau4rT1KKWaTahFqdOssn30vGzrhh07dtC2LLW8oaGBtmWp9OzxAJ+g3b9/fxKLtg5gE4gzZ86kbd95550kFqXSswnEaLKRxaP9uHsy6cpeR/b+AumWAlH/0CdwEZFMaQAXEcmUBnARkUxpABcRyVS3A7iZTTWzv5jZ62a2xcy+WYg/ZGb7zGxD4b/PVv90RSpHfVtyZ9GG7/9sYNYAoMHd15vZCACvAbgVHYVeT7n7v5d8MLMPPphImdw9qnGZqGTfrq+v93nz5nWJRZXm2eqJKO2eFS2IVmWwlR1RujhbCcPS4AGgqakpiQ0bNoy2ZSnfbPXF8ePH6eNnz55d8rHefPPNJDZ58mTa9siRI0ksWoXCVoZE7w8TXRt7fY8dO0bbFq802rBhA06ePJn07VKKGh8AcKDw9Ukz2wqAv0oiGVHfltz16B64mc0AMB/AK4XQfWa20cxWmhndNcfMlphZk5mlf8ZFBohy+zZbZyxSbSUP4GZWD+B3AL7l7icA/BTAZQCuRsenmB+zx7n7cndvdPfGCpyvSMVVom9HCV4i1VTSAG5mdejo4E+6++8BwN0Pufv77t4O4HEA11XvNEWqQ31bctbtPXAzMwArAGx190c7xRsK9xAB4PMANlfnFEWqo5J9m+0HHk1QjRw5MolFk41sUpBNyAG8cnqUss5S3tmEKQBMnz49ibHq8QDQ0tJS0nlF/2JhE4ATJ06kbdmk4NGjR2lbNmEZpbGzVPho+wI2QRtd27Zt25LY1KlTadviPhLtB17KXig3ALgLwCYz21CIPQBgsZldDcAB7AFwbwnPJTKQqG9L1kpZhfIiALY067nKn45I31HfltwpE1NEJFMawEVEMqUBXEQkUyroIFIl0SoHVpE9qnpeX1+fxNgKEoCnnEfFBdhKiaiiOyts0tbWRtuy82WPj1Z1sPONKtgzUduOBUddzZo1i7Zlq4ei95IlcEWrUNhrE60SKn4vzp8/T9vpE7iISKY0gIuIZEoDuIhIpjSAi4hkqtv9wCt6MLPDAPYWvh0HgOcE503X1X+mu3vpGzdXUKe+ncPr1Fu1em05XBft2306gHc5sFlTLe5QqOu6uNXy61Sr15bzdekWiohIpjSAi4hkqj8H8OX9eOxq0nVd3Gr5darVa8v2uvrtHriIiJRHt1BERDKlAVxEJFN9PoCb2UIze8PMdprZ0r4+fiUVKpa3mNnmTrExZvYnM9tR+D+taD6QmdlUM/uLmb1uZlvM7JuFePbXVk210rfVr/O5tj4dwM1sMID/AvBvAK5ER+mqK/vyHCpsFYCFRbGlANa4+xUA1hS+z00bgO+4+5UArgfw9cL7VAvXVhU11rdXQf06C339Cfw6ADvdfbe7vwfgVwBu6eNzqBh3XwugeO/JWwCsLny9GsCtfXpSFeDuB9x9feHrkwC2ApiMGri2KqqZvq1+nc+19fUAPhlA59LXzYVYLZnQqaL5QQAT+vNkymVmMwDMB/AKauzaKqzW+3ZNvfe10q81iVlF3rFGM9t1mmZWD+B3AL7l7ic6/yz3a5Pey/29r6V+3dcD+D4AUzt9P6UQqyWHzKwBAAr/b+nn8+kVM6tDRyd/0t1/XwjXxLVVSa337Zp472utX/f1AL4OwBVmNtPMhgK4E8AzfXwO1fYMgLsLX98N4A/9eC69Yh31p1YA2Oruj3b6UfbXVkW13rezf+9rsV/3eSammX0WwH8CGAxgpbs/3KcnUEFm9ksAC9CxHeUhAA8C+B8AvwEwDR3bi97h7mmRvQHMzD4J4AUAmwC0F8IPoON+YdbXVk210rfVr/O5NqXSi4hkSpOYIiKZ0gAuIpIpDeAiIpnSAC4ikikN4CIimdIALiKSKQ3gIiKZ+n8UHlbwK//39QAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAC4CAYAAAD61bdSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAaF0lEQVR4nO3de4zU1dkH8O8DLBe5X5cFFleuxlBdCiKtkmhrFYwNtjW2pjE0MS+9X5K20ZBWW5s3tcnb+vqHbYJBwaRor7yYN/oqElu0pYQFK3fk4loWl11EhEUuy+4+7x87NLtzvkdmZ2eWPcP3kxjZh2dmfjNzOAzznOccc3eIiEh6+lzqCxARkfxoAhcRSZQmcBGRRGkCFxFJlCZwEZFEaQIXEUlUtyZwM1toZnvNbL+ZPVioixK51DS2JQWW7zpwM+sL4C0AnwFQB2AzgHvdfddH3EaLzqWo3N26ex/5jO1+/fp5//79O8X69u1Lc1tbW4NY9m0vaGtrC2J9+vDPXefPnw9iZWVlNJf9uT937hzNZY9nxl/mWDyXx4/dvivPl71eANCvX78gFnt/2H0Uol+GPbfY/WZf29mzZ9Hc3BzcQfiscjcPwH53P5i5uOcALAYQHeQiiejy2O7fvz+mT5/eKTZs2DCa29TUFMSuvPJKmnv69OkgNmDAAJp79OjRIDZu3Diay/4SOXDgAM294oorghibEGNxNlHG/rJgf5F15fmy1wsARowYEcRGjx5Nc9l9nD17luYysUmZPbeWlhaaO3To0E4/19TU0LzufIUyEcChDj/XZWKdmNlSM6sxM34FIr1Pl8d27A+iSDEVvYjp7svdfa67zy32Y4n0pI5jO/aJVKSYujPqDgOo7PDzpExMJHVdHttlZWWoqKjoFIt9TTB27Ngg9u6779Jc9hfDyZMnaS57vLq6Opo7ePDgIDZ+/Hiay74CGT58OM1lXzWwr0Bqa2tzvq7m5uacH2vIkCE0d+DAgUFs3759NHfGjBlBjL1nAPDGG28EsSlTptBc9tVK7Cuf7K/Z2FdeQPc+gW8GMN3MrjKz/gC+BOD5btyfSG+hsS1JyPsTuLu3mNm3ALwEoC+Ap9x9Z8GuTOQS0diWVHTrizt3fwHACwW6FpFeQ2NbUqBOTBGRRGkCFxFJlNY+iRSAuwerJWLNH6y55tSpUzSX3UdszTnrIBw1ahTNZatepk2bRnMbGxuD2JYtW2hudXV1EGMrTiorK4MYwLsuWeMTwBtxqqqqaC7rgow1CLH3IrbyhzVgffjhhzSXrTiJvZdsNQ6jT+AiIonSBC4ikihN4CIiidIELiKSKBUx5d/YrnOzZ8+muQsWLAhisaLd5s2bg1islfrw4TR3Y3D3YHvTWIGKFepiLe+ssMi2UQV4232sBXvChAlBLFY4mzx5chArLy+nuez9Y/cba3lnbfuxfWbYDoOxgmdDQ0MQy9764AK2i+S2bdtoLit4xtruWdH12LFjNHfQoEGdfo5tqatP4CIiidIELiKSKE3gIiKJ0gQuIpIoTeAiIonK+1DjvB5Mhxr3CrEW4h/96EdBbNmyZTS3Kwe0Muw8QwDYtGlTEFu8eHHO91uIQ43zMXjwYJ81a1anWOxMzDNnzrDb01y2KqMrLeCx1R5sxVCsXZy148da/9k2AWxcxM4AZatIpk6dSnOzV2oA8efAVuPs2sWPOGVnV3blEI3jx4/TXLYqKXYIc3bbfW1tLc6cOROMbX0CFxFJlCZwEZFEaQIXEUmUJnARkUR1q5XezGoBNAFoBdDi7nMLcVFSOIsWLQpiDz30EM2dN29ezve7du3aIBYrYq5fvz6IHTp0KOfHuhS6OrbZfuCsdRrge2nHTnlnp6nHsBb7WBFz587wiM9YezzbD5xtuxC7hhtuuCGIsUIuAMycOTOIxdrIJ02aFMRYYRMAXnzxxSB2/fXX09wdO3YEsdj2Bez9GTp0KM1l2xfE9lXPLoDHXoNC7IVyi7u/V4D7EeltNLalV9NXKCIiieruBO4AXjazLWa2tBAXJNJLaGxLr9fdr1BucvfDZjYOwDoz2+PuGzomZAa//gBIaro0tsvKyi7FNcplrlufwN39cOb/jQDWAAiqYO6+3N3nqsApKenq2I7tWS1STHmPOjMbDKCPuzdlfn0bgEcKdmXSJddddx2NP/PMM0EsdlJ5TU1NEFuyZAnN3bNnTxeuLi35jG0zCw4e+OCDD2gua48/ceIEzWX3Efu0z9rjWas3wFe9xFYRjRw5MqfbA8CUKVNyuobbbruN3p616I8ZM4bmsjb0f/zjHzSXHYwR286hsrIyiMVWgbAVOrEVNuzwBraSBgDeey+32nl3PjaUA1iT2ROjH4DV7v5/3bg/kd5CY1uSkPcE7u4HAfCPfSIJ09iWVGgZoYhIojSBi4gkSqXzBM2YMSOIffvb36a5rOX5pz/9Kc39+c9/HsRiLcTSWUtLC95///1OMVZUBHhLNStsAvw09Dlz5tDc3bt3B7FYIfXDDz8MYrFC+IEDB4LYggULaO4777wTxObODRegxU5j37p1axC7/fbbaS7bzzu2bzcTKxyz4mhsSwJ22n3sfWcF2uztFz7qGhh9AhcRSZQmcBGRRGkCFxFJlCZwEZFEaQIXEUmUTqXvxWKb5q9atSqI3X333TR3zZo1Qezzn/989y6sF7tUp9IPHz7c58+f3ykWW2nBjBgxgsbZapFYqzZb/RBbhcJWP8QOImAHfcyePZvmsgMV2DXEVl988pOfDGJHjhyhuWwFSG1tLc1lK7di19DU1BTE2OoagG8/EFsJw+KxlTDZK1bq6+tx7tw5nUovIlIqNIGLiCRKE7iISKI0gYuIJEqt9L3YihUraJwVIZ9++mma+8ADDxT0moRjrfSx9nhWPIudps6KmLGCJyuIsYIcAAwYMCCIxfamZnthb9++neZ+9rOfDWLs9Pe6ujp6e1YULC8vp7mnT58OYnfccQfNZfuEx17z7PcRiO9JzoqmscIxM27cOBofPXp0p59jBXF9AhcRSZQmcBGRRGkCFxFJlCZwEZFEaQIXEUnURVehmNlTAO4E0OjuszKxUQB+B6AKQC2Ae9z9ePEus7SwFQALFy4MYosWLcr5PlevXk3jsZO3pbBjm51K379/f5p71VVXBbGdO3fS3PHjx1/sof/t+PHwMmMrS1hreOx6Wdt85sDnADuBvqysLIix1wAA9u7dG8RiLe8bN27M6fYA334g9nzHjh0bxNavX09z2bXFXvOWlpYgFvvzmf36xg54yOUT+EoA2bPLgwDWu/t0AOszP4ukZiU0tiVhF53A3X0DgOyFkYsBXNhRaRWAuwp8XSJFp7Etqcu3kafc3eszvz4CgK+0B2BmSwEszfNxRHpaXmM79s9xkWLqdhHT21unotvEuvtyd5/r7uHJpiK9WFfGNvueV6TY8v0E3mBmFe5eb2YVABoLeVGlIrafN9u7O9YKn6tnn32Wxllr8iuvvEJzf/CDHwSxd999t1vXlaC8xjYrYsZa6ffs2RPEYmOFFbliBURWqNuxYwfNHTx4cBCbPHkyzZ05c2YQe/XVV2nutGnTghjbzzt2cjsr5sb+dcNa/GPjleWyYiUAbNu2LYhNnTqV5rK2+YEDB9Lc119/PYjF9lVvbMxtSs33E/jzAJZkfr0EwNo870ekt9HYlmRcdAI3s2cBbAQw08zqzOx+AI8C+IyZ7QNwa+ZnkaRobEvqLvoVirvfG/mtTxf4WkR6lMa2pE6dmCIiidIELiKSKB3oUETs9HigOKfCZ28AfwFbhfLFL36R5l599dVBLNbOz04Ev5z16dMnWEnSlRU8w4YNi95vNnbgAMDb7ltbW2nunDlzglhVVRXNZe3i7OAGgK+aYbHYipeDBw8GsTvvvJPmspUsXTlEY/fu3TS3oqIiiMVec3ZgBnvPAH4oQ2w1TvZBHt1ppRcRkV5IE7iISKI0gYuIJEoTuIhIolTE7KJYyzMrDLKWeYAXFplNmzbR+Nq1YXPgo4/yfpNrr702iK1cuZLmVldXB7GvfOUrNPcXv/gFjV+u2traglPSY6fHnzp1KojFio0M21caQNDKD8RPPWeFxVtvvZXmsuJobN9tViycMGFCEPvXv/5Fb7948eIgFtsrnbWsDx06lOaydv5Y4Z/tVX7y5Emay4qLsYLjpz71qSAWWwyQPc/ECqP6BC4ikihN4CIiidIELiKSKE3gIiKJUhHzI7DuuB/+8Ic0d9myZUEsdhjra6+9FsRYETK253JXCl779+8PYm+//TbNve6664JY7IBW6axPnz7BHtusWAnw4masSMYOioh1740aNSqI3XDDDTSXFRZjB+yyol6smM86FlkRkh3sDfAC7fDhw2kuKxbGul9Zl3Esd8OGDUEs9ueAFXNjhUn2nCsrK2ludodnbA94fQIXEUmUJnARkURpAhcRSZQmcBGRRGkCFxFJ1EVXoZjZUwDuBNDo7rMysZ8A+A8AF8rWy9z9hWJdZCGx9tmvf/3rNJfFWVtxzB//+Eca//KXv5zzfXQXWy1w11135Xz72Gn3paCQY7u1tTVYdRJbjcBWdcTa49lKqFguG9uxlVDspPkpU6bQ3DfffDOIxVZPsPHG2uZj+5/v2rUriLGVVADfq37kyJE0l61YYe8DwFfzxE6JZ/eRvRrpgj179gSxWIt8rivNcvkEvhLAQhJ/zN2rM/8lMXmLZFkJjW1J2EUncHffAIAfRyGSMI1tSV13vgP/lpltM7OnzIz/uwWAmS01sxozq+nGY4n0pC6P7djXGiLFlO8E/hsAUwFUA6gH8MtYorsvd/e57j43z8cS6Ul5jW22latIseU16tz939UZM3sSwP8W7IqK7JFHHgliX/va17p9v2+99VYQu//++7t9v931jW98I+fcmprwH0mbN28u5OX0evmO7ba2tuAg2ljB+/z580Fs0KBBNJcVBWfMmEFzWaEtViQ7fvx4EDtw4ADNZe3xZ86cobnsUOEPPvggiB0+fJjefuLEiUEstsc3ex3YawsAL7wQljI+8YlP0Ny//OUvQSy21cG0adNyvgb2l3zsdcze67ygrfRm1vHY5s8B2JHP/Yj0NhrbkpJclhE+C+BmAGPMrA7AwwBuNrNqAA6gFsBXi3iNIkWhsS2pu+gE7u73kvCKIlyLSI/S2JbUqRNTRCRRmsBFRBJ12a19im0kz/z6178OYgsWLKC5H/vYx4JYrEX/sccey/kacvWFL3yBxmMHUDBLliwJYrGKunTWt2/f4KCG2NrwrqxCqa2tDWKxU8/nzJkTxGIt2Wx1ETv0IPZ4bBULAOzYEdZ82eERsdUx27dvD2LV1dU0l72+7k5zb7nlliD217/+leaygx5i98ta/8+dO0dz2Woats0AEG5roAMdRERKjCZwEZFEaQIXEUmUJnARkURddkVMJlYgWLduXRB74oknaC4rZjzwwAM0l+2xfeTIkSAWO/l71apVQezuu++muawF+Pbbb6e5bL9iyV12sa+uro7mjRkzJoixFnQAmDx5chAbO3YszWXv9U033URz//a3vwWxWFs3K+qx6wIQFHIB4L333gtisb24y8vLgxh7vQBg6tSpQeyNN96guaw4evToUZLJtyQ4ffo0zWUFXlZ4BoDrr78+iF177bU0t3///p1+ju21o0/gIiKJ0gQuIpIoTeAiIonSBC4ikihN4CIiibrsVqGw6nmsTfbjH/94ENu0aRPNffHFF3O6PQDMmzcviLEVJz/+8Y/p7VnL86FDh2guO4F+69atNFfy17dv36BVOntT/gvYyoXYKgd2wjlb6QEA06dPD2KxE93ZGIy1t8+ePTuI7dy5k+ayFSMVFRVBLLbyi7XHx/4csVUkXWnxZwdVAHw11qxZs2guWz3EVpsAfCVM9iEgF2SvOmlubqZ5+gQuIpIoTeAiIonSBC4ikihN4CIiicrlTMxKAM8AKEf7OYHL3f1xMxsF4HcAqtB+duA97s4rCL3Id77znSD2hz/8gebGiogMKwCxtmAAWLNmTc73y7zyyitB7Gc/+xnNVcEyrpBj+/z582hoaOgUYwVIgBc3s1unL2Bt87FCHSuIjR49muaysRnLZc+D7fENADfffHMQY3uSz5w5k95+48aNQYwtEACAsrKyILZlyxaay/boju2rHisSM9nvOcD3/QaAkSNHBrHYnvHZxdFYgTmXT+AtAL7v7tcAmA/gm2Z2DYAHAax39+kA1md+FkmJxrYk7aITuLvXu/vWzK+bAOwGMBHAYgAXdlVaBSBcrybSi2lsS+q6tA7czKoAzAawCUC5u9dnfusI2v8Zym6zFMDS/C9RpPi6O7ZjX4GIFFPORUwzGwLgTwC+5+6d9q309k4Y2g3j7svdfa67z+3WlYoUSSHGdmy7T5FiymkCN7MytA/w37r7nzPhBjOryPx+BYBwE12RXk5jW1KWyyoUA7ACwG53/1WH33oewBIAj2b+v7YoV1hgrE328ccfp7kPPfRQEOvKypSuYJv/P/nkkzR39erVQezgwYMFv6ZSV8ix3adPn2B1SVNTE81lq1AOHz5Mc9l9sJZ5gLfjxw4FiT0e8/bbbwex2Mnro0aNCmJsq4q///3v9PbHjh0LYrEVLy+99FIQq6mpoblshc22bdtoLmv9HzRoEM0dNmxYEGMHWAB8RRF7vkC46iS23Ucu/+67EcB9ALab2T8zsWVoH9y/N7P7AbwD4J4c7kukN9HYlqRddAJ399cB8J1ngE8X9nJEeo7GtqROnZgiIonSBC4ikiiLfTlelAcz67kHKwBWkLnvvvtobmy/YObs2bNB7OGHHw5isf2KJc7dY1+JFNWwYcN8/vz5nWKxIiZbcsjarIH4yekMa4+PrU+vrq7OKQbw1vDY2GSF1JMnTwYx1gYP8CJkbI567rnncs6dOHFiEDtx4gTNZc8h1vrPWuFjS0rZNbBWfCA82b6hoQHNzc3B2NYncBGRRGkCFxFJlCZwEZFEaQIXEUmUJnARkURpB56PwCrtsbZ7kewVELHVCGxVR319PcnkubHVLaw9PnaoBLsPtqoDAMaPHx/EYqtm2KoXtqrj5ZdfprcfMmRIEIu1prOVLLHDLtj1njp1iuayFSfDhw+nufv37w9i7KR6gD+32IEO2bmx10CfwEVEEqUJXEQkUZrARUQSpQlcRCRRKmKKFEBbW1tQFDtz5gzNZcWsWJGM7bsdK46y+2An1QPA1q1bg9jevXtpLtvHOtaiz4qm69atC2KLFi2it9+xY0cQY1sEALxYeOONN9LcN998M4hdffXVNJe9P7GWd7Z3eOz9YYXIAQMG0NzswjE7LwDQJ3ARkWRpAhcRSZQmcBGRRGkCFxFJ1EUncDOrNLNXzWyXme00s+9m4j8xs8Nm9s/Mf3cU/3JFCkdjW1KXyyqUFgDfd/etZjYUwBYzu1BWfszd/6t4lydSVAUb22VlZcGKhNhhDKyFO9aazlrD29raaO758+eDWGtrK81lrfRs9QXAD0lobGykuex6s09YB4ADBw7Q20+bNi2INTc309yqqioaZyZNmhTEBg4cSHPZ47FDKQB+OEsMu94jR47Q3EGDBuV0n7kcalwPoD7z6yYz2w0gPFpCJDEa25K6Ln0HbmZVAGYD2JQJfcvMtpnZU2ZGP0KY2VIzqzGz8KwkkV6iu2ObrdcWKbacJ3AzGwLgTwC+5+4nAfwGwFQA1Wj/FPNLdjt3X+7uc919bgGuV6TgCjG2Yw0ZIsWU0wRuZmVoH+C/dfc/A4C7N7h7q7u3AXgSwLziXaZIcWhsS8ou+h24mRmAFQB2u/uvOsQrMt8hAsDnAIQ9sCK9WCHH9rlz54LC3BVXXJHztcQKnqygFmstZ0VItkc4AEyYMCGIxQpq7BrYHuEAb29nr0OsFZ/d/tixYzSXPd/YXtyVlZVBLLZ3ODNu3DgaZ4XQWDG4fbh1Fnsds4umrJAM5LYK5UYA9wHYbmb/zMSWAbjXzKoBOIBaAF/N4b5EehONbUlaLqtQXgcQ/tUBvFD4yxHpORrbkjp1YoqIJEoTuIhIojSBi4gkSgc6iBSAmdFT0hnWCs9WhQDAvn37gljsNHW2iiTWon/ixIkgxk6PB/gqknfeeYfmsgMd2Mnr7JAIgK+2GDFiBM1lYlsHsFU+9fX1JJMf0hBz6NChnG/PDtyIXW/2a862IwD0CVxEJFmawEVEEqUJXEQkUZrARUQSZbEWzaI8mNlRABeqH2MAhMc0p0/P69K50t15dazIOoztFF6nfJXqc0vhedGx3aMTeKcHNqspxR0K9bwub6X8OpXqc0v5eekrFBGRRGkCFxFJ1KWcwJdfwscuJj2vy1spv06l+tySfV6X7DtwERHpHn2FIiKSKE3gIiKJ6vEJ3MwWmtleM9tvZg/29OMXUubE8kYz29EhNsrM1pnZvsz/+W5CvZiZVZrZq2a2y8x2mtl3M/Hkn1sxlcrY1rhO57n16ARuZn0BPAFgEYBr0H501TU9eQ0FthLAwqzYgwDWu/t0AOszP6emBcD33f0aAPMBfDPzPpXCcyuKEhvbK6FxnYSe/gQ+D8B+dz/o7s0AngOwuIevoWDcfQOA97PCiwGsyvx6FYC7evSiCsDd6919a+bXTQB2A5iIEnhuRVQyY1vjOp3n1tMT+EQAHTfQrcvESkl5hxPNjwDgR4gnwsyqAMwGsAkl9twKrNTHdkm996UyrlXELCJvX6OZ7DpNMxsC4E8AvufuJzv+XurPTfKX+ntfSuO6pyfwwwAqO/w8KRMrJQ1mVgEAmf83XuLryYuZlaF9kP/W3f+cCZfEcyuSUh/bJfHel9q47ukJfDOA6WZ2lZn1B/AlAM/38DUU2/MAlmR+vQTA2kt4LXkxMwOwAsBud/9Vh99K/rkVUamP7eTf+1Ic1z3eiWlmdwD4bwB9ATzl7v/ZoxdQQGb2LICb0b4dZQOAhwH8D4DfA5iM9u1F73H37IJQr2ZmNwF4DcB2ABcOcFyG9u8Lk35uxVQqY1vjOp3nplZ6EZFEqYgpIpIoTeAiIonSBC4ikihN4CIiidIELiKSKE3gIiKJ0gQuIpKo/weJV6VnMyKIWgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAC4CAYAAAD61bdSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAZtklEQVR4nO3da4yV1bkH8P/jcBGBKvdOuQgqaBEVLbUmmrZasdiQ4CUx2saY2Bz6oW3U4AdjGm1Oc9LGnFZOmhNTjAaMHsWm7SlpzTlaetGmakWDFxi5FEFghotyHxBh5jkfZtvD7PV/5J19m1mb/y8xzn5Ye+/3ffdised9nrWWuTtERCQ/p/X3AYiISGU0gIuIZEoDuIhIpjSAi4hkSgO4iEimNICLiGSqqgHczOaZ2Toz22hm99XqoET6m/q25MAqrQM3sxYA6wHMBbANwGsAbnP3tZ/yHBWdS125u1X7GpX07cGDB/vpp59e/jq0bXd3dxKL/h4OGTIkiR07doy27erqSmKnnVb8O9qgQYMKvy47h768X3Rtjh8/nsRaWloKv0Z0XKwtOy8AGDx4cOG27HWja9CXsba87dGjR3Hs2LHkzfgnVszlADa6+yYAMLNnACwAEHZykUz0uW+ffvrpuOyyy3rFhg4dStsePnw4iX300Ue07ZQpU5JYe3s7bdvZ2ZnEomNgA8+YMWNo24MHDyYxdg5Az3UoIvrHYs+ePUnsM5/5TOHXiK4j+0fg0KFDtO348eOT2IEDB2hbdn3POOMM2jY6NqZ8AF+9ejVtV80tlIkAtp7weFsp1ouZLTSzVWa2qor3EmmkPvft6FuxSD3VPYnp7kvcfY67z6n3e4k00ol9m/3aLVJv1dxC2Q5g8gmPJ5ViIrnrc9/u6urC/v37e8Wi2xe7du1KYmPHjuUHsj192+j+8YQJE5LYhx9+SNsyUdt9+/Ylseg2QdF72Lt376bPHzFiRBKLbrew94puA5V/NgC/VRId2/Dhw2lb9hlHt1vYvXF2DgAwcuTIkz4XqO4b+GsAppvZNDMbAuBWACuqeD2RgUJ9W7JQ8Tdwdz9uZt8D8L8AWgA87u5ranZkIv1EfVtyUc0tFLj7cwCeq9GxiAwY6tuSA83EFBHJlAZwEZFMVXULRUT+X3l1yLBhw2i70aNHJ7GoyoFNNolel1WLRFUOUZxhlSHR89lEHlZiGVXSHDlyJIlFVSgszqpNAH68bIISwCtLovNlMzSjtmyWaDRJqfw6hDNfaVRERAY8DeAiIpnSAC4ikikN4CIimVISU/6JJaDuuece2vYHP/hBEotWqLv++uuT2KpVzbW2mbsny7xGy4cePXo0ibEEJMATntF17stysn1JTLa2thZuyxKLrF+NGzeOPp8l9aIkJltqN1qT5s0330xi0XIALJHKYgBw5plnJrHo2kSfG1O+cqGSmCIiTUYDuIhIpjSAi4hkSgO4iEimNICLiGSq4k2NK3ozbWrccBMnJjuB4c4776Rt586dm8SuvPLKqo/hb3/7WxK77rrraNso219ULTY1rsTw4cP985//fK9YtM0aq5Rg1SYA32QhmobOqjVYxQvAp+5HmwezShZWAQLw82AVJ7NmzaLPZ+cQXRt2fTds2EDbsuuwd+9e2nbjxo1JLNrPkn2W0efONopgVSwAsH79+l6P9+7dSzc11jdwEZFMaQAXEcmUBnARkUxpABcRyVRVU+nNbDOAgwC6ABx39zm1OCjpuygp+NBDDyWxiy66qOr3W7Mm3SLywgsvpG0//vjjJMamfQ8kfe3b3d3dSQI2mlLNEoB79uwp3DZKNrKkXLTeNEuERmuSs2Nj0+sBYPLkyUls/vz5SSy6Nh0dHUns4osvpm3Za0ydOpW2/eCDD5LYunXraFt2zdvb22lbtqZ4tCs9KxhhywwA6fVl68IDtVkL5Wp3T6+OSP7Ut2VA0y0UEZFMVTuAO4Dnzex1M1tYiwMSGSDUt2XAq/YWylXuvt3MxgN4wczedfcXT2xQ6vz6CyC56VPfjpYxFamnqr6Bu/v20v93AfgNgMtJmyXuPkcJTslJX/t2lFgUqaeKv4Gb2XAAp7n7wdLP1wH415odmeDaa6+l8QcffDCJfelLX6Jt+zKwsOz5smXLCh/DV77yFdp27dq1SYxVpgwUlfTtlpYWjBw5sless7OTti266QHAlxZgmzEAfLp4tKHD+++/n8SiKetXXXVVEhs1ahRtyypZWAVHVDXFXnfHjh207fjx45NYtPkDq7oZM2YMbbtr164kFm2owN7vs5/9LG3LsCoWAJgwYUKvx++99x5//8LvRN4DwG9KF2YQgP9y9/+p4vVEBgr1bclCxQO4u28CcEkNj0VkQFDfllyojFBEJFMawEVEMqVd6fsB2w17wYIFSewXv/gFfT5LYkXrurM1iO+9917a9qWXXkpiW7ZsoW2ZFStWFG7bbE477bRk2nr0mbBp0dF60yzRxtYIj143SkyyssdojW/2uiyxCQBDhw5NYmPHjk1i0dIBbO34aNp+W1tbEitPJH+CnRtbKx/gfz+j42V/P6LryIoEouMt/3sbLT2gb+AiIpnSAC4ikikN4CIimdIALiKSKQ3gIiKZUhVKHUXTeh999NEkduuttxZ+XTYt+Mknn6RtH3jggSS2efPmwu8lxXR3dydVBlE1AqsoGDZsGG3LpseHi/uT/rZ9+3balu2Gfs4559C2hw8fTmJsKj4AXHPNNUmMVc0sX76cPp8tsRDtNM8qQ6Lp8WzafbQBCdsYY9KkSbQt2yiCfWYAr8Zh1xZIq3miJRH0DVxEJFMawEVEMqUBXEQkUxrARUQypSRmjbAk1BtvvEHbzpgxo9BrRgmOm2++OYn95S9/oW2jhIrUXvna61HiiYmm3bOEZzS1nCX1onWsWTyazs92mj///PNpW4YlTKM1s//4xz8msSgRyxKel156KW3LEr/RGuzHjh1LYtFO8+XLJwBxkQB7v2ipiosuuqjX46ggQt/ARUQypQFcRCRTGsBFRDKlAVxEJFMawEVEMnXSKhQzexzAfAC73H1WKTYawHIAUwFsBnCLu6fzT5tQNN2YLUQ/bty4wq/75ptvJrFol/doJ2vpm1r37fIlDqLKBbbTfFRxxNpGWKXC7NmzaVs2Xfy8886jbefPn5/E2KYiAK+aYbvKr1u3jj6fVdJElSXbtm1LYlHlF9u8Yf/+/bRteTUREC9fwD43VsUC8IqwaIwov2bRaxb5Br4UwLyy2H0AVrr7dAArS49FcrMU6tuSsZMO4O7+IoDyfxYXAFhW+nkZgBtqfFwidae+LbmrdCLPBHfvKP28A8CEqKGZLQSwsML3EWm0ivo22wtSpN6qTmJ6zxQyPo2s58+XuPscd59T7XuJNFJf+nY0U06knirtdTvNrNXdO8ysFcCuWh7UQPGFL3whiT3yyCO0LUtGdHV10baLFi1KYmwqvJKV/aKivt3d3Z0kutjO7wCfUh2tB852lWdrUAM8sRj9w1I+VRvgU94BYN++fUksmobOErebNm1KYlFyli0/EC0Twa7vDTfwO15sqYJdu/hHyxKW0TID7Dev6HOPEpFM+VrybA8AoPJv4CsA3FH6+Q4Av63wdUQGGvVtycZJB3AzexrAywDON7NtZvZtAD8BMNfMNgC4tvRYJCvq25K7k95Ccffbgj/6Wo2PRaSh1Lcld5qJKSKSKQ3gIiKZUu3Tp7jpppuSGKtMAYD169cnsRtvvJG2fffdd6s7MBlw3D2ZKj1t2jTalk0tZ9O3AaCzszOJRbvdjxo1KolFlVCsioRVpgDArFmzkhibMg8Aa9asSWJsyno0lZ5Ve0RT6dmyFlFlyRVXXJHEomoeVv0VXfOdO3cmsagKpaOjI4lFVUJFy1L1DVxEJFMawEVEMqUBXEQkUxrARUQypSQm+M7SAHDPPfcksWhK6+LFi5PYQE1WsunZAHDWWWcVfg02vZqt5XyqMLMkMRjtTs76G9thPRJ9TiNHjkxi0XrgF1xwQRJjiT6ATy1fvXo1bcuSkP/4xz+SGJsyDwBf//rXC70mAEyfPj2JTZ06lbZl1yxKHE+YkK5f9vbbb9O27e3tSYwlqQFgzJgxNM6UJ36jZLS+gYuIZEoDuIhIpjSAi4hkSgO4iEimlMQEcMstt9A4W+s3mkH27LPPVnUMbD3xiy++mLa96667khhb7zjCElhAvKkts3bt2iTGElAAT/Q0m66uriSxGyWtWMIySuoxn/vc52iczQCcNGkSbcsSgGxWIcAT99Ha1hs3bkxiLGk+Y8YM+nzW9owzzqBtWWKSzUYFeCI0WoOd/R2PPsvx48cnsWizZPa5R2v+l59blHDVN3ARkUxpABcRyZQGcBGRTGkAFxHJlAZwEZFMnbQKxcweBzAfwC53n1WK/RDAvwDYXWp2v7s/V6+DHEj27t1bOB6tC/zEE08kMTaNecqUKfT5rCqgL1UotTBz5swk9uSTT9K211xzTb0PpyK17NstLS1J5UC0RAOr4IiqEcaOHZvEoiULWHXR5ZdfTtuyNb5XrVpF2zITJ06kcVYxwvrrhg0bCj8/6tusioStcw7wKpRoTXN2zd966y3atru7O4kNHz6ctt22bVsSi9YZP3LkSK/H0TUo8g18KYB5JP6wu88u/XdKDN7SdJZCfVsydtIB3N1fBHDqrlIkTUt9W3JXzT3w75nZW2b2uJnx6nkAZrbQzFaZWfHfz0T6V5/7dvTruEg9VTqAPwLgXACzAXQA+GnU0N2XuPscd59T4XuJNFJFfbvoHoYitVRRr3P3f865NbNHAfyuZkc0wE2ePJnGf/nLXyaxefPY7dV4anBRbOPWKMnBEot9Wbc7WiN6/vz5Sawv6x0PVJX27UGDBiXJr927d9O2I0aMSGJRwptNy476IEtMHj58mLZlidRoLW2WYI2mdrN1q9k65VGCl03FjxJ9bFNitgk0wJOu0XIArCAhel0Wj9YvZ0sgREURdd3U2MxaT3h4I4B3KnkdkYFGfVtyUqSM8GkAXwUw1sy2AXgQwFfNbDYAB7AZwHfqeIwidaG+Lbk76QDu7reR8GN1OBaRhlLfltxpJqaISKY0gIuIZEq1T30ULaZ/0003FX6N9957L4m9/PLLSez3v/89ff4zzzxT+L36glULfPOb36zLezWbrq4uHDhwoFeMbQgC8OqHqC2bGn7JJZfQtqwyJKp4YlUOUeXDmWeemcTeffdd2pZhU8jZhiAAr9yJNl64+eabkxg7VoBXSG3dupW2ZdeMbcYA8Kqb6DqW949PU155E234oW/gIiKZ0gAuIpIpDeAiIpnSAC4ikiklMRHvkF2taKd6tqs8mx5fL1ES7IEHHkhibN1vgCdvfvzjH1d3YBlz9yTRFS0tUL7W8yfPZ1hCLZqqzT6TaJ1xtgM9SzYCPJH64Ycf0rbsnNkxRFPI2XT+aE1zlrCM1uLu6OhIYq+++iptu3nz5iQWrV++Y8eOwscwevToJMYKGoA0ccvWVAf0DVxEJFsawEVEMqUBXEQkUxrARUQypQFcRCRTp1wVyqJFi5LY3Xff3bD3AupTcRLtxv2tb30riS1evJi27UvFw5133pnEli9f/mmH2NSOHz+eVGZE09jZTuYsBvDqA7bkAQAcOnQoiUWbd0yaNCmJffGLX6Rt2bR3tnM7wDdZYBtYXH311fT5fdmarrW1NYm1t7fTtmz5gvfff5+2ZVUo0eYPbPmCs846i7ZllT+sMgVIN4qI+oe+gYuIZEoDuIhIpjSAi4hkSgO4iEimiuyJORnAEwAmoGefwCXu/h9mNhrAcgBT0bN34C3uzufH9oNx48bR+Pe///0kFu2wzZIOUbKQTev93e/4huYPPfRQEnvnneJ757Kp+HPnzqVt2Q7mbMo1APzhD39IYrfffjtt28ip//VSy749ZMiQZK346DqzafPRetMsecZ2qgd4YvLCCy+kbdn60lEC8YILLkhi0a7yq1evTmIsqRcleNk5RH2N/f2M1tz++9//nsSidcbZ9Hg2FR/g+wNEyxewHeyjdb7Lk5ZRXyryDfw4gEXuPhPAFQC+a2YzAdwHYKW7TwewsvRYJCfq25K1kw7g7t7h7m+Ufj4IoA3ARAALACwrNVsG4IZ6HaRIPahvS+76VAduZlMBXArgVQAT3P2T3yt2oOfXUPachQAWVn6IIvVXbd8u3wJLpBEKJzHNbASAXwG429173Wjynpt6dD1Md1/i7nPcfU5VRypSJ7Xo24MHD27AkYr0VmgAN7PB6OngT7n7r0vhnWbWWvrzVgD5Z7XklKO+LTkrUoViAB4D0ObuPzvhj1YAuAPAT0r//21djrBCbFoxALz00ktJLNp5/ec//3kSe+2112hbtpnBZZddRts+9dRTNF4Um14dbQjw5z//OYlF1TEPP/xwVceVm1r2bXdPPoOo0mLTpk1J7Nxzz6Vt2cYH0W7q7DbO66+/TtuyypLotwhWgRFNWT/vvPOS2JYtW5JY1F9ZBUh0HZ9//vlCzwf4tPnoHJjo2rB4X1432qihfFOIqFKuyD3wKwHcDuBtM/ukRuh+9HTuZ83s2wC2ALil0BGLDBzq25K1kw7g7v5XAPyfCeBrtT0ckcZR35bcaSamiEimNICLiGTKomRCXd7MrHFvFmDTZ++9917a9pVXXkliL7zwAm3L1jxmu7wD8Trh5ZYuXUrjP/rRjwo9H+AJlWjadjNw9+iWSF0NHz7cZ86c2StWdJo0EE9jZwm8c845h7YdNWpUEhs6dChty5Z++PKXv1y4LUtMAnz39vPPPz+JRWtxs6UDovdqa2tLYtu2baNt2RIYUe0+K4AYNIjfbWafTzT1ny0/EL1u+Zrvr7zyCg4cOJD0bX0DFxHJlAZwEZFMaQAXEcmUBnARkUxpABcRydQpV4Uiza2/qlCGDRvm5dPho2nSrCIiqkZg8Y8++oi2ZZuYjB8/nrZlu8qzzQkAYP369Uks2u3+7LPPTmK7d+9OYtE5sKoZthkDAEybNi2JsSoYgF/zaNOX7du3J7GoYoV9PtGu9Kw6Jbrm5a+xcuVK7NmzR1UoIiLNQgO4iEimNICLiGRKA7iISKaUxJSmMpCm0kdJMpbMOnr0KG3LptizZRuitlFylC2n0NraStvu37+/0HsBwKpVq5IYWxc/mvLOpptH78XGLracAMCTo+y8ovdjyVmAJ2NHjx5N27LEb/T5lE/nb2trQ2dnp5KYIiLNQgO4iEimNICLiGRKA7iISKZOOoCb2WQz+5OZrTWzNWZ2Vyn+QzPbbmarS/99o/6HK1I76tuSu5NWoZhZK4BWd3/DzEYCeB3ADejZ6PWQu/974TdTFYrUWV+qUGrZt0eOHOmzZ8/uFWObEwB8J/N9+/bRtmxadjQNnW0uEFW3sHi0AQWbCs+qRaLXjapImM7OziTGNpSIlO/m/omDBw8msaiap6urK4nNmDGDtu3o6Ch8bH2pEtqxY0evx+3t7Th69GjSt4tsatwBoKP080EzawMwsdARiwxg6tuSuz7dAzezqQAuBfBqKfQ9M3vLzB43M1qAaWYLzWyVmaUFoiIDRLV9O/q2LVJPhQdwMxsB4FcA7nb3AwAeAXAugNno+RbzU/Y8d1/i7nPcfU4Njlek5mrRt9ltEZF6KzSAm9lg9HTwp9z91wDg7jvdvcvduwE8CuDy+h2mSH2ob0vOTnoP3HoWNX4MQJu7/+yEeGvpHiIA3Ajgnfocokh91LpvlycBjxw5QtuxRF80/ZrtkF6L3dRZ8UKU8CzfIR2IE6lsKjtLjkbT2FnCkiUVAb7Gd7QWN/sNKVo6YOvWrYViAL++7LgAnqiOruOkSZN6Pf7ggw/4+9Nob1cCuB3A22a2uhS7H8BtZjYbgAPYDOA7BV5LZCBR35asFalC+SsAVpr1XO0PR6Rx1Lcld5qJKSKSKQ3gIiKZ0gAuIpKpIklMEanAmDFjCrdlU8gBYNiwYUksWv6CVaewyhSAV4ZE09BZpUW0Kz2rwDh8+HCh1wT48gNR1Q17r6jyh02bZ0sEALw6hU3FB4CWlpYkFlWWsOONPp/yqpNoOQJ9AxcRyZQGcBGRTGkAFxHJlAZwEZFMNXpX+t0AtpQejgXA54fmTefVf852d74VfJ2d0LdzuE6VatZzy+G8aN9u6ADe643NVjXjCoU6r1NbM1+nZj23nM9Lt1BERDKlAVxEJFP9OYAv6cf3ried16mtma9Ts55btufVb/fARUSkOrqFIiKSKQ3gIiKZavgAbmbzzGydmW00s/sa/f61VNqxfJeZvXNCbLSZvWBmG0r/pzuaD2RmNtnM/mRma81sjZndVYpnf2711Cx9W/06n3Nr6ABuZi0A/hPA9QBmomfrqpmNPIYaWwpgXlnsPgAr3X06gJWlx7k5DmCRu88EcAWA75Y+p2Y4t7posr69FOrXWWj0N/DLAWx0903u/jGAZwAsaPAx1Iy7vwigfF3NBQCWlX5eBuCGhh5UDbh7h7u/Ufr5IIA2ABPRBOdWR03Tt9Wv8zm3Rg/gEwGcuL3ztlKsmUw4YUfzHQAm9OfBVMvMpgK4FMCraLJzq7Fm79tN9dk3S79WErOOvKdGM9s6TTMbAeBXAO529wMn/lnu5yaVy/2zb6Z+3egBfDuAySc8nlSKNZOdZtYKAKX/7+rn46mImQ1GTyd/yt1/XQo3xbnVSbP37ab47JutXzd6AH8NwHQzm2ZmQwDcCmBFg4+h3lYAuKP08x0AftuPx1IRMzMAjwFoc/efnfBH2Z9bHTV7387+s2/Gft3wmZhm9g0AiwG0AHjc3f+toQdQQ2b2NICvomc5yp0AHgTw3wCeBTAFPcuL3uLufAPBAcrMrgLwEoC3AXSXwvej535h1udWT83St9Wv8zk3TaUXEcmUkpgiIpnSAC4ikikN4CIimdIALiKSKQ3gIiKZ0gAuIpIpDeAiIpn6PwwUqvwT/zDxAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "out_img = torch.squeeze(output.cpu().data)\n",
    "print(out_img.size())\n",
    "\n",
    "for i in range(2):\n",
    "    plt.subplot(1,2,1)\n",
    "    plt.imshow(torch.squeeze(image[i]).numpy(),cmap='gray')\n",
    "    plt.subplot(1,2,2)\n",
    "    plt.imshow(out_img[i].numpy(),cmap='gray')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "colab": {},
    "colab_type": "code",
    "id": "utDLVRM7Ov2l"
   },
   "source": [
    "### Convolution Stacked AutoEncoder\n",
    "\n",
    "Stacked AutoEncoder의 Model이 성능이 안좋은 것을 확인하여 Model을 ANN구조가 아닌 CNN의 구조로서 다시 작성한다.  \n",
    "\n",
    "중요한것은 **Convolution Stacked AutoEncoder에서의 Decoder**이다.  \n",
    "결국 줄어든 Feature 특성을 다시 Input Size에 맞게 늘리기 위해서는 Convolution연산과 같은 방법으로서 Data를 늘려야 하기 때문이다.  \n",
    "이러한 Convolution의 역 연산을 **DeConvolution**이라 한다.\n",
    "\n",
    "Pytorch는 이러한 연산을 **ConvTranspose2d**을 통하여 지원한다.\n",
    "\n",
    "<code>torch.nn.ConvTranspose2d(in_channels, out_channels, kernel_size, stride=1, padding=0, output_padding=0, groups=1, bias=True, dilation=1, padding_mode='zeros')</code>\n",
    "- stride: controls the stride for the cross-correlation.\n",
    "- padding: controls the amount of implicit zero-paddings on both sides for dilation * (kernel_size - 1) - padding number of points. See note below for details.\n",
    "- output_padding: controls the additional size added to one side of the output shape. See note below for details.\n",
    "- dilation: controls the spacing between the kernel points; also known as the à trous algorithm. It is harder to describe, but this link has a nice visualization of what dilation does.\n",
    "- groups: controls the connections between inputs and outputs\n",
    "\n",
    "아래와 같은 가정이 있을때 **간단한 수식을 사용하여 Output의 차원의 크기를 쉽게 예측**할 수 있다.  \n",
    "\n",
    "- 입력 크기: (N, C_{in}, H_{in}, W_{in})\n",
    "- 출력 크기: (N, C_{out}, H_{out}, W_{out})\n",
    "\n",
    "\n",
    "<p>$$H_{out}=(H_{in}−1)×stride[0]−2×padding[0]+dilation[0]×(kernel_size[0]−1)+output_padding[0]+1$$</p>\n",
    "<p>$$W_{out}=(W_{in}−1)×stride[0]−2×padding[0]+dilation[0]×(kernel_size[0]−1)+output_padding[0]+1$$</p>\n",
    "\n",
    "\n",
    "좀 더 명확히 알아보기 위하여 그림으로서 알아보자.  \n",
    "아래 그림의 Parameter는 다음과 같다.\n",
    "- input size: 2 x 2\n",
    "- output size: 4 x 4\n",
    "- kernel size: 3 x 3\n",
    "- stride: 1\n",
    "- padding: 0\n",
    "- out_padding: 0\n",
    "\n",
    "<img src=\"https://i.stack.imgur.com/YyCu2.gif\" alt=\"\" width=\"208\" height=\"233\">\n",
    "\n",
    "위와 같은 조건에서 Parameter의 Stride를 2로 증가시키고 Padding을 넣었을 경우의 Deconvolution의 결과이다.  \n",
    "<img src=\"https://i.stack.imgur.com/f2RiP.gif\" alt=\"\" width=\"208\" height=\"233\">\n",
    "\n",
    "위의 두 그림에서 이해하기 힘든 **Deconvolution에서의 Padding과 Out_padding**에 대해서 좀 더 살펴보자.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Input data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[[1., 1., 1.],\n",
      "          [1., 1., 1.],\n",
      "          [1., 1., 1.]]]])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "tensor([[[[1., 1., 1.],\n",
       "          [1., 1., 1.],\n",
       "          [1., 1., 1.]]]])"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQ8AAAD8CAYAAABpXiE9AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAANvUlEQVR4nO3dbahl1X3H8e+vd3ygNY06I3EYJ1GJJE3aBvViTSxFGgMqwSnEgnmRaIhMk0aaQAqVBBIIlCZ5kVKrRAYj0RJUaoK5CQbRqjWl+HCV0fEB61UoznQaJ2M6RpKazvTfF3cbTq7nPsw6+55zJv1+YHPW3nudvf6ukZ/77AcnVYUkHa7fmHQBko5MhoekJoaHpCaGh6QmhoekJoaHpCYjhUeSE5PcneS57vOEZfodSrKzW+ZGGVPSdMgoz3kk+SrwclV9OcnVwAlV9VdD+r1aVceNUKekKTNqeDwLnF9Ve5NsBu6vqncM6Wd4SL9mRg2P/6qq47t2gJ+8vr6k30FgJ3AQ+HJV3bHM8bYD2wF+6zdz9jvffnRzbZJW9+gTr/24qk5q+e6G1TokuQc4eciuzw+uVFUlWS6J3lZVe5KcDtybZFdVPb+0U1XtAHYAzL7n2Hr4rq2r/gNIajezeeHfW7+7anhU1QXL7UvyoySbB362vLTMMfZ0ny8kuR84E3hDeEg6cox6q3YOuLxrXw58d2mHJCckOaZrbwLOA54ecVxJEzZqeHwZ+ECS54ALunWSzCa5oevzO8B8kseB+1i85mF4SEe4VX+2rKSq9gPvH7J9Hriya/8r8HujjCNp+viEqaQmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCa9hEeSC5M8m2QhydVD9h+T5LZu/0NJTu1jXEmTM3J4JJkBrgMuAt4FfDjJu5Z0+zjwk6p6O/C3wFdGHVfSZPVx5nEOsFBVL1TVL4BbgW1L+mwDburatwPvT5IexpY0IX2ExxbgxYH13d22oX2q6iBwANjYw9iSJmSqLpgm2Z5kPsn8vv2HJl2OpBX0ER57gK0D66d024b2SbIBeDOwf+mBqmpHVc1W1exJG2d6KE3SeukjPB4BzkhyWpKjgcuAuSV95oDLu/alwL1VVT2MLWlCNox6gKo6mOQq4C5gBrixqp5K8iVgvqrmgG8A/5BkAXiZxYCRdAQbOTwAqupO4M4l274w0P5v4E/7GEvSdJiqC6aSjhyGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCaGh6QmhoekJoaHpCa9hEeSC5M8m2QhydVD9l+RZF+Snd1yZR/jSpqcDaMeIMkMcB3wAWA38EiSuap6eknX26rqqlHHkzQd+jjzOAdYqKoXquoXwK3Ath6OK2mK9REeW4AXB9Z3d9uW+lCSJ5LcnmTrsAMl2Z5kPsn8vv2HeihN0noZ1wXT7wGnVtXvA3cDNw3rVFU7qmq2qmZP2jgzptIktegjPPYAg2cSp3Tbfqmq9lfVa93qDcDZPYwraYL6CI9HgDOSnJbkaOAyYG6wQ5LNA6uXAM/0MK6kCRr5bktVHUxyFXAXMAPcWFVPJfkSMF9Vc8BfJLkEOAi8DFwx6riSJitVNekahpp9z7H18F1Dr6tK6snM5oVHq2q25bs+YSqpieEhqYnhIamJ4SGpieEhqYnhIamJ4SGpieEhqYnhIamJ4SGpieEhqYnhIamJ4SGpieEhqYnhIamJ4SGpieEhqYnhIamJ4SGpieEhqYnhIamJ4SGpieEhqYnhIamJ4SGpieEhqYnhIalJL+GR5MYkLyV5cpn9SXJNkoUkTyQ5q49xJU1OX2ce3wQuXGH/RcAZ3bId+HpP40qakF7Co6oeAF5eocs24OZa9CBwfJLNfYwtaTLGdc1jC/DiwPrubtuvSLI9yXyS+X37D42pNEktpuqCaVXtqKrZqpo9aePMpMuRtIJxhcceYOvA+indNklHqHGFxxzw0e6uy7nAgaraO6axJa2DDX0cJMktwPnApiS7gS8CRwFU1fXAncDFwALwM+BjfYwraXJ6CY+q+vAq+wv4VB9jSZoOU3XBVNKRw/CQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNSkl/BIcmOSl5I8ucz+85McSLKzW77Qx7iSJqeXv+ga+CZwLXDzCn1+WFUf7Gk8SRPWy5lHVT0AvNzHsSQdGcZ5zeO9SR5P8oMk7x7WIcn2JPNJ5vftPzTG0iQdrnGFx2PA26rqPcDfA3cM61RVO6pqtqpmT9o4M6bSJLUYS3hU1StV9WrXvhM4KsmmcYwtaX2MJTySnJwkXfucbtz94xhb0vro5W5LkluA84FNSXYDXwSOAqiq64FLgU8mOQj8HLisqqqPsSVNRi/hUVUfXmX/tSzeypX0a8InTCU1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUxPCQ1MTwkNTE8JDUZOTySbE1yX5KnkzyV5NND+iTJNUkWkjyR5KxRx5U0WX38RdcHgc9W1WNJ3gQ8muTuqnp6oM9FwBnd8gfA17tPSUeokc88qmpvVT3WtX8KPANsWdJtG3BzLXoQOD7J5lHHljQ5vV7zSHIqcCbw0JJdW4AXB9Z388aAkXQE6S08khwHfBv4TFW90niM7Unmk8zv23+or9IkrYNewiPJUSwGx7eq6jtDuuwBtg6sn9Jt+xVVtaOqZqtq9qSNM32UJmmd9HG3JcA3gGeq6mvLdJsDPtrddTkXOFBVe0cdW9Lk9HG35TzgI8CuJDu7bZ8D3gpQVdcDdwIXAwvAz4CP9TCupAkaOTyq6l+ArNKngE+NOpak6eETppKaGB6SmhgekpoYHpKaGB6SmhgekpoYHpKaGB6SmhgekpoYHpKaGB6SmhgekpoYHpKaGB6SmhgekpoYHpKaGB6SmhgekpoYHpKaGB6SmhgekpoYHpKaGB6SmhgekpoYHpKaGB6SmhgekpoYHpKajBweSbYmuS/J00meSvLpIX3OT3Igyc5u+cKo40qarA09HOMg8NmqeizJm4BHk9xdVU8v6ffDqvpgD+NJmgIjn3lU1d6qeqxr/xR4Btgy6nElTbc+zjx+KcmpwJnAQ0N2vzfJ48B/AH9ZVU8N+f52YHu3+trM5oUn+6yvB5uAH0+6iAHWs7Jpqwemr6Z3tH4xVdVLBUmOA/4Z+Ouq+s6Sfb8N/G9VvZrkYuDvquqMVY43X1WzvRTXk2mryXpWNm31wPTVNEo9vdxtSXIU8G3gW0uDA6CqXqmqV7v2ncBRSTb1MbakyejjbkuAbwDPVNXXlulzctePJOd04+4fdWxJk9PHNY/zgI8Au5Ls7LZ9DngrQFVdD1wKfDLJQeDnwGW1+u+lHT3U1rdpq8l6VjZt9cD01dRcT2/XPCT9/+ITppKaGB6SmkxNeCQ5McndSZ7rPk9Ypt+hgcfc59ahjguTPJtkIcnVQ/Yfk+S2bv9D3bMt62oNNV2RZN/AvFy5jrXcmOSlJEOfwcmia7pan0hy1nrVchg1je31iDW+rjHWOVq3V0iqaioW4KvA1V37auAry/R7dR1rmAGeB04HjgYeB961pM+fA9d37cuA29Z5XtZS0xXAtWP6c/oj4CzgyWX2Xwz8AAhwLvDQFNR0PvD9Mc3PZuCsrv0m4N+G/HmNdY7WWNNhz9HUnHkA24CbuvZNwJ9MoIZzgIWqeqGqfgHc2tU1aLDO24H3v34beoI1jU1VPQC8vEKXbcDNtehB4Pgkmydc09jU2l7XGOscrbGmwzZN4fGWqtrbtf8TeMsy/Y5NMp/kwSR9B8wW4MWB9d28cZJ/2aeqDgIHgI0913G4NQF8qDsFvj3J1nWsZzVrrXfc3pvk8SQ/SPLucQy4wusaE5ujtbxCstY56vXdltUkuQc4eciuzw+uVFUlWe4e8tuqak+S04F7k+yqquf7rvUI8z3glqp6LcmfsXhm9McTrmmaPMbivzevvx5xB7Di6xGj6l7X+Dbwmap6ZT3HWqtVajrsORrrmUdVXVBVvztk+S7wo9dP3brPl5Y5xp7u8wXgfhZTtC97gMH/ap/SbRvaJ8kG4M2s79Oyq9ZUVfur6rVu9Qbg7HWsZzVrmcOxqjG/HrHa6xpMYI7W4xWSafrZMgdc3rUvB767tEOSE5Ic07U3sfh069L/b8goHgHOSHJakqNZvCC69I7OYJ2XAvdWd8Vpnaxa05Lfy5ew+Jt2UuaAj3Z3FM4FDgz8HJ2Icb4e0Y2z4usajHmO1lJT0xyN4wr0Gq8IbwT+CXgOuAc4sds+C9zQtd8H7GLxjsMu4OPrUMfFLF6Nfh74fLftS8AlXftY4B+BBeBh4PQxzM1qNf0N8FQ3L/cB71zHWm4B9gL/w+Jv9Y8DnwA+0e0PcF1X6y5gdgzzs1pNVw3Mz4PA+9axlj8ECngC2NktF09yjtZY02HPkY+nS2oyTT9bJB1BDA9JTQwPSU0MD0lNDA9JTQwPSU0MD0lN/g+gA/MOxXXfkAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# 입력으로 1로 채워진 텐서를 생성합니다.\n",
    "img = torch.ones(1,1,3,3)\n",
    "print(img)\n",
    "\n",
    "plt.imshow(img.numpy()[0,0,...],vmin=0)\n",
    "\n",
    "transpose = nn.ConvTranspose2d(in_channels=1, out_channels=1, kernel_size=3, stride=1, padding=0, output_padding=0, bias=False)\n",
    "# 결과를 확인하기 쉽게 전치 컨볼루션 연산의 가중치를 1로 초기화합니다.\n",
    "init.constant_(transpose.weight.data,1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Kernel Size=3, stride=1, padding=0, output_padding=0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[[1., 2., 3., 2., 1.],\n",
      "          [2., 4., 6., 4., 2.],\n",
      "          [3., 6., 9., 6., 3.],\n",
      "          [2., 4., 6., 4., 2.],\n",
      "          [1., 2., 3., 2., 1.]]]], grad_fn=<ConvTranspose2DBackward>) torch.Size([1, 1, 5, 5])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPUAAAD4CAYAAAA0L6C7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAJjklEQVR4nO3d32tcBRrG8edxTGxaV0vZCqGpWwURirAtxFLpXUGIP9AbLywoeyH0ZoUKguil/4B4403QoqAosnohxUUKVkSotWmtYluFUrq1JZBdSo39sU2Tvnsxc9GVTuec6TlzMi/fDwQymXDmIZ1vTzIJM44IAcjjtqYHAKgWUQPJEDWQDFEDyRA1kMztdRx0tLUyxkburuPQlbt2R6vpCaUsjbrpCaW0Fobntyu3XVlqekJhl6/+poWlSze8M9QS9djI3Xpkw9/qOHTlLt+/pukJpczfW8s/WW3uOr3Y9ITCxk6ea3pCYftPvdf1Or79BpIhaiAZogaSIWogGaIGkiFqIBmiBpIhaiAZogaSIWogGaIGkiFqIBmiBpIhaiAZogaSIWogGaIGkikUte0p27/YPmH71bpHAehfz6httyS9JekxSRsl7bC9se5hAPpT5Ey9RdKJiDgZEQuSPpL0dL2zAPSrSNTrJP163eUznY/9H9s7bc/YnllYulTVPgAlVfZAWURMR8RkREyOtlZWdVgAJRWJ+qyk9dddnuh8DMAyVCTqg5IesH2f7VFJz0r6rN5ZAPrV85nhI2LR9ouSvpDUkrQ7Io7WvgxAXwq93ENEfC7p85q3AKgAf1EGJEPUQDJEDSRD1EAyRA0kQ9RAMkQNJEPUQDJEDSRD1EAyRA0kQ9RAMkQNJEPUQDJEDSRD1EAyhZ4koaxrd7R0+f41dRy6crOPjDQ9oZTVD881PaGU2YP3ND2hsHENx31Wkq7Ntrpex5kaSIaogWSIGkiGqIFkiBpIhqiBZIgaSIaogWSIGkiGqIFkiBpIhqiBZIgaSIaogWSIGkiGqIFkiBpIpmfUtnfbnrP90yAGAbg1Rc7U70qaqnkHgIr0jDoivpZ0bgBbAFSAn6mBZCqL2vZO2zO2Z64uXKzqsABKqizqiJiOiMmImBwZXVXVYQGUxLffQDJFfqX1oaT9kh60fcb2C/XPAtCvnq/QERE7BjEEQDX49htIhqiBZIgaSIaogWSIGkiGqIFkiBpIhqiBZIgaSIaogWSIGkiGqIFkiBpIhqiBZIgaSIaogWR6PklCP5ZGrfl7azl05VY/PNf0hFK+3fSPpieUslXPND2hsPmza5ueUNjSQXe9jjM1kAxRA8kQNZAMUQPJEDWQDFEDyRA1kAxRA8kQNZAMUQPJEDWQDFEDyRA1kAxRA8kQNZAMUQPJEDWQDFEDyfSM2vZ62/tsH7N91PauQQwD0J8iTyS2KOnliDhs+0+SDtneGxHHat4GoA89z9QRMRsRhzvv/y7puKR1dQ8D0J9SP1Pb3iBps6QDN7hup+0Z2zOL/71YzToApRWO2vadkj6R9FJEzP/x+oiYjojJiJi8fcWqKjcCKKFQ1LZH1A76g4j4tN5JAG5FkUe/LekdSccj4o36JwG4FUXO1NskPS9pu+0jnbfHa94FoE89f6UVEd9I6v4aHwCWFf6iDEiGqIFkiBpIhqiBZIgaSIaogWSIGkiGqIFkiBpIhqiBZIgaSIaogWSIGkiGqIFkiBpIhqiBZIo873dprYXQXacX6zh05WYP3tP0hFK26pmmJ5Ryfoi+vuOnrzY9obDWQnS9jjM1kAxRA8kQNZAMUQPJEDWQDFEDyRA1kAxRA8kQNZAMUQPJEDWQDFEDyRA1kAxRA8kQNZAMUQPJEDWQTM+oba+w/Z3tH2wftf36IIYB6E+RpzO6Iml7RFywPSLpG9v/jIhva94GoA89o46IkHShc3Gk89b9CZIANKrQz9S2W7aPSJqTtDciDtQ7C0C/CkUdEUsRsUnShKQtth/64+fY3ml7xvbM1YWLVe8EUFCpR78j4rykfZKmbnDddERMRsTkyOiqqvYBKKnIo99rba/uvD8m6VFJP9c9DEB/ijz6PS7pPdsttf8T+Dgi9tQ7C0C/ijz6/aOkzQPYAqAC/EUZkAxRA8kQNZAMUQPJEDWQDFEDyRA1kAxRA8kQNZAMUQPJEDWQDFEDyRA1kAxRA8kQNZAMUQPJFHnmk9Juu7KksZPn6jh05ca1pukJpcyfXdv0hFLGT19tekJhw3KfldqNdb1ugDsADABRA8kQNZAMUQPJEDWQDFEDyRA1kAxRA8kQNZAMUQPJEDWQDFEDyRA1kAxRA8kQNZAMUQPJEDWQDFEDyRSO2nbL9ve299Q5CMCtKXOm3iXpeF1DAFSjUNS2JyQ9IenteucAuFVFz9RvSnpF0rVun2B7p+0Z2zMLS5cqGQegvJ5R235S0lxEHLrZ50XEdERMRsTkaGtlZQMBlFPkTL1N0lO2T0n6SNJ22+/XugpA33pGHRGvRcRERGyQ9KykLyPiudqXAegLv6cGkin1sjsR8ZWkr2pZAqASnKmBZIgaSIaogWSIGkiGqIFkiBpIhqiBZIgaSIaogWSIGkiGqIFkiBpIhqiBZIgaSIaogWSIGkjGEVH9Qe1/S/pXxYf9s6T/VHzMOg3T3mHaKg3X3rq2/iUi1t7oilqiroPtmYiYbHpHUcO0d5i2SsO1t4mtfPsNJEPUQDLDFPV00wNKGqa9w7RVGq69A986ND9TAyhmmM7UAAogaiCZoYja9pTtX2yfsP1q03tuxvZu23O2f2p6Sy+219veZ/uY7aO2dzW9qRvbK2x/Z/uHztbXm95UhO2W7e9t7xnUbS77qG23JL0l6TFJGyXtsL2x2VU39a6kqaZHFLQo6eWI2Chpq6S/L+Ov7RVJ2yPir5I2SZqyvbXhTUXsknR8kDe47KOWtEXSiYg4GRELar/y5tMNb+oqIr6WdK7pHUVExGxEHO68/7vad751za66sWi70Lk40nlb1o/y2p6Q9ISktwd5u8MQ9TpJv153+YyW6R1vmNneIGmzpAPNLumu863sEUlzkvZGxLLd2vGmpFckXRvkjQ5D1KiZ7TslfSLppYiYb3pPNxGxFBGbJE1I2mL7oaY3dWP7SUlzEXFo0Lc9DFGflbT+ussTnY+hArZH1A76g4j4tOk9RUTEeUn7tLwfu9gm6Snbp9T+kXG77fcHccPDEPVBSQ/Yvs/2qNovfP9Zw5tSsG1J70g6HhFvNL3nZmyvtb268/6YpEcl/dzsqu4i4rWImIiIDWrfZ7+MiOcGcdvLPuqIWJT0oqQv1H4g5+OIONrsqu5sfyhpv6QHbZ+x/ULTm25im6Tn1T6LHOm8Pd70qC7GJe2z/aPa/9HvjYiB/ZpomPBnokAyy/5MDaAcogaSIWogGaIGkiFqIBmiBpIhaiCZ/wEgXwieuR2oIgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "out = transpose(img)\n",
    "print(out,out.size())\n",
    "\n",
    "plt.imshow(out.detach().numpy()[0,0,...],vmin=0)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Kernel Size=3, stride=2, padding=0, output_padding=0\n",
    "\n",
    "Stride증가시 Output Size 커짐"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[[1., 1., 2., 1., 2., 1., 1.],\n",
      "          [1., 1., 2., 1., 2., 1., 1.],\n",
      "          [2., 2., 4., 2., 4., 2., 2.],\n",
      "          [1., 1., 2., 1., 2., 1., 1.],\n",
      "          [2., 2., 4., 2., 4., 2., 2.],\n",
      "          [1., 1., 2., 1., 2., 1., 1.],\n",
      "          [1., 1., 2., 1., 2., 1., 1.]]]], grad_fn=<ConvTranspose2DBackward>) torch.Size([1, 1, 7, 7])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f880a82ddd8>"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPUAAAD4CAYAAAA0L6C7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAKxElEQVR4nO3d64tc9R3H8c/HdY2apLWiFTWhkSKCCF4IgaKENkWJVbQPCiootBRCoZZIC6J9UvwHRB+USlBbizfEC4hYL2AkFarmYqyXaFGxmGiJImLWQlzjpw/2pKyamLOzc86M332/YMnuzji/78S898xt5+ckAlDHYaMeAMBwETVQDFEDxRA1UAxRA8Uc3sWFTi5anEVHH9vFRY+16aWjW3tyz+jWlhb2dR+Fvf/9UNN7P/GBTusk6kVHH6sz16zv4qLH2rurD/h33IuTNo32qcmFfN1H4cWnbj7oadz8BoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGimkVte21tl+3/Ybt67oeCsDgDhm17QlJf5R0oaTTJV1h+/SuBwMwmDZH6lWS3kjyVpJPJd0r6dJuxwIwqDZRnyzpnVlf72y+9wW219neYnvL9N6pYc0HYI6G9kBZkg1JViZZObloybAuFsActYl6l6Tls75e1nwPwBhqE/VmSafaPsX2EZIul/Rwt2MBGNQh384oyWe2r5b0uKQJSbcneaXzyQAMpNV7lCV5VNKjHc8CYAh4RRlQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8W02fXydtu7bb/cx0AA5qfNkfovktZ2PAeAITlk1Ek2Sfqwh1kADEGrHTrasL1O0jpJOuKoY4Z1sQDmiK1sgWJ49BsohqiBYto8pXWPpH9IOs32Ttu/7H4sAINqsz/1FX0MAmA4uPkNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRQztF+9nG16qfTuandx0WPtzctuGdna39evRra2tLCv+yhMbz74aRypgWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKKbN+34vt73R9qu2X7G9vo/BAAymzW9pfSbpd0m22V4qaavtJ5O82vFsAAbQZivb95Jsaz7fI2mHpJO7HgzAYOZ0n9r2CklnS3ruAKets73F9pZ9U58MZzoAc9Y6attLJD0g6ZokH3/59Nlb2U4sWTzMGQHMQauobU9qJui7kjzY7UgA5qPNo9+WdJukHUlu7H4kAPPR5kh9rqSrJK2xvb35+EnHcwEYUJutbJ+RtPDeRRD4huIVZUAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMZ1sZTu5RzppU7q46LE2yi1VR/33vZCv+yi8v+fgp3GkBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoJg2b+Z/pO3nbb/YbGV7Qx+DARhMm9/S2itpTZKpZvudZ2z/LcmzHc8GYABt3sw/kqaaLyebj4X3u27AN0TbDfImbG+XtFvSk0m+divb6b1TX70QAL1oFXWSfUnOkrRM0irbZxzgPP/fynZy0ZJhzwmgpTk9+p3kI0kbJa3tZhwA89Xm0e/jbR/TfH6UpPMlvdb1YAAG0+bR7xMl3WF7QjM/BO5L8ki3YwEYVJtHv/8p6eweZgEwBLyiDCiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYjrZn3p6qfTuandx0WPtzctuGdnao9wfWlrY130Upjcf/DSO1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDGto27203rBNu/5DYyxuRyp10va0dUgAIaj7a6XyyRdJOnWbscBMF9tj9Q3SbpW0ucHO8PsrWz3TX0ylOEAzF2bDfIulrQ7ydavO9/srWwnliwe2oAA5qbNkfpcSZfYflvSvZLW2L6z06kADOyQUSe5PsmyJCskXS7pqSRXdj4ZgIHwPDVQzJzeoyzJ05Ke7mQSAEPBkRoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWI62cp2co900qZ0cdFjbZRbqo7673shX/dReH/PwU/jSA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRTT6rXfze4ceyTtk/RZkpVdDgVgcHP5hY4fJfmgs0kADAU3v4Fi2kYdSU/Y3mp73YHOMHsr2+m9U8ObEMCctL35fV6SXba/K+lJ268l2TT7DEk2SNogSUu+s3zh/YIrMCZaHamT7Gr+3C3pIUmruhwKwODabDq/2PbS/Z9LukDSy10PBmAwbW5+nyDpIdv7z393ksc6nQrAwA4ZdZK3JJ3ZwywAhoCntIBiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKKZV1LaPsX2/7dds77D9g64HAzCYtntp3SzpsSQ/s32EpKM7nAnAPBwyatvflrRa0s8lKcmnkj7tdiwAg2pz8/sUSe9L+rPtF2zf2uyp9QVsZQuMhzZRHy7pHEl/SnK2pE8kXfflMyXZkGRlkpWTi5YMeUwAbbWJeqeknUmea76+XzORAxhDh4w6yX8kvWP7tOZbP5b0aqdTARhY20e/fyPpruaR77ck/aK7kQDMR6uok2yXtLLjWQAMAa8oA4ohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGCcZ/oXa70v694D/+XGSPhjiOKzN2hXX/l6S4w90QidRz4ftLUlG8jpz1mbtCmtz8xsohqiBYsYx6g2szdqsPbixu08NYH7G8UgNYB6IGihmrKK2vdb267bfsP2VtyHucN3bbe+2/XJfa85ae7ntjbZftf2K7fU9rn2k7edtv9isfUNfa8+aYaJ5P/lHel73bdsv2d5ue0vPa3e6jdXY3Ke2PSHpX5LO18zbEm+WdEWSzt+51PZqSVOS/prkjK7X+9LaJ0o6Mck220slbZX0056utyUtTjJle1LSM5LWJ3m267VnzfBbzbz/3beSXNzjum9LWpmk9xef2L5D0t+T3Lp/G6skHw3r8sfpSL1K0htJ3mq29rlX0qV9LJxkk6QP+1jrAGu/l2Rb8/keSTskndzT2kmyfzuVyeajt5/ytpdJukjSrX2tOWqztrG6TZrZxmqYQUvjFfXJkt6Z9fVO9fSPe1zYXiHpbEnPff05h7rmhO3tknZLenLWpg19uEnStZI+73HN/SLpCdtbba/rcd1W21jNxzhFvaDZXiLpAUnXJPm4r3WT7EtylqRlklbZ7uXuh+2LJe1OsrWP9Q7gvCTnSLpQ0q+bu2B9aLWN1XyMU9S7JC2f9fWy5nvlNfdnH5B0V5IHRzFDcxNwo6S1PS15rqRLmvu290paY/vOntZWkl3Nn7slPaSZu3996Hwbq3GKerOkU22f0jx4cLmkh0c8U+eaB6tuk7QjyY09r3287WOaz4/SzIOUr/WxdpLrkyxLskIz/6+fSnJlH2vbXtw8KKnmpu8Fknp55qOPbazabrvTuSSf2b5a0uOSJiTdnuSVPta2fY+kH0o6zvZOSX9Iclsfa2vmiHWVpJea+7aS9Pskj/aw9omS7mieeThM0n1Jen1qaUROkPTQzM9THS7p7iSP9bh+p9tYjc1TWgCGY5xufgMYAqIGiiFqoBiiBoohaqAYogaKIWqgmP8BWWra/tXw78UAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "transpose = nn.ConvTranspose2d(in_channels=1, out_channels=1, kernel_size=3, stride=2, padding=0, output_padding=0, bias=False)\n",
    "init.constant_(transpose.weight.data,1)\n",
    "out = transpose(img)\n",
    "\n",
    "print(out,out.size())\n",
    "plt.imshow(out.detach().numpy()[0,0,...],vmin=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Kernel Size=3, stride=2, padding=1, output_padding=0\n",
    "\n",
    "Padding 추가시 결과값에서 Outline padding크기만큼 제거"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[[1., 2., 1., 2., 1.],\n",
      "          [2., 4., 2., 4., 2.],\n",
      "          [1., 2., 1., 2., 1.],\n",
      "          [2., 4., 2., 4., 2.],\n",
      "          [1., 2., 1., 2., 1.]]]], grad_fn=<ConvTranspose2DBackward>) torch.Size([1, 1, 5, 5])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f880abd0748>"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPUAAAD4CAYAAAA0L6C7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAJT0lEQVR4nO3d3WtcBR7G8efZbKwvKYisF9qUrRQRiqBCKELBi4JQX9BLFfRKKIUVKgiil/4D4l4IUlRcUBRBL0RcpGClCK42ahVrFaq4WBXqImLrQk3rsxcZoStNc2Z6zpycH98PBJJMmXk45NuTnIQZJxGAOv7U9wAA7SJqoBiiBoohaqAYogaK+XMXdzq77pKsu/iyLu66dUvr+14wntnjfS8Yz5CO75CO7cn//qilk7/4bLd1EvW6iy/Tddt3d3HXrfvuprMelzXryv3D+hXkkI7vkI7tx2/9fcXb+PYbKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBooplHUtnfY/sL2EduPdD0KwORWjdr2jKQnJd0iaYuke2xv6XoYgMk0OVNvlXQkyVdJfpX0kqQ7u50FYFJNot4g6ZszPj46+tz/sb3T9qLtxaWTJ9raB2BMrV0oS7InyUKShdl1c23dLYAxNYn6W0kbz/h4fvQ5AGtQk6gPSLra9lW2L5B0t6TXup0FYFKrPpl/klO2H5D0pqQZSc8mOdT5MgATafQKHUnekPRGx1sAtIC/KAOKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoJhGT5IwrqX10nc3uYu7bt2Xdz3V94SxbNauvieMZUjHd0jHdunAyrdxpgaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBoohaqAYogaKIWqgGKIGiiFqoBiiBopZNWrbz9o+ZvvTaQwCcH6anKmfk7Sj4x0AWrJq1En2S/pxClsAtICfqYFiWova9k7bi7YXT5/4pa27BTCm1qJOsifJQpKFmblL2rpbAGPi22+gmCa/0npR0ruSrrF91Pb93c8CMKlVX6EjyT3TGAKgHXz7DRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMas+ScIkZo9LV+5PF3fdus3a1feEsQzluP5uSMd3SMf2h+Mr38aZGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWJWjdr2Rtv7bH9m+5Dt3dMYBmAyTZ6j7JSkh5J8aHu9pA9s703yWcfbAExg1TN1ku+TfDh6/7ikw5I2dD0MwGTGejZR25sk3SDpvbPctlPSTkm64KJLW5gGYBKNL5TZnpP0iqQHk/z8x9uT7EmykGRhdt1cmxsBjKFR1LZntRz0C0le7XYSgPPR5Oq3JT0j6XCSx7ufBOB8NDlTb5N0n6Tttg+O3m7teBeACa16oSzJO5I8hS0AWsBflAHFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UMxYzyba1NJ66bubhvG8Cl/e9VTfE8ayWbv6njCWIR3fIR3bpQMr38aZGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKGbVqG1faPt92x/bPmT7sWkMAzCZJk9ndFLS9iQnbM9Kesf2P5P8q+NtACawatRJIunE6MPZ0Vu6HAVgco1+prY9Y/ugpGOS9iZ5r9tZACbVKOokp5NcL2le0lbb1/7x39jeaXvR9uLpE7+0vRNAQ2Nd/U7yk6R9knac5bY9SRaSLMzMXdLWPgBjanL1+3Lbl47ev0jSzZI+73oYgMk0ufp9haR/2J7R8n8CLyd5vdtZACbV5Or3J5JumMIWAC3gL8qAYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiiGqIFiiBoohqiBYogaKIaogWKIGiimyTOfjG32uHTl/mE8i/Bm7ep7wliGclx/N6TjO6Rj+8PxlW/jTA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxRA0UQ9RAMUQNFEPUQDFEDRRD1EAxjaO2PWP7I9uvdzkIwPkZ50y9W9LhroYAaEejqG3PS7pN0tPdzgFwvpqeqZ+Q9LCk31b6B7Z32l60vbh08kQr4wCMb9Wobd8u6ViSD87175LsSbKQZGF23VxrAwGMp8mZepukO2x/LeklSdttP9/pKgATWzXqJI8mmU+ySdLdkt5Kcm/nywBMhN9TA8WM9bI7Sd6W9HYnSwC0gjM1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFEDVQDFEDxRA1UAxRA8UQNVAMUQPFOEn7d2r/IOnfLd/tXyT9p+X77NKQ9g5pqzSsvV1t/WuSy892QydRd8H2YpKFvnc0NaS9Q9oqDWtvH1v59hsohqiBYoYU9Z6+B4xpSHuHtFUa1t6pbx3Mz9QAmhnSmRpAA0QNFDOIqG3vsP2F7SO2H+l7z7nYftb2Mduf9r1lNbY32t5n+zPbh2zv7nvTSmxfaPt92x+Ptj7W96YmbM/Y/sj269N6zDUfte0ZSU9KukXSFkn32N7S76pzek7Sjr5HNHRK0kNJtki6UdLf1vCxPSlpe5LrJF0vaYftG3ve1MRuSYen+YBrPmpJWyUdSfJVkl+1/Mqbd/a8aUVJ9kv6se8dTST5PsmHo/ePa/mLb0O/q84uy35/4fPZ0duavspre17SbZKenubjDiHqDZK+OePjo1qjX3hDZnuTpBskvdfvkpWNvpU9KOmYpL1J1uzWkSckPSzpt2k+6BCiRsdsz0l6RdKDSX7ue89KkpxOcr2keUlbbV/b96aV2L5d0rEkH0z7sYcQ9beSNp7x8fzoc2iB7VktB/1Cklf73tNEkp8k7dPavnaxTdIdtr/W8o+M220/P40HHkLUByRdbfsq2xdo+YXvX+t5Uwm2LekZSYeTPN73nnOxfbntS0fvXyTpZkmf97tqZUkeTTKfZJOWv2bfSnLvNB57zUed5JSkByS9qeULOS8nOdTvqpXZflHSu5KusX3U9v19bzqHbZLu0/JZ5ODo7da+R63gCkn7bH+i5f/o9yaZ2q+JhoQ/EwWKWfNnagDjIWqgGKIGiiFqoBiiBoohaqAYogaK+R9vbf2eEjFKwwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "transpose = nn.ConvTranspose2d(in_channels=1, out_channels=1, kernel_size=3, stride=2, padding=1, output_padding=0, bias=False)\n",
    "init.constant_(transpose.weight.data,1)\n",
    "out = transpose(img)\n",
    "\n",
    "print(out,out.size())\n",
    "plt.imshow(out.detach().numpy()[0,0,...],vmin=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Kernel Size=3, stride=2, padding=0, output_padding=1\n",
    "\n",
    "out_padding 추가시 out_padding크기만큼 결과값 width, height증가"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[[[1., 1., 2., 1., 2., 1., 1., 0.],\n",
      "          [1., 1., 2., 1., 2., 1., 1., 0.],\n",
      "          [2., 2., 4., 2., 4., 2., 2., 0.],\n",
      "          [1., 1., 2., 1., 2., 1., 1., 0.],\n",
      "          [2., 2., 4., 2., 4., 2., 2., 0.],\n",
      "          [1., 1., 2., 1., 2., 1., 1., 0.],\n",
      "          [1., 1., 2., 1., 2., 1., 1., 0.],\n",
      "          [0., 0., 0., 0., 0., 0., 0., 0.]]]],\n",
      "       grad_fn=<ConvTranspose2DBackward>) torch.Size([1, 1, 8, 8])\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "<matplotlib.image.AxesImage at 0x7f880ac9a668>"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAPUAAAD4CAYAAAA0L6C7AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAKxUlEQVR4nO3d34tc9RnH8c/HzbqabDS1tUWzoQlVAlLQyJIiKYEmWGIV7UXBBBQqhaUXSqQF0d71HxB7UZQQtYKp0kYDIlYr/iAV2tQkpq3JxpIESzZqo4gku9D88unFnpTVru6ZmfNj9uH9gsWdneHMc9Z958zMzp6vI0IA8rig7QEAVIuogWSIGkiGqIFkiBpIZkEdGx0cWhRDCy+rY9OtOrO42fsbPNncfTW5b03uV9Mu+GSqkfv5j6Z0Ok55tutqiXpo4WW6dt3mOjbdqvfWzvo9rM2VO5v7dWOT+9bkfjVt4Y5djdzPrnjlC6/j4TeQDFEDyRA1kAxRA8kQNZAMUQPJEDWQDFEDyRA1kEypqG1vsP2O7UO27697KADdmzNq2wOSfi3pJknXSNpk+5q6BwPQnTJH6tWSDkXEkYg4LelpSbfVOxaAbpWJeqmkozMuTxRf+wzbY7Z329595tRkVfMB6FBlL5RFxJaIGI2I0cGh4ao2C6BDZaI+JmnZjMsjxdcA9KEyUb8p6WrbK2xfKGmjpOfqHQtAt+Y8SUJEnLV9t6SXJA1Ieiwi9tc+GYCulDrzSUS8IOmFmmcBUAHeUQYkQ9RAMkQNJEPUQDJEDSRD1EAyRA0kQ9RAMkQNJEPUQDJEDSRD1EAyRA0kQ9RAMkQNJEPUQDJEDSRD1EAyZVboeMz2cdtvNzEQgN6UOVL/RtKGmucAUJE5o46InZI+bmAWABUodTbRMmyPSRqTpAsvXlLVZgF0iGV3gGR49RtIhqiBZMr8SuspSX+WtNL2hO2f1D8WgG6VWUtrUxODAKgGD7+BZIgaSIaogWSIGkiGqIFkiBpIhqiBZCr7g46ZziyW3lvrOjbdqsO3P9Lo/X1LP23svprctyb3q2lX7Wh7Ao7UQDpEDSRD1EAyRA0kQ9RAMkQNJEPUQDJEDSRD1EAyRA0kU+YcZctsv2b7gO39tjc3MRiA7pR57/dZST+PiL22F0vaY/vliDhQ82wAulBm2Z33I2Jv8flJSeOSltY9GIDudPSc2vZySask7ZrlujHbu23vPjc5Vc10ADpWOmrbw5KekXRvRJz4/PUzl90ZGF5U5YwAOlAqatuDmg56W0Q8W+9IAHpR5tVvS3pU0nhEPFj/SAB6UeZIvUbSnZLW2d5XfPyg5rkAdKnMsjtvSMp3biIgKd5RBiRD1EAyRA0kQ9RAMkQNJEPUQDJEDSRD1EAytaylNXhSunJn1LHpVjW9BlST38Mm9y3jz0Y/4UgNJEPUQDJEDSRD1EAyRA0kQ9RAMkQNJEPUQDJEDSRT5sSDF9n+q+2/Fcvu/LKJwQB0p8zbRE9JWhcRk8Wpgt+w/YeI+EvNswHoQpkTD4akyeLiYPHBm3eBPlX2ZP4DtvdJOi7p5Yj40mV3zpya/P+NAGhEqagj4lxEXCdpRNJq29+e5Tb/W3ZncGi46jkBlNTRq98R8Ymk1yRtqGccAL0q8+r35baXFJ9fLOlGSQfrHgxAd8q8+n2FpCdsD2j6H4HfRcTz9Y4FoFtlXv3+u6bXpAYwD/COMiAZogaSIWogGaIGkiFqIBmiBpIhaiAZogaSqWXZnTOLpffWuo5Nt+rw7Y80en9NLoXT5L41vXxRk67a0fYEHKmBdIgaSIaogWSIGkiGqIFkiBpIhqiBZIgaSIaogWSIGkimdNTFCf3fss1JB4E+1smRerOk8boGAVCNssvujEi6WdLWescB0KuyR+qHJN0n6dMvusHMtbTOTU5VMhyAzpVZoeMWSccjYs+X3W7mWloDw4sqGxBAZ8ocqddIutX2u5KelrTO9pO1TgWga3NGHREPRMRIRCyXtFHSqxFxR+2TAegKv6cGkunodEYR8bqk12uZBEAlOFIDyRA1kAxRA8kQNZAMUQPJEDWQDFEDydSy7M7gSenKnVHHplvV9HIxTX4Pm9y3jD8b/YQjNZAMUQPJEDWQDFEDyRA1kAxRA8kQNZAMUQPJEDWQDFEDyZR6m2hxJtGTks5JOhsRo3UOBaB7nbz3+3sR8VFtkwCoBA+/gWTKRh2S/mh7j+2x2W4wc9mdM6cmq5sQQEfKPvz+bkQcs/11SS/bPhgRO2feICK2SNoiScNfWcbf1gEtKXWkjohjxX+PS9ohaXWdQwHoXpkF8hbZXnz+c0nfl/R23YMB6E6Zh9/fkLTD9vnb/zYiXqx1KgBdmzPqiDgi6doGZgFQAX6lBSRD1EAyRA0kQ9RAMkQNJEPUQDJEDSRD1EAyRA0kQ9RAMkQNJEPUQDJEDSRD1EAyRA0kQ9RAMkQNJEPUQDKlora9xPZ22wdtj9u+oe7BAHSn7Hm/fyXpxYj4ke0LJS2scSYAPZgzatuXSlor6ceSFBGnJZ2udywA3Srz8HuFpA8lPW77Ldtbi/N/fwbL7gD9oUzUCyRdL+nhiFglaUrS/Z+/UURsiYjRiBgdHBqueEwAZZWJekLSRETsKi5v13TkAPrQnFFHxAeSjtpeWXxpvaQDtU4FoGtlX/2+R9K24pXvI5Luqm8kAL0oFXVE7JM0WvMsACrAO8qAZIgaSIaogWSIGkiGqIFkiBpIhqiBZIgaSMYRUflGL/Fl8R2vr3y7AKbtild0Ij72bNdxpAaSIWogGaIGkiFqIBmiBpIhaiAZogaSIWogGaIGkpkzatsrbe+b8XHC9r1NDAegc3Oeoywi3pF0nSTZHpB0TNKOmucC0KVOH36vl3Q4Iv5VxzAAelf2FMHnbZT01GxX2B6TNCZJF7F+HtCa0kfq4pzft0r6/WzXf2bZHQ1VNR+ADnXy8PsmSXsj4t91DQOgd51EvUlf8NAbQP8oFXWxdO2Nkp6tdxwAvSq77M6UpK/WPAuACvCOMiAZogaSIWogGaIGkiFqIBmiBpIhaiAZogaSqWXZHdsfSur0zzO/JumjyofpD1n3jf1qzzcj4vLZrqgl6m7Y3h0Ro23PUYes+8Z+9ScefgPJEDWQTD9FvaXtAWqUdd/Yrz7UN8+pAVSjn47UACpA1EAyfRG17Q2237F9yPb9bc9TBdvLbL9m+4Dt/bY3tz1TlWwP2H7L9vNtz1Il20tsb7d90Pa47RvanqlTrT+nLhYI+KemT5c0IelNSZsi4kCrg/XI9hWSroiIvbYXS9oj6Yfzfb/Os/0zSaOSLomIW9qepyq2n5D0p4jYWpxBd2FEfNL2XJ3ohyP1akmHIuJIRJyW9LSk21qeqWcR8X5E7C0+PylpXNLSdqeqhu0RSTdL2tr2LFWyfamktZIelaSIOD3fgpb6I+qlko7OuDyhJD/859leLmmVpF3tTlKZhyTdJ+nTtgep2ApJH0p6vHhqsbU46ea80g9Rp2Z7WNIzku6NiBNtz9Mr27dIOh4Re9qepQYLJF0v6eGIWCVpStK8e42nH6I+JmnZjMsjxdfmPduDmg56W0RkOb3yGkm32n5X00+V1tl+st2RKjMhaSIizj+i2q7pyOeVfoj6TUlX215RvDCxUdJzLc/UM9vW9HOz8Yh4sO15qhIRD0TESEQs1/T/q1cj4o6Wx6pERHwg6ajtlcWX1kuady9sdrpAXuUi4qztuyW9JGlA0mMRsb/lsaqwRtKdkv5he1/xtV9ExAstzoS53SNpW3GAOSLprpbn6Vjrv9ICUK1+ePgNoEJEDSRD1EAyRA0kQ9RAMkQNJEPUQDL/BXzlmOLqy4ELAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "transpose = nn.ConvTranspose2d(in_channels=1, out_channels=1, kernel_size=3, stride=2, padding=0, output_padding=1, bias=False)\n",
    "init.constant_(transpose.weight.data,1)\n",
    "out=transpose(img)\n",
    "\n",
    "print(out,out.size())\n",
    "plt.imshow(out.detach().numpy()[0,0,...],vmin=0)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 1) Model\n",
    "\n",
    "Encoder: 기존에 사용하던 Convolution으로서 Input Size보다 작아지게 구성  \n",
    "Decoder: DeConvolution을 사용하여 Input Size만큼 크기를 키움"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Encoder(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Encoder,self).__init__()\n",
    "        self.layer1 = nn.Sequential(\n",
    "                        nn.Conv2d(1,16,3,padding=1),                            # batch x 16 x 28 x 28\n",
    "                        nn.ReLU(),\n",
    "                        nn.BatchNorm2d(16),\n",
    "                        nn.Conv2d(16,32,3,padding=1),                           # batch x 32 x 28 x 28\n",
    "                        nn.ReLU(),\n",
    "                        nn.BatchNorm2d(32),\n",
    "                        nn.Conv2d(32,64,3,padding=1),                           # batch x 32 x 28 x 28\n",
    "                        nn.ReLU(),\n",
    "                        nn.BatchNorm2d(64),\n",
    "                        nn.MaxPool2d(2,2)                                       # batch x 64 x 14 x 14\n",
    "        )\n",
    "        self.layer2 = nn.Sequential(\n",
    "                        nn.Conv2d(64,128,3,padding=1),                          # batch x 64 x 14 x 14\n",
    "                        nn.ReLU(),\n",
    "                        nn.BatchNorm2d(128),\n",
    "                        nn.MaxPool2d(2,2),\n",
    "                        nn.Conv2d(128,256,3,padding=1),                         # batch x 64 x 7 x 7\n",
    "                        nn.ReLU()\n",
    "        )\n",
    "        \n",
    "                \n",
    "    def forward(self,x):\n",
    "        out = self.layer1(x)\n",
    "        out = self.layer2(out)\n",
    "        out = out.view(batch_size, -1)\n",
    "        return out\n",
    "    \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "class Decoder(nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Decoder,self).__init__()\n",
    "        self.layer1 = nn.Sequential(\n",
    "                        nn.ConvTranspose2d(256,128,3,2,1,1),                    # batch x 128 x 14 x 14\n",
    "                        nn.ReLU(),\n",
    "                        nn.BatchNorm2d(128),\n",
    "                        nn.ConvTranspose2d(128,64,3,1,1),                       # batch x 64 x 14 x 14\n",
    "                        nn.ReLU(),\n",
    "                        nn.BatchNorm2d(64)\n",
    "        )\n",
    "        self.layer2 = nn.Sequential(\n",
    "                        nn.ConvTranspose2d(64,16,3,1,1),                        # batch x 16 x 14 x 14\n",
    "                        nn.ReLU(),\n",
    "                        nn.BatchNorm2d(16),\n",
    "                        nn.ConvTranspose2d(16,1,3,2,1,1),                       # batch x 1 x 28 x 28\n",
    "                        nn.ReLU()\n",
    "        )\n",
    "        \n",
    "    def forward(self,x):\n",
    "        out = x.view(batch_size,256,7,7)\n",
    "        out = self.layer1(out)\n",
    "        out = self.layer2(out)\n",
    "        return out\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 2) Loss func & Optimizer\n",
    "\n",
    "Loss Function에서 중요한 점은 Encoder와 Decoder의 Parameter를 동시에 학습시키기 위해 묶어서 Optimizer에적용시켜야 한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "cuda:0\n"
     ]
    }
   ],
   "source": [
    "device = torch.device(\"cuda:0\" if torch.cuda.is_available() else \"cpu\")\n",
    "print(device)\n",
    "\n",
    "encoder = Encoder().to(device)\n",
    "decoder = Decoder().to(device)\n",
    "\n",
    "# 인코더 디코더의 파라미터를 동시에 학습시키기 위해 이를 묶는 방법입니다.\n",
    "parameters = list(encoder.parameters())+ list(decoder.parameters())\n",
    "\n",
    "loss_func = nn.MSELoss()\n",
    "optimizer = torch.optim.Adam(parameters, lr=learning_rate)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 3) Train "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "--------model not restored--------\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 모델을 불러오는 방법입니다.\n",
    "# 크게 두가지 방법이 있는데 여기 사용된 방법은 좀 단순한 방법입니다.\n",
    "# https://pytorch.org/tutorials/beginner/saving_loading_models.html\n",
    "\n",
    "try:\n",
    "    encoder, decoder = torch.load('./model/conv_autoencoder.pkl')\n",
    "    print(\"\\n--------model restored--------\\n\")\n",
    "except:\n",
    "    print(\"\\n--------model not restored--------\\n\")\n",
    "    pass\n",
    "\n",
    "for i in range(num_epoch):\n",
    "    for j,[image,label] in enumerate(train_loader):\n",
    "        optimizer.zero_grad()\n",
    "        image = image.to(device)\n",
    "        \n",
    "        output = encoder(image)\n",
    "        output = decoder(output)\n",
    "        \n",
    "        loss = loss_func(output,image)\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        \n",
    "    if j % 10 == 0:\n",
    "        # 모델 저장하는 방법\n",
    "        # 이 역시 크게 두가지 방법이 있는데 여기 사용된 방법은 좀 단순한 방법입니다.\n",
    "        # https://pytorch.org/tutorials/beginner/saving_loading_models.html\n",
    "        torch.save([encoder,decoder],'./model/conv_autoencoder.pkl')\n",
    "        print(loss)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 4) Check with Train Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([256, 28, 28])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAC4CAYAAAD61bdSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAUGUlEQVR4nO3de2zVZZoH8O9DaSmIgIiWitwpRCAC0ii4EjEjCnW4mHj9w0g0y/wxyBCNG4LGwcskJu7O7v6xXmpEWDO2g+JIQdEVHEXDptIh3qA73EWgUBDkKpfCs3/04FTe56Wn55zf6Xl/5/tJDO3Dc/p7f+3L4+nvvYmqgoiIwtOhvRtARESpYQEnIgoUCzgRUaBYwImIAsUCTkQUKBZwIqJApVXARWSyiPxdRLaIyLxMNYqovbFvUwgk1XngIlIAYBOASQB2AVgH4H5V3XiR13DSOUVKVSXdr5HrfVvEvkWu6QhPW36WVt/umMa1rwewRVW3JRpSDWA6AG8nJwpETvftwsJCM3769Okst4TSVVRUZMZPnTqV1OvTeYTSB8D3LT7flYj9gojMEpE6EalL41pE2cS+TUFI5x14UlS1EkAlwEcoFC/s29Te0inguwH0bfH51YkYUehyum/zUUl8JPuoxCedRyjrAJSJyEARKQJwH4CatFpDlBvYtykIKb8DV9UmEZkN4EMABQAWquqGjLWMqJ2wb1MoUp5GmNLF+JyQIpaJaYSpYN+mqFl9mysxiYgCxQJORBSoyKcRElH+6N69uxM7fPhwO7QkP/AdOBFRoFjAiYgCxQJORBQoFnAiokBxEDNCy5cvN+MVFRVJvX7Pnj1mfPLkyU5swwauM6FojB492ondd999Zm63bt2cWO/evZ1YY2Oj+foVK1Y4sQ8++MDMbWpqMuP5hO/AiYgCxQJORBQoFnAiokCxgBMRBYoFnIgoUJyFchEdO7rfnqeeesrMnTZtmhPr37+/mZvsDpClpaVmfOXKlU6ssrLSzH3uueeSuhbll06dOjmxqVOnmrm33nqrEysvLzdzDx06lNS1RowYYb7+p59+cmK1tbVm7v79+814PuE7cCKiQLGAExEFigWciChQLOBERIFKaxBTRHYAOArgLIAmVbVHNgI1e/ZsJzZ//vysXX/nzp1m3GrDli1bom5OXol7354+fboTe+SRR8zcc+fOObH6+nozt7i42IlZS/E//fRT8/W+JfZky8QslFtU9UAGvg5RrmHfppzGRyhERIFKt4ArgP8Rkb+JyKxMNIgoR7BvU85L9xHKTaq6W0SuBPCRiPyfqq5pmZDo/PwHQKFh36acl9Y7cFXdnfizEcBfAFxv5FSqanncBoEo3ti3KQQpvwMXkUsAdFDVo4mPbwPwTMZalkV9+/Y14w888ECWW/JLS5YsMePV1dVZbkl+yVTfLiwsNONnzpxJr4Ft0KtXLzN+5513OrFBgwaZud99950TGzJkiJlrLW9///33nZjvsBNrdko2v1+hSecRSgmAv4jI+a/zpqraR2cQhYV9m4KQcgFX1W0ARmWwLUQ5gX2bQsFphEREgWIBJyIKFPcDh3/f7lGjsvdb9MiRI53Y9u3bs3Z9wB6Y2r17t5lr7dtMv5QLg2/W0nbAXh5/+PBhM7d79+5OzNorH7D3uq+qqnJiq1atMl8flUsuucSJJcY4HMeOHYu6ORnDd+BERIFiASciChQLOBFRoFjAiYgCxQJORBQozkIB0LNnz/ZuAvbs2ePETp06Fcm1Hn/8cTM+c+ZMJ2a1C7BnodTU1Ji57777rhM7cIDbbGfD1VdfbcatWSQdOtjv56x+6JuFtHXrVif28ccfX6yJGXXLLbeY8WnTpjkx3zYDa9eudWLr1q0zc+vq6trQuuRdOEPGmt0D8B04EVGwWMCJiALFAk5EFCgWcCKiQHEQE8Azz2R3G/OVK1c6sagGLOfOnevEfPucDxs2LKmYT0VFhRmfM2eOE5s6daqZa+09Tam77bbbzLi1PH7v3r1mbu/evZ3YyZMnzdydO3c6sXS3XfDt13/77bc7seuvd87dAABMnDjRiR0/ftzMveaaa5zY2LFjzVzr3t566y0zd82aNWbc0qVLl1avA/AdOBFRsFjAiYgCxQJORBQoFnAiokCxgBMRBarVWSgishDArwE0qurIRKwngD8DGABgB4B7VPVQdM3MnKFDhzqxHj16RHIt3yj3K6+84sSsWSi+dk2ZMsWJvfTSS2Zu586dnVhBQYGZG5Xhw4c7serqajN3/PjxUTfnZ3Hr25MmTXJi5eXlZu6ll17qxAoLC83cs2fPOrFNmzaZudZMC+swBd/2FdY9WLNgAHtmyGWXXWbmnjhxwolt27bNzC0qKnJiJSUlZq51MMZdd91l5lqHo3z//fdmrq92XCiZd+CLAEy+IDYPwGpVLQOwOvE5UWgWgX2bAtZqAVfVNQAOXhCeDmBx4uPFAGZkuF1EkWPfptClupCnRFUbEh/vBWD/fgFARGYBmJXidYiyjX2bgpH2SkxVVRGx9zps/vtKAJUAcLE8olzDvk25LtUCvk9ESlW1QURKATRmslFReu+995yYb8/kdP3www9mfMWKFUm9fty4cWb8jTfecGK+E7Z9+wi3N2tgEwBmzHCfWFj7iUco2L5t7YXt69tnzpxxYkeOHDFzrWXcvv2xk90Kwbfk/d5773ViV155pZl74XJzwB5wBYBOnTo5MWs7AcAe8PQNYlqTD3zbDHTt2tWJnT592sxNVqrTCGsAPJj4+EEAy9JqBVHuYN+mYLRawEWkCsD/AhgmIrtE5GEAzwOYJCKbAdya+JwoKOzbFLpWH6Go6v2ev/pVhttClFXs2xQ6rsQkIgoUCzgRUaDy7kCHQYMGObGoZmpUVVUlnWsteX711Vcz2ZxWffjhh07MN1vAt2Q5WdbyasA/2k//cPnll5vxIUOGODFr9gVgz1ryHZywfv16J+ZbAn7w4IXrooAxY8Y4Md/p8VZ7y8rKzNzi4mIntnHjRjN3x44dTmzXrl1m7sCBA52Ybzm/NZvHN/vM+rpfffWVmctT6YmIYo4FnIgoUCzgRESBYgEnIgpU3g1idujg/j/L2tM3Ez7//POkc1etWuXErD2bfaz7Aux7W758uZlrLWO/4447zNyampqk29YWvi0B6B98/dVa1m0NKvpYg4KAvTe1ta8+ADQ1NTkxayD8iiuuMF9/1VVXObE9e/aYuWvXrnViq1evNnMPHDjgxHz7n1t9ftiwYWau9e+utLTUzD127JgZtyQ7sYLvwImIAsUCTkQUKBZwIqJAsYATEQUq7wYxrQGgbO+ZPXfuXCdmHT7clnb5VpU99NBDTsy3l3NbRPU9y9X9y3PJjz/+aMatfbt9rBWP1qpCwB6Y9A1C9uvXz4lNnDjRifl+zvv27XNiX3/9tZm7bJm70+/KlSvNXOt+J0++8DjUZg0NDU7syy+/NHOtFcXW6wH/6tV08B04EVGgWMCJiALFAk5EFCgWcCKiQLGAExEFqtVZKCKyEMCvATSq6shEbAGAfwawP5E2X1Xfj6qRofLtr2zNDCkoKEjrWr6l7R999FFaX9eaHRMXofZt3wwOayl8jx49kv66vtkt1kyLxsZGM3fEiBFObP/+/U7M1y5rK4X6+noz19q/3sfaZsDaIsDXNt8e7EeOHHFi1kwcAOjYMfOT/pJ5B74IgDXf5t9VdXTiv5zq4ERJWgT2bQpYqwVcVdcASH5HHKJAsG9T6NJ5Bj5bRL4WkYUi4j1fS0RmiUidiNSlcS2ibGLfpiCkWsBfAjAYwGgADQD+zZeoqpWqWq6q7qGPRLmHfZuCkdJTdVX9eb2riLwKYEXGWhQjL774Yns3IW0vvPBCJF/XNwjmW7KcLSH3bWuw0bdPfFFRkRPz7T9vHeh7+vRpM7d79+5OzBrM9+1pbrV3+/btZm5b9vEfO3asE5syZYqZO2HCBCfmG+C1DpLetGmTmXvo0CEnZu1/DriDoydOnDDzUnoHLiItdyy/E8C3qXwdolzDvk0hSWYaYRWAiQB6icguAL8HMFFERgNQADsA/CbCNhJFgn2bQtdqAVfV+43waxG0hSir2LcpdFyJSUQUKBZwIqJA5d2BDnHlW75rnbA9c+ZMM7e83J0N5zthO12+08Nra2sjuV4+6NWrlxM7efKkmWvN4PAtFz969KgTs2axAEC3bt2cmHUau3XAAgDs2LHDiQ0ePNjMnTNnjhOzZuIAQFlZmRMbNWqUmWvdm+8AC2uGzDvvvGPmHjzorhnz/XySxXfgRESBYgEnIgoUCzgRUaBYwImIApV3g5g7d+50YiUlJWaub6AmF1VUVJhx33LhbPrkk0+cmDUARen54osvnNiNN95o5lpLw31L061l3L4T7K19t61/R2fPnjVfP3ToUCfmW+JvLef37ZVuDZr67sHaDsC3lN0axHzzzTfN3HQHLC18B05EFCgWcCKiQLGAExEFigWciChQLOBERIHKu1koAwcOdGJPP/20mfvEE09E3Zy88PLLLzsxa3N7So/1fbaWsQPA+PHjnZh1wjoAdOnSxYlde+21SedaM1N8rBlhXbt2NXMbGhqcWGFhoZlrLWO32urL3bdvn5EJLFu2zIn5vudR4DtwIqJAsYATEQWKBZyIKFAs4EREgRLf0tOfE0T6AvhvACVoPiewUlX/U0R6AvgzgAFoPjvwHlW96MiUiFz8Yu1k2LBhZnzBggVO7O677464NakRETPe2s83VcePH3diDz/8sJm7dOlSJxZVu1TV/kYY8qFv+wYAH3vsMSd28803m7nFxcVOzFpuDtgDodbP2tdfm5qakro+YC/H9w2YHj582Il17tzZzLWWzVdXV5u5VVVVZjwKVt9O5h14E4DHVHU4gHEAfisiwwHMA7BaVcsArE58ThQS9m0KWqsFXFUbVHV94uOjAOoB9AEwHcDiRNpiADOiaiRRFNi3KXRtmgcuIgMAjAFQC6BEVc9PxNyL5l9DrdfMAjAr9SYSRY99m0KU9CCmiHQFsBTAXFX9xYMubX7IZT4DVNVKVS1XVffARaIcwL5NoUqqgItIIZo7+J9U9fyJnftEpDTx96UAGqNpIlF02LcpZMnMQhE0Pwc8qKpzW8RfAPCDqj4vIvMA9FTVf2nla+XkSL1Px47uE6YbbrjBzH322Wed2JgxY8xc3wb16WjLLBTfZvobN250Yk8++aSZay0trquru1gTs6KNs1Dytm9bp6xfd911Zu6kSZOc2KBBg8zcgoICJ+abCWOx+qt10j1gz1ixltcDwKZNm5zY1q1bzdy3337biVkHYGSb1beTeQb+TwAeAPCNiHyZiM0H8DyAJSLyMIDvANyTqYYSZQn7NgWt1QKuqp8D8L2r+VVmm0OUPezbFDquxCQiChQLOBFRoFodxMzoxQIb6EmX70T4fv36JfX6Rx991IwPHjzYiX322WdmrrUE2DrNGwBef/31pNqVy9oyiJlJ+da3+/TpY8YnTJjgxKzBUd/AZP/+/Z1YfX29mbt582Yntn79ejO3trbWjIck1aX0RESUg1jAiYgCxQJORBQoFnAiokCxgBMRBYqzUChWOAslbNaWENmsUbmMs1CIiGKEBZyIKFAs4EREgWIBJyIKVJuOVCMiihIHLNuG78CJiALFAk5EFCgWcCKiQLGAExEFqtUCLiJ9ReSvIrJRRDaIyO8S8QUisltEvkz8VxF9c4kyh32bQpfMqfSlAEpVdb2IXArgbwBmoPmg12Oq+q9JX4zLjSlibTyVnn2bgpHSqfSq2gCgIfHxURGpB2Afx0EUEPZtCl2bnoGLyAAAYwCcP59otoh8LSILReQyz2tmiUidiNSl1VKiCLFvU4iS3o1QRLoC+BTAH1T1HREpAXAAgAJ4Fs2/ij7Uytfgr5kUqVR2I2TfphBYfTupAi4ihQBWAPhQVf9o/P0AACtUdWQrX4ednCLV1gLOvk2hSOkZuDRv0PsagPqWHVxEShPPEAHgTgDfZqqhRNmQz327Qwf36em5c+ciuVZxcbETO3nyZCTXyjfJzEK5CcBnAL4BcP4nPB/A/QBGo/nXzB0AftOi0/u+Ft+lUKTaOAslb/s2C3h4Un6EkimhdXIKD0/kSQ4LeHh4Ig8RUYywgBMRBYoFnIgoUHwGTrHCZ+AUV3wGTkQUIyzgRESBYgEnIgoUCzgRUaCyfSr9AQDfJT7ulfg8bnhf7ad/O177fN8O4fuUqrjeWwj3ZfbtrM5C+cWFRepUtbxdLh4h3ld+i/P3Ka73FvJ98REKEVGgWMCJiALVngW8sh2vHSXeV36L8/cprvcW7H212zNwIiJKDx+hEBEFigWciChQWS/gIjJZRP4uIltEZF62r59JiRPLG0Xk2xaxniLykYhsTvxpnmiey0Skr4j8VUQ2isgGEfldIh78vUUpLn2b/Tqce8tqAReRAgD/BWAKgOEA7heR4dlsQ4YtAjD5gtg8AKtVtQzA6sTnoWkC8JiqDgcwDsBvEz+nONxbJGLWtxeB/ToI2X4Hfj2ALaq6TVVPA6gGMD3LbcgYVV0D4OAF4ekAFic+XgxgRlYblQGq2qCq6xMfHwVQD6APYnBvEYpN32a/Dufesl3A+wD4vsXnuxKxOClpcQDuXgAl7dmYdInIAABjANQiZveWYXHv27H62celX3MQM0LaPEcz2HmaItIVwFIAc1X1SMu/C/3eKHWh/+zj1K+zXcB3A+jb4vOrE7E42ScipQCQ+LOxnduTEhEpRHMn/5OqvpMIx+LeIhL3vh2Ln33c+nW2C/g6AGUiMlBEigDcB6Amy22IWg2ABxMfPwhgWTu2JSUiIgBeA1Cvqn9s8VfB31uE4t63g//Zx7FfZ30lpohUAPgPAAUAFqrqH7LagAwSkSoAE9G8HeU+AL8H8C6AJQD6oXl70XtU9cIBoZwmIjcB+AzANwDOJcLz0fy8MOh7i1Jc+jb7dTj3xqX0RESB4iAmEVGgWMCJiALFAk5EFCgWcCKiQLGAExEFigWciChQLOBERIH6f/Kz4QGMvlr0AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAC4CAYAAAD61bdSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAPW0lEQVR4nO3df4wUdZrH8c/DD0GZi0oIOM7isTn1CEFFJUg4SDCeZm7dRI2R6B8rRiL+sUY22UQN/rHEc40hd95hcrk4RgKXqLubwC4GzK0LIScmooLBRWHnUAEXBEaEREQUkef+mOIyUt92mu6u7n5q3q+ETPfT3+56avrhSU3Vt6rM3QUAiGdYqxMAANSGBg4AQdHAASAoGjgABEUDB4CgaOAAEFRdDdzMus2s18w+NLPHGpUU0GrUNiKwWueBm9lwSf8r6WZJ+yS9I+ked9/xA+9h0jkK5e5W72dQ22hHqdquZwt8hqQP3f1jdz8p6TeSbqvj84B2QW0jhHoaeJekvw54vi+LfY+ZLTSzLWa2pY5lAc1EbSOEEUUvwN17JPVI/JmJcqG20Wr1bIHvlzRxwPMfZTEgOmobIdTTwN+RdIWZ/djMzpN0t6RXGpMW0FLUNkKoeReKu58ys4ck/VHScEnL3f2DhmUGtAi1jShqnkZY08LYT4iCNWIaYS2obRSt0dMIAQAtRAMHgKAKn0YIoHbDhw/Pxb777rsWZIJ6jRiRb7enTp2q6zPZAgeAoGjgABAUDRwAgqKBA0BQzANvE7Nnz87FNm3alBzb0dGRix0/frzhOUXEPPD2c/XVV+di3d3dybFLly4tOp2wmAcOACVCAweAoGjgABAUDRwAgqKBA0BQnErfJiZPnpyLNXOGEFCUGTNm5GLz5s1Ljn3ppZdysX379jU8p7JgCxwAgqKBA0BQNHAACIoGDgBB1XUQ08z2SDom6TtJp9x9eiOSGoruvPPOVqeAAajtxunq6srFRo0alRz7xRdfFJ1OqTRiFsqN7n64AZ8DtBtqG22NXSgAEFS9DdwlvWZmW81sYSMSAtoEtY22V+8ulNnuvt/Mxkv6k5n9xd1fHzggK37+AyAaahttr64tcHffn/3sk/R7SblTrty9x92ncxAIkVDbiKDmLXAzGyNpmLsfyx7fIumJhmU2xFQ6Ko/mo7Yba9y4cbnYwYMHk2NPnDhRdDqlUs8ulAmSfm9mZz7nJXf/74ZkBbQWtY0Qam7g7v6xpGsamAvQFqhtRME0QgAIigYOAEFxPfAWuOCCC3Kx6dPzExnefvvt5Ps50INIRo8enYtt3749Ofbbb78tOp1SYQscAIKigQNAUDRwAAiKBg4AQdHAASAoZqG0wP3335+LdXR05GKrVq1Kvv/06dMNzwnNM2xYfrupLN9p6m7zF154YS723nvvNSOd0mMLHACCooEDQFA0cAAIigYOAEFxELMFbrzxxlzs6NGjudjKlSubkQ6arCwHLFMmTZqUix0+nL8v9PHjx5PvHzt2bC525MiRuvMqK7bAASAoGjgABEUDB4CgaOAAEBQNHACCGnQWipktl/RTSX3uPjWLjZX0W0mTJO2RNM/d89MohrjUKcSSdMstt+Ri69evz8X6+vqS7+/q6srFnn/++eTYp556Khd74403kmOHGmq7dtdff30ynppF0tvbm4vt3r07+f6pU6fmYhMnTkyOXb16dS421G52Us0W+ApJ3WfFHpO0wd2vkLQhew5Es0LUNgIbtIG7++uSzp6IeZukM5OUV0q6vcF5AYWjthFdrSfyTHD3A9njg5ImVBpoZgslLaxxOUCzUdsIo+4zMd3dzcx/4PUeST2S9EPjgHZDbaPd1drAD5lZp7sfMLNOSemjbUNI6hrPy5YtS45N3ZX+2WefrXpZqVPxu7vP3pXbzz3fV2699daqlzUEUdtVuPnmm5Pxb775JhdLHTT/9NNPk++/6aabql5W6mD+0qVLk2PLqtZphK9Imp89ni9pTWPSAVqO2kYYgzZwM3tZ0puS/t7M9pnZAklPS7rZzHZJ+sfsORAKtY3oBt2F4u73VHgp/7cOEAi1jeg4ExMAgqKBA0BQ3NDhHKVmm0jSokWLcrF77703Ofa5557LxTZu3FhfYhWMGMFXjOpceeWVyfhdd92Vi02ePDk59sUXX8zFtm7dmoulbvwgSRdffHEudskllyTHnnfeecn4UMIWOAAERQMHgKBo4AAQFA0cAILiCNc5WrMmfWJe6vT0tWvXJsc+/vjjdeUwd+7cqsfOmjUrF6t08OfkyZO1poSCjB49Ohf7+uuvC1nW3XffnYynanvz5s3JsdVea37Pnj3JeOp64hMmpK8nljptf6jVNlvgABAUDRwAgqKBA0BQNHAACIqDmOeos7Oz6rGVDugcOXL2XbzOzZw5c6oeO2bMmFys0k1iP/roo5pzQjGKOmCZUukm3Klryu/atSs59ssvv6xqWTfccEMyPn78+Fys0gHI1P/FSmdtfvLJJ1XlFQ1b4AAQFA0cAIKigQNAUDRwAAiKBg4AQQ06C8XMlkv6qaQ+d5+axZZIekDSZ9mwxe7+alFJtpPFixcn46tWrcrFHnzwweTY7du352KpU5Mffvjh5PtT11I+dOhQcmzqNORp06Ylxw61WSjU9ve9+mp6NTs6OnKx6667Ljn2vvvuy8X27t2bi1W6HMSll16ai5lZcuzRo0dzscsuuyw5dijPQlkhqTsR/zd3n5b9GxIFjtJZIWobgQ3awN39dUn1TVwG2hC1jejq2Qf+kJn92cyWm1n+PkgZM1toZlvMbEsdywKaidpGCLU28P+U9HeSpkk6IOlfKw109x53n+7u02tcFtBM1DbCqOlUenf//yNmZva8pPSFr0votddeS8bvuOOOXKynpyc5dt26dbnY4cOHc7Fx48Yl35+6Jvmbb76ZHPv0008n40gbyrW9YcOGZPzzzz/PxZYsWZIcu2DBglwsdQCx0iUCent7c7Hdu3cnx1500UW52KlTp5Jjy6qmLXAzG3gRgjskvd+YdIDWorYRSTXTCF+WNFfSODPbJ+lXkuaa2TRJLmmPpPR8OaCNUduIbtAG7u73JMIvFJAL0FTUNqLjTEwACIoGDgBBcUOHBlm/fn0uNnPmzOTYJ598MhcbOXJkLrZjx47k+5ctW5aLPfLII4OlCNRk27Ztudijjz6aHDt9enUzKivNFhk2LL9NWWk21uzZs3OxAwcOVLX8smALHACCooEDQFA0cAAIigYOAEFxELNAfX19yfjChQsbvqxrrrmm6rHHjh1r+PIxtKROef+heLXGjBmTiz3xxBPJseeff34uVuna4WXFFjgABEUDB4CgaOAAEBQNHACCooEDQFDMQgkodfR91qxZVb8/dYF+oB189dVXuVil0/NTl5+YOnVqcuyePXvqyqtdsQUOAEHRwAEgKBo4AARFAweAoKq5J+ZESf8laYL67xPY4+7LzGyspN9KmqT+ewfOc/ejxaWKM0aNGpWLdXZ2JkamrVixIhm/6qqrak0pJGq7/aROhR8/fnxybFdXVy5W6SDm2rVr60usTVWzBX5K0i/dfYqkmZJ+bmZTJD0maYO7XyFpQ/YciITaRmiDNnB3P+Du72aPj0naKalL0m2SVmbDVkq6vagkgSJQ24junOaBm9kkSddKekvSBHc/c/+ig+r/MzT1noWSGn/5PaCBqG1EVPVBTDPrkLRK0i/c/YuBr7m7q38fYo6797j7dHev7mZ5QJNR24iqqgZuZiPVX+AvuvvqLHzIzDqz1zslpS9+DbQxahuRVTMLxSS9IGmnuz8z4KVXJM2X9HT2c00hGaLhUqfiD0XUdvs5ffp0LvbZZ58lx06ePDkXu/zyyxueUzurZh/4P0j6maTtZrYtiy1Wf3H/zswWSNoraV4xKQKFobYR2qAN3N3fkFTpPkU3NTYdoHmobUTHmZgAEBQNHACC4nrgAZ04cSIXW716dWKkNGfOnFzsgQceaHhOQFE2b96cjPf29uZi69atKzqdtsIWOAAERQMHgKBo4AAQFA0cAIKigQNAUNZ/rZ4mLcyseQvDkOTulU7MKRS1jaKlapstcAAIigYOAEHRwAEgKBo4AARFAweAoGjgABAUDRwAgqKBA0BQNHAACGrQBm5mE81so5ntMLMPzGxRFl9iZvvNbFv27yfFpws0DrWN6AY9ld7MOiV1uvu7ZvY3krZKul39N3r90t3/peqFcboxCnYup9JT24gkVdvV3NT4gKQD2eNjZrZTUlfj0wOai9pGdOe0D9zMJkm6VtJbWeghM/uzmS03s4srvGehmW0xsy11ZQoUiNpGRFVfjdDMOiT9j6Rfu/tqM5sg6bAkl/TP6v9T9P5BPoM/M1GoWq5GSG0jglRtV9XAzWykpLWS/ujuzyRenyRprbtPHeRzKHIU6lwbOLWNKGq6nKyZmaQXJO0cWODZAaAz7pD0fiOSBJqF2kZ01cxCmS1pk6Ttkk5n4cWS7pE0Tf1/Zu6R9GB2UOiHPoutFBTqHGehUNsIo+ZdKI1CkaNo3JEHZcUdeQCgRGjgABAUDRwAgqKBA0BQNHAACIoGDgBB0cABICgaOAAENejlZBvssKS92eNx2fOyYb1a529buOwztR3h91Srsq5bhPVK1nZTz8T83oLNtrj79JYsvECs19BW5t9TWdct8nqxCwUAgqKBA0BQrWzgPS1cdpFYr6GtzL+nsq5b2PVq2T5wAEB92IUCAEHRwAEgqKY3cDPrNrNeM/vQzB5r9vIbKbtjeZ+ZvT8gNtbM/mRmu7KfyTuatzMzm2hmG81sh5l9YGaLsnj4dStSWWqbuo6zbk1t4GY2XNJ/SPonSVMk3WNmU5qZQ4OtkNR9VuwxSRvc/QpJG7Ln0ZyS9Et3nyJppqSfZ99TGdatECWr7RWirkNo9hb4DEkfuvvH7n5S0m8k3dbkHBrG3V+XdOSs8G2SVmaPV0q6valJNYC7H3D3d7PHxyTtlNSlEqxbgUpT29R1nHVrdgPvkvTXAc/3ZbEymTDgBrgHJU1oZTL1MrNJkq6V9JZKtm4NVvbaLtV3X5a65iBmgbx/jmbYeZpm1iFplaRfuPsXA1+Lvm6oXfTvvkx13ewGvl/SxAHPf5TFyuSQmXVKUvazr8X51MTMRqq/yF9099VZuBTrVpCy13Ypvvuy1XWzG/g7kq4wsx+b2XmS7pb0SpNzKNorkuZnj+dLWtPCXGpiZibpBUk73f2ZAS+FX7cClb22w3/3Zazrpp+JaWY/kfTvkoZLWu7uv25qAg1kZi9Lmqv+y1EekvQrSX+Q9DtJl6n/8qLz3P3sA0JtzcxmS9okabuk01l4sfr3F4ZetyKVpbap6zjrxqn0ABAUBzEBICgaOAAERQMHgKBo4AAQFA0cAIKigQNAUDRwAAjq/wCz1eBUw2GNOgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAC4CAYAAAD61bdSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAARFklEQVR4nO3dfYxUVZrH8d/TiK+YIKCk44iMbyiiKxHJmgBC2FEc3yAahSgw7rqMUSOLm7i8qDNZA06iy64mZhUCAnF0Zgy+gG4UY4iAjggCKoI4BpVBEERcdSXaCs/+0cWk5Zyiq6vurapTfD9Jh6qnT1Wd2/34ePuec+4xdxcAID1Nte4AAKA8FHAASBQFHAASRQEHgERRwAEgURRwAEhURQXczEaY2SYz+9DMJmfVKaDWyG2kwMqdB25mnSR9IOkXkrZKWiVpjLtvOMhrmHSOXLm7Vfoe5DaqxSyerrG6HMvtSs7AB0r60N03u3uLpD9IuqqC9wPqBbmNzJlZ8HXYYYdFv0pVSQE/UdJf2zzfWogd2OkJZrbazFZX8FlANZHbSELppb5M7j5L0iyJPzPRWMht1FolBfxTSSe1ef6zQgxIHbmNzMUujfzwww8VvWcll1BWSTrdzH5uZodLGi1pUUW9AeoDuY0klH0G7u4/mtltkl6S1EnSXHd/L7OeATVCbiMVZU8jLOvDuE6InGUxjbAc5Dba07lz5yDWkUsoWU8jBADUEAUcABKV+zRCAOXr1KlTENu7d28NeoJSNTXFz4srnXES/azM3xEAUBUUcABIFAUcABJFAQeARDGICdQxBizr2+GHHx7EWlpaqvb5nIEDQKIo4ACQKAo4ACSKAg4AiaKAA0CimIVSA9OmTSup3fTp03PuCZCtu+66K4gtXbo0iL322mvV6E7uqjnjJIYzcABIFAUcABJFAQeARFHAASBRFQ1imtnHkr6RtFfSj+4+IItONYohQ4ZE4zfddFMQW758ed7dQQeQ2wc3bNiwaPyKK64IYmeccUYQa5RBzFrLYhbKMHfflcH7APWG3EZd4xIKACSq0gLukpaY2VtmNiGLDgF1gtxG3av0Esogd//UzE6Q9LKZve/uy9o2KCQ//wEgNeQ26l5FZ+Du/mnh352SnpE0MNJmlrsPYBAIKSG3kYKyz8DN7BhJTe7+TeHxxZL+PbOeJWbUqFFBrNhS+E8++SSIzZgxI/M+oTzk9k8NHz48iE2cODHadu3atUGM3M5PJZdQekp6xsz2v88T7v5iJr0CaovcRhLKLuDuvlnS32XYF6AukNtIBdMIASBRFHAASBT3A++g2GClJC1YsCCIvfTSS9G2d9xxRxDbsmVLZR0DKjR27Nho/PLLLw9imzZtirZduHBhECO388MZOAAkigIOAImigANAoijgAJAoCjgAJMrcvXofZla9D8vAmWeeGcQ2bNgQbRtbNn/33Xdn3iccnLtbLT43tdy+5pprgtiECfH7cq1YsSKILVmyJNr2jTfeqKxjKCqW25yBA0CiKOAAkCgKOAAkigIOAIliEFPxwUpJWrVqVRArtjx+3LhxQWzPnj2VdQwdxiDmT916663R+OTJk4PYRx99FG07ZMiQTPuE8jCICQANhAIOAImigANAoijgAJAoCjgAJKrdWShmNlfS5ZJ2unu/QqybpD9K6i3pY0nXuvuX7X5YHYzUH3/88UHs1Vdfjbbt3r17EOvZs2fmfWoUxWbzHHPMMUHsrLPOirZ9/PHHK+pDR2ahNFpu9+rVK4g9+uij0bZHHXVUEIvNTJFYHi9JvXv3jsb79+9f8nusXLkyiG3btq3k15c7C2WepBEHxCZLesXdT5f0SuE5kJp5IreRsHYLuLsvk7T7gPBVkuYXHs+XNDLjfgG5I7eRunL3xOzp7tsLjz+TVPS6gplNkBS/zRlQf8htJKPiTY3d3Q92/c/dZ0maJdXHdUKgVOQ26l25BXyHmTW7+3Yza5a0M8tO5WnKlClBLDZYKUmXXnpp3t2pK7EB3lGjRkXbxu4d3adPn2jbo48+OogVGzyvdBAzA8nm9i233BLEunbtGm07derUINbIg5WnnXZaELv66qujbZubm4PYqaeeGm0bqxEffPBBtO3YsWODWEcGMWPKnUa4SNL4wuPxkp6rqBdA/SC3kYx2C7iZPSnpz5L6mNlWM/snSb+T9Asz+4ukfyg8B5JCbiN17V5CcfcxRb41POO+AFVFbiN1rMQEgERRwAEgURVPI6xXI0YcuMCu1cSJE4PYPffcE227Zs2aTPtUCyeffHIQmzlzZrRtbMbJ559/Hm0b+5ktXLgw2jY2+l5s2TbaF8thSRo2bFgQW716dbTt0qVLM+1TLcSONzYTR5IuuOCCILZ48eJo261btwaxL7+M300htsR+x44d0bZm2e81whk4ACSKAg4AiaKAA0CiKOAAkKiG3ZW+2D2+u3XrFsTOOeecvLuTu2nTpkXjt99+exArduuA++67L4jNnj072nbLli1BrNiy+0ceeSSIzZgxI9r2wQcfjMZLdSjsSr9kyZJo/Nhjjw1iF154Yd7dyV3stg2SdOeddwaxdevWRdu+8847QazYJIXduw+8QWV8Kb4kXXLJJUHsqaeeirZ99tlno/FSsSs9ADQQCjgAJIoCDgCJooADQKIadiXm4MGDo/Fim7ym5N577w1isfs7S/GBmmL3QV6xYkXJfZg0aVIQK7a68uabbw5izzzzTMmfdSjr0aNHEDv33HOjbYsN3NejYpsE33jjjUGs2H3mX3jhhSD22GOPRdvGVhQPGjQo2jY2GFysv/PmzQtiL7/8crRtHjgDB4BEUcABIFEUcABIFAUcABJFAQeARLU7C8XM5kq6XNJOd+9XiP1W0j9L2j+0O9Xd/yevTpajmrcIyEJsR/gFCxZE21588cVBbPr06dG2Dz30UBDbtWtXyX2YMmVKtG3sntQ33HBDtG29zjhJIbdjv6tvv/022ja2XLxejRs3LhofOnRoEFu0aFG07cMPPxzEit1zO5bbxfYM6Nq1axArNrulmjNOYko5A58nKXak/+nu5xW+6qp4AyWaJ3IbCWu3gLv7Mknh3V2AxJHbSF0l18BvM7N3zGyumR1XrJGZTTCz1WYW39sJqD/kNpJQbgH/b0mnSjpP0nZJ/1GsobvPcvcB7j6gzM8CqoncRjLKWkrv7n/btdPMZkt6PrMeZaTYsvDYYEY96NWrVxCLDVZK8Q2Fiw1ixjY1LrYpbuye4hs3boy2Pfvss4PY+++/H22bkhRy+/XXX4/GL7rooiD2/PPx7r/99tuZ9ulgTjjhhCB25ZVXRtuuX78+iG3bti3aNnav82K3GbjuuuuC2Pfffx9tG7snfbGB1For6wzczJrbPB0lKfypAwkit5GSUqYRPilpqKQeZrZV0m8kDTWz8yS5pI8l/TrHPgK5ILeRunYLuLuPiYTn5NAXoKrIbaSOlZgAkCgKOAAkqmE3dCi2fPuBBx4IYsV2U6/1EvBitwOI3Zx+5syZ0bbXX399ENu5c2e0bWwmS2yneknas2dPNI78LV68OBqPbZwR2zVdim9a0JENPYo54ogjgthll10WxE455ZTo67/44osgVmzGyvnnnx/ENm/eHG17//33B7FiM6xSmk3FGTgAJIoCDgCJooADQKIo4ACQKKvmfbPNrOY36Y4tFy+2tHzZsmVBLDbIIsUHHIvdmzjWNrbEf+TIkdHXNzWF/9/dsGFDtO0TTzwRxIotu28E7h7/oeesHnJ70qRJQWzgwIHRtt27dw9ixZas79u3L4jt3bs32jZ2r/Lhw4cHsX79+kVfv3bt2iC2bt26aNuvv/46iMXyXZLefPPNaDwlsdzmDBwAEkUBB4BEUcABIFEUcABIFAUcABJ1yM1CiYltpiBJgwcPDmLFlt336dMniPXt2zfaNjZjpCPLmGfPnh3Eii3/PdSWvB/Ks1BiBgyIbxY0evToIDZmTOzmjPGND2JL8SVp+fLlQeyrr74KYl26dIm+fs6c8GaQL774YrTtoYZZKADQQCjgAJAoCjgAJIoCDgCJancQ08xOkrRAUk+17hM4y90fNLNukv4oqbda9w681t2/bOe96nKgB42jI4OYWeZ2U1OTd+7c+SexlpaWDvUdOJhYbpdSwJslNbv7GjM7VtJbkkZK+pWk3e7+OzObLOk4d/+3dt6LAo5cdbCAZ5bbFHDkraxZKO6+3d3XFB5/I2mjpBMlXSVpfqHZfLUmPpAMchup69CWambWW1J/SSsl9XT37YVvfabWP0Njr5kgaUL5XQTyR24jRSUv5DGzLpJelTTd3Z82s/91965tvv+lux/XzntwCQW5KmchTxa5zSUU5K3shTxm1lnSQkm/d/enC+EdhWuI+68lxnfKBeoYuY2UtVvArXVXgjmSNrp7263PF0kaX3g8XtJz2XcPyE+Wue3uamlp+ckXkLdSZqEMkrRc0ruS9m/NMVWt1wr/JKmXpE/UOtVqdzvvxSUU5KqDs1DIbSSjrGmEWSLJkTduZoVGxc2sAKCBUMABIFEUcABIFAUcABJFAQeARFHAASBRFHAASBQFHAASRQEHgERRwAEgURRwAEgUBRwAEkUBB4BEdWhLNQBAdpqa4ufQ+/bti8aD12fZGQBA9VDAASBRFHAASBQFHAASVcqmxieZ2VIz22Bm75nZxEL8t2b2qZmtK3z9Mv/uAtkht5G6UjY1bpbU7O5rzOxYSW9JGinpWkn/5+4PlPxh7BuInHVwU2NyGxUxC9Mtr32GY7nd7jRCd98uaXvh8TdmtlHSidl3D6guchup69A1cDPrLam/pJWF0G1m9o6ZzTWz44q8ZoKZrTaz1RX1FMgRuY0UtXsJ5W8NzbpIelXSdHd/2sx6StolySXdq9Y/Rf+xnffgz0zkqiOXUPYjt1GuWl9CKamAm1lnSc9LesndZ0a+31vS8+7er533IcmRq44WcHIblah1AS9lFopJmiNpY9sELwwA7TdK0vosOglUC7mNSrl78FVNpcxCGSRpuaR3Je1foD9V0hhJ56n1z8yPJf26MCh0sPfiLAW56uAsFHIbySj7EkpWSHLkrZxr4Fkgt5G3si6hAADqEwUcABJFAQeARLGhA5CYI488Mhr/7rvvqtwT1Bpn4ACQKAo4ACSKAg4AiaKAA0Ciqj2IuUvSJ4XHPQrPGw3HVTsn1/Cz9+d27j+nGg5WppAD5UjhuKK5XdWVmD/5YLPV7j6gJh+eI47r0NbIP6dGPbaUj4tLKACQKAo4ACSqlgV8Vg0/O08c16GtkX9OjXpsyR5Xza6BAwAqwyUUAEgUBRwAElX1Am5mI8xsk5l9aGaTq/35WSrsWL7TzNa3iXUzs5fN7C+Ff6M7mtczMzvJzJaa2QYze8/MJhbiyR9bnholt8nrdI6tqgXczDpJeljSpZL6ShpjZn2r2YeMzZM04oDYZEmvuPvpkl4pPE/Nj5L+1d37Svp7SbcWfk+NcGy5aLDcnifyOgnVPgMfKOlDd9/s7i2S/iDpqir3ITPuvkzS7gPCV0maX3g8X9LIqnYqA+6+3d3XFB5/I2mjpBPVAMeWo4bJbfI6nWOrdgE/UdJf2zzfWog1kp5tNsD9TFLPWnamUmbWW1J/SSvVYMeWsUbP7Yb63TdKXjOImSNvnaOZ7DxNM+siaaGkf3H3r9t+L/VjQ/lS/903Ul5Xu4B/KumkNs9/Vog1kh1m1ixJhX931rg/ZTGzzmpN8t+7+9OFcEMcW04aPbcb4nffaHld7QK+StLpZvZzMztc0mhJi6rch7wtkjS+8Hi8pOdq2JeymJlJmiNpo7vPbPOt5I8tR42e28n/7hsxr6u+EtPMfinpvyR1kjTX3adXtQMZMrMnJQ1V6+0od0j6jaRnJf1JUi+13l70Wnc/cECorpnZIEnLJb0raV8hPFWt1wuTPrY8NUpuk9fpHBtL6QEgUQxiAkCiKOAAkCgKOAAkigIOAImigANAoijgAJAoCjgAJOr/ATBN7iYrdiXeAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "out_img = torch.squeeze(output.cpu().data)\n",
    "print(out_img.size())\n",
    "\n",
    "for i in range(2):\n",
    "    plt.subplot(1,2,1)\n",
    "    plt.imshow(torch.squeeze(image[i]).cpu().numpy(),cmap='gray')\n",
    "    plt.subplot(1,2,2)\n",
    "    plt.imshow(out_img[i].numpy(),cmap='gray')\n",
    "    plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 5) Check with Train Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "with torch.no_grad():\n",
    "    for j,[image,label] in enumerate(test_loader):\n",
    "       \n",
    "        image = image.to(device)\n",
    "        output = encoder(image)\n",
    "        output = decoder(output)\n",
    "        \n",
    "    if j % 10 == 0:\n",
    "        print(loss)        "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([256, 28, 28])\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAC4CAYAAAD61bdSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAARjUlEQVR4nO3deYyVVZrH8d8DgxJBsKGFlLSKUVxJhs0lLpFRGEAxYgQEjRptLU0kdpOOhhijiHEhCmrMdBsMJUgYuhuEkSiOtGULGicIjdhsYiOWCrKINooaVPCZP+raKTjnUrfqrufW95MQ6v7q3Puel3p48ta7mrsLAJCeduWeAACgdWjgAJAoGjgAJIoGDgCJooEDQKJo4ACQqLwauJkNN7NNZrbZzCYValJAuVHbSIG19jxwM2sv6QNJQyVtlbRS0nh333CY93DSOYrK3S3fz6C2UYlitZ3PFvg5kja7+xZ3/0HSHyVdmcfnAZWC2kYS8mngvSR92uT11kx2EDOrNbNVZrYqj2UBpURtIwn/VuwFuPsMSTMkfs1EdaG2UW75bIFvk3R8k9e/ymRA6qhtJCGfBr5SUh8zO8nMjpA0TtLiwkwLKCtqG0lo9S4Ud99vZhMkvSqpvaQ6d19fsJkBZUJtIxWtPo2wVQtjPyGKrBCnEbYGtY1iK/RphACAMqKBA0CiaOAAkCgaOAAkigYOAImigQNAomjgAJAoGjgAJIoGDgCJooEDQKJo4ACQKBo4ACSKBg4AiSr6E3kAlEbHjh2DbN++fWWYSfU58cQTo/nHH39c4pkcjC1wAEgUDRwAEkUDB4BE0cABIFF5PVLNzBok7ZV0QNJ+dx/UzHgeO5WYLl26RPObb745yKZPnx4de9NNNwXZ7Nmz85tYFoV6pBq13XY98MADQXbZZZdFx957771B9uqrrxZ8TlK8tgtxFsp/uPvuAnwOUGmobVQ0dqEAQKLybeAuaamZ/c3MagsxIaBCUNuoePnuQrnQ3beZWQ9JfzGz9919edMBmeLnPwBSQ22j4uW1Be7u2zJ/75K0SNI5kTEz3H1QcweBgEpCbSMFrd4CN7NOktq5+97M1/8paUrBZobopdGSNG7cuCCbM2dOdOyBAwfymsMJJ5wQze+7774gy3ZG06ZNm/KaQ6lR2+Vz6aWXBll9fX1RlnX++edH8xEjRgTZEUccER27c+fOgs6ppfLZhdJT0iIz+/lz/tvd/7cgswLKi9pGElrdwN19i6R/L+BcgIpAbSMVnEYIAImigQNAorgfeAWbOXNmNB8/fnyQ7d4dv2DwpZdeymsO2d7ftWvXIJs/f3507OrVq/OaA6rPE088Ec0vueSSIHv00UejY+fNm5fXHG644YZo3r179yBbsmRJdOyaNWvymkO+2AIHgETRwAEgUTRwAEgUDRwAEkUDB4BEcRZKhejRo0eQnXbaadGx27dvD7J169bltSxJmjIlvFo826X069evD7LYze0l6Ycffsh5bqg+J598cpANGTIkOrahoSHINmzYkPOyOnToEM1jDyAZOnRodOzatWuDLPaQh0rAFjgAJIoGDgCJooEDQKJo4ACQKA5iVohnnnkmyPr37x8de8sttwRZ7OBPNqeffno0v/XWW3P+jNGjRwfZ5s2bc34/2o7YJevZDo7X1dUF2bfffhsdO2DAgCA76qijomNjl+jv3bs3Ovbpp58Osmy3qig3tsABIFE0cABIFA0cABJFAweARNHAASBRzZ6FYmZ1kkZK2uXufTNZN0l/ktRbUoOkse7+z+JNs7rU1tYG2ahRo4Is2xOvn3vuuZyXNWzYsCBbtGhRdOyuXbuCLNuZKak9aT6G2i68iRMnBlnskvWFCxdG379s2bKcl1VTUxNksbOjJOnUU08Nsjlz5kTH1tfX5zyHcstlC3yWpOGHZJMk1bt7H0n1mddAamaJ2kbCmm3g7r5c0peHxFdKmp35erakcPMRqHDUNlLX2gt5err7z7fE2yGpZ7aBZlYrKdxnAFQmahvJyPtKTHd3M/PDfH+GpBmSdLhxQKWhtlHpWtvAd5pZjbtvN7MaSeHRL0QPskjSY489FmT79+8Psuuvvz7nZWW7hPjOO+8MsiOPPDI6dsGCBUGW71PtE0Rt5yB2cFySrr322iDbt29fkG3ZsiX6/vfeey/IrrjiiujYgQMHBtlFF10UHTt//vwgmzZtWnRsSlp7GuFiSTdmvr5R0ouFmQ5QdtQ2ktFsAzezeZL+T9JpZrbVzH4t6VFJQ83sH5KGZF4DSaG2kbpmd6G4+/gs37q0wHMBSoraRuq4EhMAEkUDB4BE8UCHAunevXuQPfzww9GxnTt3DrIVK1YEWezp2NlMnTo1mg8ffuiFhtLbb78dHXv//ffnvDy0bePGjYvmsQc1vP7660G2dOnS6PtjZ0iNGDEi52V98skn0bFPPvlkNE8dW+AAkCgaOAAkigYOAImigQNAojiIWSATJkwIstjTuKX407BnzpwZZNnuBz5mzJggu+2223Je1qxZs6Jjv/zy0BvzAdL06dODrEePHtGx27ZtC7IPPvggyNzjt47p379/kGW79UOXLl2CLNvBymz/l1LHFjgAJIoGDgCJooEDQKJo4ACQKA5ittDIkSOjeewqxmwHaj788MMgix2QefPNN6Pvv+CCC3Je1iuvvBJkc+fOjY5F25btoHvsgdvffvttdGzsKuNevXoFWbb7dl988cWHm+JBdu/eHWQfffRRzu+vBmyBA0CiaOAAkCgaOAAkigYOAImigQNAopo9C8XM6iSNlLTL3ftmssmSbpX0eWbYPe6+pFiTLJfevXsH2UMPPRQda2Y5f27scuFYlk1LlnXNNdcEWezyekmqra3N+XOrQVuu7QsvvDDIst13+6STTgqy2NPjJalr165B1qdPn5wySerbt2+Q7d+/Pzo2dr/8Y489Njq2WuWyBT5LUvhUAOkJd++X+VN1BY42YZaobSSs2Qbu7sslcZcjVB1qG6nLZx/4BDP7u5nVmdkvsg0ys1ozW2Vmq/JYFlBK1DaS0NoG/gdJJ0vqJ2m7pGnZBrr7DHcf5O6DWrksoJSobSSjVZfSu/u/bq5rZs9KeqlgMyqDbPcbnjhxYpDFDrJI8UvZs13enq8ff/wxyCZPnhwde9999wXZvn37Cj2lqlFttR07EC9Jd9xxR5CNHTs2OjZ2wPL777/PeQ79+vULsnbt4tuOX3zxRZBlu0/91q1bg6xbt27RsbF/h4aGhujYlLRqC9zMapq8vErSusJMBygvahspyeU0wnmSBkv6pZltlXS/pMFm1k+SS2qQFH8cDFDBqG2krtkG7u7jI3H4/C8gMdQ2UseVmACQKBo4ACSqzT3QIXbGyfPPPx8dO3r06LyWtWRJ/CK+Z599Nqc5HH300dH3L1iwIMgeeeSR6Nj6+vogW7NmTXQs0hY72yN2tonUsoeCvPPOO0H22muvRccOGDAgyK677rro2JjY/4N33303Ovatt94Kss8//zwysnqxBQ4AiaKBA0CiaOAAkCgaOAAkqs0dxFy2bFmQnX322Tm/f8+ePdH88ssvD7KVK1dGx44ZMybIOnXqlPOysh2wjIkdgEJ1uvrqq4Ms2wHE7777LsimTYvf9mXOnDlBdswxx0THDhw4MMhiT4r/9NNPo++PHcTcvHlzdCzYAgeAZNHAASBRNHAASBQNHAASRQMHgES1ubNQ6urqgizbWSi///3vg+ypp56Kjo0dKe/YsWN07F133RVksRvcT506Nfr+deu4RTVCGzZsCLJFixZFx8bOcFq8eHF0bKzezj333OjYM844I8hqamqCLPagEYkzTlqKLXAASBQNHAASRQMHgETRwAEgUdbck9PN7HhJz0vqqcbnBM5w96fMrJukP0nqrcZnB451938281nFeUx7BTjllFOCLNul9F27dg2y5cuXB9ngwYPznldb4+6W69i2UNu9evWK5vv27Quyr7/+Ojo29qT3KVOmRMeOHx8+pe7ll1/OaRwOL1bbuWyB75f0O3c/U9J5ku4wszMlTZJU7+59JNVnXgMpobaRtGYbuLtvd/fVma/3StooqZekKyXNzgybLWlUsSYJFAO1jdS16DxwM+stqb+kFZJ6uvv2zLd2qPHX0Nh7aiXVtn6KQPFR20hRzgcxzayzpBck/dbdD9pZ5o070qP7AN19hrsPcvdBec0UKBJqG6nKqYGbWQc1Fvhcd1+YiXeaWU3m+zWSdhVnikDxUNtIWbO7UMzMJM2UtNHdpzf51mJJN0p6NPP3i0WZYYXJdnn8yJEjg6xLly7RsWvXrg0yjsqXXluo7W3btuU8tm/fvtF86NChQRY760qS3njjjSCbPHlyznNAy+SyD/wCSddLWmtmazLZPWos7j+b2a8lfSxpbHGmCBQNtY2kNdvA3f0tSdnOrb20sNMBSofaRuq4EhMAEkUDB4BEtbn7gedr0qT4RXmx+xt/9dVX0bHDhg0Lsh07duQ3MSBP5513XjS/6qqrguybb76Jjr377ruDbNOmTflNDFmxBQ4AiaKBA0CiaOAAkCgaOAAkigYOAIniLJTDiF3enu0slIaGhiC7/fbbo2M54wTldtZZZwXZkCFDomM7d+4cZI8//nh07Pvvv5/fxNAibIEDQKJo4ACQKBo4ACSKBg4AieIg5mF06tQpyDp06BAde9dddwXZ0qVLCz4noBB69OgRZHv27ImOffDBB4Ns8eLFBZ8TWo4tcABIFA0cABJFAweARNHAASBRzTZwMzvezP5qZhvMbL2Z/SaTTzazbWa2JvPnsuJPFygcahupM3c//ACzGkk17r7azI6W9DdJo9T4oNdv3D1+TW38sw6/MCBP7p7tGZcBavtgxx13XDT/7LPPSjwTxMRqO5eHGm+XtD3z9V4z2yipV+GnB5QWtY3UtWgfuJn1ltRf0opMNMHM/m5mdWb2iyzvqTWzVWa2Kq+ZAkVEbSNFze5C+ddAs86Slkl6yN0XmllPSbsluaQH1fir6M3NfEbyv2aisrVkF8rPqO1G7EKpbLHazmkL3Mw6SHpB0lx3X5j5sJ3ufsDdf5L0rKRzCjlZoBSobaSs2X3gZmaSZkra6O7Tm+Q1mX2IknSVpHXFmSJQHG25ttu3bx9kbGmnJ5ezUC6U9KaktZJ+ysT3SBovqZ8af81skHRbk6LP9lnJ/5qJytbCs1DabG3HGviBAwfKMBPkKlbbOe8DL4TUihzpac0+8EJIrbZp4Olp9T5wAEDloYEDQKJo4ACQKB7oALRB7O8uvXbtwu3ln376KTKyBZ+Z17sBAGVDAweARNHAASBRNHAASFSpD2LulvRx5utfZl5XG9arfE4s47J/ru0U/p1aq1rXrSTrlecBy2htl/RKzIMWbLbK3QeVZeFFxHq1bdX871St65byerELBQASRQMHgESVs4HPKOOyi4n1atuq+d+pWtct2fUq2z5wAEB+2IUCAImigQNAokrewM1suJltMrPNZjap1MsvpMwTy3eZ2bomWTcz+4uZ/SPzd/SJ5pXMzI43s7+a2QYzW29mv8nkya9bMVVLbVPX6axbSRu4mbWX9F+SRkg6U9J4MzuzlHMosFmShh+STZJU7+59JNVnXqdmv6TfufuZks6TdEfm51QN61YUVVbbs0RdJ6HUW+DnSNrs7lvc/QdJf5R0ZYnnUDDuvlzSl4fEV0qanfl6tqRRJZ1UAbj7dndfnfl6r6SNknqpCtatiKqmtqnrdNat1A28l6RPm7zemsmqSc8mD8DdIalnOSeTLzPrLam/pBWqsnUrsGqv7ar62VdLXXMQs4i88RzNZM/TNLPOkl6Q9Ft3/7rp91JfN7Re6j/7aqrrUjfwbZKOb/L6V5msmuw0sxpJyvy9q8zzaRUz66DGIp/r7gszcVWsW5FUe21Xxc++2uq61A18paQ+ZnaSmR0haZykxSWeQ7EtlnRj5usbJb1Yxrm0ipmZpJmSNrr79CbfSn7diqjaazv5n3011nXJr8Q0s8skPSmpvaQ6d3+opBMoIDObJ2mwGm9HuVPS/ZL+R9KfJZ2gxtuLjnX3Qw8IVTQzu1DSm5LWSvr5Hpj3qHF/YdLrVkzVUtvUdTrrxqX0AJAoDmICQKJo4ACQKBo4ACSKBg4AiaKBA0CiaOAAkCgaOAAk6v8BBhms/ng15w8AAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAC4CAYAAAD61bdSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAR8klEQVR4nO3de4zUZZbG8efAKOqiiYAggi7GG0HkIohjBOPqzgqKARF1jCHIqgyoRONlQRJxRI2XdXU2RhdREEwGRlEQNauuoFk1btpuCSoiMthgoG1uMiiGoFzO/tHFpOV9i66uW/dbfD+JsfvpU1Vv0YeTH/W7mbsLAJCeNi29AABAfhjgAJAoBjgAJIoBDgCJYoADQKIY4ACQqIIGuJkNNbOvzWyNmU0p1qKAlkZvIwWW73HgZtZW0mpJv5O0QVK1pGvdfeVBHsNB5ygpd7dCn4PeRmsU6+1CtsAHSVrj7rXu/oukv0gaUcDzAa0FvY0kFDLAu0la3+j7DZnsV8xsvJnVmFlNAa8FlBO9jST8ptQv4O4zJc2U+GcmKgu9jZZWyBZ4naQTG33fPZMBqaO3kYRCBni1pNPM7GQzO1zS7yW9XpxlAS2K3kYS8v4Ixd33mNmtkt6R1FbSbHf/smgrA1oIvY1U5H0YYV4vxueEKLFiHEaYD3obpVbswwgBAC2IAQ4AiWKAA0CiGOAAkCgGOAAkigEOAIligANAokp+LRSk46ijjgqy/v37R2uHDBkSZLt27YrWVldXB9m6deuitXV1nLHeWJs24TbWvn37WmAlladnz57RPNbzHTt2jNZ++WV4ftfq1aujtd9//32QZfs7kyu2wAEgUQxwAEgUAxwAEsUAB4BEMcABIFEchXIIateuXTS/5557gmzq1KnRWrPwon/NubLlli1bonlVVVWQjRhx6N6OkiNOmufwww+P5nfccUeQjRkzJlrbvn37IOvevXu0dsWKFUG2ffv2aO28efOC7Pnnn4/W7t27N5ofiC1wAEgUAxwAEsUAB4BEMcABIFEF7cQ0s3WSdkjaK2mPuw8sxqJQPMOGDQuyadOmRWsHDRqU8/MuXrw4yLLtxFy6dGmQrV+/PufXagn0dus3fPjwILvzzjujtZ07dw6y2traaO2ePXuCLNbDUryP165dG61dtmxZkOW6szKbYhyF8k/uvrUIzwO0NvQ2WjU+QgGARBU6wF3S/5jZp2Y2vhgLAloJehutXqEfoQx29zoz6yzpXTNb5e4fNC7IND9/AZAaehutXkFb4O5el/n/ZkmLJAV7wdx9prsPZCcQUkJvIwV5b4Gb2T9IauPuOzJf/4uk6UVbGZqlb9++0fzFF18Msg4dOkRra2pqgmzs2LHR2lWrVjVjdWmht1uXyy+/PJrfdNNNQdapU6do7WuvvRZkCxYsiNYuX768GatrWYV8hNJF0qLMNTF+I2meu79dlFUBLYveRhLyHuDuXispvtkHJIzeRio4jBAAEsUAB4BEcT3wBJ1++ulBNmnSpGht7E7z999/f7T24YcfDrLdu3c3c3VA/vr16xdkEydOjNa2bds2yLJdv/6NN94obGGtFFvgAJAoBjgAJIoBDgCJYoADQKIY4ACQKGvOncQLfjGz8r1YBYgdQSJJc+fODbLRo0dHaxctWhRko0aNKmxhrZi7W0u8Lr1dHDNmzAiyXr16RWufeOKJIIudMl8pYr3NFjgAJIoBDgCJYoADQKIY4ACQKE6lb8VmzZoVzWM7IV944YVo7eTJk4u6JqAYpk+PX149dip9dXV1tPadd94p6ppSxBY4ACSKAQ4AiWKAA0CiGOAAkCgGOAAkqsmjUMxstqThkja7e+9M1kHSS5J6SFon6Wp3/1vplllZ2rVrF2RDhw4NsmHDhuX8nPPmzYvmW7ZsyX1hhxh6uzyuuuqqILv00kujtd26dQuy7777Llp7yimnBNmKFSuaubq05bIFPkfSgdNliqSl7n6apKWZ74HUzBG9jYQ1OcDd/QNJ2w6IR0jaf0WluZJGFnldQMnR20hdvifydHH3+szXGyV1yVZoZuMljc/zdYByo7eRjILPxHR3P9ilNN19pqSZEpfcRFrobbR2+Q7wTWbW1d3rzayrpM3FXFSlyHY979i1u7OdCp+r+fPnR/PY9d6XLFkSrb3rrruCLNsOpApGb+fgyCOPjOYTJkwIsiuuuCLIevfuHX38rl27cnq8JP3yyy9B1qZN/FPhl19+OcheeeWVaG1K8j2M8HVJYzNfj5W0uDjLAVocvY1kNDnAzWy+pP+TdIaZbTCzGyQ9Iul3ZvZXSf+c+R5ICr2N1DX5EYq7X5vlRxcXeS1AWdHbSB1nYgJAohjgAJAo7kpfQgsWLIjmpbgrvFn8ZuzN+f1+9tlnQZbtdP5Nmzbl/LzlxF3py+Oxxx6L5rfcckuQ7du3L8g2b44f3FNXVxdkxx13XLS2vr4+yAYMGBCtfe+994Js3Lhx0drt27dH85bGXekBoIIwwAEgUQxwAEgUAxwAEsVd6Zsp2+nx11xzTZDFTpmXct+xWFVVFc0XLw5PDnzkkfj5Jn369AmyOXPmRGtjdwS//vrro7WPPvpoNEflmTRpUpCdc8450dqdO3cGWW1tbZC9/fbb0cc/9dRTQXbMMcdEa0844YQge/LJJ6O1seuMx/7OStKzzz4bzVsjtsABIFEMcABIFAMcABLFAAeARLET8yBiO0/uvvvuaO3UqVODLHa9Ykn68MMPgyy2E/L999+PPn7v3r3RPGbNmjVBtnbt2mht3759g6x79+45vxbScfHF4fW6zjvvvGjt+eefH2TZenvGjBlB9tJLLwVZc24+vHXr1mge2zn68ccfR2svu+yyIOvRo0fOa2it2AIHgEQxwAEgUQxwAEgUAxwAEsUAB4BENXkUipnNljRc0mZ3753J/ijpJklbMmVT3f2/S7XIYurYsWOQTZw4MVoby48//vicXyvbXa+vu+66nJ+jULFT/0eOHJnz47Pd7b4SVFpvn3rqqUF22223RWtjR6H89NNP0dp27doFWexoE0maNWtWkGU7YqUUsl07fMuWLUH2ySeflHo5JZfLFvgcSUMj+ZPu3i/zXxINDhxgjuhtJKzJAe7uH0jaVoa1AGVFbyN1hXwGfquZfW5ms83s2GxFZjbezGrMrKaA1wLKid5GEvId4P8l6RRJ/STVS/qPbIXuPtPdB7r7wDxfCygnehvJyOtUenf/+x1tzew5SW8WbUUlNn369CCbMGFCwc+7evXqILvhhhsKft5C3XzzzTnX1tSEG5LV1dXFXE6rl3JvjxgxIship5BL0tFHHx1k2S7RsGPHjiD75ptvorXl3GEZ+/t11llnRWs///zzIFu0aFHR11RueW2Bm1nXRt9eISn3CxsArRi9jZTkchjhfEkXSupkZhsk3SfpQjPrJ8klrZP0hxKuESgJehupa3KAu/u1kTg82BNIDL2N1HEmJgAkigEOAIk65G7oEDstOJtnnnkmyIYMGRKtje39znaKfrY7ZxfiyiuvjObZbkARM3bs2CDbvXt33mtCeQ0YMCDIPv3002htmzbhtlvsaBMpfvf3I444opmry9/w4cOj+dNPPx1kX3/9dbT24YcfLuqaWgu2wAEgUQxwAEgUAxwAEsUAB4BEHXI7MWPMLJq/++67QRbbcSJJK1euDLLJkydHa2PX2N64cWOQxa7lLUlz584NstGjR0drf/zxxyC75JJLorWrVq2K5kjDt99+G2RnnHFGtDa2c3Pnzp3RWncPsnPPPTdau3z58iCL9WC26+oPHjw4yKZMmRKtjV3jO3apDElasaIyT6hlCxwAEsUAB4BEMcABIFEMcABIFAMcABJ1yB2FctJJJwVZbC+7JJ199tlBVlVVFa196623cnq8JA0aNCjIYkec3HvvvdHH9+zZM8jWr18frY3dgX7ZsmXRWqQtdjOFH374IVrbvn37INuwYUO0tlOnTkF25plnRmtjvRk7CiX2+pLUuXPnIGvbtm209sEHHwyyV199NVpbqdgCB4BEMcABIFEMcABIFAMcABJl2Xbg/b3A7ERJL0rqoob7BM509/80sw6SXpLUQw33Drza3f/WxHMd/MXKILaTZcGCBdHaXr16BdkDDzwQrY2dWpztlPWm/sybsmTJkpzX9dFHHxX0Wqlx9/h1ESIqrbd79+4dZA899FC0NrZjMdulG/r06RNkJ598crQ2djp+XV1dkMWuRy7FT/HPtmPyUNthGevtXLbA90i60917SfqtpFvMrJekKZKWuvtpkpZmvgdSQm8jaU0OcHevd/dlma93SPpKUjdJIyTtv6rSXEnh8WpAK0ZvI3XNOg7czHpI6i+pSlIXd6/P/GijGv4ZGnvMeEnj818iUHr0NlKU805MM2sv6VVJt7v7rz5A84YPdaOfAbr7THcf6O4DC1opUCL0NlKV0wA3s8PU0OB/dveFmXiTmXXN/LyrpM2lWSJQOvQ2UpbLUSimhs8Bt7n77Y3yf5f0vbs/YmZTJHVw939r4rlafE99zI033hjNp02bFmTdunXL+Xmz3Sgi9mceO435ueeeiz5+3rx5QVZbW5vzuipZM49CqfjeHjduXDQfNWpUkF100UXR2tgRI3v27InWbt26Nch+/vnnIFu4cGGQSdLjjz8eZNu2bYvWHmpivZ3LZ+DnSxoj6Qsz23+7jamSHpH0spndIOlbSVcXa6FAmdDbSFqTA9zdP5KUbavm4uIuBygfehup40xMAEgUAxwAEtXkTsyivlgr3dGTTYcOHYJszJgx0drYaczZ7Nq1K8juu+++IGPnTfM1ZydmMaXW2xdccEGQ9evXL1p77LHHBlm2U+Hr6+uDbPHixTnV4eDyPZUeANAKMcABIFEMcABIFAMcABLFAAeARHEUCioKR6GgUnEUCgBUEAY4ACSKAQ4AiWKAA0CiGOAAkCgGOAAkigEOAIligANAohjgAJCoJge4mZ1oZu+b2Uoz+9LMbsvkfzSzOjNbnvnv0tIvFygeehupy+Wu9F0ldXX3ZWZ2tKRPJY1Uw41ef3L38DbS2Z+L041RUs28Kz29jWTkdVd6d6+XVJ/5eoeZfSWpW/GXB5QXvY3UNeszcDPrIam/pKpMdKuZfW5ms80svO9Sw2PGm1mNmdUUtFKghOhtpCjnqxGaWXtJ/yvpIXdfaGZdJG2V5JIeUMM/Rf+1iefgn5koqXyuRkhvIwWx3s5pgJvZYZLelPSOuz8R+XkPSW+6+0Hv7EuTo9SaO8DpbaQir8vJmplJmiXpq8YNntkBtN8VklYUY5FAudDbSF0uR6EMlvShpC8k7cvEUyVdK6mfGv6ZuU7SHzI7hQ72XGyloKSaeRQKvY1k5P0RSrHQ5Cg17siDSsUdeQCggjDAASBRDHAASFSTZ2ICKL02beLbUvv27YvmgMQWOAAkiwEOAIligANAohjgAJCocu/E3Crp28zXnTLfVxreV8v5xxZ87f29ndefUyI7K1PogXyk8L6ivV3WMzF/9cJmNe4+sEVevIR4X4e2Sv5zqtT3lvL74iMUAEgUAxwAEtWSA3xmC752KfG+Dm2V/OdUqe8t2ffVYp+BAwAKw0coAJAoBjgAJKrsA9zMhprZ12a2xsymlPv1iylzx/LNZraiUdbBzN41s79m/h+9o3lrZmYnmtn7ZrbSzL40s9syefLvrZQqpbfp63TeW1kHuJm1lfS0pGGSekm61sx6lXMNRTZH0tADsimSlrr7aZKWZr5PzR5Jd7p7L0m/lXRL5vdUCe+tJCqst+eIvk5CubfAB0la4+617v6LpL9IGlHmNRSNu38gadsB8QhJczNfz5U0sqyLKgJ3r3f3ZZmvd0j6SlI3VcB7K6GK6W36Op33Vu4B3k3S+kbfb8hklaRLoxvgbpTUpSUXUygz6yGpv6QqVdh7K7JK7+2K+t1XSl+zE7OEvOEYzWSP0zSz9pJelXS7u//Y+GepvzfkL/XffSX1dbkHeJ2kExt93z2TVZJNZtZVkjL/39zC68mLmR2mhib/s7svzMQV8d5KpNJ7uyJ+95XW1+Ue4NWSTjOzk83scEm/l/R6mddQaq9LGpv5eqykxS24lryYmUmaJekrd3+i0Y+Sf28lVOm9nfzvvhL7uuxnYprZpZL+JKmtpNnu/lBZF1BEZjZf0oVquBzlJkn3SXpN0suSTlLD5UWvdvcDdwi1amY2WNKHkr6QtP86p1PV8Hlh0u+tlCqlt+nrdN4bp9IDQKLYiQkAiWKAA0CiGOAAkCgGOAAkigEOAIligANAohjgAJCo/wcwN/QXG30fegAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXAAAAC4CAYAAAD61bdSAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAASy0lEQVR4nO3dfYxV5bXH8d/ixYKgVVodKaC0KreiVKxUbTW+FLFgTBCbUm1LaWilf7REDf5hjVF7TdPGcAs3jRq50QLKLdi0Xo29tldtK2CxEYxFBVHUIaK8+IKC0ZYA6/4xx2TkWZs5c87ZZ+Y58/0khJk165z9bGbN4sx5nr0fc3cBAPLTr6cHAACoDQ0cADJFAweATNHAASBTNHAAyBQNHAAyVVcDN7PJZrbRzDaZ2XWNGhTQ06ht5MBqXQduZv0lvShpkqQtkp6SdIW7rz/IY1h0jlK5u9X7HNQ2eqOotut5BX6GpE3u/oq775G0TNLUOp4P6C1avrb79euX/CmLmSV/ouOXOYZWVc+/2AhJr3X6fEsl9jFmNtvM1pjZmjqOBTQTtY0sDCj7AO6+UNJCiV8z0VqobfS0ehr465JGdfp8ZCUG5K7la3v//v1NO1Y0z8Y9mBqjnrdQnpJ0opl91swOkXS5pAcbMyygR1HbyELNr8Ddfa+Z/VjSnyT1l3S3uz/fsJEBPYTaRi5qXkZY08F4nxAla8QywlpQ2yhbo5cRAgB6EA0cADJFAweATNHAASBTNHAAyBQNHAAyRQMHgEyVfi8U5GPQoEFJ7Jprrglzb7jhhiT2wQcfhLlTpkxJYmvWcP8nNM/IkSOT2JVXXhnmjhkzJont2rUrzP3zn/+cxJYvX97N0dWOV+AAkCkaOABkigYOAJmigQNApmjgAJApVqG0uBEjkp3ANGvWrDB30qRJSezss8+u+ljRKhZJmj9/fhK76KKLwtwPP/yw6uOhbxsyZEgSmzNnTpj7la98JYmdfPLJYW50h9ZoFYvUsd/ngf74xz+Gue+9914YrwevwAEgUzRwAMgUDRwAMkUDB4BM1TWJaWbtknZL2idpr7tPaMSg0H1Fk4K33nprEhs3blzdx3v++XSLyKJJoT179iSxffv21T2GMlHbvceMGTPC+M0335zEjj766DD32WefTWJbtmwJc4cNG5bE3n777TB3wIC0he7duzfMLUMjVqFc4O5vNeB5gN6G2kavxlsoAJCpehu4S/o/M1trZrMbMSCgl6C20evV+xbKOe7+upkdLekRM3vB3Vd0TqgUPz8AyA21jV6vrlfg7v565e8dku6XdEaQs9DdJzAJhJxQ28hBza/AzWyIpH7uvrvy8UWS/r1hI4MuvPDCMH7TTTclsTPPPDPM7d+/f9XHi25av3jx4qrHcN5554W569evT2LRypTegtou34QJ8f953/nOd5LYBRdcEOZ+7nOfS2Lbtm0Lc9euXZvE1q1bF+bu3r07iRWtWFm9enUSa+YKq3reQmmTdH/lXgADJP23u8c3AQDyQm0jCzU3cHd/RdKpDRwL0CtQ28gFywgBIFM0cADIlEX3vi3tYGbNO1gvduihhyaxqVOnJrE777wzfPzQoUOTWNH38c0330xi1157bZi7cuXKJLZ58+Ywt7dy9/QGzU1AbXeI7tE9ceLEJBbVuxRPWBZNxD/88MNJbPv27WFuNOmem6i2eQUOAJmigQNApmjgAJApGjgAZIoGDgCZYhVKiaKbvUvx5emXX3551c/br1/6/+4999wT5t54441JrL29vepj5YZVKD1r3rx5SSxacRKtjpLiFSerVq0Kc5cuXZrEnn766a6GmC1WoQBAC6GBA0CmaOAAkCkaOABkqhGbGkPS4MGDk1jRhMqYMWOqes4PPvggjH/9619PYo8//niY+69//auqYwFFRo4cmcTmzJkT5s6aNSuJDRo0KIkV1esTTzyRxH796193NcQ+i1fgAJApGjgAZIoGDgCZooEDQKZo4ACQqS4vpTezuyVdImmHu59SiQ2TtFzSaEntkqa7+84uD9YClxtHO2FL0t/+9rckdtRRR1X9vP/4xz+SWNEu79Gu2ejQnUvpqe2Pi1abSNKCBQuS2GmnnRbmRque1qxZk8SWLFkSPv7RRx892BD7tFovpV8kafIBseskPebuJ0p6rPI5kJtForaRsS4buLuvkPTOAeGpkj66I9NiSZc2eFxA6aht5K7WC3na3H1r5eNtktqKEs1stqTZNR4HaDZqG9mo+0pMd/eDvf/n7gslLZRa431C9B3UNnq7Whv4djMb7u5bzWy4pB2NHFRvcfrppyexO+64I8yNJiz37dsX5s6dOzeJRZcWM1nZI/pEbbe1pb9Y/OAHPwhzx40bl8S2bdsW5i5atCiJbdiwIYkV3eMb3VPrMsIHJc2sfDxT0gONGQ7Q46htZKPLBm5mv5G0WtK/mdkWM/u+pF9ImmRmL0m6sPI5kBVqG7nr8i0Ud7+i4EsTGzwWoKmobeSOKzEBIFM0cADIFBs6HMRll12WxKKVKZL04osvJrFp06aFuS+88EJ9AwPqdM011ySxL3/5y2FutMnC7bffHuZGl82jPLwCB4BM0cABIFM0cADIFA0cADLFJKakww8/PIxHEz1m8e2mo3sm99bJymHDhoXxI444ournePfdd5PYO+8ceGM/9LQTTjghjEf3tS+69cMf/vCHJFbWZOWAAWlLKrpPeXt7exI75ZRTwtz333+/qsdL0vHHH5/EXn755TC3p/EKHAAyRQMHgEzRwAEgUzRwAMgUk5iSpk+fHsY/8YlPJLGNGzeGuffdd19dY4juJ/6FL3whzL3qqquSWFebU3f2+c9/PowXTXhF1q9fn8S+9rWvhblvvPFG1c+Lxir6Xo8ePTqJrVixIsytd6PhaAxFE+bf+MY3ktjQoUPD3COPPDKJnXTSSWHuZz7zmSRW9DPzzDPPJLEbbrghzH3yySfDeLPwChwAMkUDB4BM0cABIFM0cADIFA0cADLV5SoUM7tb0iWSdrj7KZXYzZKulPRmJe16d//fsgbZm+zcubPq+MCBA8PcJUuWJLGzzjoriR177LHh46PL+buzCqURxo4dm8TuvffeMPerX/1q2cOpSV+o7SFDhoTxQYMGJbFoVYckTZo0KYkVXUofreg69dRTk9jJJ58cPn7UqFFJbO/evWHutm3bktiePXvC3Og5DjvssDA3uuf/9773vTA3h1UoiyRNDuLz3X185U+2BY4+bZGobWSsywbu7iskcZcitBxqG7mr5z3wH5vZOjO728zi370kmdlsM1tjZuy1hFxQ28hCrQ38DknHSxovaauk/yhKdPeF7j7B3SfUeCygmahtZKOmS+ndfftHH5vZf0l6qGEj6uWiSRZJ+u1vf5vEJk+O3l6VDj300LrGsGPHjiRWNIkZTSx2577d0eSqJF1yySVJ7FOf+lTVz9tbtVptF00WDh48OIlFk5VSfBn6jBkzwtxx48YlseiWFEWTjW+99VYSK5owffjhh5NY0aRtNPF//vnnh7nRpf/RRKwkHXLIIUms6NzKUNMrcDMb3unTaZKea8xwgJ5FbSMn1Swj/I2k8yV92sy2SLpJ0vlmNl6SS2qX9MMSxwiUgtpG7rps4O5+RRC+q4SxAE1FbSN3XIkJAJmigQNAptjQoZuiGXlJuuyyy6p+jldffTWJrV69OolFu4FL0rJly6o+VndElxZ/61vfKuVYaI5du3aF8ehS+qJLy88555wkdv/994e5r7zyShKLdoS/8847w8evWrWqqsdL3bt9RLTT/Je+9KUwN7qlQHTZviTt37+/6jGUgVfgAJApGjgAZIoGDgCZooEDQKaYxJQ0ZsyYUp63aKf6aFf56PL4shRdFnzjjTcmsei+35K0b9++JPbzn/+8voGhLgMGpD/OJ5xwQph7+OGHJ7EPP/wwzI1u/RB9/yXptttuS2JFE55l+OY3vxnGZ86cmcSOO+64MPfdd99NYg888ECYW3Sv8mbhFTgAZIoGDgCZooEDQKZo4ACQKRo4AGSqz61CmTt3bhK7+uqrm3YsqZwVJ9Gl0ZL07W9/O4ktWLAgzI1WG/zzn/8Mc2fNmpXEli9ffrAhomTTpk1LYhMnTgxzow0ONm7cGOZGNfDoo4+GuevWrTvYEGsyfvz4MH7SSSclse9+97th7rnnnpvEXnrppTD3Jz/5SRKLNo/oDXgFDgCZooEDQKZo4ACQKRo4AGSqmj0xR0laIqlNHfsELnT3/zSzYZKWSxqtjr0Dp7v7zvKG2j1HHXVUGJ8zZ04S69+/f5i7ffv2JFY0WfjJT34yiT30ULyh+a233prEnnuu+r1zo0vxi3YUHzVqVBIrugw6mpgq2n28mZf+lyXX2i5y5plnJrGi+9dH37+i2o7uEx7dI1yK76W9adOmJBb9vEhSW1tbEpsyZUqYO3To0CQWTWxK8QT7vHnzwtz169eH8d6omlfgeyXNdfexks6S9CMzGyvpOkmPufuJkh6rfA7khNpG1rps4O6+1d2frny8W9IGSSMkTZW0uJK2WNKlZQ0SKAO1jdx1ax24mY2WdJqkv0tqc/etlS9tU8evodFjZkuaXfsQgfJR28hR1ZOYZjZU0u8kXe3uH9tozzs2pws3qHP3he4+wd0n1DVSoCTUNnJVVQM3s4HqKPCl7v77Sni7mQ2vfH24pPxntdDnUNvIWTWrUEzSXZI2uPsvO33pQUkzJf2i8nd8x/MeUrST9cqVK5NY0c7rv/rVr5LYU089FeZGmxl88YtfDHOXLl0axqsVXQZdtEP3X//61yRWtDpm/vz5dY0rN7nWdpGdO9OFMk888USYu3nz5iQ2cODAMDfa0f2YY44Jc6PVKUcccUQSizZNkOJVU6+99lqY297ensR++tOfhrnRz3IrqOY98LMlzZD0rJk9U4ldr47ivs/Mvi9ps6Tp5QwRKA21jax12cDdfZWk9CVfh/hOOUAGqG3kjisxASBTNHAAyJQVTX6VcjCz5h2swODBg5PYtddeG+Y++eSTSeyRRx4Jc6PLeqNd3qXi+4QfaNGiRWH8lltuqerxkvTGG28ksT179lT9+Ny4e9FbIqXqDbUdKdqVPrq8vWgS8+KLL05iZ5xxRphbdIn9gTZs2BDG165dm8SWLVsW5u7evbuqY7WKqLZ5BQ4AmaKBA0CmaOAAkCkaOABkigYOAJnqc6tQ0NpYhdKzos0f+tpqkbKwCgUAWggNHAAyRQMHgEzRwAEgU0xioqUwiYlWxSQmALQQGjgAZIoGDgCZooEDQKa6bOBmNsrM/mJm683seTO7qhK/2cxeN7NnKn/SmwYDvRi1jdx1uQrFzIZLGu7uT5vZYZLWSrpUHRu9vu/u86o+GDP1KFl3VqFQ28hJVNvVbGq8VdLWyse7zWyDpBGNHx7QXNQ2ctet98DNbLSk0yT9vRL6sZmtM7O7zezIgsfMNrM1ZramrpECJaK2kaOqL+Qxs6GSHpf0M3f/vZm1SXpLkku6RR2/is7q4jn4NROlquVCHmobOYhqu6oGbmYDJT0k6U/u/svg66MlPeTup3TxPBQ5StXdBk5tIxc1XYlpZibpLkkbOhd4ZQLoI9MkPdeIQQLNQm0jd9WsQjlH0kpJz0raXwlfL+kKSePV8Wtmu6QfViaFDvZcvEpBqbq5CoXaRjZqfgulUShylI2bWaFVcTMrAGghNHAAyBQNHAAyRQMHgEzRwAEgUzRwAMgUDRwAMkUDB4BMdXk72QZ7S9LmysefrnzeajivnnNcDx77o9rO4d+pVq16bjmcV1jbTb0S82MHNlvj7hN65OAl4rz6tlb+d2rVc8v5vHgLBQAyRQMHgEz1ZANf2IPHLhPn1be18r9Tq55btufVY++BAwDqw1soAJApGjgAZKrpDdzMJpvZRjPbZGbXNfv4jVTZsXyHmT3XKTbMzB4xs5cqf4c7mvdmZjbKzP5iZuvN7Hkzu6oSz/7cytQqtU1d53NuTW3gZtZf0m2SpkgaK+kKMxvbzDE02CJJkw+IXSfpMXc/UdJjlc9zs1fSXHcfK+ksST+qfJ9a4dxK0WK1vUjUdRaa/Qr8DEmb3P0Vd98jaZmkqU0eQ8O4+wpJ7xwQnippceXjxZIubeqgGsDdt7r705WPd0vaIGmEWuDcStQytU1d53NuzW7gIyS91unzLZVYK2nrtAHuNkltPTmYepnZaEmnSfq7WuzcGqzVa7ulvvetUtdMYpbIO9ZoZrtO08yGSvqdpKvdfVfnr+V+bqhd7t/7VqrrZjfw1yWN6vT5yEqslWw3s+GSVPl7Rw+PpyZmNlAdRb7U3X9fCbfEuZWk1Wu7Jb73rVbXzW7gT0k60cw+a2aHSLpc0oNNHkPZHpQ0s/LxTEkP9OBYamJmJukuSRvc/ZedvpT9uZWo1Ws7++99K9Z106/ENLOLJS2Q1F/S3e7+s6YOoIHM7DeSzlfH7Si3S7pJ0v9Iuk/Sseq4veh0dz9wQqhXM7NzJK2U9Kyk/ZXw9ep4vzDrcytTq9Q2dZ3PuXEpPQBkiklMAMgUDRwAMkUDB4BM0cABIFM0cADIFA0cADJFAweATP0/+i4ZeIzqzZYAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "out_img = torch.squeeze(output.cpu().data)\n",
    "print(out_img.size())\n",
    "\n",
    "for i in range(2):\n",
    "    plt.subplot(1,2,1)\n",
    "    plt.imshow(torch.squeeze(image[i]).cpu().numpy(),cmap='gray')\n",
    "    plt.subplot(1,2,2)\n",
    "    plt.imshow(out_img[i].numpy(),cmap='gray')\n",
    "    plt.show()"
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [],
   "name": "9.1 기본적인 오토인코더(Basic_Autoencoder).ipynb",
   "provenance": [],
   "version": "0.3.2"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}