Chap06-Deep_learning_for_text_and_sequences/6.2-understanding-recurrent-neural-networks.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6.2 순환 신경망(RNN) 이해하기"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- [[링크](http://excelsior-cjh.tistory.com/183)] 참고\n",
    "- **순환 신경망**(RNN, Recurrent Neural Network)는 시퀀스의 원소를 순회하면서 지금까지 처리한 정보를 **상태(state)**에 저장한다.\n",
    "- RNN은 내부에 루프(loop)을 가진 신경망의 한 종류다.\n",
    "\n",
    "$$\n",
    "\\begin{align*}\n",
    "h_t &= \\tanh \\left(\\mathbf{X}_{t} \\cdot \\mathbf{W}_{x} + h_{t-1} \\cdot \\mathbf{W}_{h} + \\mathbf{b} \\right) \\\\ \\mathbf{Y}_{t} &= \\mathbf{W}^{T}_{y} \\cdot h_{t}\n",
    "\\end{align*}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![](./images/rnn04.png)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### NumPy를 이용한 RNN 구현하기\n",
    "\n",
    "1. 이 RNN은 형태가 `(timesteps, input_features)`인 2D인 텐서로 인코딩된 벡터의 시퀀스를 입력받는다.\n",
    "2. 이 시퀀스는 타임스텝(`timesteps`)을 따라 반복된다.\n",
    "3. 각 타임스텝 `t`에서 현재 상태(`state`)와 입력을 연결하여 출력을 계산한다.\n",
    "4. 이 출력을 다음 스텝(`t+1`)의 상태로 설정한다.\n",
    "    - 첫 번째 타임스텝에서는 출력이 정의되어있지 않으므로 **초기 상태**인 0벡터로 상태를 초기화한다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "timesteps = 100  # 입력 시퀀스에 있는 타임스텝의 수\n",
    "input_features = 32  # 입력 특성의 차원\n",
    "output_features = 64  # 출력 특성의 차원\n",
    "\n",
    "inputs = np.random.random((timesteps, input_features))  # 입력 데이터(X): 예제를 위해 생성한 난수\n",
    "\n",
    "state_t = np.zeros((output_features,))  # 초기 상태(h0): 모두 0인 벡터\n",
    "\n",
    "W_x = np.random.random((output_features, input_features))\n",
    "W_h = np.random.random((output_features, output_features))\n",
    "b = np.random.random((output_features,))\n",
    "\n",
    "outputs = []\n",
    "for input_t in inputs:  # input_t는 형태가 (input_featurs,)인 벡터다\n",
    "    # 입력(x_t)과 현재 상태(이전 출력, h_{t-1})을 연결하여 현재 출력(h_t)를 얻는다.\n",
    "    h_t = np.tanh(np.dot(W_x, input_t) + np.dot(W_h, state_t) + b)\n",
    "    outputs.append(h_t)  # 이 출력을 리스트에 저장\n",
    "    state_t = h_t  # 다음 타임스텝을 위해 네트워크의 상태를 업데이트\n",
    "\n",
    "# 최종 출력은 형태가 (timestpes, output_features) = (100, 64)\n",
    "final_output_sequence = np.stack(outputs, axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(100, 64)"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "final_output_sequence.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Using TensorFlow backend.\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "'2.2.4'"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import keras\n",
    "\n",
    "keras.__version__"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.2.1 케라스의 순환 층 - `SimpleRNN`\n",
    "\n",
    "넘파이로 간단하게 구현한 과정이 실제 케라스의 [`SimpleRNN`](https://keras.io/layers/recurrent/#simplernn) 층에 해당합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from keras.layers import SimpleRNN"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "케라스에 있는 모든 순환 층과 동일하게 SimpleRNN은 두 가지 모드로 실행할 수 있습니다. \n",
    "\n",
    "- 입력 시퀀스에 대한 마지막 출력만 반환할 수 있습니다(크기가 `(batch_size, output_features)` 인 2D 텐서). \n",
    "- 각 타임스텝의 출력을 모은 전체 시퀀스를 반환하거나(크기가 `(batch_size, timesteps, output_features)` 인 3D 텐서)\n",
    "    - 이 모드는 객체를 생성할 때 `return_sequences` 매개변수로 선택할 수 있습니다. \n",
    "    - return_sequences: Boolean. Whether to return the last output\n",
    "        in the output sequence, or the full sequence.\n",
    "\n",
    "예제를 살펴보죠:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 마지막 타임스텝의 출력만 얻기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_1 (SimpleRNN)     (None, 32)                2080      \n",
      "=================================================================\n",
      "Total params: 322,080\n",
      "Trainable params: 322,080\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Embedding, SimpleRNN\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32))  # 32 * 32 * 2(W_x, W_h) + 32\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### 전체 상태 시퀀스를 얻기"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_1 (SimpleRNN)     (None, None, 32)          2080      \n",
      "=================================================================\n",
      "Total params: 322,080\n",
      "Trainable params: 322,080\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "from keras import backend as K\n",
    "\n",
    "K.clear_session()\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "네트워크의 표현력을 증가시키기 위해 여러 개의 순환 층을 차례대로 쌓는 것이 유용할 때가 있습니다. \n",
    "- 이런 설정에서는 **중간 층들이 전체 출력 시퀀스를 반환하도록 설정**해야 합니다:\n",
    "\n",
    "![](./images/deep-rnn.PNG)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "simple_rnn_1 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_2 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_3 (SimpleRNN)     (None, None, 32)          2080      \n",
      "_________________________________________________________________\n",
      "simple_rnn_4 (SimpleRNN)     (None, 32)                2080      \n",
      "=================================================================\n",
      "Total params: 328,320\n",
      "Trainable params: 328,320\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n"
     ]
    }
   ],
   "source": [
    "K.clear_session()\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(10000, 32))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32, return_sequences=True))\n",
    "model.add(SimpleRNN(32))  # 맨 위 층만 마지막 출력만 반환\n",
    "model.summary()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### IMDB 영화 리뷰 분류 문제에 적용하기\n",
    "\n",
    "이제 IMDB 영화 리뷰 분류 문제에 적용해 보죠. 먼저 데이터를 전처리합니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "데이터 로딩...\n",
      "25000 훈련 시퀀스\n",
      "25000 테스트 시퀀스\n",
      "시퀀스 패딩 (samples x time)\n",
      "train_inputs shape : (25000, 500)\n",
      "test_inputs shape : (25000, 500)\n"
     ]
    }
   ],
   "source": [
    "from keras.datasets import imdb\n",
    "from keras.preprocessing import sequence\n",
    "\n",
    "max_features = 10000  # 특성으로 사용할 단어의 수\n",
    "maxlen = 500  # 사용할 텍스트의 길이(가장 빈번한 max_features 개의 단어만 사용)\n",
    "batch_size = 32\n",
    "\n",
    "print('데이터 로딩...')\n",
    "(train_inputs, train_labels), (test_inputs, test_labels) = imdb.load_data(num_words=max_features)\n",
    "print(len(train_inputs), '훈련 시퀀스')\n",
    "print(len(test_inputs), '테스트 시퀀스')\n",
    "\n",
    "print('시퀀스 패딩 (samples x time)')\n",
    "train_inputs = sequence.pad_sequences(train_inputs, maxlen=maxlen)\n",
    "test_inputs = sequence.pad_sequences(test_inputs, maxlen=maxlen)\n",
    "print('train_inputs shape :', train_inputs.shape)\n",
    "print('test_inputs shape :', test_inputs.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`Embedding` 층과 `SimpleRNN` 층을 사용해 간단한 순환 네트워크를 훈련시켜 보겠습니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 13s 670us/step - loss: 0.6435 - acc: 0.6130 - val_loss: 0.6037 - val_acc: 0.6376\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 12s 588us/step - loss: 0.4152 - acc: 0.8223 - val_loss: 0.4209 - val_acc: 0.8230\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 12s 588us/step - loss: 0.3163 - acc: 0.8721 - val_loss: 0.3826 - val_acc: 0.8232\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 12s 588us/step - loss: 0.2653 - acc: 0.8972 - val_loss: 0.3976 - val_acc: 0.8210\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 12s 589us/step - loss: 0.2131 - acc: 0.9205 - val_loss: 0.4215 - val_acc: 0.8030\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 12s 590us/step - loss: 0.1752 - acc: 0.9339 - val_loss: 0.3799 - val_acc: 0.8568\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 12s 589us/step - loss: 0.1431 - acc: 0.9497 - val_loss: 0.4237 - val_acc: 0.8314\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 12s 589us/step - loss: 0.1095 - acc: 0.9628 - val_loss: 0.4891 - val_acc: 0.8224\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 12s 588us/step - loss: 0.0873 - acc: 0.9694 - val_loss: 0.5085 - val_acc: 0.8222\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 12s 591us/step - loss: 0.0544 - acc: 0.9835 - val_loss: 0.5891 - val_acc: 0.8030\n"
     ]
    }
   ],
   "source": [
    "from keras.layers import Dense\n",
    "\n",
    "K.clear_session()\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_features, 32))\n",
    "model.add(SimpleRNN(32))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "\n",
    "model.compile(optimizer='rmsprop', \n",
    "              loss='binary_crossentropy', \n",
    "              metrics=['acc'])\n",
    "history = model.fit(train_inputs, train_labels,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "이제 훈련과 검증의 손실과 정확도를 그래프로 그립니다:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3XmcFNW5//HPI4usIsIYFWTRoOybE9ALriwOLhB3EBOXKFevmlyX3GuiV70kJv7iEo0/s2B+uERkIBIUIktQMcSdQVkEVBABB1AHRERRYeD5/XFqmJ5mlp6hZ7qn+/t+vfo13VWnqp+umXnq9DmnTpm7IyIi2eGAVAcgIiJ1R0lfRCSLKOmLiGQRJX0RkSyipC8ikkWU9EVEsoiSfhYyswZm9qWZdUhm2VQys++aWdLHH5vZUDNbG/P6PTM7MZGyNXivP5vZz2u6vUgiGqY6AKmamX0Z87IZ8C2wO3r97+4+qTr7c/fdQItkl80G7n5sMvZjZlcCl7j7KTH7vjIZ+xapjJJ+PeDue5NuVJO80t2fr6i8mTV09+K6iE2kKvp7TC9q3skAZvZLM5tiZpPNbDtwiZmdYGavm9nnZrbJzH5nZo2i8g3NzM2sU/T6yWj9bDPbbmavmVnn6paN1o8ws/fNbJuZPWRmr5jZZRXEnUiM/25mq81sq5n9LmbbBmb2WzPbYmZrgLxKjs+tZpYft+xhM7s/en6lma2MPs8HUS28on0Vmtkp0fNmZvaXKLblwHFxZW8zszXRfpeb2choeS/g/wInRk1nm2OO7Z0x218dffYtZvaMmR2eyLGpznEuicfMnjezz8zsYzP7r5j3+Z/omHxhZgVmdkR5TWlm9nLJ7zk6ngui9/kMuM3MupjZ/Og9NkfHrVXM9h2jz1gUrX/QzJpEMXeLKXe4me0wszYVfV6pgrvrUY8ewFpgaNyyXwI7gbMJJ/KmwPeAgYRvc0cB7wPXReUbAg50il4/CWwGcoFGwBTgyRqUPRTYDoyK1t0I7AIuq+CzJBLjs0AroBPwWclnB64DlgPtgTbAgvDnXO77HAV8CTSP2fenQG70+uyojAGnAV8DvaN1Q4G1MfsqBE6Jnt8LvAS0BjoCK+LKXggcHv1OLo5i+E607krgpbg4nwTujJ4Pj2LsCzQBfg+8mMixqeZxbgV8AvwEOBA4CBgQrfsZsAToEn2GvsAhwHfjjzXwcsnvOfpsxcA1QAPC3+MxwBCgcfR38gpwb8zneSc6ns2j8oOidROAu2Le5yZgeqr/D+vzI+UB6FHNX1jFSf/FKra7Gfhr9Ly8RP7HmLIjgXdqUPYK4F8x6wzYRAVJP8EYj49Z/zfg5uj5AkIzV8m6M+ITUdy+Xwcujp6PAN6rpOzfgWuj55Ul/fWxvwvgP2LLlrPfd4Azo+dVJf3HgV/FrDuI0I/TvqpjU83j/ANgYQXlPiiJN255Ikl/TRUxnF/yvsCJwMdAg3LKDQI+BCx6vRg4N9n/V9n0UPNO5vgo9oWZdTWz56Kv618A44G2lWz/cczzHVTeeVtR2SNi4/DwX1pY0U4SjDGh9wLWVRIvwFPAmOj5xdHrkjjOMrM3oqaHzwm17MqOVYnDK4vBzC4zsyVRE8XnQNcE9wvh8+3dn7t/AWwF2sWUSeh3VsVxPpKQ3MtT2bqqxP89HmZmU81sQxTDY3ExrPUwaKAMd3+F8K1hsJn1BDoAz9UwJkFt+pkkfrjinwg1y++6+0HA7YSad23aRKiJAmBmRtkkFW9/YtxESBYlqhpSOhUYambtCM1PT0UxNgWeBn5NaHo5GPhHgnF8XFEMZnYU8AdCE0ebaL/vxuy3quGlGwlNRiX7a0loRtqQQFzxKjvOHwFHV7BdReu+imJqFrPssLgy8Z/v/xBGnfWKYrgsLoaOZtaggjieAC4hfCuZ6u7fVlBOEqCkn7laAtuAr6KOsH+vg/f8O9DfzM42s4aEduKcWopxKvCfZtYu6tT778oKu/vHhCaIxwhNO6uiVQcS2pmLgN1mdhah7TnRGH5uZgdbuI7huph1LQiJr4hw/ruKUNMv8QnQPrZDNc5k4Edm1tvMDiSclP7l7hV+c6pEZcd5BtDBzK4zswPN7CAzGxCt+zPwSzM72oK+ZnYI4WT3MWHAQAMzG0fMCaqSGL4CtpnZkYQmphKvAVuAX1noHG9qZoNi1v+F0Bx0MeEEIPtBST9z3QRcSuhY/ROhw7VWufsnwEXA/YR/4qOBtwk1vGTH+AfgBWAZsJBQW6/KU4Q2+r1NO+7+OXADMJ3QGXo+4eSViDsI3zjWArOJSUjuvhR4CHgzKnMs8EbMtvOAVcAnZhbbTFOy/RxCM8z0aPsOwNgE44pX4XF2923AMOA8wonofeDkaPU9wDOE4/wFoVO1SdRsdxXwc0Kn/nfjPlt57gAGEE4+M4BpMTEUA2cB3Qi1/vWE30PJ+rWE3/O37v5qNT+7xCnpHBFJuujr+kbgfHf/V6rjkfrLzJ4gdA7fmepY6jtdnCVJZWZ5hJEyXxOG/O0i1HZFaiTqHxkF9Ep1LJlAzTuSbIOBNYS27NOBc9TxJjVlZr8mXCvwK3dfn+p4MkGVzTtmNpHQ3vapu/csZ70BDxLGSe8gjNV9K1p3KXBbVPSX7v54EmMXEZFqSqSm/xiVXOJOuNClS/QYR+hgI+rlv4NwJeAA4A4za70/wYqIyP6psk3f3RdYNO9KBUYBT0Q9+q9Hw9cOB04B5rn7ZwBmNo9w8phc2fu1bdvWO3Wq7O1ERCTeokWLNrt7ZUOkgeR05Laj7NV3hdGyipbvIxrnOw6gQ4cOFBQUJCEsEZHsYWZVXZUOpElHrrtPcPdcd8/NyanyRCUiIjWUjKS/gbKXorePllW0XEREUiQZSX8G8MPoMu3jgW3uvgmYCww3s9ZRB+7waJmIiKRIlW36ZjaZ0Cnb1swKCSNyGgG4+x+BWYThmqsJQzYvj9Z9Zma/IFwiDzC+pFO3unbt2kVhYSHffPNNTTaXOtKkSRPat29Po0YVTScjIqmWdtMw5ObmenxH7ocffkjLli1p06YN4bIASTfuzpYtW9i+fTudO3euegMRSSozW+TuuVWVS4uO3Kp88803Svhpzsxo06aNvo2J1MCkSdCpExxwQPg5aVLtvVe9mXtHCT/96XckUn2TJsG4cbBjR3i9bl14DTC2pvOqVqJe1PRFRDLVrbeWJvwSO3aE5bVBST8BW7ZsoW/fvvTt25fDDjuMdu3a7X29c+fOhPZx+eWX895771Va5uGHH2ZSbX6vE5G0s76CaeQqWr6/6k3zTnVMmhTOkuvXQ4cOcNdd+/c1qU2bNixevBiAO++8kxYtWnDzzTeXKbP3psMHlH8effTRR6t8n2uvvbbmQYpIvdShQ2jSKW95bci4mn5J+9i6deBe2j5WGxXo1atX0717d8aOHUuPHj3YtGkT48aNIzc3lx49ejB+/Pi9ZQcPHszixYspLi7m4IMP5pZbbqFPnz6ccMIJfPrppwDcdtttPPDAA3vL33LLLQwYMIBjjz2WV18NNwz66quvOO+88+jevTvnn38+ubm5e09Ise644w6+973v0bNnT66++mpKRmm9//77nHbaafTp04f+/fuzdu1aAH71q1/Rq1cv+vTpw6219b1SRPZx113QrFnZZc2aheW1IeOSfl23j7377rvccMMNrFixgnbt2nH33XdTUFDAkiVLmDdvHitWrNhnm23btnHyySezZMkSTjjhBCZOnFjuvt2dN998k3vuuWfvCeShhx7isMMOY8WKFfzP//wPb7/9drnb/uQnP2HhwoUsW7aMbdu2MWfOHADGjBnDDTfcwJIlS3j11Vc59NBDmTlzJrNnz+bNN99kyZIl3HTTTUk6OiJSlbFjYcIE6NgRzMLPCRNqpxMXMjDp13X72NFHH01ubunQ2MmTJ9O/f3/69+/PypUry036TZs2ZcSIEQAcd9xxe2vb8c4999x9yrz88suMHj0agD59+tCjR49yt33hhRcYMGAAffr04Z///CfLly9n69atbN68mbPPPhsIF1M1a9aM559/niuuuIKmTZsCcMghh1T/QIhIjY0dC2vXwp494WdtJXzIwKRfUTtYbbWPNW/efO/zVatW8eCDD/Liiy+ydOlS8vLyyh233rhx473PGzRoQHFxcbn7PvDAA6ssU54dO3Zw3XXXMX36dJYuXcoVV1yh8fMi5ajL8fHpIuOSfl23j8X64osvaNmyJQcddBCbNm1i7tzkTzU0aNAgpk6dCsCyZcvK/Sbx9ddfc8ABB9C2bVu2b9/OtGnTAGjdujU5OTnMnDkTCBe97dixg2HDhjFx4kS+/vprAD77rEazZYjUK3XZ/5dOMi7p13X7WKz+/fvTvXt3unbtyg9/+EMGDRqU9Pe4/vrr2bBhA927d+d///d/6d69O61atSpTpk2bNlx66aV0796dESNGMHDgwL3rJk2axH333Ufv3r0ZPHgwRUVFnHXWWeTl5ZGbm0vfvn357W9/m/S4RdJNXff/pYt6MffOypUr6datW4oiSi/FxcUUFxfTpEkTVq1axfDhw1m1ahUNG6bH6Fv9riQRyR5WXRMHHBBq+PHMQtt6fZPo3DvpkSkkYV9++SVDhgyhuLgYd+dPf/pT2iR8kUTU9bQDFanr8fHpQtminjn44INZtGhRqsMQqbHKmlXqMunfdVfZkw/UXf9fKmVcm76IpLe6HlZdkVT2/6WSavoiUqfSqVll7NjMT/LxVNMXkTqVymHVoqQvInUsW5tV0kVCSd/M8szsPTNbbWa3lLO+o5m9YGZLzewlM2sfs263mS2OHjOSGXxdOfXUU/e50OqBBx7gmmuuqXS7Fi1aALBx40bOP//8csuccsopxA9RjffAAw+wI6a36YwzzuDzzz9PJHSRfaTDVah1Oe2AlFVl0jezBsDDwAigOzDGzLrHFbsXeMLdewPjgV/HrPva3ftGj5FJirtOjRkzhvz8/DLL8vPzGTNmTELbH3HEETz99NM1fv/4pD9r1iwOPvjgGu9Psle2XoUqpRKp6Q8AVrv7GnffCeQDo+LKdAdejJ7PL2d9vXb++efz3HPP7b1hytq1a9m4cSMnnnji3nHz/fv3p1evXjz77LP7bL927Vp69uwJhCkSRo8eTbdu3TjnnHP2Tn0AcM011+ydlvmOO+4A4He/+x0bN27k1FNP5dRTTwWgU6dObN68GYD777+fnj170rNnz73TMq9du5Zu3bpx1VVX0aNHD4YPH17mfUrMnDmTgQMH0q9fP4YOHconn3wChGsBLr/8cnr16kXv3r33TuMwZ84c+vfvT58+fRgyZEhSjq3UrWy9ClVKJTJ6px3wUczrQmBgXJklwLnAg8A5QEsza+PuW4AmZlYAFAN3u/sz8W9gZuOAcQAdqujC/8//hHKmj98vfftClC/LdcghhzBgwABmz57NqFGjyM/P58ILL8TMaNKkCdOnT+eggw5i8+bNHH/88YwcObLC+8X+4Q9/oFmzZqxcuZKlS5fSv3//vevuuusuDjnkEHbv3s2QIUNYunQpP/7xj7n//vuZP38+bdu2LbOvRYsW8eijj/LGG2/g7gwcOJCTTz6Z1q1bs2rVKiZPnswjjzzChRdeyLRp07jkkkvKbD948GBef/11zIw///nP/OY3v+G+++7jF7/4Ba1atWLZsmUAbN26laKiIq666ioWLFhA586dNT9PPZUuwyUldZLVkXszcLKZvQ2cDGwAdkfrOkaXBl8MPGBmR8dv7O4T3D3X3XNzcnKSFFJyxTbxxDbtuDs///nP6d27N0OHDmXDhg17a8zlWbBgwd7k27t3b3r37r133dSpU+nfvz/9+vVj+fLl5U6mFuvll1/mnHPOoXnz5rRo0YJzzz2Xf/3rXwB07tyZvn37AhVP31xYWMjpp59Or169uOeee1i+fDkAzz//fJm7eLVu3ZrXX3+dk046ic6dOwOafrm+qutZaCX9JFLT3wAcGfO6fbRsL3ffSKjpY2YtgPPc/fNo3Ybo5xozewnoB3xQ04Arq5HXplGjRnHDDTfw1ltvsWPHDo477jggTGBWVFTEokWLaNSoEZ06darRNMYffvgh9957LwsXLqR169Zcdtll+zUdcsm0zBCmZi6veef666/nxhtvZOTIkbz00kvceeedNX4/qR+y9SpUKZVITX8h0MXMOptZY2A0UGYUjpm1NbOSff0MmBgtb21mB5aUAQYBlVdf01SLFi049dRTueKKK8p04G7bto1DDz2URo0aMX/+fNaVd9VJjJNOOomnnnoKgHfeeYelS5cCYVrm5s2b06pVKz755BNmz569d5uWLVuyffv2ffZ14okn8swzz7Bjxw6++uorpk+fzoknnpjwZ9q2bRvt2rUD4PHHH9+7fNiwYTz88MN7X2/dupXjjz+eBQsW8OGHHwKafrm+0nBJqTLpu3sxcB0wF1gJTHX35WY23sxKRuOcArxnZu8D3wFK6g3dgAIzW0Lo4L3b3etl0ofQxLNkyZIySX/s2LEUFBTQq1cvnnjiCbp27VrpPq655hq+/PJLunXrxu233773G0OfPn3o168fXbt25eKLLy4zLfO4cePIy8vb25Fbon///lx22WUMGDCAgQMHcuWVV9KvX7+EP8+dd97JBRdcwHHHHVemv+C2225j69at9OzZkz59+jB//nxycnKYMGEC5557Ln369OGiiy5K+H0kvWi4ZHbT1MqSVPpdiaRGolMr64pckTqSDhdFiWjCNZE6kC5zyIvUm5p+ujVDyb70O6qYLoqSdFEvkn6TJk3YsmWLkkoac3e2bNlCkyZNUh1KWtJFUZIu6kXzTvv27SksLKSoqCjVoUglmjRpQvv27asumIXSaQ55yW71Iuk3atRo75WgIvWRLoqSdFEvmndE6jtdFCXpQklfMl66DJXURVGSDupF845ITWmopEhZqulLRtNQSZGylPQlo2mopEhZSvqS0TR/vEhZSvqS0e66KwyNjKWhkpLNlPQlo2mopEhZGr0jGW/sWCV5kRKq6YuIZBElfRGRLKKkLyKSRRJK+maWZ2bvmdlqM7ulnPUdzewFM1tqZi+ZWfuYdZea2arocWkygxcRkeqpMumbWQPgYWAE0B0YY2bd44rdCzzh7r2B8cCvo20PAe4ABgIDgDvMrHXywpd0li5z3ohIqURq+gOA1e6+xt13AvnAqLgy3YEXo+fzY9afDsxz98/cfSswD8jb/7Al3ZXMebNuHbiXznmjxC+SWokk/XbARzGvC6NlsZYA50bPzwFamlmbBLfFzMaZWYGZFehGKZlBc96IpKdkdeTeDJxsZm8DJwMbgN2JbuzuE9w9191zc3JykhSSpJLmvBFJT4kk/Q3AkTGv20fL9nL3je5+rrv3A26Nln2eyLaSmTTnjUh6SiTpLwS6mFlnM2sMjAZmxBYws7ZmVrKvnwETo+dzgeFm1jrqwB0eLZMMpzlvRNJTlUnf3YuB6wjJeiUw1d2Xm9l4MxsZFTsFeM/M3ge+A9wVbfsZ8AvCiWMhMD5aJhlOc96IpCdz91THUEZubq4XFBSkOgwRkXrFzBa5e25V5XRFrohIFlHSFxHJIkr6IiJZRElfRCSLKOmLiGQRJX0RkSyipJ+BNLuliFRE98jNMCWzW5ZMdlYyuyXowigRUU0/42h2SxGpjJJ+htHsliJSGSX9DKPZLUWkMkr6GUazW4pIZZT0M4xmtxSRymj0TgYaO1ZJXkTKp5q+iEgWUdIXEckiSvoidWTVKjjhBLjoInjhBdizJ9URSTZKKOmbWZ6ZvWdmq83slnLWdzCz+Wb2tpktNbMzouWdzOxrM1scPf6Y7A8gUh/8859w/PHw/vvw/PMwdCgceyzccw98+mmqo5NsUmXSN7MGwMPACKA7MMbMuscVu41w79x+hBun/z5m3Qfu3jd6XJ2kuEXqjcceg2HD4NBD4c03YcMGePJJOPxw+K//gvbtYfRoePFFSLO7l0oGSqSmPwBY7e5r3H0nkA+MiivjwEHR81bAxuSFKFI/7dkDP/85XH45nHQSvPYaHH00NGkSRlctWADLl8O118I//gFDhpTW/ouKUh29ZKpEkn474KOY14XRslh3ApeYWSEwC7g+Zl3nqNnnn2Z2YnlvYGbjzKzAzAqK9NcuGWDHjtB2/+tfhwnvZs+Ggw/et1z37vDb34ba/1/+At/5Tmntf8wYmD9ftX9JrmR15I4BHnP39sAZwF/M7ABgE9Ahava5EXjKzA6K39jdJ7h7rrvn5uTkJCkkkdTYtAlOOQWmTYP77oM//hEaNap8m6ZN4ZJL4F//gnfegWuugTlz4LTTQu3/3nth8+Y6CV8yXCJJfwNwZMzr9tGyWD8CpgK4+2tAE6Ctu3/r7lui5YuAD4Bj9jdokXS1dCkMHBiabZ55Bm68MVwZXR09esADD8DGjfD446Ev4Kc/hXbt4OKL4aWXVPuXmksk6S8EuphZZzNrTOionRFXZj0wBMDMuhGSfpGZ5UQdwZjZUUAXYE2yghdJJ889B4MGhbb8l1+GkSP3b39Nm8IPfxj2tWwZXH11aCY69VTo2jV8i1DtX6qryqTv7sXAdcBcYCVhlM5yMxtvZiV/1jcBV5nZEmAycJm7O3ASsNTMFgNPA1e7+2e18UFEUsUdfve7kOSPOQbeeAP69Uvue/TsCQ8+GNr+H3sM2raFm28Otf+xY8OQUNX+JRHmafaXkpub6wUFBakOQyQhxcXwk5/A738P3/9+GIrZvHndvPeyZfDII/DEE7BtW2j7HzcOLr0U2rSpmxgkfZjZInfPraqcrsgVqaFt2+Dss0PC/+lPQ8dtXSV8gF69wjeMjRvh0UfhkEPgpptC7f+SS8KQ0DSr00kaUNIXqYG1a0P7/fPPh9r2b34TbkSfCs2awWWXwauvho7kq66Cv/8dTj65dEjoZ2pUlYiSvkg1vf56GKGzYUMYVnnllamOqFSvXvDQQ6H2P3FiuDbgxhvhiCPgBz8IQ0JV+89uSvoi1ZCfH8bgt2wZrrAdMiTVEZWvWbNwJfBrr8HixeHENGNGuDK4Rw+4//6wrqhIJ4Fso45ckQS4wy9/CbffDoMHw/TpYQRNffLVVzBlSriT2htvlC5v1Qq++93w6NKl7POcnOpfZyCpkWhHrpK+SBW+/TbUlJ98MjSRPPIIHHhgqqPaP2vWwMqVsHp1mPJ59erwWLsWdu8uLdeyZfkng+9+N0wZoRNC+kg06et2iUk0aRLceiusXw8dOoSbkeu2hfXb5s1wzjnhAqlf/jJMoJYJie6oo8Ij3s6dsG5d2RPBqlXw1lthdFLsCaFFi9ITQfyJ4fDDM+M4ZSIl/SSZNCmMkd6xI7xety68BiX++urdd+HMM0On6JQpcOGFqY6o9jVuHJJ3ly77rtu1K/xdx54MVq8OI4aeeSZcs1CiWbOKvyEcfnjqRjqJmneSplOn8A8Rr0GDMG7avbTDrOR5bS874IAwjW+TJuGS/ooela2vzrYHHpg5tbsXXoDzzgufacaMMFpHKlZcHL7hxjcXrVoVmpJ27Sot27Rp6YmgY0do2DD83ZT87ZQ8r8tlbdrA6aeHJqv6Ss07dWz9+vKX794d5kpJxh9mdbffswe++Qa+/rr0Z8lj2zb4+OPy19X0Nn5mZU8CJc+PPBKGD4cRI8I0Bel+YnjkEfiP/wjz28ycGU7oUrmGDUubjIYPL7tu92746KN9Twbvvgvz5oX11a3U1AazcHI/++wwpUaPHun/t1oTquknSUU1/Y4dQ+dYfeEeamWxJ4H4k0L8o7L133wTOgzffTfsv3NnyMsLJ4DTTqvbK1irsns3/Pd/h4nM8vJCk85B+0wELukimd+M164NJ/gZM2DhwrCsc+fSE8CJJ4amr3Sm0Tt1bNIk+NGPwkiPEs2aheFxatMP/1Rz5oRZIl94IQwfbNw4/DONGBEe3bqlrmb11Vfh9/Tss3DddeEq1ob6HpyVNm0KVzTPnBm+iXzzTTj5jxgRTgIjRoQpL9KNkn4KnHkmzJoVnnfsqNE7Ffn22zAapuQksHx5WN6hQ+m3gCFDwnDBurBhQ/hnXrIkzGN//fVVbyPZYceOMNXGzJnh8cknoZ9u8ODwDeDss8vv9E4FJf06tmdPaOLp3TvUEiRx69eHE8CcOeEfbPv2UMsePLj0W0DPnrXzLeCtt8I/7hdfhOacM85I/ntIZtizBwoKQhPQzJlh1BKEvp+SZqATTggnhVRQ0q9jr74aJuD6y1/CDIdSMzt3hmM5e3Y4CZT8Y7VrV/otYOjQcBXp/nr22XAnqrZtw4m6V6/936dkj7Vrw9/NjBnhbma7doVRQGeeGU4Aw4fX3bdVUNKvcz/+cWi///RTdf4lU8mkZnPmhPbVbdtCTerf/i2cAPLyoG/f6n0LcA9zz/z0p/C974Xkf9hhtfcZJPN98QXMnRtOALNmhVlNGzcOI/fOPjs8OnSo3RiU9OvQ7t3Qvn1IRNOmpTqazLVrV5jhsuRbwNtvh+WHHRaSf15eqF21bl35Pq69NgzLvOCCcA/apk3rJn7JDsXF4dtqyWig998Py/v0Ke0HOO645F+gpqRfh+bPD8MPs+WqzXSxaVOoXc2ZA//4B2zdGv6Rjj++9FtA//6l/1xbt4ZE/8ILYbqM8eN1ZajUvvfeKz0BvPJK6Bs4/HA466xwEhgyJDkVj6QmfTPLAx4EGgB/dve749Z3AB4HDo7K3OLus6J1PwN+BOwGfuzucyt7r/qY9K++OkzG9ckn6TXuPJsUF8Obb5Z+Cyj5Ezr00HCl5SmnwD33wAcfhFr+pZemNFzJUlu2hOafmTPD3+n27SHhDxsWTgBnnlnzpsakJX0zawC8DwwDCoGFwBh3XxFTZgLwtrv/wcy6A7PcvVP0fDIwADgCeB44xt13x79PifqW9HftCmftYcNg8uRURyMlPv00fAuYPTt8C9iyJYytnj49zCkvkmo7d4Yb2peMBlq3LlwF/M47NdtfMqdhGACsdvc10Y7zgVHAipgyDpR0X7YCNkbPRwH57v4t8KGZrY7291pCn6IeePHFkFBGj051JBLr0EPDNMg/+EHoc1m8OEylPnoJAAAPPUlEQVQHceihqY5MJGjcOFQWhw0L9zpetizM6lrbEkn67YCPYl4XAvHTT90J/MPMrgeaA0Njtn09btt28W9gZuOAcQAdaruLO8ny88Nonby8VEciFWnQIHSciaQrs3CNT11IVjfWGOAxd28PnAH8xcwS3re7T3D3XHfPzcnJSVJIte/bb0NzwTnn1P+baohIdkikpr8BODLmdftoWawfAXkA7v6amTUB2ia4bb01d24YN37RRamOREQkMYnUxhcCXcyss5k1BkYDM+LKrAeGAJhZN6AJUBSVG21mB5pZZ6AL8Gaygk+1KVNC5+DQoVWXFRFJB1XW9N292MyuA+YShmNOdPflZjYeKHD3GcBNwCNmdgOhU/cyD8OClpvZVEKnbzFwbWUjd+qTHTtKL+Nv1CjV0YiIJCahyWOjMfez4pbdHvN8BTCogm3vAu7ajxjT0qxZYTpejdoRkfpE1yPWUH5+uLXaySenOhIRkcQp6dfA9u3w3HPhkv5UTaMqIlITSvo1MGNGuJuORu2ISH2jpF8DU6aUzqopIlKfKOlX09atYaKkCy/UDI0iUv8obVXTM8+ESdY0akdE6iMl/WrKz4ejjoLcKueyExFJP0r61VBUFG7AcdFFtXOTbhGR2qakXw3TpoVpetW0IyL1lZJ+NeTnQ9eu0KtXqiMREakZJf0EbdwICxaEWr6adkSkvlLST9Bf/wruuiBLROo3Jf0ETZkCffqE5h0RkfpKST8B69bBa6+pli8i9Z+SfgKmTg0/lfRFpL5T0k9Afj5873vhoiwRkfpMSb8Kq1bBW29pbL6IZAYl/SpMmRJ+XnBBauMQEUmGhJK+meWZ2XtmttrMbiln/W/NbHH0eN/MPo9ZtztmXfwN1dPelCkweDAceWSqIxER2X9V3iPXzBoADwPDgEJgoZnNiO6LC4C73xBT/nqgX8wuvnb3vskLue4sXw7vvAMPPZTqSEREkiORmv4AYLW7r3H3nUA+MKqS8mOAyckILtWmTAlz5p9/fqojERFJjkSSfjvgo5jXhdGyfZhZR6Az8GLM4iZmVmBmr5vZ9yvYblxUpqCoqCjB0GuXexi1c8opcNhhqY5GRCQ5kt2ROxp42t13xyzr6O65wMXAA2Z2dPxG7j7B3XPdPTcnJyfJIdXM4sVh5I5G7YhIJkkk6W8AYrsx20fLyjOauKYdd98Q/VwDvETZ9v60lZ8PDRvCueemOhIRkeRJJOkvBLqYWWcza0xI7PuMwjGzrkBr4LWYZa3N7MDoeVtgELAiftt04x7a84cNgzZtUh2NiEjyVJn03b0YuA6YC6wEprr7cjMbb2YjY4qOBvLd3WOWdQMKzGwJMB+4O3bUT7p6440w346mXRCRTFPlkE0Ad58FzIpbdnvc6zvL2e5VoN7dcmTKFGjcGL5fbreziEj9pSty4+zZEyZYGzECWrVKdTQiIsmlpB/n5ZfDXbI0akdEMpGSfpz8fGjaFM46K9WRiIgkn5J+jOJiePppOPtsaNEi1dGIiCSfkn6M+fOhqEijdkQkcynpx5gyBVq2DJ24IiKZSEk/snMnTJsGo0aFNn0RkUykpB+ZNw8+/1yjdkQksynpR/LzoXXrMPWCiEimUtIHvv4ann02TK7WuHGqoxERqT1K+sDs2bB9u0btiEjmU9InjNrJyYFTT011JCIitSvrk/6XX8LMmeGWiA0Tmn5ORKT+yvqk//e/hzZ9jdoRkWyQ9Uk/Px+OOAIGD051JCIitS+rk/62baET98IL4YCsPhIiki2yOtU980y4ElejdkQkWySU9M0sz8zeM7PVZnZLOet/a2aLo8f7ZvZ5zLpLzWxV9Lg0mcHvr/x86NgRBg5MdSQiInWjyvEqZtYAeBgYBhQCC81sRuy9bt39hpjy1wP9oueHAHcAuYADi6Jttyb1U9TA5s3w/PNw441glupoRETqRiI1/QHAandf4+47gXxgVCXlxwCTo+enA/Pc/bMo0c8D8vYn4GT529/C/PkatSMi2SSRpN8O+CjmdWG0bB9m1hHoDLxY3W3r2pQp0KUL9O2b6khEROpOsjtyRwNPu/vu6mxkZuPMrMDMCoqKipIc0r4+/hheeinU8tW0IyLZJJGkvwE4MuZ1+2hZeUZT2rST8LbuPsHdc909NycnJ4GQ9s/TT8OePRq1IyLZJ5GkvxDoYmadzawxIbHPiC9kZl2B1sBrMYvnAsPNrLWZtQaGR8tSKj8fevaEHj1SHYmISN2qMum7ezFwHSFZrwSmuvtyMxtvZiNjio4G8t3dY7b9DPgF4cSxEBgfLUuZjz6CV15RLV9EslNCU4y5+yxgVtyy2+Ne31nBthOBiTWML+mmTg0/lfRFJBtl3RW5U6ZA//5h5I6ISLbJqqT/wQewcKHG5otI9sqqpF/StHPhhamNQ0QkVbIq6efnwwknhPl2RESyUdYk/ZUrYelSdeCKSHbLmqQ/ZUq4+vaCC1IdiYhI6mRF0ncPSf+kk8JdskREslVWJP2lS+HddzVqR0QkK5L+lCnQoAGcd16qIxERSa2MT/ruYdTOkCFQB3O5iYiktYxP+gUF8OGHatoREYEsSPr5+dCoEXz/+6mOREQk9TI66e/ZE67CzcuD1q1THY2ISOpldNJ/9VUoLNQFWSIiJTI66U+ZAk2awMiRVZcVEckGGZv0d++Gv/4VzjwTWrZMdTQiIukhY5P+P/8Jn3yiUTsiIrEyNunn50Pz5nDGGamOREQkfSSU9M0sz8zeM7PVZnZLBWUuNLMVZrbczJ6KWb7bzBZHj31uqF4bdu2CadNg1Cho1qwu3lFEpH6o8h65ZtYAeBgYBhQCC81shruviCnTBfgZMMjdt5rZoTG7+Nrd+yY57ko9/zx89plG7YiIxEukpj8AWO3ua9x9J5APjIorcxXwsLtvBXD3T5MbZvVMmQKtWsHpp6cyChGR9JNI0m8HfBTzujBaFusY4Bgze8XMXjezvJh1TcysIFpe7nWxZjYuKlNQVFRUrQ8Q75tvYPp0OOccOPDA/dqViEjGqbJ5pxr76QKcArQHFphZL3f/HOjo7hvM7CjgRTNb5u4fxG7s7hOACQC5ubm+P4HMnQtffKFROyIi5Umkpr8BODLmdftoWaxCYIa773L3D4H3CScB3H1D9HMN8BLQbz9jrlR+PrRpA6edVpvvIiJSPyWS9BcCXcyss5k1BkYD8aNwniHU8jGztoTmnjVm1trMDoxZPghYQS356iuYMQPOPz9MsiYiImVV2bzj7sVmdh0wF2gATHT35WY2Hihw9xnRuuFmtgLYDfzU3beY2b8BfzKzPYQTzN2xo36S7bnnYMcOjdoREamIue9XE3rS5ebmekFBQY22PfdceO21MMlagwZJDkxEJI2Z2SJ3z62qXMZckfvFFzBrFlxwgRK+iEhFkjV6J+W+/RauvRbGjEl1JCIi6Stjkn5ODtx3X6qjEBFJbxnTvCMiIlVT0hcRySJK+iIiWURJX0Qkiyjpi4hkESV9EZEsoqQvIpJFlPRFRLJIxiT9SZOgUyc44IDwc9KkVEckIpJ+MuKK3EmTYNy4MMMmwLp14TXA2LGpi0tEJN1kRE3/1ltLE36JHTvCchERKZURSX/9+uotFxHJVhmR9Dt0qN5yEZFslRFJ/667oFmzssuaNQvLRUSkVEYk/bFjYcIE6NgRzMLPCRPUiSsiEi+hpG9meWb2npmtNrNbKihzoZmtMLPlZvZUzPJLzWxV9Lg0WYHHGzsW1q6FPXvCTyV8EZF9VTlk08waAA8Dw4BCYKGZzYi9wbmZdQF+Bgxy961mdmi0/BDgDiAXcGBRtO3W5H8UERGpSiI1/QHAandf4+47gXxgVFyZq4CHS5K5u38aLT8dmOfun0Xr5gF5yQldRESqK5Gk3w74KOZ1YbQs1jHAMWb2ipm9bmZ51dgWMxtnZgVmVlBUVJR49CIiUi3J6shtCHQBTgHGAI+Y2cGJbuzuE9w9191zc3JykhSSiIjESyTpbwCOjHndPloWqxCY4e673P1D4H3CSSCRbUVEpI6Yu1dewKwhIYkPISTshcDF7r48pkweMMbdLzWztsDbQF+izlugf1T0LeA4d/+skvcrAtbV+BOlh7bA5lQHkUZ0PMrS8SilY1HW/hyPju5eZVNJlaN33L3YzK4D5gINgInuvtzMxgMF7j4jWjfczFYAu4GfuvsWADP7BeFEATC+soQfvV+9b98xswJ3z011HOlCx6MsHY9SOhZl1cXxqLKmL9WnP+SydDzK0vEopWNRVl0cj4y4IldERBKjpF87JqQ6gDSj41GWjkcpHYuyav14qHlHRCSLqKYvIpJFlPRFRLKIkn4SmdmRZjY/ZrbRn6Q6plQzswZm9raZ/T3VsaSamR1sZk+b2btmttLMTkh1TKlkZjdE/yfvmNlkM2uS6pjqkplNNLNPzeydmGWHmNm8aFbieWbWOtnvq6SfXMXATe7eHTgeuNbMuqc4plT7CbAy1UGkiQeBOe7eFehDFh8XM2sH/BjIdfeehGuARqc2qjr3GPtOQHkL8IK7dwFeiF4nlZJ+Ern7Jnd/K3q+nfBPvc8Ec9nCzNoDZwJ/TnUsqWZmrYCTgP8H4O473f3z1EaVcg2BptFV/82AjSmOp065+wIg/mLVUcDj0fPHge8n+32V9GuJmXUC+gFvpDaSlHoA+C9gT6oDSQOdgSLg0ai5689m1jzVQaWKu28A7gXWA5uAbe7+j9RGlRa+4+6boucfA99J9hso6dcCM2sBTAP+092/SHU8qWBmZwGfuvuiVMeSJhoS5qD6g7v3A76iFr661xdRW/UowsnwCKC5mV2S2qjSi4fx9EkfU6+kn2Rm1oiQ8Ce5+99SHU8KDQJGmtlawo13TjOzJ1MbUkoVAoXuXvLN72lKJyLMRkOBD929yN13AX8D/i3FMaWDT8zscIDo56dVlK82Jf0kMjMjtNmudPf7Ux1PKrn7z9y9vbt3InTQvejuWVuTc/ePgY/M7Nho0RBgRSWbZLr1wPFm1iz6vxlCFndsx5gBlNxL/FLg2WS/gZJ+cg0CfkCo1S6OHmekOihJG9cDk8xsKWHq8V+lOJ6Uib7xPE2Ybn0ZIRdl1ZQMZjYZeA041swKzexHwN3AMDNbRfg2dHfS31fTMIiIZA/V9EVEsoiSvohIFlHSFxHJIkr6IiJZRElfRCSLKOmLiGQRJX0RkSzy/wFol/wKqqSlIAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VNX5x/HPwyYiCAq4gRDcgCCCGKGKFFFrcQMXtCC4oBblJ0XrUlGxVdRWrXWhxYVa0AqK/rRW6lLan6CCK2ExCoggggRQFgVFUAg8vz/OBCYxJAOZ5E5uvu/Xa16Ze+fMvc/MwDNnzj2LuTsiIhIvNaIOQERE0k/JXUQkhpTcRURiSMldRCSGlNxFRGJIyV1EJIaU3KVEZlbTzNabWYt0lo2SmR1iZmnv+2tmJ5nZ4qTt+WbWLZWyu3Cux8zspl19finHvcPMHk/3cSU6taIOQNLDzNYnbdYDfgC2JLYvd/fxO3M8d98C1E932erA3Vun4zhmdhkwwN2PTzr2Zek4tsSfkntMuPu25JqoGV7m7v+3o/JmVsvdCyojNhGpfGqWqSYSP7ufMbOnzexbYICZHWNm75rZWjNbYWYjzax2onwtM3Mzy0psj0s8/qqZfWtm75hZq50tm3j8FDP7xMzWmdmfzewtM7t4B3GnEuPlZrbQzL42s5FJz61pZveb2RozWwT0LOX9udnMJhTbN8rM7kvcv8zM5iVez6eJWvWOjpVvZscn7tczsycTsc0BjipWdriZLUocd46Z9Ursbw/8BeiWaPJanfTe3pr0/CsSr32Nmf3TzPZP5b0pi5mdlYhnrZlNNrPWSY/dZGbLzewbM/s46bX+xMxmJvZ/aWZ/TPV8UgHcXbeY3YDFwEnF9t0BbALOIHyp7w4cDXQh/II7CPgEGJIoXwtwICuxPQ5YDeQAtYFngHG7UHYf4Fugd+Kxa4DNwMU7eC2pxPgi0BDIAr4qfO3AEGAO0BxoDLwZ/smXeJ6DgPXAHknHXgnkJLbPSJQx4ARgI3BE4rGTgMVJx8oHjk/cvxd4HdgLaAnMLVb2PGD/xGdyfiKGfROPXQa8XizOccCtifsnJ2LsCNQFHgImp/LelPD67wAeT9xvm4jjhMRndBMwP3G/HbAE2C9RthVwUOL+dKBf4n4DoEvU/xeq80019+plmrv/y923uvtGd5/u7u+5e4G7LwJGA91Lef5z7p7r7puB8YSksrNlTwdmu/uLicfuJ3wRlCjFGP/g7uvcfTEhkRae6zzgfnfPd/c1wF2lnGcR8BHhSwfgZ8DX7p6bePxf7r7Ig8nAa0CJF02LOQ+4w92/dvclhNp48nmfdfcVic/kKcIXc04KxwXoDzzm7rPd/XtgGNDdzJonldnRe1OavsBEd5+c+IzuInxBdAEKCF8k7RJNe58l3jsIX9KHmlljd//W3d9L8XVIBVByr16WJm+YWRsze9nMvjCzb4ARQJNSnv9F0v0NlH4RdUdlD0iOw92dUNMtUYoxpnQuQo2zNE8B/RL3z09sF8Zxupm9Z2ZfmdlaQq25tPeq0P6lxWBmF5vZB4nmj7VAmxSPC+H1bTueu38DfA00SyqzM5/Zjo67lfAZNXP3+cC1hM9hZaKZb79E0YFANjDfzN43s1NTfB1SAZTcq5fi3QAfJdRWD3H3PYHfEpodKtIKQjMJAGZmFE1GxZUnxhXAgUnbZXXVfBY4ycyaEWrwTyVi3B14DvgDocmkEfCfFOP4YkcxmNlBwMPAYKBx4rgfJx23rG6bywlNPYXHa0Bo/lmWQlw7c9wahM9sGYC7j3P3roQmmZqE9wV3n+/ufQlNb38CnjezuuWMRXaRknv11gBYB3xnZm2ByyvhnC8BnczsDDOrBVwFNK2gGJ8FrjazZmbWGLihtMLu/gUwDXgcmO/uCxIP7QbUAVYBW8zsdODEnYjhJjNrZGEcwJCkx+oTEvgqwvfcLwk190JfAs0LLyCX4GngUjM7wsx2IyTZqe6+w19COxFzLzM7PnHu6wnXSd4zs7Zm1iNxvo2J21bCC7jAzJokavrrEq9tazljkV2k5F69XQtcRPiP+yjhwmeFcvcvgV8A9wFrgIOBWYR++emO8WFC2/iHhIt9z6XwnKcIF0i3Ncm4+1rg18ALhIuSfQhfUqn4HeEXxGLgVeDvScfNA/4MvJ8o0xpIbqf+L7AA+NLMkptXCp//b0LzyAuJ57cgtMOXi7vPIbznDxO+eHoCvRLt77sB9xCuk3xB+KVwc+KppwLzLPTGuhf4hbtvKm88smssNHmKRMPMahKaAfq4+9So4xGJC9XcpdKZWc9EM8VuwC2EXhbvRxyWSKwouUsUjgMWEX7y/xw4y9131CwjIrtAzTIiIjGkmruISAxFNnFYkyZNPCsrK6rTi4hUSTNmzFjt7qV1HwYiTO5ZWVnk5uZGdXoRkSrJzMoaaQ2oWUZEJJaU3EVEYkjJXUQkhpTcRURiSMldRCSGqlRyHz8esrKgRo3wd/xOLfksIlJ9VJkFssePh0GDYMOGsL1kSdgG6F/uefBEROKlytTcb755e2IvtGFD2C8iIkVVmeT++ec7t19EpDqrMsm9xQ4WSNvRfhGR6qzKJPc774R69Yruq1cv7BcRkaKqTHLv3x9Gj4aWLcEs/B09WhdTRURKUmV6y0BI5ErmIiJlqzI1dxERSZ2Su4hIDCm5i4jEkJK7iEgMKbmLiMSQkruISAwpuYuIxFBKyd3MeprZfDNbaGbDdlDmPDOba2ZzzOyp9Ia53ZYt8N57FXV0EZF4KDO5m1lNYBRwCpAN9DOz7GJlDgVuBLq6ezvg6gqIFYDf/Q66dYNFiyrqDCIiVV8qNffOwEJ3X+Tum4AJQO9iZX4JjHL3rwHcfWV6w9xu8GCoVQtuuqmiziAiUvWlktybAUuTtvMT+5IdBhxmZm+Z2btm1rOkA5nZIDPLNbPcVatW7VLAzZrBtdfCM8/A++/v0iFERGIvXRdUawGHAscD/YC/mlmj4oXcfbS757h7TtOmTXf5ZL/5DeyzD1x3Hbjv8mFERGIrleS+DDgwabt5Yl+yfGCiu29298+ATwjJvkI0aAC33gpTp8LEiRV1FhGR9PvXv2Djxoo/TyrJfTpwqJm1MrM6QF+geEr9J6HWjpk1ITTTVOglz8sug9at4YYbYPPmijyTiEj5FRTANddAr17w4IMVf74yk7u7FwBDgEnAPOBZd59jZiPMrFei2CRgjZnNBaYA17v7mooKGqB2bbj7bpg/Hx57rCLPJCJSPqtWwcknw/33w69+Fa4bVjTziBqtc3JyPDc3t1zHcIfu3UOCX7gwNNeIiGSSmTPhrLPgyy/h0UfhoovKdzwzm+HuOWWVq9IjVM3g3nth5Uq4556ooxERKWr8eOjaFbZuhWnTyp/Yd0aVTu4AnTtD377wpz/BsuKXeUVEIlDYvj5gQMhRM2ZATpl17fSq8skd4Pe/D9MS3HJL1JGISHW3ejX8/Ofb29f/7/9C1+3KFovk3qoVDBkCjz8OeXlRRyMi1dWsWaGG/tZbMHYsjBwZOn9EIRbJHeDmm6FhwzDASUSksj31VGhf37IljMG5+OJo44lNct97bxg+HCZNgv/+N+poRKS6KGxf798fjj4acnPD36jFJrlDaJrJyoLrrw/fniIiFamk9vV99406qiBWyX233cLF1Q8+gHHjoo5GROIsk9rXSxKr5A7wi1+EN3z48MqZv0FEqp9Ma18vSeySe40aYWBTfj488EDU0YhInBQUhKkDMq19vSSxS+4QpiQ44wz4wx/CnA4iIuW1ejX07An33Zd57esliWVyhzCp2IYNMGJE1JGISFU3e3Zo7p02LTPb10sS2+Tetm2YFviRR+CTT6KORkSqqqeegmOPDU0ymdq+XpLYJncIC3rUrQvDhkUdiYhUNcnt6zk5YX6YTG1fL0msk/t++4URqy+8EH5OiYikIrl9fcgQeO21zG5fL0mskzuEkWP77x8GNmm9VREpy+zZoYZe2L7+5z9nfvt6SWKf3PfYA26/Hd59F557LupoRCSTPf10aF/fvLlqta+XJPbJHcIHdPjhcOONsGlT1NGISKYpKIDrroPzz6+a7eslqRbJvWbNsFLTp5/Cww9HHY2IZJI1a0L7+p/+VHXb10tSLZI7hA/vxBNDv/e1a6OORkQyQfH+61W1fb0k1Sa5m8Ef/whffx1GropI9Ran9vWSVJvkDnDkkWFNwwcfhCVLoo5GRKIQx/b1klSr5A5wxx3h7/Dh0cYhIpUvuX39yiszf36Y8qh2yb1FC7j66jDf+8yZUUcjIpVh61Z4551QU586FcaMgb/8BerUiTqyipNScjeznmY238wWmtmPBvOb2cVmtsrMZidul6U/1PS58UZo3Dj8NNPAJpF42bQpLKQxZkyYvbFrV9hzz6Lt6wMHRh1lxatVVgEzqwmMAn4G5APTzWyiu88tVvQZdx9SATGmXcOG8LvfwdCh8MorcNppUUckIrviu+/CymuzZoVf4rNmwUcfhSQOUL8+dOwIl1wSrrn16hUqdtVBmckd6AwsdPdFAGY2AegNFE/uVcrll4dpO3/zm7AGYq1U3gkRicxXXxVN4rNmwfz52399N2kSEvg114S/Rx4JhxwSFvCpjlJJac2ApUnb+UCXEsqdY2Y/BT4Bfu3uS4sXMLNBwCCAFi1a7Hy0aVSnDtx1F/TpE/q3/vKXkYYjIgnusHx50SQ+cyZ8/vn2MgceCJ06Qd++IYl36gTNmoUuzxKYl9HobGZ9gJ7uflli+wKgS3ITjJk1Bta7+w9mdjnwC3c/obTj5uTkeG5ubrlfQHm4w3HHwaJFsGBB+AknIpVn69Ywcjw5ic+atX0FNTM47LDtNfFOnUIzS5Mm0cYdJTOb4e45ZZVLpea+DDgwabt5Yt827r4mafMx4J5Ugoxa4cCmrl1D16jf/S7qiETia/NmmDevaBKfPRu+/TY8Xrs2tGsHp58ekviRR0KHDqp07apUkvt04FAza0VI6n2B85MLmNn+7r4isdkLmJfWKCvQscfCOeeEJH/55WEOeBFJj02bwoL1L7wAH34IP/wQ9terF2rgF164vVberh3stlu08cZJmcnd3QvMbAgwCagJjHH3OWY2Ash194nAUDPrBRQAXwEXV2DMafeHP8CLL4aa+6OPRh2NSDy8/z5cemnovdKtW+iWWFgjP/TQMKGfVJwy29wrSia0uScbOhRGjQq1i+zsqKMRqbo2bIBbboEHHggL5TzySGhqkfRItc29mnYS+rHf/ja07d1wQ9SRiFRdkydD+/ZhebpBg2DuXCX2qCi5JzRpEkauvvQSvP561NGIVC1r14buxCeeGPqVv/56WDthzz2jjqz6UnJPctVVof/sddeFLloiUraJE8PF0DFjwqDAvDzo3j3qqETJPcnuu8Odd4YpQJ9+OupoRDLbypVhEFHv3uGX73vvwd13h/9HEj0l92L69w9X82+6Cb7/PupoRDKPe5hVtW3b0MXxjjsgNzfMuCiZQ8m9mBo1Qp/3zz8PS26JyHaffx4m2rvgAmjdOgxCuvnm+CxNFydK7iU48UQ45ZTQRLNmTdnlReJu61Z46KHQtv7GG2E1s6lTQ+1dMpOS+w7cc08YFl24cpNIdTV/Phx/fFi56JhjYM6cMC5Eg5Aym5L7Dhx+eJjQf9SoMLGRSHVTUBBmTu3QIQzuGzsWJk2CrKyoI5NUKLmXYsSI0JZ4001RRyJSuWbPhi5dwtiP008PE35dfLGm1K1KlNxLccABcO218OyzoZuXSNx9/324QJqTA8uWwXPPhZsm1Kt6lNzLcP31sM8+Wm9V4u+tt8JMjb//fZitce7cMGOqVE1K7mVo0ABuuw2mTQszR4rEzbffhhkbu3ULNfdJk8Jo0733jjoyKQ8l9xRcdhm0aRMmFStceFckDiZNCp0HRo0KCf6jj+Dkk6OOStJByT0FtWqFYdWffAJ//WvU0YiU35o1cNFF0LNnWDhj2rTQd12rHsWHknuKzjgjTIZ0663wzTdRRyOya9zhf/83rFnw1FMwfHjoGXPssVFHJumm5J4is7Bc2KpVoRYvUtWsWAFnnw3nnRdmP83Nhdtv19J2caXkvhNycqBfv7AQQX5+1NGIpMY9XCBt2xb+/e8w+vrdd8PgJIkvJfed9Pvfh3k2brkl6khEyrZoEfzsZ2Et0w4dwlzr118friNJvCm576SsrNCr4Ikn4IMPoo5GpGTLl4f29Pbtw0LVjzwCU6aEhamlelBy3wWHHRba4Dt2hKZN4fHHo46o+lq5Mswpfs01cPTRYaGIM86AV1+tnqtpvf9+WJOgZcvwK/OUU8JgpMsvD9NZS/Whj3snjR8Pv/719sSxenWYYOznP4dZs6KNLe7cQzPDE0+E9TrbtIF99w0XCR96KHTpu/BCmD4dTj0VDjkktC+vXh115BVr82aYMCHM2NilC/zrXzBkCCxYEKYOaN486gglCuYRjanPycnx3NzcSM5dHllZsGTJjh/v2DEk+/79oXHjSgsrlrZsCYNqpk4Nt2nTQnMDQKNGcNxx4datGxx11PZeH5s2hdr8ww+Hucd32w3OPRf+53/gJz+Jz+RXq1fD6NHhi23ZsvBlNnRomOCrQYOoo5OKYmYz3L3Mda+U3HdSjRo7nmPmL38J06LOmBFmk+zdOyT6k0/WBaxUfP99qHVPmxaS+dtvw7p14bHmzUMS79YtJPR27VJrZpgzJyT5v/89DLPv0CEk+fPPr7oDdj78EEaODEvdff89nHQSXH11aIJR00v8KblXkB3V3Fu2hMWLw/28vJDkx40LtasDDgjNBQMHhvZ6CdauDQm8sFb+/vuh1g1hkE1hIu/WLby/5bF+fWhSe+ih8PnsuWcYoTl4cNVYTWjLFnj55TCKdPLkcG3hggtCTb1du6ijk8qUanLH3cu8AT2B+cBCYFgp5c4BHMgp65hHHXWUV0XjxrnXq+ce6u/hVq9e2F/cDz+4P/+8++mnu9esGcp27er+2GPu33xT+bFHLT/ffcIE9yuvdD/iCHez8J7UquXepYv7dde5//Of7qtWVVwMW7e6v/WWe//+7nXqhPMff7z7s8+6b9pUcefdVevWud9/v/tBB4VYmzd3v+su99Wro45MogLkeip5u8wCUBP4FDgIqAN8AGSXUK4B8CbwbpyTu3tI5C1bhuTUsmXJib245cvd777bvU2b7V8IF13k/vrrIeHEzdat7vPmuY8e7X7hhe6tWm3/MtxjD/ef/cz9ttvcJ092X78+mhhXrgyJMisrxLXffu633OK+dGk08SRbsMB96FD3+vVDbMce6/7MM5n5BSSVK53J/RhgUtL2jcCNJZR7ADgNeD3uyb08tm51f+cd90GD3Bs0CJ/AQQe53367++efRx3drtu0yf39993/9Cf3M890b9JkezJv2tT97LNDDXT6dPfNm6OOtqiCAveXX3Y/7bTwhV2jRngN//mP+5YtlRfH1q3u//1v+KVn5l67tvuAAeE9EymUanIvs83dzPoAPd39ssT2BUAXdx+SVKYTcLO7n2NmrwPXufuPGtTNbBAwCKBFixZHLSmt20k1sGED/OMfoX1+8uTQi+Okk+CSS+DMM6Fu3agjLMo9XEP45JNwmz9/+/2FC+GHH0K5gw7afvGzW7cwcKaq9FD57LPQA+Wxx8JrPfRQuOKK0AOlouY337AhXJ8ZOTJcAG7aNJxz8GDYf/+KOadUXWm7oFpWcjezGsBk4GJ3X1xack9WVS+oVpTPPgv9tx9/PFywbdQozGNzySWhm19lJscNG0If6eIJfP78cBG0UO3aofvdYYeFW05OuAB6wAGVF2tF+eGH0Ef84YfDCkV164bPY/DgMFgqHZYuDRd4R4+Gr74K3Wivugr69s28L3bJHOlM7scAt7r7zxPbNwK4+x8S2w0JbfLrE0/ZD/gK6FVagldyL9nWrWGY+Nix8Pzzoavb4YeHJD9gQKjVpcOWLaF3T0m18KVLi5Y98MDtCbx16+33W7asHl08P/ggJPlx4+C778KX2ODBIQnXq7dzx3KHd94JvV6efz5sn3lmSOrdulWdXzgSnXQm91rAJ8CJwDJgOnC+u8/ZQfnXUc09LdatCyMPx44NC3TXqhVWoh84MPRprl279Oe7h+H5JSXwTz/d3u0Qwi+F5MRdeP+QQ2CPPSr2dVYV33wDTz4Zattz54b3bODA0IRSVhfXTZvCQusPPhim2m3YMKzwNWRI6F4rkqq09nM3s1MJF0xrAmPc/U4zG0Fo2J9YrOzrKLmn3dy5Ick/+SR8+WUYdn/BBSG5tGy5PWkXT+KFg4AA6tQJbciFCTw5iTdpolpjqtxD3/yHHgq174KCcK3kf/4nzGuT/Gtm5Up49NFQ9osvwvs9dGgY91BVB1FJtDSIKaY2bw5zco8dG+YQKSj4cZkWLUquhbdoATVrVn7McfbFF/C3v4UEvnQpNGsGgwaFVbsefzysdrRpU5h76Oqrw2hljSKV8lByrwZWroRnngnNBa1bh9shh4TRi1K5CgrCCNKHHw6LTkNoj7/oolBTb9Mm2vgkPpTcRSKycGG4RnLqqbDXXlFHI3GTanKvBn0dRCrXIYeEm0iU1PonIhJDSu4iIjGk5C4iEkNK7iIiMaTkLiISQ0ruIiIxpOQuIhJDSu4iIjGk5C4iEkNK7iIiMaTkLiISQ0ruIiIxpOQuIhJDSu4iIjGk5C4iEkNK7iIiMaTkXoWNHw9ZWWFNzqyssC0iAlqJqcoaPz4sxLxhQ9hesiRsA/TvH11cIpIZVHOvom6+eXtiL7RhQ9gvIqLkXkV9/vnO7ReR6kXJvYpq0WLn9otI9ZJScjeznmY238wWmtmwEh6/wsw+NLPZZjbNzLLTH6oku/NOqFev6L569cJ+EZEyk7uZ1QRGAacA2UC/EpL3U+7e3t07AvcA96U9Uimif38YPRpatgSz8Hf0aF1MFZEgld4ynYGF7r4IwMwmAL2BuYUF3P2bpPJ7AJ7OIKVk/fsrmYtIyVJJ7s2ApUnb+UCX4oXM7ErgGqAOcEJaohMRkV2Stguq7j7K3Q8GbgCGl1TGzAaZWa6Z5a5atSpdpxYRkWJSSe7LgAOTtpsn9u3IBODMkh5w99HunuPuOU2bNk09ShER2SmpJPfpwKFm1srM6gB9gYnJBczs0KTN04AF6QtRRER2Vplt7u5eYGZDgElATWCMu88xsxFArrtPBIaY2UnAZuBr4KKKDFpEREqX0twy7v4K8Eqxfb9Nun9VmuMSEZFy0AhVEZEYUnIXEYkhJXcRkRhScpdy06IhIplHi3VIuWjREJHMpJq7lIsWDRHJTEruUi5aNEQkMym5S7lo0RCRzKTkLuWiRUNEMpOSu5SLFg0RyUzqLSPlpkVDRDKPau4iIjGk5C4iEkNK7iIiMaTkLiISQ0ruIiIxpOQuIhJDSu4iIjGk5C4iEkNK7iIiMaTkLiISQ0ruEhtaEUpkO80tI7GgFaFEilLNXWJBK0KJFKXkLrGgFaFEikopuZtZTzObb2YLzWxYCY9fY2ZzzSzPzF4zs5bpD1Vkx7QilEhRZSZ3M6sJjAJOAbKBfmaWXazYLCDH3Y8AngPuSXegIqXRilAiRaVSc+8MLHT3Re6+CZgA9E4u4O5T3L2wxfNdoHl6wxQpnVaEEikqld4yzYClSdv5QJdSyl8KvFrSA2Y2CBgE0EK/lyXNtCKUyHZpvaBqZgOAHOCPJT3u7qPdPcfdc5o2bZrOU4uISJJUau7LgAOTtpsn9hVhZicBNwPd3f2H9IQnIiK7IpWa+3TgUDNrZWZ1gL7AxOQCZnYk8CjQy91Xpj9MERHZGWUmd3cvAIYAk4B5wLPuPsfMRphZr0SxPwL1gf81s9lmNnEHhxMRkUqQUpu7u7/i7oe5+8Hufmdi32/dfWLi/knuvq+7d0zcepV+RJF40vw2kik0t4xImmh+G8kkmn5AJE00v41kEiV3kTTR/DaSSZTcRdJE89tIJlFyF0kTzW8jmUTJXSRNNL+NZBL1lhFJI81vI5lCNXcRkRhSchcRiSEldxGRGFJyF4khTYMguqAqEjOaBkFANXeR2NE0CAJK7iKxo2kQBJTcRWJH0yAIZFib++bNm8nPz+f777+POhRJQd26dWnevDm1a9eOOhRJcuedRdvcQdMgVEcZldzz8/Np0KABWVlZmFnU4Ugp3J01a9aQn59Pq1atog5HkhReNL355tAU06JFSOy6mFq9ZFRy//7775XYqwgzo3HjxqxatSrqUKQEmgZBMq7NXYm96tBnJZK5Mi65i4hI+VXp5J7uUXhr1qyhY8eOdOzYkf32249mzZpt2960aVNKxxg4cCDz588vtcyoUaMYn6Yhg8cddxyzZ89Oy7FEJD4yqs19Z1TEKLzGjRtvS5S33nor9evX57rrritSxt1xd2rUKPl7cezYsWWe58orr9y1AEWqmPHjdWE3KlW25l6Zo/AWLlxIdnY2/fv3p127dqxYsYJBgwaRk5NDu3btGDFixLayhTXpgoICGjVqxLBhw+jQoQPHHHMMK1euBGD48OE88MAD28oPGzaMzp0707p1a95++20AvvvuO8455xyys7Pp06cPOTk5ZdbQx40bR/v27Tn88MO56aabACgoKOCCCy7Ytn/kyJEA3H///WRnZ3PEEUcwYMCAtL9nIoUVsCVLwH17BUzz3FSOKltzr+xReB9//DF///vfycnJAeCuu+5i7733pqCggB49etCnTx+ys7OLPGfdunV0796du+66i2uuuYYxY8YwbNiwHx3b3Xn//feZOHEiI0aM4N///jd//vOf2W+//Xj++ef54IMP6NSpU6nx5efnM3z4cHJzc2nYsCEnnXQSL730Ek2bNmX16tV8+OGHAKxduxaAe+65hyVLllCnTp1t+0TSqbQKmGrvFS+lmruZ9TSz+Wa20Mx+lJ3M7KdmNtPMCsysT/rD/LHKHoV38MEHb0vsAE8//TSdOnWiU6dOzJs3j7lz5/7oObvvvjunnHIKAEcddRSLFy8u8dhnn332j8pMmzaNvn37AtChQwfatWtXanzvvfceJ5xwAk2aNKF27dqcf/77AnCCAAALOklEQVT5vPnmmxxyyCHMnz+foUOHMmnSJBo2bAhAu3btGDBgAOPHj9cgJKkQmgYhWmUmdzOrCYwCTgGygX5mll2s2OfAxcBT6Q5wRyp7MeI99thj2/0FCxbw4IMPMnnyZPLy8ujZs2eJo2rr1Kmz7X7NmjUpKCgo8di77bZbmWV2VePGjcnLy6Nbt26MGjWKyy+/HIBJkyZxxRVXMH36dDp37syWLVvSel4RTYMQrVRq7p2Bhe6+yN03AROA3skF3H2xu+cBWysgxhJFuRjxN998Q4MGDdhzzz1ZsWIFkyZNSvs5unbtyrPPPgvAhx9+WOIvg2RdunRhypQprFmzhoKCAiZMmED37t1ZtWoV7s65557LiBEjmDlzJlu2bCE/P58TTjiBe+65h9WrV7Oh+O9nkXKq7AqYFJVKm3szYGnSdj7QpWLC2TlRjcLr1KkT2dnZtGnThpYtW9K1a9e0n+NXv/oVF154IdnZ2dtuhU0qJWnevDm33347xx9/PO7OGWecwWmnncbMmTO59NJLcXfMjLvvvpuCggLOP/98vv32W7Zu3cp1111HgwYN0v4apHrTNAjRMncvvUBoQ+/p7pclti8Aurj7kBLKPg685O7P7eBYg4BBAC1atDhqyZIlRR6fN28ebdu23YWXET8FBQUUFBRQt25dFixYwMknn8yCBQuoVSuzroHrMxOpXGY2w91zyiqXSrPMMuDApO3miX07zd1Hu3uOu+c0bdp0Vw5Rbaxfv56uXbvSoUMHzjnnHB599NGMS+wiVUV1XHYwlWwxHTjUzFoRknpf4PwKjUpo1KgRM2bMiDoMkSqvui47WGbN3d0LgCHAJGAe8Ky7zzGzEWbWC8DMjjazfOBc4FEzm1ORQYuIpKq6LjuY0u98d38FeKXYvt8m3Z9OaK4REcko1bW/fZWdfkBEJBXVtb+9kruIxFp17W+v5J6kR48ePxqQ9MADDzB48OBSn1e/fn0Ali9fTp8+Jc++cPzxx5Obm1vqcR544IEig4lOPfXUtMz7cuutt3LvvfeW+zgiVVGUAx6jpOSepF+/fkyYMKHIvgkTJtCvX7+Unn/AAQfw3HMldvFPSfHk/sorr9CoUaNdPp6IBP37w+LFsHVr+Bv3xA4ZPCvk1VdDuteg6NgREjPtlqhPnz4MHz6cTZs2UadOHRYvXszy5cvp1q0b69evp3fv3nz99dds3ryZO+64g969i8zCwOLFizn99NP56KOP2LhxIwMHDuSDDz6gTZs2bNy4cVu5wYMHM336dDZu3EifPn247bbbGDlyJMuXL6dHjx40adKEKVOmkJWVRW5uLk2aNOG+++5jzJgxAFx22WVcffXVLF68mFNOOYXjjjuOt99+m2bNmvHiiy+y++677/A1zp49myuuuIINGzZw8MEHM2bMGPbaay9GjhzJI488Qq1atcjOzmbChAm88cYbXHXVVUBYUu/NN9/USFaRKkI19yR77703nTt35tVXXwVCrf28887DzKhbty4vvPACM2fOZMqUKVx77bWUNrr34Ycfpl69esybN4/bbrutSJ/1O++8k9zcXPLy8njjjTfIy8tj6NChHHDAAUyZMoUpU6YUOdaMGTMYO3Ys7733Hu+++y5//etfmTVrFhAmMbvyyiuZM2cOjRo14vnnny/1NV544YXcfffd5OXl0b59e2677TYgTGE8a9Ys8vLyeOSRRwC49957GTVqFLNnz2bq1KmlfmmISGbJ2Jp7aTXsilTYNNO7d28mTJjA3/72NyDMuX7TTTfx5ptvUqNGDZYtW8aXX37JfvvtV+Jx3nzzTYYOHQrAEUccwRFHHLHtsWeffZbRo0dTUFDAihUrmDt3bpHHi5s2bRpnnXXWtpkpzz77bKZOnUqvXr1o1aoVHTt2BEqfVhjC/PJr166le/fuAFx00UWce+6522Ls378/Z555JmeeeSYQJi+75ppr6N+/P2effTbNm6u3q0hVoZp7Mb179+a1115j5syZbNiwgaOOOgqA8ePHs2rVKmbMmMHs2bPZd999S5zmtyyfffYZ9957L6+99hp5eXmcdtppu3ScQoXTBUP5pgx++eWXufLKK5k5cyZHH300BQUFDBs2jMcee4yNGzfStWtXPv74412OU0QqdxoEJfdi6tevT48ePbjkkkuKXEhdt24d++yzD7Vr12bKlCkUn/SsuJ/+9Kc89VSY3v6jjz4iLy8PCNMF77HHHjRs2JAvv/xyWxMQQIMGDfj2229/dKxu3brxz3/+kw0bNvDdd9/xwgsv0K1bt51+bQ0bNmSvvfZi6tSpADz55JN0796drVu3snTpUnr06MHdd9/NunXrWL9+PZ9++int27fnhhtu4Oijj1ZyFymHyl52MGObZaLUr18/zjrrrCI9Z/r3788ZZ5xB+/btycnJoU2bNqUeY/DgwQwcOJC2bdvStm3bbb8AOnTowJFHHkmbNm048MADi0wXPGjQIHr27Lmt7b1Qp06duPjii+ncuTMQLqgeeeSRpTbB7MgTTzyx7YLqQQcdxNixY9myZQsDBgxg3bp1uDtDhw6lUaNG3HLLLUyZMoUaNWrQrl27batKicjOq+xlB8uc8rei5OTkePF+35o+turRZyaSmho1Qo29OLPQRTNV6ZzyV0REyqmyp0FQchcRqQSVPQ1CxiX3qJqJZOfpsxJJXWVPg5BRF1Tr1q3LmjVraNy4MWYWdThSCndnzZo11K1bN+pQRKqMylz3OaOSe/PmzcnPz2fVqlVRhyIpqFu3rgY2iWSojErutWvXplWrVlGHISJS5WVcm7uIiJSfkruISAwpuYuIxFBkI1TNbBVQ+gQtma8JsDrqIDKI3o/t9F4UpfejqPK8Hy3dvWlZhSJL7nFgZrmpDAOuLvR+bKf3oii9H0VVxvuhZhkRkRhSchcRiSEl9/IZHXUAGUbvx3Z6L4rS+1FUhb8fanMXEYkh1dxFRGJIyV1EJIaU3HeBmR1oZlPMbK6ZzTGzq6KOKWpmVtPMZpnZS1HHEjUza2Rmz5nZx2Y2z8yOiTqmKJnZrxP/Tz4ys6fNrNpMJWpmY8xspZl9lLRvbzP7r5ktSPzdqyLOreS+awqAa909G/gJcKWZZUccU9SuAuZFHUSGeBD4t7u3ATpQjd8XM2sGDAVy3P1woCbQN9qoKtXjQM9i+4YBr7n7ocBrie20U3LfBe6+wt1nJu5/S/jP2yzaqKJjZs2B04DHoo4lambWEPgp8DcAd9/k7mujjSpytYDdzawWUA9YHnE8lcbd3wS+Kra7N/BE4v4TwJkVcW4l93IysyzgSOC9aCOJ1APAb4CdWOY3tloBq4CxiWaqx8xsj6iDioq7LwPuBT4HVgDr3P0/0UYVuX3dfUXi/hfAvhVxEiX3cjCz+sDzwNXu/k3U8UTBzE4HVrr7jKhjyRC1gE7Aw+5+JPAdFfSzuypItCf3JnzpHQDsYWYDoo0qc3joi14h/dGV3HeRmdUmJPbx7v6PqOOJUFegl5ktBiYAJ5jZuGhDilQ+kO/uhb/kniMk++rqJOAzd1/l7puBfwDHRhxT1L40s/0BEn9XVsRJlNx3gYUFXv8GzHP3+6KOJ0rufqO7N3f3LMKFssnuXm1rZu7+BbDUzFondp0IzI0wpKh9DvzEzOol/t+cSDW+wJwwEbgocf8i4MWKOImS+67pClxAqKXOTtxOjTooyRi/AsabWR7QEfh9xPFEJvEL5jlgJvAhIedUm6kIzOxp4B2gtZnlm9mlwF3Az8xsAeGXzV0Vcm5NPyAiEj+quYuIxJCSu4hIDCm5i4jEkJK7iEgMKbmLiMSQkruISAwpuYuIxND/A3284E6M9S4OAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "3장에서 이 데이터셋을 사용한 첫 번째 모델에서 얻은 테스트 정확도는 87%였습니다. 안타깝지만 간단한 순환 네트워크는 이 기준 모델보다 성능이 높지 않습니다(85% 정도의 검증 정확도를 얻었습니다). \n",
    "- 이런 원인은 전체 시퀀스가 아니라 처음 500개의 단어만 입력에 사용했기 때문입니다. 이 RNN은 기준 모델보다 얻은 정보가 적습니다.\n",
    "- 다른 이유는 SimpleRNN이 텍스트와 같이 긴 시퀀스를 처리하는데 적합하지 않기 때문입니다. \n",
    "\n",
    "더 잘 작동하는 다른 순환 층이 있습니다. 조금 더 고급 순환 층을 살펴보죠."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.2.2 LSTM과 GRU 층 이해하기\n",
    "\n",
    "- [[링크](http://excelsior-cjh.tistory.com/185)] 참고\n",
    "- 케라스에는 `SimpleRNN`외에 다른 순환층이 있는데, [`LSTM`](https://keras.io/layers/recurrent/#lstm)과 [`GRU`](https://keras.io/layers/recurrent/#gru) 2 개가 있다.\n",
    "- `SimpleRNN`은 이론적으로 타임스텝 `t`에서 이전 모든 타임스텝의 정보를 유지할 수 있지만, 실제로는 긴 시간에 걸친 의존성은 학습할 수 없는 것이 문제다.\n",
    "- `SimpleRNN`은 **vanishing gradient** 또는 **exploding gradient** 문제가 발생할 가능성이 높다.\n",
    "- 이 문제를 해결하기 위해 고안된 것이 **LSTM**과 **GRU**이다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### LSTM Cell\n",
    "\n",
    "![](./images/lstm.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- LSTM 셀에서는 상태(state)가 두 개의 벡터 $\\mathbf{h}_t$와 $\\mathbf{c}_t$로 나누어 진다는 것을 알 수 있다.\n",
    "    - $\\mathbf{h}_t$를 단기 상태(short-term state), \n",
    "    - $\\mathbf{c}_t$를 장기 상태(long-term state)라고 볼 수 있다. \n",
    "\n",
    "\n",
    "- LSTM의 핵심은 네트워크가 장기 상태($\\mathbf{c}_t$)에서 기억할 부분, 삭제할 부분,  그리고 읽어 들일 부분을 학습하는 것이다. \n",
    "- 장기 기억 $\\mathbf{c}_{t-1}$은 셀의 왼쪽에서 오른쪽으로 통과하게 되는데,\n",
    "    1. **forget gate**를 지나면서 일부를 기억(정보)을 잃고, \n",
    "    2. 그 다음 덧셈(+) 연산으로 **input gate**로 부터 새로운 기억 일부를 추가한다. \n",
    "    3. 이렇게 만들어진 $\\mathbf{c}_t$는 별도의 추가 연산 없이 바로 출력되며, 이러한 장기 기억 $\\mathbf{c}_t$는 타임 스텝마다 일부의 기억을 삭제하고 추가하는 과정을 거치게 된다. \n",
    "    4. 그리고 덧셈 연산 후에 $\\mathbf{c}_t$는 복사되어 **output gate**의 $\\tanh$함수로 전달되어 단기 상태 $\\mathbf{h}_{t}$와 셀의 출력인 $\\mathbf{y}_t$를 만든다. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- forget, input, output 게이트(gate)의 역할은 다음과 같다.\n",
    "    - **Forget gate** : $\\mathbf{f}_{t}$에 의해 제어되며 장기 상태 $\\mathbf{c}_{t}$의 어느 부분을 삭제할지 제어한다.\n",
    "    - **Input gate** : $\\mathbf{i}_{t}$에 의해 제어되며 $\\mathbf{g}_{t}$의 어느 부분이 장기 상태 $\\mathbf{c}_{t}$에 더해져야 하는지 제어한다.\n",
    "    - **Output gate** : $\\mathbf{o}_{t}$는 장기 상태 $\\mathbf{c}_{t}$의 어느 부분을 읽어서 $\\mathbf{h}_{t}$ 와 $\\mathbf{y}_{t}$로 출력해야 하는지 제어한다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- 다음의 식은 위의 그림에서 타임 스텝 $t$에서, 셀의 장기 상태, 단기 상태, 그리고 각 레이어의 출력을 구하는 식을 나타낸 것이다.\n",
    "\n",
    "\n",
    "$$\n",
    "\\begin{eqnarray} \n",
    "\\mathbf{f}_{t} & = & \\sigma \\left( \\mathbf{W}_{xf}^{T} \\cdot \\mathbf{x}_{t} + \\mathbf{W}_{hf}^{T} \\cdot \\mathbf{h}_{t-1} + \\mathbf{b}_{f} \\right) \\\\ \\mathbf{i}_{t} & =  & \\sigma \\left( \\mathbf{W}_{xi}^{T} \\cdot \\mathbf{x}_{t} + \\mathbf{W}_{hi}^{T} \\cdot \\mathbf{h}_{t-1} + \\mathbf{b}_{i} \\right) \\\\ \\mathbf{o}_{t} & = & \\sigma \\left( \\mathbf{W}_{xo}^{T} \\cdot \\mathbf{x}_{t} + \\mathbf{W}_{ho}^{T} \\cdot \\mathbf{h}_{t-1} + \\mathbf{b}_{o} \\right) \\\\ \\mathbf{g}_{t} & = & \\tanh \\left( \\mathbf{W}_{xg}^{T} \\cdot \\mathbf{x}_{t} + \\mathbf{W}_{hg}^{T} \\cdot \\mathbf{h}_{t-1} + \\mathbf{b}_{g} \\right) \\\\ \\mathbf{c}_{t} & = & \\mathbf{f}_{t} \\otimes \\mathbf{c}_{t-1} + \\mathbf{i}_{t} \\otimes \\mathbf{g}_{t} \\\\ \\mathbf{y}_{t}, \\mathbf{h}_{t} & = & \\mathbf{o}_{t} \\otimes \\tanh \\left( \\mathbf{c}_{t} \\right) \n",
    "\\end{eqnarray}\n",
    "$$\n",
    "\n",
    "\n",
    "- $\\mathbf{W}_{xf}, \\mathbf{W}_{xi}, \\mathbf{W}_{xo}, \\mathbf{W}_{xg}$ : 입력 벡터 $\\mathbf{x}_{t}$에 연결된 네 개의 레이어에 대한 가중치 행렬\n",
    "- $\\mathbf{W}_{hf}, \\mathbf{W}_{hi}, \\mathbf{W}_{ho}, \\mathbf{W}_{hg}$ : 이전 타임스텝의 단기 상태 $\\mathbf{h}_{t-1}$에 연결된 네 개의 레이어에 대한 가중치 행렬\n",
    "- $\\mathbf{b}_{f}, \\mathbf{b}_{i}, \\mathbf{b}_{o}, \\mathbf{b}_{g}$ : 네 개의 레이어에 대한 편향(bias), [텐서플로](https://www.tensorflow.org/api_docs/python/tf/nn/rnn_cell/LSTMCell)(TensorFlow)에서는 $\\mathbf{b}_{f}$ 를 `1`로 초기화하여 학습 시작시에 모든것을 잃어버리는 것을 방지한다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### GRU Cell\n",
    "\n",
    "![](./images/gru.PNG)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **GRU(Gated Recurrent Unit)** 셀은 2014년에 K. Cho(조경현) 등에 의해 ['이 논문'](https://arxiv.org/pdf/1406.1078v3.pdf)에서 제안된 LSTM 셀의 **간소화된 버전**이다.\n",
    "\n",
    "- LSTM Cell에서의 두 상태 벡터 $\\mathbf{c}_{t}$와 $\\mathbf{h}_{t}$가 하나의 벡터 $\\mathbf{h}_{t}$로 합쳐졌다.\n",
    "\n",
    "- 하나의 gate controller인 $\\mathbf{z}_{t}$가 **forget**과 **input** 게이트(gate)를 모두 제어한다.  $\\mathbf{z}_{t}$가 `1`을 출력하면 forget 게이트가 열리고 input 게이트가 닫히며, $\\mathbf{z}_{t}$가 `0`일 경우 반대로 forget 게이트가 닫히고 input 게이트가 열린다. 즉, 이전($t-1$)의 기억이 저장 될때 마다 타임 스텝 $t$의 입력은 삭제된다. \n",
    "\n",
    "- GRU 셀은 output 게이트가 없어 전체 상태 벡터 $\\mathbf{h}_{t}$가 타임 스텝마다 출력되며, 이전 상태 $\\mathbf{h}_{t-1}$의 어느 부분이 출력될지 제어하는 새로운 gate controller인 $\\mathbf{r}_{t}$가 있다.\n",
    "\n",
    "- GRU 셀의 상태(state)와 각 레이어의 출력을 계산하는 식은 다음과 같다.\n",
    "\n",
    "\n",
    "$$\n",
    "\\begin{eqnarray} \n",
    "\\mathbf{r}_{t} & = & \\sigma \\left( \\mathbf{W}_{xr}^{T} \\cdot \\mathbf{x}_{t} + \\mathbf{W}_{hr}^{T} \\cdot \\mathbf{h}_{t-1} + \\mathbf{b}_{r} \\right) \\\\ \\mathbf{z}_{t} & =  & \\sigma \\left( \\mathbf{W}_{xz}^{T} \\cdot \\mathbf{x}_{t} + \\mathbf{W}_{hz}^{T} \\cdot \\mathbf{h}_{t-1} + \\mathbf{b}_{z} \\right) \\\\ \\mathbf{g}_{t} & = & \\tanh \\left( \\mathbf{W}_{xg}^{T} \\cdot \\mathbf{x}_{t} + \\mathbf{W}_{hg}^{T} \\cdot \\left( \\mathbf{r}_{t} \\otimes \\mathbf{h}_{t-1} \\right) + \\mathbf{b}_{g} \\right) \\\\ \\mathbf{h}_{t} & = & \\mathbf{z}_{t} \\otimes \\mathbf{h}_{t-1} + \\left( 1 - \\mathbf{z}_{t} \\right) \\otimes \\mathbf{g}_{t}\n",
    "\\end{eqnarray}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "---"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.2.3 케라스를 사용한 LSTM 예제\n",
    "\n",
    "이제 실제적인 관심사로 이동해 보죠. `LSTM` 층으로 모델을 구성하고 IMDB 데이터에서 훈련해 보겠습니다(그림 6-16과 6-17 참조). 이 네트워크는 조금 전 SimpleRNN을 사용했던 모델과 비슷합니다. \n",
    "\n",
    "`LSTM` 층은 출력 차원만 지정하고 다른 (많은) 매개변수는 케라스의 기본값으로 남겨 두었습니다. 케라스는 좋은 기본값을 가지고 있어서 직접 매개변수를 튜닝하는 데 시간을 쓰지 않고도 거의 항상 어느정도 작동하는 모델을 얻을 수 있습니다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_2 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "lstm_1 (LSTM)                (None, 32)                8320      \n",
      "_________________________________________________________________\n",
      "dense_2 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 328,353\n",
      "Trainable params: 328,353\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 53s 3ms/step - loss: 0.5066 - acc: 0.7641 - val_loss: 0.4870 - val_acc: 0.7590\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 52s 3ms/step - loss: 0.2869 - acc: 0.8866 - val_loss: 0.2794 - val_acc: 0.8866\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 52s 3ms/step - loss: 0.2301 - acc: 0.9122 - val_loss: 0.3553 - val_acc: 0.8780\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 52s 3ms/step - loss: 0.1967 - acc: 0.9280 - val_loss: 0.4495 - val_acc: 0.8524\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 52s 3ms/step - loss: 0.1741 - acc: 0.9371 - val_loss: 0.2947 - val_acc: 0.8860\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 53s 3ms/step - loss: 0.1520 - acc: 0.9454 - val_loss: 0.4889 - val_acc: 0.8206\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 53s 3ms/step - loss: 0.1391 - acc: 0.9514 - val_loss: 0.4156 - val_acc: 0.8800\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 52s 3ms/step - loss: 0.1287 - acc: 0.9555 - val_loss: 0.3136 - val_acc: 0.8798\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 52s 3ms/step - loss: 0.1176 - acc: 0.9592 - val_loss: 0.3477 - val_acc: 0.8692\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 53s 3ms/step - loss: 0.1077 - acc: 0.9642 - val_loss: 0.3739 - val_acc: 0.8862\n"
     ]
    }
   ],
   "source": [
    "from keras.layers import LSTM\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_features, 32))\n",
    "model.add(LSTM(32))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer='rmsprop', \n",
    "              loss='binary_crossentropy', \n",
    "              metrics=['acc'])\n",
    "history = model.fit(train_inputs, train_labels,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VNXZwPHfw2YI+yYICAEXICxhiYBlUUQQrcIroqLYilapC2pd6ouCgijSuoFa20oVrTXCS6UqVmwiiKIilB0FRFABA4hhFQSFkOf949yEScgySWbmTmae7+eTT2buvXPvMzeTZ84959xzRFUxxhgTHyr5HYAxxpjIsaRvjDFxxJK+McbEEUv6xhgTRyzpG2NMHLGkb4wxccSSfhwSkcoiclBEWoRyWz+JyOkiEvL+xyJyvohsDni+QUT6BLNtGY71gojcX9bXGxOMKn4HYEomIgcDniYCPwPHvOe/VdW00uxPVY8BNUO9bTxQ1Tah2I+I3ABco6rnBuz7hlDs25jiWNKvAFQ1L+l6JckbVHVeUduLSBVVzY5EbMaUxD6P0cWqd2KAiDwiIv8nIjNE5ABwjYicLSKLRWSfiOwQkWdEpKq3fRURURFJ8p6/6q1/V0QOiMinItKqtNt66y8UkS9FZL+IPCsin4jIyCLiDibG34rIJhHZKyLPBLy2sohMEZHdIvI1MKiY8zNWRGYWWPaciDzlPb5BRNZ77+crrxRe1L4yReRc73GiiPzDi20t0K3AtuNE5Gtvv2tFZLC3vCPwJ6CPV3W2K+DcTgh4/U3ee98tIm+KyCnBnJvSnOfceERknojsEZHvROTegOM84J2TH0RkmYg0LawqTUQ+zv07e+dzoXecPcA4ETlDRBZ4x9jlnbc6Aa9v6b3HLG/90yKS4MXcLmC7U0TkkIg0KOr9mhKoqv1UoB9gM3B+gWWPAEeAS3Bf5NWBs4AeuKu51sCXwGhv+yqAAkne81eBXUAqUBX4P+DVMmx7MnAAGOKtuws4Cows4r0EE+NbQB0gCdiT+96B0cBaoDnQAFjoPs6FHqc1cBCoEbDv74FU7/kl3jYCnAccBjp5684HNgfsKxM413v8BPABUA9oCawrsO0VwCne3+RqL4bG3robgA8KxPkqMMF7PNCLsTOQAPwZeD+Yc1PK81wH2AncAZwE1Aa6e+vuA1YDZ3jvoTNQHzi94LkGPs79O3vvLRu4GaiM+zyeCfQHqnmfk0+AJwLez+fe+azhbd/LWzcNmBRwnLuBN/z+P6zIP74HYD+l/IMVnfTfL+F19wD/9B4Xlsj/GrDtYODzMmx7PfBRwDoBdlBE0g8yxp4B6/8F3OM9Xoir5spdd1HBRFRg34uBq73HFwIbitn238Ct3uPikv7WwL8FcEvgtoXs93Pgl97jkpL+34FHA9bVxrXjNC/p3JTyPP8KWFrEdl/lxltgeTBJ/+sSYhiWe1ygD/AdULmQ7XoB3wDiPV8FDA31/1U8/Vj1Tuz4NvCJiLQVkXe8y/UfgIlAw2Je/13A40MU33hb1LZNA+NQ91+aWdROgowxqGMBW4qJF+A14Crv8dXe89w4LhaRJV7Vwz5cKbu4c5XrlOJiEJGRIrLaq6LYB7QNcr/g3l/e/lT1B2Av0Cxgm6D+ZiWc51Nxyb0wxa0rScHPYxMRmSUi27wYXi4Qw2Z1nQbyUdVPcFcNvUWkA9ACeKeMMRmsTj+WFOyu+DyuZHm6qtYGHsSVvMNpB64kCoCICPmTVEHliXEHLlnkKqlL6SzgfBFphqt+es2LsTrwOjAZV/VSF8gIMo7viopBRFoDf8FVcTTw9vtFwH5L6l66HVdllLu/WrhqpG1BxFVQcef5W+C0Il5X1LofvZgSA5Y1KbBNwff3R1yvs45eDCMLxNBSRCoXEccrwDW4q5JZqvpzEduZIFjSj121gP3Aj15D2G8jcMx/A11F5BIRqYKrJ24UphhnAb8TkWZeo97/Frexqn6Hq4J4GVe1s9FbdRKunjkLOCYiF+PqnoON4X4RqSvuPobRAetq4hJfFu7770ZcST/XTqB5YINqATOA34hIJxE5Cfel9JGqFnnlVIzizvMcoIWIjBaRk0Sktoh099a9ADwiIqeJ01lE6uO+7L7DdRioLCKjCPiCKiaGH4H9InIqroop16fAbuBRcY3j1UWkV8D6f+Cqg67GfQGYcrCkH7vuBq7FNaw+j2twDStV3QlcCTyF+yc+DViJK+GFOsa/APOBz4CluNJ6SV7D1dHnVe2o6j7gTuANXGPoMNyXVzDG4644NgPvEpCQVHUN8CzwX2+bNsCSgNe+B2wEdopIYDVN7uv/g6uGecN7fQtgRJBxFVTkeVbV/cAA4DLcF9GXwDne6seBN3Hn+Qdco2qCV213I3A/rlH/9ALvrTDjge64L585wOyAGLKBi4F2uFL/VtzfIXf9Ztzf+WdVXVTK924KyG0cMSbkvMv17cAwVf3I73hMxSUir+Aahyf4HUtFZzdnmZASkUG4njKHcV3+juJKu8aUidc+MgTo6HcsscCqd0yo9Qa+xtVlXwBcag1vpqxEZDLuXoFHVXWr3/HEAqveMcaYOGIlfWOMiSNRV6ffsGFDTUpK8jsMY4ypUJYvX75LVYvrIg1EYdJPSkpi2bJlfodhjDEVioiUdFc6YNU7xhgTVyzpG2NMHLGkb4wxcSTq6vQLc/ToUTIzM/npp5/8DsUUIyEhgebNm1O1alHDyRhj/FYhkn5mZia1atUiKSkJN3CjiTaqyu7du8nMzKRVq1Ylv8AY44sKUb3z008/0aBBA0v4UUxEaNCggV2NGVMGaWmQlASVKrnfaWnhO1aFKOkDlvArAPsbGVN6aWkwahQcOuSeb9ningOMKOu4qsWoECV9Y4yJVWPHHk/4uQ4dcsvDwZJ+EHbv3k3nzp3p3LkzTZo0oVmzZnnPjxw5EtQ+rrvuOjZs2FDsNs899xxp4byuM8ZEna1FDCNX1PLyqjDVO6WRlua+JbduhRYtYNKk8l0mNWjQgFWrVgEwYcIEatasyT333JNvm7xJhysV/j360ksvlXicW2+9texBGmMqpBYtXJVOYcvDIeZK+rn1Y1u2gOrx+rFwFKA3bdpEcnIyI0aMoH379uzYsYNRo0aRmppK+/btmThxYt62vXv3ZtWqVWRnZ1O3bl3GjBlDSkoKZ599Nt9//z0A48aNY+rUqXnbjxkzhu7du9OmTRsWLXITBv34449cdtllJCcnM2zYMFJTU/O+kAKNHz+es846iw4dOnDTTTeRO5rql19+yXnnnUdKSgpdu3Zl8+bNADz66KN07NiRlJQUxobrutKYKBPJBtSiTJoEiYn5lyUmuuVhkVtCjZafbt26aUHr1q07YVlRWrZUdek+/0/LlkHvoljjx4/Xxx9/XFVVN27cqCKiS5cuzVu/e/duVVU9evSo9u7dW9euXauqqr169dKVK1fq0aNHFdC5c+eqquqdd96pkydPVlXVsWPH6pQpU/K2v/fee1VV9a233tILLrhAVVUnT56st9xyi6qqrlq1SitVqqQrV648Ic7cOHJycnT48OF5x+vatavOmTNHVVUPHz6sP/74o86ZM0d79+6thw4dyvfasijN38oYP736qmpiYv48kZjolvsRS8uWqiLud1liAJZpEDk25kr6ka4fO+2000hNTc17PmPGDLp27UrXrl1Zv34969atO+E11atX58ILLwSgW7dueaXtgoYOHXrCNh9//DHDhw8HICUlhfbt2xf62vnz59O9e3dSUlL48MMPWbt2LXv37mXXrl1ccsklgLuZKjExkXnz5nH99ddTvXp1AOrXr1/6E2FMBRPpBtTijBgBmzdDTo77HY5eO7lirk4/0vVjNWrUyHu8ceNGnn76af773/9St25drrnmmkL7rVerVi3vceXKlcnOzi503yeddFKJ2xTm0KFDjB49mhUrVtCsWTPGjRtn/eeNKSDSBcRoEXMl/YjXjwX44YcfqFWrFrVr12bHjh2kp6eH/Bi9evVi1qxZAHz22WeFXkkcPnyYSpUq0bBhQw4cOMDs2bMBqFevHo0aNeLtt98G3E1vhw4dYsCAAUyfPp3Dhw8DsGfPnpDHbUygaKhLL6ogGK4CYrSIuaQ/YgRMmwYtW4KI+z1tWngvl3J17dqV5ORk2rZty69//Wt69eoV8mPcdtttbNu2jeTkZB566CGSk5OpU6dOvm0aNGjAtddeS3JyMhdeeCE9evTIW5eWlsaTTz5Jp06d6N27N1lZWVx88cUMGjSI1NRUOnfuzJQpU0IetzG5ItnZojh+FhD9FHVz5KampmrBSVTWr19Pu3btfIooumRnZ5OdnU1CQgIbN25k4MCBbNy4kSpVoqOmzv5WpiRJSYVXwbZs6eqzIynU3bv9JCLLVTW1pO2iI1OYoB08eJD+/fuTnZ2NqvL8889HTcI3JhjRVJc+YkTFTfJlFXPVO7Gubt26LF++nNWrV7NmzRoGDhzod0imgvG7Pj1e69KjRVBJX0QGicgGEdkkImMKWd9SROaLyBoR+UBEmgesOyYiq7yfOaEM3hhTOtFQnx6vdenRosSkLyKVgeeAC4Fk4CoRSS6w2RPAK6raCZgITA5Yd1hVO3s/g0MUtzGmDKKhb7qfnS1McHX63YFNqvo1gIjMBIYAgX0Fk4G7vMcLgDdDGaQxJjSipT49HuvSo0Uw1TvNgG8Dnmd6ywKtBoZ6jy8FaolIA+95gogsE5HFIvI/hR1AREZ52yzLysoqRfjGmNKw+nQTqobce4BzRGQlcA6wDTjmrWvpdSO6GpgqIqcVfLGqTlPVVFVNbdSoUYhCCp1+/fqdcKPV1KlTufnmm4t9Xc2aNQHYvn07w4YNK3Sbc889l4JdVAuaOnUqhwKuyS+66CL27dsXTOgmivjdgApWn26CS/rbgFMDnjf3luVR1e2qOlRVuwBjvWX7vN/bvN9fAx8AXcofdmRdddVVzJw5M9+ymTNnctVVVwX1+qZNm/L666+X+fgFk/7cuXOpW7dumfdnIi8aGlDB6tNNcEl/KXCGiLQSkWrAcCBfLxwRaSgiufu6D5juLa8nIiflbgP0In9bQIUwbNgw3nnnnbwJUzZv3sz27dvp06dPXr/5rl270rFjR956660TXr9582Y6dOgAuCEShg8fTrt27bj00kvzhj4AuPnmm/OGZR4/fjwAzzzzDNu3b6dfv37069cPgKSkJHbt2gXAU089RYcOHejQoUPesMybN2+mXbt23HjjjbRv356BAwfmO06ut99+mx49etClSxfOP/98du7cCbh7Aa677jo6duxIp06d8oZx+M9//kPXrl1JSUmhf//+ITm38SIaGlBzRXJwLxN9SmzIVdVsERkNpAOVgemqulZEJuKG8pwDnAtMFhEFFgK5s4G0A54XkRzcF8wfVLVcSf93v4NCho8vl86dwcuXhapfvz7du3fn3XffZciQIcycOZMrrrgCESEhIYE33niD2rVrs2vXLnr27MngwYOLnC/2L3/5C4mJiaxfv541a9bQtWvXvHWTJk2ifv36HDt2jP79+7NmzRpuv/12nnrqKRYsWEDDhg3z7Wv58uW89NJLLFmyBFWlR48enHPOOdSrV4+NGzcyY8YM/va3v3HFFVcwe/Zsrrnmmnyv7927N4sXL0ZEeOGFF3jsscd48sknefjhh6lTpw6fffYZAHv37iUrK4sbb7yRhQsX0qpVKxufp5SipQHVmKBu5VTVucDcAsseDHj8OnBC/YWqLgI6ljPGqJBbxZOb9F988UXAzUdw//33s3DhQipVqsS2bdvYuXMnTZo0KXQ/Cxcu5PbbbwegU6dOdOrUKW/drFmzmDZtGtnZ2ezYsYN169blW1/Qxx9/zKWXXpo30ufQoUP56KOPGDx4MK1ataJz585A0cM3Z2ZmcuWVV7Jjxw6OHDlCq1atAJg3b16+6qx69erx9ttv07dv37xtbPjl0on06K/GFKXC3b9fXIk8nIYMGcKdd97JihUrOHToEN26dQPcAGZZWVksX76cqlWrkpSUVKZhjL/55hueeOIJli5dSr169Rg5cmS5hkPOHZYZ3NDMhVXv3Hbbbdx1110MHjyYDz74gAkTJpT5eKZ4kya5OvzAKh5rQDV+sGEYglSzZk369evH9ddfn68Bd//+/Zx88slUrVqVBQsWsKWw4lyAvn378tprrwHw+eefs2bNGsANy1yjRg3q1KnDzp07effdd/NeU6tWLQ4cOHDCvvr06cObb77JoUOH+PHHH3njjTfo06dP0O9p//79NGvmet/+/e9/z1s+YMAAnnvuubzne/fupWfPnixcuJBvvvkGsOGXS8saUE20sKRfCldddRWrV6/Ol/RHjBjBsmXL6NixI6+88gpt27Ytdh8333wzBw8epF27djz44IN5VwwpKSl06dKFtm3bcvXVV+cblnnUqFEMGjQoryE3V9euXRk5ciTdu3enR48e3HDDDXTpEnznqAkTJnD55ZfTrVu3fO0F48aNY+/evXTo0IGUlBQWLFhAo0aNmDZtGkOHDiUlJYUrr7wy6OMYxxpQTTSwoZVNSEXj3yqWhs81pig2tLIxHO8fn1uXnts/Hizxm/hk1TsmpkVT/3hjokGFSfrRVg1lThSNfyPrH29MfhUi6SckJLB79+6oTCrGUVV2795NQkKC36HkYwOMGZNfhajTb968OZmZmdgInNEtISGB5s2bl7xhBFn/eGPyqxBJv2rVqnl3ghpTGrmNtdZ7xxinQiR9Y8rDJuww5rgKUadvjDEmNCzpm7CJhklDjDH5WfWOCQu7KcqY6GQlfRMWdlOUMdHJkr4JC7spypjoZEnfhIXdFGVMdLKkb8Ji0iR3E1QguynKGP9Z0jdhYZOGGBOdrPeOCRu7KcqY6GMlfWOMiSOW9I0xJo5Y0jfGmDhiSd8YY+KIJX1jjIkjlvSNMSaOWNI3xpg4Ykk/BtmQxsaYotjNWTHGhjQ2xhTHSvoxxoY0NsYUx5J+jLEhjY0xxbGkH2NsSGNjTHEs6ccYG9LYGFMcS/oxxoY0NsYUx3rvxCAb0tgYUxQr6RtjTByxpG+MMXHEkr4xxsQRS/rGGBNHgkr6IjJIRDaIyCYRGVPI+pYiMl9E1ojIByLSPGDdtSKy0fu5NpTBG2OMKZ0Sk76IVAaeAy4EkoGrRCS5wGZPAK+oaidgIjDZe219YDzQA+gOjBeReqEL3xhjTGkEU9LvDmxS1a9V9QgwExhSYJtk4H3v8YKA9RcA76nqHlXdC7wHDCp/2MYYY8oimKTfDPg24HmmtyzQamCo9/hSoJaINAjytYjIKBFZJiLLsrKygo3dGGNMKYWqIfce4BwRWQmcA2wDjgX7YlWdpqqpqpraqFGjEIVkjDGmoGDuyN0GnBrwvLm3LI+qbscr6YtITeAyVd0nItuAcwu89oNyxGuMMaYcginpLwXOEJFWIlINGA7MCdxARBqKSO6+7gOme4/TgYEiUs9rwB3oLTPGGOODEpO+qmYDo3HJej0wS1XXishEERnsbXYusEFEvgQaA5O81+4BHsZ9cSwFJnrLjDHG+EBU1e8Y8klNTdVly5b5HYYxxlQoIrJcVVNL2s7uyDXGmDhiSd8YY+KIJX1jjIkjlvSNMSaOWNIPobQ0SEqCSpXc77Q0vyMyxpj8bLrEEElLg1Gj4NAh93zLFvccbOpCY0z0sJJ+iIwdezzh5zp0CMacMBC18cMPP5z49zEmmixaBG+8Ef7jWNIPka1bC1+emQkdO8Lo0TBrFuzcGdm4DKxaBaefDhdeCFF2W4qJc6rw/vtw3nnQqxeMHx/+z6gl/RBp0aLw5XXrQtOm8PLLcOWV0KQJtG3rqn7S0tyXggmfJUugXz84eBAWLoS33/Y7ImNcYp871yX6/v3hiy/gqafg009BJLzHtqQfIpMmQfXq+ZclJsKf/gTp6bBvn0tAjz0GZ5zhSv3XXAOnngqtW8N117kvhm++sdJoqCxcCOefDw0awJo1cOaZcP/9cCzo8V+NCa2cHPjXv6BbN/jlL2H7dvjzn+Hrr+HOO6FGjQgEoapR9dOtWzetqMaMUXUpW7VlS9VXXy162+xs1RUrVKdMUb30UtUGDY6/tnlz1REjVKdNU/3iC9WcnIi9hZjxn/+oVq+u2q6d6rZtbtk//+nO70sv+RqaiUNHj6qmpakmJ7vP4Omnq06frnrkSOiOASzTIHKsjb0TQr//PTzzDOzZU/pv7JwcWLcOPvzQlVA//PB4/X+TJtC3L5xzjvudnOy6hZrCvfUWXHEFtGsH770HuVM0qEKPHvDdd/Dll5CQ4G+cJvYdOQKvvgqTJ8OmTdC+vev0ccUVULlyaI8V7Ng7lvRDKCUFGjaE+fPLvy9Vl5gCvwRy6/8bNHDJP/eLoFOn0H+AKqqZM121WWoqvPsu1CswI/OCBa7R7Ikn4O67/YnRb7lVDDt3uvpjEVeIKO53MNuUZtvcxzVquP+bWPv8/vQTTJ8Of/yj6+TRtSuMGwdDhoSvwBZs0ve9OqfgT0Wt3tm+3V22/eEP4dl/To7qV1+5S8KRI1VbtTpeHVSnjurFF6s+9pjqkiXuUjIeTZ+uKqLat6/qDz8Uvd0FF6jWr6+6d2/kYosmf/3r8c9ONPw0aKD6q1+56rf9+/0+O+Vz8KDqk0+qNmni3tsvfqE6d25kqmix6p3IeuUVuPZaWLECunSJzDG//fb4VcCHH7orA4CaNeEXv3BdFG+9FapWjUw8fnruOdctduBA19c5MbHobVeudCWv++6DRx+NXIzRYM8e16Ddvj28/vrx1JuTU77fZX1tVpbrxTJ3routalU491y45BL3k5Tk9xkLzv797jM4ZQrs2uV65Iwb567Ew90bJ5eV9CNsxAjVk09WPXbMvxi2b1f9v/9TveUW1fbt3b9i9+7uCiGW/fGP7r0OGaL600/Bvebqq11Db24jb7y45RbVSpVUV6/2O5L8jh5V/fBD1XvuUW3T5vhVQMeOqvffr7p4sb//W0XZtUv1gQfc1Tao/vKXqosW+RMLQZb0fU/yBX8qYtI/dky1USOX+KPJP/+pWreuau3aqjNm+B1N6OXkqI4f7z7Fw4eXrifEV1+pVq2qOmpU2MKLOitXuoQ/erTfkZRswwbVJ55QPecc1cqV3d+4cWPV669XffNNV43ipx07VH//e9UaNVxsl12muny5vzFZ0o+gFSvcmXzlFb8jOdHmza5eEVSvu87/f5ZQyclxpUJwiSA7u/T7uO02l1C++CL08UWbnBzV3r1VGzZU3bPH72hKZ/du1/35yitdAQZUTzpJ9aKLXPtEZmbkYtm61X1uEhLcF+iIEaqffx654xfHkn4ETZ7szuT27X5HUrijR1XHjXONnG3auBJfRXbsmOrNN7tzPnp02S/7d+5UrVlTddiw0MYXjV591Z2vv/3N70jK5+efVefNU73jjvydGbp2VZ0wwZW2w9Fo+tVXqjfe6K4Oq1RR/c1vVDduDP1xysOSfgT166faqZPfUZTs/fdVmzZVrVZN9emnK+ZNX0ePql57rfvk3ntv+d9DbvXQkiWhiC46/fCD6imnqKamRme9eFnl5LhS9uTJ7mpWxP0tmzVTvekm1XfeUT18uHzHWLfO9SyqXNldXdx6q7t6jkaW9CPk4EH37f/73/sdSXCyslz3TlC95BL3vKI4ckT1iitc7BMnhuZL64cfXHvMuedWzC/BYPz+9+6cLV7sdyThtXOnu9t66NDjde2Jiar/8z+qL76o+t13we9r1SrVyy93XySJiap33x29V/K5LOlHyL//7c7ie+/5HUnwcnJUp051Jf6mTVUXLPA7opIdPuy+pMA18IXSM8+4/b77bmj3Gw2++MIVSq67zu9IIuvwYff3vOUW1VNPdX9fEdWePVUnTVJds6bwL/nFi49/zmrXVh07tuIUjCzpR8jtt7uuf+W9jPTDihWqZ57p/hnGjYvem7oOHlQdMMB9Wv/859Dv/+efXf1wSkrsVX9ccIFLXqUp5caanBzXjjVxoupZZ2leO0BSkmuUzchwVZ+5n7H69VUffrji3bxnST9C2rRRHTTI7yjK7sABd4cvqPbqFX31lfv3u14nlSqpvvxy+I6TlubOQVpa+I4RaW++6d7TlCl+RxJdtm93gxlecokrsOV+CTRurPr44+5/oiIKNunbHbnlsGWLu2PwqafcsKgV2WuvwU03uTFQXngBLrvM74jcHZqDBrk7aNPS3CBV4ZKT44a73b/fjW1erVr4jhUJhw+7gflq1HDnLx7uyi6LQ4fcWFl798Lll584PHpFEuwduTZWYzlkZLjfF1zgbxyhcPXVLjmccQYMG+a+AA4f9i+e7793k5+sXu0GBwtnwgc3CNYf/uDmM3j++fAeKxIefxw2b4Znn7WEX5zERDfcw69/XbETfmlY0i+HjAxo1swN4RsLTjsNPv7YDRH9/PNw1lnw+eeRj2PbNjeC6KZN8M477p8yEgYOdF80Dz8MBw5E5pjhsGWLG8r38svd+zEmkCX9Mjp2DObNc6X8SA2oFAnVqrnZvdLT3WBYZ50Ff/1r5Gbz+uYb6NPHzSiUnu5mvooUEVfaz8qCJ5+M3HFD7e673Xt54gm/IzHRyJJ+GS1d6qZAHDjQ70jCY+BAN8Vg375w882u1Lh3b3iP+eWX7nj79rl61t69w3u8wnTv7qq3nnyyYk5iP28ezJ7tJuooat5mE98s6ZdRRoYrTUWyJBppjRu7iUgee8zNRpWS4qp/wuGzz1zCP3IEPvjAXWH4ZdIk157xyCP+xVAWR4/C7be7OZfjdYIYUzJL+mWUnu5mZ2rQwO9IwqtSJVfHv2iRaxA85xxX5x3KycWXLXNjqFeu7OYF6NQpdPsuizPPhBtucO0aX3/tbyyl8eyzsH49TJ1qU0GaolnSL4N9+2DJktit2inMWWe53j3Dh8ODD7ornG3byr/fTz5xE07Urg0ffQRt25Z/n6Hw4INQpQo88IDfkQTnu+9gwgQ3cc7FF/sdjYlmlvTLYMECV9KNha6apVG7tpvk+eWXXZtGp04wZ07Z9zd/vvvibNLEJfzWrUMWark1bQr+CLDYAAARlElEQVS/+527f2HlSr+jKdn//i/8/DM8/XRsdSwwoWdJvwzS06FWLejZ0+9IIk/ETQu5fDm0bOkmer79djcRdGm88w788pcu0S9cCM2bhyfe8rj3Xqhf302rGM0WLXLTdd51l7vPwpjiWNIvJVWX9M87L75vemnTBj79FO64w9Ul9+zp7mQNxuuvw6WXQseOrtG2ceOwhlpmdevC/fe7v/f77/sdTeGOHYPbbnP3i4wd63c0piKwpF9Kmza5Ox3jqT6/KCed5BoN337b1e936wbTpxffp/8f/4Arr3RdI+fNi/6G8FtvhVNPhTFjInevQmm8+CKsWOHuwK1Z0+9oTEVgSb+UcodesKR/3MUXu+ESevSA3/zGDemwf/+J202b5qqG+vVzpec6dSIfa2klJMDEia4NY/Zsv6PJb88edyXSt69rYDcmGJb0Syk93dVDn36635FEl6ZN4b33XB/3f/4TunRxPZxyTZkCv/0tXHQR/PvfbiCwiuJXv4L27V2CPXrU72iOe+ABd8Pcs89a460JniX9UjhyxPXcsVJ+4SpXdolx4UJX19y7N/zxj+4mp7vucne6/utfFa8PeeXKbiybjRtd9VU0WL3aDY9xyy3+39dgKpagkr6IDBKRDSKySUTGFLK+hYgsEJGVIrJGRC7ylieJyGERWeX9/DXUbyCSFi+Ggwfjr6tmaf3iF7BqlevZM2aMK5H+6lcwY0bFHbL44ouhVy946CH48Ud/Y1GF0aNdz6KJE/2NxVQ8JSZ9EakMPAdcCCQDV4lIcoHNxgGzVLULMBz4c8C6r1S1s/dzU4ji9kV6uiv12ciFJatXz1XzvPSSK+m//LK72amiEnFXLTt2uL7wfnrtNTccxuTJ7jwbUxrB/Bt2Bzap6tcAIjITGAKsC9hGgdre4zrA9lAGGS0yMuDssytGA2Q0EIGRI/2OInR69YLBg13y/+1v/el5dOCAGxYjNRWuvz7yxzcVXzDVO82AbwOeZ3rLAk0ArhGRTGAucFvAulZetc+HItKnsAOIyCgRWSYiy7KysoKPPoJ27XI3JFl9fnx79FFXxTd5sj/Hf+QRd7Xxpz+5cZGMKa1QfWyuAl5W1ebARcA/RKQSsANo4VX73AW8JiK1C75YVaepaqqqpjZq1ChEIYXWvHmuLtWSfnxr397NsvTss7B1a2SPvWGD6wU1cqTrHmtMWQST9LcBpwY8b+4tC/QbYBaAqn4KJAANVfVnVd3tLV8OfAWcWd6g/ZCR4epPU0ucgdLEuoceclVX48dH7piq7u7n6tXdRC/GlFUwSX8pcIaItBKRariG2oLDbG0F+gOISDtc0s8SkUZeQzAi0ho4A6hAg9U6uUMvnH++a8g18a1FC9d75u9/j9x0knPmuM/gQw9F77AVpmIoMemrajYwGkgH1uN66awVkYkiMtjb7G7gRhFZDcwARqqqAn2BNSKyCngduElV94TjjYTTunVu+j7rqmly3XefG3X0/vvDf6zDh+HOOyE52Q0LYUx5BNWJTlXn4hpoA5c9GPB4HdCrkNfNBqLs5vXSS093vwcM8DcOEz0aNHDDGd9/v+s+Gc6pHR9/3M0dPH9+fA/yZ0LD2v+DkJHhJvewOUdNoDvugFNOcck/XIOxbdniegpdfrkb2dWY8rKkX4KffnJT+FnVjikoMdHNVrVokRtpNBzuvts1Gj/xRHj2b+KPJf0SfPSRS/zWVdMU5vrr3Zy6990X2nmDwXUTnj3bVSHZVaYJFUv6JcjIcOPFnHOO35GYaFSlirtha906N3tVqBw96mYka90a7rkndPs1xpJ+CdLTXSNdRRoK2ETW0KFuUpgHH3Q9bULh2Wdh/Xo3SU1FG5XURDdL+sXYsQM++8yqdkzxRNwNU5mZ8Nxz5d/fd9+5toILL3SjexoTSpb0i5E7S5Y14pqS9OsHgwa5qp59+8q3rzFjXDvS1Kk2OYoJPUv6xcjIgJNPtkkqTHAmT3YzWT32WNn3sWiRu9P3rrtcA7ExoWZJvwg5OW76v4EDbTRDE5zOnd38wFOnuju4S+vYMbjtNjf15LhxoY/PGLCkX6RVqyAry+rzTek8/DBkZ7sxckrrxRdhxQrXJ79mzdDHZgxY0i9Sbn2+Db1gSqN1a7jpJpfAN2wI/nV79rj++H37wvDh4YvPGEv6RUhPh5QUaNLE70hMRTNunBsCeezY4F/zwAOuPeDZZ63x1oSXJf1CHDwIn3xiVTumbE4+2d1QNXs2LFlS8varV8Nf/wq33GKdBkz4WdIvxIcfujsiraumKau77oJGjVz3y+IGY1N1jbf168PEiZGLz8QvS/qFSE93l+e9Thgs2pjg1Krl7tD94IPjQ3MXZsYMN77To4+6mdmMCTfRcI0JW0apqam6bNkyX2No29Y1yM2dW/K2xhTlyBFo1859AaxYcWLX3wMHoE0baNYMFi+2WdlM+YjIclUtcUJXK+kXsGWL63Vh9fmmvKpVc104V692JfqCHnnEDfXxpz9ZwjeRY0m/gNyumpb0TSgMH+5u2ho3Dn7++fjyDRtgyhQYORJ69PAtPBOHLOkXkJEBzZu7y3JjyqtSJTcY2+bN8Pzzbpmqm3WrenW3zphIsqQfIDvbTVwxcKD1lTahM3Cgm+rwkUdcPf6cOa5x96GHoHFjv6Mz8caSfoBly9wIidZV04RS7tDLWVku8d95JyQnw623+h2ZiUdV/A4gmqSnu3/Q/v39jsTEmrPOgmHDjo/AOX8+VK3qb0wmPllJP0BGhvvnbNDA70hMLJo0yU2vOGyYq+4xxg9W0vfs2+dumb/vPr8jMbHqzDPdXLo2ybnxkyV9z/vvu/HMraumCaczzvA7AhPvrHrHk5Hh7pzs2dPvSIwxJnws6eP6Taenu3pWa1wzxsQyS/rApk3u5hnrqmmMiXWW9Dk+CqLV5xtjYp0lfVx9fuvWcNppfkdijDHhFfdJ/8gRWLDAqnaMMfEh7pP+p5+66RGtascYEw/iPulnZLixzO0OSWNMPIj7pJ+eDmefDbVr+x2JMcaEX1wn/awsN42dVe0YY+JFXCf9+fPdjVnWiGuMiRdxnfTT06F+fejWze9IjDEmMuI26au6Rtzzz7dJqY0x8SNuk/7atbB9u9XnG2PiS1BJX0QGicgGEdkkImMKWd9CRBaIyEoRWSMiFwWsu8973QYRiZra84wM99uSvjEmnpQ4nr6IVAaeAwYAmcBSEZmjqusCNhsHzFLVv4hIMjAXSPIeDwfaA02BeSJypqoeC/UbKa30dGjXDk491e9IjDEmcoIp6XcHNqnq16p6BJgJDCmwjQK5Pd3rANu9x0OAmar6s6p+A2zy9uerw4dh4UIr5Rtj4k8wSb8Z8G3A80xvWaAJwDUikokr5d9WitciIqNEZJmILMvKygoy9LL7+GP46SfrqmmMiT+hasi9CnhZVZsDFwH/EJGg962q01Q1VVVTGzVqFKKQipaeDtWqQd++YT+UMcZElWDmyN0GBNZ8N/eWBfoNMAhAVT8VkQSgYZCvjbiMDOjTB2rU8DsSY4yJrGBK40uBM0SklYhUwzXMzimwzVagP4CItAMSgCxvu+EicpKItALOAP4bquDLYvt2+Owzq883xsSnEkv6qpotIqOBdKAyMF1V14rIRGCZqs4B7gb+JiJ34hp1R6qqAmtFZBawDsgGbvW7585777nflvSNMfFIXG6OHqmpqbps2bKw7X/ECDfmzvbtUClub00zxsQaEVmuqqklbRdXaS8nx9XnDxhgCd8YE5/iKvWtWgW7dllXTWNM/IqrpJ+e7n4PGOBvHMYY45e4SvoZGdC5MzRu7Hckxhjjj7hJ+gcPwiefWK8dY0x8i5uk/8EHcPSoJX1jTHyLm6SfkQHVq0Pv3n5HYowx/ombpJ+eDueeCyed5Hckxhjjn7hI+ps3w5dfWldNY4yJi6Rvs2QZY4wTN0m/eXNo29bvSIwxxl8xn/Szs91YOxdcACJ+R2OMMf6K+aS/dCns22dVO8YYA3GQ9DMyXAn//PP9jsQYY/wX80k/PR3OOgvq1/c7EmOM8V9MJ/19+2DJEqvaMcaYXDGd9N9/342hb/3zjTHGiemkn54OtWpBjx5+R2KMMdEhZpO+qkv6/ftD1ap+R2OMMdEhZpP+xo2wZYvV5xtjTKCYTfq5Qy9Yfb4xxhwX00n/tNOgdWu/IzHGmOgRk0n/yBFYsMCqdowxpqCYTPqffuqmR7SqHWOMyS8mk356OlSpAv36+R2JMcZEl5hM+hkZcPbZULu235EYY0x0iZmkn5YGSUlucLXly6FxY78jMsaY6BMTST8tDUaNcv3yc739tltujDHmuJhI+mPHwqFD+Zf9/LNbbowx5riYSPpbt5ZuuTHGxKuYSPotWpRuuTHGxKuYSPqTJkFiYv5liYluuTHGmONiIumPGAHTpkFCgnvesqV7PmKEv3EZY0y0iYmkDzB0qPv9u9/B5s2W8I0xpjAxk/T37YNLL4XBg/2OxBhjolcVvwMIlVNOgdde8zsKY4yJbjFT0jfGGFMyS/rGGBNHLOkbY0wcCSrpi8ggEdkgIptEZEwh66eIyCrv50sR2Rew7ljAujmhDN4YY0zplNiQKyKVgeeAAUAmsFRE5qjqutxtVPXOgO1vA7oE7OKwqnYOXcjGGGPKKpiSfndgk6p+rapHgJnAkGK2vwqYEYrgjDHGhFYwSb8Z8G3A80xv2QlEpCXQCng/YHGCiCwTkcUi8j9ljtQYY0y5hbqf/nDgdVU9FrCspapuE5HWwPsi8pmqfhX4IhEZBYwCaGGjpBljTNgEk/S3AacGPG/uLSvMcODWwAWqus37/bWIfICr7/+qwDbTgGkAIpIlIluo2BoCu/wOIorY+cjPzsdxdi7yK8/5aBnMRsEk/aXAGSLSCpfshwNXF9xIRNoC9YBPA5bVAw6p6s8i0hDoBTxW3MFUtVEwgUczEVmmqql+xxEt7HzkZ+fjODsX+UXifJSY9FU1W0RGA+lAZWC6qq4VkYnAMlXN7YY5HJipqhrw8nbA8yKSg2s/+ENgrx9jjDGRFVSdvqrOBeYWWPZggecTCnndIqBjOeIzxhgTQnZHbnhM8zuAKGPnIz87H8fZucgv7OdD8tfGGGOMiWVW0jfGmDhiSd8YY+KIJf0QEpFTRWSBiKwTkbUicoffMflNRCqLyEoR+bffsfhNROqKyOsi8oWIrBeRs/2OyU8icqf3f/K5iMwQkQS/Y4okEZkuIt+LyOcBy+qLyHsistH7XS/Ux7WkH1rZwN2qmgz0BG4VkWSfY/LbHcB6v4OIEk8D/1HVtkAKcXxeRKQZcDuQqqodcN3Bh/sbVcS9DAwqsGwMMF9VzwDme89DypJ+CKnqDlVd4T0+gPunLnSconggIs2BXwIv+B2L30SkDtAXeBFAVY+o6r7iXxXzqgDVRaQKkAhs9zmeiFLVhcCeAouHAH/3Hv8dCPl4ZZb0w0REknBDTizxNxJfTQXuBXL8DiQKtAKygJe86q4XRKSG30H5xRue5QlgK7AD2K+qGf5GFRUaq+oO7/F3QONQH8CSfhiISE1gNvA7Vf3B73j8ICIXA9+r6nK/Y4kSVYCuwF9UtQvwI2G4dK8ovLrqIbgvw6ZADRG5xt+ooos3ukHI+9Rb0g8xEamKS/hpqvovv+PxUS9gsIhsxs3BcJ6IvOpvSL7KBDJVNffK73Xcl0C8Oh/4RlWzVPUo8C/gFz7HFA12isgpAN7v70N9AEv6ISQigquzXa+qT/kdj59U9T5Vba6qSbgGuvdVNW5Lcqr6HfCtiLTxFvUH4nkcqq1ATxFJ9P5v+hPHDdsB5gDXeo+vBd4K9QEs6YdWL+BXuFJt7rzAF/kdlIkatwFpIrIG6Aw86nM8vvGueF4HVgCf4XJRXA3JICIzcKMStxGRTBH5DfAHYICIbMRdDf0h5Me1YRiMMSZ+WEnfGGPiiCV9Y4yJI5b0jTEmjljSN8aYOGJJ3xhj4oglfWOMiSOW9I0xJo78P0Qn+Avj7qODAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VOX1+PHPYZdFdhcWCSoqQfaIWL6KuCBuIBKVxQWXov5EFLWVulSL0rrVtWjFVtwIiCiIK1LBra1IEEQBKYggYQcFRFAIOb8/zg0MMSSTZDJ3MnPer1demblz752TSXLmmec+z3lEVXHOOZcaKoUdgHPOufjxpO+ccynEk75zzqUQT/rOOZdCPOk751wK8aTvnHMpxJO+KxERqSwi20TksFjuGyYROVJEYj52WUROE5HlEfcXi8iJ0exbiuf6h4jcVtrjizjvvSLyXKzP68JTJewAXPkSkW0Rd2sCvwC7g/tXq+q4kpxPVXcDtWO9bypQ1aNjcR4RuQq4WFVPjjj3VbE4t0t+nvSTnKruSbpBS/IqVf3X/vYXkSqqmhuP2Jxz8efdOyku+Pj+soiMF5EfgYtF5AQR+VRENovIGhF5XESqBvtXEREVkbTg/kvB4++IyI8i8l8RaVnSfYPHzxSR/4nIFhF5QkT+LSKD9xN3NDFeLSJLReQHEXk84tjKIvKIiGwSkWVAryJen9tFZEKBbaNF5OHg9lUisij4eb4JWuH7O1eOiJwc3K4pIi8GsS0AOhfY9w4RWRacd4GI9A62twX+BpwYdJ1tjHht7444/prgZ98kIlNE5NBoXpviiEjfIJ7NIjJDRI6OeOw2EVktIltF5OuIn7WriHwebF8nIg9G+3yuHKiqf6XIF7AcOK3AtnuBncC5WCPgAOA44Hjsk+DhwP+AocH+VQAF0oL7LwEbgQygKvAy8FIp9j0I+BHoEzx2E7ALGLyfnyWaGF8H6gJpwPf5PzswFFgANAMaAh/Zv0Khz3M4sA2oFXHu9UBGcP/cYB8BTgF2AO2Cx04DlkecKwc4Obj9EPABUB9oASwssO+FwKHB72RgEMPBwWNXAR8UiPMl4O7gds8gxg5ADeBJYEY0r00hP/+9wHPB7dZBHKcEv6PbgMXB7TbACuCQYN+WwOHB7dnAgOB2HeD4sP8XUvnLW/oO4BNVfUNV81R1h6rOVtVZqpqrqsuAMUD3Io6fpKrZqroLGIclm5Luew4wT1VfDx57BHuDKFSUMf5FVbeo6nIsweY/14XAI6qao6qbgPuKeJ5lwFfYmxHA6cAPqpodPP6Gqi5TMwN4Hyj0Ym0BFwL3quoPqroCa71HPu9EVV0T/E6ysDfsjCjOCzAI+IeqzlPVn4ERQHcRaRaxz/5em6L0B6aq6ozgd3Qf9sZxPJCLvcG0CboIvw1eO7A371Yi0lBVf1TVWVH+HK4ceNJ3ACsj74jIMSLyloisFZGtwEigURHHr424vZ2iL97ub98mkXGoqmIt40JFGWNUz4W1UIuSBQwIbg8M7ufHcY6IzBKR70VkM9bKLuq1yndoUTGIyGAR+SLoRtkMHBPlecF+vj3nU9WtwA9A04h9SvI7299587DfUVNVXQzcjP0e1gfdhYcEu14OpAOLReQzETkryp/DlQNP+g7s436kp7HW7ZGqeiDwR6z7ojytwbpbABARYd8kVVBZYlwDNI+4X9yQ0onAaSLSFGvxZwUxHgBMAv6Cdb3UA96LMo61+4tBRA4HngKuBRoG5/064rzFDS9djXUZ5Z+vDtaNtCqKuEpy3krY72wVgKq+pKrdsK6dytjrgqouVtX+WBfeX4FXRaRGGWNxpeRJ3xWmDrAF+ElEWgNXx+E53wQ6ici5IlIFuAFoXE4xTgRuFJGmItIQuLWonVV1LfAJ8BywWFWXBA9VB6oBG4DdInIOcGoJYrhNROqJzWMYGvFYbSyxb8De/36LtfTzrQOa5V+4LsR44EoRaSci1bHk+7Gq7veTUwli7i0iJwfP/TvsOswsEWktIj2C59sRfOVhP8AlItIo+GSwJfjZ8soYiyslT/quMDcDl2H/0E9jF1zLlaquAy4CHgY2AUcAc7F5BbGO8Sms7/1L7CLjpCiOycIuzO7p2lHVzcBwYDJ2MTQTe/OKxl3YJ47lwDvACxHnnQ88AXwW7HM0ENkPPh1YAqwTkchumvzj38W6WSYHxx+G9fOXiaouwF7zp7A3pF5A76B/vzrwAHYdZi32yeL24NCzgEVio8MeAi5S1Z1ljceVjljXqXOJRUQqY90Jmar6cdjxOJcsvKXvEoaI9Aq6O6oDd2KjPj4LOSznkoonfZdI/g9YhnUdnAH0VdX9de8450rBu3eccy6FeEvfOedSSMIVXGvUqJGmpaWFHYZzzlUoc+bM2aiqRQ1zBhIw6aelpZGdnR12GM45V6GISHEzywHv3nHOuZTiSd8551KIJ33nnEshCden75yLr127dpGTk8PPP/8cdiguCjVq1KBZs2ZUrbq/0ktF86TvXIrLycmhTp06pKWlYcVNXaJSVTZt2kROTg4tW7Ys/oBCJE33zrhxkJYGlSrZ93ElWu7budT1888/07BhQ0/4FYCI0LBhwzJ9KkuKlv64cTBkCGzfbvdXrLD7AIPKXFvQueTnCb/iKOvvKila+rffvjfh59u+3bY755zbK6qkH1Q/XCwiS0VkRCGPDxaRDSIyL/i6KuKxy0RkSfB1WSyDz/fddyXb7pxLHJs2baJDhw506NCBQw45hKZNm+65v3NndGX3L7/8chYvXlzkPqNHj2ZcjPp9/+///o958+bF5FzxVmz3TlDXfDS2IHQOMFtEpqrqwgK7vqyqQwsc2wBbLCIDWy1nTnDsDzGJPnDYYdalU9h251xsjRtnn6K/+87+x0aNKls3asOGDfck0LvvvpvatWtzyy237LOPqqKqVKpUeDt17NixxT7PddddV/ogk0g0Lf0uwFJVXRasdjMBWyc0GmcA01X1+yDRT8dW24mpUaOgZs19t9Wsadudc7GTf/1sxQpQ3Xv9rDwGTixdupT09HQGDRpEmzZtWLNmDUOGDCEjI4M2bdowcuTIPfvmt7xzc3OpV68eI0aMoH379pxwwgmsX78egDvuuINHH310z/4jRoygS5cuHH300fznP/8B4KeffqJfv36kp6eTmZlJRkZGsS36l156ibZt23Lsscdy2223AZCbm8sll1yyZ/vjjz8OwCOPPEJ6ejrt2rXj4osvjvlrFo1okn5TYGXE/RwKX7C6n4jMF5FJIpK/4HNUx4rIEBHJFpHsDRs2RBn6XoMGwZgx0KIFiNj3MWP8Iq5zsRbv62dff/01w4cPZ+HChTRt2pT77ruP7OxsvvjiC6ZPn87ChQU7HGDLli10796dL774ghNOOIFnn3220HOrKp999hkPPvjgnjeQJ554gkMOOYSFCxdy5513Mnfu3CLjy8nJ4Y477mDmzJnMnTuXf//737z55pvMmTOHjRs38uWXX/LVV19x6aWXAvDAAw8wb9485s+fz9/+9rcyvjqlE6sLuW8AaaraDmvNP1+Sg1V1jKpmqGpG48bFFokr1KBBsHw55OXZd0/4zsVevK+fHXHEEWRkZOy5P378eDp16kSnTp1YtGhRoUn/gAMO4MwzzwSgc+fOLF++vNBzn3/++b/a55NPPqF///4AtG/fnjZt2hQZ36xZszjllFNo1KgRVatWZeDAgXz00UcceeSRLF68mGHDhjFt2jTq1q0LQJs2bbj44osZN25cqSdXlVU0SX8V0DzifrNg2x6quilihaN/AJ2jPdY5V3Hs7zpZeV0/q1Wr1p7bS5Ys4bHHHmPGjBnMnz+fXr16FTpevVq1antuV65cmdzc3ELPXb169WL3Ka2GDRsyf/58TjzxREaPHs3VV18NwLRp07jmmmuYPXs2Xbp0Yffu3TF93mhEk/RnA61EpKWIVAP6A1MjdxCRQyPu9gYWBbenAT1FpL6I1Ad6BtuccxVQmNfPtm7dSp06dTjwwANZs2YN06bFPpV069aNiRMnAvDll18W+kki0vHHH8/MmTPZtGkTubm5TJgwge7du7NhwwZUlQsuuICRI0fy+eefs3v3bnJycjjllFN44IEH2LhxI9sL9pXFQbGjd1Q1V0SGYsm6MvCsqi4QkZFAtqpOBYaJSG8gF/geGBwc+72I3IO9cQCMVNXvy+HncM7FQX63aSxH70SrU6dOpKenc8wxx9CiRQu6desW8+e4/vrrufTSS0lPT9/zld81U5hmzZpxzz33cPLJJ6OqnHvuuZx99tl8/vnnXHnllagqIsL9999Pbm4uAwcO5McffyQvL49bbrmFOnXqxPxnKE7CrZGbkZGhZVlERdUu5jqXaFThhhvglFPgvPPCjmavRYsW0bp167DDSAi5ubnk5uZSo0YNlixZQs+ePVmyZAlVqiRW8YLCfmciMkdVM/ZzyB5JMSMXrNXRqRNMmRJ2JM4V7tNP4Ykn4IIL4O23w47GFWbbtm1069aN9u3b069fP55++umES/hllTQ/TZMmlvgnTYK+fcOOxrlfy8qCGjWgdWvo1w/eew9OPDHsqFykevXqMWfOnLDDKFdJ09KvUsWS/RtvgJcFd4kmNxdefhnOPRemTbO5JOecA8UMA3cu5pIm6QNkZsKPP8L06WFH4ty+3n8fNmyAAQOgcWP7G61bF844A/73v7Cjc6kkqZL+KadA/frwyithR+LcvsaPtyQfzBmiefO9jZPTT4ecnPBic6klqZJ+1arQpw9MnQq//FL8/s7Fw44d8Npr1o9fo8be7UcfDe++C5s3W+LfuDG8GF3qSKqkD9bFs2WLfZx2LhG89ZZ1Ow4c+OvHOnWy61DLl9ungK1b4x5e6Hr06PGriVaPPvoo1157bZHH1a5dG4DVq1eTmZlZ6D4nn3wyxQ0Bf/TRR/eZJHXWWWexefPmaEIv0t13381DDz1U5vPEWtIl/dNOgwMPtFE8ziWCrCw45BA4+eTCHz/pJPt7nTfPPqmm2kCEAQMGMGHChH22TZgwgQEDBkR1fJMmTZhUhn/4gkn/7bffpl69eqU+X6JLuqRfvbr940yZArt2hR2NS3WbN1tLv39/qFx5//udfTY8/zx8+CFcdJGN9kkVmZmZvPXWW3sWTFm+fDmrV6/mxBNPZNu2bZx66ql06tSJtm3b8vrrr//q+OXLl3PssccCsGPHDvr370/r1q3p27cvO3bs2LPftddeu6cs81133QXA448/zurVq+nRowc9evQAIC0tjY1BX9vDDz/Msccey7HHHrunLPPy5ctp3bo1v/3tb2nTpg09e/bc53kKM2/ePLp27Uq7du3o27cvP/zww57nzy+1nF/o7cMPP9yziEzHjh358ccfS/3aFiZpxulHysyEF1+EmTOhZ8+wo3Gp7LXXYOfOwrt2Cho4EH74AYYOhSuvhLFjYT9rhpSbG2+0Txyx1KEDBPmyUA0aNKBLly6888479OnThwkTJnDhhRciItSoUYPJkydz4IEHsnHjRrp27Urv3r33u07sU089Rc2aNVm0aBHz58+nU6dOex4bNWoUDRo0YPfu3Zx66qnMnz+fYcOG8fDDDzNz5kwaNWq0z7nmzJnD2LFjmTVrFqrK8ccfT/fu3alfvz5Llixh/PjxPPPMM1x44YW8+uqrRdbHv/TSS3niiSfo3r07f/zjH/nTn/7Eo48+yn333ce3335L9erV93QpPfTQQ4wePZpu3bqxbds2akReCIqBpGvpgyX62rW9i8eFLysLjjwSMoqdHG+uuw7uuQdeeAGGD7fSDakgsosnsmtHVbntttto164dp512GqtWrWLdunX7Pc9HH320J/m2a9eOdu3a7Xls4sSJdOrUiY4dO7JgwYJii6l98skn9O3bl1q1alG7dm3OP/98Pv74YwBatmxJhw4dgKLLN4PV99+8eTPdu3cH4LLLLuOjjz7aE+OgQYN46aWX9sz87datGzfddBOPP/44mzdvjvmM4KRs6deoYZNgJk+GJ5+0iVvOxduaNTBjBtx5Z8nqQd1+O3z/PTzyCDRsCH/8Y/nFWFBRLfLy1KdPH4YPH87nn3/O9u3b6dzZqrOPGzeODRs2MGfOHKpWrUpaWlqh5ZSL8+233/LQQw8xe/Zs6tevz+DBg0t1nnz5ZZnBSjMX172zP2+99RYfffQRb7zxBqNGjeLLL79kxIgRnH322bz99tt069aNadOmccwxx5Q61oKSsqUP1sWzcaP1kToXhpdftpZ6lNcj9xCBhx6CwYPhrrusXk+yq127Nj169OCKK67Y5wLuli1bOOigg6hatSozZ85kRWGLYUc46aSTyMrKAuCrr75i/vz5gJVlrlWrFnXr1mXdunW88847e46pU6dOof3mJ554IlOmTGH79u389NNPTJ48mRNLUTejbt261K9ff8+nhBdffJHu3buTl5fHypUr6dGjB/fffz9btmxh27ZtfPPNN7Rt25Zbb72V4447jq+//rrEz1mUpG0D9+pldb4nTYJTTw07GpeKsrJsSGZpGmmVKsEzz9iF4GHDbNJhSEuqxs2AAQPo27fvPiN5Bg0axLnnnkvbtm3JyMgotsV77bXXcvnll9O6dWtat2695xND+/bt6dixI8cccwzNmzffpyzzkCFD6NWrF02aNGHmzJl7tnfq1InBgwfTpUsXAK666io6duxYZFfO/jz//PNcc801bN++ncMPP5yxY8eye/duLr74YrZs2YKqMmzYMOrVq8edd97JzJkzqVSpEm3atNmzClisJF1p5UgXXmgt/dWrix454VysLVkCRx1lLfabby79eX7+2Ub2fPihdVeee27sYsznpZUrHi+tvB8XXADr18Mnn4QdiUs148dbN81FF5XtPDVq2PDjTp3s7/mDD2ISnkthUSV9EeklIotFZKmIjChiv34ioiKSEdxPE5EdIjIv+Pp7rAKPxplnwgEH+CgeF1+q1rXTvTs0a1b289WpA++8A0ccAb17Q5JX/nXlrNikLyKVgdHAmUA6MEBE0gvZrw5wAzCrwEPfqGqH4OuaGMQctdq1LfG/+irk5cXzmV0qmzcPFi8u+QXcojRsaPX3GzSw61UxvrZHonXzuv0r6+8qmpZ+F2Cpqi5T1Z3ABKBPIfvdA9wPJNQk8sxMGzr33/+GHUlq2rkTNm0KO4r4ysqy4n/9+sX2vE2bwr/+ZdenTj/dFg2KhRo1arBp0yZP/BWAqrJp06YyTdiKZvROU2BlxP0c4PjIHUSkE9BcVd8Skd8VOL6liMwFtgJ3qOrHBZ9ARIYAQwAOO+ywEoRfvLPPttIMr7wC5bCOsivCrl1Wb+a772Dp0n0rTCarvDzrz+/Vy1rnsXbkkbYIS/fulvg//hgOOqhs52zWrBk5OTls2LAhNkG6clWjRg2alaHfsMxDNkWkEvAwMLiQh9cAh6nqJhHpDEwRkTaquk8tQVUdA4wBG71T1pgiHXigLVTx6qvw8MPxn9aeyu6+e+8nrAkTbNx5svv4Y1i1ykbtlJf27a2ez+mn25vLzJlWq7+0qlatSsuWLWMXoEto0aTAVUDziPvNgm356gDHAh+IyHKgKzBVRDJU9RdV3QSgqnOAb4CjYhF4SWRm2iIVn30W72dOXTNmwF/+AldcAW3b2uzSVOg9yMqCWrXKZ2hlpG7dbAjnV1/Zc5VyQqhLQdEk/dlAKxFpKSLVgP7A1PwHVXWLqjZS1TRVTQM+BXqraraINA4uBCMihwOtgGUx/ymK0bu39bH6KJ742LgRLrnExqk//rgV8Zo/31qkyWznTutGPO88S/zl7Ywz4KWXbEjyBRd4VVkXnWKTvqrmAkOBacAiYKKqLhCRkSLSu5jDTwLmi8g8YBJwjap+X9agS6puXSvCNmlSarQ2w6RqFSI3brS+7Vq1rHrkQQdZaz+ZTZtmVTKjqagZKxdeCH//u3X3DB7so9RcFFQ1ob46d+6s5WHsWFVQnT27XE7vAn/7m73Ojzyy7/a77rLtixeHElZc9O+v2rCh6s6d8X/u++6z1/e661Tz8uL//C58QLZGkWNT5rJm795WbdO7eMrPl19ayYEzz4Qbbtj3sWuvhWrV4LHHwomtvG3bBq+/bi3vqlXj//y33gq//z2MHm1F2pzbn5RJ+g0aWOG1V17xLp7ysH27rQ5Vrx4899yvSwkffLB1ezz3nHWBJJvXX7eLqfHs2inovvvgqqusHn+yd6W50kuZpA82imfZstivDOSshb9woS3+sb9x4zfeaG8OzzwT39jiISsLDjsMfvOb8GIQsf79Cy6Am26yN1jnCkqppH/eeTab0bt4YmvyZEs2v/td0ctTtm8Pp5xi9eGTaaTJhg12EXfAgPDngVSubEuF9uxpF9SnTAk3Hpd4UirpN2oEPXp4F08srVxpySUjA+69t/j9hw+3OROvvlr+scXLpEmwe3e4XTuRqle3tXmPP96qfL7/ftgRuUSSUkkfrItnyRKb1OLKZvduG4+/c6d1b1SrVvwxZ50FrVol12StrCxo08YmoSWKWrVsGOfRR0OfPjCrYBlEl7JSLumfd559BPcunrL7y19scY8nn7REHo1KlWxkz2efJUcRvO++s8lRAwaUbB3ceKhf37qdDj7Y3mwXLAg7IpcIUi7pH3wwnHSSJ/2y+s9/rLbOwIHW2i+Jyy6zUT5hLcIdS/kr+8WyjHIsHXooTJ9uXT49e8K334YdkQtbyiV9sC6ehQvty5Xc5s2W7A87DJ56quQt3Nq1YcgQ69cvZp3rhJeVBV27wuGHhx3J/h1+uNXi37HDirStXRt2RC5MKZn0+/a1ROWt/ZJThauvtkqS48dbFdPSGDrUfgdPPBHb+OJpwQL44ovEuYBblGOPhbfftoR/xhnJOVfCRSclk36TJlal0JN+yY0dCxMnwsiRNjqktJo3t/HkzzwDP/4Yu/jiafx4u0Zx4YVhRxKdrl1tCOfXX8M559icCZd6UjLpgyWcL7+0Ze1cdBYvhuuvt7H2v/992c93442wdau9kVQ0+evgnnaaXSeqKE47zeL+739j8zt0FU/KJv3zz7fvyTRevDz98ouVWTjgAJv8U7ly2c95/PFwwglWj2f37rKfL55mzbKLohWha6egfv1sBNXo0clf7tr9Wsom/WbNLOF4F090/vAHK18xdqx1j8XK8OFWGuPNN2N3znjIyrIRMX37hh1J6YwaZcNsr7jCisW51JGySR9sFM/cufDNN2FHktjeeccmUw0dGvsVofr2hRYtKlaBsNxcePlley1KeyE7bDVr2hv4ihXezZNqUjrp9+tn372LZ//WrrVx9W3bwoMPxv78VarYdYIPP7Q34IpgxgxYv75idu1E6tbNPmk99ZT9TC41RJX0RaSXiCwWkaUiMqKI/fqJiIpIRsS2PwTHLRaRM2IRdKy0aAHHHWe1eNyv5eVZwt+2zSYh1ahRPs9z5ZVWNqCitPazsmw1tjPPDDuSsrv3XlvW8oorKu4oKlcyxSb9YI3b0cCZQDowQETSC9mvDnADMCtiWzq2pm4boBfwZP6auYkiMxOys2H58rAjSTwPP2yTeh55BNJ/9RuPnXr1LOlMmABr1pTf88TCjh1WzKxfv/J7E4ynAw6wEswrV1qVVJf8omnpdwGWquoyVd0JTAD6FLLfPcD9wM8R2/oAE1T1F1X9FlganC9hZGbad+/i2Vd2Ntx2m41yGjKk/J/vhhusr3z06PJ/rrJ46y1rEVf0rp1IJ5xg9feffhr+9a+wo3HlLZqk3xRYGXE/J9i2h4h0Apqr6lslPTY4foiIZItI9oYNG6IKPFYOPxw6dfJRPJF+/NFqyRx8sE2eikchsSOOsCUt//53a00nqqwsOOQQOPnksCOJrZEjrSLnlVfa3AmXvMp8IVdEKgEPAzeX9hyqOkZVM1Q1o3HjxmUNqcQyM+HTT+0jrrMLq8uWwbhxtsxkvAwfDps2wUsvxe85S2LzZitlcNFFsZmnkEjyu3lycuCWW8KOxpWnaJL+KqB5xP1mwbZ8dYBjgQ9EZDnQFZgaXMwt7tiEkD+K57XXwo0jEWRlwfPPwx13WDXSeDrpJOjY0apvJmKt/cmTbZJaMnXtROra1RL+M8/YtRyXnESL+e8SkSrA/4BTsYQ9GxioqoVW5xaRD4BbVDVbRNoAWVg/fhPgfaCVqu53/mVGRoZmZ2eX4kcpm/btbcz1xx/H/akTxrJl0KEDtGsHH3xgwynj7YUXbMTQu+9aYbBEcvrpNgt3yZLEq50fKz//bN2d27ZZmZK6dcOOyEVLROaoakZx+xXb0lfVXGAoMA1YBExU1QUiMlJEehdz7AJgIrAQeBe4rqiEH6bMTPj3v2H16rAjCceuXdaCrVTJunXCSPhgpR4OOSTxhm+uWWNj2QcOTN6EDzYi6bnnrIqqd/Mkp6j69FX1bVU9SlWPUNVRwbY/qurUQvY9WVWzI+6PCo47WlXfiV3osZWZaV0KqdrFc9ddVk/mmWds/kJYqlWD666zFZ8Sab2DiRNt3kKiLpYSS1262Czdf/zDPnG55FJs9068hdW9A1ZzvFEj69pIJTNmWPXFK66wf/SwbdxopZcvvdSGESaC44+3T0Offx52JPHxyy/WzbN1q60n7d08iS9m3TupJDMTPvoI1q0LO5L42bjRljs86iirdpkIGjWymF54weIL29KltqZvsl7ALUz16tbNs2aNjeF3ycOTfoT8Lp7Jk8OOJD5UrXW/caPNhq1VK+yI9rrhBruomAgt/fHjrR+/f/+wI4mv446zbp5nn7Whqi45ePdOBFVo3drKLqfCzMTRo61y5qOPWpJNNGecYSNIli+3vv4w5P9NHHJI6nX7gXXzdO5syysuWGAlM1xi8u6dUhCx1v4HH0CcJwbH3fz5cPPNcNZZMGxY2NEUbvhw6154+eXwYpg3z1YMS6WunUjVq9u8jXXr7PfhKj5P+gVkZtoqTlOmhB1J+dm+3Uah1K9vNdUTdQjiGWdYK/uRR8KbrJWVBVWr7p3Al4o6d4YRI6yP/62ChVZcheNJv4D27a0OTDLX4rn5ZhsO+cILcNBBYUezfyK2ju7cueFMmsvLs/4QFHbKAAAZkklEQVT8Xr2gYcP4P38iufNOW1Pht7+1rh4XO6pWBmbIELj22vJ/Pk/6BYjYounvv291YJLNa69ZUbPf/c5mmCa6Sy6xhBvGZK2PP7ZJSqnatRMpfzTP+vX2RuzKbv16+Otfbaj4CSfYpMh4fOr2pF+I/C6eqb+aelaxrVwJV10FGRm2eEZFcMABcPXV8Prr8V/WMivLRjTFeonIiqpTJyu3/cIL8MYbYUdTMeXm2nrQ558PTZvarOe6dW1+zNq18OST5R+Dj94phKqVXE5PT54+zN274ZRTbHLR3Llw5JFhRxS91ashLc0++sZrLsHOnXDooda1M25cfJ6zIti504Zyrl9vo3niWYW1Ivvf/+z62fPP2+CEgw6yGlOXX27XrWLBR++UQf4onunTrZxuMvjzn23i2ZNPVqyED9CkiZUzfvZZ2LIlPs/53nvw/ffetVNQtWrWzbNxY2IO800k27bZa3XiibZWwYMP2qfsKVOshPUDD8Qu4ZeEJ/39yMy0affJ8DH23/+GP/0JBg2yPvKKaPhw+yf65z/j83xZWXYtoWfP+DxfRdKxI9x+u6178PrrYUeTWFThv/+1btRDD7WW/Pr1cP/91r06dSr06WMjwsLi3Tv7oWqFxzp0qNh9+5s324ikKlWsW+fAA8OOqPS6d4cVK6wsQnlWAd22zVYNu/RSeOqp8nueimznTivMtnatdfOk+uimtWvhxRft0+jXX9u1oIsushnvv/lNfC7QevdOGYnY2Oxp0yru8nGqNgxs9WpruVbkhA82amTFivKfQzF1qs1l8K6d/atWzfqnN21K3Ml95W3XLvtbOe88m8X/+9/bNY5//tP67f/5T+jWLfHmwXjSL8IFF1iL5s03w46kdJ59Fl55Be65x6pEVnS9e9sF9vIevpmVZVU+u3Ur3+ep6Nq3t/H7WVnJPZmxoK+/hltvhcMOs66aTz+1uS+LFllX6hVXQJ06YUdZBFVNqK/OnTtroti9W7VJE9W+fcOOpOQWLlStWVP11FPt50gWjz6qCqqzZpXP+TdsUK1SRfX3vy+f8yebnTtVO3ZUPegge+2S1datqv/8p+pvfmN/f5Urq/bpo/r66/YaJAIgW6PIsVG19EWkl4gsFpGlIjKikMevEZEvRWSeiHwiIunB9jQR2RFsnycif4/xe1a5qlTJunjeecf6eSuKNWuspk6tWjamulISfZ674grrpiqv1v6kSTaW2rt2olO1qo1Q+eEHuP76sKOJLdW9LfdDD4Urr7TurAcesNE3U6bYp88wL8qWSnHvCkBl4BvgcKAa8AWQXmCfAyNu9wbeDW6nAV9F8+6T/5VILX1V1Q8/tHf2l18OO5LobN6s2r69aq1aqp99FnY05eOmm6yltXJl7M994omq6emqeXmxP3cyu+ce+z959dWwIym71atV77tP9aij7GeqXVv1yitV//3vxP67IIYt/S7AUlVdpqo7gQlAnwJvHJGXOmsBiTUkqAy6dbORHBWhFs8vv9hMvwUL4NVXbRJNMrr+emuF/e1vsT3vd99Z6YVkXwe3PNx6q83YveaailmhdteuvS335s2twNxBB9l1sTVrbMZsvEbhlLdokn5TYGXE/Zxg2z5E5DoR+QZ4AIi8nt9SROaKyIcicmJhTyAiQ0QkW0SyNyTYX0zlypZI33rLRnQkqrw8G2I4Y4b9oZ5xRtgRlZ+0NOjbF8aMgZ9+it15J0yw76mwDm6s5XfzbN5sazRUFJs3w8iRNvqmb1+YPdtKI3z9tTUALr8catcOO8oYK+6jAJAJ/CPi/iXA34rYfyDwfHC7OtAwuN0Ze/M4sKjnS7TuHVXV99+3j3mTJoUdSeHy8lSHDbMYH3gg7Gji45NP7OcdPTp252zfXrVr19idLxXde6/9Xl55JexIivbDD6p/+pNqvXoW79lnq06dqrprV9iRlR5Rdu9Ek/RPAKZF3P8D8Ici9q8EbNnPYx8AGUU9XyIm/V27VBs3Vu3fP+xICnf//fabvPHGxO5zjKW8PNXjjlNt1So2o5MWLLDX8PHHy36uVLZrl2rnzvb/sn592NH8WsFkf955qp9/HnZUsRFt0o+me2c20EpEWopINaA/sM8cVRFpFXH3bGBJsL2xiFQObh8OtAKWRfURJIFUqWIf/d58E3bsCDuafb3wgvWn9u9vZVqToc8xGiJWmmHJktis3zp+vI1yuvDCsp8rlVWpYpO2tmyB664LO5q98rtxWraEu+6Ck0+24oOTJ1tZiZQSzTsDcBbwP2wUz+3BtpFA7+D2Y8ACYB4wE2gTbO8Xsf1z4NzinisRW/qqqu+9Zy2DKVPCjmSvt9+2USynnqr6889hRxN/O3eqNm1qP39Z5OWpHn646umnxyYup/rnPyfGqLdkbtkXRKy6d+L9lahJf+dO1QYNVC++OOxIzKxZNvmqQwfVLVvCjiY8f/mL/RV/8UXpz/Hpp3aOsWNjFlbK27XLut8aNVJdty7+z59KyT5ftEk/iabtlK+qVa3GxtSpNjQyTEuWwNln21DSd96p+DV1ymLIEKhZEx59tPTnyMqylaH69o1dXKmuShUbzbN1K/y//xe/NY69G6d4nvRLIDPT/oj/9a/wYli7du9wzHffhUMOCS+WRNCggS1GMW4crFtX8uNzc+Hll+Gcc2wFIxc76emWgF991V7j8uTJPnqe9Evg1FOhXj0rYhaGrVvhzDMtub31Fhx1VDhxJJobbrDCeKUpgzxzpr2eXnahfNx8s5VgHjq0dG/Kxdm82daKSEvzZB8tT/olUK2aVdV7/XVLMvGUP9v2q6+s5dSlS3yfP5EdfbR1dz35JPz8c8mOzS85fdZZ5RNbqsvv5tm2zZa7jFU3T2Syv/tu6NHDk320POmXUGam/cHNmBG/58zLg8GD4f33rUZ3r17xe+6K4sYbbfr/+PHRH7Njh72B9usHNWqUX2yprnVrK+89efLeWc+l5ck+BqK52hvPr0QdvZPv559V69SxAkzxkJdnk67AikC5wuXlqbZta1/RTlCbNMle1+nTyzc2p5qba7OdGzRQXbOm5Mf/8IPq3Xer1q2bOqNxSgofvVM+qle3okyTJ1uRpvL20EM2MmXYMFuZxxVOxFr7X34Z/aewrCwbAdWjR/nG5qyG1XPPWf2qa66JvpunYMv+lFNs2U9v2ZeeJ/1SyMyE77+HDz8s3+d58UVL9BdeaPXjU2W2bWkNHGiVEaOptb95s10M79/fEpIrf0cfDffea9fEsrKK3nd/yf6112zdald6nvRL4YwzbIGS8iy3PG2aLd7Qo0fyLYRSXmrUsIuFb70FixcXve/kyXZx3EftxNeNN1qJ4uuvt5LFBXmyL3+eSkrhgANsXPdrr9k471jLzraLi23aWHKqXj32z5Gsrr3WRlk99ljR+2VlwRFHJO+aA4mqcmUYO9Yuol999d5uHk/28eNJv5QuuMBGi3z8cWzPu2SJDR9s3Nhm2/qEoZI5+GBrvT//vHXBFWbtWuv398VSwnHUUfDnP8Mbb9gwW0/28eVJv5TOPNOm/8eyiyd/tq2qde8cemjszp1KbrzRLhg+80zhj0+caMNgfbGU8AwbZqvSDR3qyT7eRONVFCNKGRkZmp2dHXYYUbngAvjkE1skuawXA3/8Ebp3t77oGTPg+ONjE2OqOvVUey2//fbXC1d37Wr9+XPnhhObM8uXWzfcZZd5oo8FEZmjqhnF7ect/TLIzLTW+X/+U7bz7Nxps23nz7cSD57wy274cFi16tefxL75BmbN8gu4iSAtzUZaecKPL0/6ZXDWWTZipCxdPPmzbf/1L1t82csBxMZZZ0GrVpZUIj/M5s/Y7d8/nLicC1tUSV9EeonIYhFZKiIjCnn8GhH5UkTmicgnIpIe8dgfguMWi0hSLdddp46VRHj1VUvepfG731ki+vOfLfm72KhUyQqxzZ6995OYqlXjPOkkaN483PicC0uxST9Y7nA0cCaQDgyITOqBLFVtq6odgAeAh4Nj07HlFdsAvYAn85dPTBaZmdaN8OmnJT/2r3+Fhx+2McsjfvVW6srqssusKmp+rf0vvoCvv/auHZfaomnpdwGWquoyVd0JTAD6RO6gqlsj7tYC8j9Q9wEmqOovqvotsDQ4X9I491wbF17SLp5x4+CWW+xisM+2LR+1a9siK6+9ZhcNs7Ks6mNmZtiROReeaJJ+U2BlxP2cYNs+ROQ6EfkGa+kPK8mxFdmBB9owy0mToq8n8t571pVz8sk229bLAJSfoUPtDfWxx6wbrVcvaNgw7KicC0/MLuSq6mhVPQK4FbijJMeKyBARyRaR7A0bNsQqpLjJzISVK63/uDjZ2TZSJz0dpkzxkr7lrXlz+zT1xBM2tNa7dlyqiybprwIiL3s1C7btzwTgvJIcq6pjVDVDVTMaN24cRUiJ5dxzbSx4cV08S5faqJJGjXy2bTwNHw67d9tkut69w47GuXBFk/RnA61EpKWIVMMuzE6N3EFEWkXcPRtYEtyeCvQXkeoi0hJoBXxW9rATS/36cNppRXfxrFtnXQt5eTbbtkmT+MaYyrp0sWT/299aoTznUlmxSV9Vc4GhwDRgETBRVReIyEgRyW83DRWRBSIyD7gJuCw4dgEwEVgIvAtcp6q7y+HnCF1mps3+bNLEhgumpdnFWrDZtmedBatXw5tvWolZF1+vv753FI9zqaxKNDup6tvA2wW2/THi9g1FHDsKGFXaACuK/DVz16617ytW2MiR3FxL/l98YYmna9fwYnTOuaiSvivefff9etv27XDddfDTT/Dss7Z4t3POhcnLMMTId98Vvv2nn2DUKLj88vjG45xzhfGkHyOHHVb49jp14A9/iG8szjm3P570Y2TUKBsSGKlyZRg92mfbOucShyf9GBk0CMaMgQYN7H716raIxyWXhBuXc85F8qQfQ4MG2azPxx6zUTzej++cSzQ+eifGDjjAloJzzrlE5C1955xLIZ70nXMuhXjSd865FOJJ3znnUognfeecSyGe9J1zLoV40nfOuRTiSd8551KIJ33nnEshUSV9EeklIotFZKmIjCjk8ZtEZKGIzBeR90WkRcRju0VkXvA1teCxzjnn4qfYMgwiUhkYDZwO5ACzRWSqqi6M2G0ukKGq20XkWuAB4KLgsR2q2iHGcTvnnCuFaFr6XYClqrpMVXcCE4A+kTuo6kxV3R7c/RRoFtswnXPOxUI0Sb8psDLifk6wbX+uBN6JuF9DRLJF5FMROa+wA0RkSLBP9oYNG6IIyTnnXGnEtMqmiFwMZADdIza3UNVVInI4MENEvlTVbyKPU9UxwBiAjIwMjWVMzjnn9oqmpb8KaB5xv1mwbR8ichpwO9BbVX/J366qq4Lvy4APgI5liNc551wZRJP0ZwOtRKSliFQD+gP7jMIRkY7A01jCXx+xvb6IVA9uNwK6AZEXgJ1zzsVRsd07qporIkOBaUBl4FlVXSAiI4FsVZ0KPAjUBl4RWxD2O1XtDbQGnhaRPOwN5r4Co36cc87FkagmVhd6RkaGZmdnhx2Gc85VKCIyR1UzitvPZ+Q651wK8aSfhMaNg7Q0qFTJvo8bF3ZEzrlE4QujJ5lx42DIENgeTJVbscLuAwwaFF5czrnE4C39JHP77XsTfr7t2227c8550k8y331Xsu3OudTiST/JHHZYybY751KLJ/0kM2oU1Ky577aaNW27c8550k8ygwbBmDHQogWI2PcxY/wirnPO+OidJDRokCd551zhvKXvnHMpxJO+c86lEE/6zjmXQjzpO+dcCvGk75xzKcSTvnPOpRBP+s45l0KiSvoi0ktEFovIUhEZUcjjN4nIQhGZLyLvi0iLiMcuE5ElwddlsQzeOedcyRSb9EWkMjAaOBNIBwaISHqB3eYCGaraDpgEPBAc2wC4Czge6ALcJSL1Yxe+c865koimpd8FWKqqy1R1JzAB6BO5g6rOVNX8gr6fAs2C22cA01X1e1X9AZgO9IpN6C7R+WIuziWeaJJ+U2BlxP2cYNv+XAm8U5JjRWSIiGSLSPaGDRuiCMkluvzFXFasANW9i7l44ncuXDG9kCsiFwMZwIMlOU5Vx6hqhqpmNG7cOJYhuZD4Yi7OJaZokv4qoHnE/WbBtn2IyGnA7UBvVf2lJMe65OOLuTiXmKJJ+rOBViLSUkSqAf2BqZE7iEhH4Gks4a+PeGga0FNE6gcXcHsG21yS88VcnEtMxSZ9Vc0FhmLJehEwUVUXiMhIEekd7PYgUBt4RUTmicjU4NjvgXuwN47ZwMhgm0tyvpiLc4lJVDXsGPaRkZGh2dnZYYfhYmDcOOvD/+47a+GPGuV1/p0rLyIyR1UzitvPF1Fx5cYXc3Eu8XgZBuecSyGe9J1zLoV40nfOuRTiSd8lPS8H4dxefiHXJbX8chD5s4Pzy0GAX2R2qclb+i6peTkI5/blSd8lNS8H4dy+POm7pOblIJzblyd9l9S8HIRz+/Kk75LaoEEwZgy0aAEi9n3MGL+I61KXj95xSc/LQTi3l7f0nXMuhXjSd865FOJJ3znnUognfefixMtBuEQQVdIXkV4islhElorIiEIeP0lEPheRXBHJLPDY7mA1rT0rajmXavLLQaxYAap7y0F44nfxVmzSF5HKwGjgTCAdGCAi6QV2+w4YDGQVcoodqtoh+OpdyOPOJT0vB+ESRTRDNrsAS1V1GYCITAD6AAvzd1DV5cFjeeUQo3MVnpeDcIkimu6dpsDKiPs5wbZo1RCRbBH5VETOK2wHERkS7JO9YcOGEpzauYrBy0G4RBGPC7ktgsV6BwKPisgRBXdQ1TGqmqGqGY0bN45DSM7Fl5eDcIkimqS/Cmgecb9ZsC0qqroq+L4M+ADoWIL4nEsKiVQOwkcRpbZo+vRnA61EpCWW7PtjrfZiiUh9YLuq/iIijYBuwAOlDda5iiwRykH4ojKu2Ja+quYCQ4FpwCJgoqouEJGRItIbQESOE5Ec4ALgaRFZEBzeGsgWkS+AmcB9qrrw18/inIsHH0XkRFXDjmEfGRkZmp2dHXYYziWlSpVsnkBBIpDnY+8qNBGZE1w/LZLPyHUuhfgoIudJ37kU4qOInCd951JIoowi8hFE4fFFVJxLMWGPIvIRROHylr5zLq58BFG4POk75+LK6xCFy5O+cy6ufARRuDzpO+fiKpFGEKXiBWVP+s65uEqkEUSpuLCNz8h1zqWktDRL9AW1aAHLl8c7mrLzGbnOOVeEVL2g7EnfOZeSUvWCsid951xKStULyp70nXMpKVUvKPuFXOecC1GsLij7hVznnKsA4n1BOaqkLyK9RGSxiCwVkRGFPH6SiHwuIrkiklngsctEZEnwdVmsAnfOuWQQ7wvKxSZ9EakMjAbOBNKBASKSXmC374DBQFaBYxsAdwHHA12Au4J1c51zzhH/C8rRtPS7AEtVdZmq7gQmAH0id1DV5ao6Hyi44NoZwHRV/V5VfwCmA71iELdzziWFeF9QjqaeflNgZcT9HKzlHo3Cjm1acCcRGQIMATgs2QfJOudcAfFc4yAhLuSq6hhVzVDVjMaNG4cdjnPOJa1okv4qoHnE/WbBtmiU5VjnnHMxFk3Snw20EpGWIlIN6A9MjfL804CeIlI/uIDbM9jmnHMuBMUmfVXNBYZiyXoRMFFVF4jISBHpDSAix4lIDnAB8LSILAiO/R64B3vjmA2MDLY555wLgc/Idc65JBDtjNyES/oisgEoZFJyhdII2Bh2EAnEX499+euxl78W+yrL69FCVYsdCZNwST8ZiEh2NO+4qcJfj33567GXvxb7isfrkRBDNp1zzsWHJ33nnEshnvTLx5iwA0gw/nrsy1+Pvfy12Fe5vx7ep++ccynEW/rOOZdCPOk751wK8aQfQyLSXERmishCEVkgIjeEHVPYRKSyiMwVkTfDjiVsIlJPRCaJyNciskhETgg7pjCJyPDg/+QrERkvIjXCjimeRORZEVkvIl9FbGsgItODRaeml8f6I570YysXuFlV04GuwHWFLDiTam7Aync4eAx4V1WPAdqTwq+LiDQFhgEZqnosUBmr65VKnuPX64uMAN5X1VbA+8H9mPKkH0OqukZVPw9u/4j9U/9q/YBUISLNgLOBf4QdS9hEpC5wEvBPAFXdqaqbw40qdFWAA0SkClATWB1yPHGlqh8BBWuR9QGeD24/D5wX6+f1pF9ORCQN6AjMCjeSUD0K/J5fr6iWiloCG4CxQXfXP0SkVthBhUVVVwEPYUutrgG2qOp74UaVEA5W1TXB7bXAwbF+Ak/65UBEagOvAjeq6taw4wmDiJwDrFfVOWHHkiCqAJ2Ap1S1I/AT5fDRvaII+qr7YG+GTYBaInJxuFElFrXx9DEfU+9JP8ZEpCqW8Mep6mthxxOibkBvEVmOrat8ioi8FG5IocoBclQ1/5PfJOxNIFWdBnyrqhtUdRfwGvCbkGNKBOtE5FCA4Pv6WD+BJ/0YEhHB+mwXqerDYccTJlX9g6o2U9U07ALdDFVN2Zacqq4FVorI0cGmU4GFIYYUtu+AriJSM/i/OZUUvrAdYSpwWXD7MuD1WD+BJ/3Y6gZcgrVq5wVfZ4UdlEsY1wPjRGQ+0AH4c8jxhCb4xDMJ+Bz4EstFKVWSQUTGA/8FjhaRHBG5ErgPOF1ElmCfhu6L+fN6GQbnnEsd3tJ3zrkU4knfOedSiCd955xLIZ70nXMuhXjSd865FOJJ3znnUognfeecSyH/H7ECqZcxpdCuAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.2.4 케라스를 사용한 GRU 예제"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "_________________________________________________________________\n",
      "Layer (type)                 Output Shape              Param #   \n",
      "=================================================================\n",
      "embedding_1 (Embedding)      (None, None, 32)          320000    \n",
      "_________________________________________________________________\n",
      "gru_1 (GRU)                  (None, 32)                6240      \n",
      "_________________________________________________________________\n",
      "dense_1 (Dense)              (None, 1)                 33        \n",
      "=================================================================\n",
      "Total params: 326,273\n",
      "Trainable params: 326,273\n",
      "Non-trainable params: 0\n",
      "_________________________________________________________________\n",
      "Train on 20000 samples, validate on 5000 samples\n",
      "Epoch 1/10\n",
      "20000/20000 [==============================] - 43s 2ms/step - loss: 0.4962 - acc: 0.7468 - val_loss: 0.3444 - val_acc: 0.8566\n",
      "Epoch 2/10\n",
      "20000/20000 [==============================] - 41s 2ms/step - loss: 0.2887 - acc: 0.8848 - val_loss: 0.3360 - val_acc: 0.8596\n",
      "Epoch 3/10\n",
      "20000/20000 [==============================] - 41s 2ms/step - loss: 0.2356 - acc: 0.9113 - val_loss: 0.3380 - val_acc: 0.8554\n",
      "Epoch 4/10\n",
      "20000/20000 [==============================] - 41s 2ms/step - loss: 0.2105 - acc: 0.9210 - val_loss: 0.5398 - val_acc: 0.7910\n",
      "Epoch 5/10\n",
      "20000/20000 [==============================] - 41s 2ms/step - loss: 0.1846 - acc: 0.9323 - val_loss: 0.3841 - val_acc: 0.8522\n",
      "Epoch 6/10\n",
      "20000/20000 [==============================] - 41s 2ms/step - loss: 0.1737 - acc: 0.9384 - val_loss: 0.3977 - val_acc: 0.8442\n",
      "Epoch 7/10\n",
      "20000/20000 [==============================] - 41s 2ms/step - loss: 0.1594 - acc: 0.9436 - val_loss: 0.3611 - val_acc: 0.8542\n",
      "Epoch 8/10\n",
      "20000/20000 [==============================] - 41s 2ms/step - loss: 0.1499 - acc: 0.9457 - val_loss: 0.4977 - val_acc: 0.8468\n",
      "Epoch 9/10\n",
      "20000/20000 [==============================] - 42s 2ms/step - loss: 0.1392 - acc: 0.9492 - val_loss: 0.4272 - val_acc: 0.8540\n",
      "Epoch 10/10\n",
      "20000/20000 [==============================] - 42s 2ms/step - loss: 0.1310 - acc: 0.9535 - val_loss: 0.5781 - val_acc: 0.8362\n"
     ]
    }
   ],
   "source": [
    "from keras.models import Sequential\n",
    "from keras.layers import Dense, Embedding, GRU\n",
    "from keras import backend as K\n",
    "\n",
    "model = Sequential()\n",
    "model.add(Embedding(max_features, 32))\n",
    "model.add(GRU(32))\n",
    "model.add(Dense(1, activation='sigmoid'))\n",
    "model.summary()\n",
    "\n",
    "model.compile(optimizer='rmsprop', \n",
    "              loss='binary_crossentropy', \n",
    "              metrics=['acc'])\n",
    "history = model.fit(train_inputs, train_labels,\n",
    "                    epochs=10,\n",
    "                    batch_size=128,\n",
    "                    validation_split=0.2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAX0AAAEICAYAAACzliQjAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xl8VOXZ//HPxaLsO7iAEFwqRNYQQQqoiAtaxY0qiK07rU/dtU9xqeAC7oi21kce606l/LS22irWBR+0ihLUoIAIKmAAMSDgEgTB6/fHfZJMQpZJmGQmme/79ZrXzNmvOZNc5z73uc99zN0REZH00CDZAYiISO1R0hcRSSNK+iIiaURJX0QkjSjpi4ikESV9EZE0oqSfhsysoZl9a2ZdEzlvMpnZ/maW8PbHZnakma2IGV5qZsPimbca23rQzK6p7vIi8WiU7ACkcmb2bcxgM2ArsCMa/pW7z6jK+tx9B9Ai0fOmA3c/MBHrMbPzgTPd/fCYdZ+fiHWLVERJvw5w96KkG5Ukz3f3l8ub38waufv22ohNpDL6e0wtqt6pB8zsZjP7q5k9aWbfAGea2WAzm2dmm8xsrZnda2aNo/kbmZmbWUY0/EQ0/QUz+8bM3jKz7lWdN5p+rJl9bGabzewPZvYfMzu7nLjjifFXZrbczDaa2b0xyzY0s7vNbIOZfQqMrGD/XGtmM0uNu8/MpkafzzezJdH3+SQqhZe3rjwzOzz63MzMHo9iWwQMKDXvdWb2abTeRWY2KhrfG/gjMCyqOlsfs28nxSz/6+i7bzCzv5vZXvHsm6rs58J4zOxlM/vKzL4ws/+O2c7vo33ytZnlmNneZVWlmdkbhb9ztD/nRtv5CrjOzA4wsznRNtZH+611zPLdou+YH02/x8yaRDH3jJlvLzMrMLP25X1fqYS761WHXsAK4MhS424GtgEnEA7kTYGDgUGEs7l9gY+Bi6L5GwEOZETDTwDrgWygMfBX4IlqzNsJ+AY4MZp2BfADcHY53yWeGP8BtAYygK8KvztwEbAI6AK0B+aGP+cyt7Mv8C3QPGbdXwLZ0fAJ0TwGHAFsAfpE044EVsSsKw84PPp8J/Aa0BboBiwuNe9pwF7Rb3JGFMMe0bTzgddKxfkEMCn6fHQUYz+gCfAn4NV49k0V93NrYB1wKbA70AoYGE27GsgFDoi+Qz+gHbB/6X0NvFH4O0ffbTtwIdCQ8Pf4E2AEsFv0d/If4M6Y7/NhtD+bR/MPiaZNBybHbOdK4Jlk/x/W5VfSA9Crij9Y+Un/1UqWuwr4f9HnshL5/8TMOwr4sBrzngu8HjPNgLWUk/TjjPGQmOl/A66KPs8lVHMVTjuudCIqte55wBnR52OBpRXM+0/gN9HnipL+qtjfAviv2HnLWO+HwM+iz5Ul/UeBKTHTWhGu43SpbN9UcT//AphfznyfFMZbanw8Sf/TSmIYXbhdYBjwBdCwjPmGAJ8BFg2/D5yS6P+rdHqpeqf++Dx2wMx6mNm/otP1r4EbgQ4VLP9FzOcCKr54W968e8fG4eG/NK+8lcQZY1zbAlZWEC/AX4Cx0eczouHCOI43s7ejqodNhFJ2Rfuq0F4VxWBmZ5tZblRFsQnoEed6IXy/ovW5+9fARqBzzDxx/WaV7Od9CMm9LBVNq0zpv8c9zWyWma2OYnikVAwrPDQaKMHd/0M4axhqZr2ArsC/qhmToDr9+qR0c8UHCCXL/d29FXA9oeRdk9YSSqIAmJlRMkmVtisxriUki0KVNSmdBRxpZp0J1U9/iWJsCjwF3EKoemkD/DvOOL4oLwYz2xe4n1DF0T5a70cx662seekaQpVR4fpaEqqRVscRV2kV7efPgf3KWa68ad9FMTWLGbdnqXlKf7/bCK3OekcxnF0qhm5m1rCcOB4DziSclcxy963lzCdxUNKvv1oCm4Hvogthv6qFbf4TyDKzE8ysEaGeuGMNxTgLuMzMOkcX9X5X0czu/gWhCuIRQtXOsmjS7oR65nxgh5kdT6h7jjeGa8ysjYX7GC6KmdaCkPjyCce/Cwgl/ULrgC6xF1RLeRI4z8z6mNnuhIPS6+5e7plTBSraz88CXc3sIjPb3cxamdnAaNqDwM1mtp8F/cysHeFg9wWhwUBDMxtPzAGqghi+Azab2T6EKqZCbwEbgCkWLo43NbMhMdMfJ1QHnUE4AMguUNKvv64EziJcWH2AcMG1Rrn7OuB0YCrhn3g/4D1CCS/RMd4PvAJ8AMwnlNYr8xdCHX1R1Y67bwIuB54hXAwdTTh4xWMi4YxjBfACMQnJ3RcCfwDeieY5EHg7ZtmXgGXAOjOLraYpXH42oRrmmWj5rsC4OOMqrdz97O6bgaOAUwkHoo+Bw6LJdwB/J+znrwkXVZtE1XYXANcQLurvX+q7lWUiMJBw8HkWeDomhu3A8UBPQql/FeF3KJy+gvA7b3X3N6v43aWUwosjIgkXna6vAUa7++vJjkfqLjN7jHBxeFKyY6nrdHOWJJSZjSS0lNlCaPL3A6G0K1It0fWRE4HeyY6lPlD1jiTaUOBTQl32McDJuvAm1WVmtxDuFZji7quSHU99oOodEZE0opK+iEgaSbk6/Q4dOnhGRkaywxARqVMWLFiw3t0raiINpGDSz8jIICcnJ9lhiIjUKWZW2V3pgKp3RETSipK+iEgaUdIXEUkjKVenX5YffviBvLw8vv/++2SHIhVo0qQJXbp0oXHj8rqTEZFkqxNJPy8vj5YtW5KRkUHouFFSjbuzYcMG8vLy6N69e+ULiEhS1Inqne+//5727dsr4acwM6N9+/Y6GxOphhkzICMDGjQI7zNm1Ny26kRJH1DCrwP0G4lU3YwZMH48FBSE4ZUrwzDAuOr2q1qBOlHSFxGpr669tjjhFyooCONrQlxJ38xGmtlSM1tuZhPKmN7NzF4xs4Vm9pqZxT49aYeZvR+9nk1k8LVlw4YN9OvXj379+rHnnnvSuXPnouFt27bFtY5zzjmHpUuXVjjPfffdx4yaPK8TkZSzqpxu5Mobv8sqe4gu4Wn2nwD7Ep4wlAtklprn/wFnRZ+PAB6PmfZtVR7aO2DAAC9t8eLFO42ryBNPuHfr5m4W3p94okqLV2jixIl+xx137DT+xx9/9B07diRuQ3VUVX8rkWSqyVwRr27d3GHnV7duVVsPkOMJejD6QGC5u3/q7tuAmYS+rWNlAq9Gn+eUMb3WFNaPrVwZdl1h/VhNFKCXL19OZmYm48aN46CDDmLt2rWMHz+e7OxsDjroIG688caieYcOHcr777/P9u3badOmDRMmTKBv374MHjyYL7/8EoDrrruOadOmFc0/YcIEBg4cyIEHHsibb4YHBn333XeceuqpZGZmMnr0aLKzs3n//fd3im3ixIkcfPDB9OrVi1//+teFB2A+/vhjjjjiCPr27UtWVhYrVqwAYMqUKfTu3Zu+fftybU2dV4qkkNrMFRWZPBmaNSs5rlmzML4mxJP0O1PyyfZ57Pyw61zglOjzyUDL6LmlAE3MLMfM5pnZSbsUbRxqu37so48+4vLLL2fx4sV07tyZW2+9lZycHHJzc3nppZdYvHjxTsts3ryZww47jNzcXAYPHsxDDz1U5rrdnXfeeYc77rij6ADyhz/8gT333JPFixfz+9//nvfee6/MZS+99FLmz5/PBx98wObNm5k9ezYAY8eO5fLLLyc3N5c333yTTp068dxzz/HCCy/wzjvvkJuby5VXXpmgvSOSumo7V5Rn3DiYPh26dQOz8D59es1cxIXEXci9CjjMzN4jPF9zNbAjmtbN3bMJDzWeZmb7lV7YzMZHB4ac/Pz8XQqktuvH9ttvP7Kzs4uGn3zySbKyssjKymLJkiVlJv2mTZty7LHHAjBgwICi0nZpp5xyyk7zvPHGG4wZMwaAvn37ctBBB5W57CuvvMLAgQPp27cv//d//8eiRYvYuHEj69ev54QTTgDCzVTNmjXj5Zdf5txzz6Vp06YAtGvXruo7QqSOqfW69AqMGwcrVsCPP4b3mkr4EF/SXw3sEzPcJRpXxN3XuPsp7t4fuDYatyl6Xx29fwq8BvQvvQF3n+7u2e6e3bFjpT2DVqhr16qN31XNmzcv+rxs2TLuueceXn31VRYuXMjIkSPLbLe+2267FX1u2LAh27dvL3Pdu+++e6XzlKWgoICLLrqIZ555hoULF3Luueeq/byklNpsl16e2s4VqSKepD8fOMDMupvZbsAYwtPsi5hZBzMrXNfVwEPR+LZmtnvhPMAQYOeibwLVdv1YrK+//pqWLVvSqlUr1q5dy4svvpjwbQwZMoRZs2YB8MEHH5R5JrFlyxYaNGhAhw4d+Oabb3j66acBaNu2LR07duS5554Dwk1vBQUFHHXUUTz00ENs2bIFgK+++irhcYsUSte69FRRadJ39+3ARcCLwBJglrsvMrMbzWxUNNvhwFIz+xjYAyjcbT2BHDPLJVzgvdXdazTp13b9WKysrCwyMzPp0aMHv/zlLxkyZEjCt3HxxRezevVqMjMzueGGG8jMzKR169Yl5mnfvj1nnXUWmZmZHHvssQwaNKho2owZM7jrrrvo06cPQ4cOJT8/n+OPP56RI0eSnZ1Nv379uPvuuxMet6SOZJey07UuPVWk3DNys7OzvfRDVJYsWULPnj2TFFFq2b59O9u3b6dJkyYsW7aMo48+mmXLltGoUWrcXK3fKrWVvvsTQum2NpNdgwahhF+aWajTluoxswXR9dMKpUamkLh9++23jBgxgu3bt+PuPPDAAymT8CX1VVTKrq2k37VrqNIpa7zUPGWLOqZNmzYsWLAg2WFIHZUKLVYmTy77bKO+16WnCvW9I5JGUqHFSrrWpacKJX2RWpLsC6iQOi1WarNdupSkpC9SC1KlmaJK2aKkL1ILUqWZIqiUne6U9OMwfPjwnW60mjZtGhdeeGGFy7Vo0QKANWvWMHr06DLnOfzwwyndRLW0adOmURCTMY477jg2bdoUT+iSIlLhAqoIKOnHZezYscycObPEuJkzZzJ27Ni4lt9777156qmnqr390kn/+eefp02bNtVen9S+VLiAKgJK+nEZPXo0//rXv4oemLJixQrWrFnDsGHDitrNZ2Vl0bt3b/7xj3/stPyKFSvo1asXELpIGDNmDD179uTkk08u6voA4MILLyzqlnnixIkA3HvvvaxZs4bhw4czfPhwADIyMli/fj0AU6dOpVevXvTq1auoW+YVK1bQs2dPLrjgAg466CCOPvroEtsp9NxzzzFo0CD69+/PkUceybp164BwL8A555xD79696dOnT1E3DrNnzyYrK4u+ffsyYsSIhOzbdJEqF1BF6lw7/csugzK6j98l/fpBlC/L1K5dOwYOHMgLL7zAiSeeyMyZMznttNMwM5o0acIzzzxDq1atWL9+PYcccgijRo0q93mx999/P82aNWPJkiUsXLiQrKysommTJ0+mXbt27NixgxEjRrBw4UIuueQSpk6dypw5c+jQoUOJdS1YsICHH36Yt99+G3dn0KBBHHbYYbRt25Zly5bx5JNP8r//+7+cdtppPP3005x55pkllh86dCjz5s3DzHjwwQe5/fbbueuuu7jpppto3bo1H3zwAQAbN24kPz+fCy64gLlz59K9e/c61T/PjBmh7nzVqlCynjy59uuxC7eX7DhEVNKPU2wVT2zVjrtzzTXX0KdPH4488khWr15dVGIuy9y5c4uSb58+fejTp0/RtFmzZpGVlUX//v1ZtGhRmZ2pxXrjjTc4+eSTad68OS1atOCUU07h9ddfB6B79+7069cPKL/75ry8PI455hh69+7NHXfcwaJFiwB4+eWX+c1vflM0X9u2bZk3bx6HHnoo3bt3B+pO98up0moGdAFVUkOdK+lXVCKvSSeeeCKXX3457777LgUFBQwYMAAIHZjl5+ezYMECGjduTEZGRrW6Mf7ss8+48847mT9/Pm3btuXss8/epe6QC7tlhtA1c1nVOxdffDFXXHEFo0aN4rXXXmPSpEnV3l6qSoVuB0RSiUr6cWrRogXDhw/n3HPPLXEBd/PmzXTq1InGjRszZ84cVpbVqUiMQw89lL/85S8AfPjhhyxcuBAI3TI3b96c1q1bs27dOl544YWiZVq2bMk333yz07qGDRvG3//+dwoKCvjuu+945plnGDZsWNzfafPmzXTuHB6C9uijjxaNP+qoo7jvvvuKhjdu3MghhxzC3Llz+eyzz4C60/2yWs2IlKSkXwVjx44lNze3RNIfN24cOTk59O7dm8cee4wePXpUuI4LL7yQb7/9lp49e3L99dcXnTH07duX/v3706NHD84444wS3TKPHz+ekSNHFl3ILZSVlcXZZ5/NwIEDGTRoEOeffz79++/0jJpyTZo0iZ///OcMGDCgxPWC6667jo0bN9KrVy/69u3LnDlz6NixI9OnT+eUU06hb9++nH766XFvJ5nUakakJHWtLAmVar9VKnQlLFIb4u1aWSV9qdfU7YBISXXuQq5IVY0bpyQvUqjOlPRTrRpKdqbfSCT11Ymk36RJEzZs2KCkksLcnQ0bNtCkSZNkhyIiFagT1TtdunQhLy+P/Pz8ZIciFWjSpAldunQpGk6FO2FFpKQ6kfQbN25cdCeo1A2lW80U3gkLSvwiyVQnqnek7kml/uNFpJiSvtQI3QkrkpqU9KVG6E5YkdSkpC81Qv3Hi6QmJX2pEboTViQ11YnWO1I36U5YkdSjkr6ISBpR0hcRSSNK+vXQjBmQkQENGoT3ZDwaUERSk+r06xndCSsiFVFJv57RnbAiUhEl/XpGd8KKSEWU9OsZ3QkrIhVR0q9ndCesiFRESb+e0Z2wIlIRtd6ph3QnrIiURyV9EZE0oqQvIpJG4kr6ZjbSzJaa2XIzm1DG9G5m9oqZLTSz18ysS8y0s8xsWfQ6K5HBi4hI1VSa9M2sIXAfcCyQCYw1s8xSs90JPObufYAbgVuiZdsBE4FBwEBgopm1TVz4IiJSFfGU9AcCy939U3ffBswETiw1TybwavR5Tsz0Y4CX3P0rd98IvASM3PWwRUSkOuJJ+p2Bz2OG86JxsXKBU6LPJwMtzax9nMtiZuPNLMfMcvLz8+ONXUREqihRF3KvAg4zs/eAw4DVwI54F3b36e6e7e7ZHTt2TFBIIiJSWjzt9FcD+8QMd4nGFXH3NUQlfTNrAZzq7pvMbDVweKllX9uFeEVEZBfEU9KfDxxgZt3NbDdgDPBs7Axm1sHMCtd1NfBQ9PlF4GgzaxtdwD06GiciIklQadJ39+3ARYRkvQSY5e6LzOxGMxsVzXY4sNTMPgb2ACZHy34F3EQ4cMwHbozGiYhIEpi7JzuGErKzsz0nJyfZYYiI1ClmtsDdsyubT3fkioikESV9EZE0oqQvIpJGlPRFRNKIkr6ISBpR0hcRSSNK+gk0YwZkZECDBuF9xoxkRyQiUpIel5ggM2bA+PFQUBCGV64Mw6BHF4pI6lBJP0GuvbY44RcqKAjjRURShZJ+gqxaVbXxIiLJoKSfIF27Vm28iEgyKOknyOTJ0KxZyXHNmoXxIiKpQkk/QcaNg+nToVs3MAvv06frIq6IpBa13kmgceOU5EUktamkLyKSRpT0RUTSiJK+iEgaUdIXEUkjSvoiImlESV9EJI0o6YuIpBElfRGRNKKkLyKSRpT0RUTSiJK+iEgaUdIXEUkjSvoiImlESV9EJI0o6YuIpBElfRGRNKKkLyKSRpT0RUTSiB6XWA1btkB+Pnz5ZfErdnjzZthjj/Cc3K5di987d4ZG2uMikkRKQcC2bbB+/c7Ju/Rw4edvvy17PU2aQKdO0KoV/Oc/YZ2xGjQIiT/2QFD6vWXLmv++IpK+6mXS37EDNmwoO2GXNbxpU9nradQoJPGOHcP7/vuXHC79uXlzMCtevqAAVq0Kr5UrS35+6y2YNQu2by+5zTZtyj8gdO0Ke+4ZDh4iItVRb5L+l1/C8OEhma9fD+47z2MGHToUJ+l+/cpO3oXDbdqUTOJV1awZ9OgRXmXZsQO++KLsg8LKlTB3bqgqitW4MeyzT9kHhG7dwrSmTasfs4jUb/Um6bdsGZLrsGHlJ/J27aBhw2RHWqxhw1Dd07kzDB5c9jybN8PnnxcfFGIPDq+8AmvWwI8/llymU6dwEBg8GKZO1XUEESlWb9JB06bw9NPJjiLxWrcOr169yp7+ww+wevXO1UjLlsEf/hCWGz++dmOW1PX99/D66/Dmm9C9Oxx8MBx4YPpWGRYUQG5uuK536KG7dmZfV9SbpJ+uGjeGjIzwiuUe/oivvx7OOANatEhGdJIKli+H2bPhhRdgzpzQ+ixWq1YwYAAMHBgOAgMHQpcu9S8Bbt0KCxdCTk7xa9GiUM0KYR9MmgQ/+1n9++6xzMuq/C49k9lI4B6gIfCgu99aanpX4FGgTTTPBHd/3swygCXA0mjWee7+64q2lZ2d7Tk5OVX8GlKWefNCFc/EieGPOZ098ggsXQo//WnYJx06JDuimlNQEJJ7YaL/5JMwfv/9YeTI8Dr00HBGOH8+vPNOeM/NDWeOEBoMFB4ADj44vNq1S953qqoffggJPTbBL1xY/P06dAjfKTs7vPLzYfJk+OyzMH7SJDj22LqV/M1sgbtnVzpfZUnfzBoCHwNHAXnAfGCsuy+OmWc68J67329mmcDz7p4RJf1/uns5lRM7U9JPrNNOg+efD9U9e+2V7GiSY/Fi6NOnuEQH8JOfhANA4atnz7pbxeEOH30UEvzs2aEBwNatocrziCOKE/3++1e8nu+/D4k/9kDw0UfF0/ffv+SBoH//0Fgh2XbsCHHm5ISYc3Lg/ffDPoDQIKMwuRe+unbdOaH/8AM89hjcfDOsWAGDBoXkf8wxdSP5JzLpDwYmufsx0fDVAO5+S8w8DwCfuvtt0fx3uftPlfSTb/lyyMyEc86BBx5IdjS1zz38086fH0p6n30Wmsu++WZ4Fd5L0bp1OAMoPAgMHJja90x8/TW8+mpI8rNnh2s5EA5eI0eGUuqwYeHekV2xeTMsWBAOAoUHgry8MK1hw3DNaODA4gPBQQfVbMOBH38MBZjYEvy774azGwjVmAMGFCf3gw+GffetWtLetg0efTQk/1Wr4JBD4IYb4KijUjv5JzLpjwZGuvv50fAvgEHuflHMPHsB/wbaAs2BI919QZT0FxHOFL4GrnP318vYxnhgPEDXrl0HrCz8C5aEuPRS+OMf4cMPQ1JIJ889B6NGwT33wCWXlJzmHg6KhQeAN98MVQLuodTfp0/JA0H37sn7p3cPB63CJP/GG+EejxYt4Mgji0vz3brVfCxr15Y8G3jnneJ7XZo2hayskgeCqibdQu7hIB2b4BcsCAe8wm3171+c3LOzwxlcos7Ytm2Dhx8O1T6ffx7+Bm64AUaMSM3kX9tJ/4poXXdFJf0/A72AxkALd99gZgOAvwMHufvX5W1PJf3EW78e9tsPDjsMnn022dHUnq1bQ0m0ceNQbdG4ceXLbNoEb79dfDYwbx58802YtsceJauEsrJ2vSRdkY0b4eWXi6tt1q4N4/v0CSX5kSNDHLvtVnMxxKPw4Bl7IHj33VBdBOFaQGG1UOGBYI89dl5HXl7JKpqcnLAPIHzHfv1KVtH07Fk7zZG3boWHHoIpU0KMQ4eGap8jjkit5B9v0sfdK3wBg4EXY4avBq4uNc8iYJ+Y4U+BTmWs6zUgu6LtDRgwwCXxbrnFHdxfey3ZkdSe228P33n27OqvY/t299xc9/vvd//FL9z32y+sE9x328198GD3K690f/pp9zVrdi3eHTvc5893v+km9yFD3Bs0CNtp08b95z93f+gh99Wrd20btWXbNvf33nN/4AH3885z79On+PuAe9eu7qee6v7b37ofd5x7p07F0xo1cu/f3/2CC8LyCxa4b92a7G/k/v337n/8o3vnziHOQw91f/XVZEdVDMjxSvK5u8dV0m9EqJ4ZAawmXMg9w90XxczzAvBXd3/EzHoCrwCdgQ7AV+6+w8z2BV4Herv7V+VtTyX9mrFlSzj13WuvUHqtqxct4/XFF+H7Hn544s9u1q0rPhN4661QMi28aJiRUfJsoHfvikuj+fnw73+HkvyLL4ZhCCXZwiqbQYPqxw12330H771Xslpo5cpQYo+tounTp2bPoHbV99/Dgw/CLbeEmyMPOyxU+xx2WHLjSlhJPzooHEdI/J8A10bjbgRGRZ8zgf8AucD7wNHR+FMJZwHvA+8CJ1S2LZX0a84jj4QSypNPJjuSmnfuue6NG7t//HHNb2vrVvd589ynTnUfPdp9r72KS63Nm7sfcYT7dde5P/+8e36++5tvul9/vfvBB7ubhfnat3c/4wz3xx93X7eu5mNOFTt2JDuC6tuyxf2ee9z33DP8hsOHu8+dm7x4SFRJv7appF9zduwILRu+/hqWLIHdd092RDUjJyfUHV91Fdx+e+1v3z20+ii8OPzWW6EJYWyT0QYNQgm+sKVNVlZqdREi8duyJbSMu/XWcBY4YkQo+Q8ZUrtxJOxCbm1T0q9Z//53aMI4dSpcfnmyo0k899BUcdmy8GrVKtkRBd99V1yl0a1baP5Xl252ksoVFMD//A/cdlvoAPKoo8IF35/+tHa2r6Qv5Spst/7JJ9C2bbKjSayZM2HsWPjzn+Hcc5MdjaSjggK4//6Q/PPz4eijQ8n/kENqdrvxJv16fjlPynL77aFp4i23VD5vXVJQAL/9bajCOvvsZEcj6apZM7jyynCPwe23h+argweHarx33kl2dEr6aalvX/jlL+Hee4vv5KwPbr89tKOeNq3+t06S1Ne8eSiEfPZZqO+fPz9cx/nZz8LnZNG/Rpq66aZwY8l11yU7ksRYtSqcTo8ZE26eEUkVLVrA734Xkv+UKaHJ9MCBcMIJ4Q7j2qakn6b22QcuuwyeeCKcftZ1//3f4SB2223JjkSkbC1bwtVXh87cJk8Oz9HOzg7dhNTm/6CSfhqbMAHatw+noCl2Pb9KXn8d/vrXUJrq2jXZ0YhUrGVLuOaakPxvuin8/Q4YACedFJr21jQl/TTWunV4yEphb4110Y4doUO5ffYJBy+RuqKVMApyAAAMDElEQVRVq1C9umJFaN3z2mvhWltNF8DUZDPNbdsWul5u2jSUMuraDUIPPggXXBCaap5+erKjEam+TZtCQ4TyHo1aGTXZlLjstltouvnhh6EP8bpk8+Zwmjx0aHhYjEhd1qZN9RN+VSjpC6NHhxtHfv/7cOdoXXHTTaHb6HvuSa0ubkVSmZK+YAZ33BF6DJw2LdnRxGfp0pDszzsv9FsjIvFR0hcgVJGcdFJxvyGp7sorw52PN9+c7EhE6hYlfSly662hK4Mbb0x2JBV74QX4179Cy6PST2ASkYop6UuRAw+E8eNDN7Eff5zsaMr2ww9wxRVwwAFw8cXJjkak7lHSlxImTgxPLZowIdmRlO1Pf4KPPgpdQyf72bAidZGSvpSwxx6hS4Nnngm3iaeS/PxwUDrmmNBplYhUnZK+7OSKK8KzdFOte4brr4dvv4W771YTTZHqUtKXnTRvHtrAv/UW/O1vyY4myM2F6dPhoovCg7RFpHrUDYOUaceO0O/+1q2waFFy68/d4Ygj4IMPwiMQ69vTvkQSQd0wyC5p2DA8lGT58tCaJ5n+9rfQGdXNNyvhi+wqlfSlXO4wYkQoYS9fHnrlrG1btoQO4Vq1Cn2O17UO4URqi0r6sssKu2dYvz55DyeZOjV0PTttmhK+SCIo6UuFBgyAM84ILWby8mp326tXh8fLnXoqDB9eu9sWqa+U9KVSkyfDjz+GXjhr04QJ4YLyHXfU7nZF6jMlfalURgZccknobz83t3a2+dZb4fm9V14J3bvXzjZF0oGSvsTlmmvCQx5+97ua39aPP4ZHIO61V3iQtIgkjpK+xKVt2/A8zxdfhJdeqtltPf44zJ8fLh63aFGz2xJJN2qyKXHbuhV69AhNN999FxrUQJHhm2/gJz+Bbt3gzTdrZhsi9ZGabErC7b57aE2Tmxvq22vClCnwxRfhqVhK+CKJp38rqZLTTw/NOK+7Ltw4lUiffhra5f/ylzBoUGLXLSKBkr5USYMGcOed8PnncO+9iV33VVdB48Zwyy2JXa+IFFPSlyo7/HA4/vhQFbN+fWLW+coroQ//a6+FvfdOzDpFZGdK+lItt90W+rZPxIPJt2+Hyy4L7fEvv3zX1yci5VPSl2rJzITzzguPL/zkk11b1/Tp8OGHcNdd4VGNIlJzlPSl2m64IdTBX3NN9dfx1Vehe4fhw+GkkxIXm4iUTUlfqm2vvcLF11mz4O23q7eOSZNg06bQi6YegShS85T0ZZdcdRV06hTeq3qf36JFoXroV7+CPn1qJj4RKUlJX3ZJy5ahmueNN+DZZ+Nfzj1ctG3ZEm68sebiE5GS4kr6ZjbSzJaa2XIzm1DG9K5mNsfM3jOzhWZ2XMy0q6PllprZMYkMXlLD+efDgQeGzth++CG+ZZ57LvThc8MN0KFDzcYnIsUqTfpm1hC4DzgWyATGmllmqdmuA2a5e39gDPCnaNnMaPggYCTwp2h9Uo80ahSacC5dCn/+c+Xzb90KV1wBPXvChRfWfHwiUiyekv5AYLm7f+ru24CZwIml5nGgVfS5NbAm+nwiMNPdt7r7Z8DyaH1Sz4waBcOGwcSJodO0itxzT2jmeffdofWPiNSeeJJ+Z+DzmOG8aFysScCZZpYHPA9cXIVlMbPxZpZjZjn5+flxhi6ppPB5ul9+WfGTrr74Am66CU44AY5RZZ9IrUvUhdyxwCPu3gU4DnjczOJet7tPd/dsd8/u2LFjgkKS2jZoEJx2WrjJas2asue55ppQvXPXXbUbm4gE8STm1cA+McNdonGxzgNmAbj7W0AToEOcy0o9MmVKuJg7ceLO03Jy4OGHQ5cLBxxQ+7GJSHxJfz5wgJl1N7PdCBdmSzfOWwWMADCznoSknx/NN8bMdjez7sABwDuJCl5Sz377wX/9Fzz0UGiHX8g9PAKxU6fQLbOIJEelSd/dtwMXAS8CSwitdBaZ2Y1mNiqa7UrgAjPLBZ4EzvZgEeEMYDEwG/iNu++oiS8iqeP3vw/t72OfpztzZngS1pQp0KpV+cuKSM3S4xKlRtx2G0yYAK++CgMHhscsduoE77wDDdVoVyTh9LhESapLLoF99oHf/hZuvRXy8kJTTSV8keRqlOwApH5q2jT0tX/WWbBgAYwZA0OHJjsqEVFJX2rMmWdC377hAHDbbcmORkRAJX2pQQ0awOzZsG4ddO2a7GhEBJT0pYbtuWd4iUhqUPWOiEgaUdIXEUkjSvoiImlESV9EJI0o6YuIpBElfRGRNKKkLyKSRpT0RUTSiJK+iEgaUdIXEUkjSvoiImlESV9EJI0o6YuIpBElfRGRNKKkLyKSRpT0RUTSiJK+iEgaUdIXEUkjSvoiImlESV9EJI0o6YuIpBElfRGRNKKkLyKSRpT0RUTSiJK+iEgaUdIXEUkjSvoiImlESV9EJI0o6YuIpBElfRGRNKKkLyKSRpT0RUTSiJK+iEgaUdIXEUkjcSV9MxtpZkvNbLmZTShj+t1m9n70+tjMNsVM2xEz7dlEBi8iIlXTqLIZzKwhcB9wFJAHzDezZ919ceE87n55zPwXA/1jVrHF3fslLmQREamueEr6A4Hl7v6pu28DZgInVjD/WODJRAQnIiKJFU/S7wx8HjOcF43biZl1A7oDr8aMbmJmOWY2z8xOKme58dE8Ofn5+XGGXtKMGZCRAQ0ahPcZM6q1GhGReq3S6p0qGgM85e47YsZ1c/fVZrYv8KqZfeDun8Qu5O7TgekA2dnZXtWNzpgB48dDQUEYXrkyDAOMG1edryEiUj/FU9JfDewTM9wlGleWMZSq2nH31dH7p8BrlKzvT4hrry1O+IUKCsJ4EREpFk/Snw8cYGbdzWw3QmLfqRWOmfUA2gJvxYxra2a7R587AEOAxaWX3VWrVlVtvIhIuqo06bv7duAi4EVgCTDL3ReZ2Y1mNipm1jHATHePrZ7pCeSYWS4wB7g1ttVPonTtWrXxIiLpykrm6OTLzs72nJycKi1Tuk4foFkzmD5ddfoikh7MbIG7Z1c2X724I3fcuJDgu3UDs/CuhC8isrNEt95JmnHjlORFRCpTL0r6IiISHyV9EZE0oqQvIpJGlPRFRNKIkr6ISBpJuXb6ZpYPrEx2HLuoA7A+2UGkEO2PkrQ/imlflLQr+6Obu3esbKaUS/r1gZnlxHOTRLrQ/ihJ+6OY9kVJtbE/VL0jIpJGlPRFRNKIkn7NmJ7sAFKM9kdJ2h/FtC9KqvH9oTp9EZE0opK+iEgaUdIXEUkjSvoJZGb7mNkcM1tsZovM7NJkx5RsZtbQzN4zs38mO5ZkM7M2ZvaUmX1kZkvMbHCyY0omM7s8+j/50MyeNLMmyY6pNpnZQ2b2pZl9GDOunZm9ZGbLove2id6ukn5ibQeudPdM4BDgN2aWmeSYku1SwhPXBO4BZrt7D6AvabxfzKwzcAmQ7e69gIaEp++lk0eAkaXGTQBecfcDgFei4YRS0k8gd1/r7u9Gn78h/FN3Tm5UyWNmXYCfAQ8mO5ZkM7PWwKHAnwHcfZu7b0puVEnXCGhqZo2AZsCaJMdTq9x9LvBVqdEnAo9Gnx8FTkr0dpX0a4iZZQD9gbeTG0lSTQP+G/gx2YGkgO5APvBwVN31oJk1T3ZQyeLuq4E7gVXAWmCzu/87uVGlhD3cfW30+Qtgj0RvQEm/BphZC+Bp4DJ3/zrZ8SSDmR0PfOnuC5IdS4poBGQB97t7f+A7auDUva6I6qpPJBwM9waam9mZyY0qtXhoT5/wNvVK+glmZo0JCX+Gu/8t2fEk0RBglJmtAGYCR5jZE8kNKanygDx3Lzzze4pwEEhXRwKfuXu+u/8A/A34aZJjSgXrzGwvgOj9y0RvQEk/gczMCHW2S9x9arLjSSZ3v9rdu7h7BuEC3avunrYlOXf/AvjczA6MRo0AFicxpGRbBRxiZs2i/5sRpPGF7RjPAmdFn88C/pHoDSjpJ9YQ4BeEUu370eu4ZAclKeNiYIaZLQT6AVOSHE/SRGc8TwHvAh8QclFadclgZk8CbwEHmlmemZ0H3AocZWbLCGdDtyZ8u+qGQUQkfaikLyKSRpT0RUTSiJK+iEgaUdIXEUkjSvoiImlESV9EJI0o6YuIpJH/D4ux2iA8BkaWAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXcAAAEICAYAAACktLTqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvhp/UCwAAIABJREFUeJzt3Xuc1XP+wPHXu+uUUlRupYtrTUqXkfVLumj9SsglVhfSsrlkQ1htJUSoTWw2VmiwDYMsi0VriYofmpJREsmU6S5KKWrq/fvjfaZmpmnmzMw553vOmffz8ZjHnPM93/P9vufMzPt8zuf7+bw/oqo455xLLlWCDsA551zkeXJ3zrkk5MndOeeSkCd355xLQp7cnXMuCXlyd865JOTJ3RVLRKqKyDYRaRrJfYMkIseJSMTH/opITxHJKXB/mYh0CWffcpzrCREZVd7nl3Dce0TkqUgf1wWnWtABuMgQkW0F7tYGfgV2h+5fraoZZTmequ4G6kR638pAVU+MxHFE5CpgkKp2K3DsqyJxbJf8PLknCVXdm1xDLcOrVPW/B9pfRKqpal4sYnPOxZ53y1QSoY/dz4vIcyKyFRgkIqeJyEcisllE1orIFBGpHtq/moioiDQP3Z8RevxNEdkqIv8nIi3Kum/o8d4i8pWIbBGRh0XkAxG54gBxhxPj1SKyXER+FJEpBZ5bVUQeFJFNIrIC6FXC6zNaRDKLbJsqIpNDt68SkaWhn+ebUKv6QMfKFZFuodu1ReQfodiWAB2L7DtGRFaEjrtERM4LbW8D/A3oEury+r7Aa3tngedfE/rZN4nIKyJyZDivTWlE5IJQPJtF5F0RObHAY6NEZI2I/CQiXxb4WX8jIgtD29eLyF/CPZ+LAlX1ryT7AnKAnkW23QPsBM7F3tRrAacAp2Kf4I4BvgKuD+1fDVCgeej+DOB7IA2oDjwPzCjHvocBW4G+ocdGALuAKw7ws4QT47+AekBz4If8nx24HlgCNAEaAHPsT77Y8xwDbAMOKnDsDUBa6P65oX0E6AHsANqGHusJ5BQ4Vi7QLXR7EvAecAjQDPiiyL6XAEeGficDQjEcHnrsKuC9InHOAO4M3T4rFGM7IAV4BHg3nNemmJ//HuCp0O1WoTh6hH5Ho4BlodutgZXAEaF9WwDHhG7PB/qHbtcFTg36f6Eyf3nLvXKZp6qvqeoeVd2hqvNV9WNVzVPVFcA0oGsJz5+pqlmqugvIwJJKWfc9B1ikqv8KPfYg9kZQrDBjvE9Vt6hqDpZI8891CfCgquaq6ibg/hLOswJYjL3pAPwW+FFVs0KPv6aqK9S8C7wDFHvRtIhLgHtU9UdVXYm1xgue9wVVXRv6nTyLvTGnhXFcgIHAE6q6SFV/AUYCXUWkSYF9DvTalORS4FVVfTf0O7ofe4M4FcjD3khah7r2vg29dmBv0seLSANV3aqqH4f5c7go8OReuXxX8I6ItBSRf4vIOhH5CRgHNCzh+esK3N5OyRdRD7TvUQXjUFXFWrrFCjPGsM6FtThL8izQP3R7QOh+fhzniMjHIvKDiGzGWs0lvVb5jiwpBhG5QkQ+C3V/bAZahnlcsJ9v7/FU9SfgR6BxgX3K8js70HH3YL+jxqq6DLgZ+z1sCHXzHRHadQiQCiwTkU9E5Owwfw4XBZ7cK5eiwwAfw1qrx6nqwcBYrNshmtZi3SQAiIhQOBkVVZEY1wJHF7hf2lDNF4CeItIYa8E/G4qxFjATuA/rMqkP/CfMONYdKAYROQZ4FLgWaBA67pcFjlvasM01WFdP/vHqYt0/q8OIqyzHrYL9zlYDqOoMVe2MdclUxV4XVHWZql6Kdb09ALwkIikVjMWVkyf3yq0usAX4WURaAVfH4JyvAx1E5FwRqQbcADSKUowvADeKSGMRaQDcVtLOqroOmAc8BSxT1a9DD9UEagAbgd0icg5wZhliGCUi9cXmAVxf4LE6WALfiL3P/QFruedbDzTJv4BcjOeAK0WkrYjUxJLsXFU94CehMsR8noh0C537Vuw6ycci0kpEuofOtyP0tQf7AS4TkYahlv6W0M+2p4KxuHLy5F653QwMxv5xH8MufEaVqq4HfgdMBjYBxwKfYuPyIx3jo1jf+OfYxb6ZYTznWewC6d4uGVXdDNwEvIxdlOyHvUmF4w7sE0QO8CbwTIHjZgMPA5+E9jkRKNhP/TbwNbBeRAp2r+Q//y2se+Tl0PObYv3wFaKqS7DX/FHsjacXcF6o/70mMBG7TrIO+6QwOvTUs4GlYqOxJgG/U9WdFY3HlY9Yl6dzwRCRqlg3QD9VnRt0PM4lC2+5u5gTkV6hboqawO3YKItPAg7LuaQSVnIP/TMuC02GGHmAfS4RkS9CEx+eLW4f50JOB1ZgH/n/F7hAVQ/ULeOcK4dSu2VCH5u/wsb95rJvosIXBfY5HrsI00NVfxSRw1R1Q/TCds45V5JwWu6dgOWhCRw7gUz2TfTI9wdgqqr+COCJ3TnnghVO4bDGFJ6EkYvNVCvoBAAR+QAb93pn6Er+ATVs2FCbN28efqTOOedYsGDB96pa0vBhIHJVIasBxwPdsMkOc0SkTWgI2V4iMhQYCtC0aVOysrIidHrnnKscRKS0mdZAeN0yqyk8w27vTLUCcrFaFLtU9Vusj/74ogdS1WmqmqaqaY0alfrG45xzrpzCSe7zsWJALUSkBqGiQkX2eQVrtSMiDbFumhU455wLRKnJXW1Bh+uBWcBS4AVVXSIi4/JrT4ce2yQiXwCzgVtDVficc84FILAZqmlpaep97s45VzYiskBVSy0L7TNUnXMuCXlyd865JOTJ3TnnkpAnd+ecixFVuOUWWLIk+ufy5O6cczHyyivwwAOwYEH0z+XJ3TnnYmDPHrjjDjjhBBgwIPrni1T5AeeccyX45z/h888hIwOqxSDzesvdOeeibPdua7W3agW/+11szuktd+eci7IXX4QvvoDMTKhaNTbn9Ja7c85F0e7dcNdd0Lo1XHxx7M7rLXfnnIuizEz48kuYOROqxLA57S1355yLkrw8a7W3bQsXXBDbc3vL3TnnoiQjA77+Gl5+ObatdvCWu3PORcWuXXD33dC+PfQtuup0DHjL3TnnouAf/4BvvoHXXgOR2J/fW+7OORdhO3daq/2UU6BPn2Bi8Ja7c85F2FNPQU4OPPJIMK128Ja7c85F1K+/wvjx8JvfQK9ewcXhLXfnnIug6dNh1Sp44ongWu3gLXfnnIuYX36xVnvnztCzZ7CxeHJ3EfHoo7BsWdBROBesxx+H1ath3LhgW+3gyd1FQHY2XHcdXHtt0JE4F5wdO+C++6BrV+jePehoPLm7CEhPt++zZ8O8ecHG4lxQHnsM1q61cgNBt9rBk7uroJ07YcYMOPtsOOwwG9vrXGXz88/Wau/Rw1ru8cCTu6uQf/8bvv8ehg2zhX//8x/46KOgo3Iuth59FDZssFZ7vPDk7iokPR2OPBLOOsv63Bs08Na7q1y2bYOJE+1/4PTTg45mH0/urtzWrYM33oDLL7c1IevUgREjbFtWVtDRORcbU6fCxo3x1WoHT+6uAmbMsFVmhgzZt+3666F+fbjnnuDici5WfvrJWu29e9uM1Hjiyd2Vi6p1yZx2Gpx44r7tBx8MN90E//oXLFoUXHzOxcLDD8MPP8Rfqx08ubtymj/fFvwt2GrPN3y4JXlvvbtktmULPPAAnHuuVX+MN57cXbmkp0OtWvC73+3/WP36luBfegkWL459bM7Fwl//Cj/+CHfeGXQkxfPk7spsxw547jm46CJroRfnxhvtAuv48bGNzblY2LwZJk+G88+HDh2CjqZ4ntxdmb3yin0kLa5LJl+DBnZx9fnnbeV355LJgw/a/0C8ttrBk7srh/R0aN4cunUreb8RI6zrxlvvLpn88IMl93794OSTg47mwDy5uzJZtQr++18YPLj01dwbNbKCYs8+ayvAO5cMHnjAJi7dcUfQkZTMk7srk2eesWGQgweHt//NN0ONGnDvvdGNy7lY+P57mDIFLrkETjop6GhKFlZyF5FeIrJMRJaLyMhiHr9CRDaKyKLQ11WRD9UFTdXWhuzeHVq0CO85RxwBV19tK8F/+21Uw3Mu6iZNsiJh8d5qhzCSu4hUBaYCvYFUoL+IpBaz6/Oq2i709USE4wQgI8P6eqtUse8ZGdE4izuQuXPhm29KvpBanD/9ycoT3HdfdOJyLhY2bLBJS/37Q6tWQUdTunBa7p2A5aq6QlV3AplA3+iGtb+MDBg6FFautBbkypV23xN87KSnQ926NgSyLI46Cq66ylr9K1dGJTTnom7iRFtGb+zYoCMJTzjJvTHwXYH7uaFtRV0kItkiMlNEji7uQCIyVESyRCRr48aNZQp09GjYvr3wtu3bbbuLvm3b4MUXbdJS7dplf/5tt9n3CRMiG5eLf1dfDeecY0MHE9W6dfDIIzBoUOFyG/EsUhdUXwOaq2pb4G3g6eJ2UtVpqpqmqmmNGjUq0wlWrSrbdhdZL75ofY1l7ZLJd/TR9twnn7Q1Jl3lMGcOTJtmdf+7dYP164OOqHwmTLCFaW6/PehIwhdOcl8NFGyJNwlt20tVN6nqr6G7TwAdIxPePk2blm27i6z0dGuxnHZa+Y/x5z/Dnj328dYlvz17bK5Dkybw8svw1VdW7zwnJ+jIymbNGluM4/LL4bjjgo4mfOEk9/nA8SLSQkRqAJcCrxbcQUSOLHD3PGBp5EI048fv3x1Qu7ZPkImF5cvtYuoVV1Rsbcjmze0fZNo0W2vSJbdnn4UFC2wY7Pnn2/yITZugc+fEqjl0331W2nrMmKAjKSNVLfULOBv4CvgGGB3aNg44L3T7PmAJ8BkwG2hZ2jE7duyoZTVjhmqzZqoi9n3GjDIfwpXD6NGqVaqorl5d8WMtX65atarqiBEVP5aLXz//rNqkiWrHjqq7d+/bvnix6lFHqR5yiOqHHwYXX7hWrVKtUUP1D38IOpJ9gCwNJ2+Hs1M0vsqT3F3s5eXZP2nv3pE75mWXqdaqpbp+feSO6eLLPfdYdnn//f0f+/Zb1eOOU61dW/Wtt2IeWplce61q9eqqOTlBR7JPuMndZ6i6Er3zDuTmlv9CanFGj7YhZZMnR+6YLn6sWwf3329dMWecsf/jzZvDvHlwwglWCz0zM+YhhmXlSnjiCbjySmjWLOhoys6TuytRejoceiicd17kjnniiXDppfC3v9l0bpdc7rjD3rxLunB++OHw3nt2gX7AALtgGW/Gj7drTKNGBR1J+Xhydwf04482ymHAAKhZM7LHzp+38NBDkT2uC9bixdbaHTYMjj++5H3r1YO33rIx8NddB3ffbRMU48G331rDZuhQG8abiDy5uwPKzIRff41sl0y+1q2tZOqUKfYm4pLDLbfYAi7hzuKsVQv++U8bRTV2rC3ysmdPdGMMxz33QNWqNnw3UXlydweUng5t20L79tE5/pgxsHWrLVfmEt9bb8GsWZakDz00/OdVq2Z/azfdZG/2gwfDrl3Ri7M0y5fD00/DNddY6YxE5cndFWvJElsEe8iQio1tL0nbtnbR7a9/Teyp6Q7y8qzVfuyx1iVTVlWqWJ30e++FGTPgggv2LzcSK3ffbWWqR+5X/zaxeHJ3xUpPtxbVwIHRPc/tt9t6lH/7W3TP46Jr+nRrEEyYYImxPESsG+Sxx+CNN+B//9f+NmLpq6/szeW666xcdSITDegKRlpammZlZQVybleyXbtsynjnztYfGm3nngsffmjT0uvWjf75XGRt3WrT8k84wWrJROKT3osvWsOiVSvr6olVoh00yAYRfPstHHZYbM5ZViKyQFXTStvPW+5uP2++abWro3EhtTi3327rUj7ySGzO5yLr/vvt72Xy5Mh14V18sRUb++Yba2SsWBGZ45Zk6VIrmXD99fGb2MvCk7vbT3q6jUPu3Ts25+vUyT6CP/CAVZ50ieO77yypDxgAp5wS2WP/9rc2iW7zZkvw2dmRPX5Rd90FBx0Et94a3fPEiid3V8iGDfD663DZZdbnHitjx8LGjdbf6hJH/gSfaK2ydeqpVrSualXo2hU++CA651m8GF54AYYPh4YNo3OOWPPk7grJyLCRD7Hqksn3P/8DZ55psxp37IjtuV35zJ9vFx9vuim6pbdTUy2pH3aYtebfeCPy57jrLqhTxxZ0Txae3N1eqtYl06mT/UPF2tixtpjD44/H/tyubFQtER52WGyGDDZrZi34Vq2gb1/rG4+Uzz6DmTNtAlVZxufHO0/ubq+FC+Hzz2Pfas93xhn2NWGC1SZx8euVVyzZjhtnM1Jj4bDDYPZsW/Bj4MDIDZ+9804rhXDTTZE5Xrzw5O72Sk+HlBQr6hWUsWNt5Zv09OBicCXbuRP+9Cf7dHfllbE998EH22iu88+HP/7REnNFRnMvXGhvVDfdBIccErEw44IndwdYS/nZZ21mYP36wcXRo4f1v993nyURF38eecSm6E+aFNuL7vlSUmwc/JAh1lc+fHj569Hceaf9vd94Y0RDjAue3B0Ar75qBbyC6pLJJ2Kt9+++s/oeLr788IN1xZx1FvTqFVwc1arZYuu33GLdM4MGlb0xMH8+vPaaHaNevejEGSRP7g6wbpCjj7aWc9DOOssu6t57b7AFpNz+7r7b6gBNmhS9mkPhEoG//MUmUT33nF1oLcs8iTvusAuow4dHL8YgeXJ3rF4N//mPVeOrWjXoaOyf9vbbrRzBjBlBR+PyLV8OU6fC738PbdoEHc0+t91mi67/5z/WMAinhPRHH1nf/a23Jm/JC0/ujmeesT7LK64IOpJ9+vSxUsP33mvj7l3wbrvNioLdfXfQkezvD3+wSUhZWTbias2akve/4w6brHT99bGJLwie3Cu5/LHtZ5xh5VrjRX7f+/Ll8bvGZmUyZ44VkRs5Mn6rJV50kU1wysmx4ZLLlxe/3wcfWCv/ttts4lKy8qqQldwHH9g/Qnp6fLXcwT5NtG9vq0EtWRIfXUaV0Z49VgZg7VoriVu7dtARlWz+fKuLVK2aVZQ8+eTCj/fsaeUGVqyI/5+lOF4V0oUlPd2KJfXrF3Qk+6tSxVZrWrbMhr65YDz3nHV33HdfYiTDU06BefOgenWrRzN37r7H3n/fipHddlti/CwV4S33Suznn+0j9sUX22IL8WjPHrt4J2JVAat4cySmduyAE0+02aGffJJYr/+qVXaBdeVKaxz06QPdutmnjxUrbP3WROQtd1eql16CbduCH9tekvzW+5IltoiCi60HH7Q5Bw88kFiJHayY2dy5cNJJNqP1hhvs2sGoUYmb2MvCW+6VWPfukJtrLZmgxyyXZPdum+qekgKffpp4SSZRrV9vKyz17JnYb6xbt1pyf/ddaNzYLrSmpAQdVfl5y92VaMUKeO89u4gaz4kd7ELqmDHWLfPaa0FHU3mMHWtlKSZMCDqSiqlb11Z1uvlmeOKJxE7sZeHJvZJ6+mlL6pdfHnQk4enf34ZqjhtXsUJRLjyLF1sivO46Wxs10aWk2KzaIEsmxJon90pozx5L7r/9rZUcSATVqllf6cKFNrPQRdett1oFxrFjg47ElZcn90po9mwbQRDPF1KLc9lltmiDt96ja9YseOstKwHRoEHQ0bjy8uReCaWnW5nT888POpKyqV7dWu8ffwxvvx10NMkpL8/6po85BoYNCzoaVxGe3CuZLVtsCGT//ol5YWnwYOtK8tZ7dEyfbsNOJ06EmjWDjsZVhCf3Sub5520ERKJ1yeSrWdNmF37wgY32cZGzdat1xZx+Olx4YdDRuIry5F7JpKdD69aQVuoo2fh15ZVw5JHWeneRM2ECbNhgE5bifXisK50n90pk6VKrYz1kSGL/86akWOv9vfdsxqGruPxZqAMG2EIpLvGFldxFpJeILBOR5SIysoT9LhIRFZEEbhcmr6eesglBgwYFHUnF/eEPcPjh8VlbPBGNGmXXMO69N+hIXKSUmtxFpCowFegNpAL9RSS1mP3qAjcAH0c6SFdxeXm2KEefPpYUE13t2rb25X//C//3f0FHk9iysmzFq5tusqGmLjmE03LvBCxX1RWquhPIBPoWs9/dwATglwjG5yLkrbdg3brEvZBanGuusdV0vPVefqowYgQ0agR//nPQ0bhICie5Nwa+K3A/N7RtLxHpABytqv8u6UAiMlREskQka+PGjWUO1pVferr9A/fpE3QkkVOnjo3JfvNNW6AhSJs3WwXCRx6xN52LL7a44n245iuvWNzjxtmMVJc8Sq0KKSL9gF6qelXo/mXAqap6feh+FeBd4ApVzRGR94BbVLXEko9eFTJ2vv8ejjrK1oucPDnoaCJr61Zo3hw6d4ZXX43++XbutMVDPv/cvrKz7ft3BZo/9evbWqMbNthKUqNG2dDCeKtmuXOnjZyqUQM++8xKPLj4F25VyHB+nauBghVImoS25asLnAS8JzYE4wjgVRE5r7QE72IjIwN27UquLpl8detaX/Htt1s54PbtI3NcVSuHXDSJf/mlvZZgM2ZbtrT1Z9u0sa+2ba2s7K5d1o99//3Wim/Z0tYfHTDAnhcPHnnEyt++8YYn9mQUTsu9GvAVcCaW1OcDA1R1yQH2fw9vuceVdu3snzdZX+4tW+xCYI8etohzWf30k1VBLJrIN2/et8/RR+9L3vnfTzjBWr0l2b3bZgTfe6+1jps1gz/9CX7/+2BnCP/wg9VqT0uzWjKJPDS2solYy11V80TkemAWUBWYrqpLRGQckKWqMfgw7Mrr008tqfztb0FHEj316tkqO+PGWVJu06b4/fLybGGSokk8J2ffPnXr2vMvvXRfa7xNG+tqKY+qVeGSS6z1/sYbMH681WwZN86uF1xzjZ0z1u65x968Jk3yxJ6sfCWmJDd8ODz2mK1cf+ihQUcTPT/8YH3vvXtDZqaNDMpP3vmJfOlS+PVX279qVVsbNL8lnt8ab9o0uslO1RZpHj/ehnEecoj9jv74x9hVYFy+3Fa2GjwYHn88Nud0kRNuy92TexL79Ve7kNqzp9WUSXajR8N999mb2KZN+7YfddT+XSotWwZfGOuTTyzeV16Bgw6Ca6+1YYlHHhnd8150kXXFLF9uC6S7xBLJC6ouQb32mrVok/FCanFuvtla5w0bFu5Sidea5J062dqkixfbhdfJk+Hhh60//tZboUWLyJ9z7ly7LjFunCf2ZOct9yTWp4/1t69cad0QLr59842V2n3qKbsQO2CATSxq1Soyx9+zB0491brovvrKZvm6xOMLZFdya9bYrNTLL/fEniiOPdauj6xYYf3wL71k49AvuggWLKj48Z97zkZM3XuvJ/bKwJN7kvrHP6yldsUVQUfiyqpxY+uiycmx6wjvvGNDFnv1sm6V8tixwz4FdOiQHIXjXOk8uSchVSs30LlzcqxcX1k1amR1c1atsguvCxfahKkuXexTWVl6VB98cF9Z33ibKeuiw3/NSeijj2yKfGW5kJrsDj7YZrfm5MCUKXYNpXdv6NgRZs60T2glWb/e3hz69oVu3WIRsYsHntyTUHo61KplE2dc8qhd28bDL18OTz4J27bZ77h1a3j66X1lEYq64w5bWnHixNjG64LlyT3JbN9uk3j69fMqf8mqRg0bLrl0qc1fqFnTrq0cf7zVi/mlQNHtxYttotJ113kXXWXjyT3J/POfVinRu2SSX35pg08/hddft8law4bZ+Pi//MX+Dm691d7kx44NOloXa57ck0x6uk3D79o16EhcrIjYnIYPPoDZs+Gkk6w4WZMmduF1zJj4ncjloseTexLJyYF337WP6D4iovIRsQumb78NH39sVTK7dLE6/q7y8fIDSeTpp+374MHBxuGCl1/awFVe3r5LEnv22LT1Hj2sW8Y5V7l5ck8S779v3TJ+IdU5B57ck0Z6uo2KuPDCoCNxzsUDT+5J4KefbKbi737nBaGccybhkvuGDVau9Oefg44kfrzwghWG8i4Z51y+hBst88wzNjEDbImyJk1K/qoMszTT023JuN/8JuhInHPxIuGS+3nn2QoyubmFvxYutAJJRdWpU/obwKGHJu4iwcuWwYcf2ko+ifozOOciL+GS+wknHLhGxq+/2iozRRN//tfbb9vjRavopaSU/gbQqFFsJgbl5cHOnfazFPe96Lbnn7e4Lrss+rE55xJHwiX3ktSsaWO8SxrnnZcH69YVTvqrV++7PW+e3S9aYa96dVtEIT/ZN24M9euXnIhL+17cttLKtxanb1+rK+Kcc/mSKrmHo1q1fQn6QPbsgY0bD/wJICvLVqz/5RfrCqlZ0yr1Ff1edFudOgfeN9zvxW3zan/OuaIqXXIPR5UqcPjh9tWxY/H7qNqngGrVvK/bORd/PLmXk4h11TjnXDxKuHHuzjnnSufJ3TnnkpAnd+ecS0Ke3J1zLgl5cnfOuSTkyd0555KQJ3fnnEtCntydcy4JeXJ3zrkk5MndOeeSUFjJXUR6icgyEVkuIiOLefwaEflcRBaJyDwRSY18qM4558JVanIXkarAVKA3kAr0LyZ5P6uqbVS1HTARmBzxSJ1zzoUtnJZ7J2C5qq5Q1Z1AJtC34A6q+lOBuwcBGrkQnXPOlVU4VSEbA98VuJ8LnFp0JxEZBowAagA9ijuQiAwFhgI0bdq0rLE655wLU8QuqKrqVFU9FrgNGHOAfaapapqqpjVq1ChSp3bOOVdEOMl9NXB0gftNQtsOJBM4vyJBxbuMDFvKr0oV+56REXREzjlXWDjJfT5wvIi0EJEawKXAqwV3EJHjC9ztA3wduRDjS0YGDB0KK1faakwrV9p9T/DOuXhSanJX1TzgemAWsBR4QVWXiMg4ETkvtNv1IrJERBZh/e6DoxZxwEaPhu3bC2/bvt22O+dcvBDVYAa2pKWlaVZWViDnrogqVazFXpSILaztnHPRJCILVDWttP18hmoZHWiQjw/+cc7FE0/uZTR+PNSuXXhb7dq23Tnn4oUn9zIaOBCmTYNmzawrplkzuz9wYNCROefcPuFMYnJFDBzoydw5F9+85e6cc0nIk7tzziUhT+7OOZeEPLk751wS8uTunHNJyJO7c84lIU/uzjmXhDy5O+dcEvLYZRTWAAAPiUlEQVTk7pxzSciTu3POJSFP7s45l4Q8uTvnXBLy5O6cc0nIk7tzziUhT+7OOZeEPLk751wS8uSewDIyoHlzW7S7eXO775xz4CsxJayMDBg6FLZvt/srV9p98FWinHPeck9Yo0fvS+z5tm+37c4558k9Qa1aVbbtzrnKxZN7gmratGzbnXOViyf3BDV+PNSuXXhb7dq23TnnPLknqIEDYdo0aNYMROz7tGl+MdU5Z3y0TAIbONCTuXOueN5yd865JOTJ3TnnkpAnd+ecS0Ke3F2FeRkE5+KPX1B1FeJlEJyLT95ydxXiZRCci0+e3F2FeBkE5+JTWMldRHqJyDIRWS4iI4t5fISIfCEi2SLyjog0i3yoLh55GQTn4lOpyV1EqgJTgd5AKtBfRFKL7PYpkKaqbYGZwMRIB+rik5dBcC4+hdNy7wQsV9UVqroTyAT6FtxBVWeran7P60dAk8iG6eKVl0FwLj6FM1qmMfBdgfu5wKkl7H8l8GZxD4jIUGAoQFP/3J40vAyCc/EnohdURWQQkAb8pbjHVXWaqqapalqjRo0ieWrnnHMFhNNyXw0cXeB+k9C2QkSkJzAa6Kqqv0YmPOecc+URTnKfDxwvIi2wpH4pMKDgDiLSHngM6KWqG8obzK5du8jNzeWXX34p7yFcDKWkpNCkSROqV68edCjOuSJKTe6qmici1wOzgKrAdFVdIiLjgCxVfRXrhqkDvCgiAKtU9byyBpObm0vdunVp3rw5oeO4OKWqbNq0idzcXFq0aBF0OIDNlh092sbYN21qI3b8WoCrrMIqP6CqbwBvFNk2tsDtnpEI5pdffvHEniBEhAYNGrBx48agQwG8DIJzRcXdDFVP7Ikjnn5XXgbBucLiLrk7Vx5eBsG5whI6uUe61OymTZto164d7dq144gjjqBx48Z77+/cuTOsYwwZMoRly5aVuM/UqVPJiFBd3NNPP51FixZF5FiJzMsgOFdYwpb8jUYfa4MGDfYmyjvvvJM6depwyy23FNpHVVFVqlQp/n0xPT291PMMGzasfAG6Axo/vvDfAwRTBsEv6rp4kbAt91j2sS5fvpzU1FQGDhxI69atWbt2LUOHDiUtLY3WrVszbty4vfvmt6Tz8vKoX78+I0eO5OSTT+a0005jwwYbJTpmzBgeeuihvfuPHDmSTp06ceKJJ/Lhhx8C8PPPP3PRRReRmppKv379SEtLK7WFPmPGDNq0acNJJ53EqFGjAMjLy+Oyyy7bu33KlCkAPPjgg6SmptK2bVsGDRoU8dcs1uKhDEJ+g2PlSlDd1+DwxUtcEBK25R7rPtYvv/ySZ555hrS0NADuv/9+Dj30UPLy8ujevTv9+vUjNbVwPbUtW7bQtWtX7r//fkaMGMH06dMZOXK/opqoKp988gmvvvoq48aN46233uLhhx/miCOO4KWXXuKzzz6jQ4cOJcaXm5vLmDFjyMrKol69evTs2ZPXX3+dRo0a8f333/P5558DsHnzZgAmTpzIypUrqVGjxt5tiS7oMgglNTi89e5iLWFb7rHuYz322GP3JnaA5557jg4dOtChQweWLl3KF198sd9zatWqRe/evQHo2LEjOTk5xR77wgsv3G+fefPmcemllwJw8skn07p16xLj+/jjj+nRowcNGzakevXqDBgwgDlz5nDcccexbNkyhg8fzqxZs6hXrx4ArVu3ZtCgQWRkZPgkpAjxi7ouniRsco91qdmDDjpo7+2vv/6av/71r7z77rtkZ2fTq1evYmfV1qhRY+/tqlWrkpeXV+yxa9asWeo+5dWgQQOys7Pp0qULU6dO5eqrrwZg1qxZXHPNNcyfP59OnTqxe/fuiJ63MvKLui6eJGxyD7KP9aeffqJu3bocfPDBrF27llmzZkX8HJ07d+aFF14A4PPPPy/2k0FBp556KrNnz2bTpk3k5eWRmZlJ165d2bhxI6rKxRdfzLhx41i4cCG7d+8mNzeXHj16MHHiRL7//nu2F+1PcGXmte1dPEnYPncIro+1Q4cOpKam0rJlS5o1a0bnzp0jfo4//vGPXH755aSmpu79yu9SKU6TJk24++676datG6rKueeeS58+fVi4cCFXXnklqoqIMGHCBPLy8hgwYABbt25lz5493HLLLdStWzfiP0Nlk/+3GA+jZXzUjhNVDeTEaWlpmpWVVWjb0qVLadWqVSDxxJu8vDzy8vJISUnh66+/5qyzzuLrr7+mWrX4ej/231n8KTpMGOwThC+ikhxEZIGqppW2X3xlCrfXtm3bOPPMM8nLy0NVeeyxx+Iusbv45KN2HHhyj1v169dnwYIFQYfhEpCP2nGQwBdUnXPF81E7Djy5O5d04mnUTqTrP7nweXJ3LsnEQykG8HIMQfPk7lwSGjgQcnJgzx77HsSFVK+xHyxP7gV07959vwlJDz30ENdee22Jz6tTpw4Aa9asoV+/fsXu061bN4oO/SzqoYceKjSZ6Oyzz45I3Zc777yTSZMmVfg4zpWFX9gNlif3Avr3709mZmahbZmZmfTv3z+s5x911FHMnDmz3OcvmtzfeOMN6tevX+7jORekeLqwWxn7/uN2KOSNN0Kk16Bo1w5ClXaL1a9fP8aMGcPOnTupUaMGOTk5rFmzhi5durBt2zb69u3Ljz/+yK5du7jnnnvo27dvoefn5ORwzjnnsHjxYnbs2MGQIUP47LPPaNmyJTt27Ni737XXXsv8+fPZsWMH/fr146677mLKlCmsWbOG7t2707BhQ2bPnk3z5s3JysqiYcOGTJ48menTpwNw1VVXceONN5KTk0Pv3r05/fTT+fDDD2ncuDH/+te/qFWr1gF/xkWLFnHNNdewfft2jj32WKZPn84hhxzClClT+Pvf/061atVITU0lMzOT999/nxtuuAGwJfXmzJnjM1ld2OKpxn5lXF/XW+4FHHrooXTq1Ik333wTsFb7JZdcgoiQkpLCyy+/zMKFC5k9ezY333wzJc3uffTRR6lduzZLly7lrrvuKjRmffz48WRlZZGdnc37779PdnY2w4cP56ijjmL27NnMnj270LEWLFhAeno6H3/8MR999BGPP/44n376KWBFzIYNG8aSJUuoX78+L730Uok/4+WXX86ECRPIzs6mTZs23HXXXYCVMP7000/Jzs7m73//OwCTJk1i6tSpLFq0iLlz55b4puFcUfFyYbey9v3Hbcu9pBZ2NOV3zfTt25fMzEyefPJJwGqujxo1ijlz5lClShVWr17N+vXrOeKII4o9zpw5cxg+fDgAbdu2pW3btnsfe+GFF5g2bRp5eXmsXbuWL774otDjRc2bN48LLrhgb2XKCy+8kLlz53LeeefRokUL2rVrB5RcVhisvvzmzZvp2rUrAIMHD+biiy/eG+PAgQM5//zzOf/88wErXjZixAgGDhzIhRdeSJMmTcJ5CZ3bK+ga+1B5+/695V5E3759eeedd1i4cCHbt2+nY8eOAGRkZLBx40YWLFjAokWLOPzww4st81uab7/9lkmTJvHOO++QnZ1Nnz59ynWcfPnlgqFiJYP//e9/M2zYMBYuXMgpp5xCXl4eI0eO5IknnmDHjh107tyZL7/8stxxOheUytr378m9iDp16tC9e3d+//vfF7qQumXLFg477DCqV6/O7NmzWblyZYnHOeOMM3j22WcBWLx4MdnZ2YCVCz7ooIOoV68e69ev39sFBFC3bl22bt2637G6dOnCK6+8wvbt2/n55595+eWX6dKlS5l/tnr16nHIIYcwd+5cAP7xj3/QtWtX9uzZw3fffUf37t2ZMGECW7ZsYdu2bXzzzTe0adOG2267jVNOOcWTu0tI8TKpK9bj/uO2WyZI/fv354ILLig0cmbgwIGce+65tGnThrS0NFq2bFniMa699lqGDBlCq1ataNWq1d5PACeffDLt27enZcuWHH300YXKBQ8dOpRevXrt7XvP16FDB6644go6deoE2AXV9u3bl9gFcyBPP/303guqxxxzDOnp6ezevZtBgwaxZcsWVJXhw4dTv359br/9dmbPnk2VKlVo3br13lWlnEsk8VKKOdYF3bzkr6sQ/505F54qVazFXpSITTYLV7glf71bxjnnYiDWff+e3J1zLgZi3fcfd8k9qG4iV3b+u3IufLEe9x9XF1RTUlLYtGkTDRo0QESCDseVQFXZtGkTKSkpQYfiXMKI5bj/uEruTZo0ITc3l40bNwYdigtDSkqKT2xyLk7FVXKvXr06LVq0CDoM55xLeHHX5+6cc67iPLk751wS8uTunHNJKLAZqiKyESi5QEv8awh8H3QQccRfj338tSjMX4/CKvJ6NFPVRqXtFFhyTwYikhXONODKwl+Pffy1KMxfj8Ji8Xp4t4xzziUhT+7OOZeEPLlXzLSgA4gz/nrs469FYf56FBb118P73J1zLgl5y90555KQJ3fnnEtCntzLQUSOFpHZIvKFiCwRkRuCjiloIlJVRD4VkdeDjiVoIlJfRGaKyJcislRETgs6piCJyE2h/5PFIvKciFSaUqIiMl1ENojI4gLbDhWRt0Xk69D3Q6Jxbk/u5ZMH3KyqqcBvgGEikhpwTEG7AVgadBBx4q/AW6raEjiZSvy6iEhjYDiQpqonAVWBS4ONKqaeAnoV2TYSeEdVjwfeCd2POE/u5aCqa1V1Yej2Vuyft3GwUQVHRJoAfYAngo4laCJSDzgDeBJAVXeq6uZgowpcNaCWiFQDagNrAo4nZlR1DvBDkc19gadDt58Gzo/GuT25V5CINAfaAx8HG0mgHgL+BJRhmd+k1QLYCKSHuqmeEJGDgg4qKKq6GpgErALWAltU9T/BRhW4w1V1bej2OuDwaJzEk3sFiEgd4CXgRlX9Keh4giAi5wAbVHVB0LHEiWpAB+BRVW0P/EyUPnYnglB/cl/sTe8o4CARGRRsVPFDbSx6VMaje3IvJxGpjiX2DFX9Z9DxBKgzcJ6I5ACZQA8RmRFsSIHKBXJVNf+T3Ews2VdWPYFvVXWjqu4C/gn8T8AxBW29iBwJEPq+IRon8eReDmILvD4JLFXVyUHHEyRV/bOqNlHV5tiFsndVtdK2zFR1HfCdiJwY2nQm8EWAIQVtFfAbEakd+r85k0p8gTnkVWBw6PZg4F/ROIkn9/LpDFyGtVIXhb7ODjooFzf+CGSISDbQDrg34HgCE/oEMxNYCHyO5ZxKU4pARJ4D/g84UURyReRK4H7gtyLyNfbJ5v6onNvLDzjnXPLxlrtzziUhT+7OOZeEPLk751wS8uTunHNJyJO7c84lIU/uzjmXhDy5O+dcEvp/jZkwNKPmplsAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "acc = history.history['acc']\n",
    "val_acc = history.history['val_acc']\n",
    "loss = history.history['loss']\n",
    "val_loss = history.history['val_loss']\n",
    "\n",
    "epochs = range(1, len(acc) + 1)\n",
    "\n",
    "plt.plot(epochs, acc, 'bo', label='Training acc')\n",
    "plt.plot(epochs, val_acc, 'b', label='Validation acc')\n",
    "plt.title('Training and validation accuracy')\n",
    "plt.legend()\n",
    "\n",
    "plt.figure()\n",
    "\n",
    "plt.plot(epochs, loss, 'bo', label='Training loss')\n",
    "plt.plot(epochs, val_loss, 'b', label='Validation loss')\n",
    "plt.title('Training and validation loss')\n",
    "plt.legend()\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 6.2.5 정리\n",
    "\n",
    "• RNN이 무엇이고 동작하는 방법\n",
    "\n",
    "• LSTM이 무엇이고 긴 시퀀스에서 단순한 RNN보다 더 잘 작동하는 이유\n",
    "\n",
    "• 케라스의 RNN 층을 사용하여 시퀀스 데이터를 처리하는 방법"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "tensorflow",
   "language": "python",
   "name": "tensorflow"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}