mfnn-and-digit-recognition.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression and Multi-layer Feed-forward Neural Network for Digit Recognition\n",
    "\n",
    "This practical session is designed to implement Logistic Regression (LR) and Multi-layer Feed-forward Neural Network (MFNN) for handwritten digit recognition. [The MNIST database of handwritten digits](http://yann.lecun.com/exdb/mnist/) is used for training and testing LR and MFNN. \n",
    "\n",
    "## Dataset\n",
    "The [MNIST](http://yann.lecun.com/exdb/mnist/) dataset consists of handwritten grayscale digit images. All handwritten digit images have been size-normalized and centered in a fixed size image of 28 x 28 pixels. A grayscale image is nothing but a 2D array of integers between 0 and 255. In the original dataset each pixel of the image is represented by a value between 0 and 255, where 0 is black, 255 is white and anything in between is a different shade of grey. Below is some examples of [MNIST](http://yann.lecun.com/exdb/mnist/) handwritten digits:\n",
    "\n",
    "![MNIST](./images/mnist-samples.png)\n",
    "\n",
    "\n",
    "## References\n",
    "[1] http://yann.lecun.com/exdb/mnist/\n",
    "\n",
    "[2] https://pjreddie.com/projects/mnist-in-csv/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset Loading and Visualisation\n",
    "The following function loads train and test handwritten digit image samples from [MNIST](http://yann.lecun.com/exdb/mnist/) dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib.pyplot as pl\n",
    "\n",
    "import numpy as np"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "def load_dataset(train_dataset='mnist_train.csv', test_dataset='mnist_test.csv'):\n",
    "    \"\"\"\n",
    "    Loads and returns train and test datasets.\n",
    "    \"\"\"\n",
    "    data_train = np.genfromtxt(fname=train_dataset, delimiter=',')\n",
    "    y_train = data_train[:,0]\n",
    "    x_train = data_train[:,1:]\n",
    "    data_test = np.genfromtxt(fname=test_dataset, delimiter=',')\n",
    "    y_test = data_test[:,0]\n",
    "    x_test = data_test[:,1:]\n",
    "    return (x_train, y_train, x_test, y_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The following script calls function `load_dataset()` to train and test datasets. _Be patient as it may take some time to load datasets!_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "ename": "IOError",
     "evalue": "mnist_train.csv not found.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mIOError\u001b[0m                                   Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-6-181aa814c982>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[0;32m----> 1\u001b[0;31m \u001b[0mx_train\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my_train\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mx_test\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my_test\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mload_dataset\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m",
      "\u001b[0;32m<ipython-input-5-06ffe1e7c3e2>\u001b[0m in \u001b[0;36mload_dataset\u001b[0;34m(train_dataset, test_dataset)\u001b[0m\n\u001b[1;32m      3\u001b[0m     \u001b[0mLoads\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mreturns\u001b[0m \u001b[0mtrain\u001b[0m \u001b[0;32mand\u001b[0m \u001b[0mtest\u001b[0m \u001b[0mdatasets\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      4\u001b[0m     \"\"\"\n\u001b[0;32m----> 5\u001b[0;31m     \u001b[0mdata_train\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mgenfromtxt\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mtrain_dataset\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mdelimiter\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0;34m','\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      6\u001b[0m     \u001b[0my_train\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdata_train\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m0\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      7\u001b[0m     \u001b[0mx_train\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mdata_train\u001b[0m\u001b[0;34m[\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;36m1\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m]\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/lukhetho/anaconda2/lib/python2.7/site-packages/numpy/lib/npyio.pyc\u001b[0m in \u001b[0;36mgenfromtxt\u001b[0;34m(fname, dtype, comments, delimiter, skip_header, skip_footer, converters, missing_values, filling_values, usecols, names, excludelist, deletechars, replace_space, autostrip, case_sensitive, defaultfmt, unpack, usemask, loose, invalid_raise, max_rows, encoding)\u001b[0m\n\u001b[1;32m   1698\u001b[0m             \u001b[0mfname\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1699\u001b[0m         \u001b[0;32mif\u001b[0m \u001b[0misinstance\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mbasestring\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1700\u001b[0;31m             \u001b[0mfhd\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0miter\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlib\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_datasource\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mfname\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m'rt'\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mencoding\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mencoding\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1701\u001b[0m             \u001b[0mown_fhd\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTrue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1702\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/lukhetho/anaconda2/lib/python2.7/site-packages/numpy/lib/_datasource.pyc\u001b[0m in \u001b[0;36mopen\u001b[0;34m(path, mode, destpath, encoding, newline)\u001b[0m\n\u001b[1;32m    260\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    261\u001b[0m     \u001b[0mds\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mDataSource\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mdestpath\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 262\u001b[0;31m     \u001b[0;32mreturn\u001b[0m \u001b[0mds\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mopen\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpath\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mmode\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mencoding\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mencoding\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mnewline\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mnewline\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    263\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    264\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/home/lukhetho/anaconda2/lib/python2.7/site-packages/numpy/lib/_datasource.pyc\u001b[0m in \u001b[0;36mopen\u001b[0;34m(self, path, mode, encoding, newline)\u001b[0m\n\u001b[1;32m    616\u001b[0m                                       encoding=encoding, newline=newline)\n\u001b[1;32m    617\u001b[0m         \u001b[0;32melse\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 618\u001b[0;31m             \u001b[0;32mraise\u001b[0m \u001b[0mIOError\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"%s not found.\"\u001b[0m \u001b[0;34m%\u001b[0m \u001b[0mpath\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    619\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    620\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mIOError\u001b[0m: mnist_train.csv not found."
     ]
    }
   ],
   "source": [
    "x_train, y_train, x_test, y_test = load_dataset()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(60000, 784)"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "x_train.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Each row of `x_train` and `x_test` variables is a 1D vector comprised of 784 (28 * 28) integers and refers to an unpacked handwritten digit image. The following function takes a 1D vector array as input and returns the corresponding 2D representation as image."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def vector2image(image_vector, image_height=28, image_width=28):\n",
    "    \"\"\"\n",
    "    Returns image representation of the input 1D vector.\n",
    "    \"\"\"\n",
    "    image = np.reshape(image_vector, (image_height, image_width))\n",
    "    return image"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can use `vector2image()` function to convert a 1D feature vector to 2D image for visualisation purposes, e.g.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAEICAYAAACQ6CLfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEZdJREFUeJzt3X2wVPV9x/H3Bx9bNIIyKhIDxto21hriUNqp2KGVGEztoJ3qBG2KTVLMjNo6iVOMOhGmNWW0JiE+pMUqD/EhmvEx1lYdxsdSHdEaRYWgDCgPvWgJEawNwfvtH+dcZ73ePbvs7tmzl9/nNXPn3j2/8/DlDJ/9/c7D7lFEYGbpGVF1AWZWDYffLFEOv1miHH6zRDn8Zoly+M0S5fB3maRLJf1Lk/MulrRT0rom599P0g5Jv5T0920VWjJJ50h6uMl55+b/ph2SRja5zOv5vrulvUr3XA7/bpD0mKSvtLOOiPhWROzOOq6KiAk1NVwl6U1J70haL+mymnX/IiIOAG5tp8ZG8jeltt5cIuLWiDhlNxa5IyIOiIh38xoukrQ23w+bJH1H0t416z8a+FY7NQ5F0nGSHpL0tqRhfZOMw99Btf/5SnQT8JsR8THg94GzJf1pF7bbtC7thx8DJ+T74Tjg08Bfd2G7vwTuBL7chW2VKsnwSzpa0lZJJ+Svj8jfyacWLHMlcBJwXT78vC6fHpLOl7QGWJNPW1DTOz8n6aSa9cwdGIpKmpAvP0vSG3kNlw2x+Q9ExOqB3i/XD/xai/vhTEnPDZr2dUn3FiwzGzgH+Nt8P/w4n75O0hxJLwLvStpb0iX58Hu7pFcknVGznnMlPVXzOiR9VdIaST+TdL0k1asjIl6PiG0Di7ezH3ZHvv9vAl4ue1tlSzL8EfE6MAe4VdKvAouAxRHxWMEylwFPAhfkw88LappPB34XODZ//SwwETgYuA34kaT9C0qaAvwGcDLwTUmfKqo/D9UOYAMwMt9GK+4Hjhq0vT8HflBvgYhYSHZYcVW+H/6kpnkm8MfAqIjYBbxO9oZ5EDAPuEXS2IJ6TgN+h6wXPwv4XFHxks6W9A7wdr7MPxfNP2jZKZK2FfxMaXZdw1WS4QeIiBvJeupngLFAYY/bwD9ExNaIeC9f9y0R8T8RsSsirgH2Iwt3PfMi4r2I+AnwE7L/yEW1zwcOBE4gC+rPWyk6In4B3EEWeCT9FjABeKCV9QHfi4g3a/bDjyJiU0T0R8QdZPt7csHy8yNiW0S8ATxK9gZaVP9t+bD/14F/AvqaLTQinoqIUQU/TzVey/CWbPhzN5IdL16bB6FVb9a+yIfOr0r6uaRtZD3fmILl/7vm7/8FDmi0wcj8F/AeWa/aqiVk5w0EfBG4s419MXg//IWkFwZ6U7J93dH9ABARa8iG4TfsZr1JSzb8kg4Avkt2Am2upIObWKze2d0PpufH93PIhq2jI2IUWc9c9/i1TXsDR7e6cEQ8DewkG56fTcGQv3axRtMljSd7c70AOCTfDyvpkf0g6aT8nEW9n5Mar2V4Szb8wALgufyy27+SDRsb6QM+2WCeA4FdwFvA3pK+CXysnUIHSBoh6TxJo5WZDJwPLCtYZuCk4oSCVS8FrgN2NTncbWY/jCR7M3grr+MvyXr+jpD0FUmH5n8fC3yDgv0wWEQ8mZ+zqPfzZJ3tKj9/s2/+en9J+3Xgn9R1SYZf0gxgOvDVfNLXgBMkndNg0QXAn+Vno79XZ56HgH8DfgqsB/6PQcPhNp1BdiJtO3ALcG3+U8+ReR0bC+b5AVkwm+n1IRstHZsP54e8MhARrwDXAP9J9mbx28B/NLn+ZpwIvCTpXeDB/OfSDq6/nvFkh1oDZ/vfA1Z3YbsdJ3+ZR++SdCPZGfS+/KaVRvPvRxa0fcjOxs+TdDnwVkTUPRMu6VeALWTXzdd0pvrOyf8N3yC7xj5u0KXOesusBsaRncP4UsklDksOvyHpa8BpEfFHVddi3dONO7GGlfz6+VBOrXccOJwp+9yAyO5VqJ3+MtkQd7DzIqLU24etO9zzmyUqyRN+ZtblYb+G+aegzIaDiGjqXoq2en5J0yWtlvSapEvaWZeZdVfLx/yS9iK7lv1Zsg+YPAvMzK/v1lvGPb9ZybrR808GXouItRGxE/ghMKON9ZlZF7UT/nF8+M61Dfm0D5E0W9IKSSva2JaZdVg7J/yGGlp8ZFiff/57IXjYb9ZL2un5N5DdNz7g48Cm9soxs25pJ/zPAsdIOkrSvsAXyL4ZxsyGgZaH/RGxS9IFZJ9i2wu4OSKG/feamaWiq7f3+pjfrHxducnHzIYvh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiWr5Ed1mZbv88ssL2+fNm1fYPmJE/b5t6tSphcs+/vjjhe17grbCL2kdsB14H9gVEZM6UZSZla8TPf8fRsTbHViPmXWRj/nNEtVu+AN4WNJzkmYPNYOk2ZJWSFrR5rbMrIPaHfafGBGbJB0KPCJpVUQ8UTtDRCwEFgJIija3Z2Yd0lbPHxGb8t9bgHuAyZ0oyszK13L4JY2UdODA38ApwMpOFWZm5Wpn2H8YcI+kgfXcFhH/3pGqLAnnnntuYfucOXMK2/v7+1vedoSPQFsOf0SsBT7dwVrMrIt8qc8sUQ6/WaIcfrNEOfxmiXL4zRLlj/RaZcaPH1/Yvv/++3epkjS55zdLlMNvliiH3yxRDr9Zohx+s0Q5/GaJcvjNEuXr/FaqadOm1W278MIL21r3qlWrCttPO+20um19fX1tbXtP4J7fLFEOv1miHH6zRDn8Zoly+M0S5fCbJcrhN0uUr/NbW6ZMmVLYvmjRorptBx10UFvbvvrqqwvb169f39b693Tu+c0S5fCbJcrhN0uUw2+WKIffLFEOv1miHH6zRPk6v7Vl1qxZhe1HHHFEy+t+7LHHCtuXLl3a8rqtiZ5f0s2StkhaWTPtYEmPSFqT/x5dbplm1mnNDPsXA9MHTbsEWBYRxwDL8tdmNow0DH9EPAFsHTR5BrAk/3sJcHqH6zKzkrV6zH9YRGwGiIjNkg6tN6Ok2cDsFrdjZiUp/YRfRCwEFgJIirK3Z2bNafVSX5+ksQD57y2dK8nMuqHV8N8PDFzjmQXc15lyzKxbFFE8Epd0OzAVGAP0AVcA9wJ3Ap8A3gDOjIjBJwWHWpeH/cPMmDFjCtsbff99f39/3bZt27YVLnvWWWcVtj/66KOF7amKCDUzX8Nj/oiYWafp5N2qyMx6im/vNUuUw2+WKIffLFEOv1miHH6zRPkjvYmbMGFCYftdd91V2ravvfbawnZfyiuXe36zRDn8Zoly+M0S5fCbJcrhN0uUw2+WKIffLFG+zp+46dMHfzfrhx1//PFtrX/ZsmV12xYsWNDWuq097vnNEuXwmyXK4TdLlMNvliiH3yxRDr9Zohx+s0Q1/Orujm7MX93ddaefXvwYxcWLFxe2jxw5srB9+fLlhe1FX7/d6Gu/rTXNfnW3e36zRDn8Zoly+M0S5fCbJcrhN0uUw2+WKIffLFH+PP8eoOi798v83n2AtWvXFrb7Wn7vatjzS7pZ0hZJK2umzZW0UdIL+c/nyy3TzDqtmWH/YmCor3v5TkRMzH8e7GxZZla2huGPiCeArV2oxcy6qJ0TfhdIejE/LBhdbyZJsyWtkLSijW2ZWYe1Gv7vA0cDE4HNwDX1ZoyIhRExKSImtbgtMytBS+GPiL6IeD8i+oEbgcmdLcvMytZS+CWNrXl5BrCy3rxm1psaXueXdDswFRgjaQNwBTBV0kQggHXAeSXWaA3MmTOnblt/f3+p254/f36p67fyNAx/RMwcYvJNJdRiZl3k23vNEuXwmyXK4TdLlMNvliiH3yxR/kjvMDBx4sTC9lNOOaW0bd93332F7atXry5t21Yu9/xmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaL8iO5hYMuWLYXto0fX/Ra1hp5++unC9lNPPbWwfceOHS1v28rhR3SbWSGH3yxRDr9Zohx+s0Q5/GaJcvjNEuXwmyXKn+cfBg455JDC9na+nvuGG24obPd1/D2Xe36zRDn8Zoly+M0S5fCbJcrhN0uUw2+WKIffLFHNPKL7SGApcDjQDyyMiAWSDgbuACaQPab7rIj4WXml7rkWLVpU2D5iRHnv0cuXLy9t3dbbmvlftQv4ekR8Cvg94HxJxwKXAMsi4hhgWf7azIaJhuGPiM0R8Xz+93bgVWAcMANYks+2BDi9rCLNrPN2azwpaQLwGeAZ4LCI2AzZGwRwaKeLM7PyNH1vv6QDgLuAiyLiHamprwlD0mxgdmvlmVlZmur5Je1DFvxbI+LufHKfpLF5+1hgyG+ZjIiFETEpIiZ1omAz64yG4VfWxd8EvBoR365puh+Ylf89Cyh+nKuZ9ZRmhv0nAl8EXpL0Qj7tUmA+cKekLwNvAGeWU+Lw1+gR29OmTStsb/SR3Z07d9Ztu/766wuX7evrK2y3PVfD8EfEU0C9A/yTO1uOmXWL7/AzS5TDb5Yoh98sUQ6/WaIcfrNEOfxmifJXd3fBqFGjCtsPP/zwtta/cePGum0XX3xxW+u2PZd7frNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUf48fxesWrWqsL3RY7KnTJnSyXLMAPf8Zsly+M0S5fCbJcrhN0uUw2+WKIffLFEOv1miFBHFM0hHAkuBw4F+YGFELJA0F/gr4K181ksj4sEG6yremJm1LSLUzHzNhH8sMDYinpd0IPAccDpwFrAjIv6x2aIcfrPyNRv+hnf4RcRmYHP+93ZJrwLj2ivPzKq2W8f8kiYAnwGeySddIOlFSTdLGl1nmdmSVkha0ValZtZRDYf9H8woHQA8DlwZEXdLOgx4Gwjg78gODb7UYB0e9puVrGPH/ACS9gEeAB6KiG8P0T4BeCAijmuwHoffrGTNhr/hsF+SgJuAV2uDn58IHHAGsHJ3izSz6jRztn8K8CTwEtmlPoBLgZnARLJh/zrgvPzkYNG63POblayjw/5OcfjNytexYb+Z7ZkcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S1S3H9H9NrC+5vWYfFov6tXaerUucG2t6mRt45udsauf5//IxqUVETGpsgIK9GptvVoXuLZWVVWbh/1miXL4zRJVdfgXVrz9Ir1aW6/WBa6tVZXUVukxv5lVp+qe38wq4vCbJaqS8EuaLmm1pNckXVJFDfVIWifpJUkvVP18wfwZiFskrayZdrCkRyStyX8P+YzEimqbK2ljvu9ekPT5imo7UtKjkl6V9LKkv8mnV7rvCuqqZL91/Zhf0l7AT4HPAhuAZ4GZEfFKVwupQ9I6YFJEVH5DiKQ/AHYASwcehSbpKmBrRMzP3zhHR8ScHqltLrv52PaSaqv3WPlzqXDfdfJx951QRc8/GXgtItZGxE7gh8CMCuroeRHxBLB10OQZwJL87yVk/3m6rk5tPSEiNkfE8/nf24GBx8pXuu8K6qpEFeEfB7xZ83oDFe6AIQTwsKTnJM2uupghHDbwWLT896EV1zNYw8e2d9Ogx8r3zL5r5XH3nVZF+Id6lFAvXW88MSJOAE4Fzs+Ht9ac7wNHkz3DcTNwTZXF5I+Vvwu4KCLeqbKWWkPUVcl+qyL8G4Aja15/HNhUQR1DiohN+e8twD1khym9pG/gCcn57y0V1/OBiOiLiPcjoh+4kQr3Xf5Y+buAWyPi7nxy5ftuqLqq2m9VhP9Z4BhJR0naF/gCcH8FdXyEpJH5iRgkjQROofcePX4/MCv/exZwX4W1fEivPLa93mPlqXjf9drj7iu5wy+/lPFdYC/g5oi4sutFDEHSJ8l6e8g+7nxblbVJuh2YSvaRzz7gCuBe4E7gE8AbwJkR0fUTb3Vqm8puPra9pNrqPVb+GSrcd5183H1H6vHtvWZp8h1+Zoly+M0S5fCbJcrhN0uUw2+WKIffLFEOv1mi/h+Wo7CtauJAbAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f532fc5a650>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "index = 3\n",
    "image = vector2image(x_train[index])\n",
    "pl.figure()\n",
    "pl.imshow(image, cmap='gray')\n",
    "pl.title('x_train[' + str(index) + '], ' + 'y_train[' + str(index) + '] = ' + str(int(y_train[index])))\n",
    "pl.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You may also need the following function to convert a 2D grayscale image to 1D vector representation."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "def image2vector(image):\n",
    "    \"\"\"\n",
    "    Returns vector representation of the input 2D image.\n",
    "    \"\"\"\n",
    "    image_height, image_width = image.shape\n",
    "    image_vector = np.reshape(image, image_height*image_width)\n",
    "    return image_vector"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Thus, `vector2image(image2vector(vector2image(x_train[index])))` must produce exactly same output with that of `vector2image(x_train[index])`, i.e.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAEICAYAAACQ6CLfAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAEZdJREFUeJzt3X2wVPV9x/H3Bx9bNIIyKhIDxto21hriUNqp2KGVGEztoJ3qBG2KTVLMjNo6iVOMOhGmNWW0JiE+pMUqD/EhmvEx1lYdxsdSHdEaRYWgDCgPvWgJEawNwfvtH+dcZ73ePbvs7tmzl9/nNXPn3j2/8/DlDJ/9/c7D7lFEYGbpGVF1AWZWDYffLFEOv1miHH6zRDn8Zoly+M0S5fB3maRLJf1Lk/MulrRT0rom599P0g5Jv5T0920VWjJJ50h6uMl55+b/ph2SRja5zOv5vrulvUr3XA7/bpD0mKSvtLOOiPhWROzOOq6KiAk1NVwl6U1J70haL+mymnX/IiIOAG5tp8ZG8jeltt5cIuLWiDhlNxa5IyIOiIh38xoukrQ23w+bJH1H0t416z8a+FY7NQ5F0nGSHpL0tqRhfZOMw99Btf/5SnQT8JsR8THg94GzJf1pF7bbtC7thx8DJ+T74Tjg08Bfd2G7vwTuBL7chW2VKsnwSzpa0lZJJ+Svj8jfyacWLHMlcBJwXT78vC6fHpLOl7QGWJNPW1DTOz8n6aSa9cwdGIpKmpAvP0vSG3kNlw2x+Q9ExOqB3i/XD/xai/vhTEnPDZr2dUn3FiwzGzgH+Nt8P/w4n75O0hxJLwLvStpb0iX58Hu7pFcknVGznnMlPVXzOiR9VdIaST+TdL0k1asjIl6PiG0Di7ezH3ZHvv9vAl4ue1tlSzL8EfE6MAe4VdKvAouAxRHxWMEylwFPAhfkw88LappPB34XODZ//SwwETgYuA34kaT9C0qaAvwGcDLwTUmfKqo/D9UOYAMwMt9GK+4Hjhq0vT8HflBvgYhYSHZYcVW+H/6kpnkm8MfAqIjYBbxO9oZ5EDAPuEXS2IJ6TgN+h6wXPwv4XFHxks6W9A7wdr7MPxfNP2jZKZK2FfxMaXZdw1WS4QeIiBvJeupngLFAYY/bwD9ExNaIeC9f9y0R8T8RsSsirgH2Iwt3PfMi4r2I+AnwE7L/yEW1zwcOBE4gC+rPWyk6In4B3EEWeCT9FjABeKCV9QHfi4g3a/bDjyJiU0T0R8QdZPt7csHy8yNiW0S8ATxK9gZaVP9t+bD/14F/AvqaLTQinoqIUQU/TzVey/CWbPhzN5IdL16bB6FVb9a+yIfOr0r6uaRtZD3fmILl/7vm7/8FDmi0wcj8F/AeWa/aqiVk5w0EfBG4s419MXg//IWkFwZ6U7J93dH9ABARa8iG4TfsZr1JSzb8kg4Avkt2Am2upIObWKze2d0PpufH93PIhq2jI2IUWc9c9/i1TXsDR7e6cEQ8DewkG56fTcGQv3axRtMljSd7c70AOCTfDyvpkf0g6aT8nEW9n5Mar2V4Szb8wALgufyy27+SDRsb6QM+2WCeA4FdwFvA3pK+CXysnUIHSBoh6TxJo5WZDJwPLCtYZuCk4oSCVS8FrgN2NTncbWY/jCR7M3grr+MvyXr+jpD0FUmH5n8fC3yDgv0wWEQ8mZ+zqPfzZJ3tKj9/s2/+en9J+3Xgn9R1SYZf0gxgOvDVfNLXgBMkndNg0QXAn+Vno79XZ56HgH8DfgqsB/6PQcPhNp1BdiJtO3ALcG3+U8+ReR0bC+b5AVkwm+n1IRstHZsP54e8MhARrwDXAP9J9mbx28B/NLn+ZpwIvCTpXeDB/OfSDq6/nvFkh1oDZ/vfA1Z3YbsdJ3+ZR++SdCPZGfS+/KaVRvPvRxa0fcjOxs+TdDnwVkTUPRMu6VeALWTXzdd0pvrOyf8N3yC7xj5u0KXOesusBsaRncP4UsklDksOvyHpa8BpEfFHVddi3dONO7GGlfz6+VBOrXccOJwp+9yAyO5VqJ3+MtkQd7DzIqLU24etO9zzmyUqyRN+ZtblYb+G+aegzIaDiGjqXoq2en5J0yWtlvSapEvaWZeZdVfLx/yS9iK7lv1Zsg+YPAvMzK/v1lvGPb9ZybrR808GXouItRGxE/ghMKON9ZlZF7UT/nF8+M61Dfm0D5E0W9IKSSva2JaZdVg7J/yGGlp8ZFiff/57IXjYb9ZL2un5N5DdNz7g48Cm9soxs25pJ/zPAsdIOkrSvsAXyL4ZxsyGgZaH/RGxS9IFZJ9i2wu4OSKG/feamaWiq7f3+pjfrHxducnHzIYvh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiWr5Ed1mZbv88ssL2+fNm1fYPmJE/b5t6tSphcs+/vjjhe17grbCL2kdsB14H9gVEZM6UZSZla8TPf8fRsTbHViPmXWRj/nNEtVu+AN4WNJzkmYPNYOk2ZJWSFrR5rbMrIPaHfafGBGbJB0KPCJpVUQ8UTtDRCwEFgJIija3Z2Yd0lbPHxGb8t9bgHuAyZ0oyszK13L4JY2UdODA38ApwMpOFWZm5Wpn2H8YcI+kgfXcFhH/3pGqLAnnnntuYfucOXMK2/v7+1vedoSPQFsOf0SsBT7dwVrMrIt8qc8sUQ6/WaIcfrNEOfxmiXL4zRLlj/RaZcaPH1/Yvv/++3epkjS55zdLlMNvliiH3yxRDr9Zohx+s0Q5/GaJcvjNEuXr/FaqadOm1W278MIL21r3qlWrCttPO+20um19fX1tbXtP4J7fLFEOv1miHH6zRDn8Zoly+M0S5fCbJcrhN0uUr/NbW6ZMmVLYvmjRorptBx10UFvbvvrqqwvb169f39b693Tu+c0S5fCbJcrhN0uUw2+WKIffLFEOv1miHH6zRPk6v7Vl1qxZhe1HHHFEy+t+7LHHCtuXLl3a8rqtiZ5f0s2StkhaWTPtYEmPSFqT/x5dbplm1mnNDPsXA9MHTbsEWBYRxwDL8tdmNow0DH9EPAFsHTR5BrAk/3sJcHqH6zKzkrV6zH9YRGwGiIjNkg6tN6Ok2cDsFrdjZiUp/YRfRCwEFgJIirK3Z2bNafVSX5+ksQD57y2dK8nMuqHV8N8PDFzjmQXc15lyzKxbFFE8Epd0OzAVGAP0AVcA9wJ3Ap8A3gDOjIjBJwWHWpeH/cPMmDFjCtsbff99f39/3bZt27YVLnvWWWcVtj/66KOF7amKCDUzX8Nj/oiYWafp5N2qyMx6im/vNUuUw2+WKIffLFEOv1miHH6zRPkjvYmbMGFCYftdd91V2ravvfbawnZfyiuXe36zRDn8Zoly+M0S5fCbJcrhN0uUw2+WKIffLFG+zp+46dMHfzfrhx1//PFtrX/ZsmV12xYsWNDWuq097vnNEuXwmyXK4TdLlMNvliiH3yxRDr9Zohx+s0Q1/Orujm7MX93ddaefXvwYxcWLFxe2jxw5srB9+fLlhe1FX7/d6Gu/rTXNfnW3e36zRDn8Zoly+M0S5fCbJcrhN0uUw2+WKIffLFH+PP8eoOi798v83n2AtWvXFrb7Wn7vatjzS7pZ0hZJK2umzZW0UdIL+c/nyy3TzDqtmWH/YmCor3v5TkRMzH8e7GxZZla2huGPiCeArV2oxcy6qJ0TfhdIejE/LBhdbyZJsyWtkLSijW2ZWYe1Gv7vA0cDE4HNwDX1ZoyIhRExKSImtbgtMytBS+GPiL6IeD8i+oEbgcmdLcvMytZS+CWNrXl5BrCy3rxm1psaXueXdDswFRgjaQNwBTBV0kQggHXAeSXWaA3MmTOnblt/f3+p254/f36p67fyNAx/RMwcYvJNJdRiZl3k23vNEuXwmyXK4TdLlMNvliiH3yxR/kjvMDBx4sTC9lNOOaW0bd93332F7atXry5t21Yu9/xmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaL8iO5hYMuWLYXto0fX/Ra1hp5++unC9lNPPbWwfceOHS1v28rhR3SbWSGH3yxRDr9Zohx+s0Q5/GaJcvjNEuXwmyXKn+cfBg455JDC9na+nvuGG24obPd1/D2Xe36zRDn8Zoly+M0S5fCbJcrhN0uUw2+WKIffLFHNPKL7SGApcDjQDyyMiAWSDgbuACaQPab7rIj4WXml7rkWLVpU2D5iRHnv0cuXLy9t3dbbmvlftQv4ekR8Cvg94HxJxwKXAMsi4hhgWf7azIaJhuGPiM0R8Xz+93bgVWAcMANYks+2BDi9rCLNrPN2azwpaQLwGeAZ4LCI2AzZGwRwaKeLM7PyNH1vv6QDgLuAiyLiHamprwlD0mxgdmvlmVlZmur5Je1DFvxbI+LufHKfpLF5+1hgyG+ZjIiFETEpIiZ1omAz64yG4VfWxd8EvBoR365puh+Ylf89Cyh+nKuZ9ZRmhv0nAl8EXpL0Qj7tUmA+cKekLwNvAGeWU+Lw1+gR29OmTStsb/SR3Z07d9Ztu/766wuX7evrK2y3PVfD8EfEU0C9A/yTO1uOmXWL7/AzS5TDb5Yoh98sUQ6/WaIcfrNEOfxmifJXd3fBqFGjCtsPP/zwtta/cePGum0XX3xxW+u2PZd7frNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUf48fxesWrWqsL3RY7KnTJnSyXLMAPf8Zsly+M0S5fCbJcrhN0uUw2+WKIffLFEOv1miFBHFM0hHAkuBw4F+YGFELJA0F/gr4K181ksj4sEG6yremJm1LSLUzHzNhH8sMDYinpd0IPAccDpwFrAjIv6x2aIcfrPyNRv+hnf4RcRmYHP+93ZJrwLj2ivPzKq2W8f8kiYAnwGeySddIOlFSTdLGl1nmdmSVkha0ValZtZRDYf9H8woHQA8DlwZEXdLOgx4Gwjg78gODb7UYB0e9puVrGPH/ACS9gEeAB6KiG8P0T4BeCAijmuwHoffrGTNhr/hsF+SgJuAV2uDn58IHHAGsHJ3izSz6jRztn8K8CTwEtmlPoBLgZnARLJh/zrgvPzkYNG63POblayjw/5OcfjNytexYb+Z7ZkcfrNEOfxmiXL4zRLl8JslyuE3S5TDb5Yoh98sUQ6/WaIcfrNEOfxmiXL4zRLl8JslyuE3S1S3H9H9NrC+5vWYfFov6tXaerUucG2t6mRt45udsauf5//IxqUVETGpsgIK9GptvVoXuLZWVVWbh/1miXL4zRJVdfgXVrz9Ir1aW6/WBa6tVZXUVukxv5lVp+qe38wq4vCbJaqS8EuaLmm1pNckXVJFDfVIWifpJUkvVP18wfwZiFskrayZdrCkRyStyX8P+YzEimqbK2ljvu9ekPT5imo7UtKjkl6V9LKkv8mnV7rvCuqqZL91/Zhf0l7AT4HPAhuAZ4GZEfFKVwupQ9I6YFJEVH5DiKQ/AHYASwcehSbpKmBrRMzP3zhHR8ScHqltLrv52PaSaqv3WPlzqXDfdfJx951QRc8/GXgtItZGxE7gh8CMCuroeRHxBLB10OQZwJL87yVk/3m6rk5tPSEiNkfE8/nf24GBx8pXuu8K6qpEFeEfB7xZ83oDFe6AIQTwsKTnJM2uupghHDbwWLT896EV1zNYw8e2d9Ogx8r3zL5r5XH3nVZF+Id6lFAvXW88MSJOAE4Fzs+Ht9ac7wNHkz3DcTNwTZXF5I+Vvwu4KCLeqbKWWkPUVcl+qyL8G4Aja15/HNhUQR1DiohN+e8twD1khym9pG/gCcn57y0V1/OBiOiLiPcjoh+4kQr3Xf5Y+buAWyPi7nxy5ftuqLqq2m9VhP9Z4BhJR0naF/gCcH8FdXyEpJH5iRgkjQROofcePX4/MCv/exZwX4W1fEivPLa93mPlqXjf9drj7iu5wy+/lPFdYC/g5oi4sutFDEHSJ8l6e8g+7nxblbVJuh2YSvaRzz7gCuBe4E7gE8AbwJkR0fUTb3Vqm8puPra9pNrqPVb+GSrcd5183H1H6vHtvWZp8h1+Zoly+M0S5fCbJcrhN0uUw2+WKIffLFEOv1mi/h+Wo7CtauJAbAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f532fc5a310>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "    image = vector2image(image2vector(vector2image(x_train[index])))\n",
    "    pl.figure()\n",
    "    pl.imshow(image, cmap='gray')\n",
    "    pl.title('x_train[' + str(index) + '], ' + 'y_train[' + str(index) + '] = ' + str(int(y_train[index])))\n",
    "    pl.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, let's produce an image that is similar to what you see on the very first markdown cell by selecting samples from `x_train` and `y_train`!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD0CAYAAACVbe2MAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMS4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvNQv5yAAAIABJREFUeJzsvXdQFem2PrxQARExICggKL+RoxyGqxyHO3qVi1ozo3KNlJHCRDkyUkZK1LFMOBzHiKh8omPAQJnzcHV0RkGlzFmOoKKicCVH9yUou/v5/sDuuzc7deI4evqpegr27u61305Pv73e9a5lBYBUqFChQsXniyYfuwEqVKhQoaJxoQq9ChUqVHzmUIVehQoVKj5zqEKvQoUKFZ85VKFXoUKFis8cqtCrUKFCxWeORhN6KyurwVZWVk+trKyeW1lZ/dhYv6NChQoVKszDqjHi6K2srJoS0TMi+o6I/oeIbhNRCIAMxX9MhQoVKlSYRWP16L8moucAXgJ4T0SHiGhEI/2WChUqVKgwg8YS+o5ElKvz+X8+fKdChQoVKv7JaNZIdq2MfKfnI7KysgonovAPH79qpHaoUKFCxeeMEgDOllZqLKH/HyLy0PnsTkR5uisA2E5E24mIrKys1IQ7KlSoUCEer4Ws1Fium9tE9BcrK6v/Z2VlZUNE44noVyUMf/XVV7R7927avXs39ezZUwmTPBiGIQDUr18/Re0qhaCgIGIYhoKCgsjZ2eJDXIUJHDlyhACQmtBPHpo3b063bt0ilmUJAHl5eVGTJspJCmeXZVlZdnr16qVQi/55SEpKIpZlqbCwkOzt7eUb5C54pUlE/0X1kTcviGixhXUhhLm5uWAYBlqtFoWFhdBqtejdu7egbc1xypQpuHjxIurq6sAwDOrq6tC5c2fJ9pKTk3k7HDdt2iSrjQcOHEBubi5vb9GiRZJteXl54eXLl2BZFgzD6DE9PR1ff/21YFuOjo44cOAACgsLeRslJSV4+PAhHj58iGHDhsna74iICJw6dUrvWEZERIi2ExsbCw4eHh4gIowdOxYAEBsbK6uNL168gFarxfv37/H+/Xv+fycnJ1l2Bw4ciLy8PL3zs23bNtja2kqyZ2Njg9TUVINzzjAMAgICBNtp3bo1f42np6cjNDQUz549A8MwSE5OxqxZsyTvs7u7O65evWrQvhkzZqBVq1aC7SxevBg1NTV6NtLS0nDq1CmcPHkSJ0+ehJ+fn6zz06JFCwwbNgwMwyAlJQXffPONLHscQ0NDUVxcDD8/P3h6eqKurg4TJkwwtf4dQXrcWEIv8qFg8YAGBARAq9XyQj969Gj+s9QD2qZNG2RnZ0Oj0aCurg7p6el4/PixLKH39PREZWWlgdDLaScR4fnz5wY2R4wYIdpOq1atcOPGDTAMY1ToGYZBVFSUYHujR4822L6h3ebNm0va5z59+hg9lhqNBtbW1oLteHh4AACuXbtm0DHgkJOTI6mNgYGBeuKu+79cIamurjZ6fqQ86IgIKSkpenYqKip44Rcj9BEREbyNLl26YOfOnXp2379/L3mfr1y5YnSfGYaBv7+/YDs1NTWoqanBqVOnEB4ejvnz50Oj0eDSpUsoKCgAwzCoqqrCtm3bYGNjI7qdw4YNw+XLl/U0qaysDIMGDYKbm5ukfff09MScOXOQkZGhd+08efIEjx49MrXd5yP0SUlJ0Gq1egeV64VLFdCRI0fizJkzeiIyadIkTJkyRZbQe3l56b0ZKCX0+fn5BjYLCgpE2dizZw9KSkoMBLm6uhpv3ryRJPRlZWUWhX7hwoWi97dfv37Iz883eSwjIyMF2eFE3tT6R44cQWRkpKSevbe3N168eGFS6Fu0aCH5fDs7Oxv0SBmGwYMHD+Di4iLaXuvWrQ1s3bhxA0QkWujT09N5G4sWLQLDMCgsLERoaCiqqqpEnZ+G5B7sHCsrK/H777/zPXIhNjp06ACGYbB27Vq977OyskBEmDx5MhYsWIC8vDywLCvYLlF9p3P48OG8HulqEsfc3FxER0eL6owQEZ4+fQqWZcGyrN7348aNQ2lpKb788ktj230eQl9WVsYfwJYtW2LIkCF6bguGYdCzZ09RB7SkpITvGRYWFvICEhwcLFnog4KC9C7Qn376CUSE2bNng2EYoH5HRXPjxo16Anfo0CEcPnyY/yzUPeDq6qrXvpiYGISFheHw4cOYOHEipk6dKlroPT099WzW1tbi559/BhFh1KhRKC0t5ZeJee1eunSpnl0ABp9zc3Ph4+Nj1k7v3r0BAEeOHBH0u5zgX7t2TdD6CQkJSElJ0RP3w4cP4+jRo7zoBwcHSzrvug/QFy9e6LkfxNpasmSJgchzy2JiYkQJvZWVFW/Dw8MDo0ePRm5uLrp168avM3jwYDAMg8zMTN5NJoTr1683aKPudSt033fu3GnO1aHHgIAAFBcXIzw83Ox6zZo10xNzXZ48eZLvxW/atAnFxcX8sunTpwtqx9GjR3Hu3DncvXsXJ0+eNLo8NzfX2LafvtB37dqV7y08ePDA6DoMw2D//v2CL6aRI0fywnnhwgVe2Ovq6kBEsoRet9fJfT9r1izJPfqQkBD+oZSZmYmkpCS0bt0aK1as4G126dJFkK179+7xPePCwkLY2dmBqN7H3rRpU7Ru3RpHjx4VJfRJSUl6N2Z0dLTe8t69e2PhwoVgGAaTJk0SvN/GevDGPm/YsMGsnZycHNEP2CNHjgjepnPnzkhNTeWF/uLFi3BycsKxY8d4oR85cqTo884dA47e3t44ffq0ZKHv2rUrnj59CoZhoNFoMGfOHH6ZWKH38vICwzC4ffs2HBwcLLZf6EOWiPD48WN+u1WrVoGI0LRpU0RHR4va91evXmH48OGCf3f+/PlIS0uDo6OjyXUePHhgIPArV6406p4bNGgQSkpKoNVqkZeXZ/H33d3dUVRUBAcHB1y+fBm7du0yWGfJkiUoKysztv2nLfS2trb49ddfUVFRgUGDBqFdu3YmLyihF8CUKVNQUVEBjUaDCxcuwMfHB1OmTMGdO3f4AaSWLVvi6tWrooX+1q1bvBC9fv2a/15XlMXY69Gjh57A6S7TFTwhQt+rVy9UVFTwD01j63Tu3Jl/RRYi9L1790ZtbS1/Yy5btgzNmjUzeY5qa2sFD5zr7reuj37NmjXo0qULwsLCUFdXh5KSErMCBUjzvQsV+sePH/OCnp6ezvdedd04Yn+7d+/evM/74sWLOHv2LIjqB44rKirw6NEjuLu7i7Kp26PX7RS1b98eZ8+elST0SUlJZte7du0aGIbBuXPnBNkdNmyY3luM7oBzVFSUqPucZVlRQm9nZweGYRAUFGT2mjxy5Agv8sePH0f79u1Nrh8eHs6va+n3169fz7trVqxYYVLoG7p0PvDTFnruQhFyUwq5APz8/FBUVCTIZmpqqmih1xUnXXeKFB+9vb09EhISAADp6elwdXXVW56QkMC7MBouM8ZTp07x7hquJ9+QYl03Dx8+BMMw+PXXXwUdG86/bG69jh078v7furo6jB07FkT1roBBgwbx63FvT8ePHzfrupIq9NzvmmNwcLCeu0Z3WUxMDC/0YqKjFi1axPt8mzZtarC8e/fuYBhxg7Ft2rTBoUOHwDAMfH199Zb5+/sb9WWbI9e77tGjh9n1/Pz8DNxEpqjrAuQebMauIaFCf+XKFUybNk3UOWdZFjt27DC5fODAgbxwC/EgcNfHlStXzK4XGBgIAOYGW0FUf89/dkI/dOhQVFdXC3oaMgyD+Ph4i+u9ePHCaO+4IaX26IF6P/KTJ08M2mfuAjbG9evX8w+HGTNmGCzn9iM9Pd2irYULF/IDPEOGDDG7bmJiIliWxfz58y3aTU9PB8uySE5OtrjuqlWrAAAPHz40u15ISIieT97U24rueMjgwYNN2rt27Zrg3rkuhQh9fn6+yTBKqT167iFXW1trdLmPj49ood+9e7dJwZUi9Nu3b1dc6MPCwvh1TQ00ixH6uXPnIj093WSnxhjr6ur4wWlj5K7Juro6fPfdd2ZtdevWjQ+eWL58ucVrjWVZ3Lx50+Q67u7uKC0tRXV1tbHlgoT+T5mP3s7OjmxsbKioqMjkOra2trRq1SoiIlq0aJFFm506dRL026NHj6avv/5aWEM/oF+/fsSyLD/BQ9cW9/3WrVsF29OdCObs7EzW1tZG15syZYpFW7on+8yZM2bX5dqvuw+W1j1w4ICgNjQ8Ng0xevRoio+P548Xy7L0ww8/mP19SzaPHj1KRES9e/e22EaxKCkp4f93cnLSWzZx4kT+/6SkJME2fXx8qLS0lIYPHy6/gQ3akpeXZ3KdrKwswfbu3LlDRET+/v7yGqaD77//XjFbRETnzp0jHx8fUcexqqqK/v3f/93kcpZlSavV0t///nf6448/TK5nY2NDM2fOJGdnZ7p8+TL9/PPPZn/Xx8eHiIiuXbtm1qaNjQ2VlpZa2Asz+Ni9eWM9+jFjxkCr1ZqcZOTn58f7yYwtN0bdEEpT63ADVvn5+YLtEtUPuOr6kYnqX7W439R1Owjh06dPUVdXh5s3b6JTp07897oDvkJtvnz50mLPytbWFlFRUaiurkZycrKgnhDnusnIyLA4TiDEdaM72Orj44MNGzZgw4YN8Pb21jvO3JsOd5wtUczgKkch7h5uAHbLli0Gy6TE0c+YMYMfeDW23N/fH7m5uaIGYx0cHMAwDDZs2GAQ6ufp6clPchNzbDg3y8mTJ02Oyeie8507d1q0qTtBquEyZ2dn5ObmIj8/Hx07dhTczsuXL4NhGEGDoVx7zbluGIZBdnY22rRpY3IdV1dXvcFaIfNHOnXqxI+9mFrn/v37YFkW/fv3N7b803XdjB07FgzD6A1qcoyMjERZWRn27dsn6gI1FlrGsWXLljhx4gQYhkFWVpZeqJgQ6gp9ly5d8NNPP/GfLfmRjfHAgQO8C2PLli28q4J7fYyLixNsa8GCBRaFnhvsYhgGJ06cEGR3//79/DaZmZnw8vKyeOzNDeBxIm/u4Z2ens4LvdAYdQ8PD+Tk5AiO/uDi7i2tx82GNXaDKi30PXr04GeF9+nTR7DNwMBAMMz/hfrq8vbt24JdK6bOp6nQyU6dOoFhGFy4cMFkEIUuTQl9TEwM4uLiwDAM+vbtK6qNnTt3RlZWFliWxbJly0xG1LRq1QpbtmyBRqMxGMPQ5cGDB6HVanHmzBmjtiIiIpCamgqtVovi4mJs3bpVcFufPHlitnNZW1uLI0eOmOqAfbpCz/Xo3717Bz8/P3h4eGDMmDF4/fo1/2QVm/qAC1Osq6tDSkoK5syZg8GDByM1NRVXr17l0wqIFXmi+lh5ToQ5fyP3WawtIn0fvbGwQjGTW4QIve4MTKHTuB0dHXHs2DF+u4yMDKNT87mY6qNHj6Jt27Ym7XF2jD3EnJ2d+QFlKceUE29LE6J69+6NnJwcQT36W7du8YKuK8663ysh9NbW1vxbmdg3TU7o4+Pj4eXlxXPr1q1gmPqZoZZ8yObOlTHxDQgI4N+MBw4cKMiertAHBQWhT58+vA2p55y7bhITE8EwDHJycrBv3z5MmTIFTk5OcHJywoIFC/Dq1SswDIMxY8aYteXl5cX31B89eoQpU6YgKCgIP/30E65cuYLc3Fxe5BuGGVtiTEyM0YHWtm3bYsWKFXj9+rW5SXKfrtC7u7vj+vXrBrPO5OaL6dixIx/uyPHNmzc4dOgQWrduLdnuxIkTDUT5yZMnmDt3ruQLlJv1q9vT/eGHH0TbCg4O1pv1qktuBmtFRYXRQV8hPH78uJ7NqqoqVFZWory8nH+AtGrVCh8ylJqk7oP42bNnePr0KU/uFVxu7pixY8fyA7TGEBsbK3iCz6JFi/QmRhmbGSu2fcbOEcMwKCsrE9WTF2KzuLgYAwYMkGSTe1AwDINp06Zh0aJFyMrK4r8zN7BojFu2bDHZzrNnz8q6N4nq7/v58+fj3r17ePPmDR4/fozHjx/j1KlTmD9/Pjw9PQXZ8fb2xokTJwzi6RmGwdWrV0UNajfkN998wwdNcKyrq8Ply5ctbfvpCj1Rvb+LC+XSarWyE0/pknMBpKeni34lNHdDGZswJZWdO3fmbXK9EKm2dHvDxoR+5syZkm07Ojqib9++WLt2rYFd7iEixM6oUaOQmJho8k1G6oPIGCMjI3HkyBE9Com0aUgnJyfs2bPHqNBfvHhRtL3k5GSDFADv378XNau4IRs+iOX0kDl27dqVzxfTkDdv3hT9AGnZsiUfAqrLkJAQdOjQQbHzTkSy7VlbW6NDhw6Ii4vj6eLiIivdBcddu3ahtrYWBw4cwLp164R6LT5tof/UuGTJEn6ylNjB18amt7c3ysvLjQr93r17zQ6qieH06dNx6dIlAOB7Y+YmoTSksTxBixYtwuDBg/UGpf9MdHJyMkhkxrkbpdgbMmQIkpOTsXXrVoSEhEh6AOmyVatWGD58OKZMmcKfd6ED2So/CQoS+kYpDi4WauERFSpUqJCEuwAsxrr+KePoVahQoUKFclCFXoUKFSo+c6hCr0KFChWfOVShV6E4Dhw48EnW6fyzoX///pSamvqxm6HiM8C/tNBv2rSJ0tPTqXPnzh+7KZ8VOnfuTElJSSZz9PyZYGtrS6dOnZJdgFppfPvtt5ScnEyBgYGKFat3c3OjiooKGjx4sCL2GhuBgYHEsiw9fvz4YzfFJCZNmqSYLU9PT2IYpnH06GOHVkoNr+TqvEqNCba3t8fJkyeh1WoVDYd0cnJCXl6epHJvxtiiRQvk5ubi+++/b7QQLQcHB74I9dSpU2XZ8vX1BcuyOHHiBJo0aSK7bTY2Nrhw4QI/8UrpfT9y5IgiseUNKWTqvyk2rFamZLsePnwouT6uJepWQ1PCXnh4OD8nQ2wVOXO8e/cuGIZRJGT3/fv3mDx5siLt4gqBX7hwQcx2n272SiFo3ry5rO2rqqro9evXCrVGHy4uLuTi4qKIrerqasrKyiJnZ2dF7BlDREQEdejQgYiIrl+/LtlOs2bNaOHChUREdPDgQUV6yc2bN6dWrVoRUX1WU6UxatQoIiJ6/vy5bFseHh4UEhJCRCT5fAUFBYnKeCkF7u7ujWJ3woQJRERUUVGhmE2dzqAKGfgkhf7bb78lIqInT57QF198IclGmzZtqEePHko2i4eVlZWi9rZs2UJ//etfFbWpCy7dMxFRRkaGZDsbNmyg0NBQJZrEg2VZYhiGiIjKy8sVta2L48ePy7YRGhpK+/bto+XLl0u2MWvWLGrbti1duXKF3r9/L7tNDfHy5UvFbRIRjRgxglq3bk1E4lIzW4KVlZWi95Ovry//oOOuqz8bunXrprzRj+22Eeu62bFjBxiGkTTFvCGXLl3K56uQkwZAl05OTmBZVjF7RPVJuViWlZTrxhK5Yhfl5eUWCyqYY1hYGFiWxZ07d8CyrMUkUULJFdtQ2oVBRDh37hyqqqrQr18/2bb69++PmpoalJSUoGfPnibTDVuinZ0dvLy80KpVK0lphC1xxowZpioVSaanpydu3boFhjEsvCOHuuk05OY54vjLL7+ISs1hiUq6btzd3fmcTyK2+/xSIDg5OfHT46UmZGpIpYW+TZs2YFlWVCphS+SEnqtrqyS5G2nlypWitmvZsiX69u2LrVu3orS0lG9fly5dFBV67sGutOBt2LABAHD+/HlF7J07dw4syyI0NBRE9WMLcm1y4yZK7vf48eMVF3quTkBdXZ3kRH7GqJvUUCmbDx48AMMwOHDggCL2lBR6IsLy5csbReib0SeE8+fP8/8rFXamtJtFSf9kY2PDhg1kZWVFFy5coJiYGFHburu7065du6hr165UWVlJO3bsoK1btyru/22MyJ3AwEAaN24cAaCNGzcqYrNdu3ZE9H+VpxrD7aIEsrOzFbVnY2PDV9j6+9//Tlu2bFHMttJuGyKif/u3fyMiooKCAkXt/unxsXvzQnv0I0eO5NN2KpnRTukePVF9L1lsYRRzbIwePZe6WOneXbt27ZCXlye4gIklcjUIiouLFT0/LMvi8ePHitiLjIwEy7IAAH9/f8Xa2Rg9+mXLlil6zg8cOMD35pVMPMcVRmFZFoWFhYrY7NevH4qLi40WYpHCYcOGgWVZWRFWxpidnY3du3cLXf/z6dF7enryg2UvX76kwsLCj9wiy1Cy7mdjwNXVlYiIMjMzFbVbWlpK2dnZdOnSJUXscW8IV69eVcSep6cnH8WRmJioiM2wsDAiIgLA11SVi4CAAH5wU0l8+eWXitr7z//8T2rSpD6mIycnRxGb3t7e5O3tzZ8nS3VXhWLx4sVERLRv3z5F7HFvcbJquRqBpVrIUvBJRN0sXLiQLwa9evVqRW1zr4eBgYGK2m0MPHr0SBE7c+fO5Z/03333nSI2GyI/P79R7MpBkyZNaPbs2URUL/KxsbGK2PX09CQiIo1Go4g9IqK//OUvRkOIW7ZsKcuut7e3rO0bwsXFhViWpYSEBMVshoaGUosWLcjKyoru3btHmzZtkm2zbdu2lJmZSVqtVjER/cc//kEAFH94nj59WlF7REQf3W1jyXVTVFTEvxq2bNlS0VckItKrFKOUTZZlUV1dja5duypij3PduLu7y7Lj5eWFjIwMxXOSe3l54euvv4avry/c3NwwevRojB49Gs7OzujcuTOGDx8u2TY3EHvq1CnZ7RwyZAgYxnztWqnnm2VZHDp0iP9OatQNx5SUFH7fbW1tMXXqVDAMg+TkZMk227dvj+LiYsVcNwkJCairq8OdO3dk7y/Hx48f8wECYuqumqOjoyM/SUpMgXFLHDNmjOKuT6L6CC6GYbB3714h6zd+1A0RvSKidCJ6wP0gETkS0R9ElPXhb1s5Qq9bhELpA0rUOEKfkZEBlmWRkJCgiD1O6Lt06SLLDid0DCO+2LIx2tjYIDY2FhqNBizLora2FiUlJbzwFRYWory8XFaJNa69csc8Bg8ezFdFMle7VgpZlkVVVRW++uor/ju554rzzzMMg0ePHvH/r1+/XrLNNWvWgGVZ5ObmKrLf3L25YcMGxY6lbqSNUrNhly5dCoZhFIuw4jhlypRGE3oAePHihZD1/2lC79Tgu7VE9OOH/38kojVyhB5Ao8VRExGePXvG25d7c3LcuHEjKisr0bx5c0XsKTUYy9W2VULsOnTogIMHD+rVuMzNzcX58+exdu1aPfr6+kr+HaUeTFwR6Ma4jliWRXh4uGL23N3dTdZ5tbOzk2z37t27AJQLKQWg6NuRs7MzGIbh73ml7GZkZCA/P1+xtw6Oo0aNAsuyiqfmaNOmDRiGwfPnz4Ws/9GE/ikRuX7435WInkoVeu4VRuAOS2JQUBDfo1dS6FmWRXBwsGLtZFkWv/76qywbxcXFYBgGy5Ytk90eT09PPHz4EGFhYY12boiUE/r8/HzelpI35oEDB/Do0SNFI8FGjBjB119NSUlBv379YG1tLdvu3bt3wbIsli5dKttWcnKyop2jyMhI/vzcvn1bMVEePHgw7t+/3yh5koiUj6Pn+PjxY9TV1Qm5V/8pUTcgot8/lAL8BcB2IuoAIJ+ICEC+lZVVe2MbWllZhRNRuMzfl42MjAzKzMxUNMXA2LFj6d27d4pHtMgdiHNwcCAionfv3sluy6tXrxothURD3L17l+7evSvLhm5c+9u3b+U2icd//Md/0Jo1axSNBDt9+jQ1bdpUMXsc9u3bRzY2NnTixAnFbcvF8ePHydHRkUpKShQZfCUi6tSpE50+fZp69eql6DnXxZo1axrF7rp162jHjh3KGZTZo3f78Lc9ET0kokAiqmiwTrnUHr2Li0uj9+gbg4cOHcLDhw/RuXNnxWxev34d06dPl2Xj9evXKC4uhp+f30c/RkLJMAzWrVv30dthitnZ2R+9DR+DSvfoG4Oenp6N5vL9E/GfWxzcysoqmoj+l4imEVH/D715VyK6BMBslh61OLgKFSpUSELjFge3srKyt7KycuD+J6KBRPQPIvqViCZ/WG0yETVCUKgKFSpUqBAKOT76DkR08kMuimZEdADAOSsrq9tEdMTKymoqEeUQ0Rj5zVShQoUKFVKhmOtGViNU140KFXrIzc2ljh07Uk5ODj/zVoUKI2hc140KFRyaN29Ow4cPp4cPH9L+/fsVnxL+rwg3NzcCQB4eHnwuHRV/fkRHR1NqaipJ7UA7ODhQeHg4rV27VtmGyYm6UYr08UeuFeOoUaMQFxcHAEhJSVG01qWSHDlyJE85dqKiorBr1y69GcYMw+jNElUpnhEREejevTs2bNiAuro6RVNWfCr09fVFREQEIiIiPnpbhJCb0cpBig1PT0+wLIv8/Hyh23x+hUe2bt2KBw8e8IKyefNmSQfz1KlTYBgGpaWlSE5O5qfwS7HVvXt3zJgxw+hMxoa0NEN05MiRJrd98uQJcnJycO/ePdy4cQN9+vQR3MZu3bohMDAQWq1WbyZrYWEhCgsLUVVVhW+++UbS/nMVqnSFPjc3F1qtFmlpaZLy83h7e2Pbtm3Ytm0bxo4dCw8PD9k34ebNm/l8J6ZoKcVwkyZNMGrUKHz//ffIzs7GwYMHcfz4cb2JPps2bcKAAQNga2sru80c+/btKzv9QUNyVazk2Pjyyy8xd+5cbN26FZWVlaisrER8fDzmzp0recJT69at+WLwXM6bhhQzeS4uLg6bN2/myYWF3rx5U7Ew49TUVERHR6N///4gIkRHR0sWeiLC6NGjwbIsoqOjhaz/+Qg9d+NzvcVXr15Bq9WioKBA0gVVUlKCVatWwdHRkbf/9u1bSTNGG4p8dnY2nj9/zlN3maWeybRp0wQ9MBiGQW5urqDc576+vkhPT+fF/ffff0dkZCQiIyMRFBSEIUOGgGVZ3Lx5U9JFqSv0OTk56Nu3L3x8fHjRX7FihSh7XEm+hg8kqW8I4eHhqKmpEXRMKysrzdqKiIgwu72uMJ07d07yjd6Q3bt3lzWfpGXLlvDz84Ofnx8SExNx5MgRXuxCQkJE2/v++++RmJiIt2/fmjwWr1+/xqRJk0TbNnY8z549K0no4+LizJ6v8vJy2W+eqampvMBzlCv0RPUz4V+/fo2qyBHuAAAgAElEQVQePXpYWvfzEPpt27ahvLycF44nT56gc+fO/OeAgABRB/C7777DwYMHDb7/6aefJE1+mTFjBo4fP443b95gxYoVBhk2xQh9XV2dYKFnGAbv3r2zmLOmsLAQLMtCo9GgT58+/MONY6dOnWQLPcuyePnypd73UnPLFBYWIi0tDatWrcKTJ094sffy8pLUvhs3boBhGNTU1CAjIwNlZWXIyMjgWVtby7dz3rx5Fm8+oUKv5EQdTuhFvM6DqD7p3OjRo3H9+nWz7RZjc9++fXyHi+OxY8ewY8cOntnZ2WCY+jdmMR0x3XZWVFTAw8MDHh4ecHV11fs9oW9LmzdvNnj4ZGRk6OW3ev36teQ3xtTUVKOCrpTQsywrJC/R5yH0nKA/f/5cr4KNVKE3lWisW7duYFlW9uxTjtbW1li3bp1iN/68efMwb948lJaW6tkU22PW5aBBg/h6r2KPoyXa29vj0aNH0Gq1suqnJicny6oGtXLlShw6dAjNmjUDEfFJwdzc3PD999+LOj/cG5pGo8GkSZMQEBCAgIAAfv8CAgIwfvx43p7Uh1NDJiQk8L8rdBvuoVNaWopDhw7h0KFDer3DgIAAPveRmLakp6ebTRfu6OiIqKgoMAyD3377TfS+9u7dW+/zw4cPwTAMHj9+LMlXb8w91bx5cxw8eJA/T0qlE+fIQY6NX3/99V9L6KOjo9GnTx+0b99e73upQm+K1tbWYFkWsbGxsm0NGDAAv/76q56IZGVlKdJOBwcHHDp0CJWVlWAYBtevX5dkp1mzZti1axdYlsWlS5fQunVrRS92IsLUqVOh1WrRr18/Sdvb2NjwbpyYmBjF2jV37lzcu3ePPzd1dXWCcp+7urrCz8/PbC/V1tYWGzduBMMwkseQGjItLQ0Mw+CPP/4QvM2uXbuQnp6OadOmGV1eVlYGhhFforF9+/Zo2rSp0WXt2rXDmTNn+IeS1HEfjtu2bePfcqOiohQ5li1atMCmTZv03Deenp6KXVtcb14VepFCb4pKCz0RKSb079+/1xP55cuXC/G1CWZ4eDg0Gg0YhsG4ceMk2Vi/fj3/eqjEYKcx2tvb49mzZ6KFvkmTJggKCsKlS5eg0WiQlJSkWMpnIn132tOnTy26bMTSz8+P94ErkTWRa2toaKio7Vq1amX0uO3bt4+3qWTmRa6ojZS26rJNmzYYPHgwWJbF27dvJXdmjPHQoUN8G+UEIZgiF3mTmpoqy86/nNC3aNEC/v7+/Gg5UJ+rOjc3V9GESpzrhnt1dHJywjfffIMlS5YI2t7b2xtPnz4FwzB4//49qqqqUFVVpehFRFTfo9LtiUqx4ejoCJZlkZaWJvlBocvWrVtjzJgxeuSWpaenY8aMGYJtNWvWjB+AFVphqH379nBwcDC7To8ePXD69Gm8e/cOlZWVyMvLU1zgOXJCr1txSgpDQkJ4QVKiMPz06dMV6W3qctKkSbh48SIqKipQUVGBadOmwcnJSbK9c+fO8R0Qpdo4c+ZMvtg4587avn27aDu6MLeOXJEnImRmZoJlWSEF1z99obe2tuZD9TQaDY4cOYK3b99Cq9UiPz8f8+fPl+X/Jap/1e7SpQvCwsLAsiwePHiAXbt24fXr12BZ1mIkBkcxg65S2LRpU0ydOhXXrl3jBxel+Oe7dOmC9PR0aLVaLFq0SFabmjRpguXLl+PSpUsGcfQFBQUoKCjAu3fvRPXo+/fvj3fv3pldx9vbG4MHD+Z9zyzL4tmzZybXHzx4sN75kepKEsLmzZujb9++YFkWUVFRaNKkiSQ73333HYqKikS7bEzR1taWd9co9YBzdnbm3yzLysrQq1cvWfbCwsL4a2fEiBGKtJEbyOa4fv16dOvWTbSdhjHypsQcgEEUjhSyLItNmzbxY0tm+GkLvY2NDYYPHw6tVoulS5eib9++cHR01Iuj12q1onukdnZ2CA4Oxpo1a3D79m290EOWZVFXV4fs7GwsX74c/v7+Fv139vb26NGjB38hXbhwQZELtCG5mqEcpfYalixZApZlTfpuxXDChAl6pd90hV7388qVKwXbjI+PB8MwiI2N5V+rhw4diqlTp+L+/fu4f/8+ysrKDCpbzZkzx6i90aNH8+MZDMMgLy/P5G83bdoUAwcOlFQ1qWXLlggJCeHDTTnu2bMHmzZtEmXL3t6eHyhlGEZ2rWA7Ozvs3LkT5eXlmDFjhkEREy8vL3h5eSE2Nha7du0SZDMsLIwf58jMzDRZZMfd3R3du3dH9+7dTdqysrLCuHHjoNFoUFVVhQEDBsi+Nh0cHLB7927+YZmXl4f9+/ebHF8YOHAgT2PLde83LtqmoeD3799fkd68nZ0dWJbFTz/9JGT9T1foV69eDa1Wi8rKSj580N/fnw+vHDBgAFq1aoXBgwcjKSmJFxRz4ZHnz58Hy7KoqanB5s2b8c033/D+fS6MT2zxcW9vb7x+/RoMw+D33383GDAmIvj4+ODEiRO4e/cu7t69i8uXLwue7NSnTx8cPnwYjx8/5m/6q1evSr6AXrx4oVjN0DFjxvBtunLlit5bTGBgIHbu3KkneGIegCtWrEBubq6emLMsi4sXL2L8+PEIDAwUZOfo0aP87+uO5XCzeRmGweHDh2WHHFZXV4NhTE/wEWLPzs4OCQkJqKio4Le5fPmynhvEwcEBt2/fFiWEurVns7KywDD1tVO5SUkc79y5I2jwuG3btnrbxcXFISkpCUlJSQb7/P79e7x48QIJCQlmJwtybwW///67xWM0ZMgQnubWbXjsZ8+ejaioKERFRSE6OtroOTI3FmBp8lKqiVBLsdy9ezdKS0tx+fJlodt8ukKv1Wrx9u1bjB8/HkFBQThy5Ag0Gg2WLVtmdOAwOTkZycnJZsPZAODly5cGs+HWrFkDjUaDgoIC0SdFNx43ISGBn8gRHR2NmJgYxMTEGFxMYiaRjB07lt/u7du3WLNmDdq0aSP5IkpLSwPLsnj//r3sC3LMmDF6k6K4sY1BgwZh8+bNej36/Px80THgbm5uWLJkCbZu3YqtW7di48aNotsI/F+94devX+PJkyd48uSJSTGuq6tDamoqEhMTRf0Ot70coT9//rzJbePi4hAXF8cPds6ePVt02xqyurqav28CAgIEd3K4QtvGWFRUhNzcXOTm5iIuLk7Q27ZuwfrevXujU6dOPDt37sz/n5SUpPdwshQtpNuuJ0+emJ2jcuPGDaSmphqEdupSd+ZrQ3IiL7c3HxAQgLy8PAAQ8xb3aQt9VVUV7t69qycYpl67hJBlWdy7d0/P59W8eXO+ly/Fr7Zt2za9C6akpAQvX740mFCiSzF+cV1Rio2NRWBgoKzoE19fX753HBUVhaFDh4KI9B5+XK/HUjibrtBzwsH55HVdNxqNBl999RUGDx4s6yaQwm3btqGqqsogCqqh4HED51JniRoT+pKSEgQEBGDBggWCIsM491JdXR1qampQU1OjN5mLo9jZnNHR0Vi3bh38/Pz4MQ1/f398+eWXko7pvHnzUFRUpDcrtrS0FCtWrEDHjh1F2zt37pzJc2PqwbllyxaL6QtMzYbWPd8lJSWYMGGCoMio6OhopDaYBcuJPwCh6QrMUvftlXNb6noeTPDTFfqJEyciLi4OixYtkj3YqsuYmBjU1NTg3bt3ePfuHY4ePSrb5tChQw1egzm+evUKrVu35inWtpCp+9evXxf1Kj9s2DA+ft4Yd+3aJXgCSe/evU366IuLixUtpSiXXbp0ga+vL3x9fY262KQwKCjIQJjKy8vx3XffffT9/VTYcFIh9+A4c+YMHj58iAcPHuD69evw8fGRZJ87576+vhYjsyyRE3tO2JUQd11y9yAAsCyL5ORkfP/995a2++eWEpSDTz0ffcuWLWnWrFn097//nYiIJkyYQEREL1++pJs3b0q2O3nyZJo+fTq5u7uTm5ubwfLs7GwqLS2lL774gpydnUXZHj58uNHvf/31V0lt/VdEUFAQ/fd//zf/2crKipYtW8ZfByos48svv6T//M//1PsuKyuLLl68+JFa9MlBUD76j96bN9ajV6lPf39/TJ06FSdOnNBzAXXr1g3t27fH1KlTP3ob/xXZrFkzJCQk8G4Me3v7j94mlf9yVHv0KlSoUPGZQ60wpUKFChUqVKFXoUKFis8eqtCrUKFCxWcOVehVqFCh4jNHs4/dgI8FHx8fGjp0KPn5+dGDBw+IiJSvvK7iT4WWLVuSr68vDRw4kP72t7+Rq6sr9enTh1iW/dhN+6QxYsQImjt3LgUGBvLfNW3a9CO26J+HefPm0YIFC6iyspJ2795NRES///473b179yO3rAE+dmilpfDKdevW4dWrV8jJyTGYmMNRbM5vR0dHFBQUGEwWMpXQqCHXrl1rcsIRR61Wi0ePHokuatCtWzckJiYiMTFRL3e2MT569OijhXUlJSUhJiYGX331lQGdnZ1l2w8LC+PzCDWcjJWbm4uffvpJVGWgs2fPYvPmzVi+fDk6deoEX19feHl54eTJk5Lax02Watg2sQm5bG1tMWzYMGzYsEHv+klKSkJYWJhBAjIp9PLyQlZWlsEx7Nmzp2SbHTt2xIsXL4zej1qtFikpKYLCTbt164YePXpg2LBhCA0NNXovcRCbjO/06dN8osJTp04hKSlJctFyjq1bt8akSZP4nESm9n/hwoWCbX7zzTd49uyZ3j4zDIMTJ04I2f7TnRmrS3NZETmKqQrPcfr06dBoNHj16hV/cIUWHXn+/Dm/TXFxMS5evMgzISEBd+/e5ZdbSr7UkNwDTSgtJaKyt7dHu3btMHnyZD5nSkNKueB183vrpnzgksuJuaHc3NwMLvyKigqLmTHnzp0r66Ylqk9IJqU4yN27d4227datW6JmYOoet8rKSj7BF0e5lcnGjBnDp+WIjY3Fd999hxEjRkCr1WLPnj2S7Xp5efGdmZkzZ/KMjo7mj8egQYPM2mjSpAnmz5+PZ8+emS003pBC27h+/XrU1tYaZKe9f/++5H339vbWO9/Pnj3DsWPHsHr1anTr1k200AcHB/M1sf39/TF58mS4ubmBZVmL6bo/8PMS+ry8PKxcuZJPFiZX6IkI9+/fx6NHj/iL4IsvvhC0XZcuXRAUFIQuXbrA1dXVYLmDgwP/APnll19EtUl3SnhZWRkiIyORmZlptldvKm9JSEgIHjx4YPHGkZoS4IcffsDt27cNmJGRgTt37gi207CX5u7ujkWLFvF5c0pKSnDnzh1cvHiRF8WnT5/Krsnaq1cvHD58GB/mcYjinj17cPv2bezZswd79uzBzJkz+baZy9TYkG/evMGcOXP4qe7t2rXDnDlzEBsby++/nH3k6rc2TAImV+iJCKNGjTL4jnsACBH6hpkwG/Lt27d8GUUpQk9Un0l16tSpmDp1Ko4dO6Yn+o6OjqL3uVOnTjh//jzOnz8Pf39/vTQfrVu3Fi30Fy5cQF5enkFG1n85oTdFjUbDH1Q5F+vDhw/5E69E7hsbGxtUV1dLrpJjrgRbly5dcP36db2L3lQecCLC06dPMXbsWKPLPDw8sHfvXos2pBCAYjYjIyP5ghkMU19hbNu2bbJsOjk5ISUlRZEsnhzz8/MBAMePH5dty9bWlt/fBw8eKHpuiOrdYufOnVPc7qlTpwSLvC779++PFStW4Nq1aybXefnypSShb8i4uDi8f/8eLMtaTIwmlF988QUOHDgArVaLwsJC0W/xxsiyLDIyMoSs+/kK/fjx4/V69FIPZmhoKJ+MimVZ2a6AAQMGIDExkX8a//DDD7JPuL29PTw9PXHr1i2DFLtCq18Zo67rRW6yJ10uXrxY9s1IVN+znTZtmp575M6dO4iIiJCVxdTFxQX37t0Dy7Kora3FzJkzJfXsONrZ2WH9+vW8yyU8PBx2dnay9n3FihX8ucnJyTFbtEMMu3btyhewCQsLU+yccywtLZV1TxobM+jQoQN69uypl8lTim17e3ts3LiRv9fNFaARw5YtW6KsrIzfb2NvOWLZuXNn1NbW8tllLVAZoSeiRCIqIqJ/6HznSER/EFHWh79tP3xvRUSbieg5ET0iop5KCn1oaCjS09NRU1Ojd/OLPZDe3t7IyMjgn+xiXTfG+PXXX6Ouro63VVtbi6CgIFkn3N7eHgcPHjT6WltWViap19yiRQsMHz5c8quwOXL592/fvi3bFtdD4oT+1KlTRt1kYtivXz9kZ2cbDPY9fPhQss2ffvrJYPxg2LBhstq5ceNGPHr0iH94vHr1CosXL5aVydXBwUFv3GPatGmKPeDbtGmD1NRU2ddUwxKP69atM7Ar1fbRo0f5lORJSUmy3X6tWrVCfHw8UlNTodVqUVtbq0hwhJeXFx48eIDk5GSh2ygm9IFE1JP0hX4tEf344f8fiWjNh///i4h+o3rB701EN+UIva+vr8mBuNOnT8s6oLNnz9Zzsch13fTp0wdnzpxBUVGR0Qic9evXi7bp6OiImzdvGhX6vLw8Sa+euonRjFFqbzkyMhKvXr0SlHtdCF1dXTF37lycOHGCP+eNVes1OTkZ5eXlorcbOnQo0tPT8eWXX6JJkyYYO3Ys31YlU9jOnz8fVVVVYBhGdBQXRzc3Nzx69Ahr1qxBhw4d+IgZOe2MiIjQuz8zMzNRWVkpukc/bNgwPfecMT59+lRWxMz27dtRW1uL9PR0hIaGokWLFrLOCedG5Nr37NkzTJ48WbbNhpE32dnZlurGKue6ISJP0hf6p0Tk+uF/VyJ6+uH/X4goxNh6UoVe92TrHliGYWT3mIOCghASEoKJEyeioqJCER99p06dEBQUhB07dhicOCmFor28vDBu3DiMGzfOoBDF2rVrRdvz8fFBQEAAz1GjRmHEiBGYPXs2ampqkJaWJqQgsQG3bduG7OxshIeHIzw8XHYYmy6XLVuGN2/egGEYREZGii75aIkjR46UJPTGyJ2b+Ph4Rds4ffp0VFdX4/Dhw7LePDkOHDhQ9hvdxo0b+QpiM2fOBBHhzJkzooV+yZIlFgMG5PbAiQhfffUVfy/+9ttvsmxt2rQJmzdvxqZNm/gHncDBU5N88eIFXr58ifv37+Ply5f82+fGjRvNhdk2qtBXNFhe/uHvfxNRgM73F4nIX47rZvHixfD399crIBAXFwetVitb6DlaWVkhOjoaWVlZihbLaBgXvGDBAln2bG1t9apayR1M7Nu3Ly5dusR/7tmzJxiGkVTkITAwELdv34ZGo9HrkSg1KMvVDGYYRlIlI3McOXIkNBqNIraUGDsyxa+//hoMw+DgwYOSOg26dHV1RXZ2tqx2jh49GtOnT9f77uzZs6IfHt7e3ggODsbRo0cRFxeH4OBgjB07lq8EVV1dLTkMuCGzsrL461PuWApHuWMTpsi9JbIsa+6e/ChCf4YMhf4rEzbDiejOB4o6ANwrslJCz02AslQ+TwpjY2P1yvfJtefp6ckLfUJCgmQ7x44dw507d/ji6xy5twa57Rw8eLBi/nqOXJiplJJ/5lhUVKRYjz4kJKTRhJ6I+HBD3WLsUnn//n1F26kbdaPkPtvb24NhGGRmZiryNufm5oaff/4Z5eXlsLW1lW2Pe2A21jkvLy/H0qVLTS0XJPRSc90UWllZuRIRffhb9OH7/yEiD5313Ikoz5gBANsB+MNILmVra2saMmQI2dnZGWwXFhZGO3fuFN3gdu3a0enTpykkJETve1dXVwoPDyciohMnToi2y9kYO3as0WU9evTg/79x44Yk+xy8vLzo6NGj/Of//d//lWzLycmJ/va3v9HBgwf578aMGUPW1tay2shh5MiRBIB+/vlnRewREf3lL38hItPVsUyhWbNmJvcrISGBnJycaPPmzbLbR0Tk6empiB1TOH78OBERubi4yLLTv39/+utf/6pEk3g0rBSlFJo3b05ERF27dlUktUJtbS0BIFtbW+rdu7fF9T08PEwu69Chg9nlSuBDx1i+EQk9+nWkPxi79sP/Q0h/MPaWQPt6T6nffvsNWq0WHh4eet9PmDCBn0Wm0WhETTdPSkoCy7LIzMzk/X1fffUVH2rHsqykwtsuLi54+PCh0ZqwHTp00HPdWLI1YMAApKenIz09HX379oWXlxecnZ3h5uaGrVu36sUSS/XRc9y4cSNvJzs7G9nZ2XwR7SlTpsjuhbAsi8ePH4vaZsKECQaumcjISERGRvL7fufOHdH1d/v3748zZ84YpEz44osv+KgbJycn0fs4btw4bN++Xe87rp1KuYIacu7cuWAYRnLqBo5r164V7aMfMWIEZs2apffdrFmzMGvWLD46prS01GDyjzG2bt0aPj4+gtyE3377Ld9WKbWXdX8zKioKb9++FRxOffDgQWRmZiImJkbv+/bt28Pb2xs3btzgZzU3xtwEovp7yYz3QrGom4NElE9EdVTfY59KRO2o3i2T9eGv44d1rYhoCxG9IKJ0EuCfNyb0pvJHcNEmYmebcly/fj0iIiIMomJKSkokn4Rp06bxdu7evYuUlBS9FAgsy+Lt27d8VXdzHDlypMVBKYapn+WoxGCnjY0NHB0dERsby1PKLNGGPH78OJKSkkRt4+DggB07dhg951qtFuXl5ejVq5fkNunOgNYtwCxn0szatWuh1WqRlpbGu5WUirgZPny4QWoIovoOC8MwGD58uGBbLi4u0Gq1CAkJwcCBA/mQwNzcXLRp00aQjadPnxq9J58/f47MzExkZmaKmiR19epVvWs6JycHL1++xMuXLxEYGIh+/frxkUYMw2D//v2iw0Hnzp2L69ev8+eaK97+448/Cr5/9u7di/fv35vUo/LycgP3p1RyKTBSUlKwePFivhP69ddfm9vu050wtWvXLoOD+vTpU2zevFnU9HJjtLW1xfz587F//37+xMtJ7uTp6YkDBw6YTG62du1awQLl5+eHmpoakwK/e/du7Ny5U5GLqrHICZGUbd3c3Pg3Ni6K4fXr15g6daroZGEN6e3tjRUrVqC8vBzJyckIDQ0VLHKmGBYWZnDjT5s2TZFBvqysLNy7dw9z587VI/c7YpKdcUKv287o6GiDN2ZznDhxItLT06HVavVy20jtYXOTFYXwzZs3okV+5MiRqK2tRV5eHurq6pCamop79+5J6iDNmDEDe/fuNdCkH3/8UdbchoYcMGAASktL9fRDQCTPpyv0tra2+OGHH1BcXIxjx47hhx9+gIuLi2IHVGna2tpi+/btiIqKwpUrV7B9+3Zs374d48ePF22Ly02iy2XLlmH48OGyZoQ2NiMjI/kJUxs2bJBsp1+/fnwCrj970XMHBwde/G7cuIE1a9YoduPb2dkZzfNy8+ZNDB8+XNSbl247tVot1qxZ89GPnY2NDdzc3JCcnGw0l9OtW7eQnJyM5ORkSZ27nTt3Ijw8HEQkO2b+n0lHR0csXrwYSUlJSEpKgr+/v6Vt1OLgKlSoUPGZQy0OrkKFChUqVKFXoUKFis8eqtCrUKFCxWcOVehVqFCh4jOHKvQmcPHiRUpJSZFtJyUlhVxdXRVokQoVKlRIw59O6HUryQvBjz/+SN26dVO0DatXr6Z///d/p23bthldHhMTQ1qtVpCtwMBAys3NJS8vL8Xa9+bNG/L19VXMnhzopnjgYGtrSwzDEMMwlJqaKtpmcXExPX782GjagvHjx1Ntba2ktprDvHnz6PDhw5K2LS4uJpZl6datW3rfN2/enJKTk4llWSWayKOgoIAYhqHJkycLWt/a2pr8/PwMvv/iiy9Io9Eo2jYOM2bMoJqaGr2UHUqgWbNmxLIs5efn0/jx40Vta29vr/fZzs6OIiIiKD8/X3ZKCQ6XL18mhmEMrgUhMNcOLg2EZHzsGPqGcfTLli0THHPapEkT/PLLL4qmxCUiXLp0CX/88YfJ5VzN2hEjRli0xcUuS0mz2rVrV3Tq1Mng++zsbEWzbCYlJYnO9962bVtMnjwZBQUFBpN3dEvhpaamimoLlwyNYRijMenBwcGorq7G3r17Fdn3Jk2a8PsvtZITy7IoLS01mGXbokULvlSlWJtjx4412p7Q0FBUV1dDq9Wid+/egmyFhIRg1apVBt97e3srksDOGLVaLcrKygzSTsjljBkzwLIsRo8eLXrbo0ePYuDAgfznJUuW8PenUrH2Wq0WJSUloovkuLu7o7i42Ghm1vHjx5ub+/BpTph6/vy54IPTsWNHSTcRx8DAQIMyciEhISgpKUGPHj1MbscJfVlZmdkJDS1btuQvpEWLFolu39u3bxEZGan3Xe/evbFixQrZF2SHDh34vObG8n6MGTPG7Pa9e/c2KchShd7e3h43b95EXV0d4uPjTabiff78OZ4+fapIhSR/f3/J9X05msqb0qZNGxQWFqKurk60Ta1Wiz/++MNgH7OyskRnSjx69KjRTKdKCP3WrVsNvnNzc4NWqzWXcVESZ8yYgczMTOzevVvUzGCi+pQSGo1GL60ylwqioqJCsTZqtVpcuHBB9Ha9evVCWlqa0WWlpaXmypJ+ekLfvXt3VFVVCT44165dk3WD3rt3D99++y3/2c/PD2/fvrV48Tdt2hRLly7lb7jvv//e6HrcBS+1R28sadm+fftk1xC1sbEBy7K4evUq+vfvj759+xqsY6lajtJC37NnTxQVFYFhGIspKZ4/fy45+Zqfnx/Kysr46f8sy2L37t2SpvK3a9cO+fn5+PHHH40u53IezZkzx6ydcePG8f87ODjg2LFj0Gq1Brnera2todVq8c0334iqRwwAq1evNvh+2LBhsu6fyMhIaLVavYeIg4MDDh8+jFu3bsm6RisqKjB//nz+c01NDViWNTgmQqibVoP7rmPHjtBqtTh69ChatWol2uaRI0f0ksJZW1sjNTVV0qxjPz8/VFdXG13m5OSkVzPCCD89of/xxx8FX3gdOnRAfn6+5As1JiYGdXV1cAfGHFYAACAASURBVHZ25r87ePAgtFotrl69anH71q1b8xfP/fv3TZ5AqULv5uZm9KKpqqqS9TpsZ2eHmJgY3LlzBx06dMDcuXPh7u5usJ6luqe6Qt9wXSlCX15eLjib4p49e/gcKGL2vU2bNtizZw9fXcjLywvl5eWSct7Y2Njg0KFDAGDy7Y97UxB67ps3b86L/KZNmwwEaNasWfjtt9+wZs0a9O/fX3BbWZY16rqRU8g9Pj4eRUVFSEpK0nNTODg44N69e5g3b54ku+3bt8f58+dRV1en1wFhWRZJSUmik9AFBwfz9+DPP//MX587d+7EvXv3JIk80f+5ZIuLi0FE6NGjB7RarajzwnHEiBFGz4O1tTUSEhIsVRT79IR+9+7dgoWbSzv85MkT0QfWw8MDBQUFqKmpMTh5OTk5ok92w9doLnXrlStXZPXo4+PjDQojsCyLpk2bSiqYYGNjg4MHD4JlWfTp0wcuLi54+/atJKHTzcmTmpqKli1bIjg4GHPmzMHFixdFCz23vpBi73Z2dmAYBgUFBaLavHHjRrAsy7vbcnNzsXDhQtH7TlRfXcmcy6d58+ZgWRbnzp0T7GJ69eoVtNr6QtMNl/n7+6OsrAxVVVUYO3asqLaaEvrVq1dLEnoucV1ubq7Bm9DQoUOh0WiMdh4ssVmzZigsLATLsigoKICVlRXatm2L6OhoaDQaUeNSDg4OmDlzJt+bX7lyJaytrWFtbY3ExERZWUajoqKg1Wpx69YtPg8PVyReSq4jU0Lft29fk2NVOvw8hN7Pzw9RUVHYuHEjn8azqKiIL0IsZiC2WbNmmDJlCoD62rM3btyAra0tXF1dcfPmTSQmJkq64IVQSoKuEydO8Cl19+7di7S0NAAw8NsLZdOmTfljrMutW7di69atyMjIAMuyGDJkiFk7gYGBvIvLEi2kWOXJMIzFgu8uLi7w8/ODn58fb1/og557+/v111+xcuVKvuQhd42JrVrFsizKysqMDpYT1WdgTU9PF2TL3t4eR48eRXV1tdHi6qNGjeKvd7E90LCwMD2h79q1K6ZOnYqwsDBkZWVhx44douxxPWSGYZCWlsb/X1RUpJdWWkwGzyZNmiA2NpZ/mx40aBCsrKwwa9YssCwrKbe/bieM63i1aNGCb19SUhI6duwoKt0z0f/VydU9T1y66hcvXiAiIgIvXrwQXY5z0aJFqKio4F3JQ4cOBcuyJv32Ovz0hD4hIQEMw+D+/fs8WZbF+/fvUVFRgbi4OISGhsLd3R2FhYWia6ZOmDCBvxi5/No3btxAbm6upDJguq4ZSxR7QxHVC3NISAiOHz+Oe/fu8cWCpRTv5mhra4sRI0YgMTGRF/rXr1/zjI+PR7du3czacHZ25muDmmNpaalelIM5cj3EVatWwd3dnef+/ft5PnjwwOA3ysrKBNk3lkIaAC5cuICCggJRD8+oqCiwLGuynB+XTljooPmKFSv4tMzPnz/H8+fPMWfOHFy6dAnPnz+HRqMRPQDLkescPHjwwCAXv5RBaC67KCeY3P+FhYV6/4uxuXLlSr4tum8Ily5d4t/axdbINXYPcp0TrVaLqqoqbNu2De/evcPNmzcF2fT09OSL1Lu4uPDkOo4vX77kz5XYqKA2bdoAAKqrq7F8+XL8/PPPBp06Ly8vY66rT0/oiQgLFy7E6dOneYaFhRkNI2NZVlSEzrhx41BXV4eamhoMGDAAfn5+uHjxot5Fm5ubiy5dugi2aUronzx5gtjYWCxbtkyW0DfkxIkTZQ2e6XL58uVgWRbx8fFGl1sqIuHk5IQZM2YgJyeH59KlSzF9+nRUVlaCYRi8evUKx44dE9QeoW9GuqysrBRcS2Dy5MmYPHkyv995eXm4ePEiLly4gD179ojqgV27dg3Hjh3TSxXs5eWFGTNm4N27d2AYBizLCnbZ6Fb7MsWcnBxJBXe4QjsMwyA5ORmrVq1Ct27d0K1bN5SWluLIkSOibbq7uyM0NBTr1q3DunXr4O7ujvbt20Orra+0ZGxw3xw5ka+pqeE7GS1atODbPWXKFNH1CIYOHYpFixZhy5YtqKmp4e9zTpR1KeaNgauz2zC/v+7nvLw8SfdkYmIiX+WNu4YeP36MBQsWYMGCBbh586axmrmfptCLuTiExhETEebMmWNQaCEwMBBarRZjx46Fj4+P6NfiVq1aITExEZcuXTKaM1v3QfD69WtJJ5+jra0t7ty5g0ePHsmyo3v8FixYoIithhw+fLhoH721tTUcHBywYMECJCQkICEhAaGhoYiMjISDgwMcHBz0xiWkRN1YWVlh+fLlkl1fHK9duwaNRoPKykq9njFXJJplWRw+fFj2cQwKCkJGRobs9hqjg4MDSktLFemA2NraIiEhQXKkzcCBAzFmzBgcPXoUz58/549nbm4url27psiciYCAAGg0Gklhzg3p7OyMPXv24Pz587zQX758WZHKYhwzMzMxf/58zJo1C7Nnz4a3t7epUqeq0Jtj69atER8fj2fPnil2chqyYY9fji0XFxcwDMNHDshhy5YtkZubi/bt2zfKfq9Zs0byhCkh9PX1lST0o0aNUuSNaM2aNXoCf+rUKb7IzPv378GyrMmQSzHn+8WLF41We5YbAFRC6AcMGACtVot27drJtuXj48MfV0dHR0UKubRt2xbnzp0TPaZniV27dkVBQQGio6Ml1Zs2xYCAAGRmZgpd//MWegCYNGmS5IO5aNEiaLVaSdEBQunp6akn9HJ861OnTgXDMPjqq69ktysyMlJUHLZY6roiGkPoicT36O3s7HDixAnBPn1LDAgIwMKFCw3qutbV1YFlWdEzIxsyOTkZWq1WclSQJXbu3Blv375VROi5gWIl2nXs2DGwLCsq+s0S7969C61WixMnTih6DIODg8EwjJAqUKI4YsQIjBo1Suj6n7fQsywruUj4s2fPoNVqsWfPHkVPkDH6+/ujoKBAdIoBY/srYATeIuPj43H79u1G2193d3c9/2djCb3YB/SDBw9QU1Mjqz6w0PN04MAByds3bdoUq1evRk5OjqX4adksKiqSLfQajQa7d+9WpMM0ZcoUpKWlGcxWl8OFCxdKDm82Ry6c8vz584rabdWqldj5IZ+30AOQLPRcbz44OFjRk2SKISEhuHfvHuzt7SXbYBgGcXFxstrh5uaGt2/fYv369Y22r25ubtBoNGCY+vqmgwcP/qccY0ssKSmRffyEUK7QL168GFqt1mLkkxKUK/Rubm64deuW7LcXjqdPn5bVGTLGpKQkHD9+XHE35ZUrV5Cbm6t4x2HOnDli5zZ83kIvp0f/KbKoqEh2bpeYmBgcPny40XzzHAcOHAiGYRAWFvbRjxvHsrIyo3lZlCbDMFiyZInk7ffv36+YG8QS09LSZAn9tGnTRM8/MMdevXopWsh7ypQpqK6uNpooTC61Wi1++uknxe0uWLCgUYReLQ7+L4RZs2ZRfHz8x26GChUqlIOg4uCq0KtQoULFpwtBQv+nKzyiQoUKFSqUhSr0KlSoUPGZQxV6FSpUqPjMoQq9ij81IiIi6M2bN4rXXeUwYsQISkhIIADk4eHRKL/xOSM1NZUuXbpEs2bN+thNkQVbW9uP3YTGhYDQx0QiKiKif+h8F01Eb4jowQf+l86yRUT0nIieEtEgOeGVnTp1wqRJk7B9+3a9ii4Mw+D169c4duwYny5VSsL/fyY9PDxw7do1AMC1a9dk2YqOjkZ0dDQa4mPvY5MmTeDo6Iju3btj9erV0Gg0ACB5Fq6vry+f2Kq2tlaR0DuuEpDutVRTU4MbN2589OP3KbFNmzY4c+aM0UyW+fn5H719pmhjY4O+ffuipKTEIJOpXNtff/017t+/j2XLlqG2tpavmaDEzFk7Ozu4uroam7OgTBw9EQUSUU8yFPooI+v6ENFDIrIlov9HRC+IqKkUoedKvpnK5jdx4kQEBwcjODhY8cklnTt3xooVK1BUVISioiJZtmJjY3mB5yAlYyBHYwLfGEJvY2MjajZh69atDfLzl5WV4dmzZ6IygupSV+hPnjypyH7pVsbicuDLyZlkiT169MCAAQOQlJQkqvC9Obq6usLPz89kmmSx9PT0xKpVq7B//37Ex8dbzAdjZWWF1A/1VrVaLWbOnMnP/uZEv7GOJ1F9plCOQrdxdHRE79698csvv+gJvEajwZs3bzBz5kxRbVi/fj3Wr1+PsLAwrF+/HtnZ2XzmUt003VLP+YwZM7Br1y7s2rULK1as0Cvm02Bd5SZMEZEnCRP6RUS0SOfzeSL6DylC7+joiKysLL0Dd+3aNZw9e9agMpRS/Pbbb5GQkKBX1k7k5AUQ1ffeY2NjkZOToyfEYisDNWT//v2NCnxqaqoimfOaN2+O2bNnY8+ePbh//76ofY+JidE7Zjdu3JAtoJ6enrzQKzExJzw8HIWFhXzaXyUyGRqjg4MDBgwYgKysLH6WMMMwJuuCCmG7du0QEhKC7OxslJWVSU6c1rFjR36/HRwcMGfOHJSUlPBpcVmWtVgsW1fkAwMDDZaLLbohhFFRUbh+/Tpu3LjBH0+hacptbW2RkpIClmWxbNkyrF27Fj/++CMiIyPRp08fSe0RkkZbak8+IiLCpM27d+82XL/Rhf4VET2ietdO2w/f/39ENEFnvV1ENFqK0HO0srLC9u3bJQmuELZr1w7bt28Hy7K4ffs2Ro0aJfsNIScnB0eOHFGkp9i/f39exJV0T/Xr1w8jR45EWloaf4O/fPkSZ8+exZkzZ4yWnjNF7qEgNQ+3OXJCf+jQIdm27OzskJaWhrq6Oty/f1+xFBg//PADzp8/j9raWlRWViI7OxvPnj2Dl5cXXFxcRBfNIKrPjJiXl8fvf05ODg4fPowRI0ZIssdx7969vE3unM+bN09UOwcNGoTMzEy9PO8pKSmKnXMfHx8kJCRgy5YtAMAXBff09MSECRNE2erRowdYlkVVVRV2796tWBuNibDcXFQHDx7kXT7JyclISkpCQECApfTpjSr0HYioKdUP5q4kosQP328hQ6EfZcJmOBHd+UCzB2DdunVgGEbWBW6KXKbFTZs2KWJv7NixuHbtmkHueylMTU3le+1K7nObNm30ChycPXsWjo6Okn3gu3btAsMwWLp0qeLnhxMlpdL1tm/fXu/mNFLIQRC3bNmCNWvWoGPHjmAYBhUVFTh06JBiaXCXLFkClmVRXl6umGtp165dfBrlgwcP4rvvvpOVf2nixIlISUnh3TUbN24UXVmpIQMDA3H//n1UVVVh9uzZGDVqlKwkZ/v27eML7HTt2lWx65Lb5/T0dERGRqJz586yU5Tk5eVJ8SI0ntCbWkYKum50af//t/emUVVc3droLEUQlNggKsSGY3yRTxmRV73qjYxoXpMoJ0blqEkc2MRjovEajX6aKNcmKMdeo4bP2CZ2JxpNbDl2REX0xNhL5EVARUUOAREU2JdOaq3n/ti7Vvam3dUg6qg5xjPYu3bVZNaqqllrzTXXfBo1wunTp52mpXMGHh4eWLBgAQYPHowhQ4YYVk/6/PnzWLVqFT744APdD6giRhEa1K9fH4sWLUJ6ejqKi4uRnJyM9evXayIHt8fIkSPBGKuVWLfykmesRpJkp3Hw4EERUlmzZo0mR6LY9P3332P69OmGVlwkssaAOee65nOqstnoiot9+/bFwYMHwRjDmTNnVPOwKqhfvz7i4+NRWFhoSFitVatWSE9PFyPWnJwcDB8+3JA5PcYYCgoKDK0bZd8BUfgNnECt9uh97D5PJ6KfbJ+7kONk7F3SOBlbGfLy8kRD6K2YpwwJjbpIyoSrwga0atUq3T1xRWJjYw0J2/j7+4sbPz09HR9++KEh5644eqPasjyCg4PBOUdYWJhhOps3b44+ffqI+0kF0QOICD/99BOKi4vBmJWgfOzYsZpHB5W1p/2EoRGjhAMHDohzffr0aZUUknqxbt060dNVe+zvv/+OpUuXokGDBobZExAQgJiYGMybNw8xMTG4du2aaFc1BOblMX36dDDG1JYUrhYeHh4IDw93mO8qKyurKTpgWNbNbiLKJKIyIvofIhpPRDuJKIGsMfrD5Oj455A12yaFiEKcfJE41RCBgYGIiYkBYwzfffedrqp0Fy9eBGMM77zzjiEXyd7JExnr6I3s2fv5+eHo0aNiTsKIc38Wjp4xhsOHDxuu++rVq+KhatGihapjx40bJ2KqisM3wqaAgABBjr5jxw5Dar3369cPJSUlKCkpEUTktREK7dq1K6KiolRn3rzyyivYvn07ACA7Oxs7d+7U9P/r1atXbZZXs2bNMGvWLHDO0b9/f83n2aRJE3z77be1et937NgRAHDkyJHq9ns5yxQ3bdoUo0ePBmMMv/76q+rG69WrF4isPbqIiAjk5+cb0mMq/9ZVMm706i2fM2/UhKyWXmxV8Pb2RmZmZq0RZeh19CNGjEBmZiYyMzMxYcIEh9+aN2+Oa9eugTGmOV0xIiIC6enpYIyJ+8sIdOjQAYmJiapfQPYYOnRohW0hISHgnCMyMtLQssAKOnfurNoBtmjRAl27dsXixYtx48YNMMZw+fJl1dfk888/r5EeNDQ0FJxzzJw5U9d5uru7IyYmBiEhIaqPXblyZY3zI02bNsXVq1cRHR1d3X4vn6N//fXXsXDhQhw7dgyMMVy/ft3pXklmZiYKCwsd8mUjIyNx4sQJ1aTgNeHBgwd48OCBIROyChRnH6uBsWnJkiUVshWMnjxVFq8pOH36tGEkEnFxcZod/c8//wzGGP744w94e3tXuo/CQetsPDw1NbXCtl69eiE7O1sNBZzAH3/8gV9//RUfffQRfvnlFxw9ehQ3btwA51xV9lN5AKgyLZUxK6G1Fr2SJGH8+PF4/fXXK/xmv5BqzZo1TumLjo6udHR+69YtzJ8/XxUF57p168A5rzYOHxERAc452rVrp/vedHFxEfd8s2bNnD4uKioKDx8+xLffflvlegClgxMcHFydrpfL0Xfq1AkZGRkOccajR4863bDh4eEoLCwU8bmUlBRwznUxxJw/f94hN14J1+zdu9dQJ0/0VwaOFkfPOXfo5SiZIj179jTMvnr16mHgwIGYO3cu/vjjDzDGsGrVKt16Z82aha+//lpzmG3EiBHIz88HYwwbNmyo4OzbtWuHlJQUMMaczhjhnMPX1xe+vr5o3rw5xo0bh0ePHmmeMD527JhDTJ5zjqysLHz99de6MjkOHjyIR48eYeHChcJeX19f9O3bF5xz5Ofno3379qr1SpIEWZYr7Wna59g76+j37dvn8L1Hjx4ICwsDYwx9+vRRZZu3tzc450hLS6t0QnPmzJkoLS0F51w1h/NHH31U6TGKT5oyZYoqfQqbVE5OToUXcteuXfHgwQNnXiAvh6Nv3bo1pk+fjtTUVIceo5aZ/ZkzZ+LHH3/Ew4cPkZ2djejoaNSvX1/zgwRA5Mwrore8AZG1x2Hv0O0XSmmJ0+/duxeccyQlJSE9PR0AsGjRIt12VgUfHx8wxnDt2jVd7Uv0V3qlnljoBx98IHRcvnwZLVq0QIsWLfDWW28JJ69G/4YNG8QxWVlZ4vOZM2c02deyZUssWLAACxYswNSpUzFw4EA0adJE93VYsmRJlQtvZFnGmDFjNOmVJKnSNlM6I4wxVR0SzjkOHjyIPXv2AIB42X355ZeabGvdujU455BlGevXr8fkyZMxefJkLF26VDh5LYkYu3fvhre3N7y9veHm5gYPDw907dpVtMWlS5dU6zx79qw4/uzZswJKKNCJ+/LFdfTNmzfHzz//XGFl7NChQ3XNlL8IqEqMmIjt2LGjrrxpNUhMTARjTFMGRWRkJMrKysSinuTkZNVL1Mujffv2GDt2bAWHd/HixUrj2DWhc+fO6Nev33NfY4nIWvZBsbVfv36GzEm1b98e165dq1DrJiUlRVN2VPv27dGxY0f4+voacs5ubm746KOPkJCQ4DBS6tSpk+ZU6vv371f54iwtLcWgQYM02/vqq69iy5YtAh07dnR2tPViOvpevXqJIYsCi8VSqz3Q5wn2E6+xBpU2qAso11Ctow8MDERERIRIXZw0aZJhOeqSJKFNmza4desWUlJSsGTJkpe+41CbaN++vYOjN2L1stFwcXGBq6urgB5dISEhogNij1u3bmleO2AAXkxHv3TpUtGACQkJiIyM1L2gx8SzhzIBqjd0Y8LE84Tg4GCHjmhCQoLhi+VUwiQHN8UUU0x5ycXkjDXFFFNMMcV09KaYYoopL728kI7e39+f0tLS6tqMaiUyMpIAUOPGjevalJdKYmNjKSIiQreeBg0a0OLFi2n69On6jbITAIbYV158fHzo5s2bNHLkSMN1m/J8So8ePSg/P59u375Nfn5+unS9cI4+KiqK/vu//5uuX79uqN4OHTrQ06dPKSAgQLeuiIgImjFjBh0/fpw6dOhggHVEAwYMoNjYWAoPDzdE34soAKhfv37Ut29f3bpCQ0Np6NCh9DzMUdUka9eupZiYGOrUqROlp6cbojMnJ4cA0Ny5cw3R5+fnR7m5uRQSEkL169c3RGerVq1oyJAhdObMGUpOTqawsDBD9BIRdevWjRhjNGvWLN26ABBjjM6dO2eAZVaZOXMmnT17lho3bkyurq7k6uqqT2FdZ9yUz7qpDr/99ptI5fLy8jJs5vqNN95AQUEBVq9erWoZc1XgnGPx4sWGzq5HRUWJvHIjcuHr1asnUjmNtJPIWvcHsBanCgwMNEyvIkbpW758uebiWZVBWdhmdEqskuFRVFSEVq1a6dbXo0cPoXPy5MmG3Et79uwRuepGLPYispaGsE/fTE9PN0TvkiVLkJGRAVmW8dVXX+nS5evrK+zTSzyioEGDBmI1N2OsJl/3YqZXVoXevXuLi24ErZyCwsJCHD9+3LBKfj/88IMmirfq8Nprr4mLrmXpdmVQ9JWUlBhm5zvvvCPqvJ87dw6MOU/3Vh30lH+oCkoVVKM6DP369UOsQeWkiawrwhWKv7lz5xpSwdLNzQ1btmwBYwzr1q1TxblaFX766SdwzpGbm4t69eohLi5O1/3v7+8vSirfvHkTK1as0F2S3B7Kwi4jeG1DQkIgyzLy8vLQunVrQ+zr3bs3MjMzce/ePWevz8vj6P39/XH37l0wxjBkyBDDLnrDhg1x4cIFw4qa9e3bF2VlZYbZp0Aph6q1Ymdl7anosy+trBc3b94Uet3d3XHw4EHdjt6+/INRdvbs2RMZGRmGsVYRGT/aUEoz1FSJUQ1at24tro8RRDsNGzYUvfgVK1aAiLBs2TJd617Onz8PxhgAGOrgiayL8RQnb0RJ6d9++w2yLGPlypWG2Ofu7o7o6Gi1lVRfHkcfGRkJWZZ11Z+vDCtWrDCkp6Rg8uTJhpKZKDYqhNZ66r0oaNWqFe7cuSP01VAZTxUUnQp3bLNmzXQz8NiLUXYqdUSMCtsYPeL44YcfkJaWBsaYao7U6mBPPmKEvs8++wycc3z77beG2ZidnQ1ZlpGdnW2YTiJCt27dcPfuXciyjCVLluhmmbKvn2SUjQEBAWCMqe0svhyOXol/6SnXWhnmzZtnaI+uWbNmyMrKMrRUQ5cuXUTFzs2bNxuic/bs2SK0YkS8l4gwePBgQcDhbMVCZ6A4UCPrySisUm+//bbY1qJFC800iMqIw95GhdBdrd3e3t6CEGf69OmGrir28PAQL+EOHTrA3d0drVu31nwPbNmyBZxzTdUvq0Lbtm1FITKjdBJZuViVnrxRtZ7S0tIgyzLu3btnmJ2Kr/v000/VHPfiO/qmTZuKky8/abJy5UpdQ6Zz587h+PHjhl2kRYsWgXOumbyiMqxdu9bQSTMiwvbt28EYc4gpNm3aFBs3btQ8iThq1ChRfc+oB0lxoEbG5T08PHDy5ElBht6sWTPs2LFDlFUOCgpSrTM2NtbBRvtaRWpHIpMnT3Z2Ak41jhw5gtLSUsyZMwdEJOZQSktLVU+eDhgwAGVlZYaOsogImzZtAmPM0Mns5s2bC1YtWZYRGhqKuLg4xMXF6SoBbbFYIMsy4uPjxbZWrVphz549mnVq7NC9+I7e09MTgLXsqdLjmj59uuBrVGJ5ahs0ODgYZWVlDlX89PYak5OTwTk3NLyk9O6McvQKWXL54ebly5fF/1GbddSmTRtcuXKl1nrzCsNWeWjROWDAADDGhJNX4qEKtPColu/NVybO2quUk64s/FevXj14enpi0aJFqpMRFAYwpYzuu+++63DeagnIhw8fDs458vLyDLveRFZH9/DhQ0MJtydNmuQwAWsPPcQjsizjzp07YsJ0/vz5SE9PhyzLmuYRIyMjwTnXUjr6xXb0np6emDJlCtLT07Fw4UIQEX788UdxMwQFBaFv375gjKkePj59+lSQTNy7dw/Dhw+Hp6cnioqKMHDgQE0XnnOOGTNmOGzz8fHB5s2bUVBQgEePHmm68cuDc47Y2FhNaaCbN28GYww3btwQ23bs2CFCOevWrVOdPaD0DBljOHLkSAV7taTEVeYoFdi/ANToVJz84cOHMWLECOTl5SEzM1NQ6Q0aNEiVo69skrh8715tuqWScGDfSxw9ejSysrKQkpIiyuSqIQwZOHCgQyxZiS0vWLAALi4umDRpEhhjePfdd50+94kTJ4Jzjnv37lUgTCkrK9OUZqh03uwrYCqduu7du6vWpyAsLMzBuefm5ope85MnT1Sdt4L58+eLdEolJGb/P9RGGtq1a4c7d+44dBgCAwORmprqUC2zivvoxXb0SurS/PnzQWTtjSqpTFFRUQgMDERSUpKmXgXnXNQgt48HRkVFaaZX45zjvffeE9+HDRuG+/fvY/PmzdiwYYOmSdqqHD1jDLNmzVLNJaqQZNgz+pw5cwYZGRlo164dxowZo9rR2xMkKDh+/Ljg/dQyWWUvlY20tDj6U6dOgTGGDz745JRiiwAAIABJREFUQOQonzhxQvw+ZswYVemG5akdFaduv09l8XtnrndGRgaIrKGMp0+fCruCgoIqOMOaYO/oW7ZsicLCQjDGRIpuvXr1VDv6kydPOjj36Oho7Nq1S9yfnPNKaQarw/Tp0wX3QGxsrAPhTEFBAXr06KE6rbhBgwY4duyYgxNOTk5Gjx49QKTd0a9cuVI4+uXLl4vO58GDByHLMqZOnapK39y5cys8Jzt37hTnn5ubW93I68V29LNmzXLIdVVSmRQeUq1vTyKrU1bCNvaxUC2kxvY6P/vsMxBZyYdLS0uRkpICIivhhxZHr1xgxqxsTfv27XN4ANSmWirHKfSH3bt3R0FBgQiLxcXF6XL0eXl52L9/P1q0aIHXX39dt6Ov7DfFwap19Ip9Ss9+06ZNIszm5+eHxMRETTYqTryyxWfK6EPt9WHMyudbXFyMXbt2id87duyo2tFHRkYKnfbplcrvCluUGoc3cuRI4dC7d+8uJo2bNGmCMWPGoLi4WPVITkn5VZ7ryj5PnDhRlc6AgACkpKQ4OHp7Ksm7d++qvjeJ/nL0ly5dwuHDh8EYQ0hICEJCQvD999+r1qekJSvfGzVqJJ7z1atXo6CgAIyxqghyXmxHv2TJEuHog4KCkJ6ejh9++AEPHjxAdnY2Xn31VTRu3FjThQJQ6UTM+++/rznu+OTJE0RFRUGSJGzevFmEgBRi6/DwcNU68/LyHB7+W7duoWfPng7b1OhTcrOLi4vRu3dvoSMyMhLFxcW4ffu2ahsVR3/+/HkHZiGlZndpaamm61NV7xjQNkHLGENycjK+/PJLpKamYtKkSZg0aRJKS0vBmHVxjl4blReQ1vDSiRMnHK6t8hKxHx1t3LhRFVmKMpKJiopycPTt2rVDYGAg4uPjVYc+lRh9+fN3cXFxmPdQy7i0ceNGyLIscvLt9SqO2tmMlAEDBiAvL08cFxkZiXbt2mHjxo148OABjhw5ovoeUrBq1SqH6wRAfFY6omqg+IgOHTqAiPDKK68AAEaPHo3Tp08jJiamSmJ7etEdvbIStnfv3vjss8/ERVOGSVovEpG1912Zoz9y5IimCTmivyZjx40bByLrHENISAg455qcPJFj/Luym0pteqgSo2fMmmZXXve0adNU26g4+tTUVLHNzc1N6Lx+/bpqnYoocXl70eLkiayOvqysDN9//32F8/7qq69UdxrK26gn20bBW2+95WDX48ePcevWLfH90aNHqlezKo5+//79cHNzEy+T9PR0FBcXa8o8e/PNNwX3qv0I8Pjx46Knr2WNwqBBgyp9Sbi4uIjtahb43bt3Tzj6zMxMLF68WHw/efKkpvuIyDpysR8lKCMOrX4pJCQEJSUlePz4MbZv344ff/xRPOcWi6WmEO2L7eiVsEL5iQ4lRq/1IhERMjIyKqRBNmjQAImJiarj3gqGDBkCi8WCkpISPHjwAJmZmWKCViuFmY+Pj6jvU5mj/+WXX1Tpa9euHZKSkio4OsYY5syZo6m0wvr164WO119/HT179sT+/fvBGHPISlCDqkRPZtTWrVsrPe/XXntNU/mLqkRL/rz9PfjFF19UOu/x008/aUq5/O6774SOsWPHIjs7W3yfP38+bKQ/qjFu3DgUFxcjMzMTkZGRiI6OFk6+tLRUc3ro6dOnIcsyLBYLevfujd69eyM6Olo8/x999JHTuoYNG1Zptk1cXJwIX2pFUVGRg6O/ePEiQkJCNOuzz7Czf85DQ0NrOvbFdvS1jcmTJ6OoqEjEbpXMnucVs2fPxunTp3Hs2DHMnj1bd561QrxuxMrYzp07V3BORi9wM6ENDRo0wPTp00UYMDs7G0uWLDFsMdb48eNx7tw5cM6xYcMGQ+rneHp6ijCO4kh79OihyeaQkBBMnDgRCxYs0PSyqA7l5xHrCCaVoClVS3R0NHl6etI//vEP4pwbonPr1q1UWlpK6enptGjRIkN0mmKKKdWKU1SCpqM3xRRTTHlxxeSMNcUUU0wxxXT0pphiiikvvdTo6CVJaitJUqwkSUmSJCVKkvSFbXtzSZJ+lSTptu1vM9t2SZKkbyVJuiNJ0g1JkrrV9kmYYooppphStTjTo5eJaAaA/0VEvYlosiRJnYloNhGdAvA3Ijpl+05EFEJEf7NhAhGtN9xqU0wxxRRTnJYaHT2ATADXbJ8tRJRERK8S0RAi2m7bbTsRDbV9HkJEO2xpxReIqKkkST6GW26KKaaYYopToipGL0mSHxH9nYguElErAJlE1pcBEbW07fYqEdlT1f+PbVt5XRMkSboiSdIV9WabYoopppjirDjt6CVJakxE+4hoGoCC6natZFuF9EkAmwD0cCY1yBRTTDFFrfTu3ZsWLVpEly9fJlmWSZZl+vTTT+varGpl8uTJxDmnX3/9le7evUu5ubmG6HUqj16SpAZE9F9EdALAN7ZtKUTUD0CmLTRzBkAnSZI22j7vLr9fNfqdzqOfMWMGubq60n/8x38QEVFycjJ16dLF2cMNlbZt2xIRUZ8+fcjf39/ht7KyMvrP//xPSk9Pr+xQ1eLr60sjRoyg4cOHU1hYGD148EDV8ZIk0eeff+6wrUWLFjR37lyaNm0aHT58mNLS0gyxtbbEz8+Pxo0bR3PnzqXmzZtTfn5+lft6e3tTVlYWbd68udLfJ0yYQADowIEDlJOTQwcOHKATJ07UlumaxM3NjYKDg2n58uXUrZs1p0GSKutH6f8/RESMMZJlWZMOSZLoww8/pLfffpv+/d//ncLCwmj37t1GmqlZ3Nzc6O9//zvNmDGDBg0aRNOnT6cNGzbUyv+6evUq/f3vf6eDBw/Sv/3bv6k69q233qJjx45RaWkpNWnShDp37kwJCQlUv379av+lU51lJ8oTSES0g4jWlNu+gohm2z7PJqLlts/vEdEx23G9ieiSESUQ+vbti71796KsrMyhbsXTp08NWUpcv359dOzYER07doSbm5tTx5QnXagMeksVtGnTRpCOK1BTj8fb2xtjxoypUK61PNLT07F48WJdtrq6usLLywuLFi0SnKKccwwbNkyzTnv7FQYfWZaxdevWao8LDg6ustytLMtITEyssF3vPRQeHi5KDdy6dQvh4eEIDAxUrScwMBDh4eE4efIkGGMOpamNuNeJrGUGZs+ejWXLlonrtHTpUlU6goODsXbtWqxdu9ah5hFjDCkpKdVVXKwSCtnI9OnT8fjxY4faL9euXdNVvkAhCSkqKsInn3xiWFv6+Pjgq6++wtWrV4W9Wqrgrlq1ChaLBW+88Ya494uKimpqR2Nq3RBRsE3hDSKKt+FficiLrNk2t21/m9u9GNYRUSoRJRBRDy2O/u2333aoPifLMjp16gQvLy906tQJ9+/fF9vVNmhYWBgWLlyIuLg4cM5x5coVzJs3T9zwRUVFTulp0KBBlb+5uLggLi6uQslVZ/DWW28J9p6NGzeKejRBQUGq69rbO/iIiAh8/vnnFZCVleXQzgMGDHBaf5s2bTB48GCcPXtWOKWnT58iJSVFlCresmWLKpslScLs2bOrfTG99tprhj2okZGRqu+j9957D+Hh4Q7E0+UL8MmyjCdPnjhNPK4U8WOMITY2FjNnzkSbNm0wZMgQzY7excUFPj4+6NatGw4cOFBtp+Tzzz+vUZ/CSGWPdevW4ZtvvsHAgQPh4eGBc+fO4dy5c05VBO3fvz+ys7MrbT9Fj1LFVevzXv7ZkmUZ9+/f16Vn7dq1FcqI21cZbdiwoWqdjDH4+fk5bNu1a1dN1IovdlGzmJgYcWELCwsr3ITKb3fu3FHdoEotbc459u3bJxpS2b5hwwZdNwGRtZQp51yw2ajBiBEjcODAAQwdOtShsqIWR9+5c2ds3boVsizjxo0bVdYe79ixo2pH7+Hh4VANUektDRo0CB4eHhg3bhwYY1i0aJHT9vbt27fCi2f//v1YvXq1w3a918ceAJx2ou+//z4WLFggyKErK1f7xx9/iG1paWlO29GvXz/Rm7Xf/s0334Bzrqnksz0PrYLCwkKMHz8e48ePR1RUFDjnOHfunFNFwyIiIkT56bi4OAwcONCh6qnCguVsB2fEiBEObTV9+nQMHz5cUH0qUNpWLW9AeXh5eYlOopbj/f39sXbtWod7/uHDh9i3b5/4/uWXX2rSHRMT49B5lCQJe/fuRaNGjao77sV19O+++y4KCgogyzLu3r2LPn36VDhB5eZYtWqV6gYtLCzE+fPn8d577wlH+uWXX4JzjtTUVPj4+Oi6mYj0OfqqcPToUU1MVQ0aNBCV9ubOnVups1ccfWZmplMcne7u7g717e/fv4/FixfjnXfeARFh8ODB4jeFl7UmhISEIDc3V1zbGzduYPny5eJ4++1GtWlAQICq0I09mYV9GOjEiRPo0aMHevTogZEjR4rfZs+erdtGhTBm9erVTu2/du1a7NixAzt27HBw8DExMRg9ejTeeustsa8ykp01a5ZT5Zrd3d0RHByMV155pdLfhw8f7kCiURP8/f3x/fffV8uF0LRpU3GNvvrqK93tuWfPHk2O3t/fH6mpqeK+Tk9Px44dOxAYGCjCbMeOHauU68IZHDhwwOHYVq1aOdMBeXEd/bBhw3D16lWcPn26womFh4eLXpNa2rvK8OWXX8JisYBzbkiJVQXTpk0D5xxNmzYFkXWoP2zYMAEtOq9fv45Lly4ZZqOCzp07IzU1FbIso0mTJrr1ubu7gzGGnTt3qib0ePPNNyts8/b2xrVr13Djxg2MGjXK0HO3WCxgjKFbt25O7T9o0KAa9/35558hy7JToRBnwDnHb7/9pmr/LVu2YOjQoRg6dCjatGlT6X7z589HYWGh06GlmrBkyRIUFBTgzz//NOz6fPbZZ5BlGUlJSRXCGlrQtm1bERbScnxAQEAFfgnF8duT7xiBjIyMl9vRV4X3338fxcXFkGW5AnGIVihMOVpCQFXdCP3790dGRgY458jPzxdk1AUFBYiLi8MXX3yhWm+bNm1QUlJSFUGwLtizGOnV1ahRI6SmpiIhIcHpnnxNOHr0KBhjmklhqoMyGjFK95o1a/D48WPDwktBQUGQZdmBeL4mhISE1OgUu3TpggsXLoBzrpkYpzx+//13ZGVlGTaK7d+/vxhBqaU7rApKjF4N7251GDlyJBhjiI6O1jTxrqBevXro0KEDOnTogKCgIPz6668ArOHl5s2bV3fsy+fojZqQUfDee++Bc4779+9r7s03atQIISEhiI6OxvXr11FcXOwwXM7Ly0NiYiJWr16tawJRybwxgiikPJSXpxEhEYUOr7JwmxZMmjRJhPGMPu+AgADIsuwMi49TcHNzQ35+PkpKSlTNS1SHbdu24eLFi4af+5YtW1BYWIirV68aoq9x48ZgjOlmf7PHkSNHIMsyUlJSDNOZn59vWAjIy8sLN27cQF5ensiUUYsmTZqgf//++Pnnn0Vnyz7L6smTJwgKCqpOx8vl6F1dXYWjnzx5sq4L9P7776OgoACccwQEBGjW88orr+DWrVuCMrBly5bw8vJCu3btRI/eqBs0NjYW165dM0yfgjVr1kCWreTJenWVlpaitLTU0HkJZd7g9ddfN/S8vb29ce/ePU2jq8qgkJf/8ccfmtIKK4NC4K6GJ9UZ9OnTR3AZO5tKXB0aN26My5cv48MPPzTEvkGDBiE2NhZJSUmYOnWqoedeXFzsMEehFUr2kZY0SnsMGzYM4eHhDiPK+vXrgzGGt99+2xkdL4+j9/DwwIQJE0TcV4l7a8Uff/wBzjmysrI0c2YqCAkJQVhYWIXt169ff+4d/fjx48EYw927d3W98JRrxBjD9u3bDbGtadOmiI2NNcS2yrBhwwbDwkGenp6CltIoJ+/i4iJ4WI0+94MHDxo6Oly+fLlhdnp5eeHcuXOQZRnh4eGGnvdnn31mWP68klo5btw4w6/PwoULcfLkSWc5nF8OR+/p6YmffvoJsixj6tSpmoic7dGiRQsRrunbt6/hF0mB0Y6+qKgIMTExhunz9vbGxYsXIcsyevXqpUtXjx49cPbsWRw+fNgw+wYPHgxZlrF8+fJauT4PHz40LBz0yy+/iAQBo+zr2bOn4YukiAjbt29HYWEh3njjDUN684GBgbBYLE6vPakJ9qmpRs0deHp64uTJk4bZGBoaivz8fBw6dEj3gsjy6NSpEzjn6N+/v7PHvByOfvPmzYbkzyro2rUr9u/fb+jFqQxK1o1R+jjnNcXqVMF+IZUePaGhocjLyzM008I+xbI2rs2xY8fAGMPPP/+sS0+XLl0QHx+P/Px8Q8NVRNZVkowxHDp0yDCdc+fOFXNHRuksKSnBtm3b0LZtW926lNBs+Rx6PfDy8hIhli5duujWd/jwYTDGMHLkSEOvN9FfKcnZ2dlqjns5HL3ywBuVZUNEVeYAGwmjHX1eXh5effVVw/QpvcW9e/di8ODBmvUkJCQgIyMDPXv2NMQuSZLEEvraujaJiYngnOual2jSpAl27doFWZYrTQnVi6tXr4JzbljqI5H1Bco5x/z58w3R9+6774Ixpjvl1dPTE7GxsQBgWHYNkXWOY+XKlWCMYcqUKbr1hYSEGDp6KY9p06aBMabW1734jr5Lly6QZRnfffcdOnXqVCuNW1to3ry5YZknRIS7d+8aap/9gh+toZtmzZqBMYaIiAjD7Jo3bx4yMzNVlWHQcu4PHz5Eu3btNOvYtWsXLBYLYmNja8VGJfPCqIwgRaeyOMoIfadOncL06dN1h1M3bdokVhYbda6DBg0Sq5cHDx4Md3d33TpzcnJQUlJi2KRzeYwaNUpLFtSL7+j79++PIUOG1EqjvkhYvHixszPwTkNx8lrziZs2bYpz5845vQLSGSirc41YtFUdsrKywBhDQkKC6mNdXV3xyy+/OL3ASiuuXLmC9PR0TJw40TCdrq6ucHV11e2YFVy7ds2QRYu1GaYzCl999ZXh8XiD4JSjd6pMcW2LmjLFpjwfsmHDBho8eDD5+vrWtSnPVA4dOkQ+Pj7Us2fPujblpRHZVhrZxcWlji15IcWpMsWmozfFFFNMeXHFKUevikrQFFNMMcWUF09MR2+KKaaY8pKL6eifgcycOZMsFktdm2HKM5b8/Hz64IMP6tqMWpWOHTtS3759adu2bcQ5J845rVu3Trfebt26UUBAgAEWvlwSHR1Na9euVX3cC+fofX19qWvXrtSuXTsaO3Yscc7J09NTsz7GWAXk5ORo1ufl5UVFRUWCf9PPz4+IiDw8PDTrJCL65ZdfqLi4mE6ePEnZ2dn02muv6dLXtm1bWrhwIS1cuFDMzE+ZMkW8lC5cuKBKnyRJov0sFgt98skn9PTpU5ozZw5FRUVRTEwMvfLKK6rtHDlyJF2+fFm0pwKt4uXl5aBr7969FBoaSm3atCHGGA0fPly1zvDwcOHk7OHp6UnNmjWjevXUP2b9+vWrKkNNtXTo0IFiYmIoJSWF4uPjaenSpbR06VLatWsXZWVladJJRLRgwQK6desWxcbG0pgxY2jr1q20aNEi8vT0JM45vfHGG5r0+vj40OnTpyk5OVmzbYp4eHgQY4xyc3MpNTWVOOdUWFhIGRkZ1KNHzVSr1UloaChdu3bN4b7cunWr+P3XX3+l3r17a9J9/vx54pyLZ+r48eOUmZlJmzZtoi+++EK9wrpOrawuvVJBYGAgNm3ahBMnTuDevXuCtejAgQMAoCvtSVk4dOXKFRw6dAiHDh2CxWLRpCs4OBgnT550YBsKDAzEjRs31K52A5E1Tz00NBSffvqpYNrq378/0tLSdK0UbtiwoSCzqAolJSWqdLq6uiItLQ2hoaFikU9paSksFgvy8vLQuXNnVfqCgoJw/PhxkXp3/PhxfPLJJ4iPj9eVijdnzpwKPLJ5eXkYOHAgGGOqV2W2atUKd+/eBeccJSUlYlFSUVERJk2ahPz8/JrKzFaK6kTtPVlQUID4+PhKGaSaN2+OyZMna1o1mpycjDt37mDt2rUV6rJER0drIl2JiIhAenp6hcWGoaGhOHLkiGpSoNmzZ+Pu3btiHc7+/fvFPf748WMMHTpU873022+/QZZljB49Gr1790bv3r3Fugd/f3/Ex8erJiFp1qwZPv74Y3GPxsXFOVSyrOSYFz+PXsHUqVMdqOq2b9/uUNJT64Uisjr65ORkQSfYqlUrpKen4+OPP1ZVoEpZNSfLMqKiohAVFYUbN25g5cqVePr0qeoSAQ0aNBBOwx4WiwU3btxASUmJprLHr776KhITE4WDW716tSjwpeDs2bOqV3seO3aswurY0tJSMMZw8uRJVbrGjh0ratEcPHjQYXVos2bNkJ2drao+u4L169ejqKgIhw8fRps2bdCmTRswxrB+/XqEhYVpcvQzZsxASUkJFi9ejPfeew9hYWHgnItCdyNHjtRUU0aRiIgIURlTi6PfsmULCgoK0KpVq0p/9/T0xJEjRzTVlUlOTq5Sb6NGjdCrVy8EBQWpqlekUCbGxcU5bH/69CmioqJU27lx40bMmTNHfG/cuDHat2+P9u3bC5YoteetQHH05bePHDkSaWlpNXG9Vnk/Mcawa9cuwdamOPkqXkovh6NfsWIFOOe4d++e2NaxY0fh9PSWRlBqS7z55pu4ePGicHZlZWWq6lkob+BNmzaJbW+//TZkWVZd7Ktr164oKyursL1p06bo0qULEhISNNEdrlu3DoyxCi8deyevdYFaXFwcfv/990p1qiVKTk1NBQBER0c7cGi6ublh27ZtmlZQdu3aVVRFLN+mSUlJujhEHz9+jAEDBqBRo0a4evUqmjVrpuueJPrL0ffr1w8RERGanLyCPXv2gDFWoUcfERGB4uJizTbOnTu3yt+SkpLE6l5nygIr3BDleSFWr14NzrlTfLblERwcDMZYlcULOedV9ZKdQnlHP2jQIMiyjPj4eE2r4m/evAnGGBYvXgwia7FAJ57Jl8PR9+rVCwkJCWCMoVGjRvD19cXhw4eRk5NjSJ3uqkIXalckLlu2TIQDgoKCEBQUhPT0dKSnp6suRhYQEICnT59WqMnj4eGBqKgojB8/XtO5Ko7envJM6dUqqIGIuEo0atQIsixj27ZtSEpKAmMMhYWFmkrhzpw5U7w47av4hYWFifr0WmyMjIyELMvw9/cX265cuaJrZWb79u3BOUdubi7Onj2LzZs3674nif5y9EoNGD2Ovn79+vjiiy+wZ88eEeaMjIxEaWmpZiJr5T61D9nUq1cPM2bMQEJCAjjnOHHihNNlgRcsWADOOa5evYrDhw/j8OHDGDRoEIqKinSN2h8/flylo9fbo586daq4bzZt2oS7d+/ip59+0kxipDBVNWjQAOvXr8fjx4+dCSO/HI7ezc1NkFAPHTpUxOiNKFKkNK494uPjNS3rbtCgAQ4dOgRZlpGVlSVCD1qXyi9btgzZ2dmYOHGiGK5u2rQJaWlpms+1b9++yMnJAWNMvERiYmLEuWvl0aysLS0Wi+aCXH5+fjh9+jRkWcaJEyeEY87OzoYsy5oLqPn5+SE9PR0PHjyAv78//P39xQvl66+/1nze9iE2vVwJCux78fZOX4/O/Px8PHr0CAMHDkRGRoZuAo5mzZrh/fffF98V8nrOOaZMmaIqZBUUFISdO3dWCFVyzrFjxw7NNu7bt6/SWkz9+vUDY0wXeXvXrl2Rl5eHtLQ0FBUV6Zo78vb2BmNMlCQpLi5GaWkpQkJCajr25XD0RI4xes45oqOjdd2gLi4ueOedd3D//n0Rq96zZ48unQpCQ0PBGAMAXL9+Xbe+Tz/9FHl5eeCcG8KM4+7ujlGjRjk45dzcXGduqGrRrl07WCwW8SIxoi19fX0dJk0XLlyoW6cygjGyvsqjR4+EU1q3bp0hOon+6tUDMKxw3KJFi8AYM4Rom4hw+fJl5OTkgHOO4uJiwbRmhG57Z69VR69evUS4csaMGXjttddw4cIFMOY8IXxViIyMFMkGRhSfO3XqlDhfFZVVXw5HX79+ffz888+iAfQ6eSJrkTB7R2dUCeTu3bsjPT0dmZmZyMrKQk5OjiF6f/nlF3DO8fnnnxuij8ix961lYrM8kpKSEBQUhA8//BCMMUNILYgIGRkZ4sVphL62bdsa6ugbN24MzjkuXryI8+fP4/Hjx4a0J9Ffjt4oJ+/l5YXc3Fx88cUXSElJMaRI17p168Szad+714tmzZrh/v37CA8P151wcefOHWHjn3/+Cc657tHr1KlTBXG5Xl0KvvnmG/FMXrp0CStXrnQmlPpyOHp70lzGmG4Wo+7du1cI1xhRGVKJySsx+uXLl+t2JPXq1cOmTZvAGMPp06fBOcdnn32m29ZGjRo5nL8eXR06dMCaNWtEeVUPDw9cuXJFDUNOtcjJyTGshK2vry+2bt1qqKMfPHgwbt68iSZNmgin/+TJE11E8ES105vfv3+/yICJi4vDlStXNKV+2uPatWuGk5kQWQmHJkyYABcXF928rH5+fvj9999FRIAxhvz8fF06lfvn0qVLhjn6zMxMHDt2DB9//DGys7PBOcexY8dqOu7Fd/SjR48WJMaenp5gjOGHH37Q3JBKWmF0dDT69u2LL774wjBHrzgPZYZccfRasmMUPH78GJxzwd7Tq1cvWCwWXTynkZGRYIxh37598Pf3101PmJGRgZKSEod84cGDB2PVqlW69Hp5eQknb4Sj37t3L9LS0vDw4UPRDuUnZtVC4Urt3r272MY5R1pami6SmPLxeb33pr299t8/+OADTe360UcfQZZlcM6xfft2dO3aFa1btzaMaH3y5MnixeHr64vCwkLNugIDA4VjX7t2LYgI8+fPB2NM03xKaGgoLBYL2rRpAyLC9OnTdTt6d3d3bN68GWPGjBHbvvvuO3DOkZGRUdPxL7aj79evH548eYJBgwahYcOG8PPzA2NMMzvOqFGjUFZWhov715kWAAAZlElEQVQXL4qUvV27dhni6EePHg0AWLNmjdimZLjo0cs5x+3btx1yhz/55BPND9S8efOQm5tbYTJWj42MWRev2W8LCwvDvHnzNOvs0aMHjhw5AlmW0aVLF1E/Xq+dsixj6NCh8PT0xLVr13T16D08PHD27NkKPVnOua5zJyJDMm0qQ3ki6/r162PHjh2qM61KSkrAOcejR49ARPDx8cG+ffuwevVqQ+w8fPiwaNcJEyboGi0MHToUFovFYYRZr149HD9+XNOcV/mUyu+++053YsiCBQscnsk2bdqgrKwMZWVlzoSVX2xHv2jRInGBGzRogM8//xycc83x9Lt374IxhvPnzzvcsJmZmXj99dd1Xahdu3ahtLRUxGa7dOkCxphD7r8WcM4xc+ZMh22tWrXS/LJ7/PgxGGM4ePAgiKwz/VrINxR4eHjg3Llz8PDwcNiut4cTHR0NALh16xZGjx4tnLRWfUp2zYQJE+Dq6mqIo/fx8QHnHHfu3BHbpkyZgpiYGIfcf7VQnLy9s9fTlvYo7+iJrIQaajo6QUFBIlTz5ptv4vvvv8e9e/fAOTeMeUlx9B4eHkhOTtbl6H18fPDNN99U2H706FGMHTtWla7u3bujqKgI69evB5F1ZFBaWqqLvapFixZ49OgRbt68CR8fH8yfPx/3799HYmKis2FaYxw9EbUlolgiSiKiRCL6wrY9gogyiCjehn+1OyaciO4QUQoRDdDi6Ddu3AjGGM6cOWNILNk+w0RJ0bR/i+rBjz/+6BD3VegP9S6cKSkpcSBd9vf3x+7du/Hpp5+q1jV//nxYLBax2m7cuHEoKirSnEfcuHFjnD9/Hh988AGIrDnVSihs48aNus5blmXk5ubi3XffFRNeerJEpk6dCovFgjVr1mDs2LHiWlXmAJzF06dPRbzXPjtEy8IeBeVXwOpNpywPi8VSYV3DtGnT8MYbbzitQ1k3YI+0tDSEh4cbZueWLVvAOUdZWRlyc3N1TRoPHToU33//vcO2t956CxaLRTWV5J49eyDLMn788UfxnCtOXyvKzxempKSoXSdjmKP3IaJuts+eRHSLiDqT1dHPrGT/zkT0BxG5EdG/EFEqEdVX6+gVolzOOXJycrBgwQJdDfrnn39WaFStNW3KQ5nglWUZFy5cMCx17eHDh0hISMCMGTOwZMkSsRpYi66lS5fi8OHDmDZtGqZNm4aSkhIwxjBo0CBN+lq2bInU1FTk5+fj9u3byMnJQW5uLubPn696JWx52NcKkmUZN27c0KVPSc9V9FksFowePVp1HRJ7hIWFOTjlXbt26eY1Lu/o+/XrZ8h9pGD//v0oKyvDN998g2HDhmH37t2a7id7Jz9hwgTNi+yqQseOHXHy5EkkJCSgR48eunQFBwejtLRUlPRwcXHBuXPnwBhTTamozOsoiImJ0Z25FBoaiuvXr+PPP//EuHHjtDw7tRO6IaJDRPQOVe3ow4ko3O77CSL6v9U6+mbNmuGrr75CbGysIStgAwMDkZmZKZz80aNHDclLr028++67IkeZcy4mULXoWrp0aYUX3TfffKOrB9qxY0eHxVZGjI6ICAUFBcIxf/zxx2jcuLEufR4eHkLfpUuXdE2Q1zaUiVijnTyRtfCckkfPGMOePXs0rVx+0aDk0f/www9i1fbjx49V63Fzc8PMmTORl5eH9PR0tG/fvs7PjWqDM1aSJD8iOktEgUT0v4noYyIqIKIrRDQDwBNJkv4PEV0A8J+2Y74nomMAfqlGr/NGmGKKKaaYooixVIKSJDUmon1ENA1AARGtJ6LXiCiIiDKJaJWyayWHV3DkkiRNkCTpiiRJV5y1wRRTTDHFFPXilKOXJKkBWZ38jwD2ExEBeAiAAeBEtJmIetp2/x+yTuAq0oaI/iyvE8AmAD2ceRuZYoopppiiXWp09JIkSUT0PRElAfjGbruP3W6hRPRP2+fDRPSRJElukiT9CxH9jYguGWeyKaaYYoopasTFiX36ENFoIkqQJCnetu3/JaKRkiQFkTUsc5+IJhIRAUiUJGkvEd0kIpmIJgNgRhtuiimmmGKKk6I266Y2QHU/c23CxHOF5cuXCyKcrKwsTJ48WddCLBMvLZzKunnhyMFfFGnfvj2tWbOGOOe0a9euujangrRv357i4uIEkXViYiJ5e3vXtVlERBQUFEQ5OTnEOSdZlikkJKSuTXqmsnz5cho7diytWbOG3N3d6W9/+xuNHTuW5syZU9emVRBvb29atGgRcc4pODi4rs0xpSqp6968sz36Xr164caNGyIH+MMPP0RUVBR69eql6404adIkpKengzFm2Oq+cePGIT8/H1euXBEFlAICAnTpDAwMxMSJE8E5BwAcP35cVw35ffv2iYVJSo65E5XyqsWJEycAQOQtR0ZGqqmrLTBlyhTB2rRu3Trk5eXpzlkOCgrCvHnzRO34yq714sWLER0drZkwJSAgAA8fPgTnHImJiZp0KOUzyl+LrVu3OqyS1oJOnTph3rx5SEtLE/cR51w1A5oCFxcXPHjwQNxD6enpCAwM1F3nnchKLXjmzBmsWLECK1as0K2PyFryAwBOnTplSGlyPz8/7N27F5xzJCUlYevWrZrWfLi5uaF58+ZYunQpli5dip07d6KkpMTZuv4vdq0be3z44YeisNWpU6dw48YN4aQURhYt6N27d4XSBVu3btV18V1dXVFcXIxFixahadOm6NatGxhjuqoZjhgxAoWFhRUWPKklHFcwatQoWCwWXL58GRMmTMDAgQPFAiot+tzd3TFgwADRhkq5Zq2lgBVHv2zZMsyfP79Kx6wGT58+dWg7WZYdKOaaNm0qftNCrzdnzhxBDq/o10JGodAolucJ7du3L7p06aLZQXXp0qVCGyh4+vQptm/frlqnUrgvKSlJlL/QW64kMjISO3bsqHC/f/zxx7quv7e3N3bt2uWwqrd8jSY1GDJkCNLS0vDkyRNcvHgR8+bNQ1JSkkNhQ2fw6quvYvfu3eI8ExIS8Mknnzjcm23btsWmTZuqqn3zcjh6xVnY93BGjRoFWZaRlpYGb29vTReqMgb3R48ewWKxoGvXrpp0Nm/eHNnZ2Q48mnfu3NHMcUpE+PXXXx2IPFxdXcXK0coIxGvCnDlzHIpEvfnmm+Cca67ZoXCuZmdnIzAwUGxPS0uDLMuaiGIqc/RbtmzRZJ/CZ+Dl5VVlbyssLAzFxcV49OiRqiXo3t7euHz5MhISEtCtWzd4eHggICBALJXX4kg6d+4Mxhg2bdokarEoZWxlWdb0ct+wYYMoz+3l5SWKcLm4uAgGpl27djmtLzQ0FEVFRaJdle3Hjh3T7Oi7dOmC7OxsMMawe/dubN++XXCv+vv763qGrl+/jtLSUvz444+QJAktW7ZEXl6eqNOkBeWr3tavXx/Lly9XVfpYWbGdnp6OwYMHO/z26quvIjo6WnQcHj16VNWq+Bff0SsPzLfffuuwvD4pKcmh9rsWnDx5slJHn5GRoan37eLigqSkpArVIDnnuoaeCu1ZvXr1MGjQILz33nviYdIy+rh8+XKFB5Expqn0cePGjQVHrv1SeldXV/GC1rLE/pNPPgHnXIzeOOd48OCBpvZTavpU9fupU6dE71EtQ5JSeM9+m1JWV0+v9u7du2JkdPz4cdEpkWUZv//+uypdAQEBomqpfflkb29vnDp1SoyU1YxiFCdvXwlWkiSHEYPac05NTQXnHHfv3q1A2rJs2TJdjl6WZVFC2dfXF7/99hs455prHb322muIiooS5UPc3NwwcuRI1eetlBApXz3U29tb0JwqpdSjoqKq0vNiO3oltn3w4EHRA2nYsCEGDx6MwsJC3UXOTp06BcYYGjZsiC5dumDFihVISEhw6JWqwUcffQTGmMOQa+TIkSgoKECrVq0023no0CHx1k9OThYP0r59+zT1GAMCAsA5x4YNGzBgwADs27cP9+/f10RmEhQUBFmWHQqjBQYG4vfff9fl6F1dXbF161ZB+bZnzx7NpWqVzJXypWQHDBiA8PBwXeGGuLi4Cp0FhQ5Obwnkbdu2ORCvpKenY9u2bRXKVteE27dvgzGGO3fuoEOHDhg1ahS8vb3FvBRjDKGhoaqcnnKcUgmVyEo2o7Utx4wZUylLVZMmTbBixQpwzpGVlaW5PTnnmDFjBj788EPBv6yn9PHs2bMxceJE8V1x8lodvVI+Ojg4GF9//TUyMzORmJiIsWPHwtXVFQkJCdWNNF9cR9+yZUtxg7ds2RKzZs3Cb7/9Jh4gLSGL8khMTBQ9OVmWUVBQoEsf5xxz584FEeGVV14R9hpR6MvV1RVNmjTBhAkTRE9eb6rdzp07cfnyZdy8eRMTJkzQpGPPnj24fv06xo0bh3HjxiE7OxsWiwV+fn6QZVkw+uiFu7s7jh49infffVfT8S1atMDVq1fFwxgbG4sDBw4gLi4OjDGkpqZqYhtSUh8nTJiACRMm4NixY2KorWWE5O7ujoiIiGqrQart2TLGkJOTU4HaUWmL1NRUTeetcC1ERkYiIyNDsKkpnRI1+hTOAaUa5okTJ5CcnIzbt2+LEWg1PdoasXnzZsTGxmL58uV4/fXXwTnH06dPNesrT8VZXFzsLElIBQwfPhzx8fH4888/sXPnTgeWKSJCaWkphg4dWp2OF9fRN23aFJmZmQ4TevaTfHqGcfawn4zdtm2bLl2cczEauH//PjjnuHXrliF2Ell74kqP3ghSZwWMMc2OPjg42OH6XLhwAS1btkSjRo2QmZmpO9PIHsOGDdPFndqiRQt0794d3bt3F460T58+YIxh6tSpmnRevnwZFotFOCmFqlLrCGn06NGQZbnaXrvaGP2TJ09EjHfJkiUYP3489u3bJ5yUlnNXji0tLQVjDFeuXEHfvn3RpEkTTY6+T58+KCwsFCEv+9Bkfn4+GGOqRzJVYdWqVeCca5o7skdERAQ++OADdOzYEaWlpYKi0kh4eXmhtLS0pv1eXEdPZE2nfPToEVJSUrB8+XJ07twZZ86ccYi3GQHF0evhDiWyxi1v3ryJ9PR05OfnA4BhHJpEf8WDS0pKDNMZGhqqawjr6+uL+/fvC0evONDQ0FBcuXLF8Bv/+vXrhi4aGj16NG7duqXrxRkQEID169cjLCwMHh4ekGUZO3bs0GxPTY4+NzdXVZnlnj17inme8sjPz3cINToLex0JCQkiDVBx9E+ePFGtMyQkxMHRX7hwQXTEyodE9aCsrAycc7Ru3dqw+4gxhhkzZhimT8HKlSudCQe92I6+qgaVZVlzpk1VOk+dOmUYWYgkSSgtLdVNmGCPrl27QpZlXcPX8lBSKi9fvmzozRkaGgpZlkXGhJHgnGPAgAGG6FLIJ5zMVXYKly9f1hxeIiJMnDixRkefn5+Pzp07a9I/YMAAdO/eHVu3bsXJkycNPXeivxy9noloe9jPoRh5Dx06dMgwfatXr8bnn39uaDsSkWBCc4IYxnT0NSEgIABZWVkYPny4YReoX79+WLhwoWH6fHx8cPToUcMd8sOHDzXneleH+Ph4XROR1YFzbshDpXDGGsUwRmSNVet1SEqsujK7/P398fDhQ00xdQVNmjTBd999h5ycHMOdPJE1dq0wuRmhb8eOHYY6+kGDBqGoqEhzwkV5BAcH4/Hjx3B1dTW8LaOiosAYw6JFi2ra13T0Nd30aWlphg+5/vzzT92MSPa4du0aGGOGrDa0B+fc8Lhi69atxeSUkXrtbTbC0SsO9ezZs4bYFRAQoDvThojQoUMHpKSkgDGGBQsWiEyhdu3aiTx6rb15IsL48eOd7SVqhp4MqfLIzc0FYwxLliwxRF94eLjq9NTq8N133xk62rCH8oJT1s9Ug5fP0QPW5fV6Hf3bb7+NwsJCQ2/4hg0b4tq1ayIbwQg8ePAA+fn5mDJliqE30b59+3QRY1eF5ORkWCwW+Pr66ta1bNky3L59G7IsY8GCBejYsaNhjp4xhqKiIlWk2NVBlmURozdCX+/evSskIsiyjFmzZmnWqSzOOXfuXK305hUMHjwYnHNDQqGccyQnJxvSsZs8eTI454bN8cTFxeH8+fPo2bOn4W3YuHFjMMacpZN8+Rz9rVu3IMuy5lokCnJzc1FYWGho2GLSpElgjBk2LFR6icuWLTN04ig0NBSMMU1ZITVBlmXk5+cboqtDhw5o0KABfv/9d5EO9+jRI0PsZozh8OHDhtg5Z84cnDlzxvC27NmzJ6KjoxEVFYUvv/wSLVu2dFhxrQYNGzbEzz//jOvXrxs6v1UZmjdvjqysLMyePduQ63T8+HFD7Nq4caNhIw0iQnZ2tqHzcPYIDg7GiRMnnOnNg15GR//xxx9DlmWcOnVKcyO6u7tDlmXs2bPH0Itz8+ZNXL9+XfPDaI8mTZogPj4ep0+f1l3Iyh6NGjVCYmIiYG10w+Dn54cLFy5AlmWMGjXKUN316tXDl19+iYKCAkNI4on+Wr6u91qFhobCYrEYHlYzGkonxH5hW20iIiICSUlJhlynTz75RLceJZZ+8eJFQ87Pz88PI0aMqLX2Cw4Oxg8//ICOHTuiXr16Ne3/8jl6IutipKioKE2xyp49eyI9PR1t2rR5Jje8Fmzbtg27d+/Wlb3xrDFixAiH9MrnHVu3bsWGDRt0hxc2bNhQoeTF84ZHjx6BMYZr167VuS1qYVSnoaSkBLt27TJk/cn69et1r8qvCS4uLliyZAmGDRsGSZJq2t8pRy/ZHG2diu1kTDHFFFNMUSdX4QTvtjNUgs9Ccoio0Pb3eZIWZNrkrDyPdpk2OS/Po13Po01Ez5dd7Z3Z6bno0RMRSZJ0xZk307MU0ybn5Xm0y7TJeXke7XoebSJ6fu2qTkwqQVNMMcWUl1xMR2+KKaaY8pLL8+ToN9W1AZWIaZPz8jzaZdrkvDyPdj2PNhE9v3ZVKc9NjN4UU0wxxZTakeepR2+KKaaYYkotSJ07ekmSBkqSlCJJ0h1JkmbXoR33JUlKkCQpXpKkK7ZtzSVJ+lWSpNu2v82egR0/SJKULUnSP+22VWqHZJVvbW13Q5Kkbs/QpghJkjJs7RUvSdK/2v0WbrMpRZKkAbVkU1tJkmIlSUqSJClRkqQvbNvruq2qsqvO2kuSpIaSJF2SJOkPm00LbNv/RZKki7a22iNJkqttu5vt+x3b737P0KZtkiTds2unINv2Z3L97OyrL0nSdUmS/sv2vc7ayhCp4xWx9YkolYg6EJErEf1BRJ3ryJb7RNSi3LblRDTb9nk2ES17Bna8SUTdiOifNdlBRP9KRMeISCKi3kR08RnaFEFEMyvZt7PtOroR0b/Yrm/9WrDJh4i62T57EtEt2/+u67aqyq46ay/bOTe2fW5ARBdtbbCXiD6ybd9ARJNsn/8fItpg+/wREe2phXaqyqZtRDS8kv2fyfWz+3//m4h2EdF/2b7XWVsZgbru0fckojsA7gJ4SkQ/EdGQOrbJXoYQ0Xbb5+1ENLS2/yGAs0T02Ek7hhDRDljlAhE1lSTJ5xnZVJUMIaKfAJQCuEdEd8h6nY22KRPANdtnCxElEdGrVPdtVZVdVUmtt5ftnP8/29cGNoCI/kFEv9i2l28rpQ1/IaL+kiRJz8imquSZXD8iIkmS2hDRe0S0xfZdojpsKyOkrh39q0SUbvf9f6j6h6I2BUQUI0nSVUmSJti2tQKQSWR9gImoZR3ZVpUddd1+n9uG0T/YhbWeuU224fLfydorfG7aqpxdRHXYXrZQRDwRZRPRr2QdOeQBkCv5v8Im2+/5RORV2zYBUNppka2dVkuS5FbepkrsNVrWENFXRMRt372ojttKr9S1o6/szVdXaUB9AHQjohAimixJ0pt1ZIcaqcv2W09ErxFREBFlEtGqurBJkqTGRLSPiKYBKKhu10q2PUu76rS9ADAAQUTUhqwjhv9Vzf+tE5skSQokonAiCiCi/4uImhPRrGdpkyRJg4goG8BV+83V/O/nyYdVKXXt6P+HiNrafW9DRH/WhSEA/rT9zSaiA2R9GB4qw0Pb3+y6sK0aO+qs/QA8tD2onIg201/hhmdmkyRJDcjqTH8EsN+2uc7bqjK7nof2stmRR0RnyBrnbipJklLvyv7/Cptsvzch50N3emwaaAt9AUApEW2lZ99OfYhosCRJ98kaSv4HWXv4z0VbaZW6dvSXiehvthltV7JOZhx+1kZIktRIkiRP5TMRvUtE/7TZMta221giOvSsbbNJVXYcJqIxtoyE3kSUr4QtalvKxUdDydpeik0f2bIR/oWI/kZEl2rh/0tE9D0RJQH4xu6nOm2rquyqy/aSJMlbkqSmts/uRPQ2WecOYolouG238m2ltOFwIjoN22xjLduUbPeSlsgaB7dvp1q/fgDCAbQB4EdWf3QaQBjVYVsZInU9G0zW2fRbZI0ZzqkjGzqQNfPhDyJKVOwga6ztFBHdtv1t/gxs2U3WoX0ZWXsL46uyg6zDxnW2tksgoh7P0Kadtv95g6w3u4/d/nNsNqUQUUgt2RRM1iHyDSKKt+Ffn4O2qsquOmsvInqdiK7b/vc/iWi+3X1/iawTwD8TkZtte0Pb9zu23zs8Q5tO29rpn0T0n/RXZs4zuX7lbOxHf2Xd1FlbGQFzZawppphiyksudR26McUUU0wxpZbFdPSmmGKKKS+5mI7eFFNMMeUlF9PRm2KKKaa85GI6elNMMcWUl1xMR2+KKaaY8pKL6ehNMcUUU15yMR29KaaYYspLLv8/7eb7kQ8IZKoAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x7f534c40da50>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "C = 10 # number of digits\n",
    "R = 16 # images per row\n",
    "for c in np.arange(C):\n",
    "    for r in np.arange(R):\n",
    "        image = vector2image(x_train[y_train==c][r])\n",
    "        if r == 0:\n",
    "            image_digits_row = image\n",
    "        else:\n",
    "            image_digits_row = np.hstack((image_digits_row, image))\n",
    "    if c == 0:\n",
    "        image_digits = image_digits_row\n",
    "    else:\n",
    "        image_digits = np.vstack((image_digits, image_digits_row))\n",
    "pl.figure()\n",
    "pl.imshow(image_digits, cmap='gray')\n",
    "# save a copy of this image as png file under ./images folder\n",
    "pl.imsave('./images/mnist-samples.png', image_digits, cmap='gray') "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We are ready to implement LR classifier for digit recognition.\n",
    "\n",
    "## Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "metadata": {},
   "outputs": [],
   "source": [
    "class LR:\n",
    "    def __init__(self, W):\n",
    "        self.W = W\n",
    "        self.delta = 0.000001\n",
    "        self.epoch_max = 100000\n",
    "        \n",
    "    def parameters(self):\n",
    "        return self.W\n",
    "                \n",
    "    def activation(self, s):\n",
    "        return 1.0 / (1 + np.exp(-s)) # sigmoid function as activation function\n",
    "    \n",
    "    def activation_prime(self, s):\n",
    "        return (self.activation(s) * (1.0 - self.activation(s))) # the derivative of activation function\n",
    "    \n",
    "    def feedforward(self, X):\n",
    "        S = np.dot(X, self.W)\n",
    "        U = self.activation(S)\n",
    "        return (U)        \n",
    "        \n",
    "    def learn(self, X, Y, learning_rate):        \n",
    "        C =  len(np.unique(Y))\n",
    "        N = X.shape[0]\n",
    "        Yv = np.zeros((N, C))\n",
    "        Yv[np.arange(N), Y.astype(np.int)] = 1.0\n",
    "        is_converged = False\n",
    "        epoch = 1\n",
    "        while not is_converged:\n",
    "            U = self.feedforward(X)\n",
    "      \n",
    "            E = (U - Yv)\n",
    "    \n",
    "            D = U * (1-U) * E\n",
    "            print(np.mean(D))\n",
    "            print(X.shape)\n",
    "            MSE = 1.0 / (2 * N) * np.sum(np.sum(E**2, axis=1))\n",
    "            if (epoch%100) == 0:\n",
    "                ACC = self.accuracy(X, Y)\n",
    "                print('epoch = {0}, MSE = {1:.3f}, ACC = {2:.3f}'.format(epoch, MSE, ACC))\n",
    "            Wprev = self.W.copy()\n",
    "            self.W = self.W - learning_rate * (1.0/N )* np.dot(X.T, D)\n",
    "            epoch += 1\n",
    "            print( (np.max(np.abs(Wprev - self.W))))\n",
    "            if (np.max(np.abs(Wprev - self.W)) <= self.delta) or (epoch >=self.epoch_max):\n",
    "                is_converged = True        \n",
    "        \n",
    "    def classify(self, X):\n",
    "        U = self.feedforward(X)\n",
    "        L = np.argmax(U, axis=1)\n",
    "        return L\n",
    "    \n",
    "    def accuracy(self, X, Y):\n",
    "        N = X.shape[0]\n",
    "        L = self.classify(X)\n",
    "        acc = np.sum(L == Y) / N\n",
    "\n",
    "        return acc\n",
    "    \n",
    "    def confusion_matrix(self, X, Y):\n",
    "        L = self.accuracy(X, Y)\n",
    "        labels = np.unique(Y).astype(int)\n",
    "        conf_mat = np.zeros((len(labels), len(labels)))\n",
    "    \n",
    "        for i in np.arange(len(labels)):\n",
    "            for j in np.arange(len(labels)):\n",
    "                #print(L)\n",
    "                #print(Y==labels[j])\n",
    "                conf_mat[i, j] = np.sum((L==labels[i]) and (Y==labels[j]))\n",
    "            \n",
    "    \n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Train Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.018469302403356272\n",
      "(60000, 785)\n",
      "0.03489064009953802\n",
      "0.017796360625813405\n",
      "(60000, 785)\n",
      "0.032790023354145714\n",
      "0.01701708806586914\n",
      "(60000, 785)\n",
      "0.02933359078295228\n",
      "0.016374605055426143\n",
      "(60000, 785)\n",
      "0.027200945707488988\n",
      "0.01588303611449703\n",
      "(60000, 785)\n",
      "0.03017764143597068\n",
      "0.015308646645962196\n",
      "(60000, 785)\n",
      "0.03270862769931382\n",
      "0.01438194325626776\n",
      "(60000, 785)\n",
      "0.034182822694123094\n",
      "0.013130849555161547\n",
      "(60000, 785)\n",
      "0.032942708418340905\n",
      "0.011758989213258264\n",
      "(60000, 785)\n",
      "0.028893370309470146\n",
      "0.01041654516360978\n",
      "(60000, 785)\n",
      "0.023703705092530947\n",
      "0.009218621059081259\n",
      "(60000, 785)\n",
      "0.022112752625663645\n",
      "0.008226386290215939\n",
      "(60000, 785)\n",
      "0.02043203893584561\n",
      "0.007362093048340321\n",
      "(60000, 785)\n",
      "0.01848139581461139\n",
      "0.0066160138984899045\n",
      "(60000, 785)\n",
      "0.016389059348971224\n",
      "0.0059833581642678765\n",
      "(60000, 785)\n",
      "0.014639021376393724\n",
      "0.005439442917044483\n",
      "(60000, 785)\n",
      "0.013423044616044141\n",
      "0.004974567051397508\n",
      "(60000, 785)\n",
      "0.012289422842377296\n",
      "0.004578482362574982\n",
      "(60000, 785)\n",
      "0.011268456418602198\n",
      "0.004235884120042742\n",
      "(60000, 785)\n",
      "0.01036154458938457\n",
      "0.003933895059713521\n",
      "(60000, 785)\n",
      "0.009548156474918926\n",
      "0.0036658169979933595\n",
      "(60000, 785)\n",
      "0.008821557830485424\n",
      "0.003428472182914821\n",
      "(60000, 785)\n",
      "0.008180593129950647\n",
      "0.00321837136093385\n",
      "(60000, 785)\n",
      "0.007614846288900828\n",
      "0.0030318784359294176\n",
      "(60000, 785)\n",
      "0.007116685674151663\n",
      "0.002865760388482607\n",
      "(60000, 785)\n",
      "0.006682158828205817\n",
      "0.0027170929906958317\n",
      "(60000, 785)\n",
      "0.006305190589349113\n",
      "0.002583194892047159\n",
      "(60000, 785)\n",
      "0.005976527499285983\n",
      "0.0024617612604891427\n",
      "(60000, 785)\n",
      "0.00568690372700463\n",
      "0.0023509499943532366\n",
      "(60000, 785)\n",
      "0.005429390406184578\n",
      "0.0022493245636826683\n",
      "(60000, 785)\n",
      "0.005210170216659232\n",
      "0.0021557413971581762\n",
      "(60000, 785)\n",
      "0.005017545493785636\n",
      "0.0020692600047590506\n",
      "(60000, 785)\n",
      "0.004843038273763034\n",
      "0.0019890904451562857\n",
      "(60000, 785)\n",
      "0.004684157929973537\n",
      "0.0019145644444080612\n",
      "(60000, 785)\n",
      "0.004538629846290609\n",
      "0.0018451158307287241\n",
      "(60000, 785)\n",
      "0.0044045274037953774\n",
      "0.00178026310003644\n",
      "(60000, 785)\n",
      "0.004280314903748306\n",
      "0.001719592729020792\n",
      "(60000, 785)\n",
      "0.004164807419049565\n",
      "0.001662744268309546\n",
      "(60000, 785)\n",
      "0.00405708999258983\n",
      "0.001609398301462436\n",
      "(60000, 785)\n",
      "0.003956434290786182\n",
      "0.0015592675714085646\n",
      "(60000, 785)\n",
      "0.0038622326779379768\n",
      "0.001512090886049294\n",
      "(60000, 785)\n",
      "0.003773954046932454\n",
      "0.0014676291035165124\n",
      "(60000, 785)\n",
      "0.0036911175920923966\n",
      "0.0014256624956626334\n",
      "(60000, 785)\n",
      "0.0036132785746389173\n",
      "0.0013859889370386133\n",
      "(60000, 785)\n",
      "0.0035400209965573692\n",
      "0.0013484225473410223\n",
      "(60000, 785)\n",
      "0.0034709538195404344\n",
      "0.0013127925700454123\n",
      "(60000, 785)\n",
      "0.0034057089358240644\n",
      "0.0012789423796794389\n",
      "(60000, 785)\n",
      "0.003343940173273663\n",
      "0.0012467285750455\n",
      "(60000, 785)\n",
      "0.0032853231567586683\n",
      "0.0012160201449320884\n",
      "(60000, 785)\n",
      "0.0032295559668131246\n",
      "0.0011866976993982162\n",
      "(60000, 785)\n",
      "0.0031763604253412514\n",
      "0.0011586527559239435\n",
      "(60000, 785)\n",
      "0.0031254836811118736\n",
      "0.0011317870644337142\n",
      "(60000, 785)\n",
      "0.003076699682842604\n",
      "0.0011060119529173545\n",
      "(60000, 785)\n",
      "0.003029810150879275\n",
      "0.0010812476771295\n",
      "(60000, 785)\n",
      "0.0029846447669811815\n",
      "0.001057422762736468\n",
      "(60000, 785)\n",
      "0.0029410604493465\n",
      "0.0010344733347498426\n",
      "(60000, 785)\n",
      "0.002898939724585714\n",
      "0.0010123424356758318\n",
      "(60000, 785)\n",
      "0.002858881793891918\n",
      "0.000990979339418745\n",
      "(60000, 785)\n",
      "0.002821639914161969\n",
      "0.000970338871914697\n",
      "(60000, 785)\n",
      "0.002785437731633672\n",
      "0.0009503807514359803\n",
      "(60000, 785)\n",
      "0.002750243974510902\n",
      "0.0009310689615425688\n",
      "(60000, 785)\n",
      "0.002716038558281647\n",
      "0.0009123711680902261\n",
      "(60000, 785)\n",
      "0.0026828091181944347\n",
      "0.0008942581890361773\n",
      "(60000, 785)\n",
      "0.00265054801005489\n",
      "0.000876703522570335\n",
      "(60000, 785)\n",
      "0.0026192498508620238\n",
      "0.0008596829358458133\n",
      "(60000, 785)\n",
      "0.0025889096153654673\n",
      "0.0008431741136641734\n",
      "(60000, 785)\n",
      "0.0025595212609133416\n",
      "0.0008271563641150848\n",
      "(60000, 785)\n",
      "0.002531076822602729\n",
      "0.0008116103764648311\n",
      "(60000, 785)\n",
      "0.002503565903329702\n",
      "0.0007965180255183814\n",
      "(60000, 785)\n",
      "0.002476975477014004\n",
      "0.0007818622161669507\n",
      "(60000, 785)\n",
      "0.0024512899254260645\n",
      "0.000767626761769599\n",
      "(60000, 785)\n",
      "0.0024264912368910374\n",
      "0.0007537962902903569\n",
      "(60000, 785)\n",
      "0.002402559306166216\n",
      "0.0007403561726168484\n",
      "(60000, 785)\n",
      "0.0023794722869669577\n",
      "0.0007272924681318965\n",
      "(60000, 785)\n",
      "0.002357206960505523\n",
      "0.0007145918833216074\n",
      "(60000, 785)\n",
      "0.0023357390940984407\n",
      "0.0007022417399228281\n",
      "(60000, 785)\n",
      "0.0023150437729201126\n",
      "0.0006902299497946561\n",
      "(60000, 785)\n",
      "0.0022950956951869397\n",
      "0.0006785449943105296\n",
      "(60000, 785)\n",
      "0.002275869426514629\n",
      "0.0006671759065882729\n",
      "(60000, 785)\n",
      "0.002257339613094031\n",
      "0.0006561122552943676\n",
      "(60000, 785)\n",
      "0.002239481155928695\n",
      "0.0006453441290735958\n",
      "(60000, 785)\n",
      "0.002222269349933824\n",
      "0.0006348621208719901\n",
      "(60000, 785)\n",
      "0.0022056799924616843\n",
      "0.0006246573115525587\n",
      "(60000, 785)\n",
      "0.0021896894660121055\n",
      "0.0006147212522683873\n",
      "(60000, 785)\n",
      "0.0021742747996935027\n",
      "0.0006050459450801125\n",
      "(60000, 785)\n",
      "0.0021594137135674124\n",
      "0.0005956238213113213\n",
      "(60000, 785)\n",
      "0.0021450846494522058\n",
      "0.000586447717154527\n",
      "(60000, 785)\n",
      "0.0021312667911624616\n",
      "0.0005775108460989528\n",
      "(60000, 785)\n",
      "0.0021179400765782425\n",
      "0.0005688067678721254\n",
      "(60000, 785)\n",
      "0.0021050852034090912\n",
      "0.000560329353784978\n",
      "(60000, 785)\n",
      "0.0020926836300618826\n",
      "0.0005520727486484453\n",
      "(60000, 785)\n",
      "0.002080717572646329\n",
      "0.000544031329778227\n",
      "(60000, 785)\n",
      "0.0020691699988555634\n",
      "0.0005361996639985093\n",
      "(60000, 785)\n",
      "0.0020580246192336143\n",
      "0.0005285724639561905\n",
      "(60000, 785)\n",
      "0.0020615469111930906\n",
      "0.0005211445454154916\n",
      "(60000, 785)\n",
      "0.0020659795484612165\n",
      "0.0005139107874652355\n",
      "(60000, 785)\n",
      "0.0020701649617318463\n",
      "0.0005068660976873896\n",
      "(60000, 785)\n",
      "0.002074115492020301\n",
      "0.0005000053842665628\n",
      "(60000, 785)\n",
      "0.0020778441689104366\n",
      "0.0004933235367455371\n",
      "(60000, 785)\n",
      "0.002081364586631773\n",
      "0.0004868154166541258\n",
      "(60000, 785)\n",
      "epoch = 100, MSE = 0.498, ACC = 0.000\n",
      "0.0020846907699838546\n",
      "0.0004804758585851824\n",
      "(60000, 785)\n",
      "0.002087837030646078\n",
      "0.00047429968151301515\n",
      "(60000, 785)\n",
      "0.0020908178149168233\n",
      "0.0004682817093148097\n",
      "(60000, 785)\n",
      "0.0020936475445048064\n",
      "0.0004624167986459767\n",
      "(60000, 785)\n",
      "0.0020963404526002094\n",
      "0.0004566998716211702\n",
      "(60000, 785)\n",
      "0.0020989104180075824\n",
      "0.0004511259502466525\n",
      "(60000, 785)\n",
      "0.0021013708005468423\n",
      "0.00044569018930901545\n",
      "(60000, 785)\n",
      "0.0021037342811989213\n",
      "0.00044038790450460016\n",
      "(60000, 785)\n",
      "0.002106012710539895\n",
      "0.00043521459302240924\n",
      "(60000, 785)\n",
      "0.0021082169688959596\n",
      "0.00043016594456643645\n",
      "(60000, 785)\n",
      "0.0021103568413582963\n"
     ]
    },
    {
     "ename": "KeyboardInterrupt",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m                         Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-30-0e3d6d913b3b>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m     11\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     12\u001b[0m \u001b[0mlogreg\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mLR\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mW\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 13\u001b[0;31m \u001b[0mlogreg\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mlearn\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mx_train_t\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0my_train\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m2\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     14\u001b[0m \u001b[0;31m#logreg.confusion_matrix( x_train_t, y_train)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-29-01705bb8e6cf>\u001b[0m in \u001b[0;36mlearn\u001b[0;34m(self, X, Y, learning_rate)\u001b[0m\n\u001b[1;32m     27\u001b[0m         \u001b[0mepoch\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;36m1\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     28\u001b[0m         \u001b[0;32mwhile\u001b[0m \u001b[0;32mnot\u001b[0m \u001b[0mis_converged\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 29\u001b[0;31m             \u001b[0mU\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfeedforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     30\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     31\u001b[0m             \u001b[0mE\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mU\u001b[0m \u001b[0;34m-\u001b[0m \u001b[0mYv\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m<ipython-input-29-01705bb8e6cf>\u001b[0m in \u001b[0;36mfeedforward\u001b[0;34m(self, X)\u001b[0m\n\u001b[1;32m     15\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     16\u001b[0m     \u001b[0;32mdef\u001b[0m \u001b[0mfeedforward\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mself\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mX\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m---> 17\u001b[0;31m         \u001b[0mS\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mdot\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mX\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mW\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m     18\u001b[0m         \u001b[0mU\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mactivation\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mS\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m     19\u001b[0m         \u001b[0;32mreturn\u001b[0m \u001b[0;34m(\u001b[0m\u001b[0mU\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mKeyboardInterrupt\u001b[0m: "
     ]
    }
   ],
   "source": [
    "N_train = x_train.shape[0]\n",
    "x_train_t = np.hstack((np.ones((N_train, 1)), x_train))\n",
    "x_train_t = x_train_t / np.max(x_train_t)\n",
    "N_test = x_test.shape[0]\n",
    "x_test_t = np.hstack((np.ones((N_test, 1)), x_test))\n",
    "x_test_t = x_test_t / np.max(x_train_t)\n",
    "C = len(np.unique(y_train))\n",
    "d = x_train_t.shape[1]\n",
    "\n",
    "W = np.random.randn(d, C) \n",
    "\n",
    "logreg = LR(W)\n",
    "logreg.learn(x_train_t, y_train, 2)\n",
    "#logreg.confusion_matrix( x_train_t, y_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(5,)"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "y_train[:][0:5].shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "ename": "IndexError",
     "evalue": "index 5 is out of bounds for axis 1 with size 5",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mIndexError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m<ipython-input-12-53a2754f0653>\u001b[0m in \u001b[0;36m<module>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0mYv\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mnp\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mzeros\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mN\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0mC\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m Yv[np.arange(N), \n\u001b[0;32m----> 7\u001b[0;31m    Y.astype(np.int)] = 1.0\n\u001b[0m\u001b[1;32m      8\u001b[0m \u001b[0mYv\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mshape\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mIndexError\u001b[0m: index 5 is out of bounds for axis 1 with size 5"
     ]
    }
   ],
   "source": [
    "X = x_train[:][0:5]\n",
    "Y = y_train[:][0:5]\n",
    "C =  len(np.unique(Y))\n",
    "N = X.shape[0]\n",
    "Yv = np.zeros((N, C))\n",
    "Yv[np.arange(N), \n",
    "   Y.astype(np.int)] = 1.0\n",
    "Yv.shape"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test Accuracy of the Trained Logistic Regression"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print('{0:.3f}'.format(logreg.accuracy(x_test_t, y_test)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}