Titanic_K-nearest_neighbors_classification.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Predicting Titanic survivors with K-nearest neighbors classification"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Demonstration of K-nearest neighbors classification on the Titanic dataset. This time using the preprocessed data. For the preprocessing see the Titanic_K-nearest_neighbors_classification.ipynb notebook. Since we use KNN classifier it is important to scale continous data.\n",
    "\n",
    "\n",
    "Source of data: https://www.kaggle.com/c/titanic/data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Import the preprocessed data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = pd.read_csv('train_clear.csv')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Unnamed: 0</th>\n",
       "      <th>PassengerId</th>\n",
       "      <th>Survived</th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>male</th>\n",
       "      <th>Q</th>\n",
       "      <th>S</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>22.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>7.2500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>38.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>71.2833</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>2</td>\n",
       "      <td>3</td>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>26.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>7.9250</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>3</td>\n",
       "      <td>4</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>35.0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>53.1000</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>4</td>\n",
       "      <td>5</td>\n",
       "      <td>0</td>\n",
       "      <td>3</td>\n",
       "      <td>35.0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>8.0500</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   Unnamed: 0  PassengerId  Survived  Pclass   Age  SibSp  Parch     Fare  \\\n",
       "0           0            1         0       3  22.0      1      0   7.2500   \n",
       "1           1            2         1       1  38.0      1      0  71.2833   \n",
       "2           2            3         1       3  26.0      0      0   7.9250   \n",
       "3           3            4         1       1  35.0      1      0  53.1000   \n",
       "4           4            5         0       3  35.0      0      0   8.0500   \n",
       "\n",
       "   male  Q  S  \n",
       "0     1  0  1  \n",
       "1     0  0  0  \n",
       "2     0  0  1  \n",
       "3     0  0  1  \n",
       "4     1  0  1  "
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [],
   "source": [
    "train = train.drop(['Unnamed: 0', 'PassengerId'], axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Standard scaling\n",
    "\n",
    "First let's create a data frame with the countinous variables only."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [],
   "source": [
    "train_cont = train.drop(['Survived', 'male','Q','S'],axis=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a Standard scaler object and fitting only the continous data."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.827377</td>\n",
       "      <td>-0.552360</td>\n",
       "      <td>0.432793</td>\n",
       "      <td>-0.473674</td>\n",
       "      <td>-0.502445</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1.566107</td>\n",
       "      <td>0.659475</td>\n",
       "      <td>0.432793</td>\n",
       "      <td>-0.473674</td>\n",
       "      <td>0.786845</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.827377</td>\n",
       "      <td>-0.249401</td>\n",
       "      <td>-0.474545</td>\n",
       "      <td>-0.473674</td>\n",
       "      <td>-0.488854</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-1.566107</td>\n",
       "      <td>0.432256</td>\n",
       "      <td>0.432793</td>\n",
       "      <td>-0.473674</td>\n",
       "      <td>0.420730</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.827377</td>\n",
       "      <td>0.432256</td>\n",
       "      <td>-0.474545</td>\n",
       "      <td>-0.473674</td>\n",
       "      <td>-0.486337</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Pclass       Age     SibSp     Parch      Fare\n",
       "0  0.827377 -0.552360  0.432793 -0.473674 -0.502445\n",
       "1 -1.566107  0.659475  0.432793 -0.473674  0.786845\n",
       "2  0.827377 -0.249401 -0.474545 -0.473674 -0.488854\n",
       "3 -1.566107  0.432256  0.432793 -0.473674  0.420730\n",
       "4  0.827377  0.432256 -0.474545 -0.473674 -0.486337"
      ]
     },
     "execution_count": 59,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_cont.head()\n",
    "from sklearn.preprocessing import StandardScaler\n",
    "scaler = StandardScaler()\n",
    "scaler.fit(train_cont)\n",
    "scale_feat_cont = scaler.transform(train_cont)\n",
    "scale_feat_contDF = pd.DataFrame(scale_feat_cont, columns=train_cont.columns)\n",
    "scale_feat_contDF.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's create a dataframe for the encoded categorical data only, then concatenate the scaled and categorical dataframes."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>Pclass</th>\n",
       "      <th>Age</th>\n",
       "      <th>SibSp</th>\n",
       "      <th>Parch</th>\n",
       "      <th>Fare</th>\n",
       "      <th>Survived</th>\n",
       "      <th>male</th>\n",
       "      <th>Q</th>\n",
       "      <th>S</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.827377</td>\n",
       "      <td>-0.552360</td>\n",
       "      <td>0.432793</td>\n",
       "      <td>-0.473674</td>\n",
       "      <td>-0.502445</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>-1.566107</td>\n",
       "      <td>0.659475</td>\n",
       "      <td>0.432793</td>\n",
       "      <td>-0.473674</td>\n",
       "      <td>0.786845</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.827377</td>\n",
       "      <td>-0.249401</td>\n",
       "      <td>-0.474545</td>\n",
       "      <td>-0.473674</td>\n",
       "      <td>-0.488854</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>-1.566107</td>\n",
       "      <td>0.432256</td>\n",
       "      <td>0.432793</td>\n",
       "      <td>-0.473674</td>\n",
       "      <td>0.420730</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>0.827377</td>\n",
       "      <td>0.432256</td>\n",
       "      <td>-0.474545</td>\n",
       "      <td>-0.473674</td>\n",
       "      <td>-0.486337</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "     Pclass       Age     SibSp     Parch      Fare  Survived  male  Q  S\n",
       "0  0.827377 -0.552360  0.432793 -0.473674 -0.502445         0     1  0  1\n",
       "1 -1.566107  0.659475  0.432793 -0.473674  0.786845         1     0  0  0\n",
       "2  0.827377 -0.249401 -0.474545 -0.473674 -0.488854         1     0  0  1\n",
       "3 -1.566107  0.432256  0.432793 -0.473674  0.420730         1     0  0  1\n",
       "4  0.827377  0.432256 -0.474545 -0.473674 -0.486337         0     1  0  1"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train_dis = train.drop(['Pclass','Age', 'SibSp', 'Parch', 'Fare'], axis=1)\n",
    "train_scaled = pd.concat([scale_feat_contDF, train_dis],axis=1)\n",
    "train_scaled.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Classification\n",
    "\n",
    "Let's create the KNN classifier object, with the initial value of K=2."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.neighbors import KNeighborsClassifier\n",
    "knn = KNeighborsClassifier(n_neighbors=2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Splitting the dataset to train and test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X = train_scaled.drop('Survived', axis=1)\n",
    "y = train_scaled['Survived']\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Fitting the training data and predicting based on the test set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [],
   "source": [
    "knn.fit(X_train, y_train)\n",
    "pred_knn = knn.predict(X_test)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Evaluation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[148  21]\n",
      " [ 52  47]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.74      0.88      0.80       169\n",
      "           1       0.69      0.47      0.56        99\n",
      "\n",
      "    accuracy                           0.73       268\n",
      "   macro avg       0.72      0.68      0.68       268\n",
      "weighted avg       0.72      0.73      0.71       268\n",
      "\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix, classification_report\n",
    "print(confusion_matrix(y_test, pred_knn))\n",
    "print(classification_report(y_test, pred_knn))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Optimization\n",
    "\n",
    "We can see that the model made worse predictions than the random forest classifier did. Let's see if we can improve this accuracy by optimizing the K value."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "error_rate_knn = []\n",
    "for i in range(1, 50):\n",
    "    knn = KNeighborsClassifier(n_neighbors=i)\n",
    "    knn.fit(X_train, y_train)\n",
    "    predict_i = knn.predict(X_test)\n",
    "    error_rate_knn.append(np.mean(predict_i != y_test))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Text(0, 0.5, 'Error rate')"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYgAAAEJCAYAAACOr7BbAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjAsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+17YcXAAAgAElEQVR4nO3deXyb1ZXw8d+RbHmR7XiN4yWJHRISm6zOylpKA2VNSkILtEyh7TudTkvXKX27zHR9O50pLVMK7XSZLtMNCjhAKFBIKUuhxJDY2ffYTuIl8b7Hli3d9w/JjmxLtmxLtmyd7+eTD9LzXMnnIY6OnrucK8YYlFJKqaEsUx2AUkqp8KQJQimllE+aIJRSSvmkCUIppZRPmiCUUkr5pAlCKaWUTyFNECJyvYgcFZETIvJFH+c/JyKHRGSfiLwkIvO9zn1XRA6KyGER+aGISChjVUopNVjIEoSIWIEfATcAhcCdIlI4pFkZsMYYsxx4Aviu57WXAZcDy4GlwFrgHaGKVSml1HBRIXzvdcAJY0w5gIg8CmwGDvU3MMa87NV+J3BX/ykgFrABAkQD50b6Yenp6SYvLy9YsSulVETYvXt3gzEmw9e5UCaIHOCM1/MqYP0I7T8CPA9gjHlTRF4GanEniIeNMYeHvkBEPgp8FGDevHns2rUrSKErpVRkEJFT/s6FcgzC15iBz7oeInIXsAa43/N8IVAA5OJONNeIyFXD3syYnxlj1hhj1mRk+EyASimlximUCaIKmOv1PBeoGdpIRDYCXwE2GWN6PIdvBXYaYzqMMR247yw2hDBWpZRSQ4QyQbwNLBKRfBGxAXcA270biMgq4Ke4k0Od16nTwDtEJEpEonEPUA/rYlJKKRU6IUsQxpg+4F7gBdwf7o8ZYw6KyDdFZJOn2f1AAvC4iOwRkf4E8gRwEtgP7AX2GmOeCVWsSimlhpOZUu57zZo1RgeplVJqbERktzFmja9zupJaKaWUT5oglFJK+RTKdRBKKaXGoLGjh9+XnKbP6Rp2Ltpq4fa1c5mdFDtp8WiCUEqpMPH7ktM8sOMYvirPGQM7Dp/jiY9dhi1qcjp/NEEopVSYKKlopCAriec/feWwc38+UMvHflfKd/98hH+9eWhZu9DQMQillAoDjj4Xu081sz4/1ef565dm8cFL5/M/r1fw1yMjlqYLGk0QSikVBvZXt9Dd62LDAt8JAuDLNxZQmJXEvzy2l9rW8yGPSROEUkqFgZ3lTQCsy0/z2yY22spD719FT5+LTz+6x+dgdjBpglBKqTBQUtHExZkJpNptI7a7KCOBb21eylsVTTz01xMhjUkThFJKTbE+p4vdlU2sH+HuwdvW1blsKcrhob8e582TjSGLSxOEUkpNsQM1bXQ6nKwfYfxhqG9tXkpeup1PP1pGY0fP6C8YB00QSikVAucdTm798Ru8caJh1LYl5e67gHV+ZjD5Yo+J4uE7i2g538u/PL4Xlyv4dfU0QSilVAjsOtVE2ekWfv638lHbllQ0sSDDzuzEsa2SLsxO4uu3XMLVF2f4XFw3UbpQTimlQqDEMyvptWP11LV3+/3wd7oMb1c0cfOKrHH9nPevnzfuGEejdxBKKRUCJRWNzEmKxWXg6bJhm2kOOFzbRntPX8AD1JNJE4RSSgVZd6+TvWda2bwym5VzkykurfLbdqdn/GEsA9STRROEUkoFWenpZhxOF+sXpLK1KIcjZ9s5WNPqs21JRRPzUuPJmhU3yVGOThOEUkoFWUl5ExaBNXmp3LIim2irsK20elg7l8vwdmWT3/pLU00ThFJKBVlJRSOF2UkkxUaTHG/jXUsyeXpPNb1DSmMcq2unpauX9QvCb/wBNEEopVRQ9fQ5KTvdMmjQeUtRDg0dDv52vH5Q2/6ZTnoHoZRSEWDvmVZ6+lyDPvSvXjybVLuN4t2Du5lKKhrJSY5jbmr8ZIcZEE0QSikVRG9VuGclrc27kCBsURY2rchmx+FztHb1AmCM4a2K8B1/AE0QSikVVCUVTSyZk0jKkKqsW4tycfS5+NN+95qIk/UdNHQ4wnJ6az9NEEopFSS9Tv+7wi3NSWLR7ISB2Uw7B8YfwnOAGjRBKKVU0OyvbqXL4fQ5K0lE2FKUy+5TzVQ0dFJS0URmUgzz08Jz/AE0QSilVNCUDOwK57vb6NZVOYjAttIqSsobWZ+fhoSiyl6QaLE+pZQKkpKKRhbOTiA9Icbn+TmzYrliYTr/+/dK2rr7wnr8AfQOQimlgqLP6WJXpe/xB29bi3Jp6+4Dwnv8AfQOQimlguJQbRsdPX2jroq+7pJM7DYrcTYrF2XYJym68dEEoZRSQdA//rBhlDuIeFsU9717MQbCevwBNEEopVRQlFQ0kp9uZ3bS6LvC3XN5/iRENHE6BqGUUhPkdIX/qujx0AShlFITdORs27SYlTRWmiCUUmqCSqbBqujx0AShlFITVFLRyNzUOLKTw29XuInQBKGUUhPgGhh/mFl3D6AJApfLUNNynpYux1SHopSaho7XddDc1TvjBqghxAlCRK4XkaMickJEvujj/OdE5JCI7BORl0Rkvte5eSLyoogc9rTJC0WM59q7uew//sqf9tWG4u2VUjPcttIqADaE6bahExGyBCEiVuBHwA1AIXCniBQOaVYGrDHGLAeeAL7rde43wP3GmAJgHVAXijgzE2OJjbZQ0dAZirdXSs1grx9v4Gd/K+f2NXPDdle4iQjlHcQ64IQxptwY4wAeBTZ7NzDGvGyM6fI83QnkAngSSZQxZoenXYdXu6CyWIS8NLsmCKXUmNS39/CZP+5hYUYCX990yVSHExKhTBA5wBmv51WeY/58BHje8/hioEVEtolImYjc77kjGUREPioiu0RkV319/dDTAVuQoQlCKRU4l8vwucf20N7dy8PvLyLONuzjaUYIZYLwVWTE+GwochewBrjfcygKuBL4PLAWWADcM+zNjPmZMWaNMWZNRkbGuAPNT7dzpqmLXqdr3O+hlIocP3ntJH873sDXN13C4jmJUx1OyIQyQVQBc72e5wI1QxuJyEbgK8AmY0yP12vLPN1TfcBTQFGoAs1PT6DPZahqPh+qH6GUmiF2n2ri+y8e46blWdyxdu7oL5jGQpkg3gYWiUi+iNiAO4Dt3g1EZBXwU9zJoW7Ia1NEpP+24BrgUKgCzU93Dy5VNHSE6kcopWaAli4Hn3pkDznJcXxny7Kwr8Y6USFLEJ5v/vcCLwCHgceMMQdF5JsissnT7H4gAXhcRPaIyHbPa524u5deEpH9uLurfh6qWPPTEwAor9dxCKWUb8YYvvDEPurau3nozlUkxUZPdUghF9Jy38aY54Dnhhz7qtfjjSO8dgewPHTRXZASH82suGgqGzVBKDWTlJQ3Eh1loWheSkDtnS7Db96s5Gxb97Bz51q7efHQOf71pgJWzE0OcqThSfeDwL1pR366zmRSaiZx9Ln42O9209Pn4plPXsFFGQmjvuaHLx3nwZeOY4uy+Jxls2VVDh+5Ynrs5RAMmiA88tPtlJQ3TnUYSqkgeeVoHc1dvURbhU/8vpSnPnE5sdH+p6O+ebKRh/56nC1FOTzwvpWTGGn4ivhaTP3y0+3UtHZz3uGc6lCUUkFQXFpFekIMP/7Aao6cbefbzx7227axo4dPP1pGXrqdb21eOolRhjdNEB756e7Nw081aTeTUtNdc6eDvx6p4z0rs7m2MJN/vDKf3+48xfP7h9dcc7kM//L4XlrO9/LwnUXYY7RjpZ8mCI/+BFGhM5mUmvae2VdDr9OwpSgXgPvevYQVubP4QvE+zjQNrtrzi9creOVoPf92UwGF2UlTEW7Y0gThkedJEOU6UK3UtFdcWs2SOYkDH/i2KAsP3VkEBj71aNlA1YQ9Z1r4zz8f4Yalc7hrw/yR3jIiaYLwSIiJYnZijM5kUmqaO1HXwd4zLdy2OnfQ8Xlp8Xxn6zLKTrfw/ReP0dbdyycfKSUzKZb/2Lp8xi96Gw/tbPOiU12Vmv62lVZhtQibVmYPO3fz8mzeONHIT149yZsnG6ht6eaxj13KrLiZv+htPPQOwsuCDDuVmiCUmracLsOTZdVctSid2YmxPtt87ZZCFmcmsreqlc+/e3HAi+gikd5BeMlLs9PY6aC1q5dZ8fqNQqnpZmd5I7Wt3Xz5xgK/bWKjrfzP3Wt45Vg9H1g3bxKjm370DsLLwEwmLbmh1LRUXFpFYmwU1xZmjthubmo8/7BhPhaLjjuMRBOElwUZngShVV2VmnY6e/r484Gz3Lw8a8QV0ypwmiC8zE2NxyJQ0RCS3U2VUiH05wNn6XI42VqUO3pjFRBNEF5ioqzkpsTrTCalpqHi0irmpcazer4OOgeLJogh8tLt2sWk1DRT3XKeN8sb2VKUo+sZgkgTxBAL0u1U1HdijM/ts5VSYeipsmqMgS2rtHspmHSa6xD56XY6HU7qO3r8zqNWSk3cq8fqaT3f6/PchvxUZicF9u/PGENxaRXr8lKZlxYfzBAjniaIIbyL9mmCUCo03q5s4u5fvuX3fE5yHM996sqA1iM9vquK8vpOPvaOi4IZokITxDADCaKhk/UL0qY4GqVmpid2VRFvs1L8z5cRbR3c032mqYt//M0uvlC8l5/ctXrEMYXj59r56vYDXHZRms5eCgFNEENkJ8dhs1p0JpNSIdLd6+TZ/bXcsDSLgqzh5bUXzk7g/16/hG8/d5jf7jzFBy/N8/s+9/6hDLstih/cvhKrLnoLOh2kHsJqEeanxWvZb6VC5IWDZ+no6WPr6hy/bT5yRT7vXJzB//vTYQ7WtPps841nDnH0XDvff9+KgMcr1NhogvAhP12L9ikVKttKq8lJjmNDvv8uXItF+P77VpJij+aTfyijs6dv0Pk/7avhkbdO80/vWMDVi2eHOuSIpQnCh/x0O6cau3C6dKqrUsFU19bN347Xc+uqnFHrIKXabTx4xyoqGzv5t6cODBw/3djFl4r3s2peMp+/bnGoQ45omiB8yE+343C6qGk5P9WhKDWjPLWnGpeBW4v8dy9527AgjU+9axHbyqp5YncVjj4X9z5Sigj88I5Vwwa4VXDpILUP+V7bj85N1XnVSgWDMYbi3dWsnJvMRRkJAb/uk9cs4s2TjfzbUwd49Vg9+6pa+cldRfpvcxJo+vUh31PVVcchlArMf79ykkffOj1im4M1bRw9187W1WObjmq1CA/esYo4m5Vn9tbwDxvmc/3SrImEqwKkdxA+ZCTEYLdZdaqrUgEwxvDjV07Q2dPHgowE1uWn+my3rbQam9XCLcvH/uE+Z1YsP/5AEU/vqeErN/nfDEgFl95B+CAi5GfYdaqrUgGoaj5Pe7d7ltGnHy2judMxrE2v08X2vdW8q2A2yfG2cf2cDQvS+M6WZbrXwyTSBOFHfnqCVnVVKgCHa9sA+PqmS2jo6OG+J/YNK3b52rF6GjocbNHVztOKJgg/8tPtVDefp6fPOdWhKBXWDtW2IQJbi3L50g0F/OXwOX7998pBbbaVVpNqt3H14oypCVKNiyYIPxak23EZd10YpZR/h2vbyEuzY4+J4kOX57GxIJPvPHeEA9XuFdCtXb3sOHSOTSuydVrqNKN/W37k9U91rddxCKVGcri2nYKsRMA9fnf/bctJS7Bx7x9K6ejp40/7a3A4Xdw2xtlLauppgvAjP+1CVVellG/t3b2cbuqiYM6Fonspdhs/vHMVZ5rP85Un91O8u4qLMxO4JHt4YT4V3jRB+DErPpo0u00ThFIjOHK2HYDCIR/+a/NS+ezGRTy9p4bS0y1sLcrVrUCnoYAShIhcISIf8jzOEJH80IYVHvLS7VQ2aoJQyp/+GUy+ynb/89ULueyiNKIswntWBVZaQ4WXURfKicjXgDXAYuBXQDTwO+Dy0IY29TISYjhRr1NdlfLncG0bs+KiyZo1vNy21SL84u61VDV3kanluKelQO4gbgU2AZ0AxpgaIDGUQYWL1AQbTT4W/Sil3A55Bqj9dR/F2awsyoyIj4sZKZAE4TDuVS8GQETsgb65iFwvIkdF5ISIfNHH+c+JyCER2SciL4nI/CHnk0SkWkQeDvRnBlOa3UZzl0PLfivlg9NlOHq2zWf3kpoZAkkQj4nIT4FkEflH4C/A/4z2IhGxAj8CbgAKgTtFpHBIszJgjTFmOfAE8N0h578FvBpAjCGRardhDLR06V2EUkNVNHTS3euiUBPEjDVqgjDGfA/3h3cx7nGIrxpjfhjAe68DThhjyo0xDuBRYPOQ937ZGNO/Em0nMDBRWkRWA5nAi4FcSCik2t01Y7SbSanhRhqgVjPDqAlCRP7TGLPDGHOfMebzxpgdIvKfAbx3DnDG63mV55g/HwGe9/xMC/B94L5RYvuoiOwSkV319fUBhDQ2afYYABo1QSg1zOHaNqIswqLMwPd2UNNLIF1M1/o4dkMAr/M1auWzM19E7sI9U+p+z6GPA88ZY874aj/wZsb8zBizxhizJiMj+DVe9A5CKf8O17ZxUUYCMVFaXXWm8jvNVUT+GfcH9QIR2ed1KhF4I4D3rgLmej3PBWp8/JyNwFeAdxhjejyHLwWuFJGPAwmATUQ6jDHDBrpDKS3BnSD0DkKp4Q7VtnHZRelTHYYKoZHWQfwBd5fPdwDvD+Z2Y0xTAO/9NrDIs6iuGrgDeL93AxFZBfwUuN4YU9d/3BjzAa829+AeyJ7U5ACQ4qlb39ShCUIpb02dDs619QzUYFIzk98EYYxpBVqBOwFEZDYQCySISIIxZsT9BY0xfSJyL/ACYAV+aYw5KCLfBHYZY7bj7lJKAB73zKM+bYzZFITrCgpblIXE2CiaOntGb6xUBNEB6sgQyErqW4AHgGygDpgPHAYuGe21xpjngOeGHPuq1+ONAbzHr4Ffj9YuVNLsNu1iUmoITRCRIZBB6v8HbACOGWPygXcR2BjEjJBq19XUSg11qLaNjMQY0hNipjoUFUKBJIheY0wjYBERizHmZWBliOMKG6n2GE0QSg1xqKZNF8hFgEASRIuIJACvAb8XkQeBvtCGFT60i0mpwRx9Lk7Wd2j3UgQIJEFsBrqAzwJ/Bk4Ct4QyqHCSmmCjudMxbBN2pSLViboOep1GZzBFgBEHqT31lJ72DCa7gP+dlKjCSJrdRp/L0Ha+j1nx0VMdjlJTrn+AWruYZr4R7yCMMU6gS0RmTVI8Yad/NXWjTnUNa23dvWx6+HX+UDLi7GsVgKfKqrnxwb/R7Kdr9XBtGzFRFvLTAy7srKapQLqYuoH9IvILEflh/59QBxYutNxG+DPG8KXi/eyrauWlw+emOpxp77Xj9RyqbePzj+/12bV6qLaNxXMSibLqjsUz3ajrIIBnPX8iUn/BPk0Q4euRt87w7P5aEmOjBro/1PhVNHQSG23hpSN1/OqNSj58xYUdho0xHK5t47rCOVMYoZosoyYIY0zEjTt4S03QO4hwduRsG9945iBXLkrn0ovS+O6fj9LS5SDZUyZFjV1lQydbinKpa+vhO88fZm1eKsty3b3M59p6aO7q1QHqCKH3iKNIs2vBvnDV5ejj3j+UkRQXzQPvW8kl2e4PsUN6FzFuzZ0Omrt6WZBu5/7blpOeEMO9j5TS3t0L6ArqSKMJYhSx0VbibVa9gwhD39h+iJP1HfzX+1aSkRgz8K32cG37FEc2fVU0dgKQn24nxW7jwTtWcaapi688eQBjzEDyLcjWBBEJRkwQImIVkftHahMJtNxG+Hl6TzV/3HWGj199EVcscpecnp0YS3qCTcchJqCi/kKCAFiXn8pnN17M9r01PLbrDIdq28hNiSMpVqd8R4IRxyCMMU4RWS0iYiJ4pZiupg4vlQ2dfHnbftbMT+GzGy8edK4gK0kTxARUNHRitQhzU+MHjn38nQt5s7yRr20/SEJMFKvmpUxhhGoyBTKLqQx4WkQeBzr7DxpjtoUsqjCTardR36HrICZTTct5/n6y0ee5X71RQZTVwoN3rho21bIwK4lfvVFJr9NFtE7DHLOKxk7mpcYP+n9ntQg/uH0lNzz4Nxo6HDr+EEECSRCpQCNwjdcxA0RQgojh6Fnt155M/7d4H3873uDzXJRF+PEHishJjht2riArCYfTXStoyRz9IBurivpOnwvgZifF8sDtK/nIr99mXV7qFESmpkIg01w/NBmBhLO0BHcXkzEGz8ZGKoTOtnbz+okG/s8V+dx9Wd6w8/aYqIEFjEP1f7s9XNumCWKMjDFUNHSyYUGaz/PvuDiDfV+/jnhbIN8r1Uww6j24iOSKyJMiUici50SkWERyJyO4cJFqt9HT56LL4ZzqUCLCk2XVGAN3bZjP3NT4YX/8JQeABRl2bFaLzmQah3NtPZzvdZKf4b+EhiaHyBJIJ+2vgO24d5TLAZ7xHIsYWm5j8hhj2FZaxer5KeSNo9ZPtNXCxXMSOFSjA9VjVd7QAcACrbGkPAJJEBnGmF8ZY/o8f34NZIQ4rrCii+Umz/7qVo7XdbC1aPw3qQVz3DOZInji3bhUNnQBaBE+NSCQBNEgInd51kRYReQu3IPWEePCHYTOZAq1baXV2KIs3LQ8a9zvUZCVRGOng/p2/fsai4qGDmKiLMxJip3qUFSYCCRBfBh4H3AWqAVu8xyLGP0F+xo79A4ilBx9LrbvreHawkxmxY1/IVb/QLWW3Bibigb3DCaLRSdiKLdANgzaaozZNEnxhCUt2Dc5XjlaR1Ong61FORN6n8KBmUztXL14djBCiwjlDZ0sztQifOqCQDYM2jxJsYQtu82KLcqiCSLEtpVWk55g46pFExvimhUfTU5ynN5BjEGf08WZpi4df1CDBDJn7Q0ReRj4I4NXUpeGLKowIyJabiPEmjsdvHTkHB+8NC8oG9EUZCVqyY0xqG45T6/TjGvmmJq5AkkQl3n++02vY4bBK6tnPC3YF1p/2ldDr9OwZYLdS/0KspL465E6unudxEZbg/KeM1l5g/u7n05xVd5GG4OwAP9tjHlskuIJW6l6BxFST5RWs2RO4sCeDhNVkJWEy8Cxc+0sz00OynvOZEOruCoFo49BuIB7JymWsJZmt+k01xA5UdfB3jMtE1r7MFT/QLUumAtMRUMnSbH+S5ioyBRIZ+8OEfm8iMwVkdT+PyGPLMyk2mNo0mmuIbGttAqLwOZV2UF7z3mp8dhtVh2HCFBlYyf5GQlaa0wNEsgYRP+ah094HTPAguCHE77SEmx0Opzapx1kLpfhybJqrro4g9mJwVugZbEIi+ckak2mAJXXd7I2T/d5UIONegdhjMn38SeikgNoPaZQ2VneSG1rd1C7l/oVZCVx+KyW3BhNd6+Tmtbz5KcnTHUoKsz4TRAi8gWvx+8dcu7fQxlUONIEMX7GGE7Wd3Copm3Yn9+VnCIxNoprCzOD/nMLspJo7+6jqvm83zYtXTP/77NxlM2uTjV2YQwjVnFVkWmkLqY7gO96Hn8JeNzr3PXAl0MVVDjSgn3j98rRej7067f9nr9z3byQdNsVZl8oueG9hWa/R946zZef3M/3blvB1tUzs4L9yfoOrn3gVX72D2vY6CcJV2gVV+XHSAlC/Dz29XzG04J94/fqsXpioy384PaVDP3VEYFLL/K9Qc1ELZmTiIh786B3XzJn0LkjZ9v4+vaDWEX416cOsHJeMhdlzLwuloM1bbiMOxn6TxDuKq66SE4NNVKCMH4e+3o+42nBvvErqWhi9fwUrl86/gqt4xFviyIvzT5sJlOXo497/1BGUlw0v7pnLR/85Vt84velPPWJy2fcBIT+9Q2vHKunoaOH9ISY4W0aOshIjCEhRjcDUoONNEi9QkTaRKQdWO553P982STFFzaS4qKIsoiOQYxRa1cvR862sS4vNHcJo3GX3Bg8k+nr2w9ysr6DH9y+kqU5s/j+e1dw5Gw733728JTEGEoVDR3ERVtxugzb99T4aeN7H2ql/CYIY4zVGJNkjEk0xkR5Hvc/H38t5mlKREjRchtj9lZlE8bA+gVTs3SmMCuJ001dtHf3AvD0nmoe21XFJ65eyOUL0wF455LZ/OOV+fx25yme3187JXGGSkVjF6vnp7AsZxbFpVW+2zR06viD8mniVdEiiBbsG7uS8kZsURZWzp2achf9e0McOdtOZUMnX962nzXzU/jMxkWD2t337iWsmJvMF4r3caapaypCDTpjDBX1HeSn29lSlMPBmjaOnB3c3dbW3UtDh0PHH5RPIU0QInK9iBwVkRMi8kUf5z8nIodEZJ+IvCQi8z3HV4rImyJy0HPu9lDGGSgt2Dd2JRVNrJybPGV9+/0JYu+ZFu59pJQoq4UH71w1rGKsLcrCQ3esAgOferSMXqdrKsINqqZOB23dfeSn29m0Ipsoi7CttHpQm8oGrcGk/AtZgvBsNvQj4AagELhTRAqHNCsD1hhjlgNPcGFabRfwQWPMJbin1P5ARKa84pomiLFp6+7lYE0rG/KnrjJL1qxYZsVF88COYxyobuN7711BTnKcz7bz0uL5j63LKTvdwvdePDrJkQZfhdeHf1pCDFcvns2TZdX0eSW/Cq3iqkYQymkL64ATxphyABF5FPfmQ4f6GxhjXvZqvxO4y3P8mFebGhGpAzKAlhDGO6o0u23URUfqgt2VzbgMrF8wNQPU4B47KshKZGd5E/dcljfqgryblmfx+ol5/PTVcho7HMREDf8OVZidxAfWzw84huLdVSycncCKIHSzPb7rDAtnJ7Bq3uhlMSqG3B3ctjqHvxw+x+snGgZ22iuv70TEnRyVGiqUCSIHOOP1vApYP0L7jwDPDz0oIusAG3AyqNGNQ6o9hrbuPnqdLqKDsKnNTFdS0US0VSgK4MMslG5enk1ctJUv3bgkoPZfu6WQ6pbzvHK0bti5nl4Xj7x1mnctyWTOrNFrR9W1d3PfE3tZOTeZbR+/fMyxe/vzgbPc98Q+3rVkNr+4Z+2o7SsaOomyCLkp7jumdy6Zzay4aLaVVg8kiIqGTnJT4oiJmlnTe1VwhDJB+FpM53P9hIjcBawB3jHkeBbwW+BuT+nxoa/7KPBRgHnz5k003lH1703d3OlgdlLwCsvNVCUVjSzPTSbONrUfPndtmM9dGwL/xuvpaFUAABm+SURBVB8bbeU3H17n81xFQyfv/N4rPLWnmo+946JR3+vpshpcBkpPt1Be38GCcS7Gq2ru4gtP7AUIuEJtRUMn89LiB8ZbYqKsbFqRzWO7ztDe3UtibDSVjZ3kpWn3kvItlF+Dq4C5Xs9zgWETsUVkI/AVYJMxpsfreBLwLPCvxpidvn6AMeZnxpg1xpg1GRkT28c4EFpuI3Bdjj72V7WyfgrHH0IhP91O0bxkindXBVQEsLi0igXpdiwCT5ZVj9rel16ni089UobLuMuS1LR2B1RDqqKhk/whH/5binLo6XPx3P5azywnneKq/AtlgngbWCQi+SJiw13babt3AxFZBfwUd3Ko8zpuA54EfmOM8a4BNaW0YF/gdp9qps9lpnT8IVS2rs7leF0HB0fZjOhQTRtHzrZzz+V5XLEog22l1bhcYy9C8MCOY5SebuHftyzj+qXukiGjlTF3uYx7j4chH/4r5yazIN1OcWk1DR0O2nv6dAaT8itkCcIY04d7N7oXgMPAY8aYgyLyTRHZ5Gl2P5AAPC4ie0SkP4G8D7gKuMdzfI+IrAxVrIHSO4jAlZQ3YbUIq+fPvD0Gbl6WjS3KwhO7fS8861dcWkW0VbhleTZbi3KobjlPSUXTmH7Wa8fq+e9XTnLH2rlsWpFNQVYiMHo309m2brp7XcMqtIoIW1fn8lZFE68dqwcgfwbWoFLBEdKRVmPMc8aYi40xFxljvu059lVjzHbP443GmExjzErPn02e478zxkR7HV9pjNkTylgDMXAHoTOZRlVS0cjSnFkzsr7PrPhori3IZPveGr/rJfqcLp7eU801S2aTYrdxXeEcEmKi2OZnNbMvde3dfO6xPVycmcDXbrkEgNmJsaQn2Dg0SoIYOoPJ23tW5SACD7503N1GxyCUHzoVZwyS422IaBfTaLp7new9M7XrH0JtS1EOTZ0OXjla7/P8a8fraehwsMWzEVKczcqNy+bw3P5auhx9o76/y2X47B/30NHTx8PvLxo00F+QlTTqHUT5CAkiJzmOSxekcbqpi2irkJPie12IUpogxsBqEVLitdzGaEpPN+Nwuqas/tJkuOriDNITbH7vCIpLq0mJj+adnumkAFuLcul0OHnx4LlR3/+/Xz3JGyca+fotl3BxZuKgc4VZSRw/1zHiau/Khk7ioq1k+tnGtT9xzU+zY7VEXPV+FSBNEGOkq6lHV1LehEVgTd7MTRDRVgubVuTw0uG6YTOKWrt62XHoHJtWuMcq+q3NSyU3Jc5v0bx+uyqbeGDHMW5Zkc3ta+cOO1+QlYTD6aLcU8rbl4qGTvLS7Vj8fPjfsHQOcdFWHaBWI9IEMUapWrBvVCUVjRRmJ5EUO7OL/m4pysHhdPHMvsEVYJ/dX4ujzzVslzqLRdhSlMvrJxo429rt8z1buhx86pEyclPi+PdblyIy/AO+v77UodpWv7GNVqHVHhPFzz64ms9ft9hvG6U0QYxRmt5BjKinz0nZ6RbW58+86a1DXZKdxJI5icO6mbaVuktrLMuZNew1W1blYIzvNRHGGD7/+D7qO3p46M5VJPpJsAsy7NisFr9TXXudLk43dZGXPnL5jCsXZbB4TuKIbVRk0wQxRtrFNLK9Z1rp6XPNuAVyvogIW4pyKDvdwsl6977OlQ2d7DrVzNaiXJ/f/vPS7ayen8K20uEL7X7990r+cvgcX7yhgOW5/us2RVstXDwnwe9A9ZmmLpwuQ366Tl9VE6MJYozS7Daauxw4x7HgKRKUlDciAusiIEEAvGdljnuVtKeM9rayakTgPauy/b5ma5F7od2B6gsf8AeqW/nOc0fYWDCbD1+eN+rPLZjjfyZTZaOW8FbBoQlijFLtNowhoFIHkaikoonFmYkkx9umOpRJMTsplisXZfBkWTVOl2FbaRVXLEwna5b/qaM3Lc/CFmUZGKzu6Onj3j+UkpZg4/7bVvi88xiqICuJhg4Hde3DxzL6B6+1hIaaKE0QY5Tq2fRdu5mG63W62H2qOSK6l7xtXZ1Ldct5Hvrrcaqaz7OlKGfE9rPiorm20L3QztHn4l+f3M/ppi4evGMVKfbAEuvAQLWPch8VDZ0kx0cH/F5K+aMJYoy03IZ/+6tbOd/rnJH1l0ZyXWEmiTFR/PCl49htVt59yZxRX7PVs9Dus4/t4ak9NXx248Vj6pYr9CQIXwPVFQ1aoVUFhyaIMZrMgn1/P9lAW3dvwO17+py8fLQuoCqj/c40dQ3bp3i8SsrddYYiZfyhX2y0lZuWZ+EycMOyLOJto5cXuWqRe6Hds/tqueyiND7+zoVj+pmz4qPJSY7zOQ5ROcoUV6UCpQlijCbrDqKioZP3/7yEB148Nnpjj1+9UcmHfvU2O8sDLwj3qUfL+Kff7h5PiMO8fKSOxZmJpHu64SLJHevmYbNauHNdYPuSRFktfPDSPHKS4/jB7SvHtZq5ICtxWII473BS09qtA9QqKDRBjFHKQMG+0CaI/rn1/f3UozHGUOypLhpoQbiT9R2UnW7hVGPXmO5UfDnd2MVblU1sWul/9s5MtnJuMvu/cd2Yqtd+8pqFvHrf1ePefKogK4mT9R109zoHjg3MYMrQBKEmThPEGEVbLSTFRtHUGbqKri6XYVtp9cCivFeP+S4I5+1AdRvH6zpIs9t4bn8t5x3OUV/TPzUT4Mgo+wuMZltZlWd658gDtDPZWLftFJGB3d7GoyArCZeBY+cu/N31V3HVMQgVDJogxiEtISakXUwlFU1Ut5znyzcWkJ5gG7gzGElxaRW2KAvf2bKMToeTFw6eHbG9y2V4sqx6YDZMoNtY+mKMO6FduiCNnGStDDpZCn383Y1U5lupsdIEMQ6hXk29rbSKhJgoblyWxeaVObx05NyI6y4cfS62763h2sJMNhZkBlQQbmdFI9Ut5/nYOxaQEh89oQSx61Qzp5u62FqUO3pjFTTzUuOx26yDZjJVNHSSmRSDfQbuw6EmnyaIcQhlguhy9PHc/lpuXDaHOJuVLUU59DoNz+wdtp33gFeO1tHU6WBrUY67INyqHN4YoSAcQPHuahJjonj3JXMC2l9gJNtKq4i3WQe2w1STw2IRFs9JHLQWoqJh+DajSo2XJohxCGXBvhcPnqPT4Rz4Nn5J9iyWzEmkuNT/hvfbSqtJT7Bx1aIMwF3r32XgqT2+X9Pl6OP5A7XctDyL2GgrBVlJHDnbTt8I+wv4093r5E97a7l+6Rz91joFCrKSOHy2bWBqsyYIFUyaIMYh1VOPaSzrDQJVXFpFbkoca732UthalMueMxcKwnlr6XLw0pFzbF6ZMzDg2V8Qrnj38IJwAC8cPEuXwzmwaUxBVhI9fa6BGTBjsePQOdp7+rR7aYoUZifR3t1HVfN5Wrt6aep0aIJQQaMJYhxS7TZ6nYa286NvHTkWZ1u7ef1EA1tW5Qza6GXzymws4nv66jN7a+h1mmHlHbYU5QwrCNeveHc1c1PjWOOZklk4sL/A2GcyFZdWkT0rlksjbPV0uPCeZFAxUKRPq7iq4NAEMQ7Znpk6VS1dQX3fp/ZUY8yF7SD7zU6K5aqLM3iytBrXkCqyxaXVLJmTyCXZg/ceuHlZ9qCCcP1qW8/zxskGtqzKHUhCC2cnEG2VMY9D1LV389qxet4zJKGpybNkTiIicKi2jYoG9x2m3kGoYNEEMQ79/wD7pxQGQ/9Ct9XzU8jz8Q98S1EuNa3d7CxvHDh2sr6DPWdafHbvzIqP5tqCzGEL7Z4s609CF+44bFEWLsrwv7+AP9v31ODykdDU5Im3RZGXZnffQdR3YhH37CalgkETxDj0L0KqDGKC6F/o5q8vv78gnPdg9bbSKiwCm/3sPbB1tbsg3CtH64AL6xXW5qUwf8hCqsKsJJ+VQUfyxO4qVsxNZuFs7dKYSu6SG+2UN3SSmxI/aB9spSZCf5PGIc5mJWtWLOVBTBD9C91uWp7l83x/QbjnD9TS2dPnXuhWWs1VF2cwO9F3qYYrPQXhtnmSyr6qVk7Udfj8xl+QlURdew+NHYGtED9U08aRs+3cNkppaxV6hVlJnG7q4kB1q3YvqaDSBDFO+en2oHUxeS90mxXnex9icO870OVZJb2zvJGa1u4RZw9FWy0DC+2aOx1sGyEJFWb7Lx/tS3FpFdFW4eblkVl7KZz0D1RXNnZpglBBpQlinIKZILwXuo1kzfwU5qbGsa20muJS90K3awszR3xN/0K7bWXVbN9bw3WFmSTFDk9CYym50ed08fSeat61JFM3pQkD/X93oAPUKrg0QYxTfrqdlq5emoOwYK5/oduVnoVu/ogIW1bl8sbJBp7dXzOw0G0khVlJLJmTyAMvHqW5q9fvHUeq3UZmUkxACeK14/U0dDhG3TlNTY6sWbEDd56aIFQwaYIYp4GZTONYXOatufPCQrfoACp7bi3KxRjo7nWxdfXos4dEhK1FuXQ6nKQnxHDlonS/bQuykjgUQIIoLq0m1W7j6sWzR22rQk9EBtayaIJQwaQJYpwGEkT9xBLET1496XOhmz/z0uJZn59Kfrp9YKHbaDavzCbaKmwtyhmxvHRBVhIn6jro6fNfKrz1fC87Dp1j04psnS0TRlbOSyYxNmpgjY5SwaDFc8Zpbmo8VotMaBzi1WP1/PS1cu5cN2/YQreR/PgDRfQ6DSKBLU6bnRTLnz9z1ailuAuzkuhzGU7UdfiN59l9tTj6XFpaI8x88pqF3Ll23rh2plPKH/0KOE7RVgvzUuPHnSDq2rr53B/3sDgzka/dUjim16YlxDBn1th2IbsoI2HU8YoLA9X+ZzIVl1axaHYCS3OS/LZRky/eFsW8NF0gp4JLE8QEjHcmk9Nl+Mwf99Dp6OPh968a9YN7suSn24mNtvgdqK5s6GT3qWa2FOUGfPeilJq+NEFMQF6aO0GMtarrj18+wd9PNvLNTUtZlJkYoujGzmoRFmcm+l1Rva2sGhG4NYK3FVUqkmiCmID8DDvne52cawt8f+q3Kpr4r78cY/PKbN67Jvz68YfuL9DPvU92FVcsTB9z95ZSanrSBDEBCzwzmcobhu/T4Etzp4NPP1rGvNR4vn3rsrDspinMTqKlq5ezbYN3o3u7somq5vM6OK1UBNEEMQFjqepqjOHzj++lscPBw+8vIiFMd1/zt6K6uLQKu83KdZeMvHJbKTVzaIKYgDlJscREWQKq6vqbN0/x0pE6vnTjEpbmBD6ldbItmeMeE/GeyXTe4eS5/We5cVkW8bbwTGxKqeALaYIQketF5KiInBCRL/o4/zkROSQi+0TkJRGZ73XubhE57vlzdyjjHC+LRQKeyfS7nadYm5fCPZflhT6wCUiMjWZuatyggeoXD52lo6dP931QKsKELEGIiBX4EXADUAjcKSJDJ/yXAWuMMcuBJ4Dvel6bCnwNWA+sA74mIoEtG55k+en2Uct+N3b0cLyug2uWZIbluMNQhVlJg7qYikuryUmOY31+6givUkrNNKG8g1gHnDDGlBtjHMCjwGbvBsaYl40x/ft27gT6v6K+G9hhjGkyxjQDO4DrQxjruOWn2znd2EWf0+W3zVsVTQCsXzA9PmALspKoaOyky9HHubZuXj9ez5Yi3VZUqUgTygSRA5zxel7lOebPR4Dnx/naKZOfbqfPZahqPu+3TUlFE/E2K8vCeOzBW0FWEsbA0bPtPFVWjcvo2gelIlEoE4Svr5s+V5SJyF3AGuD+sbxWRD4qIrtEZFd9ff24A52IQKq67ixvZPX8lICqtYaD/sqgh2rbKC6tomheMgsydFtRpSJNKD+xqoC5Xs9zgZqhjURkI/AVYJMxpmcsrzXG/MwYs8YYsyYjY+S9FEJltKquLV0Ojp5rn1b997kpce79r3dXceyc7y1KlVIzXygTxNvAIhHJFxEbcAew3buBiKwCfoo7OdR5nXoBuE5EUjyD09d5joWdVLuNpNgovzOZ3qpowhhYl582yZGNn4hQkJVE6ekWbFYLN/vZJ1spNbOFLEEYY/qAe3F/sB8GHjPGHBSRb4rIJk+z+4EE4HER2SMi2z2vbQK+hTvJvA1803Ms7IgI+RkJfhNESUUTMVEWVsydHuMP/Qqy3OshNhbOJjletxVVKhKFdNWTMeY54Lkhx77q9XjjCK/9JfDL0EUXPPlp8bxd2ezzXElFI6vmJRMTFR4VWwNVmO0eh9iySruXlIpUuiw2CPLTE3h6bw3dvc5Bpbvbuns5VNPGJ69ZNIXRjc+mFe4tUK9ZotuKKhWppse0mjCXn2HHGDjV2DXo+K7KJlxm+qx/8BZns7KlKFfXPigVwTRBBMGCgaJ9g6u6lpQ3YbNaKJoXlovAlVJqRJoggiBvoOz34IHqnRVNrJg7K2x2jFNKqbHQBBEECTFRZCTGDKrq2tHTx4HqVtZPo+mtSinlTRNEkAyt6rr7VDNOl5mW4w9KKQWaIIJmwZAEUVLeSJRFWD1fxx+UUtOTJoggyU+309DhoPV8L+BeILcsd5ZusKOUmrY0QQRJ/0B1ZUMn5x1O9lW16PiDUmpa06+3QdI/1bWysZOOnj56nTr+oJSa3jRBBMm8tHhEoLy+k5N1HVgE1uj4g1JqGtMEESQxUVZyU+KoaOjkbFs3S3NmkRgbPdVhKaXUuOkYRBDlpdk5craNPWdaptX+D0op5YsmiCBakG7n2LkOHH0uHaBWSk17miCCqH93ORFYq3cQSqlpThNEEOV79m0umJPErDgdf1BKTW+aIIIoP819B6HTW5VSM4HOYgqiualxfOqahbxnVc5Uh6KUUhOmCSKIRITPXbd4qsNQSqmg0C4mpZRSPmmCUEop5ZMmCKWUUj5pglBKKeWTJgillFI+aYJQSinlkyYIpZRSPmmCUEop5ZMYY6Y6hqAQkXrg1CjN0oGGSQgnXEXy9UfytUNkX79e+8jmG2MyfJ2YMQkiECKyyxizZqrjmCqRfP2RfO0Q2dev1z7+a9cuJqWUUj5pglBKKeVTpCWIn011AFMskq8/kq8dIvv69drHKaLGIJRSSgUu0u4glFJKBUgThFJKKZ8iJkGIyPUiclRETojIF6c6nlATkV+KSJ2IHPA6lioiO0TkuOe/KVMZY6iIyFwReVlEDovIQRH5tOf4jL9+EYkVkbdEZK/n2r/hOZ4vIiWea/+jiNimOtZQERGriJSJyJ88zyPp2itFZL+I7BGRXZ5j4/69j4gEISJW4EfADUAhcKeIFE5tVCH3a+D6Ice+CLxkjFkEvOR5PhP1Af9ijCkANgCf8Px9R8L19wDXGGNWACuB60VkA/CfwH95rr0Z+MgUxhhqnwYOez2PpGsHeKcxZqXX+odx/95HRIIA1gEnjDHlxhgH8CiweYpjCiljzGtA05DDm4H/9Tz+X+A9kxrUJDHG1BpjSj2P23F/WOQQAddv3Do8T6M9fwxwDfCE5/iMvHYAEckFbgL+x/NciJBrH8G4f+8jJUHkAGe8nld5jkWaTGNMLbg/RIHZUxxPyIlIHrAKKCFCrt/TxbIHqAN2ACeBFmNMn6fJTP79/wHwBcDleZ5G5Fw7uL8MvCgiu0Xko55j4/69jwpBgOFIfBzT+b0znIgkAMXAZ4wxbe4vkzOfMcYJrBSRZOBJoMBXs8mNKvRE5GagzhizW0Su7j/so+mMu3YvlxtjakRkNrBDRI5M5M0i5Q6iCpjr9TwXqJmiWKbSORHJAvD8t26K4wkZEYnGnRx+b4zZ5jkcMdcPYIxpAV7BPQ6TLCL9Xwhn6u//5cAmEanE3Y18De47iki4dgCMMTWe/9bh/nKwjgn83kdKgngbWOSZzWAD7gC2T3FMU2E7cLfn8d3A01MYS8h4+p1/ARw2xjzgdWrGX7+IZHjuHBCROGAj7jGYl4HbPM1m5LUbY75kjMk1xuTh/jf+V2PMB4iAawcQEbuIJPY/Bq4DDjCB3/uIWUktIjfi/jZhBX5pjPn2FIcUUiLyCHA17nK/54CvAU8BjwHzgNPAe40xQweypz0RuQL4G7CfC33RX8Y9DjGjr19EluMeiLTi/gL4mDHmmyKyAPe36lSgDLjLGNMzdZGGlqeL6fPGmJsj5do91/mk52kU8AdjzLdFJI1x/t5HTIJQSik1NpHSxaSUUmqMNEEopZTySROEUkopnzRBKKWU8kkThFJKKZ80QSg1AhHp8Hp8o6ci5rwJvuc9IvLwxKNTKrQipdSGUhMiIu8CHgKuM8acnup4lJoMegeh1ChE5Erg58BNxpiTQ85ZPDX4k72OnRCRTBG5xbMPQZmI/EVEMn28969F5Dav5953LPeJyNsisq9/XwelJpMmCKVGFoO7NMF7jDHDCp8ZY1ye87cCiMh6oNIYcw54HdhgjFmFeyXvFwL9oSJyHbAIdy2dlcBqEblqgtei1JhoglBqZL3A3xl5k5k/Ard7Ht/heQ7uwnAviMh+4D7gkjH83Os8f8qAUmAJ7oSh1KTRBKHUyFzA+4C1IvJlP23eBBaKSAbuzVj6q8c+BDxsjFkG/BMQ6+O1fXj+HXqKDPZvhynAdzw7g600xiw0xvwiKFekVIA0QSg1CmNMF3Az8AERGXYnYdwFzZ4EHsBdQbbRc2oWUO15fPfQ13lUAqs9jzfj3gEO4AXgw549LRCRHE+Nf6Umjc5iUioAxpgmEbkeeE1EGowxQ0sm/xF3Wfl7vI59HXhcRKqBnUC+j7f+OfC0iLyFe7/gTs/Pe1FECoA3PRsddQB3McP3sFDhRau5KqWU8km7mJRSSvmkCUIppZRPmiCUUkr5pAlCKaWUT5oglFJK+aQJQimllE+aIJRSSvn0/wF+9ZgfbzLctQAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.plot(list(range(1, 50)), error_rate_knn)\n",
    "plt.xlabel('K value')\n",
    "plt.ylabel('Error rate')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's clear that further increasing the K value, the error grows along with it.\n",
    "Let's choose a new K value: K=4 then refit and revaluate the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[153  16]\n",
      " [ 35  64]]\n",
      "              precision    recall  f1-score   support\n",
      "\n",
      "           0       0.81      0.91      0.86       169\n",
      "           1       0.80      0.65      0.72        99\n",
      "\n",
      "    accuracy                           0.81       268\n",
      "   macro avg       0.81      0.78      0.79       268\n",
      "weighted avg       0.81      0.81      0.80       268\n",
      "\n"
     ]
    }
   ],
   "source": [
    "knn_opt = KNeighborsClassifier(n_neighbors=4)\n",
    "knn_opt.fit(X_train, y_train)\n",
    "pred_knn_opt = knn_opt.predict(X_test)\n",
    "print(confusion_matrix(y_test, pred_knn_opt))\n",
    "print(classification_report(y_test, pred_knn_opt))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With a new K value the performance slightly improved, but still did not reached the accuracy of the random forest estimator."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}