muinar06_act3_grupal.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# ACTIVIDAD 3 (GRUPAL): REDES NEURONALES RECURRENTES (RNN)\n",
        "\n",
        "---\n",
        "\n",
        "En esta actividad grupal, vamos a utilizar las RNN para identificar la intensidad de odio en mensajes escritos en las redes sociales. Esta actividad forma parte del proyecto transversal del Máster, ¡así que seguro que ya estás familiarizado/a con el dataset!\n",
        "\n",
        "Por sencillez, se adjunta un documento csv que es una versión resumida del dataset original del proyecto transversal. En este dataset, solo se incluyen los mensajes que han sido catalogados como contenedores de odio. También se han eliminado las features innecesarias para este trabajo simplemente para hacer todo el proceso más rápido computacionalmente hablando.\n",
        "\n",
        "El dataset a utilizar consiste en 12280 mensajes de odio. Los mensajes pueden pertenecer a cinco categorias (numeradadas del 1 al 5), siendo 1 la menor intensidadd y 5 la mayor intensidad de odio del mensaje. El dataset ya ha sido procesado para eliminar ciertos caracteres que podrían darte problemas y para utilizar una codificación más amigable."
      ],
      "metadata": {
        "id": "KVfeIL4OiTON"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Empezamos, como siempre, incluyendo todo lo que vamos a necesitar. Según lo que vayas a hacer, es posible que necesites incluir alguna librería extra."
      ],
      "metadata": {
        "id": "Qeb4aqTDjtZu"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 101,
      "metadata": {
        "id": "2gvn3m1cgZ7-",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "860ea99d-4b60-4b76-ade7-bbadd6eaa97a"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "[nltk_data] Downloading package stopwords to /root/nltk_data...\n",
            "[nltk_data]   Package stopwords is already up-to-date!\n",
            "[nltk_data] Downloading package wordnet to /root/nltk_data...\n",
            "[nltk_data]   Package wordnet is already up-to-date!\n"
          ]
        }
      ],
      "source": [
        "import tensorflow as tf\n",
        "from tensorflow import keras\n",
        "from keras import layers\n",
        "\n",
        "import matplotlib.pyplot as plt\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import string\n",
        "import re\n",
        "import nltk\n",
        "nltk.download('stopwords')\n",
        "nltk.download('wordnet')\n",
        "\n",
        "from nltk.stem import WordNetLemmatizer\n",
        "from nltk.corpus import stopwords\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "from tensorflow.keras.preprocessing.text import Tokenizer\n",
        "from tensorflow.keras.preprocessing.sequence import pad_sequences"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Cargamos el dataset. Lo encontrarás adjunto a este Notebook, al descargar la actividad del Campus Virtual."
      ],
      "metadata": {
        "id": "_xvORCesj6S3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Leemos el fichero\n",
        "data = pd.read_csv('dataset.csv', sep=\";\", encoding='utf-8')\n",
        "# Mostramos las 5 primeras filas\n",
        "data.head()"
      ],
      "metadata": {
        "id": "sI11OfLxgh9H",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "outputId": "1f398392-24c9-400b-ee4a-76f53ed579d1"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                CONTENIDO A ANALIZAR  INTENSIDAD\n",
              "0  cristina cifuentes podrí­a haber sido la presi...           3\n",
              "1  habrí­a que reabrir el caso. el supremo se ded...           3\n",
              "2  me parece un poco exagerado pedir más de tres ...           3\n",
              "3  parece que todos los delincuentes niegan las e...           3\n",
              "4  preguntárselo al fracasado, media carrera en 6...           4"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-11d14c69-3c55-4faa-ba63-2de143d1c5bd\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>CONTENIDO A ANALIZAR</th>\n",
              "      <th>INTENSIDAD</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>cristina cifuentes podrí­a haber sido la presi...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>habrí­a que reabrir el caso. el supremo se ded...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>me parece un poco exagerado pedir más de tres ...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>parece que todos los delincuentes niegan las e...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>preguntárselo al fracasado, media carrera en 6...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-11d14c69-3c55-4faa-ba63-2de143d1c5bd')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-11d14c69-3c55-4faa-ba63-2de143d1c5bd button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-11d14c69-3c55-4faa-ba63-2de143d1c5bd');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-2bfedd86-9b49-47a6-aa37-a35567595da5\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-2bfedd86-9b49-47a6-aa37-a35567595da5')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-2bfedd86-9b49-47a6-aa37-a35567595da5 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "data",
              "summary": "{\n  \"name\": \"data\",\n  \"rows\": 12279,\n  \"fields\": [\n    {\n      \"column\": \"CONTENIDO A ANALIZAR\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 11189,\n        \"samples\": [\n          \"el fmi es facha.\",\n          \"son unos sinverg\\u00fcenzas, nos mienten a la cara y lo saben.\",\n          \"@lavanguardia co\\u00f1o. espero a los de podemos diciendo que son tan idiotas como ayuso. por cierto en el hospital de ayusito ya hay casi 300 enfermos y enfermas de covid ingresados  e ingresadas descongestionando los otros hospitales, y no oigo a los progreburgueses desdecirse de nada.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"INTENSIDAD\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 1,\n        \"max\": 5,\n        \"num_unique_values\": 5,\n        \"samples\": [\n          4,\n          5,\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 49
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Cambiamos el nombre de las features para hacer su manejo más amigable:"
      ],
      "metadata": {
        "id": "WYJWQvEqkB2B"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data = data.rename(columns={'CONTENIDO A ANALIZAR': 'mensaje', 'INTENSIDAD': 'intensidad'})\n",
        "data.head()"
      ],
      "metadata": {
        "id": "nTawYDb6NmmH",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "outputId": "e39d1121-a6cf-4377-9609-03993a5d27b2"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                             mensaje  intensidad\n",
              "0  cristina cifuentes podrí­a haber sido la presi...           3\n",
              "1  habrí­a que reabrir el caso. el supremo se ded...           3\n",
              "2  me parece un poco exagerado pedir más de tres ...           3\n",
              "3  parece que todos los delincuentes niegan las e...           3\n",
              "4  preguntárselo al fracasado, media carrera en 6...           4"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-8437844b-6f34-46a9-be2e-2eed0521e902\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>mensaje</th>\n",
              "      <th>intensidad</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>cristina cifuentes podrí­a haber sido la presi...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>habrí­a que reabrir el caso. el supremo se ded...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>me parece un poco exagerado pedir más de tres ...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>parece que todos los delincuentes niegan las e...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>preguntárselo al fracasado, media carrera en 6...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-8437844b-6f34-46a9-be2e-2eed0521e902')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-8437844b-6f34-46a9-be2e-2eed0521e902 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-8437844b-6f34-46a9-be2e-2eed0521e902');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-c652ae0b-d34b-4bdc-b34b-230e44bff678\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-c652ae0b-d34b-4bdc-b34b-230e44bff678')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-c652ae0b-d34b-4bdc-b34b-230e44bff678 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "data",
              "summary": "{\n  \"name\": \"data\",\n  \"rows\": 12279,\n  \"fields\": [\n    {\n      \"column\": \"mensaje\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 11189,\n        \"samples\": [\n          \"el fmi es facha.\",\n          \"son unos sinverg\\u00fcenzas, nos mienten a la cara y lo saben.\",\n          \"@lavanguardia co\\u00f1o. espero a los de podemos diciendo que son tan idiotas como ayuso. por cierto en el hospital de ayusito ya hay casi 300 enfermos y enfermas de covid ingresados  e ingresadas descongestionando los otros hospitales, y no oigo a los progreburgueses desdecirse de nada.\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"intensidad\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 1,\n        \"max\": 5,\n        \"num_unique_values\": 5,\n        \"samples\": [\n          4,\n          5,\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 50
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Mostramos información básica para saber a qué nos enfrentamos:"
      ],
      "metadata": {
        "id": "rSrgxwcHkIpk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "data.shape"
      ],
      "metadata": {
        "id": "D8sOe2k-jb9r",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "276c4236-c61e-4307-9837-cae0bf7aba5b"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(12279, 2)"
            ]
          },
          "metadata": {},
          "execution_count": 51
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "data.info()"
      ],
      "metadata": {
        "id": "r70C7LsWnOTP",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a491e8f0-e65e-42de-8dc2-d44ed298b2fd"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<class 'pandas.core.frame.DataFrame'>\n",
            "RangeIndex: 12279 entries, 0 to 12278\n",
            "Data columns (total 2 columns):\n",
            " #   Column      Non-Null Count  Dtype \n",
            "---  ------      --------------  ----- \n",
            " 0   mensaje     12279 non-null  object\n",
            " 1   intensidad  12279 non-null  int64 \n",
            "dtypes: int64(1), object(1)\n",
            "memory usage: 192.0+ KB\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Mostramos ahora la distribución de mensajes de acuerdo a su intensidad. Verás rápidamente que estamos en problemas: hay un gran desbalanceo."
      ],
      "metadata": {
        "id": "uDpDVmNAkNDZ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "plt.pie(data['intensidad'].value_counts().values,\n",
        "        labels = data['intensidad'].value_counts().index,\n",
        "        autopct='%1.1f%%')\n",
        "plt.show()"
      ],
      "metadata": {
        "id": "HATeYyBcnTry",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 406
        },
        "outputId": "c3bfa5f6-8d0f-4163-b46a-71db7a6789c3"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYUAAAGFCAYAAAASI+9IAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA/fElEQVR4nO3dd5hU5cE28Hv6zM7O9t5Zdll6EUEFRFEsiEQ0akRUsJsgn5qYEI0xmmiM9dXX19gSO2piQWIBbCBh6Z0FlrLL9t5mp+z08/0xZHBD2zK7z5yZ+3ddXFw75cw9yzL3nud5zjkKSZIkEBERAVCKDkBERKGDpUBERAEsBSIiCmApEBFRAEuBiIgCWApERBTAUiAiogCWAhERBbAUiIgogKVAREQBLAUiIgpgKRARUQBLgYiIAlgKREQUwFIgIqIAlgIREQWwFIiIKIClQEREASwFIiIKYCkQEVEAS4GIiAJYCkREFMBSICKiAJYCEREFsBSIiCiApUBERAEsBSIiCmApEBFRAEuBiIgCWApERBTAUiAiogCWAhERBbAUiIgogKVAREQBLAUiIgpgKRARUQBLgYiIAlgKREQUwFIgIqIAlgIREQWwFIiIKIClQEREASwFIiIKYCkQEVEAS4GIiALUogMQBZvD7UWrzYV2mwvtdhfa7W502F1ot7nh8HghSYAkSZBw9G8J8EmABOm/7gM0KiXiozSIN2qRYNQiPuro30YNEqK0UKv4exWFF5YCyY7Z7saBRgsONlpQ29GFxk4HmjqdaOx0oKHTAYvDM2hZTHp1t7JIMemQn2xEQUo0ClNMyIo3QKFQDFoeov5SSJIkiQ5BdCIOtxcHGy040OAvgNKjfzd2OkVH6zGDRvWjkohGQUo0ClJMyEuM4l4GhSSWAoWEdpsLm460Ym9dZ6AEqtrs8IXpT6dGpUBeohHD0kw4Iycek/LiMSojFiol9ypILJYCCWFzerDpSCvWH27F+rJW7G/oRKT/JBq1KozPicPE3AScNSQBE3PjodeoRMeiCMNSoEHh9HixrbIdG8paUXy4BbtrzPCE625AkOjUSkzMjcfUgiRMLUjCmEzuSdDAYynQgNlbZ8aaA81YX9aCbZXtcLh9oiPJWoxejakFSbhsTDouHJGCKC3XiVDwsRQoqCpabFi+sw7/2lWLsmab6Dhhy6BR4YIRKZgzNh3nF6VwmImChqVA/dbU6cC/dtXhX7vqsLvGLDpOxDFqVZg5MhWXj83A9GFJ0KlZENR3LAXqE3OXGyv21GP5zjpsOtIatquE5MakV+OikamYMzYD0wqToOGyV+ollgL1mMvjw9f7GvDZjjqsPdgMl5dzBKEs1qDBlRMysXBKHvKSjKLjkEywFOi0mjodeG9jJd7fXI0Wq3wOHCM/pQK4YHgKbp46BFMLkkTHoRDHUqCT2l7VjreKK7CipB5uL39MwkFRqgkLp+bhygmZnJymE2IpUDc+n4RVexvw6tpy7KzuEB2HBkh8lAbXTc7BTefkIj3WIDoOhRCWAgHwH1z26fZavL62HOUtXEoaKdRKBS4dnYabpw7BxNx40XEoBLAUIpzV6cE7GyrwZnEFmi2cL4hkk/MSsGRWESbmJoiOQgKxFCKUx+vDB5ur8MJ3h9BidYmOQyFk5ohULLm0CIWpJtFRSACWQgRaWdKAp1aWcpiITkqlVOCqCZm476JhyIjjnEMkYSlEkO1V7Xjiq/3YUtEuOgrJhE6txIIpefjF+UMRF6UVHYcGAUshAlS22vDUygP4ck+96CgkUzF6Ne46fyhumTqES1nDHEshjLXbXHjhu0NYuqmSxxlQUKTG6HDPhcPws0nZPI13mGIphCG314e/rzuCl1YfHtTrFVPkGJ0ZgyeuHIsxWbGio1CQsRTCzN46M+7/aDf213eKjkJhTqVUYME5efjVxcNg1PHaDuGCpRAmXB4f/u/7Q/jrmjJe0YwGVUasHo9eMRoXjUwVHYWCgKUQBkpqzbj/o10obbCIjkIRbPaYdPzxilFIjNaJjkL9wFKQMZfHh//97hBe+YF7BxQaEo1a/GnuaFw2Jl10FOojloJM7a7pwK8/2o0Djdw7oNAze2w6/nTFaCQYeWyD3LAUZMbp8eL5bw/htbXl8HLvgEJYUrQWT1w1lnMNMsNSkJHShk4sfn8HDjVZRUch6rE7p+fjN5cO53ENMsFSkIl/7arDbz/ZDbvLKzoKUa+dnZ+AF+edgWQTJ6FDHUshxHm8PjyxohR/X3dEdBSifkmN0eGl68/AmXk8NXcoYymEsBarE3e/vx0by9tERyEKCrVSgd/OGo7bzs0XHYVOgqUQonZWd+Dn721DvdkhOgpR0M0ek44nrx6LaB4JHXJYCiHog81V+MO/9sLl8YmOQjRghiYb8coNE3kxnxDDUgghTo8Xf1i+Fx9uqRYdhWhQRGlVeOKqMbhifKboKHQUSyFE1Ju7cNd727GrukN0FKJBt/iCAvzq4iLRMQgshZCwt86MhW9uQbPFKToKkTDzz8rBn64YDSWPZxCKpSDYhrJW3PHOVlicvO4B0WVj0vD8zyZAq1aKjhKxWAoCrSypx//7cCcnlIl+ZGpBIl678Uxeo0EQloIgSzdV4veflYCnLyI63tisWLy5cBJPwy0AS0GAv645jKdWHhAdgyik5ScZ8c6tk5EVHyU6SkRhKQyyZ1YdwP+tPiw6BpEspMXo8c6tkzGMxzIMGpbCIHrsi334G89hRNQrsQYN3lg4CRNz40VHiQgshUEgSRJ+v7wE722sEh2FSJYMGhXeWDgJ5wxNFB0l7HHd1yD47Sd7WAhE/dDl9uL2d7ZiT41ZdJSwx1IYYE+s2I9/bOVpK4j6y+r0YOGbm1HWzItMDSSWwgD627/L8eoP5aJjEIWNVpsLN/5tE+o6ukRHCVsshQGyfGctHv9qv+gYRGGnzuzAjX/fhDabS3SUsMRSGABrDzbj/o92gVP4RAOjrNmGBW9shpWnhwk6lkKQ7Tp6cRy3l41ANJD21Jpx+9tb4fTwuuXBxFIIovJmK255awtsLv6QEg2GDeWtuPv9HfDyfDFBw1IIkqZOB256YzNaOc5JNKi+2deIJZ/sBg+5Cg6WQhB0OtxY8OYW1LRzRQSRCB9vq8FTq3g+sWBgKfSTx+vDHe9sxf76TtFRiCLay2vK8PmuOtExZI+l0E9/WVGKjeVtomMQEYDffLwb++r4C1p/sBT6YWVJA09wRxRCutxe3PneVrRzbq/PWAp9VNFiw68/3iU6BhH9l+q2Ltz9wXauSOojlkIfONxe/HzpdlgcPHCGKBQVH27FU6tKRceQJZZCHzy8vIQTy0Qh7rW15fh2X6PoGLLDUuilf26txj+31oiOQUSnIUnArz7aheo2u+gossJS6IX99Z14eHmJ6BhE1EPmLjfufn87XB6f6CiywVLoIYvDjV8s3Q6Hmz9cRHKyq8aMx7/cJzqGbLAUeug3H+/GkRab6BhE1Advb6jE6tIm0TFkgaXQA0s3VWJFSYPoGETUDw8u2wOLwy06RshjKZxGg9mBv3zFpW1EcldvduDPvPDVabEUTuOhz/bAwgt5EIWFDzZXo/hwi+gYIY2lcApf7K7Dt/s5DkkUTn776W7YXfxF72RkXQp/+ctfoFAocO+99wZ92x12Fx75F1csEIWb6rYuPLWSp9k+GdmWwpYtW/Dqq69i7NixA7L9x77cjxarc0C2TURivb2hAlsqeHbjE5FlKVitVsyfPx+vv/464uPjg7794sMt+Hgbj1omCleSBCz5eDccbl4697/JshQWLVqE2bNnY+bMmUHfdpfLiwc+3RP07RJRaClvseF/vjkoOkbIUYsO0Fsffvghtm/fji1btgzI9p/75gCqeK4Uoojwt3VHcNmYdIzLjhMdJWTIak+huroa99xzD5YuXQq9Xh/07e+pMeON4oqgb5eIQpPXJ+G3n+6Bj9deCFBIkiSb78Znn32GK6+8EiqVKnCb1+uFQqGAUqmE0+nsdl9v+HwSLn9xHfbxlNhEEefZa8bhpxOzRMcICbIaPrrwwguxZ0/38f6bb74Zw4cPx5IlS/pcCADw8bYaFgJRhHrum4O4fFw6dOq+f4aEC1mVgslkwujRo7vdZjQakZiYeNztveFwe/EcJ5yIIlZtRxfeXl+BO6YPFR1FOFnNKQyUv687goZOh+gYRCTQS6vLYLbzhHmymlMYCK1WJ85/eg3Pb0REuHN6Ph64bIToGEJF/J7Ci98fZiEQEQDgrfUVqOvoEh1DqIguheo2O5ZuqhQdg4hChNPjw7NfR/b8YkSXwovfH4LbG9GjZ0T0X5btqMH+CF6JGLGlUNFiw6fba0XHIKIQ45OAJ1dG7oW1IrYUXvjuEDw8ipGITmDNgWasL4vMi/FEZCkcbrJg+U7uJRDRyb3w7SHREYSIyFJ48fvD4E4CEZ3KpiNtKKk1i44x6CKuFBrMDny5u150DCKSgTfWHREdYdBFXCks3VTJuQQi6pEvdtejyRJZZzuIqFJwerz4YHOV6BhEJBMurw/vboisY5kiqhS+2FWPFqtLdAwikpGlm6oi6rKdEVUKb2+oEB2BiGSmzebCsh2Rs1oxYkphW2UbdtdE3koCIuq/SJpwjphSeGt9ZI0LElHwHGqyYu3BZtExBkVElEJjpwMrS7gMlYj67u8RsrcQEaWwdGMlT3xHRP2y9lAzDjdZRMcYcGFfCi6PD+9zGSoR9ZMkISKWp4Z9KXy5p47LUIkoKD7fXQ+P1yc6xoAK+1L4bEed6AhEFCbabC78EOYTzmFdCh12V8Se/paIBsanYX7MQliXwtd7GznBTERB9d3+RlgcbtExBkxYl8KXe7gMlYiCy+H2YUVJg+gYAyZsS4FDR0Q0UD7fFb5zlWrRAQaKHIaOJJ8X5nXvw7pvDXy2dqiiE2AcfSFip1wHhUJx3ONbV/0frDtXIv6C2xEz6YqTbrdj3VKYiz/odps6IQuZt78S+Lrtu9dhK/kOCo0ecectQPSoGYH7bKXrYCv5DilX/yEI75Io/Gwoa0WH3YW4KK3oKEEXtqUgh6Gjzk2fwLJzBRJn3wdtUg6c9YfQuuIFKHVGxJz5k26PtR9cD2fdAaiiE3q0bU1SDlJ/9vixG5THdgrthzfBtv8HpFz7J3ja69C64gUYhpwBVVQsfE4bOta+g9TrHgvKeyQKRx6fhK/3NuLaSdmiowRdWA4fme1uWQwdOWv3w1BwFqKGToI6NhXG4dNgyJsAV/3Bbo/zWFrQ9s2rSLr8fkDZwx5XqqCKjj/2Jyo2cJe7tRr67DHQpRfCOPI8KLRR8JgbAQDtq9+EacJlUMekBO19EoWjr8L01DlhWQqr9jaE/NARAOgyR8BRuQvuNv8SN1dTORw1+6DPnxh4jCT50PLFc4g56ypok3N7vG1Pex1qXroJta/ciubPn4ansylwnzZ5CFwNh+F1WOFsOAzJ44Q6PgOOmr1wNZbBNHFO8N4kUZgqPtwCc1f4rUIKy+EjOQwdAUDM2VfD57Sj7vW7/MM7Ph/ipt/YbXy/c+PHUChVME38ySm21J0uvQiJl90HTUImvNY2mIs/QMPSJci45SUodVEw5E+EcdT5aHj7PijUWiTNvg9KjQ5tq/6KxNn3wbLjK1i2fwGVIQYJl9zdqzIiihRur4Rv9zXipxOzREcJqrArBbkMHQGAff+/Ydu3Bklz7ocmOReuxnK0f/c6VNGJiB5zIZwNh9G57V9IX/DCCSeeT8Yw9MxjX6QMgS6jCDUv3wJb6TqYxl0MAIibNh9x0+YHHtax7n3o88ZDoVTBvOEfyLjlJXQd3ozWL59D+sIXgvaeicLJDwebWQqh7vsDob/q6D/a17yJ2LOvhnHkeQAAbXIePJ1NMG/8yF8K1Xvhs5lR+/LNx54k+dC++u/o3LocWT9/o0evo9RHQ5OQCU/HiZfRuVurYdu3GukL/xfW3d9AnzUaqqhYRA0/F60rXoDPaYdSF9Xv90sUbjaUt4qOEHRhVwoby9pER+gxye0EFN2ndRQKJSD5T7hlHD0D+rxx3e5v+ufDMI66ANFjZvb4dXyuLng66qEyzjjuPkmS0LrqJcRfcBuUWgMg+SD5PEefePRvKbxPAEbUV80WJw43WVCQYhIdJWjCrhQ2V8inFAwFk2Fe/w+oYpKhTcqBq7EMnVs+Q/TYiwAAKkMMVIaY7k9SqqEyxkOTeGyXtfHDB2EoPAcxRyeI27//OwwFk6GOTYHH0gbzuqWAQhnYI/kx665VUBliEFVwFgD/5HfHuvfhrC1FV/k2aBJzoNRHD9B3gEj+NpS1shRCVZPFgSMtNtExeixh5p3o+Pd7aPv6r/DZzVBFJyB6/CzETb2uV9txtzdA19UZ+NpjaUHL50/D29UJlSEWuqyRSLvx2W7LUgHAa2uHecM/kXbD04HbdBlFiJl8JZo+fhTKqFgkzb6vf2+SKMxtKG/FjefkiY4RNApJkuQxAN8Dn++qw+IPdoiOQUQRJMGoxbaHZvZqMUgoC6vjFDYfkc/QERGFhzabCwcaw+cynSwFIqJ+2lAWPquQwqYU2m0uHIyAi2oTUehhKYSgzRVtCJ/ZESKSk01H2uDzhccHUPiUAoeOiEgQc5cb++o7T/9AGQibUth0JHx234hIfjaGydHNYVEKFocb++s5n0BE4uyt455CyNhdY4Y3TMbziEieDobJstSwKIXShvD4xyAi+SprtobFZHNYlMJBlgIRCeZw+1Ddbhcdo9/CoxR4fAIRhYCDjVbREfotLErhUBj8QxCR/IXDvILsS6Gm3Q6r0yM6BhERDrEUxDvcxL0EIgoNHD4KAXK6fgIRhbeyZqvsl8fLvhQqW+U/209E4cHp8aGqTd6fSbIvBe4pEFEokftks+xLobKVpUBEoUPu85yyLgWP14ea9i7RMYiIAuo65P2ZJOtSqDc74JH5pA4RhZc2m0t0hH6RdSm02+X9zSei8NNqlffnksxLwS06AhFRN602p+gI/SLrUujgngIRhRgOHwnUwT0FIgoxHV1uWR/AJutS4JwCEYUaSZL33oKsS4F7CkQUilgKgnBOgYhCkZwnm2VdClx9REShSM7LUmVdCh1dLAUiCj0cPhKEw0dEFIpaWQpitMv4G09E4cvqkO/VIGVbCpIkwcLLcBJRCPJJPE5BCBl/34kojHl8PtER+ky2paBQKKBQiE5BRHQ8OR/RrBYdoD9UCgU83F0gGhSO6hJ0bvoErsYyeK1tSL7yd4gads5JH28/sB6WHV/B1VQOyeuGJikHcVOvhyF/YuAx1r2r0fHD25BcXTCOmYmEC28P3OcxN6LxH79H+oLnodRFDeh7CzY5l4Js9xQAQMldBaJBI7kc0KTkI+Giu3r0eEd1CfRDxiPlmkeQvuB56HPGoumTP8HVWAYA8NrNaFv5IuJn3IKUn/0Jtn1rYD+8OfD81q//ivjzFsquEADI+jovst5TUCoBeEWnoHBgVPvwzpDv0YZktDvjoHdEwWDXwas0wKOKgkuhhUfSwO1RwuNRAPL9P993oy7x/wGwcNmfcZ3WhIn6xJM//vIHun+dMRYPlm3FWRV7cEXuZJS3NuB5XTSemXAlAOCvQ85EnrkVl+kTsXHP19ikicI94+YM1LsZUBmSQXSEPpN1Kai4p0BBYvMosdmVi583/gFehQp7ssZgQ14m2uxKGFo8yG/XIbfZA1NtBxS1zfAa4uBNSIM3Nhme6AR4ouLg0cXArTHCpTTABR1cXhWcbgWcTsBp94TdwghHuxPWBnuPH++TfOjqskHtNsDaYEe0JwkulwP7S3YhIToVZVX7cGbuRWiqbMTH376Ke+Y826vthxJFXky/nv/EE0/g008/RWlpKQwGA6ZMmYInn3wSRUVFQUp4crIuBQ4fUTA9WVmI2QWXI6fmC4yv3onx1TsBAOaoeGwYPhZfTIzG+i47mru6MNptwgSbEwUdLUhvbkDcYTOUlXWQuk58fV5JqYIUlwxvfBq8cSnwmBLhMcTBY4iFWxMNl8oAF7Rw+TRwuRVwOAGn3QufjIch/tt3u/4Jp7sLZww9DwAQpTPhxhlL8M7qJ+H2ODF52EUYmT0JS9c8jfNGzUVrZwNeXfl7eH0eXHbmTZiQf57gd9Bz/f1o+uGHH7Bo0SJMmjQJHo8HDz74IC6++GLs27cPRqMxOCFPQt6loGQpUHDNq/kp1hq3QmVrCNwWa2/HpQd+wKVHvz6UWoT1qUkoTvHiw84SuIb4D6JUSMAIdzom2JMwrN2A9BYvYurMUFXWAzYbFG0NULY1QNOLPL7YJP8eSVwqPKZEeKPi4NbHwqONhksVBbdCB6dPA5dHCacLcNi98HpCr0i2HPoOK7a9izsu+SNMhvjA7eOGTMO4IdMCXx+q24XatiO4ZupiPPLhTbj5wt8hJioBTy9bhIL0sd2eG8r6+9m0cuXKbl+/9dZbSElJwbZt2zB9+vR+bft0ZF0KKpYCBVmtQ4fnUxbjV7bfnfQxhY0HUNh4AAsAODQGbMkZj/WxiSh2tWKfrRb7tM1AHIAhx55T5E7FGfYUDOvQI6PZh9i6Tqiq6iFZrKfMozS3QGlugQYlPX4PvuhYeBPS4YtLgceUBG90Atz6WLi1JrjVUXApdHBJGrg8Kn+RdPngcQ3cuvqth7/H+2ufxa0zH8bwrIknfZzb68I/1r2ABTN+i+bOWvh8XhRmjAMApMRmoaJxP8bkTRmwnMGk0qiCuj2z2QwASEhICOp2T0TWpcDhIxoIL1YNwRWFP0VB9Senfaze3YVzyzbg3KNf18dnozhjOIp1KmyyVsHi9n/oH9C04kBsKxALIPfY8wvdqZhgT0KR2YjMZi9i6y1QVzZA6uzsc36l1Qyl1QxUlULXw+f49Eb4EtPhjUv1z5MY4+ExxMGti4ZbZYRbqceOI/vxafE/UV5/EABQ0bS/22/5J7L18PdYuuZpzJp4E15b9XukJwzBA1e/Frh/y6FvsXzT3+B0dyElNgsjsychO3kYqlsOwetz49EPb8JvrnoZXp8HPkk+B4SpNMFb2Onz+XDvvfdi6tSpGD16dNC2ezIyLwXRCShcXV/1E6yP2QS1paZXz0tvr8bV7dW4GoBHqcaezDEoTszAesmGvZaK4z7YDmlaceg/ZZFz7PZ8TzIm2lIwrDMKWS0S4uosUFc1QOow9//NnYDSYYOy9jDUtYdP+phDnZ0YYrNhpkGLp6yAsWIVUlRVMCWmIyU1Dy+UbEWTswu/vfL/wa2OwtclG/Hu6mcx74LbsWLzMhRkjEWnvR1dTisMumhYu8x4/4dnccP5vwEAvPX9n3HhuGsBAKlxOXB5nBieORFl9XvQ2FGF3JSBn2QNFo02eKWwaNEilJSUYN26dUHb5qnIuhQ4fEQDpcmpwRPaxXgIv4Wij+tP1T4PJlTvwITqHbgbQEdUAjZmj8U6YxQ2dNWjydF60ueWq9tRHtvuL4tsABP8t+d6kjCxKwVFHVHIbgXi66z+smjv6FPG3ohTq/BJ57FS+qyjHZ9tW4+5MTH4c3oGOuvr0Op2I+XjxwAAa6oq4fV58d63rwAAWjsbAQD//uou/HHyBdjl8cKk1eLWsSn4+UcvYlT2MCgUNcjOkPD9rg3ISy3E3upN2Hnk37h26mLEGZMH/D0Gi1oXnOGju+++G1988QXWrl2LrKysoGzzdBSSJN+Fcpf8z1ocaLSIjkFh7KvCzzGy+oMB2fbB1OFYnzoE65Ru7LBUwOXr+1l/szyxmOhIwYiOaGS1Skiot0FT1QiptS2IiY8ZeaAU/5uRiZkm0ykf96m5A//o6MDSnFy80tqC76xWLMvzT7aYvV7MLC/D29k5yNBocG1lBR5OTcUYvQHXVlbgrewcpGs0kBQK+OJT4I1PhTc2BT5TEtxR8fDoY+DWRMOt9i8BdvrUcLmV/pVbXV74vOI+2s6bNwyjz+v7h7gkSVi8eDGWLVuGNWvWoLCwMIjpTk3WewrJJh1LgQbU/IpZ2JywCRpzedC3PayxFMMaS7EQQJc2Cluyx2N9TAKK3a2osNX2als1ajNqos1YHg0gC4B/fhbp3nic2ZWKEZ3RyG5RIKHBBm1VE6TmliC/m+NVuFz4n+ZmvJuTC/UJ5v9iVSo8kZaOB+rr4ZB8+ElMDKYZo/FQQz3mx8Wj1u3GotoaeCQJi5LMuKStsVev7zMl+FduxafCa0qCxxgfWLnlVhngUujhkjRwupWBCXevOzjzFrqo3qwxO96iRYvw/vvvY/ny5TCZTGho8K+Gi42NhcEwsAfGyb4UiAZSu1uN3ysX4QnFr6EYwIlOg8uO6WXr8Z/FhnXxOSjOKEKxVoVNtkpY3bY+bbdeZcHn0RZ8Hg0gA8BY/+0pvlicaU/DKIsJ2S0KJDbYoa1ugtTYHIy3A68k4Tf1dViUlIQ8rfakj5tpMnXb29hit+Og04nfpaTi0vJyPJORgSS1Cj+rrMSZhigkqnv+kaW0tEFpaYOmcl+Pn+MzxsCXkA5PXAp8MUnwGBP8x5IEVm4dLZKjK7ecDh/czuN/LnTG/n20vvzyywCA888/v9vtb775JhYuXNivbZ8OS4HoND6sT8fPCm/EhOq3B+01M9qrcE17Fa6Bf8J6d9ZYFCeko1iyYr+lst8rcZqUNnwVXYavogGkAxjjvz3JF4tJXWkY0RmN3DYVEutt0FU3+8uiFyPNNp8PJQ4H9jsceLzR/xu+D/6zg4w5UIrXs7Jx9n8dhOXy+fDHxgY8mZ6BKpcLXkiYFOU/71GeVovdji7MiD71cFV/KW2dUNo6oa4+0OPnSDoDvInp8MWlwhOTDG90AuK0hQBOcQqQ021T4Ki+vEshmqVAg+OGIzOxPXkjdO09/7AIFrXPgzOqtuOMKmAxgHZjIjZkj0VxVBQ2dNWi2RG8eYMWpQ0rjGVYYYS/LEb5b4/3RWNSVxpGWmKQ26YEniqFIiEO8PhOWBbRSiWW5w3pdtsHHe3YZLfj+YxMZGqOH155pa0V04xGjNTrsc/h6HYGZLckQeAUwSkpnF1Q15UDdeX4zz6R4cEbhWbqD3mXAvcUaJDYPCr8xvdzPK+8Hwqf2Cv+xdtacVnpalx29OsDaSOwPiUPxUo3tluOwO1zB/0125VdWKE6jOVOF3D0F/zHx7Tib8MTcIY6A1NUafh22U5Ym614Ji8fyvpGFOq6//9MUKmgVSiOux0ADjudWNHZiU+OFkm+VgulQoFPOjqQpFbjiMuFMXp90N/XQFENwkFmA0XWpZDEPQUaRMsbUzCv8GacXf266CjdFDXsR1HDftwMwK41Ymv2OBTHJqDY1YxKW32/t287YEPLVy2wHbbBZzs2bNXwQQMa0IDKqXXYcnsWatS1cGlcOMdpRsf+juO2E6/RIEXpX7//eacZ/9PcDLvPh7kxMShxOrEkJRVRSiVq3S7cVl2N36ek4JnmZrgkCQ+lpCL1BHsXoUgRFQVlPwrs5Zdfxssvv4yKigoAwKhRo/Dwww9j1qxZQUp4arJeknqgwYJLnl8rOgZFEIPKi+3pT8LQ0vPTTohUk5CD9RnDUaxVYrO1bxPWlt0W2A/ZYcgzoOrFKuQszkHMxJOfBdRr98L349Nm+IDDvz+MhJkJGDo3GyNaEvDeA6tx/+wpGOHT4d6VP+Cx9HScb/DPH9xZU42rY+Nw0WmWu4Yq7ZAhGLriqz4///PPP4dKpUJhYSEkScLbb7+Np59+Gjt27MCoUaOCmPTEZL2nwOEjGmxdXhXucdyFV1W/gsLrFB3ntLLaqnBtWxWuhX/CelfWOBQnpKHYZ8F+SyWkHhyYZxprgmlszz+gVVEqqKKOHbzVua0TXrsX8efGw6pw4QdLOaQoBT6/sh2fA/A0GvDnPA92zciD899muH1WTJ0+FahpA2rqAY/Y4bre0qSn9ev5c+Z0v4bE448/jpdffhkbN25kKZxOfJQGGpUC7lCdgaKw9HVLAtYU3oYZ1S+JjtIrap8HE6u2YWIV8P8AtBmTsCF7DNZHRWG9vRYtzoE50K19bTuMI43QJvmnYXWpOvhcPnRVdkGTqEHXkS7EnxuPtb5KlH1bhrwleViYuNf/WEmFM5xZGGONQ367FqlNLhhr2oDqesAd/LmTYFCnpQdtW16vFx999BFsNhvOOefklz4NJlmXgkKhQKJRh4ZOh+goFGHuKJuCHZkbEN28XXSUPkuwtWB26WrMBiBBgQPpI1CcnIv1Sjd2BGnC2t3uhmWPBdl3ZQduUxlVyLo9CzWv10BySYibEgfTGBNq/l6DhAsT4G5xo+qFKkheCSlzU7Bhkhcb9DVAEoCjB/ZqJRXGu3IxzhKP/A4tUpvciK5pg6K6HpKr70eGB4MmI6Pf29izZw/OOeccOBwOREdHY9myZRg5cmQQ0p2erEsB8A8hsRRosLl9Ctxlux3van4FhVueVwf7MQUkDK/fh+H1+3ArALsuGluyx2GdKQ7rXS2osvdtwrqjuAOqKBVMZ3QffoqZGNNtXsJWaoOzxomMGzJwcMlBZN+VDXWsGmV/LIOxyAh1TPePKpfCi826WmzW1frLosB/u1pSYpwrG+OsiSg4Wham2g4oquogOQdnuE+TldnvbRQVFWHnzp0wm834+OOPsWDBAvzwww+DUgyyL4WseAP21A7MmSOJTmVdWyxWFtyBWTXPi44SdFFOK847XIz/XOusOjEX69OLcB1KoFP1bC5PkiS0/7sdcVPioFSf/KyhPrcPde/UIeuOLLiaXJC8EozD/etedWk62MvsiJnQs8tbehQ+bNPVY5uu3n/s2FD/7SooMMaVhfG2RBS06ZDW7IGprgPKqjpIXcH9pVIbhBPXabVaFBT4m27ixInYsmULXnjhBbz66qv93vbpyL4UClKiRUegCPaLsknYmXM2Yhs3io4yoLJbK/Gz1kpcB+DPDQ3I9U3C+oRUFHstKD3JhLWt1AZXowvx0099tbTmfzUjekw0DHkGdFV2+Q99PkrySN2+7isvJOzUNmCntgGIR7eyGOXKwHhbEgo69Ehv8iCm7tSXVj0dTXbO6R/USz6fD85B2tNhKRD1gyQpcGvHzfhIWwKF69RXUZMrq0vC4bZjn8zVbS4UbNqEKw0K3BOrxL1rldjtMeGs+6divb0Wbc52AP4JZkO+Afqsk6/Zd9Q6YN5sRsEf/b8V69J1gAJo+6ENmlgNnPVOGPIH7gRwXkjYrW3Cbm2TvyyOHoStkIBR7gyMsyceu7Rqrdl/tTzbyZf1KqOjoUlN6VemBx54ALNmzUJOTg4sFgvef/99rFmzBqtWrerXdntK9qVQmCLPtcwUPraaTVg29Oe4qvZp0VEGxNY6L2a8fWze5JdfOwE4sWCcBm/NNaCjzQZfhwVPbP8KEhQoTR+Bb4wZ+OW2/ciYf/JJV0mSUPdWHdLmpUGp8w8vKbVKZN6Wifp36yG5JaTfmA5N/OAftCYpgBJtE0q0TccurTrJf99wdxrOsCWj0KxHZrMPMT+6tKp2yJBTbLVnmpqacNNNN6G+vh6xsbEYO3YsVq1ahYsuuqjf2+4JWR+8BgAOtxej/rAKXp+s3waFgW1DXkFiPQ+m/DGbzoTNOeNQbIrFemczqu0NoiMNmEJ3Iq7NnI3rZv1adJR+kf2egl6jQna8ARWt8l8BQvJ2U8uN+Fy/G0pHh+goIcPotGDGoXWYcfTr6sQ8FKcPQ7FGgc3WStg94fP/9pCmFc6c/g0dhYLgXUhUoKI0DiGReHstRryfsEh0jJCW3VqB60q+xos7VmFdWRnekFJxa9wYjDDlQgH5X163IK5AdIR+k/3wEQD873eH8Nw3B0XHIAIAbMx/E2l134iOITst0SnYkD0axQY9Nthr0ObsEB2p17675jukRMl7b0H2w0cAMDqzZ2uYiQbDjY3XYZVhO5RdraKjyEqStQlz9n+POfAfYb0/YyTWJ+dincKJXZYj8Ag+ZfnpJOgTZF8IQJiUwqiMWNERiAIO2Qz4W/Ji3NH1iOgosqWAhJF1ezGybi9ug3/CelPOeKw3xaDY2YyaEJywHp00WnSEoAiLOYXUGD2Sok9+HViiwfbnimGoyZotOkbYMDotuODQv/HQ9i+xYu9mfGlR4QHjcJwXNwIG9cBeyL6n+lsKjzzyCBQKRbc/w4cPD1K6nguLPQUAGJkRi7UHg3PRcaJguK7mavxg3AqVrVF0lLCT03IE17ccwfUA3CotdmSPQ3F8Coq9Zhy0VPfolODBNiZpTL+3MWrUKHz77beBr9Xqwf+IDptSOCMnjqVAIaXGocOLKYtxr+0h0VHCmsbrwuSKLZhcAdwHoMWUivVZo1Fs0GHjIE1YK6AISimo1WqkpfXvegz9FRbDRwAwtSBJdASi4zxflY/yrCtFx4goSZZG/GT/d3hy+1dYU7oHHzpjsDhmNM6ILYBaMTC/Bw+NG4pYXf/nNg8dOoSMjAzk5+dj/vz5qKqqCkK63gmLJakA4PH6MP6P38DqDO0VChR50nQurIv5HdSWWtFRIp5VH4NN2eOOTlg3odYenKG9nxb+FI9MeaRf21ixYgWsViuKiopQX1+PRx99FLW1tSgpKYFpEC9NGjalAAC3vLUF35c2iY5BdJw7sqrwQMsDUAgY66aTq0geiuK0AqxXA1usFejy9O3MqI9NfQxXFFwR1GwdHR3Izc3Fc889h1tvvTWo2z6VsJlTAPxDSCwFCkWv1eTgqsJrMbz6H6Kj0I/kNZchr7kM8+GfsN6eMw7FcSko9phx0NrzoZvJaZODni0uLg7Dhg3D4cOHg77tUwmbOQUAmMZ5BQph8ypmwx3b/7No0sDQeF0468gW/HLHl/hkzzp83+rCY/pCzIofjXjtyecLcmNykR4dvOsy/4fVakVZWRnS04O/7VMJqz2FojQTkk06NFsG52IURL3R7lbjEeUiPKb4DRRSEK4cQwMqubMBV3Q24AoAPoUS+zJGoTgpG+sVDuzurIBH8s9fnp1+dlBe7/7778ecOXOQm5uLuro6/OEPf4BKpcK8efOCsv2eCqtSAICpQxPx2c460TGITmhpfQauHTYf46reFR2FekEp+TC6dg9G1+7BnQAs+lhszh6LdaYYXJA+JSivUVNTg3nz5qG1tRXJycmYNm0aNm7ciOTk5KBsv6fCaqIZAD7aWo1ff7xbdAyikzKqvdie/Cfo2nkSR9lTaoAlRwBd+JypOazmFAAer0Chz+ZR4QHfLyApw25HPfLkTgmrQgDCsBQy4gzITzKKjkF0Sp82pmBz5kLRMai/hl0iOkHQhV0pAMC0Qu4tUOhbWH4euhJHiY5B/VHIUpCFi0eKPXcIUU90eVW4z3UXJBXP8CtLCUOBJPlfae2/hWUpTBmaiBSTTnQMotNa2ZyItRmDd7QqBVEYDh0BYVoKSqUCc8ZliI5B1CO3lU2DLXm86BjUWyODe1qLUBGWpQAAc8dnio5A1CNunwK/sN8OKYgXi1lb6cGcD+zIeNYCxaOd+KzU3e3+T/e7cfG7NiQ+5b9/Z4P3tNt8fZsL575pQ/yTnYh/shMz37Fhc2335z2z3omUpy1IedqCZ9d3P4h0U40HE1+zwuMLg1XwiQVATnAOWgs1YVsKY7JikZ/MVUgkDz+0xuPrtDuCtj2bS8K4VCVeukx/0vun5ajx5MyeD7OuqfRg3mgNVi8wYsOtRmTHKnHxuzbUdvqPzt7d6MXDq5348GoDPvipAQ+tdmJPo780PD4Jd33pwCuzDVArFf1/g6KNG9yjjAdTWC+Unjs+E899wwOESB7uKpuMXTlnIaZxU7+3NatQg1mFmqNfHX/mzxvH+Se3Kzp6frqNpVdFdfv6b3P0+GSfG98d8eCmcVqUtvgwNlWFC4b4P1bGpipR2uLDmFQVni52YXqOGpMyVX17Q6FEoQzrUgjbPQUAuGI85xVIPiRJgdvMN0PSymMP1+4G3D4gweD/zX9MihIHW72oMvtQ2eHDwVYfRqcoUdbmw5s73XjsgjBZ/DHkPCA2fIenw7oUchONmJATJzoGUY9t7ojB8uSfi47RI0u+dSDDpMDMfP+ewYhkFf58oR4XvWvHxe/Z8cSFeoxIVuHOL7rw1EU6rCrzYPRfrZjwqhVrK2V8Mazx80UnGFBhPXwE+IeQdlR1iI5B1GP3lp2B6XnTkNCwTnSUk/rLOic+LHFjzUIj9OpjcwR3nanFXWceO+7i7Z0umHQKnJOlQtH/WbHldiNqOiVc93EXjtwTDZ1aZvMLulhgxOWiUwyosN5TAIDLx6aHx8QWRZSFbTdB0sWIjnFCz6x34i/rnPj6RiPGpp58jqDF7sOjPzjx4iw9NtV6MSxRicJEFWYMUcPtAw62yvD04aPmAprgrRILRWFfConROp72gmRnd2c0PkhcJDrGcZ4qduJPa51YeUMUzsw49aTxfaucuO9sHbJilPD6/PMP/+HxSfDKcWXqhBtEJxhwYV8KAHDVGVmiIxD12oPlY9CUcWGfnmt1SdjZ4A0cf3Ck3YedDf5JYABo6/Lfv6/Zf/+BFv/9DdZjn9w3LevCA986Al8/uc6J36924o2fGJAXp0SD1YcGqw9W1/Gf7t+UeXCw1YtFk/0roCZlqlDa4sOKQ268ts0FlUKBokSZffwkFgLZwb/sZqgJ+zkFALhsdBqeiNWj3uw4/YOJQsgNjddjpWEHlF1tvXre1jovZrxtD3z9y6+dAJxYME6Dt+Ya8K8Dbty8/Nj/h+s+8S9b/cN5Wjxyvv/YhiqzD0rFsQ/ul7e64PICV3/UfYnrj58DAF1uCXevcOAfVxugVPiHbrNilHhxlh43L3dApwbenquHQSOzYd3x4bsM9cfC7iI7J/Pa2jL8+atS0TGIeu2hvAO4reFR0TEim0IJ3LcXiAn/Ze4y23/ru3mTc2DSRcSOEYWZxyqKUJs5S3SMyDbs0ogoBCCCSsGk1+C6ydmiYxD1yby6a+A1poiOEbnOvV90gkETMaUAALdMGwKNSmbjmEQAqrr0+Gv0YtExItOQ84CsiaJTDJqIKoX0WANmj0kXHYOoT56tHIojWXNFx4g80yNnLwGIsFIAgNvOzRcdgajP5lXPhccUvufdCTlZk4Eh00WnGFQRVwqjM2MxZWii6BhEfdLg1OJZ/d2QwGHQQXHur0QnGHQRVwoAcPt07i2QfL1cnYuD2deIjhH+UscARZeKTjHoIrIUZhSlYFhqtOgYRH12feVsuGNyRccIb+f+UnQCISKyFADgjulDRUcg6rNWlwZ/Ui+GpIjY/8IDK7EQGDlXdAohIvYn6soJmRjKy3WSjL1Tl4GS7OtFxwhP0+4FlJH58RiZ7xqASqnAry8pEh2DqF+uL78YrrgC0THCS2w2MPZnolMIE7GlAACXjk7nldlI1iweNR6QFkFShMG1j0PF1HsAleb0jwtTEV0KAPDbS4eLjkDUL580pmJr1gLRMcJDYgFwRmR/LyO+FM7KT8QFw3lOGZK3heUXwJE4UnQM+bv0SUCtPf3jwljElwIAPDBrOC/ZSbJm8ypxn+suSKrI/kDrl6LLgMKZolMIx1IAUJhqwvyzckTHIOqXFc1JWJdxi+gY8qTWA5c+ITpFSGApHHXfRcMQFxW5k0sUHm4pOxe25PGiY8jPlP8HxOeJThESWApHxUVpcd/MYaJjEPWL26fAItvtkNT60z+Y/GJzIvbo5RNhKfzI/LNyUJjC01+QvK1pi8e3abeLjiEflzwOaAyiU4QMlsKPqFVKPPqTUaJjEPXbXWVnoTNlkugYoS9/BjDyJ6JThBSWwn+ZUpCEeZM56Uzy5pWUuL3zVkgansrlpJQaYNZTolOEHJbCCTw0ewSy4rk7SfK2qSMGn6feKTpG6Dr7LiCZ84j/jaVwAkadGk/9dCwUPHSBZO6esoloT5sqOkboiU4DzlsiOkVIYimcxJSCJNxwFs9XT/ImSQrc3LYAki5GdJTQMvsZQGcSnSIksRRO4YHLhiM7gcNIJG87O6Pxz8RfiI4ROsbPB0bMEZ0iZLEUTiFKq8bTV4/jMBLJ3pLysWjOuEB0DPHicoBZT4pOEdJYCqdxdn4iFpyTJzoGUb/d2HQ9fPp40THEUSiBK1/lsNFpsBR6YMmlw5GbGCU6BlG/lFqj8Hbc3aJjiDNlMZA7RXSKkMdS6AGDVoVnrhkHnkiV5O7RihGoy7xUdIzBlzYWmPGQ6BSywFLooUl5Cbh12hDRMYj67Yb6a+GLShYdY/Boo4Fr3or46yT0FEuhF5ZcOhyThySIjkHUL+V2Pf5qWiw6xuC5/HkgcajoFLLBUugFtUqJv84/A+mxPAMlydszlQWozIqAc/5MuAEYe43oFLLCUuilpGgdXrlhInRqfutI3q6vuRLe6AzRMQZO8ghg1tP92sTatWsxZ84cZGRkQKFQ4LPPPgtOthDGT7Y+GJcdh8fmjhYdg6hfah06PGMI09VIGiNwzZuAtn+rBm02G8aNG4eXXnopSMFCn0KSJEl0CLl6eHkJ3tlQKToGUb98U7gMhdUfiY4RPAolcN37QNGs4G5WocCyZcswd+7coG431HBPoR9+f/lITM7jxDPJ2/WVl8MTE0ani7/kiaAXQiRhKfSDRqXES5x4JplrdmnwJ/ViSAiDA3Em3+k/JTb1GUuhn5JNOrx8w0RoOfFMMvZ2XSb2Zl8vOkb/DJsFXPoX0Slkj59kQTA+Ow6PXcGJZ5K3+RWXwBUn0/X86eOAq/8OKPmR1l/8DgbJtZOyced5+aJjEPWZ2a3G77AIkkIlOkrvxGQB1/8T0PLSo8HAUgiiB2aNwPyzwmjCjiLORw1p2J51k+gYPaeLAeb/EzClDcjmrVYrdu7ciZ07dwIAjhw5gp07d6KqqmpAXi8UcElqkPl8En75z534bGed6ChEfWJU+bAt9THo20pFRzk1pdq/h1Bw4YC9xJo1azBjxozjbl+wYAHeeuutAXtdkVgKA8Dj9eHnS7fjm32NoqMQ9cnlyS140fYrKHxu0VFO7vLngTNvFp0i7HD4aACoVUr83/UTMK0gSXQUoj75ojkJGzJD+AP33F+xEAYIS2GA6NQqvHbTRJyREyc6ClGf3Fw2HfaksaJjHG/6r4ELHxadImyxFAZQlFaNN2+ejJHpMaKjEPWa06fE4q47IKl0oqMcM+N3wAW8WM5AYikMsFiDBu/eOhn5yVwuR/LzXWsCvs+4XXQMv5mPAuf9RnSKsMdSGASJ0Tosve0sZMYZREch6rU7Dp8NS8qZYkNc8gQw7V6xGSIES2GQpMca8OEdZyMvsX+n8iUabF5Jibsst0LSiPjZVQCXPQOc8wsBrx2ZWAqDKDshCh/dNYVzDCQ7xe2x+DL1zsF9UYUSmPMCMDlEhq8iBI9TEKDT4cZtb23F5oo20VGIekyhkLAj9yXENawfhBdTAVe8BIyfN/CvRd1wT0GAGL0G79w6GTNHpIiOQtRjkqTALe0LIelMA/tCSjVw1WssBEFYCoLoNSq8csNEXHtmlugoRD223RyNjxN/PnAvoNYDV78JjLl64F6DTonDRyHgxe8O4dlvDoqOQdRjW/NfQ1LdmuBuNCYTuG4pkDEhuNulXmEphIhPt9dgySe74fbyn4NC34hoO75U/xpKR3twNpg1GfjZe4ApNTjboz7j8FGIuOqMLLx982SY9GrRUYhOa781Cu/GLwrOxibcACz8koUQIrinEGIONVpw13vbUNZsEx2F6LQ2Dn0LabVf9+3JSjVw8eO8pnKIYSmEIJvTgweX7cFyXpOBQtzQqC58o18Cpb2ld080xAPXvAXknz8QsagfOHwUgow6NV64bgIev3I0dGr+E1HoKrMb8Kppce+elDISuH01CyFEcU8hxO2tM2PR0u2oaLWLjkJ0UmsLPkBOzeenf+Dwy4ErXwV00QMfivqEpSADFocbSz7Zja/2NIiOQnRCmXon1kY/CJW1/iSPUPivgzDjQUChGNRs1DssBRl5s/gInviqFC6vT3QUouPcnV2B+5sfPP6OmExg7stA/nmDH4p6jaUgMzurO7Bo6XbUdnSJjkJ0nG8LP0FB9SfHbhh9NTD7WcAQJywT9Q5LQYbMdjd+/fEufL2vUXQUom5SdG6sj30IalcnMPs5nq5ChlgKMvbF7jo8+vk+NFucoqMQBTwy0YWFF08CYnleLzliKcicucuNv6zYjw+3VIP/kiSSSafGg7NHYN7kHNFRqB9YCmFiU3krHli2B+U8EpoEmFGUjD9fNQbpsbzkrNyxFMKI0+PFS6vL8MqaMq5QokERa9Dg4ctH4qcTOVQULlgKYehQowUPfLoHWyuDdAZLov+iViow/6wc3DtzGOKNWtFxKIhYCmFKkiS8t6kKT60ohcXpER2HwsgFw1Pw4GUjUJDCo5LDEUshzDV2OvD8twfx0dYaeHz8p6a+G55mwkOzR2JaYZLoKDSAWAoRoqLFhue+OYjPd9dxlRL1SlK0DvdfPAzXnpkNpZKnqAh3LIUIs7++E8+sOoDvSptER6EQp1Mrcdu5Q/CL8wtg1PHiT5GCpRChtlW24+lVpdhY3iY6CoUYhQKYMzYDS2YNR2Ycl5hGGpZChPv3oWY8s+oAdtWYRUchwbQqJa4Yn4E7puejMNUkOg4JwlIgAMCqvQ147uuDONBoER2FBplJr8b1Z+XglqlDkBqjFx2HBGMpUIAkSVh9oAlvrKvAusO9vLwiyU5ajB63TMvDvMk5MOk1ouNQiGAp0AkdaLDgjXVH8NnOWjg9PDo6nBSlmnD79HxcMT4DGhUv90rdsRTolFqtTnywuQofbK7mNRxk7uz8BNw5fShmDE8RHYVCGEuBesTnk7DmYBOWbqzC6gNN4HFw8pAeq8dPxmfgqglZKErj5DGdHkuBeq22owv/2FyFT7bXcu8hBEXr1Jg1Og1XTsjE2fmJPOCMeoWlQP2yu6YDX+1pwMqSelS02kXHiVhqpQLThyVj7oRMXDwyFXqNSnQkkimWAgXN3jozVpY04Ks99SjjdR0GxbisWMydkIk54zKQFK0THYfCAEuBBsTBRgu+2lOPlSUNKG3gsQ/BolUpcWZePM4vSsaFI1IxNJlnKqXgYinQgCtvtmLl3gZsKGvF9sp22Fxe0ZFkJTPOgPOKknH+sGRMLUjieYhoQLEUaFB5vD7sq+/E5iNt2FLRhq0V7Wi1uUTHCinJJh3OyU/EOUMTMWVoInITjaIjUQRhKZBwZc1WbDnShs0V/qKoboucFU0GjQrD0kwYmW7CyIxYnJOfgIIULh0lcVgKFHIazA5sq2zHwUYLyltsKG+24kiLDXaZDzulx+oxIj0GI9JNR/+OwZBEI5eMUkhhKZBsNJgdKG+2ouxoUZQ321DeYkVte1fIHEwXH6VBikmPlBgd0mP1GJ4Wg+HpJoxMj0FcFK9lTKGPpUCy5/R4UdlqR1OnE602J9psrm5/zF1u2JweWJweWB0e2Jwe2N3ewBXolApApVRApVRArVQe/Vtx7G+V/3a9RoUUkw6pMbrAB/+xv3VINumgU/P4AJI3lgJFJJ9PgleSoFYqoFBw+IboP1gKREQUwPPmEhFRAEuBiIgCWApERBTAUiAiogCWAhERBbAUiIgogKVAREQBLAUiIgpgKRARUQBLgYiIAlgKREQUwFIgIqIAlgIREQWwFIiIKIClQEREASwFIiIKYCkQEVEAS4GIiAJYCkREFMBSICKiAJYCEREFsBSIiCiApUBERAEsBSIiCmApEBFRAEuBiIgCWApERBTAUiAiogCWAhERBbAUiIgogKVAREQBLAUiIgpgKRARUQBLgYiIAlgKREQUwFIgIqIAlgIREQWwFIiIKIClQEREASwFIiIKYCkQEVEAS4GIiAJYCkREFMBSICKiAJYCEREF/H+fI0703o8gZwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Para hacer todo más sencillo y evitar asociaciones indeseadas, vamos a trabajar solo con minúsculas. Además, quitamos los signos de puntuación de los mensajes"
      ],
      "metadata": {
        "id": "fTYQhBW_keNX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Convertimos a minúsculas\n",
        "data['mensaje'] = data['mensaje'].map(lambda x: x.lower())\n",
        "\n",
        "# Función para eliminar signos de puntuación\n",
        "def eliminar_signos(texto):\n",
        "    return re.sub(f\"[{re.escape(string.punctuation)}]\", \"\", texto)\n",
        "\n",
        "# Aplicamos la función al dataset\n",
        "data['mensaje'] = data['mensaje'].map(eliminar_signos)\n",
        "data.head()"
      ],
      "metadata": {
        "id": "q-uS0vz0pxW2",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "outputId": "e36aed56-3c96-42d1-dc1c-2fcdf688dfbd"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                             mensaje  intensidad\n",
              "0  cristina cifuentes podrí­a haber sido la presi...           3\n",
              "1  habrí­a que reabrir el caso el supremo se dedi...           3\n",
              "2  me parece un poco exagerado pedir más de tres ...           3\n",
              "3  parece que todos los delincuentes niegan las e...           3\n",
              "4  preguntárselo al fracasado media carrera en 6 ...           4"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-75b0a443-a3af-4a62-8127-631521a05a64\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>mensaje</th>\n",
              "      <th>intensidad</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>cristina cifuentes podrí­a haber sido la presi...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>habrí­a que reabrir el caso el supremo se dedi...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>me parece un poco exagerado pedir más de tres ...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>parece que todos los delincuentes niegan las e...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>preguntárselo al fracasado media carrera en 6 ...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-75b0a443-a3af-4a62-8127-631521a05a64')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-75b0a443-a3af-4a62-8127-631521a05a64 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-75b0a443-a3af-4a62-8127-631521a05a64');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-567173bd-427f-4245-adc0-f379cd085f8a\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-567173bd-427f-4245-adc0-f379cd085f8a')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-567173bd-427f-4245-adc0-f379cd085f8a button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "data",
              "summary": "{\n  \"name\": \"data\",\n  \"rows\": 12279,\n  \"fields\": [\n    {\n      \"column\": \"mensaje\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 11006,\n        \"samples\": [\n          \"entre gilipollas anda el juego todo hay que torcerlo todo hay que manipularlo\\r\\nentonces \\u00bf\\u00bf\\u00bf un payaso cuando se disfraza tambi\\u00e9n va en contra de los muchos payasos que campan por la pol\\u00ed\\u00adtica y otras ocupaciones \\r\\n\\u00e2\\u00a1\\u00e2\\u00a1 ocup\\u00e1os de lo importante los que viv\\u00ed\\u00ads de lo p\\u00fablico \",\n          \"la doble vara de medir de lo pa\\u00ed\\u00ads el mentiroso hablando de mentiras\",\n          \"hombre es que dnd va pararque te llamen puta o zorra es lo mas\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"intensidad\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 1,\n        \"max\": 5,\n        \"num_unique_values\": 5,\n        \"samples\": [\n          4,\n          5,\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 54
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Eliminamos las \"stopwords\" o palabras vacías. Las stopwords son palabras comunes que, en general, no contienen información relevante para el análisis de texto y suelen ser filtradas antes de procesar los datos. Ejemplos típicos de stopwords en español son: \"el\", \"la\", \"los\", \"las\", \"un\", \"una\", \"de\", \"y\", \"o\", etc."
      ],
      "metadata": {
        "id": "MjXltaASlHiK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Obtenemos las stopwords en español\n",
        "stop_words = set(stopwords.words('spanish'))\n",
        "lemmatizer = WordNetLemmatizer()\n",
        "\n",
        "# Definimos una función para eliminar stopwords y lematizar\n",
        "def filtrar_palabras(texto):\n",
        "    palabras_filtradas = []\n",
        "    for palabra in texto.split():\n",
        "        if palabra not in stop_words:\n",
        "            palabra_lema = lemmatizer.lemmatize(palabra)\n",
        "            palabras_filtradas.append(palabra_lema)\n",
        "    return ' '.join(palabras_filtradas)\n",
        "\n",
        "# Aplicamos la función\n",
        "data['mensaje'] = data['mensaje'].apply(filtrar_palabras)\n",
        "data.head()"
      ],
      "metadata": {
        "id": "O4eVyt4eq2dA",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 206
        },
        "outputId": "0bdde8f7-b6e1-4d2f-e82e-a0e24450c1cf"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                             mensaje  intensidad\n",
              "0  cristina cifuentes podrí­a haber sido presiden...           3\n",
              "1  habrí­a reabrir caso supremo dedica proteger s...           3\n",
              "2  parece exagerado pedir tres años prisión cifue...           3\n",
              "3  parece delincuentes niegan evidencias pruebas ...           3\n",
              "4  preguntárselo fracasado medium carrera 6 meses...           4"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-fad08722-6e21-4f82-b2e5-e8bc76ba3475\" class=\"colab-df-container\">\n",
              "    <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>mensaje</th>\n",
              "      <th>intensidad</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>cristina cifuentes podrí­a haber sido presiden...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>habrí­a reabrir caso supremo dedica proteger s...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>parece exagerado pedir tres años prisión cifue...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>parece delincuentes niegan evidencias pruebas ...</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>preguntárselo fracasado medium carrera 6 meses...</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "    <div class=\"colab-df-buttons\">\n",
              "\n",
              "  <div class=\"colab-df-container\">\n",
              "    <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-fad08722-6e21-4f82-b2e5-e8bc76ba3475')\"\n",
              "            title=\"Convert this dataframe to an interactive table.\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\" viewBox=\"0 -960 960 960\">\n",
              "    <path d=\"M120-120v-720h720v720H120Zm60-500h600v-160H180v160Zm220 220h160v-160H400v160Zm0 220h160v-160H400v160ZM180-400h160v-160H180v160Zm440 0h160v-160H620v160ZM180-180h160v-160H180v160Zm440 0h160v-160H620v160Z\"/>\n",
              "  </svg>\n",
              "    </button>\n",
              "\n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    .colab-df-buttons div {\n",
              "      margin-bottom: 4px;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "    <script>\n",
              "      const buttonEl =\n",
              "        document.querySelector('#df-fad08722-6e21-4f82-b2e5-e8bc76ba3475 button.colab-df-convert');\n",
              "      buttonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "      async function convertToInteractive(key) {\n",
              "        const element = document.querySelector('#df-fad08722-6e21-4f82-b2e5-e8bc76ba3475');\n",
              "        const dataTable =\n",
              "          await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                    [key], {});\n",
              "        if (!dataTable) return;\n",
              "\n",
              "        const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "          '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "          + ' to learn more about interactive tables.';\n",
              "        element.innerHTML = '';\n",
              "        dataTable['output_type'] = 'display_data';\n",
              "        await google.colab.output.renderOutput(dataTable, element);\n",
              "        const docLink = document.createElement('div');\n",
              "        docLink.innerHTML = docLinkHtml;\n",
              "        element.appendChild(docLink);\n",
              "      }\n",
              "    </script>\n",
              "  </div>\n",
              "\n",
              "\n",
              "<div id=\"df-a984b7cb-fbea-4d9a-9720-44def43a74a7\">\n",
              "  <button class=\"colab-df-quickchart\" onclick=\"quickchart('df-a984b7cb-fbea-4d9a-9720-44def43a74a7')\"\n",
              "            title=\"Suggest charts\"\n",
              "            style=\"display:none;\">\n",
              "\n",
              "<svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "     width=\"24px\">\n",
              "    <g>\n",
              "        <path d=\"M19 3H5c-1.1 0-2 .9-2 2v14c0 1.1.9 2 2 2h14c1.1 0 2-.9 2-2V5c0-1.1-.9-2-2-2zM9 17H7v-7h2v7zm4 0h-2V7h2v10zm4 0h-2v-4h2v4z\"/>\n",
              "    </g>\n",
              "</svg>\n",
              "  </button>\n",
              "\n",
              "<style>\n",
              "  .colab-df-quickchart {\n",
              "      --bg-color: #E8F0FE;\n",
              "      --fill-color: #1967D2;\n",
              "      --hover-bg-color: #E2EBFA;\n",
              "      --hover-fill-color: #174EA6;\n",
              "      --disabled-fill-color: #AAA;\n",
              "      --disabled-bg-color: #DDD;\n",
              "  }\n",
              "\n",
              "  [theme=dark] .colab-df-quickchart {\n",
              "      --bg-color: #3B4455;\n",
              "      --fill-color: #D2E3FC;\n",
              "      --hover-bg-color: #434B5C;\n",
              "      --hover-fill-color: #FFFFFF;\n",
              "      --disabled-bg-color: #3B4455;\n",
              "      --disabled-fill-color: #666;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart {\n",
              "    background-color: var(--bg-color);\n",
              "    border: none;\n",
              "    border-radius: 50%;\n",
              "    cursor: pointer;\n",
              "    display: none;\n",
              "    fill: var(--fill-color);\n",
              "    height: 32px;\n",
              "    padding: 0;\n",
              "    width: 32px;\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart:hover {\n",
              "    background-color: var(--hover-bg-color);\n",
              "    box-shadow: 0 1px 2px rgba(60, 64, 67, 0.3), 0 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "    fill: var(--button-hover-fill-color);\n",
              "  }\n",
              "\n",
              "  .colab-df-quickchart-complete:disabled,\n",
              "  .colab-df-quickchart-complete:disabled:hover {\n",
              "    background-color: var(--disabled-bg-color);\n",
              "    fill: var(--disabled-fill-color);\n",
              "    box-shadow: none;\n",
              "  }\n",
              "\n",
              "  .colab-df-spinner {\n",
              "    border: 2px solid var(--fill-color);\n",
              "    border-color: transparent;\n",
              "    border-bottom-color: var(--fill-color);\n",
              "    animation:\n",
              "      spin 1s steps(1) infinite;\n",
              "  }\n",
              "\n",
              "  @keyframes spin {\n",
              "    0% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "      border-left-color: var(--fill-color);\n",
              "    }\n",
              "    20% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    30% {\n",
              "      border-color: transparent;\n",
              "      border-left-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    40% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-top-color: var(--fill-color);\n",
              "    }\n",
              "    60% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "    }\n",
              "    80% {\n",
              "      border-color: transparent;\n",
              "      border-right-color: var(--fill-color);\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "    90% {\n",
              "      border-color: transparent;\n",
              "      border-bottom-color: var(--fill-color);\n",
              "    }\n",
              "  }\n",
              "</style>\n",
              "\n",
              "  <script>\n",
              "    async function quickchart(key) {\n",
              "      const quickchartButtonEl =\n",
              "        document.querySelector('#' + key + ' button');\n",
              "      quickchartButtonEl.disabled = true;  // To prevent multiple clicks.\n",
              "      quickchartButtonEl.classList.add('colab-df-spinner');\n",
              "      try {\n",
              "        const charts = await google.colab.kernel.invokeFunction(\n",
              "            'suggestCharts', [key], {});\n",
              "      } catch (error) {\n",
              "        console.error('Error during call to suggestCharts:', error);\n",
              "      }\n",
              "      quickchartButtonEl.classList.remove('colab-df-spinner');\n",
              "      quickchartButtonEl.classList.add('colab-df-quickchart-complete');\n",
              "    }\n",
              "    (() => {\n",
              "      let quickchartButtonEl =\n",
              "        document.querySelector('#df-a984b7cb-fbea-4d9a-9720-44def43a74a7 button');\n",
              "      quickchartButtonEl.style.display =\n",
              "        google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "    })();\n",
              "  </script>\n",
              "</div>\n",
              "\n",
              "    </div>\n",
              "  </div>\n"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "dataframe",
              "variable_name": "data",
              "summary": "{\n  \"name\": \"data\",\n  \"rows\": 12279,\n  \"fields\": [\n    {\n      \"column\": \"mensaje\",\n      \"properties\": {\n        \"dtype\": \"string\",\n        \"num_unique_values\": 10750,\n        \"samples\": [\n          \"vaya pedazo sinverg\\u00fcenza mientras sanitarios a\\u00fan vacunar rey primero asco\",\n          \"sparta\\u00e2 aitor tilla jajajajjajaja encontraras comentario mio dicho absolutamente retrasado gallego raza superior si asi ponlo retrasado\",\n          \"menudo horterada propia 34furgolistas34 narcotraficantes serie b\"\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    },\n    {\n      \"column\": \"intensidad\",\n      \"properties\": {\n        \"dtype\": \"number\",\n        \"std\": 0,\n        \"min\": 1,\n        \"max\": 5,\n        \"num_unique_values\": 5,\n        \"samples\": [\n          4,\n          5,\n          2\n        ],\n        \"semantic_type\": \"\",\n        \"description\": \"\"\n      }\n    }\n  ]\n}"
            }
          },
          "metadata": {},
          "execution_count": 55
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Creamos los conjuntos de entrenamiento y de test:"
      ],
      "metadata": {
        "id": "ucxI_eXQlZ_1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "X = data['mensaje']\n",
        "y = data['intensidad']\n",
        "\n",
        "# Dividimos el dataset en conjuntos de entrenamiento y prueba\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)\n",
        "\n",
        "# Convertimos a numpy arrays\n",
        "X_train = X_train.values\n",
        "X_test = X_test.values\n",
        "y_train = y_train.values\n",
        "y_test = y_test.values\n",
        "\n",
        "# ajuste de las etiquetas para el entrenamiento\n",
        "y_train = np.array([(value-1) for value in y_train])\n",
        "y_test = np.array([(value-1) for value in y_test])\n",
        "\n",
        "# Creamos datasets de TensorFlow\n",
        "train_dataset = tf.data.Dataset.from_tensor_slices((X_train, y_train))\n",
        "test_dataset = tf.data.Dataset.from_tensor_slices((X_test, y_test))"
      ],
      "metadata": {
        "id": "FZxfXScRYVts"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "np.unique(y_train), np.unique(y_test)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "MoO0l_MV6NoS",
        "outputId": "e174cb5c-520f-485f-a22f-669b69b0af76"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([0, 1, 2, 3, 4]), array([0, 1, 2, 3, 4]))"
            ]
          },
          "metadata": {},
          "execution_count": 57
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Mostramos un mensaje y su intensidad, simplemente para ver que todo va bien:"
      ],
      "metadata": {
        "id": "NTEHtlXpnCOI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for men, inte in train_dataset.take(1):\n",
        "  print('mensaje: ', men.numpy())\n",
        "  print('nivel de odio: ', inte.numpy())"
      ],
      "metadata": {
        "id": "WWz4VaBCYsjL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5dcd0e63-385e-4a9a-a3e7-f5215d70bd57"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "mensaje:  b'tarado abascal dice gran lider tarado trump'\n",
            "nivel de odio:  3\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Mezclamos los datos de forma aleatoria:"
      ],
      "metadata": {
        "id": "iGisxv3GnZvN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "BUFFER_SIZE = 10000\n",
        "BATCH_SIZE = 64"
      ],
      "metadata": {
        "id": "ecZdI0CFYz1Y"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "train_dataset = train_dataset.shuffle(BUFFER_SIZE).batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)\n",
        "test_dataset = test_dataset.batch(BATCH_SIZE).prefetch(tf.data.AUTOTUNE)"
      ],
      "metadata": {
        "id": "wOJwWckbY9ns"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "for men, inte in train_dataset.take(1):\n",
        "  print('mensaje: ', men.numpy()[:3])\n",
        "  print()\n",
        "  print('nivel de odio: ', inte.numpy()[:3])"
      ],
      "metadata": {
        "id": "mGW0FVeyY_fZ",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5ab8f310-49e0-4383-f6e0-5c989e2e5f75"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "mensaje:  [b'gentuza debe ser expulsada dejado entrar encarcelados cargando espa\\xc3\\xb1a'\n",
            " b'pserranooceja tipo publicidad sitio p\\xc3\\xbablico parece repugnante'\n",
            " b'menos patri\\xc3\\xb3tico robar multar empresas aut\\xc3\\xb3nomos ciudadanos mantener gobierno clase pol\\xc3\\xad\\xc2\\xadtica hdlgp']\n",
            "\n",
            "nivel de odio:  [3 0 3]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Definimos y configuramos un codificador de texto utilizando la capa TextVectorization"
      ],
      "metadata": {
        "id": "dwTWx49snn-L"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Empezamos definiendo el tamaño del vocabulario\n",
        "tamaño_vocabulario = 9000\n",
        "\n",
        "# Creamos una capa de vectorización de texto\n",
        "vectorizador = tf.keras.layers.TextVectorization(\n",
        "    max_tokens=tamaño_vocabulario\n",
        ")\n",
        "\n",
        "# Se extraen solo los textos del conjunto de datos de entrenamiento\n",
        "def extraer_texto(texto, etiqueta):\n",
        "    return texto\n",
        "\n",
        "# Adaptamos el vectorizador al texto del conjunto de datos de entrenamiento\n",
        "dataset_texto = train_dataset.map(extraer_texto)\n",
        "vectorizador.adapt(dataset_texto)"
      ],
      "metadata": {
        "id": "V8W8LqSRZCnl"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Obtenemos el vocabulario aprendido por la capa de vectorización\n",
        "vocabulario = np.array(vectorizador.get_vocabulary())\n",
        "\n",
        "# Mostramos las primeras 20 palabras del vocabulario\n",
        "primeras_20_palabras = vocabulario[:20]\n",
        "primeras_20_palabras"
      ],
      "metadata": {
        "id": "ps6hMfc3ZII1",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3093dbaa-0f3b-4acc-d075-1c639957373e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array(['', '[UNK]', 'si', 'mierda', 'puta', 'ser', 'asco', 'gobierno',\n",
              "       'panfleto', 'q', 'españa', 'paí\\xads', 'gente', 'vergüenza',\n",
              "       'gentuza', 'vaya', 'basura', 'va', 'gilipollas', 'solo'],\n",
              "      dtype='<U129')"
            ]
          },
          "metadata": {},
          "execution_count": 63
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Ejercicio\n",
        "\n",
        "Construye una RNN que pueda detectar el nivel de odio de un mensaje.\n",
        "\n",
        "**Tenéis plena liberta para realizar la red y para hacer cualquier modificación a los datos.**"
      ],
      "metadata": {
        "id": "n1ztdBqGo6xb"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Construcción de las redes neuronales.\n",
        "\n",
        "En esta actividad vamos a probar tres modelos con diferntes arquitecturas:\n",
        "Un modelo RNN simple con una capa de embeddings, 2 un modelo con embeddings preentrandos y 3 un modelo profundo.\n",
        "\n"
      ],
      "metadata": {
        "id": "bBTGwwgY1HYw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Modelo Base\n",
        "\n",
        "* tamaño_vocabulario = 9000\n",
        "* dim_embedding = 128\n",
        "* unidades_lstm = 64\n",
        "* dropout_rate = 0.5\n",
        "* tasa_aprendizaje = 0.001\n"
      ],
      "metadata": {
        "id": "Ntq1nVYs30Im"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "modelo_base = tf.keras.Sequential([\n",
        "    vectorizador,\n",
        "    tf.keras.layers.Embedding(input_dim=tamaño_vocabulario, output_dim=128, mask_zero=True),\n",
        "    tf.keras.layers.LSTM(64),\n",
        "    tf.keras.layers.Dropout(0.001),\n",
        "    tf.keras.layers.Dense(64, activation='relu'),\n",
        "    tf.keras.layers.Dense(5, activation='softmax')  # 5 categorías\n",
        "])"
      ],
      "metadata": {
        "id": "SQ3Z0yqX3w4t"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Compilar el modelo\n",
        "modelo_base.compile(\n",
        "    loss='sparse_categorical_crossentropy',\n",
        "    optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),\n",
        "    metrics=['accuracy']\n",
        ")"
      ],
      "metadata": {
        "id": "dpBgPoDv3fgm"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "# Entrenar el modelo\n",
        "historial_base = modelo_base.fit(\n",
        "    train_dataset,\n",
        "    epochs=10  # Ajusta según el desempeño\n",
        ")"
      ],
      "metadata": {
        "id": "dRVvSwS51Dnu",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "5c05f10d-b977-479a-ce61-b0032812c3c0"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m24s\u001b[0m 137ms/step - accuracy: 0.8085 - loss: 1.0166\n",
            "Epoch 2/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m37s\u001b[0m 114ms/step - accuracy: 0.8756 - loss: 0.4031\n",
            "Epoch 3/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 111ms/step - accuracy: 0.9167 - loss: 0.2710\n",
            "Epoch 4/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m19s\u001b[0m 126ms/step - accuracy: 0.9426 - loss: 0.1770\n",
            "Epoch 5/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m25s\u001b[0m 157ms/step - accuracy: 0.9542 - loss: 0.1319\n",
            "Epoch 6/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m35s\u001b[0m 120ms/step - accuracy: 0.9659 - loss: 0.0983\n",
            "Epoch 7/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m17s\u001b[0m 108ms/step - accuracy: 0.9739 - loss: 0.0774\n",
            "Epoch 8/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m20s\u001b[0m 105ms/step - accuracy: 0.9747 - loss: 0.0692\n",
            "Epoch 9/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m22s\u001b[0m 113ms/step - accuracy: 0.9796 - loss: 0.0632\n",
            "Epoch 10/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m19s\u001b[0m 104ms/step - accuracy: 0.9869 - loss: 0.0435\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "resultados_base = modelo_base.evaluate(test_dataset)\n",
        "print(f\"Pérdida en prueba: {resultados_base[0]}, Precisión: {resultados_base[1]}\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "hPkCFCU78K54",
        "outputId": "71bc731d-78f8-4900-eeb9-e4113a78b479"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[1m39/39\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m1s\u001b[0m 37ms/step - accuracy: 0.8314 - loss: 0.9686\n",
            "Pérdida en prueba: 0.9139609932899475, Precisión: 0.8367263674736023\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Modelo Con embeddings preentrenados\n",
        "* tamaño_vocabulario = 9000\n",
        "* dim_embedding = 100\n",
        "* unidades_lstm = 128\n",
        "* dropout_rate = 0.5\n",
        "* tasa_aprendizaje = 0.001"
      ],
      "metadata": {
        "id": "Ig1J9d-19JWo"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# download glove and unzip it in Notebook.\n",
        "!wget http://nlp.stanford.edu/data/glove.6B.zip\n",
        "!unzip glove*.zip"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "AMKNsT411DmG",
        "outputId": "3fe8b3c9-c0f1-41d6-e798-dea3a580732e"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "--2024-12-08 20:31:50--  http://nlp.stanford.edu/data/glove.6B.zip\n",
            "Resolving nlp.stanford.edu (nlp.stanford.edu)... 171.64.67.140\n",
            "Connecting to nlp.stanford.edu (nlp.stanford.edu)|171.64.67.140|:80... connected.\n",
            "HTTP request sent, awaiting response... 302 Found\n",
            "Location: https://nlp.stanford.edu/data/glove.6B.zip [following]\n",
            "--2024-12-08 20:31:50--  https://nlp.stanford.edu/data/glove.6B.zip\n",
            "Connecting to nlp.stanford.edu (nlp.stanford.edu)|171.64.67.140|:443... connected.\n",
            "HTTP request sent, awaiting response... 301 Moved Permanently\n",
            "Location: https://downloads.cs.stanford.edu/nlp/data/glove.6B.zip [following]\n",
            "--2024-12-08 20:31:50--  https://downloads.cs.stanford.edu/nlp/data/glove.6B.zip\n",
            "Resolving downloads.cs.stanford.edu (downloads.cs.stanford.edu)... 171.64.64.22\n",
            "Connecting to downloads.cs.stanford.edu (downloads.cs.stanford.edu)|171.64.64.22|:443... connected.\n",
            "HTTP request sent, awaiting response... 200 OK\n",
            "Length: 862182613 (822M) [application/zip]\n",
            "Saving to: ‘glove.6B.zip’\n",
            "\n",
            "glove.6B.zip        100%[===================>] 822.24M  5.02MB/s    in 2m 39s  \n",
            "\n",
            "2024-12-08 20:34:30 (5.16 MB/s) - ‘glove.6B.zip’ saved [862182613/862182613]\n",
            "\n",
            "Archive:  glove.6B.zip\n",
            "  inflating: glove.6B.50d.txt        \n",
            "  inflating: glove.6B.100d.txt       \n",
            "  inflating: glove.6B.200d.txt       \n",
            "  inflating: glove.6B.300d.txt       \n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# uso de emebddings\n",
        "def embedding_for_vocab(filepath, word_index,\n",
        "                        embedding_dim):\n",
        "    vocab_size = len(word_index) + 1\n",
        "\n",
        "    # Adding again 1 because of reserved 0 index\n",
        "    embedding_matrix_vocab = np.zeros((vocab_size,\n",
        "                                       embedding_dim))\n",
        "\n",
        "    with open(filepath, encoding=\"utf8\") as f:\n",
        "        for line in f:\n",
        "            word, *vector = line.split()\n",
        "            if word in word_index:\n",
        "                idx = word_index[word]\n",
        "                embedding_matrix_vocab[idx] = np.array(\n",
        "                    vector, dtype=np.float32)[:embedding_dim]\n",
        "\n",
        "    return embedding_matrix_vocab\n"
      ],
      "metadata": {
        "id": "XENfbxZG1Dkk"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "tokenizer = Tokenizer()\n",
        "tokenizer.fit_on_texts(X)"
      ],
      "metadata": {
        "id": "09Ypeebv-knv"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "embedding_dim = 50\n",
        "embedding_matrix_vocab = embedding_for_vocab(\n",
        "    'glove.6B.50d.txt', tokenizer.word_index,\n",
        "  embedding_dim)\n",
        "\n",
        "print(\"El vector denso para el primer comentario\",f'\\n {X[2]}', \"es =>\",\"\\n\",\n",
        "      embedding_matrix_vocab[3])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "mMKbXRpg-jYz",
        "outputId": "f875f9f6-06f5-44d6-fecd-4836c88bac0c"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "El vector denso para el primer comentario \n",
            " parece exagerado pedir tres años prisión cifuentes caso habrí­a investigar destapó caso igual cremas seguro fuego amigo alguien interesaba destapara casos internos corrupción peperos madrileños amenazó menos mujer conoce llevara dinero público abasteciera partido estorbaba mucha gente amiga grave tí­tulo académico casadete regalado convalidaciones master tres dí­as harvard aravaca comparada actual caótica punto comparación habla sube pan costa decir sandez tras sandez mientras presidenta tí­tulo nobiliario llenó menos partido dinero ilí­cito sigue campando anchas es => \n",
            " [ 0.0076261  -0.30181    -0.54163998 -0.48793    -0.40406001 -1.49580002\n",
            "  1.19659996  0.0064523  -0.43487     0.27230999  0.36748001  0.40961999\n",
            "  0.13790999 -0.84524    -0.37463999 -0.88265997 -0.25148001  1.05180001\n",
            "  0.44995001 -0.18942    -1.27269995  0.13495    -0.14572001 -0.081864\n",
            "  0.41113001  0.97237003 -0.73806     0.22671001 -0.51042998  0.18915001\n",
            " -0.69155002 -0.60968    -0.88050002  0.35352001 -0.31342     0.17443\n",
            " -0.19108    -0.94510001  0.75283998 -0.092328    0.17389999 -0.65232003\n",
            " -0.26159    -0.64372998  0.58126003 -0.35635999  0.76591998 -0.49579999\n",
            "  1.10619998 -0.13865   ]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "tamaño_vocabulario = 22766\n",
        "dim_embedding = 50  # Dimensiones del embedding de GloVe\n",
        "unidades_lstm = 128\n",
        "dropout_rate = 0.3\n",
        "tasa_aprendizaje = 0.001\n",
        "\n",
        "modelo_embeddings = tf.keras.Sequential([\n",
        "    vectorizador,\n",
        "    tf.keras.layers.Embedding(\n",
        "        input_dim=tamaño_vocabulario,\n",
        "        output_dim=dim_embedding,\n",
        "        weights=[embedding_matrix_vocab],  # Inicializar con GloVe\n",
        "        trainable=False,  # Fijar los embeddings preentrenados\n",
        "        mask_zero=True\n",
        "    ),\n",
        "    tf.keras.layers.LSTM(unidades_lstm),\n",
        "    tf.keras.layers.Dropout(dropout_rate),\n",
        "    tf.keras.layers.Dense(64, activation='relu'),\n",
        "    tf.keras.layers.Dense(5, activation='softmax')\n",
        "])\n",
        "\n",
        "# Compilar el modelo\n",
        "modelo_embeddings.compile(\n",
        "    loss='sparse_categorical_crossentropy',\n",
        "    optimizer=tf.keras.optimizers.Adam(learning_rate=tasa_aprendizaje),\n",
        "    metrics=['accuracy']\n",
        ")\n",
        "\n",
        "# Entrenar el modelo\n",
        "historial_experimento2 = modelo_embeddings.fit(\n",
        "    train_dataset,\n",
        "    epochs=10  # Ajusta según el desempeño\n",
        ")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "iae06y8UEy2N",
        "outputId": "9e1c2f13-af1c-4fff-f7c5-59684e739f2c"
      },
      "execution_count": 100,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m36s\u001b[0m 217ms/step - accuracy: 0.7928 - loss: 0.8516\n",
            "Epoch 2/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m30s\u001b[0m 198ms/step - accuracy: 0.8482 - loss: 0.5738\n",
            "Epoch 3/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m39s\u001b[0m 184ms/step - accuracy: 0.8475 - loss: 0.5465\n",
            "Epoch 4/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m29s\u001b[0m 185ms/step - accuracy: 0.8614 - loss: 0.5052\n",
            "Epoch 5/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m40s\u001b[0m 183ms/step - accuracy: 0.8528 - loss: 0.5261\n",
            "Epoch 6/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m41s\u001b[0m 183ms/step - accuracy: 0.8524 - loss: 0.5176\n",
            "Epoch 7/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m44s\u001b[0m 199ms/step - accuracy: 0.8560 - loss: 0.4992\n",
            "Epoch 8/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m28s\u001b[0m 185ms/step - accuracy: 0.8592 - loss: 0.4879\n",
            "Epoch 9/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m28s\u001b[0m 182ms/step - accuracy: 0.8601 - loss: 0.4703\n",
            "Epoch 10/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m28s\u001b[0m 181ms/step - accuracy: 0.8601 - loss: 0.4588\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "resultados_base = modelo_embeddings.evaluate(test_dataset)\n",
        "print(f\"Pérdida en prueba: {resultados_base[0]}, Precisión: {resultados_base[1]}\")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "KgMs6kxtJAqQ",
        "outputId": "06cca74f-bf11-4fb0-ec0a-8ef1d5c2f865"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[1m39/39\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m5s\u001b[0m 111ms/step - accuracy: 0.8569 - loss: 0.5192\n",
            "Pérdida en prueba: 0.4950958788394928, Precisión: 0.8623778223991394\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Experimento 3: Modelo Profundo\n",
        "\n",
        "* Dos capas LSTM (64 y 32 unidades).\n",
        "* Embedding de 256 dimensiones.\n",
        "* Dropout: 0.5.\n",
        "* Tasa de aprendizaje: 0.0005."
      ],
      "metadata": {
        "id": "UVrkfg8lL6Xh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "dim_embedding = 256  # Dimensión del embedding inicializado aleatoriamente\n",
        "unidades_lstm_1 = 64\n",
        "unidades_lstm_2 = 32\n",
        "dropout_rate = 0.5\n",
        "tasa_aprendizaje = 0.0005\n",
        "\n",
        "modelo_profundo = tf.keras.Sequential([\n",
        "    vectorizador,\n",
        "    tf.keras.layers.Embedding(input_dim=tamaño_vocabulario, output_dim=dim_embedding, mask_zero=True),\n",
        "    tf.keras.layers.LSTM(unidades_lstm_1, return_sequences=True),  # Primera capa LSTM\n",
        "    tf.keras.layers.LSTM(unidades_lstm_2),                        # Segunda capa LSTM\n",
        "    tf.keras.layers.Dropout(dropout_rate),\n",
        "    tf.keras.layers.Dense(64, activation='relu'),\n",
        "    tf.keras.layers.Dense(5, activation='softmax')  # 5 categorías\n",
        "])\n",
        "\n",
        "# Compilar el modelo\n",
        "modelo_profundo.compile(\n",
        "    loss='sparse_categorical_crossentropy',\n",
        "    optimizer=tf.keras.optimizers.Adam(learning_rate=tasa_aprendizaje),\n",
        "    metrics=['accuracy']\n",
        ")"
      ],
      "metadata": {
        "id": "2HJi9lFaMEhU"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "modelo_profundo.compile(\n",
        "    loss='sparse_categorical_crossentropy',\n",
        "    optimizer=tf.keras.optimizers.Adam(learning_rate=tasa_aprendizaje),\n",
        "    metrics=['accuracy']\n",
        ")"
      ],
      "metadata": {
        "id": "akION6lmMOc7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "historial_profundo = modelo_profundo.fit(\n",
        "    train_dataset,\n",
        "    epochs=10\n",
        ")\n"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "D3ttiUBaMOU2",
        "outputId": "9ffe4ef8-d1ff-4372-f7b2-28876391dabf"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m47s\u001b[0m 251ms/step - accuracy: 0.7834 - loss: 1.2224\n",
            "Epoch 2/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m43s\u001b[0m 266ms/step - accuracy: 0.8346 - loss: 0.6207\n",
            "Epoch 3/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m39s\u001b[0m 255ms/step - accuracy: 0.8501 - loss: 0.5129\n",
            "Epoch 4/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m40s\u001b[0m 248ms/step - accuracy: 0.8593 - loss: 0.4190\n",
            "Epoch 5/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m44s\u001b[0m 266ms/step - accuracy: 0.8998 - loss: 0.3119\n",
            "Epoch 6/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m83s\u001b[0m 277ms/step - accuracy: 0.9179 - loss: 0.2874\n",
            "Epoch 7/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m80s\u001b[0m 260ms/step - accuracy: 0.9289 - loss: 0.2395\n",
            "Epoch 8/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m42s\u001b[0m 276ms/step - accuracy: 0.9423 - loss: 0.1982\n",
            "Epoch 9/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m81s\u001b[0m 268ms/step - accuracy: 0.9399 - loss: 0.1931\n",
            "Epoch 10/10\n",
            "\u001b[1m154/154\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m78s\u001b[0m 242ms/step - accuracy: 0.9446 - loss: 0.1803\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "resultados_profundo = modelo_profundo.evaluate(test_dataset)\n",
        "\n",
        "print(f\"Pérdida en prueba: {resultados_profundo[0]}, Precisión: {resultados_profundo[1]}\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "moehQcM4MZ_l",
        "outputId": "c1c58b0e-6110-4cc6-c179-b5421e209001"
      },
      "execution_count": null,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[1m39/39\u001b[0m \u001b[32m━━━━━━━━━━━━━━━━━━━━\u001b[0m\u001b[37m\u001b[0m \u001b[1m6s\u001b[0m 129ms/step - accuracy: 0.8482 - loss: 0.7507\n",
            "Pérdida en prueba: 0.7165290117263794, Precisión: 0.854234516620636\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [],
      "metadata": {
        "id": "6mCgtavZ_ern"
      }
    }
  ]
}