arima_temperature_forecasting.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The below notebook shows an ARIMA model tweaked to predict average monthly temperatures. Its purpose is purely educational. Firstly the user has to choose the country they want to run predictions for. After that they get to choose the parameters of the ARIMA model - both the numerical and temporal parameters (beginnings and endings of both train and test periods)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "220d585071724b77b6b4793b9ab66b77",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Dropdown(description='Country:', options=('Select a country', 'Åland', 'Afghanistan', 'Africa', 'Albania', 'Al…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "import ipywidgets as widgets\n",
    "from IPython.display import display\n",
    "\n",
    "# Load the CSV file into a DataFrame\n",
    "df = pd.read_csv('GlobalLandTemperaturesByCountry.csv')\n",
    "df['dt'] = pd.to_datetime(df['dt'])\n",
    "\n",
    "# Set the 'Date' column as the index\n",
    "df.set_index('dt', inplace=True)\n",
    "# Global variable to store the filtered DataFrame\n",
    "global filtered_df\n",
    "filtered_df = pd.DataFrame()\n",
    "\n",
    "# Get unique countries\n",
    "countries = df['Country'].unique()\n",
    "\n",
    "# Create a dropdown widget for countries\n",
    "country_dropdown = widgets.Dropdown(\n",
    "    options=['Select a country'] + list(countries),\n",
    "    description='Country:',\n",
    "    disabled=False,\n",
    ")\n",
    "\n",
    "# Function to update the global DataFrame\n",
    "def update_dataframe(change):\n",
    "    global filtered_df\n",
    "    if change['new'] != 'Select a country':\n",
    "        filtered_df = df[df['Country'] == change['new']]\n",
    "        print(f\"DataFrame updated for {change['new']}\")\n",
    "\n",
    "# Observe changes in the dropdown\n",
    "country_dropdown.observe(update_dataframe, names='value')\n",
    "\n",
    "# Display the widget\n",
    "display(country_dropdown)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above cell downloads data from the CSV file GlobalLandTemperaturesByCountry. The file consists of average monthly temperatures for all the countries spanning from the 18th century up to 2013.\n",
    "There is a dropdown list which allows us to change the country for which we have saved the monthly temperatures. Every time we want to change the country we need to change the country back in \n",
    "the above cell. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>AverageTemperature</th>\n",
       "      <th>AverageTemperatureUncertainty</th>\n",
       "      <th>Country</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>dt</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>1743-11-01</th>\n",
       "      <td>3.937</td>\n",
       "      <td>2.057</td>\n",
       "      <td>Poland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1744-04-01</th>\n",
       "      <td>8.889</td>\n",
       "      <td>3.110</td>\n",
       "      <td>Poland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1744-05-01</th>\n",
       "      <td>11.952</td>\n",
       "      <td>1.839</td>\n",
       "      <td>Poland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1744-06-01</th>\n",
       "      <td>14.867</td>\n",
       "      <td>1.799</td>\n",
       "      <td>Poland</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1744-07-01</th>\n",
       "      <td>17.313</td>\n",
       "      <td>1.803</td>\n",
       "      <td>Poland</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            AverageTemperature  AverageTemperatureUncertainty Country\n",
       "dt                                                                   \n",
       "1743-11-01               3.937                          2.057  Poland\n",
       "1744-04-01               8.889                          3.110  Poland\n",
       "1744-05-01              11.952                          1.839  Poland\n",
       "1744-06-01              14.867                          1.799  Poland\n",
       "1744-07-01              17.313                          1.803  Poland"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Example code to process the filtered DataFrame\n",
    "# Ensure that a country has been selected and the DataFrame is not empty\n",
    "preprocessed_df = filtered_df.dropna()\n",
    "\n",
    "if not preprocessed_df.empty:\n",
    "    # Example processing: display the first few rows\n",
    "    display(preprocessed_df.head())\n",
    "else:\n",
    "    print(\"Please select a country first.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above cell's sole purpose is to ensure that we have correctly selected the country."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "processed_df = preprocessed_df.copy()\n",
    "processed_df['Year'] = processed_df.index.year"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "db2b3ae8eb8d47d9bd4de495300a4907",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "interactive(children=(IntSlider(value=2003, continuous_update=False, description='Start Year:', max=2013, min=…"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "\n",
    "year_range = (processed_df.index.year.min(), processed_df.index.year.max())\n",
    "\n",
    "start_year_slider = widgets.IntSlider(\n",
    "    value=year_range[1]-10,\n",
    "    min=year_range[0],\n",
    "    max=year_range[1],\n",
    "    step=1,\n",
    "    description='Start Year:',\n",
    "    continuous_update=False\n",
    ")\n",
    "\n",
    "end_year_slider = widgets.IntSlider(\n",
    "    value=year_range[1],\n",
    "    min=year_range[0],\n",
    "    max=year_range[1],\n",
    "    step=1,\n",
    "    description='End Year:',\n",
    "    continuous_update=False\n",
    ")\n",
    "def update_plot(start_year, end_year):\n",
    "    # Filter the DataFrame\n",
    "    plotted_df = processed_df[(processed_df.index.year >= start_year) & (processed_df.index.year <= end_year)].copy()\n",
    "    plotted_df['PrevYearTemp'] = plotted_df['AverageTemperature'].shift(12)\n",
    "    plotted_df['TempDifference'] = plotted_df['AverageTemperature'] - plotted_df['PrevYearTemp']\n",
    "    # Plotting\n",
    "    plt.figure(figsize=(10, 5))\n",
    "    plt.plot(plotted_df.index, plotted_df['AverageTemperature'])  # Replace 'AverageTemperature' with your column name\n",
    "    plt.plot(plotted_df.index, plotted_df['PrevYearTemp'], label='Previous Year Temp', linestyle='--')\n",
    "    plt.title(f'Average Temperature from {start_year} to {end_year}')\n",
    "    plt.xlabel('Year')\n",
    "    plt.ylabel('Average Temperature')  # Replace with your column label\n",
    "    plt.grid(True)\n",
    "    plt.show()\n",
    "    # Plotting the Temperature Difference\n",
    "    plt.figure(figsize=(10, 5))\n",
    "    plt.plot(plotted_df.index, plotted_df['TempDifference'], color='red', marker='o', linestyle='-')\n",
    "    plt.title(f'Temperature Difference from {start_year+1} to {end_year}')\n",
    "    plt.xlabel('Year')\n",
    "    plt.ylabel('Temperature Difference')\n",
    "    plt.grid(True)\n",
    "    plt.show()\n",
    "\n",
    "widgets.interactive(update_plot, start_year=start_year_slider, end_year=end_year_slider)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the above cell we find the average monthly temperatures for years between January of the starting year and the last available month of the ending year (August for 2013, December for earlier years)\n",
    "We first plot the averages for month X in year Y and month X in year Y-1. After that we plot the differences between the averages of the same month vs the same month 1 year before. It shows, that\n",
    "these differences can be as large as more than 10 degrees Celsius, especially in winter months, which show the largest standard deviations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "5a6b74ffe42b4a1881bd0a6af37e4ca5",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "VBox(children=(Label(value='Adjust the SARIMA Model Parameters and Date Ranges:', layout=Layout(justify_conten…"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b1c8bff2a3db4481862d84fe184a1973",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "b1c8bff2a3db4481862d84fe184a1973",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Output()"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from statsmodels.tsa.arima.model import ARIMA\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np \n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "caption = widgets.Label(\n",
    "    value='Adjust the SARIMA Model Parameters and Date Ranges:',\n",
    "    layout=widgets.Layout(width='100%', justify_content='center')\n",
    ")\n",
    "\n",
    "train_start_year_widget = widgets.IntSlider(value=1990, min=year_range[0], max=year_range[1], description='Train Start Year', tooltip='Train Start Year')\n",
    "train_end_year_widget = widgets.IntSlider(value=2011, min=year_range[0], max=year_range[1], description='Train End Year', tooltip='Train End Year')\n",
    "test_end_year_widget = widgets.IntSlider(value=2013, min=year_range[0], max=year_range[1], description='Test End Year', tooltip='Test End Year')\n",
    "\n",
    "# Month widgets\n",
    "train_start_month_widget = widgets.IntSlider(value=1, min=1, max=12, step=1, description='Train Start Month', tooltip='Train Start Month')\n",
    "train_end_month_widget = widgets.IntSlider(value=12, min=1, max=12, step=1, description='Train End Month', tooltip='Train End Month')\n",
    "test_end_month_widget = widgets.IntSlider(value=8, min=1, max=12, step=1, description='Test End Month', tooltip='Test End Month')\n",
    "\n",
    "p_widget = widgets.IntSlider(value=1, min=0, max=5, step=1, description='p:', tooltip='Autoregressive order (p)')\n",
    "d_widget = widgets.IntSlider(value=1, min=0, max=2, step=1, description='d:', tooltip='Degree of Differencing (d)')\n",
    "q_widget = widgets.IntSlider(value=1, min=0, max=5, step=1, description='q:', tooltip='Moving Average Order (q)')\n",
    "P_widget = widgets.IntSlider(value=1, min=0, max=5, step=1, description='P:', tooltip='Seasonal AutoRegressive Order (P)')\n",
    "D_widget = widgets.IntSlider(value=1, min=0, max=2, step=1, description='D:', tooltip='Seasonal Differencing Order (D)')\n",
    "Q_widget = widgets.IntSlider(value=1, min=0, max=5, step=1, description='Q:', tooltip='Seasonal Moving Average Order (Q)')\n",
    "S_widget = widgets.IntSlider(value=12, min=1, max=24, step=1, description='S:', tooltip='Seasonal Periodicity (S)')\n",
    "\n",
    "def plot_forecast(p, d, q, P, D, Q, S, train_start_year, train_start_month, train_end_year, train_end_month, test_end_year, test_end_month):\n",
    "    # Setting the testing start year to one year after the training end year\n",
    "    train_start = f\"{train_start_year}-{train_start_month:02d}-01\"\n",
    "    train_end = f\"{train_end_year}-{train_end_month:02d}-01\"\n",
    "    if train_end_month < 12:\n",
    "        test_start = f\"{train_end_year}-{train_end_month + 1:02d}-01\"  # One month after training ends\n",
    "    else:\n",
    "        test_start = f\"{train_end_year + 1}-01-01\"  # One month after training ends\n",
    "    test_end = f\"{test_end_year}-{test_end_month:02d}-01\"\n",
    "\n",
    "    train_series = processed_df[(processed_df.index >= train_start) & (processed_df.index <= train_end)]['AverageTemperature'].asfreq('MS')\n",
    "    test_series = processed_df[(processed_df.index >= test_start) & (processed_df.index <= test_end)]['AverageTemperature'].asfreq('MS')\n",
    "\n",
    "    # Fit the SARIMA model and plot as before\n",
    "    model = ARIMA(train_series, order=(p, d, q), seasonal_order=(P, D, Q, S))\n",
    "    model_fit = model.fit()\n",
    "\n",
    "    forecasts = model_fit.get_forecast(steps=len(test_series))\n",
    "    forecast_series = forecasts.predicted_mean\n",
    "\n",
    "    rmse = np.sqrt(mean_squared_error(test_series, forecast_series))\n",
    "    print(f'RMSE: {rmse}')\n",
    "\n",
    "    plt.figure(figsize=(10, 5))\n",
    "    plt.plot(train_series.index, train_series, label='Training Data')\n",
    "    plt.plot(test_series.index, test_series, label='Actual Test Data', color='blue')\n",
    "    plt.plot(forecast_series.index, forecast_series, label='Forecast', color='red', linestyle='--')\n",
    "    plt.title('SARIMA Forecast')\n",
    "    plt.xlabel('Year')\n",
    "    plt.ylabel('Average Temperature')\n",
    "    plt.legend()\n",
    "    plt.grid(True)\n",
    "    plt.show()\n",
    "\n",
    "# Grouping the existing widgets\n",
    "parameter_widgets = widgets.VBox([p_widget, d_widget, q_widget, P_widget, D_widget, Q_widget, S_widget])\n",
    "date_widgets = widgets.VBox([train_start_year_widget, train_start_month_widget, \n",
    "                             train_end_year_widget, train_end_month_widget, \n",
    "                             test_end_year_widget, test_end_month_widget])\n",
    "\n",
    "# Creating a vertical box layout with the caption at the top\n",
    "vbox_layout = widgets.VBox([caption, parameter_widgets, date_widgets])\n",
    "\n",
    "\n",
    "# Now use this vbox_layout in your interactive plot\n",
    "interactive_plot = widgets.interactive_output(plot_forecast, \n",
    "                                              {'p': p_widget, 'd': d_widget, 'q': q_widget,\n",
    "                                               'P': P_widget, 'D': D_widget, 'Q': Q_widget, 'S': S_widget,\n",
    "                                               'train_start_year': train_start_year_widget,\n",
    "                                               'train_start_month': train_start_month_widget,\n",
    "                                               'train_end_year': train_end_year_widget,\n",
    "                                               'train_end_month': train_end_month_widget,\n",
    "                                               'test_end_year': test_end_year_widget,\n",
    "                                               'test_end_month': test_end_month_widget})\n",
    "\n",
    "display(vbox_layout, interactive_plot)\n",
    "\n",
    "display(interactive_plot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the last cell we perform ARIMA. We provide several customizable parameters for the user - the beginning and ending months for both the training and testing period, as well as parameters for the\n",
    "seasonal ARIMA model. Once again large differences between seasons year to year can be seen, for Poland we see winter months with averages both above 0 and below -10 degrees Celsius. The changes of months do not change much in the appearance of the graph. The most interesting is the S (seasonality) parameter. Based on the training period ending we can possible achieve a prediction of endless summer, endless winter or even an ice age. \n",
    "\n",
    "p (AutoRegressive Order):\n",
    "This parameter represents the number of lagged observations in the model.\n",
    "In other words, it's the number of past data points that the model uses to predict the current value.\n",
    "\n",
    "d (Degree of Differencing):\n",
    "This parameter is the number of times the data have had past values subtracted.\n",
    "Differencing is a method of transforming a time series dataset to make it stationary (i.e., to have constant mean and variance over time).\n",
    "\n",
    "q (Moving Average Order):\n",
    "This parameter represents the size of the moving average window.\n",
    "It's the number of lagged forecast errors that the model uses.\n",
    "A moving average term in an ARIMA model is a past error (difference between an observed value and a predicted value).\n",
    "\n",
    "P (Seasonal AutoRegressive Order):\n",
    "This is similar to p but for the seasonal component of the model.\n",
    "It represents the number of seasonal lags of the autoregressive model.\n",
    "\n",
    "D (Seasonal Differencing Order):\n",
    "Similar to d, it's the number of seasonal differences applied to the series.\n",
    "If a series has a stable seasonal pattern over time, D can help in making the series stationary.\n",
    "\n",
    "Q (Seasonal Moving Average Order):\n",
    "Analogous to q, this is for the seasonal part of the time series.\n",
    "It's the number of seasonal moving average terms.\n",
    "\n",
    "s (Seasonal Periodicity):\n",
    "This parameter represents the length of the seasonal cycle.\n",
    "For example, s is 12 for monthly data with an annual cycle, 4 for quarterly data, etc."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}