Part1_1 copy.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c557c6fae90fa361",
   "metadata": {},
   "source": [
    "# Part 1: Perceptron\n",
    "## (i)\n",
    "### Step 1: Data loading and initialisation"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e3f951606ed45d4a",
   "metadata": {},
   "source": [
    "Step 1.1: The read_csv method of the pandas library was used to load a dataset containing fish annular scale diameters (freshwater and brackish) and the classification of the fish (Canadian or Alaskan)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "7dd07f28930ac354",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-20T02:42:30.175341Z",
     "start_time": "2024-08-20T02:42:30.167124Z"
    }
   },
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# Load data\n",
    "data = pd.read_csv('Fish_data.csv')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af7927650fa61d3a",
   "metadata": {},
   "source": [
    "Step 1.2: The first and last rows from the dataset are extracted as samples for training the model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "65a4efa581d63b59",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-20T02:42:32.254245Z",
     "start_time": "2024-08-20T02:42:32.247624Z"
    }
   },
   "outputs": [],
   "source": [
    "# Extract the first and last rows\n",
    "first_row = data.iloc[0]\n",
    "last_row = data.iloc[-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "840248e817189c2f",
   "metadata": {},
   "source": [
    "### Step 2: Define the function and set initial parameters"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "98db4459f03ec991",
   "metadata": {},
   "source": [
    "- Activation function: computes a linear combination of the input x and the weights weights, the output of this function is used to determine the classification (1 or 0).\n",
    "- Weight update function: updates the weights according to the prediction error (difference between the target value t and the predicted value y), the adjustment of the weights depends on the learning rate learning_rate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "73339531bafa7178",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-20T02:42:40.542041Z",
     "start_time": "2024-08-20T02:42:40.535479Z"
    }
   },
   "outputs": [],
   "source": [
    "# Initial weights and learning rates\n",
    "weights = np.array([102, -28, 5.0])  # w1, w2, bias\n",
    "learning_rate = 0.5\n",
    "\n",
    "# Activation function\n",
    "def activation(x, weights):\n",
    "    return np.dot(x, weights[:2]) + weights[2]\n",
    "\n",
    "# Weight update function\n",
    "def update_weights(x, y, t, weights, learning_rate):\n",
    "    error = t - y\n",
    "    weights[:2] += learning_rate * error * x  # Update w1 and w2\n",
    "    weights[2] += learning_rate * error  # Update Bias\n",
    "    return weights"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bbf094d27af367fd",
   "metadata": {},
   "source": [
    "### Step 3: Plot and train the model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ada4a6c266af695a",
   "metadata": {},
   "source": [
    "- In two epochs, the model is trained using two sample points.\n",
    "- After each weight update, new classification boundaries are drawn to visualise the learning process."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "af0bb2bf39909728",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2024-08-20T02:42:43.408217Z",
     "start_time": "2024-08-20T02:42:43.216870Z"
    }
   },
   "outputs": [],
   "source": [
    "# Drawing classification boundaries in a single plot\n",
    "\n",
    "# Drawing classification boundaries with equations and point annotations\n",
    "def plot_boundary(points, epoch_data):\n",
    "    plt.figure(figsize=(10, 6))\n",
    "    colors = ['blue', 'green', 'red', 'cyan', 'magenta', 'yellow', 'black', 'orange']\n",
    "    \n",
    "    for index, (epoch, weights) in enumerate(epoch_data):\n",
    "        x_values = np.array([min(points[:, 0]) - 10, max(points[:, 0]) + 10])\n",
    "        m = -weights[0] / weights[1]  # Slope\n",
    "        c = -weights[2] / weights[1]  # Intercept\n",
    "        y_values = m * x_values + c\n",
    "        label = f'{index + 1}: Epoch {epoch}, Line: $x_2 = {m:.2f}x_1 + {c:.2f}$'\n",
    "        plt.plot(x_values, y_values, color=colors[index % len(colors)], label=label)\n",
    "    \n",
    "    \n",
    "    # Plot points with annotations\n",
    "    for point, label in zip(points, ['Point 1', 'Point 2']):\n",
    "        plt.scatter(point[0], point[1], c='black', s=50)  # Draw the point\n",
    "        plt.text(point[0], point[1], f'{label} ({point[0]}, {point[1]})', fontsize=12, verticalalignment='bottom')\n",
    "\n",
    "    plt.xlabel('Ring diameter - freshwater')\n",
    "    plt.ylabel('Ring diameter - saltwater')\n",
    "    plt.title('Perceptron Classification Boundaries with Annotations')\n",
    "    plt.legend(loc='upper left', bbox_to_anchor=(1, 1))  # Move legend outside of plot\n",
    "    plt.grid(True)\n",
    "    plt.show()\n",
    "\n",
    "# Two point data for plotting\n",
    "points = np.array([\n",
    "    [first_row['RingDiam_fresh_water'], first_row['RingDiam_salt_water'], first_row['Canadian_0_Alaskan_1']],\n",
    "    [last_row['RingDiam_fresh_water'], last_row['RingDiam_salt_water'], last_row['Canadian_0_Alaskan_1']]\n",
    "])\n",
    "\n",
    "# Train the model and draw the classification boundary\n",
    "\n",
    "epoch_data = []  # Used to store weights, biases, and epoch info\n",
    "\n",
    "for epoch in range(2):\n",
    "    for row in [first_row, last_row]:\n",
    "        x = np.array([row['RingDiam_fresh_water'], row['RingDiam_salt_water']])\n",
    "        target = row['Canadian_0_Alaskan_1']\n",
    "        output = 1 if activation(x, weights) > 0 else 0\n",
    "        weights = update_weights(x, output, target, weights, learning_rate)\n",
    "        epoch_data.append((epoch, weights.copy())) \n",
    "       \n",
    "# Plot all boundaries after training\n",
    "plot_boundary(points, epoch_data)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "591268fc2337328a",
   "metadata": {},
   "source": [
    "## Weights update analysis\n",
    "\n",
    "### Input data\n",
    "Two specific data points (from the first and last rows of the dataset) are used to train the perceptron model:\n",
    "\n",
    "- **First row** (Sample 1):\n",
    "  - `RingDiam_fresh_water`: 112\n",
    "  - `RingDiam_salt_water`: 394\n",
    "  - `Canadian_0_Alaskan_1`: 0 (Target)\n",
    "\n",
    "- **Last row** (Sample 2):\n",
    "  - `RingDiam_fresh_water`: 129\n",
    "  - `RingDiam_salt_water`: 420\n",
    "  - `Canadian_0_Alaskan_1`: 1 (Target)\n",
    "\n",
    "### Initialising weights and biases\n",
    "The initial weights and biases are set as follows:\n",
    "$$\n",
    "\\( w1 = 102 \\)\n",
    "$$\n",
    "$$\n",
    "\\( w2 = -28 \\)\n",
    "$$\n",
    "$$\n",
    "\\( \\text{bias} = 5.0 \\)\n",
    "$$\n",
    "\n",
    "### Weight Update Steps\n",
    "Learning rate used \n",
    "$$\n",
    "\\( \\beta = 0.5 \\)\n",
    "$$\n",
    "\n",
    "#### Epoch 0, Sample 1 calculations:\n",
    "$$\n",
    "\\( x1 = 112 \\)\n",
    "$$\n",
    "$$\n",
    "\\( x2 = 394 \\)\n",
    "$$\n",
    "$$\n",
    "\\( t = 0 \\)\n",
    "$$\n",
    "\n",
    "Activation value:\n",
    "$$\n",
    "\\text{Activation} = 102 \\times 112 + (-28) \\times 394 + 5 = 11424 - 11032 + 5 = 397\n",
    "$$\n",
    "$$\n",
    "y = 1 \\quad (\\text{because Activation > 0})\n",
    "$$\n",
    "\n",
    "**Weights update**(weights need to be reduced due to \\( t = 0, y = 1 \\)):\n",
    "$$\n",
    "w1_{\\text{new}} = 102 - 0.5 \\times (1 - 0) \\times 112 = 102 - 56 = 46\n",
    "$$\n",
    "$$\n",
    "w2_{\\text{new}} = -28 - 0.5 \\times (1 - 0) \\times 394 = -28 - 197 = -225\n",
    "$$\n",
    "$$\n",
    "\\text{bias}_{\\text{new}} = 5 - 0.5 \\times (1 - 0) = 4.5\n",
    "$$\n",
    "\n",
    "Classification boundary line:\n",
    "$$\n",
    "x2 = -\\frac{w1}{w2}x1 - \\frac{\\text{bias}}{w2} = -\\frac{46}{-225}x1 - \\frac{4.5}{-225} = \\frac{46}{225}x1 + \\frac{4.5}{225}\n",
    "$$\n",
    "After simplifying:\n",
    "$$\n",
    "x2 = 0.2044x1 + 0.02\n",
    "$$\n",
    "\n",
    "#### Epoch 0, Sample 2 calculations:\n",
    "$$\n",
    "\\( x1 = 129 \\)\n",
    "$$\n",
    "$$\n",
    "\\( x2 = 420 \\)\n",
    "$$\n",
    "$$\n",
    "\\( t = 1 \\)\n",
    "$$\n",
    "\n",
    "Activation value:\n",
    "$$\n",
    "\\text{Activation} = 46 \\times 129 + (-225) \\times 420 + 4.5 = 5934 - 94500 + 4.5 = -88561.5\n",
    "$$\n",
    "$$\n",
    "y = 0 \\quad (\\text{because Activation < 0})\n",
    "$$\n",
    "\n",
    "**Weights update**(weights need to be increased due to \\( t = 1, y = 0 \\)):\n",
    "$$\n",
    "w1_{\\text{new}} = 46 + 0.5 \\times (1 - 0) \\times 129 = 46 + 64.5 = 110.5\n",
    "$$\n",
    "$$\n",
    "w2_{\\text{new}} = -225 + 0.5 \\times (1 - 0) \\times 420 = -225 + 210 = -15\n",
    "$$\n",
    "$$\n",
    "\\text{bias}_{\\text{new}} = 4.5 + 0.5 \\times (1 - 0) = 5.0\n",
    "$$\n",
    "\n",
    "Classification boundary line:\n",
    "$$\n",
    "x2 = -\\frac{110.5}{-15}x1 - \\frac{5.0}{-15} = \\frac{110.5}{15}x1 + \\frac{5.0}{15}\n",
    "$$\n",
    "After simplifying:\n",
    "$$\n",
    "x2 = 7.3667x1 + 0.3333\n",
    "$$\n",
    "\n",
    "#### Epoch 1, Sample 1 calculations:\n",
    "$$\n",
    "\\( x1 = 112 \\)\n",
    "$$\n",
    "$$\n",
    "\\( x2 = 394 \\)\n",
    "$$\n",
    "$$\n",
    "\\( t = 0 \\)\n",
    "$$\n",
    "\n",
    "Activation value:\n",
    "$$\n",
    "\\text{Activation} = 110.5 \\times 112 + (-15) \\times 394 + 5 = 12376 - 5910 + 5 = 6471\n",
    "$$\n",
    "$$\n",
    "y = 1 \\quad (\\text{because Activation > 0})\n",
    "$$\n",
    "\n",
    "**Weights update**(weights need to be reduced due to \\( t = 0, y = 1 \\)):\n",
    "$$\n",
    "w1_{\\text{new}} = 110.5 - 0.5 \\times (1 - 0) \\times 112 = 110.5 - 56 = 54.5\n",
    "$$\n",
    "$$\n",
    "w2_{\\text{new}} = -15 - 0.5 \\times (1 - 0) \\times 394 = -15 - 197 = -212\n",
    "$$\n",
    "$$\n",
    "\\text{bias}_{\\text{new}} = 5 - 0.5 \\times (1 - 0) = 4.5\n",
    "$$\n",
    "\n",
    "Classification boundary line:\n",
    "$$\n",
    "x2 = -\\frac{54.5}{-212}x1 - \\frac{4.5}{-212} = \\frac{54.5}{212}x1 + \\frac{4.5}{212}\n",
    "$$\n",
    "After simplifying:\n",
    "$$\n",
    "x2 = 0.2571x1 + 0.0212\n",
    "$$\n",
    "\n",
    "#### Epoch 1, Sample 2 calculations:\n",
    "$$\n",
    "\\( x1 = 129 \\)\n",
    "$$\n",
    "$$\n",
    "\\( x2 = 420 \\)\n",
    "$$\n",
    "$$\n",
    "\\( t = 1 \\)\n",
    "$$\n",
    "\n",
    "Activation value:\n",
    "$$\n",
    "\\text{Activation} = 54.5 \\times 129 + (-212) \\times 420 + 4.5 = 7030.5 - 89040 + 4.5 = -8200\n",
    "$$\n",
    "$$\n",
    "y = 0 \\quad (\\text{because Activation < 0})\n",
    "$$\n",
    "\n",
    "**Weights update**(weights need to be increased due to \\( t = 1, y = 0 \\)):\n",
    "$$\n",
    "w1_{\\text{new}} = 54.5 + 0.5 \\times (1 - 0) \\times 129 = 54.5 + 64.5 = 119\n",
    "$$\n",
    "$$\n",
    "w2_{\\text{new}} = -212 + 0.5 \\times (1 - 0) \\times 420 = -212 + 210 = -2\n",
    "$$\n",
    "$$\n",
    "\\text{bias}_{\\text{new}} = 4.5 + 0.5 \\times (1 - 0) = 5.0\n",
    "$$\n",
    "\n",
    "Classification boundary line:\n",
    "$$\n",
    "x2 = -\\frac{119}{-2}x1 - \\frac{5.0}{-2} = \\frac{119}{2}x1 + \\frac{5.0}{2}\n",
    "$$\n",
    "After simplifying:\n",
    "$$\n",
    "x2 = 59.5x1 + 2.5\n",
    "$$\n",
    "\n",
    "### Table of changes in weights and biases\n",
    "\n",
    "| Epoch | Sample | \\( w1 \\) | \\( w2 \\) | Bias  |\n",
    "|-------|--------|----------|----------|-------|\n",
    "| 0     | 1      | 46       | -225     | 4.5   |\n",
    "| 0     | 2      | 110.5    | -15      | 5.0   |\n",
    "| 1     | 1      | 54.5     | -212     | 4.5   |\n",
    "| 1     | 2      | 119      | -2       | 5.0   |\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}