src/MLC/notebooks/logistic_regression.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Logistic Regression \n",
    "\n",
    "Logistic regression is a statistical method used for binary classification, which means it is used to predict the probability of an event occurring or not. It is a type of generalized linear model that is used when the dependent variable is binary or categorical.\n",
    "\n",
    "In logistic regression, the dependent variable is binary (i.e., it can take on one of two values, usually 0 or 1), and the independent variables can be either continuous or categorical. The goal of logistic regression is to find the relationship between the independent variables and the dependent variable by estimating the probability of the dependent variable being 1 given the values of the independent variables.\n",
    "\n",
    "The logistic regression model uses a logistic function (also known as the sigmoid function) to map the input values of the independent variables to a value between 0 and 1, which represents the probability of the dependent variable being 1. The logistic function is defined as:\n",
    "\n",
    "css\n",
    "Copy code\n",
    "p = 1 / (1 + e^(-z))\n",
    "where p is the predicted probability of the dependent variable being 1, e is the base of the natural logarithm, and z is the linear combination of the independent variables.\n",
    "\n",
    "The logistic regression model estimates the values of the coefficients of the independent variables that maximize the likelihood of observing the data given the model. This is typically done using maximum likelihood estimation or gradient descent optimization.\n",
    "\n",
    "Once the model is trained, it can be used to make predictions on new data by inputting the values of the independent variables into the logistic function and obtaining the predicted probability of the dependent variable being 1. The model can then classify the new observation as 1 or 0 based on a threshold probability value that is chosen by the user.\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Code \n",
    " Here's an example implementation using gradient descent optimization:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "class LogisticRegression:\n",
    "    \n",
    "    def __init__(self, learning_rate=0.01, n_iters=1000):\n",
    "        self.learning_rate = learning_rate\n",
    "        self.n_iters = n_iters\n",
    "        self.weights = None\n",
    "        self.bias = None\n",
    "        \n",
    "    def fit(self, X, y):\n",
    "        # initialize weights and bias to zeros\n",
    "        n_samples, n_features = X.shape\n",
    "        self.weights = np.zeros(n_features)\n",
    "        self.bias = 0\n",
    "        \n",
    "        # gradient descent optimization\n",
    "        for i in range(self.n_iters):\n",
    "            # calculate predicted probabilities and cost\n",
    "            z = np.dot(X, self.weights) + self.bias\n",
    "            y_pred = self._sigmoid(z)\n",
    "            cost = (-1 / n_samples) * np.sum(y * np.log(y_pred) + (1 - y) * np.log(1 - y_pred))\n",
    "            \n",
    "            # calculate gradients\n",
    "            dw = (1 / n_samples) * np.dot(X.T, (y_pred - y))\n",
    "            db = (1 / n_samples) * np.sum(y_pred - y)\n",
    "            \n",
    "            # update weights and bias\n",
    "            self.weights -= self.learning_rate * dw\n",
    "            self.bias -= self.learning_rate * db\n",
    "            \n",
    "    def predict(self, X):\n",
    "        # calculate predicted probabilities\n",
    "        z = np.dot(X, self.weights) + self.bias\n",
    "        y_pred = self._sigmoid(z)\n",
    "        # convert probabilities to binary predictions\n",
    "        return np.round(y_pred).astype(int)\n",
    "    \n",
    "    def _sigmoid(self, z):\n",
    "        return 1 / (1 + np.exp(-z))\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 1]\n"
     ]
    }
   ],
   "source": [
    "# create sample dataset\n",
    "X = np.array([[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]])\n",
    "y = np.array([0, 0, 1, 1, 1])\n",
    "\n",
    "# initialize logistic regression model\n",
    "lr = LogisticRegression()\n",
    "\n",
    "# train model on sample dataset\n",
    "lr.fit(X, y)\n",
    "\n",
    "# make predictions on new data\n",
    "X_new = np.array([[6, 7], [7, 8]])\n",
    "y_pred = lr.predict(X_new)\n",
    "\n",
    "print(y_pred)  # [1, 1]\n",
    "\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Improvements \n",
    "here are some possible improvements you could make to the code:\n",
    "\n",
    "1. Add regularization: Regularization can help prevent overfitting and improve the generalization performance of the model. You could add L1 or L2 regularization to the cost function and adjust the regularization strength with a hyperparameter. Here's an example of how to add L2 regularization to the code:"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "2. Use a more sophisticated optimization algorithm: Gradient descent is a simple and effective optimization algorithm, but it may not be the most efficient or accurate for large or complex datasets. You could try using a more sophisticated algorithm, such as stochastic gradient descent (SGD), mini-batch SGD, or Adam, which can converge faster and find better optima. Here's an example of how to use mini-batch SGD:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "class LogisticRegression:\n",
    "    \n",
    "    def __init__(self, learning_rate=0.01, n_iters=1000, regularization='l2', reg_strength=0.1, batch_size=32):\n",
    "        self.learning_rate = learning_rate\n",
    "        self.n_iters = n_iters\n",
    "        self.regularization = regularization\n",
    "        self.reg_strength = reg_strength\n",
    "        self.batch_size = batch_size\n",
    "        self.weights = None\n",
    "        self.bias = None\n",
    "        \n",
    "    def fit(self, X, y):\n",
    "        n_samples, n_features = X.shape\n",
    "        self.weights = np.zeros(n_features)\n",
    "        self.bias = 0\n",
    "        n_batches = n_samples // self.batch_size\n",
    "        for i in range(self.n_iters):\n",
    "            batch_indices = np.random.choice(n_samples, self.batch_size)\n",
    "            X_batch = X[batch_indices]\n",
    "            y_batch = y[batch_indices]\n",
    "            z = np.dot(X_batch, self.weights) + self.bias\n",
    "            y_pred = self._sigmoid(z)\n",
    "            cost = (-1 / self.batch_size) * np.sum(y_batch * np.log(y_pred) + (1 - y_batch) * np.log(1 - y_pred))\n",
    "            if self.regularization == 'l2':\n",
    "                reg_cost = (self.reg_strength / (2 * n_samples)) * np.sum(self.weights ** 2)\n",
    "                cost += reg_cost\n",
    "            elif self.regularization == 'l1':\n",
    "                reg_cost = (self.reg_strength / (2 * n_samples)) * np.sum(np.abs(self.weights))\n",
    "                cost += reg_cost\n",
    "            dw = (1 / self.batch_size) * np.dot(X_batch.T, (y_pred - y_batch))\n",
    "            db = (1 / self.batch_size) * np.sum(y_pred - y_batch)\n",
    "            if self.regularization == 'l2':\n",
    "                dw += (self.reg_strength / n_samples) * self.weights\n",
    "            elif self.regularization == 'l1':\n",
    "                dw += (self.reg_strength / n_samples) * np.sign(self.weights)\n",
    "            self.weights -= self.learning_rate * dw\n",
    "            self.bias -= self.learning_rate * db\n",
    "            \n",
    "    def predict(self, X):\n",
    "        z = np.dot(X, self.weights) + self.bias\n",
    "        y_pred = self._sigmoid(z)\n",
    "        return np.round(y_pred).astype(int)\n",
    "    \n",
    "    def _sigmoid(self, z):\n",
    "        return 1 / (1 + np.exp(-z))\n"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This implementation includes the following improvements:\n",
    "\n",
    "1. Regularization: You can choose between L1 or L2 regularization by setting the regularization parameter to either 'l1' or 'l2', and adjust the regularization strength with the reg_strength parameter.\n",
    "\n",
    "2. Mini-batch stochastic gradient descent: The model uses mini-batch SGD (instead of simple gradient descent) to update the weights and bias, which can converge faster and find better optima.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Test "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1 1]\n"
     ]
    }
   ],
   "source": [
    "# create sample dataset\n",
    "X = np.array([[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]])\n",
    "y = np.array([0, 0, 1, 1, 1])\n",
    "\n",
    "# initialize logistic regression model\n",
    "lr = LogisticRegression(learning_rate=0.01, n_iters=1000, regularization='l2', reg_strength=0.1, batch_size=2)\n",
    "\n",
    "# train model on sample dataset\n",
    "lr.fit(X, y)\n",
    "\n",
    "# make predictions on new data\n",
    "X_new = np.array([[6, 7], [7, 8]])\n",
    "y_pred = lr.predict(X_new)\n",
    "\n",
    "print(y_pred)  # [1, 1]\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Visualize "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is difficult to visualize logistic regression since it is a high-dimensional problem. However, we can visualize the decision boundary of a logistic regression model for a two-dimensional dataset.\n",
    "\n",
    "Here's an example of how to visualize the decision boundary of the LogisticRegression class on a 2D dataset using the matplotlib library:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW0AAAD8CAYAAAC8TPVwAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjQuMywgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/MnkTPAAAACXBIWXMAAAsTAAALEwEAmpwYAAASPUlEQVR4nO3dbWyd9XnH8e91jh3jJHZiCE0CyZp0aukoK4G6dAWVbdBWRGXdeLMVqRWgSXnT9WEPQqVv0Dq1UqWpaidVoBTaUZUHMQJSVXUdoI21ldaUhLJSCLSBZSUQyAOBkAec2L72wocsD3bOMTnHt//H349kxT7ntvM7ivLLnf+5//cVmYkkqQy1qgNIklpnaUtSQSxtSSqIpS1JBbG0JakglrYkFaSl0o6Iv46IJyPiVxFxd0Sc0elgkqSTNS3tiDgX+CwwnJkXAHXgE50OJkk6WavLIz1Af0T0APOBFzsXSZI0lZ5mB2TmCxHxj8BvgUPAg5n54InHRcQ6YB1Af733fasHh9qdVW3S2ztCzzlL2DEyzoF9dXpr9aojSXPe7uef2Z2ZZzc7LpptY4+IIWAD8BfAq8C/APdl5vem+p73nLk07/rItdMKrJmzctk2hv7her689SCPPryYpf2DVUeS5rxvffZDmzNzuNlxrSyPfBj4n8zclZlHgPuBS083oCRp+lop7d8CfxAR8yMigCuBLZ2NJUmaTNPSzsyNwH3AY8ATje9Z3+FckqRJNH0jEiAzbwZu7nAWSVIT7oiUpIJY2pJUEEtbkgpiac8xK5dtY/GXruPLv9nPxocWVh1H0jS19EakusP5NwRb338N191aBwapRY8ba6TCWNpzxMpl2+i59Ho2bD1oWUsFc3lEkgpiaUtSQSxtSSqIpS1JBbG0JakglrYkFcTSlqSCWNqSVBBLW5IKYmlLUkEsbUkqiKUtSQVpWtoRcV5EPH7Mx76I+PwMZJMknaDpXf4y8xlgDUBE1IEXgAc6G0uSNJnpLo9cCTybmf/biTDqnIG1q3n6yGsOPpAKN937aX8CuLsTQdQZb06q+crWA2y8bRAI76UtFazl0o6IecDHgZumeH4dsA5g+fyBtoTT6Tl+Us0ils8fqjqSpNM0neWRtcBjmfnyZE9m5vrMHM7M4aG+/vak0+mLAMLClrrEdEr7WlwakaRKtVTaETEf+Ahwf2fjSJJOpaU17cw8CJzV4SySpCbcESlJBbG0JakglrYkFcTSlqSCWNqSVBBLW5IKYmlLUkEsbUkqiKUtSQWxtCWpIJa2JBXE0pakgljaklQQS7tLTUytWcOGbfWqo0hqo+nOiNQsN9lMSKfWSN3D0u4ixxX2w4PUoschvlKXcXmky0QERM3ClrqUpS1JBXF5RNKslgePMLrtVXLfCDG/l/qqxdQG+6qOVZlWB/sujoj7IuLpiNgSER/sdDBJGt9/mMM/f4HxHfvJA0cY33WQI5t3ML7nYNXRKtPq8sg3gB9l5ruBC4EtnYskSRNGt74CY3n8g+PJkaf3kJmTf1OXa7o8EhGDwOXA9QCZeRg43NlYkgT56huTPzEyOlHmPTGzgWaBVs603wHsAr4TEb+IiNsiYsGJB0XEuojYFBGb9o4cantQSXNQ7xSbwyKgNvcKG1or7R7gYuCWzLwIOAB84cSDMnN9Zg5n5vBQX3+bY0qai+q/M3hyOdegtnwhYWlPaTuwPTM3Nr6+j4kSl6SOqq8YpLZiYKK46xNn17UlC+h555lVR6tM0zXtzHwpIp6PiPMy8xngSuCpzkeTNNdFBL3vPItctZg8NEr09RB9c/t+Oq1ep/0Z4M6ImAc8B9zQuUiSdLzorRNTrW/PMS2VdmY+Dgx3NookqRm3sUtSQSxtSSqIpS1JBfGGUV1iYlLNNVx3Sw1YOFf3HUhdz9IunJNqpLnF0i7cwNrVPDO6j40PDzn4QJoDXNPuBhFAWNjSHGBpS1JBLG1JKoilLUkFsbQlqSCWtiQVxNKWpIJY2pJUEEtbkgpiaUtSQSxtSSqI9x6RCpfjyfjug4zvOQi9dXrOGSDm91YdSx3SUmlHxDbgdWAMGM1MR49Js0COJ0ce20HuPwxjCQGHn99Hz3uWUH/bwqrjqQOmc6b9x5m5u2NJJE3b2Iuvk68fhvGceCCBTEaf2k1tyQLCG6t3Hde0pYKNv3zg/wv7BPnaGzOcRjOh1dJO4MGI2BwR6yY7ICLWRcSmiNi0d+RQ+xJKmlLUT3EmXfOcrBu1+qd6WWZeDKwFPh0Rl594QGauz8zhzBwe6utva0hJk6udOwCTFXdPjRicN/OB1HEtlXZmvtj4dSfwAHBJJ0OpuZXLtvH76/+Qr579Pm781uKq46gitSXzqZ0zALWY+KgH9NbovXApEa5nd6Omb0RGxAKglpmvNz7/KPCljifTlI4O8b21DixyJuQcFhH0vusscuUg43vfgN4atbPm+wZkF2vl6pGlwAONf7V7gLsy80cdTaXmGiPGLGwBRH8v9X6vzZ4LmpZ2Zj4HXDgDWSRJTfj2siQVxNKWpIJY2pJUEEtbkgpiaUtSQSxtSSqIpS1JBbG0JakglrYkFcTSlqSCWNqSVBBLW5IKYmlLUkEs7cJM3Et7jYMPpDlqOtPYVaGVy7ax+EvX8ZWtB9h42yC16GFp/2DVsSTNMEu7AEcL+9mDbHzYSTXSXObySCGiMammFvWqo0iqkGfamhPy8Bjjuw9CTAzDjV7/8VOZWi7tiKgDm4AXMvPqzkWS2mv0hX2M/foVCCAB9lD/vSX0LFtYcTJp+qazPPI5YEungkidkIeOTBT2eMJYTvw6noxt2U2OjFYdT5q2lko7IlYAHwNu62wcqb3GXj4AmZM+N77r4AynkU5fq2faXwduBManOiAi1kXEpojYtHfkUDuySadvPBtLIifIJMcnL3NpNmta2hFxNbAzMzef6rjMXJ+Zw5k5PNTX37aA0umonT0fanHyExHUl8yf+UDSaWrlTPsy4OMRsQ24B7giIr7X0VRSm9QG+qidO3B8cdeC+tsXEfN7qwsmvUVNrx7JzJuAmwAi4o+Av8vMT3Y2ltQ+ve86i/GlCybWt4H6soXUBvsqTiW9NV6nrTmhtugMaovOqDqGdNqmVdqZ+QjwSEeSSJKachu7JBXE0pakgljaklQQS1uSCmJpS1JBLG1JKojXac9yEzMhr+G6W2rAwkl3ZEuaOyztWerEmZAQjhmTZGnPVgNrV/PM6D42PjzkEF9JR7mmPZs15kJa2JLeZGlLUkEsbUkqiKUtSQWxtCWpIJa2JBXE0pakgljaklQQS1sTMul/aRcLt20nRkerTiNpCk13REbEGcCPgb7G8fdl5s2dDqaZ07dnL+9efxd9r+wlazXI5Lk//xP2DL+36miSTtDKNvYR4IrM3B8RvcBPI+JfM/NnHc6mmZDJ+d+8g3mvvEot8+jDv3vP9zm0/G0cPHdZheEknajp8khO2N/4srfxkaf4FhVk4bbt9Ow/cFxhA8ToKMt+8vOKUkmaSktr2hFRj4jHgZ3AQ5m5cZJj1kXEpojYtHfkUJtjqlN69x9o3OPkeLVM5r22r4JEkk6lpdLOzLHMXAOsAC6JiAsmOWZ9Zg5n5vBQX3+bY6pTXl+1gtro2EmPj/X2svf8d1WQSNKpTOvqkcx8FXgEuKoTYTTzRgcW8sIVlzI2r/foY2O9PRweWsSuD6ypLpikSbVy9cjZwJHMfDUi+oEPA1/teDLNmO0fu5IDb1/Bsv/8GT2H3mDPhefz0ocuYXzevKqjSTpBK1ePLAfuiIg6E2fm92bmDzobSzNt7wXnsfeC86qOIamJpqWdmb8ELpqBLOKEMWPfcviBpOM5bmwWOTrE99Y6MOiYMUknsbRniZXLttFz6fVs2HrQspY0Je89IkkFsbQlqSCWtiQVxNKWpIJY2pJUEEtbkgpiaUtSQSxtSSqIpS1JBbG0JakglrYkFcTSlqSCWNqSVBBLW5IKYmnPEgNrV/P0kdfY+NDCqqNImsW8n/Y0ZCbjY+PU6jUioi0/87hJNbcNAuG9tCVNqZXBviuB7wLLgHFgfWZ+o9PBZpPMZMeTL7PzN3sYHxun94weVly4nKGVi0/r5x4t7GcPsvHhRSyfP9SewJK6Vitn2qPA32bmYxExAGyOiIcy86kOZ5s1XvjlS+x6dg85lgAcOTTKtke3U59XZ3DpwFv+uQNrVxPz+oBD1KLeprSSulnTNe3M3JGZjzU+fx3YApzb6WCzxfjYOLuPKew35Viy48mdFaWSNFdN643IiFjFxGT2jZM8ty4iNkXEpr0jh9oUr3qjI6PkFM+N7B+Z0SyS1HJpR8RCYAPw+czcd+Lzmbk+M4czc3ior7+dGSvVe0bvlG869i/untcpqQwtlXZE9DJR2Hdm5v2djTS7RC1Yfv7bqNWPL+6oB+dcsLSiVJLmqlauHgngdmBLZn6t85Fmn6XnnU29r87LT+3iyBtH6F/cz7nvXcaCM+dXHU3SHNPK1SOXAZ8CnoiIxxuPfTEzf9ixVLPQklVnsmTVmVXHkDTHNS3tzPwp0J6dJJKk0+I2dkkqiKUtSQWxtCWpIJa2JBXE0pakgljaklQQS1uSCmJpS1JBLG1JKojjxipy/g3B1vev4cZ/GgUGqLnnVFILLO0ZNtlMSMeMSWqVpT3DBtau5pnRfWx8eIha9DjEV9K0uKZdhQicui7prbC0JakglrYkFcTSlqSCWNqSVBBLW5IK0rS0I+LbEbEzIn41E4EkSVNr5Uz7n4GrOpxDktSCpqWdmT8GXpmBLJKkJlzTlqSCtK20I2JdRGyKiE17Rw6168dKko7RttLOzPWZOZyZw0N9/e36sZKkY7g8IkkFaeWSv7uB/wLOi4jtEfGXnY8lSZpM01uzZua1MxFkLli5bBv1D17HhmcPVh1FUqG8n/YMmZhUcw3X3VpnYlJNvepIkgpkaXeYk2oktZOlPQMiAqLmpBpJp82rRySpIJa2JBXE0pakgljaklQQS1uSCmJpS1JBLG1JKoilLUkFsbQlqSCWtiQVxNKWpIJY2pJUEEtbkgpiaUtSQSxtSSqIpS1JBWmptCPiqoh4JiK2RsQXOh2qmwysXc3TR15j40MLq44iqQs0nVwTEXXgm8BHgO3AoxHx/cx8qtPhSjbZmDGn1kg6Xa2MG7sE2JqZzwFExD3AnwKW9hSOH+K7yJmQktqmldI+F3j+mK+3Ax848aCIWAesa3w5subeb/zq9OPNSkuA3ac84t6ZCdIhzV9f2Xx9Zevm13deKwe1UtoxyWN50gOZ64H1ABGxKTOHWwlQmm5+beDrK52vr1wRsamV41p5I3I7sPKYr1cAL76VUJKk09NKaT8KvDMiVkfEPOATwPc7G0uSNJmmyyOZORoRfwX8G1AHvp2ZTzb5tvXtCDdLdfNrA19f6Xx95WrptUXmScvTkqRZyh2RklQQS1uSCtLW0u7m7e4R8e2I2BkRXXn9eUSsjIj/iIgtEfFkRHyu6kztFBFnRMTPI+K/G6/v76vO1G4RUY+IX0TED6rO0m4RsS0inoiIx1u9NK4kEbE4Iu6LiKcbfwc/OOWx7VrTbmx3/zXHbHcHru2W7e4RcTmwH/huZl5QdZ52i4jlwPLMfCwiBoDNwJ910Z9fAAsyc39E9AI/BT6XmT+rOFrbRMTfAMPAYGZeXXWedoqIbcBwZnblxpqIuAP4SWbe1rhKb35mvjrZse080z663T0zDwNvbnfvCpn5Y+CVqnN0SmbuyMzHGp+/DmxhYjdsV8gJ+xtf9jY+uuZd+IhYAXwMuK3qLJqeiBgELgduB8jMw1MVNrS3tCfb7t41f+nnkohYBVwEbKw4Sls1lg8eB3YCD2VmN72+rwM3AuMV5+iUBB6MiM2NW2Z0k3cAu4DvNJa3bouIBVMd3M7Sbmm7u2a3iFgIbAA+n5n7qs7TTpk5lplrmNjVe0lEdMUyV0RcDezMzM1VZ+mgyzLzYmAt8OnGcmW36AEuBm7JzIuAA8CU7wm2s7Td7l64xlrvBuDOzLy/6jyd0viv5yPAVdUmaZvLgI831n3vAa6IiO9VG6m9MvPFxq87gQeYWI7tFtuB7cf8z+8+Jkp8Uu0sbbe7F6zxRt3twJbM/FrVedotIs6OiMWNz/uBDwNPVxqqTTLzpsxckZmrmPh79++Z+cmKY7VNRCxovDlOY9ngo0DXXMWVmS8Bz0fEm3f5u5JT3Pq6lbv8tfobv5Xt7sWIiLuBPwKWRMR24ObMvL3aVG11GfAp4InGui/AFzPzh9VFaqvlwB2Nq5xqwL2Z2XWXxnWppcADE+cV9AB3ZeaPqo3Udp8B7myc8D4H3DDVgW5jl6SCuCNSkgpiaUtSQSxtSSqIpS1JBbG0JakglrYkFcTSlqSC/B/dAeYbl24tJgAAAABJRU5ErkJggg==",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "\n",
    "# create 2D dataset\n",
    "X = np.array([[1, 2], [2, 3], [3, 4], [4, 5], [5, 6]])\n",
    "y = np.array([0, 0, 1, 1, 1])\n",
    "\n",
    "# initialize logistic regression model\n",
    "lr = LogisticRegression(learning_rate=0.01, n_iters=1000, regularization='l2', reg_strength=0.1, batch_size=2)\n",
    "\n",
    "# train model on dataset\n",
    "lr.fit(X, y)\n",
    "\n",
    "# plot decision boundary\n",
    "x1 = np.linspace(0, 6, 100)\n",
    "x2 = np.linspace(0, 8, 100)\n",
    "xx, yy = np.meshgrid(x1, x2)\n",
    "Z = lr.predict(np.c_[xx.ravel(), yy.ravel()])\n",
    "Z = Z.reshape(xx.shape)\n",
    "plt.contourf(xx, yy, Z, cmap=plt.cm.Spectral, alpha=0.8)\n",
    "\n",
    "# plot data points\n",
    "plt.scatter(X[:,0], X[:,1], c=y, cmap=plt.cm.Spectral)\n",
    "\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.7"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}