Chapter01_Review.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "5ca21ee6-db11-4a9b-905e-19f51e187307",
   "metadata": {},
   "source": [
    "# Chapter 1 - Review of Basic Libraries\n",
    "This section reviews the basic libraries that we are using in this course. (Approximately 2hours). But before starting, let's review our environment (jupyter) for a second."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adc4a51a-d6b7-48ab-b83a-988f0a7c5252",
   "metadata": {
    "jp-MarkdownHeadingCollapsed": true,
    "tags": []
   },
   "source": [
    "<img alt=\"NumPy logo 2020.svg\" src=\"//upload.wikimedia.org/wikipedia/commons/thumb/3/31/NumPy_logo_2020.svg/220px-NumPy_logo_2020.svg.png\">\n",
    "Numpy is Python library for high performance calculations.\n",
    "Numpy provides the following:\n",
    "\n",
    "<br>\n",
    "<ol>\n",
    "<li> High performance storing, computation</li>\n",
    "    <li> Support, multi dimensional arrays and matrices</li>\n",
    "    <li> A large collections of high level mathemtaical operations</li>\n",
    "</ol>\n",
    "\n",
    "To install numpy in anaconda use:\n",
    "`conda install -c anaconda numpy`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a657be1-dbe0-4c37-b0fa-8cdabdd0dbd8",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np # it is usually loaded"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b6b5ac42-2484-4e74-a191-cb10b844a2a9",
   "metadata": {},
   "source": [
    "## Numpy ndarray\n",
    "The building block for numpy is homogeneous multidimensional array called __ndarray__. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dca2027a-7ea9-48e5-ae2f-c079745af63f",
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.arange(15).reshape(3, 5) # Return evenly spaced values within a given interval. \n",
    "a"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "636622eb-7afe-4527-a4df-7adff659007b",
   "metadata": {},
   "outputs": [],
   "source": [
    "type(a)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdf397b1-67f5-4b89-95bb-c20133a5d1bc",
   "metadata": {},
   "source": [
    "#### Properties\n",
    "ndarray object has some properties. The most important ones are __shape__, __dtype__ and __size__."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "74eef723-f5d4-4a24-b3d8-73e3ef677b28",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"a.shape\", a.shape) # the dimensions of the array \n",
    "print(\"a.ndim\", a.ndim) # the number of axes (dimensions) of the array.\n",
    "print(\"a.size\", a.size) # the total number of elements of the array.\n",
    "print(\"a.dtype\", a.dtype) # an object describing the type of the elements in the array. \n",
    "print(\"a.itemsize\", a.itemsize) # the size in bytes of each element of the array. \n",
    "print(\"a.data\", a.data) # the buffer containing the actual elements of the array.\n",
    "# You can call a.data.tolist() to see the buffer"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1d0901a0-eb77-4e59-ab11-f676c84770e5",
   "metadata": {},
   "source": [
    "### Array creation\n",
    "There are certain numpy functions for creating arrays. Let's review some of them.\n",
    "\n",
    "#### Arrays from Py lists"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b8fc150a-838a-4cbf-9500-9d404c434167",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating from list objects\n",
    "a = np.array([1,2,3])\n",
    "print(a)\n",
    "print(a.dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7b70a0ba-460a-4319-b9ad-227ef595dc85",
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.array([1,2,3], dtype=np.float64)\n",
    "print(a)\n",
    "print(a.dtype)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06fdf09a-665f-4ecf-90c2-055a992c16a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.array([[1,2,3], [4,5,6], [7,8,9]], dtype=np.float64)\n",
    "print(a)\n",
    "print(\"shape:\", a.shape)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ab4f9d12-5324-4b0f-9298-2c8f483c7fa2",
   "metadata": {},
   "source": [
    "#### Arrays with initial values\n",
    "You can initialize ndarrays of arbitrary shape with __zero__, __ones__,  __eye__, __arange__, __linespace__, __logspace__ and __empty__ functions."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7a2e7acc-40cc-475f-b389-1f83a9361fef",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.zeros(shape=(4,4), dtype=np.float64) # all zero ndarray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42451c19-d099-4d4a-98e0-e1b7d8b3d83b",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.ones((4,4))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c25addd0-36bf-41cf-8944-c179b0b81b96",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.eye(4) # The identity matrix. The argument is a single number like n and return nxn matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "de37a12c-f3be-4d25-8f2e-1c946675ae6c",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.arange(10) # It is like python range builtin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cfc37508-e3b1-4fcd-8a49-c20926aaebfd",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.arange(start=5, stop=100, step=10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "643b0a3a-210b-45f8-bab4-5005731cc567",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.linspace(start=0, stop=1, num=10) # Equally spaced num points between start and stop"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0af3580e-09b8-43bc-88e9-284e37671b69",
   "metadata": {},
   "outputs": [],
   "source": [
    "# The end point is included unless it is excluded.\n",
    "np.linspace(start=0, stop=1, num=10, endpoint=False) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "32d3f26e-2cdc-4c86-b46f-602a009d29fa",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.empty((4,4)) # This returns an array without any initialization like C"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1bfa2023-d35c-4aa2-94a3-b4f5793c9a07",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "np.logspace(1,4,4) # It is a little bit different than logspace in MATLAB and the syntax of linspace"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "76e74c2f-a79e-4e9e-89d6-fa4eede530c9",
   "metadata": {},
   "source": [
    "#### Random Generator"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aa8b87d5-dd77-436f-ba0b-8860642c0666",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Unifrom Random\n",
    "np.random.rand(5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1b496905-ef04-4bd4-a5e5-58e87a43277e",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.random.rand(5,5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0746a049-76a3-4bdb-81fc-0386f3f87f86",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Normally Distributed\n",
    "np.random.randn(5)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45127db4-8121-45df-87b8-aef4589e589d",
   "metadata": {},
   "source": [
    "<img src=\"https://img.icons8.com/color/344/light.png\" width=70>  __np.random:__<br>\n",
    "\n",
    "<br>\n",
    "np.random offers more distributions. Also beside random generation, it has more functions for working with pdf and cdf functions. Check the documents for more."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f755f5e-c79d-44fa-b238-1baf7de01c1b",
   "metadata": {},
   "source": [
    "#### Reviewing Some Basic Operations on ndarray"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed8cee20-4800-4e34-9e8e-758fc87052c1",
   "metadata": {},
   "outputs": [],
   "source": [
    "a = np.array([[1,2], [3,4]],dtype=np.float32)\n",
    "b = np.array([[-1,-2], [-3,-4]],dtype=np.float32)\n",
    "c = np.arange(9).reshape((3,3))\n",
    "print(a, b, c, sep=\"\\n\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fea4bd67-bd1f-450a-b92b-3a071f0a058f",
   "metadata": {},
   "outputs": [],
   "source": [
    "a + b # Matrix (element wise) addition and subtraction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d1185b75-925f-433a-9e7b-af10e4c50905",
   "metadata": {},
   "outputs": [],
   "source": [
    "a * b # Element wise product"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cdcaaa86-65aa-40e0-b371-cf3a75c50a1b",
   "metadata": {},
   "outputs": [],
   "source": [
    "3 * a # Scalar-Matrix multiplication"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7ea8fdf1-6293-4339-bb78-00bd8c26dc61",
   "metadata": {},
   "outputs": [],
   "source": [
    "a/b # Element-wise division"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a5b3e201-e368-4e18-b680-6d9ceb628441",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.matmul(a,b) # Matrix Multiplication"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "10121ee0-c299-472c-bc33-d36827ae899c",
   "metadata": {},
   "outputs": [],
   "source": [
    "a**2 # Power"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7aca32fb-9e30-492b-8806-120a4d439cfb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Indexing (zero-based, row-wise)\n",
    "print(a[0])\n",
    "print(b[0][0])\n",
    "# Two dimensional indexing\n",
    "print(a[0,0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53706d9a-e73e-41a1-8e55-d5e7fd1fccfd",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(c[0:2]) # Slicing\n",
    "print(c[:,1]) # Slicing with default start \n",
    "print(c[1,:]) # slicing with default end\n",
    "print(c[0:2, 1:2]) # 2D Slicing"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7f0f5ef1-e5de-4d5b-8893-c753669826b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Iteration over ndarray\n",
    "for row in c:\n",
    "    print(row, end=\"\\n----------\\n\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "752fbaa0-443e-49a6-9bb9-eb441d75fd64",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Iteration over elements in flat mode\n",
    "for el in c.flat:\n",
    "    print(el, end=' ')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d160982f-4650-4504-ada8-4e2477bdcd4d",
   "metadata": {},
   "source": [
    "#### Shape Operations\n",
    "You can change the shape of a matrix. Think about it as representation of stored numbers."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "de723576-7d3b-4043-8028-73e2c7bd7087",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(c.T) # The transpose of matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "653efa05-3eb2-471a-9459-4d92d88a3198",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(c.ravel()) # Vectorizing matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9693066b-20be-4a20-ada8-0dde858ccbdb",
   "metadata": {},
   "outputs": [],
   "source": [
    "d = np.eye(4)\n",
    "print(d, end=\"\\n----------\\n\")\n",
    "print(d.reshape((2,8)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4af83c09-2aeb-4ff2-b000-ca5b126ad0cc",
   "metadata": {},
   "outputs": [],
   "source": [
    "d.resize((2,8)) # Change the size in place\n",
    "print(d)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c41887f8-5659-473c-9a2c-adfe7422e107",
   "metadata": {},
   "source": [
    "#### Some useful numpy functions\n",
    "Let's review some of useful functions in numpy"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d21832a2-cb7b-4fa1-8d87-17fe24541303",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.log(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a8f7875b-bfc5-4a71-8626-7a58592ca09b",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.sin(a)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86e0b794-d69a-4bf7-979e-838aee9ce55c",
   "metadata": {},
   "outputs": [],
   "source": [
    "a.sum(), a.cumsum()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5a422926-a90f-44e0-8a4f-3c81557ca4a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "np.sum(a), np.sum(a, axis=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37b2733d-0f83-45bc-88f9-70314fdfd912",
   "metadata": {},
   "outputs": [],
   "source": [
    "a.max(), a.max(axis=0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b066a765-6073-4e66-962f-a6d942495ce3",
   "metadata": {},
   "outputs": [],
   "source": [
    "a.argmax(), a.flat[a.argmax()]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "665b1831-5fd6-4693-b156-42a9b598a694",
   "metadata": {},
   "source": [
    "<img src=\"https://img.icons8.com/color/344/light.png\" width=70>  __Hint:__<br>\n",
    "What is np.reshape(-1,1)?\n",
    "<br>\n",
    "in reshape you can pass one of arguments as -1 and let python decide about that using the original size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "96708dfa-1c94-4121-9ac2-1e66a3398078",
   "metadata": {},
   "outputs": [],
   "source": [
    "d = np.eye(6)\n",
    "d.reshape(-1,9) # The original one has 6x6 =36 elements. so if you say reshape to (-1, 9) it will be 4x9"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e5306608-4de9-4f5d-bc19-6cdad2e3c3e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "# reshape(-1,1) is just converting a matrix to a 1D matrix\n",
    "print(a.reshape(-1,1))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "31c05378-2415-4edb-98bb-10ce60657768",
   "metadata": {},
   "source": [
    "<img src= \"https://img.icons8.com/external-flaticons-flat-flat-icons/344/external-question-100-most-used-icons-flaticons-flat-flat-icons.png\" width=70> __Question__:<br>\n",
    "Write a program which calculates the values of function y = sin(x*x) for 100 points in range of [0, pi]. The find maximum and minimum value of the function and corresponding x values. <br>\n",
    "Hint: Use np.pi for the pie number. (3 lines of code - Approx. 5 mins)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6b7a198-4740-440a-b24c-3071d1888fa3",
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.linspace(0,np.pi,100)\n",
    "f = np.sin(x*x)\n",
    "f.max(), x[f.argmax()]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c90dc4eb-a931-4988-9f8b-eb2abc439fca",
   "metadata": {},
   "source": [
    "<br><br>\n",
    "<img src=\"https://pandas.pydata.org/docs/_static/pandas.svg\" class=\"logo\" alt=\"Pandas\" width=300><br>\n",
    "Pandas is a python library for working and manipulating data (tabular format mostly).<br>\n",
    "For installing pandas use the following command:<br>\n",
    "`conda install -c anaconda pandas`\n",
    "<br> Pandas uses numpy underhood. \n",
    "The building blocks in pandas are series and dataframes. Series are used for storing a sequence of objects (let's a single column in a table) and dataframes are used for storing tabular data. a Dataframe is collection of series with the same length. Pandas dataframes are similar to Excel spreadsheets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c8e0ff1d-2acf-4ffd-82b8-7be641a62045",
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "628a94d4-a267-40b2-b5fe-f6ef578b8ecd",
   "metadata": {},
   "source": [
    "### Creating Dataframes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c330b95b-5537-4810-85f9-e5c918ece0a8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create Dataframes\n",
    "df = pd.DataFrame({\"name\":[\"Alice\", \"Bob\", \"Craig\", \"Sara\", \"John\"], \"Age\":[26, 34, 22,50, 45], \"experience\":[1, 6, 2, 10, 8],\n",
    "                   \"Title\":[\"IT specialist\", \"Security Expert\", \"Developer\", \"Manager\", \"Scientist\"],\n",
    "                   \"salary\":[60000, 80000, 65000, 110000, 100000]}) \n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cee62b4b-3733-4f76-a333-5b82340738a6",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Creating by loading from files\n",
    "customer_data = pd.read_csv(\"Wholesale customers data.csv\") # UCI dataset: https://archive.ics.uci.edu/ml/datasets/Wholesale+customers\n",
    "customer_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d3a127f-adc9-4a76-b37a-464abaa8c50f",
   "metadata": {},
   "source": [
    "<img src=\"https://img.icons8.com/color/344/light.png\" width=30 alt=\"Hint\"> __Hint:__ Pandas has functions for reading excel, parquet, html, orc and so on. They all have the format of `pd.read_ext` where `ext` is the format. Also, you can write a dataframe to disk using functions like `to_csv` or `to_excel`, ..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee78f8e8-7c5d-4145-bd1c-d40ab1ae0e3d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Writing data back to disk\n",
    "df.to_csv(\"test_dataframe.csv\")\n",
    "customer_data.to_excel(\"Wholesale customers data.xlsx\")  # you need to install openpyxl. conda install -c anaconda openpyxl\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ebc1ad3-c276-4f9f-a4d0-2f82be084003",
   "metadata": {},
   "source": [
    "### Dataframe Properties"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95432986-7498-4b05-95f5-f9bd28fca0e4",
   "metadata": {},
   "outputs": [],
   "source": [
    "end_line = \"\\n\" + '-'*40 + \"\\n\"\n",
    "# Shape\n",
    "print(\"shape: \", customer_data.shape, end=end_line)\n",
    "# Size\n",
    "print(\"size\", customer_data.size, end=end_line)\n",
    "# Column names\n",
    "print(\"columns: \", customer_data.columns, end=end_line)\n",
    "# Data types\n",
    "print(\"data types: \", customer_data.dtypes, end=end_line)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "61c5aa57-2af6-474c-bce0-545643bdb867",
   "metadata": {},
   "source": [
    "### Indexing and Iterating Dataframes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cfa017d8-5f6c-451d-a506-78146acd4f94",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Indexing operator in dataframe\n",
    "customer_data[\"Frozen\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "311368f5-223d-4052-89f6-4c377b9f555b",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(type(customer_data[\"Frozen\"]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "360e5725-0937-4bf0-ae86-00883205344b",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data[0] # This raise an exception since we don't have a column called 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0832e17a-6588-4827-b399-0b7f77998f11",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data[\"Channel\"][0] # If you index a column it returns a series and you can index a series"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49a8d381-af8a-4689-bc37-0ffe4a966693",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.at[0,\"Channel\"] # 2D indexing - row, col -> col has to be column name"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "678413fd-fcdd-4626-8897-d8d08f9cd3d6",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.iat[0, 0] # this is addressing both col and row using numbers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d5ddbf6c-07d2-47e8-891c-11e1a6fd5321",
   "metadata": {},
   "outputs": [],
   "source": [
    "# A more powerful tool to index 2D is loc\n",
    "customer_data.loc[0, \"Channel\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "19ad84eb-3c7e-4bc6-bb86-fb1e2899b53b",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.loc[[1,3,5], [\"Channel\", \"Frozen\"]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60b9f6a1-19a4-4fe1-9060-ec2aacafc21b",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.iloc[0] # This returns row 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fabf2e59-f3ce-4ee1-879e-277c0295e4a3",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.iloc[[0,1,2,3], [0,1,2]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b2682666-3071-4877-90c4-4fc27db1f320",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Selecting based on condition\n",
    "customer_data[customer_data[\"Frozen\"]<40]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "edb032f9-0359-48fd-b73d-8d9d28257ef4",
   "metadata": {},
   "source": [
    "<img src= \"https://img.icons8.com/external-flaticons-flat-flat-icons/344/external-question-100-most-used-icons-flaticons-flat-flat-icons.png\" width=70> __Question__:<br>\n",
    "Guess the output of the following command:<br>\n",
    "<ol><li>\n",
    "    <code>customer_data[customer_data[\"Frozen\"]<40][\"Grocery\"][2]</code> \n",
    "                                            </li>\n",
    "    <li><code>sub_df = (customer_data[customer_data[\"Frozen\"]<40]);sub_df[sub_df[\"Channel\"]==1]</code></li>\n",
    "</ol>                                           "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "806fabe8-f6c6-442a-9935-e6a51264b162",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Iterating over dataframes\n",
    "for row in df.iterrows(): # Iterating over rows as tuples of (index, series)\n",
    "    print(row)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29c3141c-e7e2-47f8-a757-e92876848282",
   "metadata": {},
   "source": [
    "<img src=\"https://img.icons8.com/color/344/light.png\" width=30 alt=\"Hint\"> __Hint:__ There are two similar functions for iterating rows: `iteritems` and `itertuples`"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f351db39-7fcf-46df-a055-d46b8f063aea",
   "metadata": {},
   "source": [
    "#### Adding, Editing and Deleting Columns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c50985b-fc2e-4ea1-b2fd-82904d40ca63",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"Contract\"] = \"Fulltime\"\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e6a5ef8f-780a-412b-bb08-ad3cb656d7c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"Senior\"] = df[\"experience\"] > 5\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3787bcb3-fb4d-4be5-85eb-d3acb034643b",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"IC\"] = df[\"Title\"].apply(lambda x: x !=\"Manager\")\n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "69265ad9-4375-46f0-9bcb-f2acf0ef1377",
   "metadata": {},
   "outputs": [],
   "source": [
    "df[\"SalaryPerExperience\"] = df.apply(lambda row: row[\"salary\"]/row[\"experience\"], axis=1) # don't forget axis = 1 for column-wise \n",
    "df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cbf30a12-b26d-42c5-967b-8e3d52a6cb49",
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.drop([\"IC\"], axis=1)\n",
    "df"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "71f9fe19-1390-4f66-b270-3c10994dad23",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### Data Summarization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "08646636-07b8-4e29-8b22-c980c325b0aa",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.info()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec1178b3-24b8-4fb1-9f05-2b070b38aed5",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.value_counts()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "087f618e-a096-47cb-a61b-79603ac56b4b",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.describe().T"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1153ba1f-b4eb-4efb-87ac-5f9652b0fe09",
   "metadata": {},
   "source": [
    "#### Data Aggregations / Grouping"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cad9e60c-e395-42bb-bec6-cfc756b4acd9",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(customer_data.max(), end=end_line)\n",
    "print(customer_data[[\"Grocery\", \"Frozen\", \"Milk\"]].mean(), end=end_line)\n",
    "# also there are min, max, count, std, var, .... functions on groups"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0ed7ae37-33cf-4763-8d0c-bd083551a63b",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.groupby(\"Region\").count()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b4097531-7b20-460c-96ad-a7eed0515d56",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.groupby(\"Region\")[\"Region\"].count().reset_index(name=\"count\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3b44c233-b44d-453e-bdfb-24f1ee19f7d2",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.groupby([\"Region\", \"Channel\"])[\"Channel\"].count().reset_index(name=\"count\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2a44758b-af2b-4a2c-90d3-4e72c8d69dc9",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.groupby(\"Region\")[[\"Milk\", \"Frozen\"]].agg([\"sum\", \"max\", \"min\", \"mean\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9138faa2-1ade-4d9a-9540-bb39f1331364",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data.groupby(\"Region\")[[\"Milk\", \"Frozen\"]].describe().T"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4beee2e7-f975-40ed-b4bb-11d765a3225a",
   "metadata": {},
   "source": [
    "<img src=\"https://img.icons8.com/color/344/light.png\" width=30 alt=\"Hint\"> __Pandasql:__ you can treat padnas dataframe as a SQL table and run queries on it using padasql library. <br>\n",
    "The library can be installed using the following command:<br>\n",
    "`conda install -c anaconda pandasql` <br>\n",
    "The library supports many of SQL functions, but it is not still supporting everything. Another solution would be using pyspark and converting the dataframe to a pyspark dataframe and running hive queries on it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "49206cfe-da7d-4450-a1c3-1e67febc7490",
   "metadata": {},
   "outputs": [],
   "source": [
    "# pandasql example\n",
    "import pandasql as ps\n",
    "query = \"\"\"\n",
    "SELECT \n",
    "    Region, sum(Frozen) as sum_frozen, sum(Milk) as sum_milk \n",
    "FROM \n",
    "    customer_data \n",
    "GROUP BY\n",
    "    Region\n",
    "\"\"\"\n",
    "ps.sqldf(query, locals())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ec58b2c9-79db-47fa-9607-1cfb7df3b437",
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "2f2e59c4-8c9e-4726-803d-485005f2e917",
   "metadata": {},
   "source": [
    "<br><br>\n",
    "<img alt=\"Matplotlib logo.svg\" src=\"//upload.wikimedia.org/wikipedia/en/thumb/5/56/Matplotlib_logo.svg/300px-Matplotlib_logo.svg.png\" decoding=\"async\" width=\"300\" height=\"55\" srcset=\"//upload.wikimedia.org/wikipedia/en/thumb/5/56/Matplotlib_logo.svg/450px-Matplotlib_logo.svg.png 1.5x, //upload.wikimedia.org/wikipedia/en/thumb/5/56/Matplotlib_logo.svg/600px-Matplotlib_logo.svg.png 2x\" data-file-width=\"540\" data-file-height=\"99\"> &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;\n",
    "\n",
    "### Visualization Using Python\n",
    "Matplotlib is the fundamental visualization library for data science work. It is designed to be compatible with MATLAB and it uses numpy under hood.\n",
    "<br>For installation run the following command: `conda install -c conda-forge matplotlib`\n",
    "<br> There are two important extensions for Matplotlib: __Seaborn__ and __Mplot3d__. Seaborn focus on providing high level API and supports for Pandas and Mplot3d focuses on 3D plotting. Use the following commands to install them:\n",
    "<br> Seaborn: `conda install -c anaconda seaborn`\n",
    "<br> Mplot3D: `conda install -c conda-forge mpld3`\n",
    "\n",
    "<br><br>* For using Matplotlib in jupyter notebooks you need to to call the following magic (before) improting it: <br>`%matplotlib inline`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50e3b88a-f14e-428d-97ce-a217d2ccd0f2",
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import matplotlib as mpl\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e36e4abf-4ea3-4e91-8e33-320924c9562d",
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.linspace(0, 4*np.pi, 500)\n",
    "y = np.sin(2*x)\n",
    "plt.plot(x,y)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bb91504e-23c0-4fc8-8e02-3b918341f775",
   "metadata": {},
   "source": [
    "<table>\n",
    "   <tr>\n",
    "       <th><img src=\"https://img.icons8.com/external-flaticons-lineal-color-flat-icons/344/external-coffee-cup-bakery-flaticons-lineal-color-flat-icons.png\" alt=\"icon\" width=80></th>\n",
    "    <td><img src =\"https://matplotlib.org/stable/_images/anatomy.png\" width=500></td>\n",
    "       <td><h3><h2>Matplotlib Figures' Anatomy</h2> Did you know that you can set every element in a Matplotlib graphs?</h3> <br>You can use the set methods (for example `set_title`) to adjust every piece of it. <br> Check it later!</td>\n",
    "       \n",
    "  </tr>\n",
    "</table>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5fc4baf9-fcc8-4969-9bfe-b4262bcc57fd",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig, ax = plt.subplots(figsize=(10, 8))\n",
    "ax.plot(x, x, label='linear')  # Plot some data on the axes.\n",
    "ax.plot(x, x**2, label='quadratic')  # Plot more data on the axes...\n",
    "ax.plot(x, x**3, label='cubic')  # ... and some more.\n",
    "ax.set_xlabel('x label')  # Add an x-label to the axes.\n",
    "ax.set_ylabel('y label')  # Add a y-label to the axes.\n",
    "ax.set_title(\"Simple Plot\")  # Add a title to the axes.\n",
    "ax.legend();  # Add a legend.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "65e37304-509d-40dd-b04c-30f97c6c1ad7",
   "metadata": {},
   "source": [
    "#### Visualizing Random Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c491e349-d2e0-47ad-a5a3-5c8ea1ae7580",
   "metadata": {},
   "outputs": [],
   "source": [
    "x = np.random.randn(300,1)\n",
    "y = np.random.randn(300,1)\n",
    "fig, ax = plt.subplots(1,4, figsize=(20, 5))\n",
    "ax[0].scatter(x,y)\n",
    "ax[1].hist(x)\n",
    "ax[2].boxplot(x)\n",
    "ax[3].violinplot(x)\n",
    "# Set title for each one\n",
    "ax[0].set_title(\"scatter\")\n",
    "ax[1].set_title(\"hist\")\n",
    "ax[2].set_title(\"boxplot\")\n",
    "_=ax[3].set_title(\"violinplot\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1a1ef473-1a4d-4931-b6ac-f192811dd45f",
   "metadata": {},
   "source": [
    "#### 2D visualization of 3D graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d48a821-3937-489a-a4de-e3211cccd611",
   "metadata": {},
   "source": [
    "<img src=\"https://img.icons8.com/color/344/light.png\" width=30 alt=\"Hint\"> __hint:__ np.meshgrid(x, y) creates the coordinate systems for given values in x and y. It is used for 3D plotting"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fe2478cc-9245-4eb8-8acf-53562cb8451a",
   "metadata": {},
   "outputs": [],
   "source": [
    "# SCHWEFEL FUNCTION\n",
    "x, y = np.meshgrid(np.linspace(-512, 512, 1000), np.linspace(-512, 512, 1000))\n",
    "z = 418.9829*2 - x * np.sin( np.sqrt( abs( x )))-y*np.sin(np.sqrt(abs(y)))\n",
    "\n",
    "minz = z.min()\n",
    "maxz = z.max()\n",
    "\n",
    "fig, axs = plt.subplots(2, 2, figsize=(24,15))\n",
    "\n",
    "# pcolormesh\n",
    "pc = axs[0, 0].pcolormesh(x, y, z, vmin=minz, vmax=maxz, cmap='RdBu_r', shading='auto')\n",
    "fig.colorbar(pc, ax=axs[0, 0])\n",
    "axs[0, 0].set_title('pcolormesh()')\n",
    "\n",
    "# filled Contour\n",
    "co = axs[0, 1].contourf(x, y, z, levels=np.linspace(minz, maxz, 8))\n",
    "fig.colorbar(co, ax=axs[0, 1])\n",
    "axs[0, 1].set_title('contourf()')\n",
    "\n",
    "# As Image\n",
    "pc = axs[1, 0].imshow(z, cmap='plasma', interpolation='nearest')\n",
    "fig.colorbar(pc, ax=axs[1, 0], extend='both')\n",
    "axs[1, 0].set_title('imshow()')\n",
    "\n",
    "# Contour lines\n",
    "co = axs[1, 1].contour(x, y, z, levels=np.linspace(minz, maxz, 8))\n",
    "fig.colorbar(co, ax=axs[1, 1])\n",
    "_=axs[1, 1].set_title('contour()')\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cb9efbc7-e5d7-4563-8c32-3ac1577ad591",
   "metadata": {
    "tags": []
   },
   "source": [
    "#### 3D Visualizations"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fcbaaa47-9e85-44cb-b0b5-87be63b03c15",
   "metadata": {},
   "outputs": [],
   "source": [
    "from mpl_toolkits.mplot3d import axes3d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1b8bc66-97a6-44ca-80cc-effd9a3dfedd",
   "metadata": {},
   "outputs": [],
   "source": [
    "fig = plt.figure(figsize=(24,15))\n",
    "\n",
    "# Wireframe\n",
    "ax = fig.add_subplot(1,2,1, projection='3d')\n",
    "ax.view_init(-140, 60)\n",
    "ax.plot_wireframe(x,y,z, rstride=10, cstride=10)\n",
    "ax.set_title(\"plot_wireframe\")\n",
    "# Surface\n",
    "ax = fig.add_subplot(1,2,2, projection='3d')\n",
    "ax.view_init(-140, 60)\n",
    "ax.plot_surface(x,y,z, cmap= mpl.cm.coolwarm)\n",
    "ax.set_title(\"plot_surface\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b38f068e-87cf-4b26-9257-c4deb25947de",
   "metadata": {},
   "source": [
    "<img src=\"https://img.icons8.com/color/344/light.png\" width=30> __Schwefel function:__ The function that we used in our presentation is called Schwefel function and is a benchmark (test) function for optimization. There are many of these functions and each one designed for a specific scenario. Check here for a comprehensive list of benchmarks. https://en.wikipedia.org/wiki/Test_functions_for_optimization"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1c207bfc-5100-4461-a25e-8f28d19be44c",
   "metadata": {},
   "source": [
    "<img src=\"https://img.icons8.com/external-kosonicon-lineal-color-kosonicon/344/external-lab-tool-back-to-school-kosonicon-lineal-color-kosonicon.png\" alt=\"icon\" width=80  alt=\"Labl\"> __Himmelblau Function__:<br>\n",
    "The function defined by the following equation is called Himmelblau's function\n",
    "<img src=\"https://wikimedia.org/api/rest_v1/media/math/render/svg/7af38b9b0297dba9457f986bf09644df396b2af3\" class=\"mwe-math-fallback-image-inline\" aria-hidden=\"true\" style=\"vertical-align: -1.838ex; width:39.051ex; height:5.676ex;\" alt=\"f(x,y)=2x^{2}-1.05x^{4}+{\\frac {x^{6}}{6}}+xy+y^{2}\">. Its domain is <img src=\"https://wikimedia.org/api/rest_v1/media/math/render/svg/5a3e7bb9757ca235b6171fd392210eeab77c12cc\" class=\"mwe-math-fallback-image-inline\" aria-hidden=\"true\" style=\"vertical-align: -0.671ex; width:13.849ex; height:2.509ex;\" alt=\"-5\\leq x,y\\leq 5\">.\n",
    "\n",
    "<br>Write a program to the followings:\n",
    "<ol>\n",
    "    <li> 3D plot (surface and wireframe) of this function on a 2x1 subplot.</li>\n",
    "    <li> plot contours (lines, filled) of this function for 10 points between its minimum and maximum (of calculated points)</li>\n",
    "    <li> boxplot and histogram of function values</li>\n",
    "</ol>\n",
    "(Approx 10 min - Do it in groups of 3 to 4 people)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3c439775-041f-44cc-bdf7-fa52931e86f1",
   "metadata": {},
   "source": [
    "### Animations\n",
    "Matplotlib supports different types of animations. They are useful for visualizing iterative processes. One category of animations is animating by calling a function iteratively. Let's demonstrate it using an example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b76840b0-afcd-4aee-aef4-d043e8a750c7",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.animation\n",
    "import numpy as np\n",
    "from IPython.display import display, clear_output\n",
    "\n",
    "t = np.linspace(0,2*np.pi)\n",
    "x = np.sin(t)\n",
    "\n",
    "fig, ax = plt.subplots()\n",
    "l, = ax.plot([0,2*np.pi],[-1,1])\n",
    "\n",
    "animate = lambda i: l.set_data(t[:i], x[:i])\n",
    "\n",
    "for i in range(len(x)):\n",
    "    animate(i)\n",
    "    clear_output(wait=True) # this and next line are needed for working in Jupyter env.\n",
    "    display(fig)\n",
    "clear_output(wait=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fbecf348-9b76-45f3-8a9f-0ad20f1b3635",
   "metadata": {},
   "source": [
    " <img src=\"https://seaborn.pydata.org/_static/logo-wide-lightbg.svg\" width=200><br>\n",
    " Seaborn is an extension to MPL which adds more styling also API for working with pandas dataframes.\n",
    " <br> Conda installation: `conda install -c anaconda seaborn`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2db819aa-8fb0-4e5e-a89e-dd8b947f534c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import seaborn as sns"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bba036a0-c83d-433c-b24b-08db46ece697",
   "metadata": {},
   "outputs": [],
   "source": [
    "y = np.random.randn(1000)\n",
    "x = np.linspace(1,1000,1000)\n",
    "sns.lineplot(x=x, y=y) # Nothing new, this is similar to MPL"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "91e8ebd9-dc6f-4669-9743-46eda1baec54",
   "metadata": {},
   "outputs": [],
   "source": [
    "# But it has interface for pandas too. Let's make an example\n",
    "df = pd.DataFrame({\"x\":x, \"y\":y})\n",
    "sns.lineplot(data=df, x=x, y=y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e1653491-e0f2-4710-bbd3-becf73a5f9fb",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.scatterplot(data=customer_data, x=\"Frozen\", y=\"Fresh\", hue=\"Region\", size=\"Channel\").set_xlim([0., 10000])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d5bb08d-d7b9-4c02-8e9c-37d21f0c45ce",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.set_theme(style=\"ticks\", palette=\"pastel\")\n",
    "sns.boxplot(data=customer_data, x=\"Region\", y=\"Frozen\", hue=\"Channel\", showfliers=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e7b9f881-c861-4894-bec6-5482824212b9",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.set_theme(style=\"darkgrid\")\n",
    "sns.displot(\n",
    "    customer_data, x=\"Grocery\" , col=\"Region\", row=\"Channel\")\n",
    "#    binwidth=3, height=3, facet_kws=dict(margin_titles=True),)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "1f2686cb-cda9-4032-8376-92983fd77f83",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.pairplot(customer_data[[\"Milk\", \"Frozen\", \"Fresh\", \"Grocery\"]])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "710488fc-7962-4aa1-902e-1eebbc558d17",
   "metadata": {},
   "outputs": [],
   "source": [
    "customer_data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ff3147b-eb60-4642-8d63-e26d28822efc",
   "metadata": {},
   "source": [
    "<img src=\"https://img.icons8.com/external-kosonicon-lineal-color-kosonicon/344/external-lab-tool-back-to-school-kosonicon-lineal-color-kosonicon.png\" alt=\"icon\" width=80  alt=\"Labl\"> __Online Retail Data Visualization__:<br>\n",
    "The file `Online_Retail.xlsx` contains online retail information (UCI data: https://archive.ics.uci.edu/ml/datasets/Online+Retail#). It has the following fields:\n",
    "<ul>\n",
    "<li>\n",
    "    <b>InvoiceNo:</b> Invoice number. Nominal, a 6-digit integral number uniquely assigned to each transaction. If this code starts with letter 'c', it indicates a cancellation. </li>\n",
    "    <li><b>StockCode:</b> Product (item) code. Nominal, a 5-digit integral number uniquely assigned to each distinct product.\n",
    "    <li><b>Description:</b> Product (item) name. Nominal. </li>\n",
    "    <li><b>Quantity:</b> The quantities of each product (item) per transaction. Numeric.</li>\n",
    "<li><b>InvoiceDate:</b> Invice Date and time. Numeric, the day and time when each transaction was generated.</li>\n",
    "    <li><b>UnitPrice:</b> Unit price. Numeric, Product price per unit in sterling.</li>\n",
    "    <li><b>CustomerID:</b> Customer number. Nominal, a 5-digit integral number uniquely assigned to each customer.</li>\n",
    "    <li><b>Country:</b> Country name. Nominal, the name of the country where each customer resides.</li>\n",
    "</ul>\n",
    "\n",
    "<br>__Hint:__ for converting a column of string to date use the following format for example `online_retail[\"InvoiceDate\"].dt` like  `online_retail[\"InvoiceDate\"].dt.to_pydatetime()`.\n",
    "<br><br>\n",
    "Load data and based on what you have learned about seaborn and pandas write a program to the followings:\n",
    "\n",
    "<ol>\n",
    "    <li> Explore the dataset with pandas describe and info functions</li>\n",
    "    <li> Find and print the most popular product per country</li>\n",
    "    <li>Plot the distribution of prices for the following countries (France, Germany, EIRE)</li>\n",
    "    <li> Plot number of transactions per day for year of 2010 and for United Kingdom </li>\n",
    "    \n",
    "</ol>"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}