Практическое_занятие_2 803-804.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Практическое задание № 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Задание:\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Шаг 1. Загрузить данные с файла расширения csv (adult.data.csv)\n",
    "    - Изучить содержание признаков\n",
    "    - Определить количествро строк, колонок\n",
    "    \n",
    "* Шаг 2. Провести очистку данных\n",
    "    - Выполнить проверку на пустые значения\n",
    "    - Определить типы данных \n",
    "    - Поиск дубликатов\n",
    "    - Поиск аномалий   \n",
    "* Шаг 3. С помощью библиотеки Pandas ответить на несколько вопросов по данным набора Adult по доходу населения, каждый объект которого содержит социальные характеристики некоторого человека (возраст, пол, профессиональная деятельность и т.п.): \n",
    "    - **1. Сколько неженатых граждан** (признак *relationship* = Unmarried) **представлено в этом наборе данных?**\n",
    "    - **2. Каков средний возраст** (признак *age* ) **женщин** (признак *sex* = Female)), **которые зарабатывают много?** ( признак *salary* = >50K)\n",
    "    - **3. Какое наибольшее количество часов человек работает в неделю?** (признак *hours-per-week*)\n",
    "    - **4. Сколько граждан  много зарабатывают** (признак *salary* = >50K), **которые закончили 9 классов**?  (признак *education* = 9th)\n",
    "    - **5. Посчитайте среднее время работы** (признак *hours-per-week*) **зарабатывающих мало и много** (признак *salary*). Используйте метод pivot_table\n",
    "* Шаг 4. Выполните запись результатов в файл с разделителями.\n",
    "* Шаг 5. Опубликуйте файл расширения ipynb на платформу Odin.\t"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Уникальные значения признаков набора данных по доходу населения:\n",
    "\n",
    "______________\n",
    "\n",
    "Признак         | Описание | Значения\n",
    "----------------|----------| -------------\n",
    "age             | возраст | количественный \n",
    "workclass       | вид работы | <code>Private</code>, <code>Self-emp-not-inc</code>, <code>Self-emp-inc</code>, <code>Federal-gov</code>, <code>Local-gov</code>, <code>State-gov</code>, <code>Without-pay</code>, <code>Never-worked</code>\n",
    "fnlwgt          | вес кейса | количественный\n",
    "education       | уровень образования | <code>Bachelors</code>, <code>Some-college</code>, <code>11th</code>, <code>HS-grad</code>, <code>Prof-school</code>, <code>Assoc-acdm</code>, <code>Assoc-voc</code>, <code>9th</code>, <code>7th-8th</code>, <code>12th</code>, <code>Masters</code>, <code>1st-4th</code>, <code>10th</code>, <code>Doctorate</code>, <code>5th-6th</code>, <code>Preschool</code>\n",
    "education-num   | код уровня образования | количественный\n",
    "marital-status  | статус гражданина | <code>Married-civ-spouse</code>, <code>Divorced</code>, <code>Never-married</code>, <code>Separated</code>, <code>Widowed</code>, <code>Married-spouse-absent</code>, <code>Married-AF-spouse</code>\n",
    "occupation      | род занятий | <code>Tech-support</code>, <code>Craft-repair</code>, <code>Other-service</code>, <code>Sales</code>, <code>Exec-managerial</code>, <code>Prof-specialty</code>, <code>Handlers-cleaners</code>, <code>Machine-op-inspct</code>, <code>Adm-clerical</code>, <code>Farming-fishing</code>, <code>Transport-moving</code>, <code>Priv-house-serv</code>, <code>Protective-serv</code>, <code>Armed-Forces</code>\n",
    "relationship    | Состав семьи |  <code>Wife</code>, <code>Own-child</code>, <code>Husband</code>, <code>Not-in-family</code>, <code>Other-relative</code>, <code>Unmarried</code>\n",
    "race            | расса |  <code>White</code>, <code>Asian-Pac-Islander</code>, <code>Amer-Indian-Eskimo</code>, <code>Other</code>, <code>Black</code>\n",
    "sex             | пол |  <code>Female</code>, <code>Male</code>\n",
    "capital-gain    | исключим из анализа |  количественный\n",
    "capital-loss    | исключим из анализа |  количественный\n",
    "hours-per-week  | количество часов в неделю |  количественный\n",
    "native-country  | страна |  <code>United-States</code>, <code>Cambodia</code>, <code>England</code>, <code>Puerto-Rico</code>, <code>Canada</code>, <code>Germany</code>, <code>Outlying-US(Guam-USVI-etc)</code>, <code>India</code>, <code>Japan</code>, <code>Greece</code>, <code>South</code>, <code>China</code>, <code>Cuba</code>, <code>Iran</code>, <code>Honduras</code>, <code>Philippines</code>, <code>Italy</code>, <code>Poland</code>, <code>Jamaica</code>, <code>Vietnam</code>, <code>Mexico</code>, <code>Portugal</code>, <code>Ireland</code>, <code>France</code>, <code>Dominican-Republic</code>, <code>Laos</code>, <code>Ecuador</code>, <code>Taiwan</code>, <code>Haiti</code>, <code>Columbia</code>, <code>Hungary</code>, <code>Guatemala</code>, <code>Nicaragua</code>, <code>Scotland</code>, <code>Thailand</code>, <code>Yugoslavia</code>, <code>El-Salvador</code>, <code>Trinadad&amp;Tobago</code>, <code>Peru</code>, <code>Hong</code>, <code>Holand-Netherlands</code>\n",
    "salary          | уровень дохода |  <code>&gt;50K</code>, <code>&lt;=50K</code>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1.23.3\n",
      "1.4.4\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "pd.set_option(\"display.precision\", 1)\n",
    "\n",
    "print (np.__version__)\n",
    "print (pd.__version__)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "CAT_OS = 'https://github.com/OlesiaAngel/DataAnalitics/blob/main/%D0%B4%D0%B0%D0%BD%D0%BD%D1%8B%D0%B5/'\n",
    "df = pd.read_csv(CAT_OS+'adult.data.csv?raw=True')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "scrolled": true
   },
   "source": [
    "* Шаг 1. Загрузить данные с файла расширения csv (adult.data.csv)\n",
    "    - Изучить содержание признаков"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "RangeIndex(start=0, stop=32561, step=1)"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.index # изучаю порядок индексации файла (Шаг 1.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>workclass</th>\n",
       "      <th>fnlwgt</th>\n",
       "      <th>education</th>\n",
       "      <th>education-num</th>\n",
       "      <th>marital-status</th>\n",
       "      <th>occupation</th>\n",
       "      <th>relationship</th>\n",
       "      <th>race</th>\n",
       "      <th>sex</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "      <th>native-country</th>\n",
       "      <th>salary</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39</td>\n",
       "      <td>State-gov</td>\n",
       "      <td>77516</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Never-married</td>\n",
       "      <td>Adm-clerical</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>2174</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>Self-emp-not-inc</td>\n",
       "      <td>83311</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Exec-managerial</td>\n",
       "      <td>Husband</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38</td>\n",
       "      <td>Private</td>\n",
       "      <td>215646</td>\n",
       "      <td>HS-grad</td>\n",
       "      <td>9</td>\n",
       "      <td>Divorced</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Not-in-family</td>\n",
       "      <td>White</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53</td>\n",
       "      <td>Private</td>\n",
       "      <td>234721</td>\n",
       "      <td>11th</td>\n",
       "      <td>7</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Handlers-cleaners</td>\n",
       "      <td>Husband</td>\n",
       "      <td>Black</td>\n",
       "      <td>Male</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>United-States</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28</td>\n",
       "      <td>Private</td>\n",
       "      <td>338409</td>\n",
       "      <td>Bachelors</td>\n",
       "      <td>13</td>\n",
       "      <td>Married-civ-spouse</td>\n",
       "      <td>Prof-specialty</td>\n",
       "      <td>Wife</td>\n",
       "      <td>Black</td>\n",
       "      <td>Female</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "      <td>Cuba</td>\n",
       "      <td>&lt;=50K</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   age         workclass  fnlwgt  education  education-num  \\\n",
       "0   39         State-gov   77516  Bachelors             13   \n",
       "1   50  Self-emp-not-inc   83311  Bachelors             13   \n",
       "2   38           Private  215646    HS-grad              9   \n",
       "3   53           Private  234721       11th              7   \n",
       "4   28           Private  338409  Bachelors             13   \n",
       "\n",
       "       marital-status         occupation   relationship   race     sex  \\\n",
       "0       Never-married       Adm-clerical  Not-in-family  White    Male   \n",
       "1  Married-civ-spouse    Exec-managerial        Husband  White    Male   \n",
       "2            Divorced  Handlers-cleaners  Not-in-family  White    Male   \n",
       "3  Married-civ-spouse  Handlers-cleaners        Husband  Black    Male   \n",
       "4  Married-civ-spouse     Prof-specialty           Wife  Black  Female   \n",
       "\n",
       "   capital-gain  capital-loss  hours-per-week native-country salary  \n",
       "0          2174             0              40  United-States  <=50K  \n",
       "1             0             0              13  United-States  <=50K  \n",
       "2             0             0              40  United-States  <=50K  \n",
       "3             0             0              40  United-States  <=50K  \n",
       "4             0             0              40           Cuba  <=50K  "
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.head() # изучил первые 5 строк (Шаг 1.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Index(['age', 'workclass', 'fnlwgt', 'education', 'education-num',\n",
       "       'marital-status', 'occupation', 'relationship', 'race', 'sex',\n",
       "       'capital-gain', 'capital-loss', 'hours-per-week', 'native-country',\n",
       "       'salary'],\n",
       "      dtype='object')"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.columns # изучаю имена серий (Шаг 1.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>fnlwgt</th>\n",
       "      <th>education-num</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>count</th>\n",
       "      <td>32561.0</td>\n",
       "      <td>3.3e+04</td>\n",
       "      <td>32561.0</td>\n",
       "      <td>32561.0</td>\n",
       "      <td>32561.0</td>\n",
       "      <td>32561.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>mean</th>\n",
       "      <td>38.6</td>\n",
       "      <td>1.9e+05</td>\n",
       "      <td>10.1</td>\n",
       "      <td>1077.6</td>\n",
       "      <td>87.3</td>\n",
       "      <td>40.4</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>std</th>\n",
       "      <td>13.6</td>\n",
       "      <td>1.1e+05</td>\n",
       "      <td>2.6</td>\n",
       "      <td>7385.3</td>\n",
       "      <td>403.0</td>\n",
       "      <td>12.3</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>min</th>\n",
       "      <td>17.0</td>\n",
       "      <td>1.2e+04</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>25%</th>\n",
       "      <td>28.0</td>\n",
       "      <td>1.2e+05</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>50%</th>\n",
       "      <td>37.0</td>\n",
       "      <td>1.8e+05</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>40.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>75%</th>\n",
       "      <td>48.0</td>\n",
       "      <td>2.4e+05</td>\n",
       "      <td>12.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>45.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>max</th>\n",
       "      <td>90.0</td>\n",
       "      <td>1.5e+06</td>\n",
       "      <td>16.0</td>\n",
       "      <td>99999.0</td>\n",
       "      <td>4356.0</td>\n",
       "      <td>99.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "           age   fnlwgt  education-num  capital-gain  capital-loss  \\\n",
       "count  32561.0  3.3e+04        32561.0       32561.0       32561.0   \n",
       "mean      38.6  1.9e+05           10.1        1077.6          87.3   \n",
       "std       13.6  1.1e+05            2.6        7385.3         403.0   \n",
       "min       17.0  1.2e+04            1.0           0.0           0.0   \n",
       "25%       28.0  1.2e+05            9.0           0.0           0.0   \n",
       "50%       37.0  1.8e+05           10.0           0.0           0.0   \n",
       "75%       48.0  2.4e+05           12.0           0.0           0.0   \n",
       "max       90.0  1.5e+06           16.0       99999.0        4356.0   \n",
       "\n",
       "       hours-per-week  \n",
       "count         32561.0  \n",
       "mean             40.4  \n",
       "std              12.3  \n",
       "min               1.0  \n",
       "25%              40.0  \n",
       "50%              40.0  \n",
       "75%              45.0  \n",
       "max              99.0  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.describe() # изучаю серии с числовыми значениями (Шаг 1.1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>age</th>\n",
       "      <th>fnlwgt</th>\n",
       "      <th>education-num</th>\n",
       "      <th>capital-gain</th>\n",
       "      <th>capital-loss</th>\n",
       "      <th>hours-per-week</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>39</td>\n",
       "      <td>77516</td>\n",
       "      <td>13</td>\n",
       "      <td>2174</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>50</td>\n",
       "      <td>83311</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>13</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>38</td>\n",
       "      <td>215646</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>53</td>\n",
       "      <td>234721</td>\n",
       "      <td>7</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>28</td>\n",
       "      <td>338409</td>\n",
       "      <td>13</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>...</th>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "      <td>...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32556</th>\n",
       "      <td>27</td>\n",
       "      <td>257302</td>\n",
       "      <td>12</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>38</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32557</th>\n",
       "      <td>40</td>\n",
       "      <td>154374</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32558</th>\n",
       "      <td>58</td>\n",
       "      <td>151910</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32559</th>\n",
       "      <td>22</td>\n",
       "      <td>201490</td>\n",
       "      <td>9</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>20</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>32560</th>\n",
       "      <td>52</td>\n",
       "      <td>287927</td>\n",
       "      <td>9</td>\n",
       "      <td>15024</td>\n",
       "      <td>0</td>\n",
       "      <td>40</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "<p>32561 rows × 6 columns</p>\n",
       "</div>"
      ],
      "text/plain": [
       "       age  fnlwgt  education-num  capital-gain  capital-loss  hours-per-week\n",
       "0       39   77516             13          2174             0              40\n",
       "1       50   83311             13             0             0              13\n",
       "2       38  215646              9             0             0              40\n",
       "3       53  234721              7             0             0              40\n",
       "4       28  338409             13             0             0              40\n",
       "...    ...     ...            ...           ...           ...             ...\n",
       "32556   27  257302             12             0             0              38\n",
       "32557   40  154374              9             0             0              40\n",
       "32558   58  151910              9             0             0              40\n",
       "32559   22  201490              9             0             0              20\n",
       "32560   52  287927              9         15024             0              40\n",
       "\n",
       "[32561 rows x 6 columns]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.select_dtypes(include=['float64', 'int64'])  # узнать все числовые серии данных фрейма (Шаг 1.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " **- Определить количество строк, колонок**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "32561"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape[0] # определяю общее кол-во строк в файле (Шаг 1.2 из задания) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "15"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.shape[1] # определяю общее кол-во колонок в файле (Шаг 1.2 из задания) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Шаг 2. Провести очистку данных**\n",
    "    -**Выполнить проверку на пустые значения**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age               0\n",
       "workclass         0\n",
       "fnlwgt            0\n",
       "education         0\n",
       "education-num     0\n",
       "marital-status    0\n",
       "occupation        0\n",
       "relationship      0\n",
       "race              0\n",
       "sex               0\n",
       "capital-gain      0\n",
       "capital-loss      0\n",
       "hours-per-week    0\n",
       "native-country    0\n",
       "salary            0\n",
       "dtype: int64"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum() # Проверка на пустые значения (Шаг 2.1) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age               0.0\n",
       "workclass         0.0\n",
       "fnlwgt            0.0\n",
       "education         0.0\n",
       "education-num     0.0\n",
       "marital-status    0.0\n",
       "occupation        0.0\n",
       "relationship      0.0\n",
       "race              0.0\n",
       "sex               0.0\n",
       "capital-gain      0.0\n",
       "capital-loss      0.0\n",
       "hours-per-week    0.0\n",
       "native-country    0.0\n",
       "salary            0.0\n",
       "dtype: float64"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.isnull().sum() / df.shape[0] * 100   # Процент пропущенных значений (Шаг 2.1) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**- Определить типы данных** "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "<class 'pandas.core.frame.DataFrame'>\n",
      "RangeIndex: 32561 entries, 0 to 32560\n",
      "Data columns (total 15 columns):\n",
      " #   Column          Non-Null Count  Dtype \n",
      "---  ------          --------------  ----- \n",
      " 0   age             32561 non-null  int64 \n",
      " 1   workclass       32561 non-null  object\n",
      " 2   fnlwgt          32561 non-null  int64 \n",
      " 3   education       32561 non-null  object\n",
      " 4   education-num   32561 non-null  int64 \n",
      " 5   marital-status  32561 non-null  object\n",
      " 6   occupation      32561 non-null  object\n",
      " 7   relationship    32561 non-null  object\n",
      " 8   race            32561 non-null  object\n",
      " 9   sex             32561 non-null  object\n",
      " 10  capital-gain    32561 non-null  int64 \n",
      " 11  capital-loss    32561 non-null  int64 \n",
      " 12  hours-per-week  32561 non-null  int64 \n",
      " 13  native-country  32561 non-null  object\n",
      " 14  salary          32561 non-null  object\n",
      "dtypes: int64(6), object(9)\n",
      "memory usage: 3.7+ MB\n"
     ]
    }
   ],
   "source": [
    "df.info() # общее инфо # (Шаг 2.2) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age                int64\n",
       "workclass         object\n",
       "fnlwgt             int64\n",
       "education         object\n",
       "education-num      int64\n",
       "marital-status    object\n",
       "occupation        object\n",
       "relationship      object\n",
       "race              object\n",
       "sex               object\n",
       "capital-gain       int64\n",
       "capital-loss       int64\n",
       "hours-per-week     int64\n",
       "native-country    object\n",
       "salary            object\n",
       "dtype: object"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.dtypes # узнать типы данных по сериям (Шаг 2.2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " **- Поиск дубликатов**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "24"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.duplicated().sum() # Поиск дубликатов (Шаг 2.3) - 24 строки индентичны"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(32537, 15)"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df = df.drop_duplicates() # удаление дублей (Шаг 2.3) \n",
    "df.shape      #  проверка очистки фрейма от дублей (Шаг 2.3)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " **- Поиск аномалий**  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "age                  73\n",
       "workclass             9\n",
       "fnlwgt            21648\n",
       "education            16\n",
       "education-num        16\n",
       "marital-status        7\n",
       "occupation           15\n",
       "relationship          6\n",
       "race                  5\n",
       "sex                   2\n",
       "capital-gain        119\n",
       "capital-loss         92\n",
       "hours-per-week       94\n",
       "native-country       42\n",
       "salary                2\n",
       "dtype: int64"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df.nunique()  # сравнил количества по каждой серии с исходными данными + выявил кол-во уникальных значений (Шаг 2.4) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['State-gov', 'Self-emp-not-inc', 'Private', 'Federal-gov',\n",
       "       'Local-gov', '?', 'Self-emp-inc', 'Without-pay', 'Never-worked'],\n",
       "      dtype=object)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['workclass'].unique()   # Найдена следующая аномалия в данной серии - \"?\" (Шаг 2.4) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['Adm-clerical', 'Exec-managerial', 'Handlers-cleaners',\n",
       "       'Prof-specialty', 'Other-service', 'Sales', 'Craft-repair',\n",
       "       'Transport-moving', 'Farming-fishing', 'Machine-op-inspct',\n",
       "       'Tech-support', '?', 'Protective-serv', 'Armed-Forces',\n",
       "       'Priv-house-serv'], dtype=object)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['occupation'].unique()  # Найдена следующая аномалия в данной серии - \"?\" (Шаг 2.4) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array(['United-States', 'Cuba', 'Jamaica', 'India', '?', 'Mexico',\n",
       "       'South', 'Puerto-Rico', 'Honduras', 'England', 'Canada', 'Germany',\n",
       "       'Iran', 'Philippines', 'Italy', 'Poland', 'Columbia', 'Cambodia',\n",
       "       'Thailand', 'Ecuador', 'Laos', 'Taiwan', 'Haiti', 'Portugal',\n",
       "       'Dominican-Republic', 'El-Salvador', 'France', 'Guatemala',\n",
       "       'China', 'Japan', 'Yugoslavia', 'Peru',\n",
       "       'Outlying-US(Guam-USVI-etc)', 'Scotland', 'Trinadad&Tobago',\n",
       "       'Greece', 'Nicaragua', 'Vietnam', 'Hong', 'Ireland', 'Hungary',\n",
       "       'Holand-Netherlands'], dtype=object)"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['native-country'].unique()  # Найдена следующая аномалия в данной серии - \"?\" (Шаг 2.4) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [],
   "source": [
    "df = df.replace({'?': np.nan}) # заменил все значения '?' на 'NaN'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Шаг 3. С помощью библиотеки Pandas ответить на несколько вопросов по данным набора Adult по доходу населения, каждый объект которого содержит социальные характеристики некоторого человека (возраст, пол, профессиональная деятельность и т.п.):\n",
    "    - **1. Сколько неженатых граждан** (признак *relationship* = Unmarried) **представлено в этом наборе данных?**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3445"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df['relationship'].value_counts()['Unmarried']  # определяю количество неженатых граждан (Шаг 3.1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " - **2. Каков средний возраст** (признак *age* ) **женщин** (признак *sex* = Female)), **которые зарабатывают много?** ( признак *salary* = >50K)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "42.125530110262936"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "reach_woman = df[(df['sex'] == 'Female') & (df['salary'] == '>50K')]  # Вычислил средний возраст богатых женщин (Шаг 3.2)\n",
    "reach_woman.age.mean()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **3. Какое наибольшее количество часов человек работает в неделю?** (признак *hours-per-week*)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "99"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "df_max_work_hours = df['hours-per-week'].max() # получилось 14 часов в день, тоже аномалия (Шаг 3.3)\n",
    "df_max_work_hours"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **4. Сколько граждан  много зарабатывают** (признак *salary* = >50K), **которые закончили 9 классов**?  (признак *education* = 9th)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "27"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "rich_with_secondary_education = df[(df['education'] == '9th') & (df['salary'] == '>50K')] # (Шаг 3.4)\n",
    "rich_with_secondary_education.shape[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **5. Посчитайте среднее время работы** (признак *hours-per-week*) **зарабатывающих мало и много** (признак *salary*). Используйте метод pivot_table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "38.84286177018382"
      ]
     },
     "execution_count": 25,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "people_with_low_salary = df[df['salary'] == '<=50K']['hours-per-week'].mean() # Среднее время работы, зарабатывающих мало (Шаг 3.5)\n",
    "people_with_low_salary"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "45.473402219670874"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "people_with_high_salary = df[df['salary'] == '>50K']['hours-per-week'].mean()  # Среднее время работы, зарабатывающих много (Шаг 3.5)\n",
    "people_with_high_salary "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>salary</th>\n",
       "      <th>hours-per-week</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>&lt;=50K</td>\n",
       "      <td>38.8</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>&gt;50K</td>\n",
       "      <td>45.5</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  salary  hours-per-week\n",
       "0  <=50K            38.8\n",
       "1   >50K            45.5"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "average_timework = df.pivot_table(             # Среднее время работы, зарабатывающих много и мало(Шаг 3.5)\n",
    "    ['hours-per-week'],\n",
    "    ['salary'],\n",
    "    aggfunc='mean')\n",
    "average_timework.reset_index()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "* Шаг 4. Выполните запись результатов в файл с разделителями."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.to_csv('practice#2.csv', sep='|')   # Запись данных в файл с разделителями (Шаг 4)"
   ]
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}