O'Reilly 07July 20.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "6e582532-511e-43b8-9c57-876e8f3932eb",
   "metadata": {},
   "source": [
    "# Agenda, week 3: Dictionaries and files\n",
    "\n",
    "1. Recap (loops, lists, and tuples) + Q&A\n",
    "2. Dictionaries\n",
    "    - Defining them\n",
    "    - Retrieving from them\n",
    "    - Assigning to them / modifying them\n",
    "    - Accumulating in dictionaries\n",
    "    - Accumulating the unknown\n",
    "    - Looping over dicts\n",
    "    - How do dicts work?\n",
    "3. Files\n",
    "    - Reading from text files\n",
    "    - Looping over text files\n",
    "    - Writing to files and the `with` construct\n",
    "  \n",
    "Download files from : https://files.lerner.co.il/exercise-files.zip"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97d5c258-5193-4dea-a9e6-120f24037f84",
   "metadata": {},
   "source": [
    "# Extracting a value from a list in a list\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "8c428245-7f44-460c-ba04-e029cb005473",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "20"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mylist = [10, 20, 30]\n",
    "\n",
    "# I want the item at index 1, aka 20\n",
    "mylist[1]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "c7250819-3b9e-494a-ba78-bbe4440a9579",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[40, 50, 60]"
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mylist = [[10, 20, 30], [40, 50, 60], [70, 80, 90]]\n",
    "\n",
    "# this is a list of 3 elements\n",
    "# each of those elements is itself a list of 3 integers\n",
    "\n",
    "mylist[1]   # what will this return? A list -- [40, 50, 60]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "803f842e-61fe-4eff-884b-4c90f3943ad8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "50"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# How can we retrieve 50 from that inner list?\n",
    "\n",
    "inner_list = mylist[1]  # will this work?\n",
    "inner_list[1]    # get index 1 from inner_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "ada3e634-f673-4f5e-8720-ef6576a44ed1",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "40"
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "inner_list[0]  # get index 0 from inner_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "4b45576e-a369-46fb-b4e0-4ba7d2b3f36d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "40"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# now, let's do that WITHOUT another variable\n",
    "# we'll get index 0 from the list at mylist[1]\n",
    "\n",
    "mylist[1][0]   # read the expression from left to right -- mylist, get its element [1]\n",
    "               # that returns [40, 50, 60].  Apply [0] to that list, and we get 40"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a18244cc-c4b0-461a-b8e0-99b43b90ccbf",
   "metadata": {},
   "source": [
    "# Data structures so far\n",
    "\n",
    "So far, we've talked about some simple data structures:\n",
    "\n",
    "- Integers and floats\n",
    "- Booleans (`True` and `False`)\n",
    "\n",
    "Then we discussed three types of data which are *sequences*:\n",
    "\n",
    "- Strings (immutable, and contain characters)\n",
    "- Lists (mutable, and contain anything -- but traditionally, all elements are of the same type)\n",
    "- Tuples (immutable and contain anything -- but traditionally, elements are of different types)\n",
    "\n",
    "We saw that with all sequences, we can run a `for` loop or we can use `in` to search in the sequence.\n",
    "\n",
    "Two big questions:\n",
    "1. How long does it take to run that search for an element in a sequence (string, list, tuple)?\n",
    "2. Wouldn't it be nice if we could use something other than integers (0, 1, 2, etc.) as indexes in our sequences?\n",
    "\n",
    "Right now, we're able to store lots of interesting and important data in a list, but the indexes are 0, 1, 2, etc.  Which means that if I'm storing information about different people, and I have an ID number (SSN or national ID or employee ID) that uniquely identifies the person, I can't use that ID number easily as the index in my list. Rather, I have to use 0, 1, 2, 3, etc.\n",
    "\n",
    "Dictionaries are especially useful because they let us use *anything* as the key (which is the term we use in dicts instead of \"index\"). We can use ID numbers as our indexes (aka our keys). We can use names (yes, strings!) as our keys. We can actually use anything for a key that is immutable -- basically meaning numbers and strings, but theoretically anything immutable.\n",
    "\n",
    "# Dicts in a nutshell\n",
    "\n",
    "A dictionary contains key-value pairs.  Dicts exist in many programming languages, not just Python, but often with other names:\n",
    "\n",
    "- Hash tables\n",
    "- Hashes\n",
    "- Associative arrays\n",
    "- Key-value stores\n",
    "- Name-value stores\n",
    "- Hashmaps\n",
    "- Maps\n",
    "\n",
    "Every key has one value, and every value has one key. There is no such thing, in a dict, of a key without a value or a value without a key.\n",
    "\n",
    "Keys are guaranteed to be unique -- they cannot repeat -- and must be immutable (again, basically numbers or strings).\n",
    "\n",
    "Values can be absolutely anything at all in Python.\n",
    "\n",
    "In this way, we have amazing flexibility when choosing our keys and values. Often, the keys will be a unique index into our data set."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "874bfbbe-fd6c-428e-b4b2-4f38a9317fca",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's define a dict!\n",
    "\n",
    "# 1. We use curly braces to define the dictionary\n",
    "# 2. Each key-value pair has a : between the key and the value\n",
    "# 3. We put commas between each key-value pair\n",
    "# 4. Keys can be anything immutable, and values can be anything at all\n",
    "# 5. Values can repeat if you want, but keys cannot.  Keys are guaranteed to be unique.\n",
    "\n",
    "d = {'a':10, 'b':20, 'c':30}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "d5d2fce3-0b01-45b2-bd93-801bbe1970ac",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(d)   # what kind of data do we have here?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "eaaea113-d8ce-4359-9a93-23bb2728ef5d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# How do I retrieve from a dict?\n",
    "# Just like with a string, list, or tuple with [] and the key we want inside of them\n",
    "\n",
    "d['a']   # retrieve the value with the key 'a'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "d5cf4325-063e-4b53-8463-757c4caa1338",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "k = 'a'\n",
    "d[k]    # will this work, using a variable that refers to a string?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "669a73c6-b4b5-4980-bbdd-59790769b3cd",
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'q'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "Cell \u001b[0;32mIn [14], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[38;5;66;03m# what if I try to retrieve the value for a key that doesn't exist?\u001b[39;00m\n\u001b[0;32m----> 2\u001b[0m \u001b[43md\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mq\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\n",
      "\u001b[0;31mKeyError\u001b[0m: 'q'"
     ]
    }
   ],
   "source": [
    "# what if I try to retrieve the value for a key that doesn't exist?\n",
    "d['q']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "48a201c0-5a87-4963-90bf-a6acbafce7cd",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10\n"
     ]
    }
   ],
   "source": [
    "# we can avoid these problems if we first check whether a key is in the dict\n",
    "# we can do that with the \"in\" operator\n",
    "\n",
    "if 'a' in d:       # 'in' only checks the keys, not the values. \n",
    "    print(d['a'])\n",
    "else:\n",
    "    print(f'a is not a key in d')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "454212ea-a7a7-4ec1-8689-40955b292e52",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "q is not a key in d\n"
     ]
    }
   ],
   "source": [
    "if 'q' in d:       # 'in' only checks the keys, not the values. \n",
    "    print(d['q'])\n",
    "else:\n",
    "    print(f'q is not a key in d')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ca1eedfa-c623-4bc9-9624-3da9361f45b3",
   "metadata": {},
   "source": [
    "# Variable names and value types\n",
    "\n",
    "You can give ANY NAME YOU WANT to ANY VALUE YOU WANT in Python.  No restrictions there, aside from reserved words.\n",
    "\n",
    "I typically use `t` for tuples and `d` for dictionaries.  This is just my lazy teaching convention. You should normally aim to use longer names for all of your variables that make your code clearer.\n",
    "\n",
    "Python couldn't care less what you call your variables. The names are for *you* and anyone who will be reading/maintaining your code."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9b83fe72-0e58-45a1-bc7e-34197d018d5f",
   "metadata": {},
   "source": [
    "# Paradigms for dict use\n",
    "\n",
    "I like to talk about three paradigms for dictionary use. We're going to use paradigm #1 in this exercise, namely creating a dict and using it as a read-only database in our program.\n",
    "\n",
    "We'll define it at the top of the program, then use it, but never modify it."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b20c56cc-740d-45df-9db3-12ff1c0429ab",
   "metadata": {},
   "source": [
    "# Exercise: Restaurant\n",
    "\n",
    "1. Define a dict called `menu` in which the keys are strings representing items on a restaurant menu, and the values are the prices of those items.\n",
    "2. Define `total` to be 0.\n",
    "3. Ask the user, repeatedly, to order something.\n",
    "    - If they give you the empty string, then stop asking and print the total.\n",
    "    - If they give you the name of something on the menu, then print the price and the new total. (And don't forget to add the price to the total.)\n",
    "    - If they ask for something *not* on the menu, then tell them it's out of stock.\n",
    "4. Print the total.\n",
    "\n",
    "Example:\n",
    "\n",
    "    Order: sandwich\n",
    "    Sandwich is 10, total is 10\n",
    "    Order: tea\n",
    "    Tea is 5, total is 15\n",
    "    Order: elephant\n",
    "    We are fresh out of elephant today!\n",
    "    Order: [ENTER]\n",
    "    Total is 15\n",
    "\n",
    "Some things to remember:\n",
    "1. We can use a `while True` loop, which is infinite. Just don't forget to `break` when you want to exit from the loop, or it'll run forever.\n",
    "2. When we use `input` to get input from the user, check to see if we got an empty string; if so, you can `break` out of the loop.\n",
    "3. Ask the user for their input. If it's blank, `break`. If not, check to see if it's a key in the dict."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "b67a88f3-e0b1-4b76-aeb7-7ca3f4224d66",
   "metadata": {},
   "outputs": [],
   "source": [
    "# define my dict, a variable called \"menu\"\n",
    "\n",
    "menu = {'sandwich':10   ,   'tea':5   ,   'cake':7,     'apple':3  }"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "9621f9c6-d7ba-4969-a776-ba69ea5253a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what is the length of my dict?\n",
    "\n",
    "len(menu)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "d0aaaea0-57f9-40f0-908d-8c278e0e49aa",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "menu['sandwich']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "a3fd4a3b-bb64-44d2-bddf-9237e42221c4",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "5"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "menu['tea']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "dbfee38e-e115-4f96-a790-96da1ce3d163",
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Order:  sandwich\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sandwich costs 10; new total is 10\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Order:  tea\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tea costs 5; new total is 15\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Order:  elephant\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Sorry, we are all out of elephant today. Try something else.\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Order:  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total is 15\n"
     ]
    }
   ],
   "source": [
    "# in my \"menu\" dict, the convention is that the keys are the menu items,\n",
    "# and the values are the prices for the corresponding items.\n",
    "\n",
    "menu = {'sandwich':10   ,   'tea':5   ,   'cake':7,     'apple':3  }\n",
    "total = 0\n",
    "\n",
    "while True:   # infinite loop -- be sure to have an escape plan with \"break\" later on\n",
    "    order = input('Order: ').strip()   # ask the user for an order, remove whitespace, and assign to order\n",
    "\n",
    "    if order == '':      # this is my escape hatch!\n",
    "        break\n",
    "\n",
    "    # not an empty string? Let's find out if the string is a key in our \"menu\" dict\n",
    "    if order in menu:\n",
    "        price = menu[order]    # grab the price from the menu dict\n",
    "        total += price         # add the price to our total\n",
    "        print(f'{order} costs {price}; new total is {total}')\n",
    "\n",
    "    else:\n",
    "        print(f'Sorry, we are all out of {order} today. Try something else.')\n",
    "\n",
    "print(f'Total is {total}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "e698cd53-4eca-40f9-ae20-d3be0263190c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'sandwich': 10, 'tea': 5, 'cake': 7, 'apple': 3}"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "menu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "fa527db5-3619-462c-8c4c-373006dfa2a7",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 24,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "menu['sandwich']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "519152b8-72a9-4470-a417-a31efa9d7712",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10\n"
     ]
    }
   ],
   "source": [
    "price = menu['sandwich']   # this assigns the value 10 to the variable price\n",
    "print(price)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "id": "7854632e-b16d-4047-ad12-8d8e84fa3b5b",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "10"
      ]
     },
     "execution_count": 26,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "total = 0\n",
    "price = menu['sandwich']    # this assigns the value 10 to price\n",
    "total += price               # this adds the value of price to total\n",
    "total"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 27,
   "id": "e8246488-f596-4960-bdf3-c26ad9bac939",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': 10, 'b': 20, 'c': 30}"
      ]
     },
     "execution_count": 27,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# if you want, you can create a dict from a list of tuples (or even a list of lists)\n",
    "# if each inner tuple/list contains two elements, that works:\n",
    "\n",
    "mylist = [('a',10), ('b',20), ('c',30)]\n",
    "\n",
    "dict(mylist)   # this returns a new dict based on mylist"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 28,
   "id": "17a545e1-d53b-469d-af54-6fcb4ecae086",
   "metadata": {},
   "outputs": [
    {
     "ename": "SyntaxError",
     "evalue": "invalid syntax (754900947.py, line 6)",
     "output_type": "error",
     "traceback": [
      "\u001b[0;36m  Cell \u001b[0;32mIn [28], line 6\u001b[0;36m\u001b[0m\n\u001b[0;31m    if order menu:  # this is not what you mean!\u001b[0m\n\u001b[0m             ^\u001b[0m\n\u001b[0;31mSyntaxError\u001b[0m\u001b[0;31m:\u001b[0m invalid syntax\n"
     ]
    }
   ],
   "source": [
    "# what if we don't use in?\n",
    "# I assume you aren't asking: What if we just drop the word \"in\" from our query?\n",
    "\n",
    "order = 'sandwich'\n",
    "\n",
    "if order menu:  # this is not what you mean!\n",
    "    print('Good!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "c1e109b7-6d57-4fc4-9d5f-3656d11e0796",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "10\n"
     ]
    }
   ],
   "source": [
    "# I think you were asking: What happens if we don't check, using \"in\", whether the order\n",
    "# is in the dict? Then, what value do we get back?\n",
    "\n",
    "price = menu['sandwich']\n",
    "print(price)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "id": "f407c79f-b252-4cd5-8660-9fa3ea2f3e3b",
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'elephant'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "Cell \u001b[0;32mIn [30], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m price \u001b[38;5;241m=\u001b[39m \u001b[43mmenu\u001b[49m\u001b[43m[\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43melephant\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m]\u001b[49m\n\u001b[1;32m      2\u001b[0m \u001b[38;5;28mprint\u001b[39m(price)\n",
      "\u001b[0;31mKeyError\u001b[0m: 'elephant'"
     ]
    }
   ],
   "source": [
    "price = menu['elephant']\n",
    "print(price)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66c33ffe-3273-4d5b-b52a-ef2298414c86",
   "metadata": {},
   "source": [
    "# Changing dictionaries\n",
    "\n",
    "Dicts, like lists, are *mutable*. They can be changed!  Let's see."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "id": "8d893048-7b02-47a0-b7e7-9c6b34d74e2f",
   "metadata": {},
   "outputs": [],
   "source": [
    "d = {'a':10, 'b':20, 'c':30}\n",
    "\n",
    "# what happens if I assign to an existing key?\n",
    "d['b'] = 1234"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "id": "7670599a-930c-4ad9-9c3d-9de7fd62c80a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': 10, 'b': 1234, 'c': 30}"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# assigning to an existing key keeps the key, but replaces the value. \n",
    "# (note that it does not add the old value to the new one. It just replaces it.)\n",
    "\n",
    "d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "id": "fde3c7cf-d404-4d7c-8381-cad1e5cf272b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# what if I assign to a key that does *not* exist?\n",
    "# we add a new key-value pair to a dict\n",
    "d['x'] = 5678"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "id": "268ebf7e-71a6-4b9b-80c9-c669453177eb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': 10, 'b': 1234, 'c': 30, 'x': 5678}"
      ]
     },
     "execution_count": 38,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ee62844-f32a-42b4-a6c6-72c3b6bae3ac",
   "metadata": {},
   "source": [
    "# Lists vs. dicts\n",
    "\n",
    "If you thought that there would be an `append` method for dicts, allowing us to add new elements, surprise! All we have to do is assign to a key-value pair. If the key is new, we get a new key-value pair. If the key already exists, then we replace the value.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "id": "93e78b78-4522-44a6-9568-19dc3a91661d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'sandwich': 10, 'tea': 5, 'cake': 7, 'apple': 3, 'water': 2.5}"
      ]
     },
     "execution_count": 39,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# FD asks: can floats be in dicts?\n",
    "# they can *definitely* be values in dicts, because ALL Python  objects an be values\n",
    "\n",
    "menu['water'] = 2.5\n",
    "menu"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "979f1238-52be-4da2-b463-dd736cb2d642",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.5\n"
     ]
    }
   ],
   "source": [
    "price = menu['water']\n",
    "print(price)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "b448aea0-ba61-446b-a1d2-f5376a033bdc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "float"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "type(price)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "6030dfb1-575e-4f65-8e39-48f2c2be916b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# can floats be *keys* in a dict?\n",
    "# answer: yes! They are immutable, just like integers"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "402e8bcb-68a9-4f67-8b4a-08105fb1d137",
   "metadata": {},
   "outputs": [],
   "source": [
    "special_numbers = {3.14:'pi', 2.718281828459045:'e', -1:'i'}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "37d51db2-afea-4d74-94d3-ab0cb062ed73",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'pi'"
      ]
     },
     "execution_count": 44,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "special_numbers[3.14]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1705c864-49fb-497b-baa5-77c693ac0164",
   "metadata": {},
   "source": [
    "# CH asks: More than 2 things?\n",
    "\n",
    "Every key has exactly one value. Every value has exactly one key.\n",
    "\n",
    "HOWEVER, a value can be a list, tuple, or dict. So a key could refer to a value that is a dict, whose values are lists, whose values are tuples, etc. etc."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aed8cf22-1b9f-410d-b81c-9b22058d4cf0",
   "metadata": {},
   "source": [
    "# FJ : Can we make the values in a dict immutable?\n",
    "\n",
    "If the values are immutable (e.g., strings), then we cannot change them. But that wasn't your question - you want to know if we can make it impossible to change the values in a dict. Answer: No.\n",
    "\n",
    "There *might* be a \"frozendict\" in Python that is created once, and then not changeable. There is a \"frozenset,\" and sets are second cousins of dictionaries."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "73288d3c-a85f-4f68-88a4-6a5074edf785",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "2.5"
      ]
     },
     "execution_count": 45,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# AK Formatting output, such as $2.50\n",
    "\n",
    "# We can actually use F-strings for this!\n",
    "\n",
    "price"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "id": "e5592c2e-775a-461b-8b8a-c07d6a06676e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "the price is $2.50.\n"
     ]
    }
   ],
   "source": [
    "# you can put lots of magic stuff after : in the {} in an f-string to modify\n",
    "# how things are displayed.\n",
    "\n",
    "print(f'the price is ${price:0.2f}.')  # :0.2f means: show 2 digits after the . for float values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "e53f2058-031e-45b5-95af-e72bda669e19",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': 40, 'b': 50, 'c': 60}"
      ]
     },
     "execution_count": 51,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# keys cannot exist more than once in a dict \n",
    "d = {'a':10, 'b':20, 'c':30, 'a':40, 'b':50, 'c':60}\n",
    "\n",
    "d  # did we have duplicate keys? No way"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "2edfd846-3d00-4914-84cf-d30cf00cafbf",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': 999, 'b': 50, 'c': 60}"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what if I assign a key that already exists?\n",
    "d['a'] = 999\n",
    "d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 54,
   "id": "337e201d-19b5-48f6-8298-5fc55a0af579",
   "metadata": {},
   "outputs": [],
   "source": [
    "# How do we handle case-sensitive keys?\n",
    "\n",
    "# (1) are keys case sensitive? YES, definitely\n",
    "\n",
    "# (2) how can we deal with that? Answer: Use lowercase when creating the dict (if you can)\n",
    "# and then use .lower() on the user's input.\n",
    "\n",
    "# if you have mIxEd cAsE as keys in your dict.... good luck!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7056ac0d-b171-48b5-9040-a4982f5d83b5",
   "metadata": {},
   "source": [
    "# What about removing items from a dict?\n",
    "\n",
    "First: I rarely find myself removing items from a dict in a program.\n",
    "\n",
    "Second: You can use the `.pop` method, which takes a key as an argument, and returns the value. Plus, it removes the key-value pair you specified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "id": "b8e13862-b7ac-49ff-83f7-a61d10f54266",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': 999, 'b': 50, 'c': 60}"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "id": "e132bae6-928f-48aa-9e46-726a22444afb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "60"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d.pop('c')   # remove the key-value pair with the key 'c'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "id": "a02210db-5e84-47f5-b407-0fca4554a4c3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': 999, 'b': 50}"
      ]
     },
     "execution_count": 57,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "id": "3828280c-34a1-4179-8d16-b9bc8ad0cb64",
   "metadata": {},
   "outputs": [
    {
     "ename": "KeyError",
     "evalue": "'c'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "Cell \u001b[0;32mIn [58], line 1\u001b[0m\n\u001b[0;32m----> 1\u001b[0m \u001b[43md\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43mpop\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[38;5;124;43mc\u001b[39;49m\u001b[38;5;124;43m'\u001b[39;49m\u001b[43m)\u001b[49m   \u001b[38;5;66;03m# remove it again?!?\u001b[39;00m\n",
      "\u001b[0;31mKeyError\u001b[0m: 'c'"
     ]
    }
   ],
   "source": [
    "d.pop('c')   # remove it again?!?"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "07f9d552-b030-4a76-bd6a-01c39f174f35",
   "metadata": {},
   "source": [
    "# Keys are primary in a dict\n",
    "\n",
    "The values in a dict are kind of along for the ride -- you cannot search on them, they don't determine any of the back-end storage (which we'll discuss later), and they can repeat.\n",
    "\n",
    "You always need to think about a dict as a one-way street -- via the key, you can get to the value. But via the value, you *cannot* easily get to the key."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59527d23-edb7-4df9-8db2-d83ff8a6d6e2",
   "metadata": {},
   "source": [
    "# Handling bad keys\n",
    "\n",
    "If you try to retrieve from a dict, and the key you specify doesn't exist in the dict, then you'll get an exception. There is something in Python known as \"exception handling\" that we won't cover in this course. If you encounter that exception, you can \"trap\" it with an exception handler, and then give the user a more normal error message.\n",
    "\n",
    "We're going to stick to searching in the dict via `in` for a key, and thus avoiding such trouble.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "id": "61ea9245-fd15-4fa9-9099-24cf99b817ee",
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Order:  sandwich\n",
      "How many?  3\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sandwich costs 10; you ordered 3; new total is 30\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Order:  apple\n",
      "How many?  10\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "apple costs 3; you ordered 10; new total is 60\n"
     ]
    },
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Order:  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total is 60\n"
     ]
    }
   ],
   "source": [
    "# variation on our exercise code, with quantities (blame this on AD)\n",
    "\n",
    "menu = {'sandwich':10   ,   'tea':5   ,   'cake':7,     'apple':3  }\n",
    "total = 0\n",
    "\n",
    "while True:   # infinite loop -- be sure to have an escape plan with \"break\" later on\n",
    "    order = input('Order: ').strip()   # ask the user for an order, remove whitespace, and assign to order\n",
    "\n",
    "    if order == '':      # this is my escape hatch!\n",
    "        break\n",
    "\n",
    "    # not an empty string? Let's find out if the string is a key in our \"menu\" dict\n",
    "    if order in menu:\n",
    "        quantity_string = input('How many? ').strip()\n",
    "\n",
    "        if quantity_string.isdigit():\n",
    "            quantity = int(quantity_string)\n",
    "        else:\n",
    "            print(f'Non-numeric quantity; try again. ')\n",
    "            continue   # go back to the top of the while loop\n",
    "        \n",
    "        price = menu[order]          # grab the price from the menu dict\n",
    "        total += price * quantity    # add the price to our total\n",
    "        print(f'{order} costs {price}; you ordered {quantity}; new total is {total}')\n",
    "\n",
    "    else:\n",
    "        print(f'Sorry, we are all out of {order} today. Try something else.')\n",
    "\n",
    "print(f'Total is {total}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b14b8c55-d53e-49b5-852f-a06daa12a233",
   "metadata": {},
   "source": [
    "# Next up:\n",
    "\n",
    "1. Accumulating in dicts\n",
    "2. Accumulating the unknown"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d8264ac-680c-4b6c-a759-9c7282d12ef5",
   "metadata": {},
   "source": [
    "# Dicts so far\n",
    "\n",
    "We've used paradigm #1, namely: We define a dict and use it in our program, never changing it. We just read it from it.  This is pretty common:\n",
    "\n",
    "1. Dict keys are month names, and dict values are month numbers.\n",
    "2. Dict keys are month numbers, and dict values are month names.\n",
    "3. Dict keys are country names, and dict values are international dial prefixes.\n",
    "4. Dict keys are user IDs, and dict values are also dicts, with user names, e-mail addresses, and salary."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a97093b-ae36-442c-a647-9833e8976d70",
   "metadata": {},
   "source": [
    "# Sometimes, we want to modify a dict!\n",
    "\n",
    "In paradigm #2, we define a dict at the start of the program with keys and empty values -- 0, empty list, etc. Over the course of the program, we don't add/remove keys, but we do update/modify the values.\n",
    "\n",
    "In this paradigm, we know what we want to count or keep track of, but we don't know in advance how many there will be.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 60,
   "id": "455d8b1e-0c23-49e0-801a-2baab3553204",
   "metadata": {},
   "outputs": [],
   "source": [
    "# odds and evens\n",
    "\n",
    "counts = {'odds':0, 'evens':0}\n",
    "\n",
    "# let's create a list of integers\n",
    "mylist = [10, 15, 20, 25]\n",
    "\n",
    "for one_number in mylist:\n",
    "    if one_number % 2 == 1:   # if the remainder after dividing by 2 is 1, we have an odd number!\n",
    "        counts['odds'] += 1   # add 1 to the score of odd numbers\n",
    "    else:\n",
    "        counts['evens'] += 1  # add 1 to the score of even numbers\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "id": "5e613e75-86f3-467e-a014-c133c1a8f8de",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'odds': 2, 'evens': 2}"
      ]
     },
     "execution_count": 61,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "counts"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "219d54c8-c9e3-4b58-be82-4b3c3c952d3f",
   "metadata": {},
   "source": [
    "# Exercise: Digits, vowels, and others (dict edition)\n",
    "\n",
    "1. Create a dict called `counts` in which you have three keys: `vowels`, `digits`, and `others`. The values should all be set to 0.\n",
    "2. Ask the user to enter a string.\n",
    "3. Go through the string, one character at a time:\n",
    "    - If the character is a digit, add 1 to `digits`\n",
    "    - If the character is a vowel, add 1 to `vowels`\n",
    "    - In other cases, add 1 to `others`\n",
    "4. Print the dict with the counts.\n",
    "\n",
    "Example:\n",
    "\n",
    "    Enter a string: hello! 123\n",
    "    {'digits':3, 'vowels':2, 'others':5}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 62,
   "id": "36a4e71c-cbd3-4663-a5fe-150a6c512a75",
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Enter a string:  hello! 123\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'digits': 3, 'vowels': 2, 'others': 5}\n"
     ]
    }
   ],
   "source": [
    "counts = {'digits':0, \n",
    "          'vowels':0,\n",
    "          'others':0}\n",
    "s = input('Enter a string: ').strip()\n",
    "\n",
    "for one_character in s:\n",
    "    if one_character.isdigit():\n",
    "        counts['digits'] += 1    # counts['digits'] = counts['digits'] + 1\n",
    "    elif one_character in 'aeiou':\n",
    "        counts['vowels'] += 1\n",
    "    else:\n",
    "        counts['others'] += 1\n",
    "\n",
    "print(counts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "id": "9880fb64-7f70-41c5-bba0-c307b123ffdc",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 63,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# how do I check odd vs. even?\n",
    "\n",
    "10 % 3     # this means: what's the remainder after dividing 10 / 3 ?   it'll be 3, remainder 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "id": "cb285b5c-f92d-4da6-9f29-a37405efa725",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0"
      ]
     },
     "execution_count": 64,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# if I divide a number by 2 and get its remainder:\n",
    "# - if the remainder is 0, the number is even\n",
    "# - if the remainder is 1, the number is odd\n",
    "\n",
    "10 % 2    # remainder is 0, 10 is even"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 65,
   "id": "dfffa61e-ec6d-44b6-afb3-b742bff01820",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 65,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "13 % 2    # remainder is 1, 13 is odd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "id": "ca3f2b84-df7e-4839-bb7d-b8f63c6aed9c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Enter a string:  hello! 123\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'digits': 3, 'vowels': 2, 'consonants': 3, 'others': 2}\n"
     ]
    }
   ],
   "source": [
    "# AK extension -- also check for non-vowel letters\n",
    "\n",
    "# we can use the \"string\" module in Python's standard library\n",
    "# which provides pre-defined variables with all sorts of useful stuff\n",
    "\n",
    "import string\n",
    "\n",
    "counts = {'digits':0, \n",
    "          'vowels':0,\n",
    "          'consonants':0,\n",
    "          'others':0}\n",
    "s = input('Enter a string: ').strip()\n",
    "\n",
    "for one_character in s:\n",
    "    if one_character.isdigit():\n",
    "        counts['digits'] += 1    # counts['digits'] = counts['digits'] + 1\n",
    "    elif one_character in 'aeiou':\n",
    "        counts['vowels'] += 1\n",
    "    # elif one_character in 'bcdfghjklmnpqrstvwxyz':\n",
    "\n",
    "    # the string module contains many predefined variables with different characters\n",
    "    # ascii_lowercase means: lowercase letters in the Latin/English alphabet\n",
    "    # ASCII is an old standard for computers to turn numbers into letters (and vice versa)\n",
    "    elif one_character in string.ascii_lowercase:\n",
    "        counts['consonants'] += 1\n",
    "    else:\n",
    "        counts['others'] += 1\n",
    "\n",
    "print(counts)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e52e805-aba7-4312-a741-41fbaf45eb2a",
   "metadata": {},
   "source": [
    "# When .isdigit doesn't work\n",
    "\n",
    "I've often seen two mistakes/problems with `isdigit`:\n",
    "\n",
    "1. Remember that `isdigit` is a string method. It only works on strings.  So don't try to convert something to an integer, and then run `isdigit` on the integer. You'll get an error saying that integers don't have any attribute named `isdigit`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "id": "463621ce-9d4b-4bba-bce4-98cd60d8a23c",
   "metadata": {},
   "outputs": [
    {
     "ename": "AttributeError",
     "evalue": "'int' object has no attribute 'isdigit'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mAttributeError\u001b[0m                            Traceback (most recent call last)",
      "Cell \u001b[0;32mIn [70], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m x \u001b[38;5;241m=\u001b[39m \u001b[38;5;241m5\u001b[39m\n\u001b[0;32m----> 2\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[43mx\u001b[49m\u001b[38;5;241;43m.\u001b[39;49m\u001b[43misdigit\u001b[49m():    \u001b[38;5;66;03m# isdigit is a string method, not an int method\u001b[39;00m\n\u001b[1;32m      3\u001b[0m     \u001b[38;5;28mprint\u001b[39m(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124myes, it is numeric!\u001b[39m\u001b[38;5;124m'\u001b[39m)   \u001b[38;5;66;03m# this code will fail\u001b[39;00m\n",
      "\u001b[0;31mAttributeError\u001b[0m: 'int' object has no attribute 'isdigit'"
     ]
    }
   ],
   "source": [
    "x = 5\n",
    "if x.isdigit():    # isdigit is a string method, not an int method\n",
    "    print('yes, it is numeric!')   # this code will fail"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f335ad6b-fe1e-47f6-b65e-a8ad22230fa8",
   "metadata": {},
   "source": [
    "2. A second mistake is that they forget that you need to call a method with `()` after its name. If you don't do that, then you're asking if `isdigit` exists, and the answer is always \"yes\" to that. Which means you'll get errors, because you'll get a `True` result, regardless of whether it's an integer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "id": "344b517c-16a9-4b39-94db-5f0d05ce3127",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Yes, 5 contains digits!\n"
     ]
    }
   ],
   "source": [
    "x = '5'\n",
    "\n",
    "if x.isdigit:   # notice: no (), which is A BIG BUG!\n",
    "    print(f'Yes, {x} contains digits!')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "id": "d437516f-692c-4670-9877-697d3390fc7a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Yes, ! contains digits!\n"
     ]
    }
   ],
   "source": [
    "x = '!'\n",
    "\n",
    "if x.isdigit:   # notice: no (), which is A BIG BUG!\n",
    "    print(f'Yes, {x} contains digits!')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b69791bb-2aae-49a8-bb91-7b208ccea3de",
   "metadata": {},
   "source": [
    "# Paradigm 3: Start with an empty dict, build it over time\n",
    "\n",
    "In this third paradigm, we don't know what keys we'll get, and we don't know what values we'll get. But we know what we want to do with them over time.\n",
    "\n",
    "As we get new keys, we add them to the dict. As we get values, we add them (as before) to the existing count.\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "id": "2732d9a6-a5ea-4ae1-b763-4557f37757a6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Enter a string:  hello out there!\n"
     ]
    },
    {
     "ename": "KeyError",
     "evalue": "'h'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "Cell \u001b[0;32mIn [74], line 8\u001b[0m\n\u001b[1;32m      5\u001b[0m s \u001b[38;5;241m=\u001b[39m \u001b[38;5;28minput\u001b[39m(\u001b[38;5;124m'\u001b[39m\u001b[38;5;124mEnter a string: \u001b[39m\u001b[38;5;124m'\u001b[39m)\u001b[38;5;241m.\u001b[39mstrip()\n\u001b[1;32m      7\u001b[0m \u001b[38;5;28;01mfor\u001b[39;00m one_character \u001b[38;5;129;01min\u001b[39;00m s:\n\u001b[0;32m----> 8\u001b[0m     \u001b[43mcounts\u001b[49m\u001b[43m[\u001b[49m\u001b[43mone_character\u001b[49m\u001b[43m]\u001b[49m \u001b[38;5;241m+\u001b[39m\u001b[38;5;241m=\u001b[39m \u001b[38;5;241m1\u001b[39m   \u001b[38;5;66;03m# my ideal -- this will not work!\u001b[39;00m\n\u001b[1;32m     10\u001b[0m \u001b[38;5;28mprint\u001b[39m(counts)\n",
      "\u001b[0;31mKeyError\u001b[0m: 'h'"
     ]
    }
   ],
   "source": [
    "# Example: How many times does each character appear in a user's input?\n",
    "\n",
    "counts = {}   # empty dict!\n",
    "\n",
    "s = input('Enter a string: ').strip()\n",
    "\n",
    "for one_character in s:\n",
    "    counts[one_character] += 1   # my ideal -- this will not work!\n",
    "\n",
    "print(counts)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "id": "43ee79bd-51c8-4f5d-81fb-761a469080e1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Enter a string:  hello out there\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'h': 2, 'e': 3, 'l': 2, 'o': 2, ' ': 2, 'u': 1, 't': 2, 'r': 1}\n"
     ]
    }
   ],
   "source": [
    "# Example: How many times does each character appear in a user's input?\n",
    "# now, let's do it the working way, making sure that a key is there before we add to / retrieve it\n",
    "\n",
    "counts = {}   # empty dict!\n",
    "\n",
    "s = input('Enter a string: ').strip()\n",
    "\n",
    "for one_character in s:\n",
    "    if one_character in counts:      # if we've seen this letter before,\n",
    "        counts[one_character] += 1   #    add 1 to its count\n",
    "    else:                            # if we have NOT seen this letter before,\n",
    "        counts[one_character] = 1    #    set the new key-value pair with a value of 1\n",
    "        \n",
    "print(counts)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ad4effdc-a33e-45c0-937c-6727ea6e290c",
   "metadata": {},
   "source": [
    "# Exercise: Rainfall\n",
    "\n",
    "The goal of this exercise is to create a dict in which the keys are city names, and the values are quantities of rain (measured in mm).  Over time, we'll gather information about cities and rainfall, and then be able to produce a report showing the totals.\n",
    "\n",
    "1. Create an empty dict, called `rainfall`.\n",
    "2. Ask the user to enter the name of a city.  (We have no restrictions -- we don't have a list of cities in advance.)\n",
    "    - If the user enters an empty string, we stop asking and print the entire `rainfall` dict.\n",
    "3. If we got a city name, ask the user to enter how many mm of rain fell since the last report. We can assume that the user will give us digits here.\n",
    "4. If this is the first time we're seeing a report from this city, add the key-value pair to `rainfall`.\n",
    "5. If this is *not* the first time we're seeing a report from this city, then add the new value to the existing one in `rainfall`.\n",
    "6. Print the entire dict.\n",
    "\n",
    "Example:\n",
    "\n",
    "    City: Jerusalem\n",
    "    Rain: 3\n",
    "    City: Tel Aviv\n",
    "    Rain: 4\n",
    "    City: Tel Aviv\n",
    "    Rain: 5\n",
    "    City: [ENTER]\n",
    "    {'Jerusalem':3, 'Tel Aviv':9}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "id": "6bc9a654-e51d-4358-abde-f478e7a502b5",
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "City:  a\n",
      "Rain:  5\n",
      "City:  b\n",
      "Rain:  4\n",
      "City:  a\n",
      "Rain:  3\n",
      "City:  \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'a': 8, 'b': 4}\n"
     ]
    }
   ],
   "source": [
    "# empty dict in which we'll store city names + mm rain\n",
    "rainfall = {}\n",
    "\n",
    "while True:   # potentially infinite loop!\n",
    "    city_name = input('City: ').strip()\n",
    "\n",
    "    if city_name == '':     # this is our escape hatch -- if we got an empty string, break out of the loop\n",
    "        break\n",
    "\n",
    "    mm_rain = int(input('Rain: ').strip())\n",
    "\n",
    "    if city_name in rainfall:            # is this city already a key in \"rainfall\"?\n",
    "        rainfall[city_name] += mm_rain   #     add to the existing rainfall\n",
    "    else:\n",
    "        rainfall[city_name] = mm_rain    # if this is the first time with this city, we'll just assign\n",
    "\n",
    "print(rainfall)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "398555bf-ba39-4f21-9d4b-af47994e94b4",
   "metadata": {},
   "source": [
    "# Next up\n",
    "\n",
    "1. Looping and dicts\n",
    "2. How dicts are implemented (behind the scenes)\n",
    "3. Start with files!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "id": "b31970bf-a65e-45bd-acc3-a85e600e54ab",
   "metadata": {},
   "outputs": [
    {
     "name": "stdin",
     "output_type": "stream",
     "text": [
      "Enter a city:  a\n",
      "Rain:  5\n"
     ]
    },
    {
     "ename": "KeyError",
     "evalue": "'a'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mKeyError\u001b[0m                                  Traceback (most recent call last)",
      "Cell \u001b[0;32mIn [80], line 16\u001b[0m\n\u001b[1;32m     14\u001b[0m         rainfall[city] \u001b[38;5;241m+\u001b[39m\u001b[38;5;241m=\u001b[39m mm    \u001b[38;5;66;03m#    add mm to the current rainfall\u001b[39;00m\n\u001b[1;32m     15\u001b[0m     \u001b[38;5;28;01melse\u001b[39;00m:\n\u001b[0;32m---> 16\u001b[0m         \u001b[43mrainfall\u001b[49m\u001b[43m[\u001b[49m\u001b[43mcity\u001b[49m\u001b[43m]\u001b[49m \u001b[38;5;241m==\u001b[39m mm    \u001b[38;5;66;03m# we'll compare the current rainfall with the user's new rainfall\u001b[39;00m\n\u001b[1;32m     18\u001b[0m \u001b[38;5;28mprint\u001b[39m(rainfall)\n",
      "\u001b[0;31mKeyError\u001b[0m: 'a'"
     ]
    }
   ],
   "source": [
    "# TC's code\n",
    "\n",
    "rainfall = {}\n",
    "\n",
    "while True:\n",
    "    city = input('Enter a city: ').strip()\n",
    "    \n",
    "    if city == '':\n",
    "        break\n",
    "    \n",
    "    mm = int(input('Rain: ').strip())\n",
    "    \n",
    "    if city in rainfall:        # if the city that the user entered is in the dict as a key...\n",
    "        rainfall[city] += mm    #    add mm to the current rainfall\n",
    "    else:\n",
    "        rainfall[city] == mm    # we'll compare the current rainfall with the user's new rainfall\n",
    "        \n",
    "print(rainfall)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7ead2e00-73c6-4f3f-8e96-8ce1eeef8ca6",
   "metadata": {},
   "source": [
    "# Loops and dicts\n",
    "\n",
    "We've seen that we can do lots of things with dicts:\n",
    "\n",
    "- Define them\n",
    "- Retrieve from them via a key\n",
    "- Update a value based on a key\n",
    "- Remove a key-value pair\n",
    "\n",
    "What happens if we run a `for` loop on our dict?\n",
    "\n",
    "We know that every type of value works differently with `for` loops:\n",
    "\n",
    "- integers *don't* work with them\n",
    "- `range()` does work with loops, giving us one integer at a time\n",
    "- strings give us one character at a time\n",
    "- lists give us one element at a time\n",
    "- tuples give us one element at a time\n",
    "\n",
    "What about dicts?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "id": "958108f0-c6c6-4bfe-8283-f84195a42bd2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a\n",
      "b\n",
      "c\n"
     ]
    }
   ],
   "source": [
    "d = {'a':10, 'b':20, 'c':30}\n",
    "\n",
    "# iterating over a dictionary gives us the keys (not the values)!\n",
    "for one_item in d:    \n",
    "    print(one_item)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "id": "4fcf938f-40e8-44eb-85ef-2a156610fa99",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a: 10\n",
      "b: 20\n",
      "c: 30\n"
     ]
    }
   ],
   "source": [
    "# what if I want to print a dict -- all of its keys and values?\n",
    "\n",
    "for key in d:\n",
    "    print(f'{key}: {d[key]}')   # print keys and values"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "id": "cc4c07bc-36e9-42eb-92b4-a3db38d30e4d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "('a', 10)\n",
      "('b', 20)\n",
      "('c', 30)\n"
     ]
    }
   ],
   "source": [
    "# there is a nicer/better way to print both keys and values\n",
    "# this is my favorite way: the .items method\n",
    "# items returns, with each iteration, one 2-element tuple (key, value)\n",
    "\n",
    "for t in d.items():    # get the tuple, print the tuple\n",
    "    print(t)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "id": "22495998-ab87-401f-8eff-a82ec554fffb",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a: 10\n",
      "b: 20\n",
      "c: 30\n"
     ]
    }
   ],
   "source": [
    "for t in d.items():             # get the tuple,\n",
    "    print(f'{t[0]}: {t[1]}')    # print t[0] (key) and t[1] (value)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "id": "0fc06874-cbf2-4a70-b1d9-de77b94c0fba",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a: 10\n",
      "b: 20\n",
      "c: 30\n"
     ]
    }
   ],
   "source": [
    "# let's turn t[0] and t[1] into variables with names!\n",
    "# let's do this with unpacking!   there are 2 values in t, and we can assign them to 2 variables\n",
    "\n",
    "for t in d.items():\n",
    "    key, value = t    # unpacking here\n",
    "    print(f'{key}: {value}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "id": "bc72c6eb-51a5-4a00-9d7e-eeccde9d5e03",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a: 10\n",
      "b: 20\n",
      "c: 30\n"
     ]
    }
   ],
   "source": [
    "# now, let's show you the final way to do this\n",
    "\n",
    "for key, value in d.items():\n",
    "    print(f'{key}: {value}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "id": "422a11b9-8d3b-4636-9f41-047ad5f39b6a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_items([('a', 10), ('b', 20), ('c', 30)])"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d.items()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "id": "8ecad630-2898-4c4a-9333-ba12ab4fa715",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "dict_values([10, 20, 30])"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what if you want to search in the values?\n",
    "# you can call the .values() method on a dict\n",
    "\n",
    "d.values()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "id": "334cdc6c-73f1-4ebe-90a2-733d078594be",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 93,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "20 in d.values()   # this is relatively rare and VERY slow (comparatively)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "id": "333a6441-d7a8-4685-9bb7-6254c9a802ad",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 94,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# you can also say:\n",
    "\n",
    "'a' in d.keys()   # this is also VERY slow, and you should never do it -- it's much faster to say 'a' in d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "id": "794e4079-a3c6-4fa8-96df-cc30036d2927",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'a': 10, 'b': (20, 30, 40), 'c': {'x': 100, 'y': 200}}"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "d = {'a':10, 'b':(20, 30, 40), 'c':{'x':100, 'y':200}}\n",
    "d"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "id": "6a1cc9fc-73f7-4842-9f1e-6ec45df50b1b",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "a: 10 -- value is <class 'int'>\n",
      "b: (20, 30, 40) -- value is <class 'tuple'>\n",
      "c: {'x': 100, 'y': 200} -- value is <class 'dict'>\n"
     ]
    }
   ],
   "source": [
    "for key, value in d.items():\n",
    "    print(f'{key}: {value} -- value is {type(value)}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bdb30a6-4187-43e3-b983-ac20b0fb35ab",
   "metadata": {},
   "source": [
    "# Dicts: Behind the scenes\n",
    "\n",
    "Dicts are also known as \"hash tables,\" because a \"hash function\" is used to determine where things are stored in memory.\n",
    "\n",
    "Let's start by talking about lists. If I want to search in a list for a value, how long will it take? In CS theory notation, we say `O(n)`, aka \"linear time.\"  Meaning: The longer the list, the longer we might need to search for a value.\n",
    "\n",
    "When Python searches in a list (with a `for` loop or with `in`), it might need to go through the entire list before discovering that the value is not there. \n",
    "\n",
    "How can we do better? Hash functions!\n",
    "\n",
    "When Python needs to decide where to store a key-value pair in memory, it runs `hash(key)`. This function returns a number that is very hard to predict, looks random, but gives the same answer for the same key over time. It tries to randomly distributed value, so that the odds of two different value having the same hash result is very small. \n",
    "\n",
    "When you say `d['a'] = 10`, Python runs `hash('a')`, and uses the number it gets back to store the key-value pair. When you then ask `'a' in d`, Python again runs `hash('a')`, jumps to that location in memory, and checks if the key-value pair is there. If so, then great! If not, we get an error.\n",
    "\n",
    "This means that we can have any number of key-value pairs, and the search speed won't change. It'll always be what CS theory people call O(1), or \"constant time.\" \n",
    "\n",
    "This explains why only immutable data can be used as a key. If the data can change, then the hash function's result on the data will change, too. Then what? Do we move things around in the data structure? Do we keep a record of where it used to be? No. That's too complicated.  As a result, Python forbids us from using mutable data as keys, to avoid such problems."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "be241b6d-a6dd-4583-8db4-9fd4308cd75c",
   "metadata": {},
   "source": [
    "# Files\n",
    "\n",
    "The point of a file is to avoid having to enter data each time we start a program, or turn on a computer. They provide us with storage for information that we might need later on.\n",
    "\n",
    "There are many types of files, but they all come down to some numbers on the disk. Sometimes, those numbers can be interepreted as simple text. (Those are the files we'll be dealing with today.) Other times, those numbers have to be interpreted as complex documents such as Excel, PowerPoint, PDF, or other.  Those are typically quite complicated to work with.\n",
    "\n",
    "We're just going to work with simple text files.\n",
    "\n",
    "It used to be, long ago, that a computer program could just read from the disk on its own. Nowadays, that's very much not the case; the operating system has to protect the files on disk from unauthorized access. Also, it wants to make sure that we don't do anything truly dangerous to the files.\n",
    "\n",
    "Thus, if we want to read from a file, we need to ask the OS for help in doing so, It returns a \"file object,\" or a \"file handle,\" through which we can actually work with a a file.\n",
    "\n",
    "To work with a file:\n",
    "1. Get the name of the file we want to work with\n",
    "2. `open` the filename, a string, for reading. This returns a new file object.\n",
    "3. Read data from the file, via the file object.\n",
    "4. Close the file object, so that we don't waste resources on the computer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "id": "facee470-7730-44c3-9313-2ed69cb49f5c",
   "metadata": {},
   "outputs": [],
   "source": [
    "# let's open a file.\n",
    "# I'm going to read through the file in the zipfile I mentioned earlier,\n",
    "# one called linux-etc-passwd.txt.  This is a \"password file\" from a Linux system\n",
    "# I used to use."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "id": "6af2fd98-8022-4634-932c-fd060d70ac9c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# This is a comment\n",
      "# You should ignore me\n",
      "root:x:0:0:root:/root:/bin/bash\n",
      "daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n",
      "bin:x:2:2:bin:/bin:/usr/sbin/nologin\n",
      "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n",
      "sync:x:4:65534:sync:/bin:/bin/sync\n",
      "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n",
      "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n",
      "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n",
      "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n",
      "\n",
      "\n",
      "\n",
      "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n",
      "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n",
      "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n",
      "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n",
      "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n",
      "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n"
     ]
    }
   ],
   "source": [
    "# here, I use a Unix command to see the top of the file\n",
    "!head -20 linux-etc-passwd.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "id": "49736662-5538-4def-8f36-9c6bfe166cc6",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# This is a comment\n",
      "# You should ignore me\n",
      "root:x:0:0:root:/root:/bin/bash\n",
      "daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n",
      "bin:x:2:2:bin:/bin:/usr/sbin/nologin\n",
      "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n",
      "sync:x:4:65534:sync:/bin:/bin/sync\n",
      "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n",
      "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n",
      "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n",
      "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n",
      "\n",
      "\n",
      "\n",
      "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n",
      "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n",
      "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n",
      "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n",
      "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n",
      "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n",
      "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n",
      "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n",
      "\n",
      "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n",
      "syslog:x:101:104::/home/syslog:/bin/false\n",
      "messagebus:x:102:106::/var/run/dbus:/bin/false\n",
      "landscape:x:103:109::/var/lib/landscape:/bin/false\n",
      "jci:x:955:955::/home/jci:/bin/bash\n",
      "sshd:x:104:65534::/var/run/sshd:/usr/sbin/nologin\n",
      "user:x:1000:1000:user,,,:/home/user:/bin/bash\n",
      "reuven:x:1001:1001:Reuven M. Lerner,,,:/home/reuven:/bin/bash\n",
      "postfix:x:105:113::/var/spool/postfix:/bin/false\n",
      "colord:x:106:116:colord colour management daemon,,,:/var/lib/colord:/bin/false\n",
      "postgres:x:107:117:PostgreSQL administrator,,,:/var/lib/postgresql:/bin/bash\n",
      "dovecot:x:108:119:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false\n",
      "dovenull:x:109:120:Dovecot login user,,,:/nonexistent:/bin/false\n",
      "postgrey:x:110:121::/var/lib/postgrey:/bin/false\n",
      "debian-spamd:x:111:122::/var/lib/spamassassin:/bin/sh\n",
      "memcache:x:113:124:Memcached,,,:/nonexistent:/bin/false\n",
      "genadi:x:1002:1003:Genadi Reznichenko,,,:/home/genadi:/bin/bash\n",
      "shira:x:1003:1004:Shira Friedman,,,:/home/shira:/bin/bash\n",
      "atara:x:1004:1005:Atara Lerner-Friedman,,,:/home/atara:/bin/bash\n",
      "shikma:x:1005:1006:Shikma Lerner-Friedman,,,:/home/shikma:/bin/bash\n",
      "amotz:x:1006:1007:Amotz Lerner-Friedman,,,:/home/amotz:/bin/bash\n",
      "mysql:x:114:125:MySQL Server,,,:/nonexistent:/bin/false\n",
      "clamav:x:115:126::/var/lib/clamav:/bin/false\n",
      "amavis:x:116:127:AMaViS system user,,,:/var/lib/amavis:/bin/sh\n",
      "opendkim:x:117:128::/var/run/opendkim:/bin/false\n",
      "gitlab-redis:x:999:1009::/var/opt/gitlab/redis:/bin/nologin\n",
      "gitlab-psql:x:998:1010::/var/opt/gitlab/postgresql:/bin/sh\n",
      "git:x:1007:1008:GitLab,,,:/home/git:/bin/bash\n",
      "opendmarc:x:118:129::/var/run/opendmarc:/bin/false\n",
      "dkim-milter-python:x:119:130::/var/run/dkim-milter-python:/bin/false\n",
      "deploy:x:1008:1011:Deploy,,,:/home/deploy:/bin/bash\n",
      "redis:x:112:123:redis server,,,:/var/lib/redis:/bin/false\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# to read this file from Python, we'll need to \n",
    "# (1) open it for reading\n",
    "# (2) get its data\n",
    "# (3) close it\n",
    "\n",
    "f = open('linux-etc-passwd.txt')   # same directory, so no need to use / (Unix) or \\ (Windows)\n",
    "print(f.read())                    # read() returns the entire contents of the file as a string\n",
    "f.close()                          # we don't need the file open any more."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "id": "ad86c5ce-b22c-42e1-8333-e9fcc158961a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# This is a comment\n",
      "\n",
      "# You should ignore me\n",
      "\n",
      "root:x:0:0:root:/root:/bin/bash\n",
      "\n",
      "daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n",
      "\n",
      "bin:x:2:2:bin:/bin:/usr/sbin/nologin\n",
      "\n",
      "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n",
      "\n",
      "sync:x:4:65534:sync:/bin:/bin/sync\n",
      "\n",
      "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n",
      "\n",
      "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n",
      "\n",
      "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n",
      "\n",
      "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n",
      "\n",
      "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n",
      "\n",
      "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n",
      "\n",
      "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n",
      "\n",
      "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n",
      "\n",
      "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n",
      "\n",
      "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n",
      "\n",
      "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n",
      "\n",
      "\n",
      "\n",
      "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n",
      "\n",
      "syslog:x:101:104::/home/syslog:/bin/false\n",
      "\n",
      "messagebus:x:102:106::/var/run/dbus:/bin/false\n",
      "\n",
      "landscape:x:103:109::/var/lib/landscape:/bin/false\n",
      "\n",
      "jci:x:955:955::/home/jci:/bin/bash\n",
      "\n",
      "sshd:x:104:65534::/var/run/sshd:/usr/sbin/nologin\n",
      "\n",
      "user:x:1000:1000:user,,,:/home/user:/bin/bash\n",
      "\n",
      "reuven:x:1001:1001:Reuven M. Lerner,,,:/home/reuven:/bin/bash\n",
      "\n",
      "postfix:x:105:113::/var/spool/postfix:/bin/false\n",
      "\n",
      "colord:x:106:116:colord colour management daemon,,,:/var/lib/colord:/bin/false\n",
      "\n",
      "postgres:x:107:117:PostgreSQL administrator,,,:/var/lib/postgresql:/bin/bash\n",
      "\n",
      "dovecot:x:108:119:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false\n",
      "\n",
      "dovenull:x:109:120:Dovecot login user,,,:/nonexistent:/bin/false\n",
      "\n",
      "postgrey:x:110:121::/var/lib/postgrey:/bin/false\n",
      "\n",
      "debian-spamd:x:111:122::/var/lib/spamassassin:/bin/sh\n",
      "\n",
      "memcache:x:113:124:Memcached,,,:/nonexistent:/bin/false\n",
      "\n",
      "genadi:x:1002:1003:Genadi Reznichenko,,,:/home/genadi:/bin/bash\n",
      "\n",
      "shira:x:1003:1004:Shira Friedman,,,:/home/shira:/bin/bash\n",
      "\n",
      "atara:x:1004:1005:Atara Lerner-Friedman,,,:/home/atara:/bin/bash\n",
      "\n",
      "shikma:x:1005:1006:Shikma Lerner-Friedman,,,:/home/shikma:/bin/bash\n",
      "\n",
      "amotz:x:1006:1007:Amotz Lerner-Friedman,,,:/home/amotz:/bin/bash\n",
      "\n",
      "mysql:x:114:125:MySQL Server,,,:/nonexistent:/bin/false\n",
      "\n",
      "clamav:x:115:126::/var/lib/clamav:/bin/false\n",
      "\n",
      "amavis:x:116:127:AMaViS system user,,,:/var/lib/amavis:/bin/sh\n",
      "\n",
      "opendkim:x:117:128::/var/run/opendkim:/bin/false\n",
      "\n",
      "gitlab-redis:x:999:1009::/var/opt/gitlab/redis:/bin/nologin\n",
      "\n",
      "gitlab-psql:x:998:1010::/var/opt/gitlab/postgresql:/bin/sh\n",
      "\n",
      "git:x:1007:1008:GitLab,,,:/home/git:/bin/bash\n",
      "\n",
      "opendmarc:x:118:129::/var/run/opendmarc:/bin/false\n",
      "\n",
      "dkim-milter-python:x:119:130::/var/run/dkim-milter-python:/bin/false\n",
      "\n",
      "deploy:x:1008:1011:Deploy,,,:/home/deploy:/bin/bash\n",
      "\n",
      "redis:x:112:123:redis server,,,:/var/lib/redis:/bin/false\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# if we aren't going to use the read() method, then how can we read from our file\n",
    "# in a safe way?\n",
    "\n",
    "# answer: for loops!\n",
    "\n",
    "# if you iterate over a file object, you'll get the file, one line at a time. Each iteration\n",
    "# will give you the next line in the file, up to and including its trailing \\n character.\n",
    "\n",
    "f = open('linux-etc-passwd.txt')\n",
    "for one_line in f:\n",
    "    print(one_line)  # one \\n from print, another \\n from the line itself\n",
    "f.close()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "id": "434d78e3-55d9-48dc-8a64-4d2bbc67c534",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# This is a comment\n",
      "# You should ignore me\n",
      "root:x:0:0:root:/root:/bin/bash\n",
      "daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n",
      "bin:x:2:2:bin:/bin:/usr/sbin/nologin\n",
      "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n",
      "sync:x:4:65534:sync:/bin:/bin/sync\n",
      "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n",
      "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n",
      "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n",
      "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n",
      "\n",
      "\n",
      "\n",
      "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n",
      "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n",
      "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n",
      "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n",
      "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n",
      "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n",
      "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n",
      "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n",
      "\n",
      "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n",
      "syslog:x:101:104::/home/syslog:/bin/false\n",
      "messagebus:x:102:106::/var/run/dbus:/bin/false\n",
      "landscape:x:103:109::/var/lib/landscape:/bin/false\n",
      "jci:x:955:955::/home/jci:/bin/bash\n",
      "sshd:x:104:65534::/var/run/sshd:/usr/sbin/nologin\n",
      "user:x:1000:1000:user,,,:/home/user:/bin/bash\n",
      "reuven:x:1001:1001:Reuven M. Lerner,,,:/home/reuven:/bin/bash\n",
      "postfix:x:105:113::/var/spool/postfix:/bin/false\n",
      "colord:x:106:116:colord colour management daemon,,,:/var/lib/colord:/bin/false\n",
      "postgres:x:107:117:PostgreSQL administrator,,,:/var/lib/postgresql:/bin/bash\n",
      "dovecot:x:108:119:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false\n",
      "dovenull:x:109:120:Dovecot login user,,,:/nonexistent:/bin/false\n",
      "postgrey:x:110:121::/var/lib/postgrey:/bin/false\n",
      "debian-spamd:x:111:122::/var/lib/spamassassin:/bin/sh\n",
      "memcache:x:113:124:Memcached,,,:/nonexistent:/bin/false\n",
      "genadi:x:1002:1003:Genadi Reznichenko,,,:/home/genadi:/bin/bash\n",
      "shira:x:1003:1004:Shira Friedman,,,:/home/shira:/bin/bash\n",
      "atara:x:1004:1005:Atara Lerner-Friedman,,,:/home/atara:/bin/bash\n",
      "shikma:x:1005:1006:Shikma Lerner-Friedman,,,:/home/shikma:/bin/bash\n",
      "amotz:x:1006:1007:Amotz Lerner-Friedman,,,:/home/amotz:/bin/bash\n",
      "mysql:x:114:125:MySQL Server,,,:/nonexistent:/bin/false\n",
      "clamav:x:115:126::/var/lib/clamav:/bin/false\n",
      "amavis:x:116:127:AMaViS system user,,,:/var/lib/amavis:/bin/sh\n",
      "opendkim:x:117:128::/var/run/opendkim:/bin/false\n",
      "gitlab-redis:x:999:1009::/var/opt/gitlab/redis:/bin/nologin\n",
      "gitlab-psql:x:998:1010::/var/opt/gitlab/postgresql:/bin/sh\n",
      "git:x:1007:1008:GitLab,,,:/home/git:/bin/bash\n",
      "opendmarc:x:118:129::/var/run/opendmarc:/bin/false\n",
      "dkim-milter-python:x:119:130::/var/run/dkim-milter-python:/bin/false\n",
      "deploy:x:1008:1011:Deploy,,,:/home/deploy:/bin/bash\n",
      "redis:x:112:123:redis server,,,:/var/lib/redis:/bin/false\n"
     ]
    }
   ],
   "source": [
    "\n",
    "f = open('linux-etc-passwd.txt')\n",
    "for one_line in f:\n",
    "    print(one_line.strip())   # use strip to remove whitespace, then we only have \\n from print\n",
    "f.close()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "id": "175cde8c-6f42-4cbc-800e-0750ceeb9ce3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# This is a comment\n",
      "# You should ignore me\n",
      "root:x:0:0:root:/root:/bin/bash\n",
      "daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n",
      "bin:x:2:2:bin:/bin:/usr/sbin/nologin\n",
      "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n",
      "sync:x:4:65534:sync:/bin:/bin/sync\n",
      "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n",
      "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n",
      "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n",
      "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n",
      "\n",
      "\n",
      "\n",
      "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n",
      "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n",
      "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n",
      "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n",
      "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n",
      "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n",
      "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n",
      "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n",
      "\n",
      "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n",
      "syslog:x:101:104::/home/syslog:/bin/false\n",
      "messagebus:x:102:106::/var/run/dbus:/bin/false\n",
      "landscape:x:103:109::/var/lib/landscape:/bin/false\n",
      "jci:x:955:955::/home/jci:/bin/bash\n",
      "sshd:x:104:65534::/var/run/sshd:/usr/sbin/nologin\n",
      "user:x:1000:1000:user,,,:/home/user:/bin/bash\n",
      "reuven:x:1001:1001:Reuven M. Lerner,,,:/home/reuven:/bin/bash\n",
      "postfix:x:105:113::/var/spool/postfix:/bin/false\n",
      "colord:x:106:116:colord colour management daemon,,,:/var/lib/colord:/bin/false\n",
      "postgres:x:107:117:PostgreSQL administrator,,,:/var/lib/postgresql:/bin/bash\n",
      "dovecot:x:108:119:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false\n",
      "dovenull:x:109:120:Dovecot login user,,,:/nonexistent:/bin/false\n",
      "postgrey:x:110:121::/var/lib/postgrey:/bin/false\n",
      "debian-spamd:x:111:122::/var/lib/spamassassin:/bin/sh\n",
      "memcache:x:113:124:Memcached,,,:/nonexistent:/bin/false\n",
      "genadi:x:1002:1003:Genadi Reznichenko,,,:/home/genadi:/bin/bash\n",
      "shira:x:1003:1004:Shira Friedman,,,:/home/shira:/bin/bash\n",
      "atara:x:1004:1005:Atara Lerner-Friedman,,,:/home/atara:/bin/bash\n",
      "shikma:x:1005:1006:Shikma Lerner-Friedman,,,:/home/shikma:/bin/bash\n",
      "amotz:x:1006:1007:Amotz Lerner-Friedman,,,:/home/amotz:/bin/bash\n",
      "mysql:x:114:125:MySQL Server,,,:/nonexistent:/bin/false\n",
      "clamav:x:115:126::/var/lib/clamav:/bin/false\n",
      "amavis:x:116:127:AMaViS system user,,,:/var/lib/amavis:/bin/sh\n",
      "opendkim:x:117:128::/var/run/opendkim:/bin/false\n",
      "gitlab-redis:x:999:1009::/var/opt/gitlab/redis:/bin/nologin\n",
      "gitlab-psql:x:998:1010::/var/opt/gitlab/postgresql:/bin/sh\n",
      "git:x:1007:1008:GitLab,,,:/home/git:/bin/bash\n",
      "opendmarc:x:118:129::/var/run/opendmarc:/bin/false\n",
      "dkim-milter-python:x:119:130::/var/run/dkim-milter-python:/bin/false\n",
      "deploy:x:1008:1011:Deploy,,,:/home/deploy:/bin/bash\n",
      "redis:x:112:123:redis server,,,:/var/lib/redis:/bin/false\n"
     ]
    }
   ],
   "source": [
    "# let's now shorten it more:\n",
    "\n",
    "for one_line in open('linux-etc-passwd.txt'):\n",
    "    print(one_line.strip()) "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 107,
   "id": "b7b21565-81fd-4981-b8f0-ad2f81e1739d",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "root\n",
      "daemon\n",
      "bin\n",
      "sys\n",
      "sync\n",
      "games\n",
      "man\n",
      "lp\n",
      "mail\n",
      "news\n",
      "uucp\n",
      "proxy\n",
      "www-data\n",
      "backup\n",
      "list\n",
      "irc\n",
      "gnats\n",
      "nobody\n",
      "syslog\n",
      "messagebus\n",
      "landscape\n",
      "jci\n",
      "sshd\n",
      "user\n",
      "reuven\n",
      "postfix\n",
      "colord\n",
      "postgres\n",
      "dovecot\n",
      "dovenull\n",
      "postgrey\n",
      "debian-spamd\n",
      "memcache\n",
      "genadi\n",
      "shira\n",
      "atara\n",
      "shikma\n",
      "amotz\n",
      "mysql\n",
      "clamav\n",
      "amavis\n",
      "opendkim\n",
      "gitlab-redis\n",
      "gitlab-psql\n",
      "git\n",
      "opendmarc\n",
      "dkim-milter-python\n",
      "deploy\n",
      "redis\n"
     ]
    }
   ],
   "source": [
    "# how can I print only the usernames (i.e., the values before the first : on each line)\n",
    "# from this file?\n",
    "\n",
    "for one_line in open('linux-etc-passwd.txt'):  # file is in the same directory as Jupyter is running\n",
    "    if one_line[0] == '#':\n",
    "        continue     # go onto the next line if this line is a comment\n",
    "\n",
    "    if one_line.strip() == '':\n",
    "        continue     # go onto the next line if this line is empty\n",
    "    \n",
    "    # fields = one_line.split(':')   # this returns a list of strings\n",
    "    # print(fields[0])\n",
    "\n",
    "    print(one_line.split(':')[0])    # split the line, get a list based on it, and return index 0"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "id": "1c134943-048d-486c-9b40-5e92c6473773",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# This is a comment\n",
      "\n",
      "# You should ignore me\n",
      "\n",
      "root:x:0:0:root:/root:/bin/bash\n",
      "\n",
      "daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n",
      "\n",
      "bin:x:2:2:bin:/bin:/usr/sbin/nologin\n",
      "\n",
      "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n",
      "\n",
      "sync:x:4:65534:sync:/bin:/bin/sync\n",
      "\n",
      "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n",
      "\n",
      "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n",
      "\n",
      "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n",
      "\n",
      "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n",
      "\n",
      "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n",
      "\n",
      "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n",
      "\n",
      "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n",
      "\n",
      "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n",
      "\n",
      "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n",
      "\n",
      "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n",
      "\n",
      "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n",
      "\n",
      "\n",
      "\n",
      "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n",
      "\n",
      "syslog:x:101:104::/home/syslog:/bin/false\n",
      "\n",
      "messagebus:x:102:106::/var/run/dbus:/bin/false\n",
      "\n",
      "landscape:x:103:109::/var/lib/landscape:/bin/false\n",
      "\n",
      "jci:x:955:955::/home/jci:/bin/bash\n",
      "\n",
      "sshd:x:104:65534::/var/run/sshd:/usr/sbin/nologin\n",
      "\n",
      "user:x:1000:1000:user,,,:/home/user:/bin/bash\n",
      "\n",
      "reuven:x:1001:1001:Reuven M. Lerner,,,:/home/reuven:/bin/bash\n",
      "\n",
      "postfix:x:105:113::/var/spool/postfix:/bin/false\n",
      "\n",
      "colord:x:106:116:colord colour management daemon,,,:/var/lib/colord:/bin/false\n",
      "\n",
      "postgres:x:107:117:PostgreSQL administrator,,,:/var/lib/postgresql:/bin/bash\n",
      "\n",
      "dovecot:x:108:119:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false\n",
      "\n",
      "dovenull:x:109:120:Dovecot login user,,,:/nonexistent:/bin/false\n",
      "\n",
      "postgrey:x:110:121::/var/lib/postgrey:/bin/false\n",
      "\n",
      "debian-spamd:x:111:122::/var/lib/spamassassin:/bin/sh\n",
      "\n",
      "memcache:x:113:124:Memcached,,,:/nonexistent:/bin/false\n",
      "\n",
      "genadi:x:1002:1003:Genadi Reznichenko,,,:/home/genadi:/bin/bash\n",
      "\n",
      "shira:x:1003:1004:Shira Friedman,,,:/home/shira:/bin/bash\n",
      "\n",
      "atara:x:1004:1005:Atara Lerner-Friedman,,,:/home/atara:/bin/bash\n",
      "\n",
      "shikma:x:1005:1006:Shikma Lerner-Friedman,,,:/home/shikma:/bin/bash\n",
      "\n",
      "amotz:x:1006:1007:Amotz Lerner-Friedman,,,:/home/amotz:/bin/bash\n",
      "\n",
      "mysql:x:114:125:MySQL Server,,,:/nonexistent:/bin/false\n",
      "\n",
      "clamav:x:115:126::/var/lib/clamav:/bin/false\n",
      "\n",
      "amavis:x:116:127:AMaViS system user,,,:/var/lib/amavis:/bin/sh\n",
      "\n",
      "opendkim:x:117:128::/var/run/opendkim:/bin/false\n",
      "\n",
      "gitlab-redis:x:999:1009::/var/opt/gitlab/redis:/bin/nologin\n",
      "\n",
      "gitlab-psql:x:998:1010::/var/opt/gitlab/postgresql:/bin/sh\n",
      "\n",
      "git:x:1007:1008:GitLab,,,:/home/git:/bin/bash\n",
      "\n",
      "opendmarc:x:118:129::/var/run/opendmarc:/bin/false\n",
      "\n",
      "dkim-milter-python:x:119:130::/var/run/dkim-milter-python:/bin/false\n",
      "\n",
      "deploy:x:1008:1011:Deploy,,,:/home/deploy:/bin/bash\n",
      "\n",
      "redis:x:112:123:redis server,,,:/var/lib/redis:/bin/false\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# what does \"with\" do? \n",
    "# let's look at some code for opening a file:\n",
    "\n",
    "f = open('linux-etc-passwd.txt')\n",
    "for one_line in f:\n",
    "    print(one_line)  # one \\n from print, another \\n from the line itself\n",
    "f.close()    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "id": "c4ba5456-5a36-4574-8da5-21fcbda061cf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "# This is a comment\n",
      "\n",
      "# You should ignore me\n",
      "\n",
      "root:x:0:0:root:/root:/bin/bash\n",
      "\n",
      "daemon:x:1:1:daemon:/usr/sbin:/usr/sbin/nologin\n",
      "\n",
      "bin:x:2:2:bin:/bin:/usr/sbin/nologin\n",
      "\n",
      "sys:x:3:3:sys:/dev:/usr/sbin/nologin\n",
      "\n",
      "sync:x:4:65534:sync:/bin:/bin/sync\n",
      "\n",
      "games:x:5:60:games:/usr/games:/usr/sbin/nologin\n",
      "\n",
      "man:x:6:12:man:/var/cache/man:/usr/sbin/nologin\n",
      "\n",
      "lp:x:7:7:lp:/var/spool/lpd:/usr/sbin/nologin\n",
      "\n",
      "mail:x:8:8:mail:/var/mail:/usr/sbin/nologin\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "\n",
      "news:x:9:9:news:/var/spool/news:/usr/sbin/nologin\n",
      "\n",
      "uucp:x:10:10:uucp:/var/spool/uucp:/usr/sbin/nologin\n",
      "\n",
      "proxy:x:13:13:proxy:/bin:/usr/sbin/nologin\n",
      "\n",
      "www-data:x:33:33:www-data:/var/www:/usr/sbin/nologin\n",
      "\n",
      "backup:x:34:34:backup:/var/backups:/usr/sbin/nologin\n",
      "\n",
      "list:x:38:38:Mailing List Manager:/var/list:/usr/sbin/nologin\n",
      "\n",
      "irc:x:39:39:ircd:/var/run/ircd:/usr/sbin/nologin\n",
      "\n",
      "gnats:x:41:41:Gnats Bug-Reporting System (admin):/var/lib/gnats:/usr/sbin/nologin\n",
      "\n",
      "\n",
      "\n",
      "nobody:x:65534:65534:nobody:/nonexistent:/usr/sbin/nologin\n",
      "\n",
      "syslog:x:101:104::/home/syslog:/bin/false\n",
      "\n",
      "messagebus:x:102:106::/var/run/dbus:/bin/false\n",
      "\n",
      "landscape:x:103:109::/var/lib/landscape:/bin/false\n",
      "\n",
      "jci:x:955:955::/home/jci:/bin/bash\n",
      "\n",
      "sshd:x:104:65534::/var/run/sshd:/usr/sbin/nologin\n",
      "\n",
      "user:x:1000:1000:user,,,:/home/user:/bin/bash\n",
      "\n",
      "reuven:x:1001:1001:Reuven M. Lerner,,,:/home/reuven:/bin/bash\n",
      "\n",
      "postfix:x:105:113::/var/spool/postfix:/bin/false\n",
      "\n",
      "colord:x:106:116:colord colour management daemon,,,:/var/lib/colord:/bin/false\n",
      "\n",
      "postgres:x:107:117:PostgreSQL administrator,,,:/var/lib/postgresql:/bin/bash\n",
      "\n",
      "dovecot:x:108:119:Dovecot mail server,,,:/usr/lib/dovecot:/bin/false\n",
      "\n",
      "dovenull:x:109:120:Dovecot login user,,,:/nonexistent:/bin/false\n",
      "\n",
      "postgrey:x:110:121::/var/lib/postgrey:/bin/false\n",
      "\n",
      "debian-spamd:x:111:122::/var/lib/spamassassin:/bin/sh\n",
      "\n",
      "memcache:x:113:124:Memcached,,,:/nonexistent:/bin/false\n",
      "\n",
      "genadi:x:1002:1003:Genadi Reznichenko,,,:/home/genadi:/bin/bash\n",
      "\n",
      "shira:x:1003:1004:Shira Friedman,,,:/home/shira:/bin/bash\n",
      "\n",
      "atara:x:1004:1005:Atara Lerner-Friedman,,,:/home/atara:/bin/bash\n",
      "\n",
      "shikma:x:1005:1006:Shikma Lerner-Friedman,,,:/home/shikma:/bin/bash\n",
      "\n",
      "amotz:x:1006:1007:Amotz Lerner-Friedman,,,:/home/amotz:/bin/bash\n",
      "\n",
      "mysql:x:114:125:MySQL Server,,,:/nonexistent:/bin/false\n",
      "\n",
      "clamav:x:115:126::/var/lib/clamav:/bin/false\n",
      "\n",
      "amavis:x:116:127:AMaViS system user,,,:/var/lib/amavis:/bin/sh\n",
      "\n",
      "opendkim:x:117:128::/var/run/opendkim:/bin/false\n",
      "\n",
      "gitlab-redis:x:999:1009::/var/opt/gitlab/redis:/bin/nologin\n",
      "\n",
      "gitlab-psql:x:998:1010::/var/opt/gitlab/postgresql:/bin/sh\n",
      "\n",
      "git:x:1007:1008:GitLab,,,:/home/git:/bin/bash\n",
      "\n",
      "opendmarc:x:118:129::/var/run/opendmarc:/bin/false\n",
      "\n",
      "dkim-milter-python:x:119:130::/var/run/dkim-milter-python:/bin/false\n",
      "\n",
      "deploy:x:1008:1011:Deploy,,,:/home/deploy:/bin/bash\n",
      "\n",
      "redis:x:112:123:redis server,,,:/var/lib/redis:/bin/false\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# let's switch to using \"with\"\n",
    "# here's the above code using \"with\"\n",
    "\n",
    "# (1) we say with OBJECT as VARIABLE\n",
    "# (2) everything you want to do with the file is indented, in the block\n",
    "# (3) when you exit the \"with\" block, the file is automatically closed -- no need to do it yourself\n",
    "\n",
    "with open('linux-etc-passwd.txt') as f:\n",
    "    for one_line in f:\n",
    "        print(one_line)  # one \\n from print, another \\n from the line itself"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "id": "b047175d-4e80-47af-ad66-17064ba31d04",
   "metadata": {},
   "outputs": [
    {
     "ename": "FileNotFoundError",
     "evalue": "[Errno 2] No such file or directory: 'C:\\\\Users\\\\sXXXX\\\\Desktop\\\\linux-etc-passwd.txt'",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mFileNotFoundError\u001b[0m                         Traceback (most recent call last)",
      "Cell \u001b[0;32mIn [111], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[38;5;66;03m# put r before the opening quote and all backslashes will be doubled\u001b[39;00m\n\u001b[0;32m----> 2\u001b[0m f \u001b[38;5;241m=\u001b[39m \u001b[38;5;28;43mopen\u001b[39;49m\u001b[43m(\u001b[49m\u001b[38;5;124;43mr\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[38;5;124;43mC:\u001b[39;49m\u001b[38;5;124;43m\\\u001b[39;49m\u001b[38;5;124;43mUsers\u001b[39;49m\u001b[38;5;124;43m\\\u001b[39;49m\u001b[38;5;124;43msXXXX\u001b[39;49m\u001b[38;5;124;43m\\\u001b[39;49m\u001b[38;5;124;43mDesktop\u001b[39;49m\u001b[38;5;124;43m\\\u001b[39;49m\u001b[38;5;124;43mlinux-etc-passwd.txt\u001b[39;49m\u001b[38;5;124;43m\"\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[1;32m      3\u001b[0m \u001b[38;5;28mprint\u001b[39m(f\u001b[38;5;241m.\u001b[39mread())\n\u001b[1;32m      4\u001b[0m f\u001b[38;5;241m.\u001b[39mclose    \n",
      "\u001b[0;31mFileNotFoundError\u001b[0m: [Errno 2] No such file or directory: 'C:\\\\Users\\\\sXXXX\\\\Desktop\\\\linux-etc-passwd.txt'"
     ]
    }
   ],
   "source": [
    "# put r before the opening quote and all backslashes will be doubled\n",
    "f = open(r\"C:\\Users\\sXXXX\\Desktop\\linux-etc-passwd.txt\")\n",
    "print(f.read())\n",
    "f.close    "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5bfbbbc4-e79a-4283-8c19-e08a618c4191",
   "metadata": {},
   "source": [
    "# Next up\n",
    "\n",
    "1. Practice with reading files\n",
    "2. Make some reports from files\n",
    "3. A little of writing to files"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "id": "0d96b741-fda5-4a83-b541-934644306e42",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "linux-etc-passwd.txt  mini-access-log.txt  nums.txt  shoe-data.txt  wcfile.txt\n"
     ]
    }
   ],
   "source": [
    "!ls *.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9804f80c-cf60-417c-a20a-62b2479c652f",
   "metadata": {},
   "source": [
    "# Exercise: Sum numbers\n",
    "\n",
    "1. The file `nums.txt` contains about 10 lines in it.  Each line could be blank, or could contain an integer. If it contains an integer, there might be whitespace (spaces, tabs, etc.) around it.\n",
    "2. Set `total` to 0.\n",
    "3. Go through `nums.txt`, one line at a time.\n",
    "    - If there is only whitespace, then ignore the line\n",
    "    - If there is an integer (with potential whitespace around it), then invoke `int` on the line, and add it to `total`.\n",
    "4. Print `total`, which should be 83."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "id": "8d51bb73-0591-4d86-8d35-8bd3b62febdf",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5\n",
      "\t10     \n",
      "\t20\n",
      "  \t3\n",
      "\t\t   \t20        \n",
      "\n",
      " 25\n"
     ]
    }
   ],
   "source": [
    "!cat nums.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 120,
   "id": "b33b6521-4ca0-4e85-8da2-b25e006fc838",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "83\n"
     ]
    }
   ],
   "source": [
    "total = 0\n",
    "\n",
    "for one_line in open('nums.txt'):\n",
    "    s = one_line.strip()     # remove the outer whitespace from the string, assign to s\n",
    "    if s.isdigit():          # does s only contain digits?\n",
    "        total += int(s)      # if so, turn it into an int and add to total\n",
    "\n",
    "print(total)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 119,
   "id": "424b2bf5-f287-4253-a401-54c0a25f27ed",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "83\n"
     ]
    }
   ],
   "source": [
    "total = 0\n",
    "\n",
    "for one_line in open('nums.txt'):\n",
    "    if one_line.strip().isdigit():\n",
    "        total += int(one_line)\n",
    "print(total)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "id": "dfc22fe0-3d79-484b-b2a1-33fd72423899",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'a     b     c'"
      ]
     },
     "execution_count": 121,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "s = '    a     b     c      '\n",
    "\n",
    "s.strip()   # this will return a new string, without s's leading/trailing whitespace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "id": "84f9d594-63db-482f-9117-e7ce73822f34",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'abc'"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# what if I want to remove all whitespace, not just on the outside?\n",
    "# then I can use str.replace(' ', '')\n",
    "\n",
    "s.replace(' ', '')   # every time you see ' ', replace it with ''"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 124,
   "id": "b54b2d4e-36c9-4b2d-a7ad-7920c9ceb5d4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67.218.116.165 - - [30/Jan/2010:00:03:18 +0200] \"GET /robots.txt HTTP/1.0\" 200 99 \"-\" \"Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html)\"\n",
      "66.249.71.65 - - [30/Jan/2010:00:12:06 +0200] \"GET /browse/one_node/1557 HTTP/1.1\" 200 39208 \"-\" \"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)\"\n",
      "65.55.106.183 - - [30/Jan/2010:01:29:23 +0200] \"GET /robots.txt HTTP/1.1\" 200 99 \"-\" \"msnbot/2.0b (+http://search.msn.com/msnbot.htm)\"\n",
      "65.55.106.183 - - [30/Jan/2010:01:30:06 +0200] \"GET /browse/one_model/2162 HTTP/1.1\" 200 2181 \"-\" \"msnbot/2.0b (+http://search.msn.com/msnbot.htm)\"\n",
      "66.249.71.65 - - [30/Jan/2010:02:07:14 +0200] \"GET /browse/browse_applet_tab/2593 HTTP/1.1\" 200 10305 \"-\" \"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)\"\n",
      "66.249.71.65 - - [30/Jan/2010:02:10:39 +0200] \"GET /browse/browse_files_tab/2499?tab=true HTTP/1.1\" 200 446 \"-\" \"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)\"\n",
      "66.249.65.12 - - [30/Jan/2010:03:13:34 +0200] \"GET /robots.txt HTTP/1.1\" 200 99 \"-\" \"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)\"\n",
      "66.249.65.12 - - [30/Jan/2010:03:13:34 +0200] \"GET /browse/one_node/2715 HTTP/1.1\" 200 26433 \"-\" \"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)\"\n",
      "66.249.65.12 - - [30/Jan/2010:03:43:39 +0200] \"GET /browse/download_model/1969 HTTP/1.1\" 200 31713 \"-\" \"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)\"\n",
      "66.249.65.12 - - [30/Jan/2010:04:05:43 +0200] \"GET /browse/one_node/1406 HTTP/1.1\" 302 118 \"-\" \"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)\"\n"
     ]
    }
   ],
   "source": [
    "!head mini-access-log.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3aa8f7f-8307-4584-9319-706ffad8b012",
   "metadata": {},
   "source": [
    "# Exercise: Count IP addresses\n",
    "\n",
    "The goal of this exercise is to create a dict whose keys are IP addresses (strings), the numbers that represent locations on the Internet.  The values will be integers, telling us how many times each address appears in `mini-access-log.txt`.\n",
    "\n",
    "1. Create an empty dict called `counts`.\n",
    "2. Go through `mini-access-log.txt` one line at a time in a `for` loop.\n",
    "3. Retrieve the IP address from the current line. Examine the lines to understand how you might do that.\n",
    "4. If the address is new, add a new key-value pair to `counts`.\n",
    "5. If the address is already in `counts`, increase the count by 1.\n",
    "6. Use a `for` loop to iterate over `counts`, showing each IP address and how often it appeared in the file."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "id": "5caba191-d9f6-4003-8814-6ccecda542c8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'/Users/reuven/Courses/Current/oreilly-2023-07July'"
      ]
     },
     "execution_count": 125,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# the current directory is wherever you ran Jupyter\n",
    "# you can get it with the magic %pwd command\n",
    "\n",
    "%pwd"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 135,
   "id": "0604c57c-bba8-4088-ad76-783a2ff9e644",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67.218.116.165 - - [30/Jan/2010:00:03:18 +0200] \"GET /robots.txt HTTP/1.0\" 200 99 \"-\" \"Mozilla/5.0 (Twiceler-0.9 http://www.cuil.com/twiceler/robot.html)\"\n",
      "66.249.71.65 - - [30/Jan/2010:00:12:06 +0200] \"GET /browse/one_node/1557 HTTP/1.1\" 200 39208 \"-\" \"Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)\"\n",
      "65.55.106.183 - - [30/Jan/2010:01:29:23 +0200] \"GET /robots.txt HTTP/1.1\" 200 99 \"-\" \"msnbot/2.0b (+http://search.msn.com/msnbot.htm)\"\n",
      "65.55.106.183 - - [30/Jan/2010:01:30:06 +0200] \"GET /browse/one_model/2162 HTTP/1.1\" 200 2181 \"-\" \"msnbot/2.0b (+http://search.msn.com/msnbot.htm)\"\n"
     ]
    }
   ],
   "source": [
    "!head -4 mini-access-log.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 129,
   "id": "66b672b1-906c-47cb-b85b-e56230ce9e7d",
   "metadata": {},
   "outputs": [],
   "source": [
    "counts = {}   # new, empty dict\n",
    "\n",
    "for one_line in open('mini-access-log.txt'):\n",
    "    ip_address = one_line.split()[0]\n",
    "\n",
    "    if ip_address in counts:     # have we already seen this IP address?\n",
    "        counts[ip_address] += 1\n",
    "    else:\n",
    "        counts[ip_address] = 1"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 131,
   "id": "9e1170ef-0811-4f90-a3f0-53a65cc8b63b",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "67.218.116.165: 2\n",
      "66.249.71.65: 3\n",
      "65.55.106.183: 2\n",
      "66.249.65.12: 32\n",
      "65.55.106.131: 2\n",
      "65.55.106.186: 2\n",
      "74.52.245.146: 2\n",
      "66.249.65.43: 3\n",
      "65.55.207.25: 2\n",
      "65.55.207.94: 2\n",
      "65.55.207.71: 1\n",
      "98.242.170.241: 1\n",
      "66.249.65.38: 100\n",
      "65.55.207.126: 2\n",
      "82.34.9.20: 2\n",
      "65.55.106.155: 2\n",
      "65.55.207.77: 2\n",
      "208.80.193.28: 1\n",
      "89.248.172.58: 22\n",
      "67.195.112.35: 16\n",
      "65.55.207.50: 3\n",
      "65.55.215.75: 2\n"
     ]
    }
   ],
   "source": [
    "for key, value in counts.items():\n",
    "    print(f'{key}: {value}')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "82502261-a05d-49b0-a7dc-2d5d97acf5a3",
   "metadata": {},
   "source": [
    "# How I approached this problem:\n",
    "\n",
    "1. I saw that each line in the file starts with an IP address, followed by a space. Everything, from that space onwards in each line, can be thrown away. I just care about the IP address.\n",
    "2. How can I get the IP address, if it is before the first space on each line? I can use split, which returns a list of strings based on the line. If I call `split()`, it'll use the whitespace, and the resulting list's index 0 will be the IP address.\n",
    "3. I then take the IP address I found and stick it into `ip_address`\n",
    "4. Then I check to see if `ip_address` is already a key in `counts`\n",
    "    - If so, I can just increment the count by 1\n",
    "    - If not, then I add a new key-value pair, with a count of 1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 132,
   "id": "7b444140-6c4c-45b8-a790-ba931feb0ac4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# if you want to see how to sort a dictionary (or anything else), \n",
    "# I gave a talk at Euro Python 2020 called \"How to sort anything\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 134,
   "id": "708ef24c-909c-4b70-8bdb-aa3868db97bd",
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['67.218.116.165', '-', '-', '[30/Jan/2010:00:03:18', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.0\"', '200', '99', '\"-\"', '\"Mozilla/5.0', '(Twiceler-0.9', 'http://www.cuil.com/twiceler/robot.html)\"']\n",
      "['66.249.71.65', '-', '-', '[30/Jan/2010:00:12:06', '+0200]', '\"GET', '/browse/one_node/1557', 'HTTP/1.1\"', '200', '39208', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['65.55.106.183', '-', '-', '[30/Jan/2010:01:29:23', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['65.55.106.183', '-', '-', '[30/Jan/2010:01:30:06', '+0200]', '\"GET', '/browse/one_model/2162', 'HTTP/1.1\"', '200', '2181', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['66.249.71.65', '-', '-', '[30/Jan/2010:02:07:14', '+0200]', '\"GET', '/browse/browse_applet_tab/2593', 'HTTP/1.1\"', '200', '10305', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.71.65', '-', '-', '[30/Jan/2010:02:10:39', '+0200]', '\"GET', '/browse/browse_files_tab/2499?tab=true', 'HTTP/1.1\"', '200', '446', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:03:13:34', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:03:13:34', '+0200]', '\"GET', '/browse/one_node/2715', 'HTTP/1.1\"', '200', '26433', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:03:43:39', '+0200]', '\"GET', '/browse/download_model/1969', 'HTTP/1.1\"', '200', '31713', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:04:05:43', '+0200]', '\"GET', '/browse/one_node/1406', 'HTTP/1.1\"', '302', '118', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:04:05:51', '+0200]', '\"GET', '/browse/one_model/1406', 'HTTP/1.1\"', '200', '2179', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['65.55.106.131', '-', '-', '[30/Jan/2010:04:24:33', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['65.55.106.131', '-', '-', '[30/Jan/2010:04:25:36', '+0200]', '\"GET', '/browse/one_model/1200', 'HTTP/1.1\"', '200', '2180', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:04:34:36', '+0200]', '\"GET', '/browse/download_model/2170', 'HTTP/1.1\"', '200', '11772', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:04:39:37', '+0200]', '\"GET', '/browse/browse_preview_tab/2499?tab=true', 'HTTP/1.1\"', '200', '452', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:05:03:34', '+0200]', '\"GET', '/browse/download_model/2244', 'HTTP/1.1\"', '200', '7215', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:05:32:31', '+0200]', '\"GET', '/browse/download_model/1002', 'HTTP/1.1\"', '200', '3658', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:06:01:22', '+0200]', '\"GET', '/browse/download_model/1200', 'HTTP/1.1\"', '200', '31950', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:06:30:19', '+0200]', '\"GET', '/browse/download_model/2463', 'HTTP/1.1\"', '200', '3357', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:06:59:14', '+0200]', '\"GET', '/browse/download_model/1861', 'HTTP/1.1\"', '200', '5136', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['65.55.106.186', '-', '-', '[30/Jan/2010:07:07:13', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['65.55.106.186', '-', '-', '[30/Jan/2010:07:08:06', '+0200]', '\"GET', '/browse/about', 'HTTP/1.1\"', '200', '2315', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:07:28:09', '+0200]', '\"GET', '/browse/download_model/1712', 'HTTP/1.1\"', '200', '11605', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:07:57:06', '+0200]', '\"GET', '/browse/download_model/1192', 'HTTP/1.1\"', '200', '31656', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:08:26:00', '+0200]', '\"GET', '/browse/download_model/1140', 'HTTP/1.1\"', '200', '4370', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['74.52.245.146', '-', '-', '[30/Jan/2010:08:40:26', '+0200]', '\"GET', '/', 'HTTP/1.0\"', '302', '110', '\"-\"', '\"Mozilla/5.0', '(X11;', 'U;', 'Linux', 'i686;', 'en-US;', 'rv:1.7.5)', 'Gecko/20041107', 'Firefox/1.0\"']\n",
      "['74.52.245.146', '-', '-', '[30/Jan/2010:08:40:26', '+0200]', '\"GET', '/account/login', 'HTTP/1.0\"', '200', '5801', '\"-\"', '\"Mozilla/5.0', '(X11;', 'U;', 'Linux', 'i686;', 'en-US;', 'rv:1.7.5)', 'Gecko/20041107', 'Firefox/1.0\"']\n",
      "['66.249.65.43', '-', '-', '[30/Jan/2010:08:44:51', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.43', '-', '-', '[30/Jan/2010:08:44:52', '+0200]', '\"GET', '/', 'HTTP/1.1\"', '302', '112', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.43', '-', '-', '[30/Jan/2010:08:44:52', '+0200]', '\"GET', '/account/login', 'HTTP/1.1\"', '200', '1935', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:08:54:53', '+0200]', '\"GET', '/browse/download_model/2152', 'HTTP/1.1\"', '200', '7285', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:09:23:46', '+0200]', '\"GET', '/browse/download_model/796', 'HTTP/1.1\"', '200', '3241', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:09:52:42', '+0200]', '\"GET', '/browse/one_node/2177', 'HTTP/1.1\"', '200', '1556', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:10:21:43', '+0200]', '\"GET', '/browse/one_node/2163', 'HTTP/1.1\"', '200', '4124', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:10:50:35', '+0200]', '\"GET', '/browse/one_node/1193', 'HTTP/1.1\"', '200', '29168', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:11:19:27', '+0200]', '\"GET', '/browse/one_node/1241', 'HTTP/1.1\"', '200', '7032', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['65.55.207.25', '-', '-', '[30/Jan/2010:11:43:56', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['65.55.207.25', '-', '-', '[30/Jan/2010:11:44:35', '+0200]', '\"GET', '/help', 'HTTP/1.1\"', '304', '-', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:11:48:31', '+0200]', '\"GET', '/browse/download_model/2508', 'HTTP/1.1\"', '200', '11374', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:12:17:24', '+0200]', '\"GET', '/browse/one_node/2509', 'HTTP/1.1\"', '200', '4766', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:12:46:17', '+0200]', '\"GET', '/browse/one_node/1233', 'HTTP/1.1\"', '200', '79301', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:13:15:09', '+0200]', '\"GET', '/browse/download_model/2508?version=1819', 'HTTP/1.1\"', '200', '11374', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:13:44:01', '+0200]', '\"GET', '/browse/download_model/2508?version=1821', 'HTTP/1.1\"', '200', '11374', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:14:12:55', '+0200]', '\"GET', '/browse/download_model/902', 'HTTP/1.1\"', '200', '5050', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:14:42:01', '+0200]', '\"GET', '/browse/download_model/1679', 'HTTP/1.1\"', '200', '11290', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['65.55.207.94', '-', '-', '[30/Jan/2010:15:02:52', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['65.55.207.94', '-', '-', '[30/Jan/2010:15:03:36', '+0200]', '\"GET', '/browse/one_model/900', 'HTTP/1.1\"', '200', '2165', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:15:10:51', '+0200]', '\"GET', '/browse/download_model/1612', 'HTTP/1.1\"', '200', '30540', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['65.55.207.71', '-', '-', '[30/Jan/2010:15:32:04', '+0200]', '\"GET', '/account/new', 'HTTP/1.1\"', '304', '-', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:15:39:45', '+0200]', '\"GET', '/browse/download_model/892', 'HTTP/1.1\"', '200', '4195', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:15:52:21', '+0200]', '\"GET', '/system/avatars/7/thumb/fractal-tree-on-transparent-sm.png?1237131134', 'HTTP/1.1\"', '304', '-', '\"-\"', '\"Googlebot-Image/1.0\"']\n",
      "['66.249.65.12', '-', '-', '[30/Jan/2010:16:08:36', '+0200]', '\"GET', '/browse/download_model/1865', 'HTTP/1.1\"', '200', '19432', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['98.242.170.241', '-', '-', '[30/Jan/2010:16:10:49', '+0200]', '\"GET', '/dvdrental.com', 'HTTP/1.1\"', '404', '947', '\"http://www.dvdrental.com/modelingcommons.org\"', '\"Opera/9.80', '(Windows', 'NT', '5.2;', 'U;', 'en)', 'Presto/2.2.15', 'Version/10.10\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:16:37:33', '+0200]', '\"GET', '/browse/download_model/914', 'HTTP/1.1\"', '200', '6181', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:16:45:12', '+0200]', '\"GET', '/browse/browse_applet_tab/2593?tab=true', 'HTTP/1.1\"', '200', '10237', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:28', '+0200]', '\"GET', '/javascripts/application.js?1264605648', 'HTTP/1.1\"', '200', '698', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:29', '+0200]', '\"GET', '/stylesheets/browse.css?1264605648', 'HTTP/1.1\"', '200', '422', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:30', '+0200]', '\"GET', '/javascripts/jquery.livequery.js?1264605648', 'HTTP/1.1\"', '200', '2135', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:30', '+0200]', '\"GET', '/javascripts/jrails.js?1264605648', 'HTTP/1.1\"', '200', '1093', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:31', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.all.css?1264605648', 'HTTP/1.1\"', '200', '52', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:31', '+0200]', '\"GET', '/stylesheets/jquery.autocomplete.css?1264605648', 'HTTP/1.1\"', '200', '447', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:32', '+0200]', '\"GET', '/stylesheets/nlcommons.css?1264605648', 'HTTP/1.1\"', '200', '1179', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:32', '+0200]', '\"GET', '/javascripts/jquery.dataTables.js?1264605648', 'HTTP/1.1\"', '200', '15537', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:33', '+0200]', '\"GET', '/javascripts/jquery.js?1264605648', 'HTTP/1.1\"', '200', '19740', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:34', '+0200]', '\"GET', '/javascripts/nlcommons.js?1264605648', 'HTTP/1.1\"', '200', '20', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:34', '+0200]', '\"GET', '/javascripts/jquery-ui.js?1264605648', 'HTTP/1.1\"', '200', '72080', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:02:35', '+0200]', '\"GET', '/javascripts/jquery.autocomplete.js?1264605648', 'HTTP/1.1\"', '200', '6433', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:03:16', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.base.css', 'HTTP/1.1\"', '200', '111', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:03:16', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.theme.css', 'HTTP/1.1\"', '200', '2862', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:03:30', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.core.css', 'HTTP/1.1\"', '200', '624', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:03:31', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.accordion.css', 'HTTP/1.1\"', '200', '295', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:03:31', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.datepicker.css', 'HTTP/1.1\"', '200', '965', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:03:32', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.dialog.css', 'HTTP/1.1\"', '200', '442', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:03:32', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.slider.css', 'HTTP/1.1\"', '200', '318', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:03:33', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.progressbar.css', 'HTTP/1.1\"', '200', '114', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:03:34', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.tabs.css', 'HTTP/1.1\"', '200', '414', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:03:34', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.resizable.css', 'HTTP/1.1\"', '200', '307', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:06:29', '+0200]', '\"GET', '/browse/download_model/743', 'HTTP/1.1\"', '200', '10552', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:17:35:24', '+0200]', '\"GET', '/browse/download_model/2547', 'HTTP/1.1\"', '200', '40914', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['65.55.207.126', '-', '-', '[30/Jan/2010:17:46:11', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['65.55.207.126', '-', '-', '[30/Jan/2010:17:48:32', '+0200]', '\"GET', '/browse/one_model/1002', 'HTTP/1.1\"', '200', '2173', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:18:04:31', '+0200]', '\"GET', '/browse/download_model/1210', 'HTTP/1.1\"', '200', '11948', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:18:33:30', '+0200]', '\"GET', '/browse/download_model/849', 'HTTP/1.1\"', '200', '6836', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:18:53:53', '+0200]', '\"GET', '/tags/one_tag/27', 'HTTP/1.1\"', '200', '2033', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:19:02:20', '+0200]', '\"GET', '/browse/download_model/739', 'HTTP/1.1\"', '200', '11302', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['82.34.9.20', '-', '-', '[30/Jan/2010:19:18:24', '+0200]', '\"GET', '/account/models/33', 'HTTP/1.1\"', '200', '23774', '\"http://www.google.co.uk/search?hl=en&q=%22netlogo%22+sokoban&btnG=Search&meta=&aq=f&oq=\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '8.0;', 'Windows', 'NT', '5.1;', 'Trident/4.0;', '.NET', 'CLR', '2.0.50727;', '.NET', 'CLR', '3.0.4506.2152;', '.NET', 'CLR', '3.5.30729;', 'InfoPath.1)\"']\n",
      "['82.34.9.20', '-', '-', '[30/Jan/2010:19:18:18', '+0200]', '\"GET', '/account/models/33', 'HTTP/1.1\"', '200', '23774', '\"http://www.google.co.uk/search?hl=en&q=%22netlogo%22+sokoban&btnG=Search&meta=&aq=f&oq=\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '8.0;', 'Windows', 'NT', '5.1;', 'Trident/4.0;', '.NET', 'CLR', '2.0.50727;', '.NET', 'CLR', '3.0.4506.2152;', '.NET', 'CLR', '3.5.30729;', 'InfoPath.1)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:19:31:07', '+0200]', '\"GET', '/browse/download_model/838', 'HTTP/1.1\"', '200', '7691', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:20:00:20', '+0200]', '\"GET', '/browse/download_model/2316', 'HTTP/1.1\"', '200', '48709', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:20:28:55', '+0200]', '\"GET', '/browse/download_model/840', 'HTTP/1.1\"', '200', '6063', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:20:47:05', '+0200]', '\"GET', '/?id=7', 'HTTP/1.1\"', '200', '3058', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:20:57:43', '+0200]', '\"GET', '/browse/download_model/806', 'HTTP/1.1\"', '200', '4263', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:21:07:44', '+0200]', '\"GET', '/account/mypage/17', 'HTTP/1.1\"', '200', '3103', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:21:26:36', '+0200]', '\"GET', '/browse/download_model/1891', 'HTTP/1.1\"', '200', '8570', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:21:56:01', '+0200]', '\"GET', '/browse/download_model/1881', 'HTTP/1.1\"', '200', '9618', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:22:25:13', '+0200]', '\"GET', '/browse/download_model/735', 'HTTP/1.1\"', '200', '16848', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['65.55.106.155', '-', '-', '[30/Jan/2010:22:25:26', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['65.55.106.155', '-', '-', '[30/Jan/2010:22:26:11', '+0200]', '\"GET', '/account/send_password', 'HTTP/1.1\"', '304', '-', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:22:53:40', '+0200]', '\"GET', '/browse/download_model/1779', 'HTTP/1.1\"', '200', '8576', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:23:21:25', '+0200]', '\"GET', '/browse/follow/2029.atom', 'HTTP/1.1\"', '200', '653', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:23:22:21', '+0200]', '\"GET', '/browse/download_model/788', 'HTTP/1.1\"', '200', '3865', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:23:28:41', '+0200]', '\"GET', '/images/default-person.png?1245832850', 'HTTP/1.1\"', '200', '677', '\"-\"', '\"Googlebot-Image/1.0\"']\n",
      "['66.249.65.38', '-', '-', '[30/Jan/2010:23:53:12', '+0200]', '\"GET', '/browse/download_model/2340', 'HTTP/1.1\"', '200', '30305', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['65.55.207.77', '-', '-', '[31/Jan/2010:00:06:37', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['65.55.207.77', '-', '-', '[31/Jan/2010:00:09:36', '+0200]', '\"GET', '/browse/one_model/2152', 'HTTP/1.1\"', '200', '2195', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:01:15:50', '+0200]', '\"GET', '/account/follow/2029.atom', 'HTTP/1.1\"', '404', '577', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['67.218.116.165', '-', '-', '[31/Jan/2010:01:23:08', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.0\"', '200', '99', '\"-\"', '\"Mozilla/5.0', '(Twiceler-0.9', 'http://www.cuil.com/twiceler/robot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:01:35:43', '+0200]', '\"GET', '/browse/one_model/2099', 'HTTP/1.1\"', '200', '2179', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['208.80.193.28', '-', '-', '[31/Jan/2010:01:50:14', '+0200]', '\"GET', '/', 'HTTP/1.0\"', '302', '106', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1;', 'FunWebProducts;', '.NET', 'CLR', '1.0.3705;', '.NET', 'CLR', '1.1.4322;', 'Media', 'Center', 'PC', '4.0;', 'Hotbar', '10.0.356.0)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:02:17:46', '+0200]', '\"GET', '/browse/download_model/985', 'HTTP/1.1\"', '200', '3204', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:02:48:17', '+0200]', '\"GET', '/browse/download_model/2162', 'HTTP/1.1\"', '200', '9362', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:02:51:11', '+0200]', '\"GET', '/browse/display_preview/2099', 'HTTP/1.1\"', '200', '7563', '\"-\"', '\"Googlebot-Image/1.0\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:03:04:16', '+0200]', '\"GET', '/browse/download_model/993', 'HTTP/1.1\"', '200', '3175', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:03:29:31', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:03:29:31', '+0200]', '\"GET', '/browse/download_model/977', 'HTTP/1.1\"', '200', '2992', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:03:49:42', '+0200]', '\"GET', '/tags/one_tag/42', 'HTTP/1.1\"', '200', '2014', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:03:54:44', '+0200]', '\"GET', '/browse/download_model/746', 'HTTP/1.1\"', '200', '12754', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:04:19:51', '+0200]', '\"GET', '/browse/download_model/815', 'HTTP/1.1\"', '200', '6284', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:32:52', '+0200]', '\"GET', '/account/mypage/7', 'HTTP/1.1\"', '200', '14292', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:03', '+0200]', '\"GET', '/', 'HTTP/1.1\"', '302', '106', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:03', '+0200]', '\"GET', '/account/login', 'HTTP/1.1\"', '200', '5998', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:04', '+0200]', '\"GET', '/browse/list_models', 'HTTP/1.1\"', '200', '21752', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:05', '+0200]', '\"GET', '/browse/about', 'HTTP/1.1\"', '200', '6801', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:06', '+0200]', '\"GET', '/help', 'HTTP/1.1\"', '200', '5683', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:06', '+0200]', '\"GET', '/account/login', 'HTTP/1.1\"', '200', '5801', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:07', '+0200]', '\"GET', '/account/follow/7.atom', 'HTTP/1.1\"', '200', '4893', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:07', '+0200]', '\"GET', '/browse/one_model/1210', 'HTTP/1.1\"', '200', '6905', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:08', '+0200]', '\"GET', '/browse/one_model/1390', 'HTTP/1.1\"', '200', '6957', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:09', '+0200]', '\"GET', '/browse/one_model/2495', 'HTTP/1.1\"', '200', '7013', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:09', '+0200]', '\"GET', '/browse/one_model/2518', 'HTTP/1.1\"', '200', '6869', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:10', '+0200]', '\"GET', '/browse/one_model/1408', 'HTTP/1.1\"', '200', '6893', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:11', '+0200]', '\"GET', '/tags/one_tag/48', 'HTTP/1.1\"', '200', '6313', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:11', '+0200]', '\"GET', '/tags/one_tag/8', 'HTTP/1.1\"', '200', '6295', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:12', '+0200]', '\"GET', '/browse/one_model/815', 'HTTP/1.1\"', '200', '6839', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:13', '+0200]', '\"GET', '/browse/one_model/2547', 'HTTP/1.1\"', '200', '6890', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:13', '+0200]', '\"GET', '/browse/one_model/1578', 'HTTP/1.1\"', '200', '6913', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:14', '+0200]', '\"GET', '/browse/one_model/2510', 'HTTP/1.1\"', '200', '6829', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:14', '+0200]', '\"GET', '/browse/one_model/999', 'HTTP/1.1\"', '200', '6860', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:15', '+0200]', '\"GET', '/account/models/7', 'HTTP/1.1\"', '200', '8787', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['89.248.172.58', '-', '-', '[31/Jan/2010:04:33:16', '+0200]', '\"GET', '/?id=17', 'HTTP/1.1\"', '200', '14542', '\"-\"', '\"Mozilla/4.0', '(compatible;', 'MSIE', '6.0;', 'Windows', 'NT', '5.1;', 'SV1)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:04:45:02', '+0200]', '\"GET', '/browse/download_model/2182', 'HTTP/1.1\"', '200', '16607', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:05:10:16', '+0200]', '\"GET', '/browse/download_model/900', 'HTTP/1.1\"', '200', '3440', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:05:17:05', '+0200]', '\"GET', '/?id=8', 'HTTP/1.1\"', '200', '3018', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:05:35:29', '+0200]', '\"GET', '/browse/download_model/797', 'HTTP/1.1\"', '200', '11950', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:06:00:42', '+0200]', '\"GET', '/browse/download_model/2606', 'HTTP/1.1\"', '200', '357240', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:06:03:13', '+0200]', '\"GET', '/browse/one_model/993', 'HTTP/1.1\"', '200', '2170', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:06:25:55', '+0200]', '\"GET', '/browse/download_model/2601', 'HTTP/1.1\"', '200', '2498832', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:06:51:01', '+0200]', '\"GET', '/browse/download_model/2596', 'HTTP/1.1\"', '200', '220990', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:07:06:26', '+0200]', '\"GET', '/browse/display_preview/2244', 'HTTP/1.1\"', '200', '4072', '\"-\"', '\"Googlebot-Image/1.0\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:07:16:18', '+0200]', '\"GET', '/browse/download_model/2752', 'HTTP/1.1\"', '200', '3146', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:07:41:27', '+0200]', '\"GET', '/browse/download_model/2492', 'HTTP/1.1\"', '200', '1680079', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:08:06:40', '+0200]', '\"GET', '/browse/download_model/1408', 'HTTP/1.1\"', '200', '25038', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:08:31:50', '+0200]', '\"GET', '/browse/download_model/2511', 'HTTP/1.1\"', '200', '39499', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:08:56:59', '+0200]', '\"GET', '/browse/download_model/2479', 'HTTP/1.1\"', '200', '39598', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:09:22:13', '+0200]', '\"GET', '/browse/download_model/2754', 'HTTP/1.1\"', '200', '3279', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:09:47:25', '+0200]', '\"GET', '/browse/download_model/2751', 'HTTP/1.1\"', '200', '3146', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:10:12:33', '+0200]', '\"GET', '/browse/download_model/2656', 'HTTP/1.1\"', '200', '117532', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:10:30:53', '+0200]', '\"GET', '/browse/one_node/2701', 'HTTP/1.1\"', '200', '39424', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:10:37:46', '+0200]', '\"GET', '/browse/download_model/935', 'HTTP/1.1\"', '200', '5014', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:11:03:00', '+0200]', '\"GET', '/browse/download_model/717', 'HTTP/1.1\"', '200', '30442', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:11:28:14', '+0200]', '\"GET', '/browse/download_model/751', 'HTTP/1.1\"', '200', '7485', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:11:53:26', '+0200]', '\"GET', '/browse/download_model/2549', 'HTTP/1.1\"', '200', '5052', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:12:18:38', '+0200]', '\"GET', '/browse/download_model/2650', 'HTTP/1.1\"', '200', '57739', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:12:43:41', '+0200]', '\"GET', '/browse/download_model/729', 'HTTP/1.1\"', '200', '13146', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:12:45:28', '+0200]', '\"GET', '/browse/browse_applet_tab/2593', 'HTTP/1.1\"', '200', '10305', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:13:09:00', '+0200]', '\"GET', '/browse/download_model/814', 'HTTP/1.1\"', '200', '3066', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:13:34:05', '+0200]', '\"GET', '/browse/download_model/2629', 'HTTP/1.1\"', '200', '62903', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:13:59:27', '+0200]', '\"GET', '/browse/download_model/2644', 'HTTP/1.1\"', '200', '12122', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:14:24:30', '+0200]', '\"GET', '/browse/download_model/804', 'HTTP/1.1\"', '200', '4085', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:44', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.0\"', '200', '99', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:44', '+0200]', '\"GET', '/account/new', 'HTTP/1.0\"', '200', '2226', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:47', '+0200]', '\"GET', '/stylesheets/nlcommons.css?1264605648', 'HTTP/1.0\"', '200', '1179', '\"http://modelingcommons.org/account/new\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:50', '+0200]', '\"GET', '/stylesheets/browse.css?1264605648', 'HTTP/1.0\"', '200', '422', '\"http://modelingcommons.org/account/new\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:52', '+0200]', '\"GET', '/stylesheets/jquery.autocomplete.css?1264605648', 'HTTP/1.0\"', '200', '447', '\"http://modelingcommons.org/account/new\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:55', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.all.css?1264605648', 'HTTP/1.0\"', '200', '52', '\"http://modelingcommons.org/account/new\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:57', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.theme.css', 'HTTP/1.0\"', '200', '2862', '\"http://modelingcommons.org/stylesheets/themes/nlcommons/ui.all.css?1264605648\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:57', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.base.css', 'HTTP/1.0\"', '200', '111', '\"http://modelingcommons.org/stylesheets/themes/nlcommons/ui.all.css?1264605648\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:59', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.slider.css', 'HTTP/1.0\"', '200', '318', '\"http://modelingcommons.org/stylesheets/themes/nlcommons/ui.base.css\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:59', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.core.css', 'HTTP/1.0\"', '200', '624', '\"http://modelingcommons.org/stylesheets/themes/nlcommons/ui.base.css\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:59', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.tabs.css', 'HTTP/1.0\"', '200', '414', '\"http://modelingcommons.org/stylesheets/themes/nlcommons/ui.base.css\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:59', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.accordion.css', 'HTTP/1.0\"', '200', '295', '\"http://modelingcommons.org/stylesheets/themes/nlcommons/ui.base.css\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:59', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.dialog.css', 'HTTP/1.0\"', '200', '442', '\"http://modelingcommons.org/stylesheets/themes/nlcommons/ui.base.css\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:59', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.resizable.css', 'HTTP/1.0\"', '200', '307', '\"http://modelingcommons.org/stylesheets/themes/nlcommons/ui.base.css\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:59', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.datepicker.css', 'HTTP/1.0\"', '200', '965', '\"http://modelingcommons.org/stylesheets/themes/nlcommons/ui.base.css\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['67.195.112.35', '-', '-', '[31/Jan/2010:14:45:59', '+0200]', '\"GET', '/stylesheets/themes/nlcommons/ui.progressbar.css', 'HTTP/1.0\"', '200', '114', '\"http://modelingcommons.org/stylesheets/themes/nlcommons/ui.base.css\"', '\"Mozilla/5.0', '(compatible;', 'Yahoo!', 'Slurp/3.0;', 'http://help.yahoo.com/help/us/ysearch/slurp)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:14:49:40', '+0200]', '\"GET', '/browse/download_model/1420', 'HTTP/1.1\"', '200', '73293', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['65.55.207.50', '-', '-', '[31/Jan/2010:15:07:49', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['65.55.207.50', '-', '-', '[31/Jan/2010:15:09:06', '+0200]', '\"GET', '/', 'HTTP/1.1\"', '302', '109', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['65.55.207.50', '-', '-', '[31/Jan/2010:15:09:14', '+0200]', '\"GET', '/account/login', 'HTTP/1.1\"', '200', '1935', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:15:14:55', '+0200]', '\"GET', '/browse/download_model/766', 'HTTP/1.1\"', '200', '10923', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:15:40:01', '+0200]', '\"GET', '/browse/download_model/756', 'HTTP/1.1\"', '200', '4632', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:16:05:28', '+0200]', '\"GET', '/browse/download_model/757', 'HTTP/1.1\"', '200', '11079', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:16:30:36', '+0200]', '\"GET', '/browse/download_model/725', 'HTTP/1.1\"', '200', '16082', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:16:55:37', '+0200]', '\"GET', '/browse/download_model/2473', 'HTTP/1.1\"', '200', '211110', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:17:20:49', '+0200]', '\"GET', '/browse/download_model/1510', 'HTTP/1.1\"', '200', '35814', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['65.55.215.75', '-', '-', '[31/Jan/2010:17:39:03', '+0200]', '\"GET', '/robots.txt', 'HTTP/1.1\"', '200', '99', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['65.55.215.75', '-', '-', '[31/Jan/2010:17:40:02', '+0200]', '\"GET', '/browse/list_models', 'HTTP/1.1\"', '304', '-', '\"-\"', '\"msnbot/2.0b', '(+http://search.msn.com/msnbot.htm)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:17:46:08', '+0200]', '\"GET', '/browse/download_model/802', 'HTTP/1.1\"', '200', '4800', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:18:11:17', '+0200]', '\"GET', '/browse/download_model/2194', 'HTTP/1.1\"', '200', '11380', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:18:36:25', '+0200]', '\"GET', '/browse/download_model/1572', 'HTTP/1.1\"', '200', '10239', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:19:01:55', '+0200]', '\"GET', '/browse/download_model/1806', 'HTTP/1.1\"', '200', '12739', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:19:26:45', '+0200]', '\"GET', '/browse/download_model/930', 'HTTP/1.1\"', '200', '8271', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:19:52:00', '+0200]', '\"GET', '/browse/download_model/2041', 'HTTP/1.1\"', '200', '18891', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:20:17:07', '+0200]', '\"GET', '/browse/download_model/1800', 'HTTP/1.1\"', '200', '14802', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:20:42:19', '+0200]', '\"GET', '/browse/one_node/1613', 'HTTP/1.1\"', '200', '27080', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n",
      "['66.249.65.38', '-', '-', '[31/Jan/2010:21:08:00', '+0200]', '\"GET', '/browse/one_node/1892', 'HTTP/1.1\"', '200', '1296', '\"-\"', '\"Mozilla/5.0', '(compatible;', 'Googlebot/2.1;', '+http://www.google.com/bot.html)\"']\n"
     ]
    }
   ],
   "source": [
    "for one_line in open('mini-access-log.txt'):\n",
    "    print(one_line.split())\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 136,
   "id": "a4d56aa1-2b22-404d-989c-539e47dfd536",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "linux-etc-passwd.txt  mini-access-log.txt  nums.txt  shoe-data.txt  wcfile.txt\n"
     ]
    }
   ],
   "source": [
    "# what txt files do you see in the current directory?\n",
    "\n",
    "!dir *.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3dc19e23-2bc2-47ec-8c34-9013bd13ae5e",
   "metadata": {},
   "source": [
    "# Writing to files\n",
    "\n",
    "You will almost certainly read from *many* more files than write to them. But you can still know how to write to a file.\n",
    "\n",
    "When we `open` a file, the assumption is that we want to read from it. However, we can pass a second argument to `open`, the letter `w`, which means: We want to open this file for writing. And yes, computers require that you decide between reading from a file and writing to a file.\n",
    "\n",
    "In other words, to open a file for reading, I say:\n",
    "\n",
    "    open(filename, 'r')  # if you want to be very explicit\n",
    "    open(filename)       # the default\n",
    "\n",
    "To open a file for writing, you say:\n",
    "\n",
    "    open(filename, 'w')   # must pass 'w'!\n",
    "\n",
    "If you open a file for writing, then after you do so, one of two things has happened:\n",
    "\n",
    "1. The file now exists, and you can write to it. Also, it currently contains 0 bytes.\n",
    "2. You cannot open the file for some reason, and the program exited with an exception.\n",
    "\n",
    "What if the file already existed? It now contains zero bytes.\n",
    "\n",
    "Also: You can write to a file (when you've opened it for writing in 'w' mode) with the `write` method. This is similar to `print`, but doesn't add a `'\\n'` after each line.\n",
    "\n",
    "Finally: You really must close a file when you're done writing to it, to ensure the contents were written right away. Meaning: Use `with` when you're writing to a file, even if you don't use it when reading."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 138,
   "id": "e2c17304-4c5e-4fef-b020-99dbc3e80340",
   "metadata": {},
   "outputs": [],
   "source": [
    "with open('myfile.txt', 'w') as f:    # open the file for writing, assign to f\n",
    "    f.write('abcd\\n')                 # write any text I want to the file\n",
    "    f.write('another line!\\n')\n",
    "    f.write('end of the road\\n')\n",
    "    # at the end of the with block, the file is automatically closed"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 139,
   "id": "b9646e39-58bb-4c3d-931c-cf76f14ba7e1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "abcd\n",
      "another line!\n",
      "end of the road\n"
     ]
    }
   ],
   "source": [
    "!cat myfile.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 140,
   "id": "f7ad2bef-94f4-4d0a-b7e7-21e9defd525a",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 140,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# AK : check for a file before writing to it\n",
    "\n",
    "# open has a lot of options:\n",
    "# - r (for reading)\n",
    "# - w (for writing)\n",
    "# - a (for appending -- write, but add to what's there already, at the end)\n",
    "# - x (for writing -- but if the file already exists, give an error. This avoids 'w' destruction)\n",
    "\n",
    "# you can also use os.path.exists to find out if a file already exists\n",
    "\n",
    "import os\n",
    "os.path.exists('/etc/passwd')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 141,
   "id": "fa572dcd-ad9a-4296-9575-78a4db96a8d2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 141,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "os.path.exists('unicorns')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7fe5af8e-1df8-43b2-b997-df5e04ece980",
   "metadata": {},
   "source": [
    "# Next time: Functions\n",
    "\n",
    "- Writing them\n",
    "- Calling them\n",
    "- Arguments and parameters\n",
    "- A little about local vs. global variables"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d58c428-6e83-4b7b-94a8-902a520eba7c",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}