Global network of computers or The Internet works in a way that is, as you might imagine, combination and interconnection of many different components.
Today, we're going to talk about network and the internet parts. And how they provide the infrastructure for the higher levels.
The Network mainly works in a way that's layered.
"EVERYTHING WE DO ONLINE WE DO WITH SOME KIND OF METAPHOR"
- cyberspace
- internet superhighways
- infobahns
- global villages
- coffee houses with a thousand rooms
- town squares
- freshman lookbooks
- cloud
Since the information is mostly formless, we use metaphors. And like all metaphors, metaphors for the internet is there to explain one or more aspects of the whole. But definitely not the whole. So i would advise you to keep tabs on the metaphors being used and try and look at the underlying system, sooner or later they will come short, and you'll be alone to tie the ends.
To illustrate the importance, implications, costs and benefits of the layered architecture. We'll use an anology of physical mail transportation to go over the stack, again, there is no one to one correspondence between one and the other. This is not even the best example, and should not be extended beyond there.
The city
- Provides the roads, highways, streets. also the street names and door numbers.
Drivers
- Takes your truck from one location to another. But someone else needs to fill the vehicle. And give the adress.
Transport company
- Keeps track of the packet and makes sure it is not lost between transportation paths. Notifies you once it gets to its destination.
People buying and selling stuff online
- They are the end users of this infrastructure.
Notice that the interaction within layers is not regular.
- Underlying layers does not wholly exists to serve the one above it.
- There are multiple ways to reach the goals of a layer. you, as a transport company, can buy a fleet of drones and pass the complications of city infrastructure and people driving the trucks.
- But it stays the same that we somehow want to deliver packets from one location to another.
The main goal here is abstraction. When you use abstractions, you get to stay away from most of the details. Like, you don't care if a driver is not able to come to work that day, the transport company has a lot employees that can do the job. Furthermore, this liberation from the details, help you communicate in a more effective manner. In this case, when you are talking about sending mail, even if the person you are talking to doesn't know anything about sending mail, you can use the simple terms of the transport company.
So if it's build to keep you away from details, why do you have to know that it exists?
Once in a while, you hit the limits. Referring it in more precise terms other than magic, is helpfull for precise communication and choosing among alternatives. For example:
- You know that it's cheaper to send a piece of paper as a
document(in transport company's terms). - You buy a plan ticket if you want to get somewhere faster.
- You try not to order food in rush hour, as the courier will get stuck on the traffic and your dinner will get cold.
This is the layer that provides you with computer to computer communication. All you can send in this layer is small packets. And it leaves you with a physical adress.
You can enter these addresses in your browser
216.58.209.174 -- to access google.com
178.62.200.105 -- for pisano.co
which you can't remember for every place you want to visit online.
That's where DNS comes in. Standing for Domain Name System. It translates domain names to the physical address of the machine hosting the website/application you intend to access.
Think of it as a very big phone book.
google.com -> 216.58.209.174
pisano.co -> 178.62.200.105
...
This has couple levels of indirection. What you need to know is, when you buy a domain name, you register a name server with your domain provider. Name Server, once queried with a domain name, returns the actual physical adress.
This is a simple layer on top of the ip layer. It provides the ability for multiple applications to share a host computer. Other than that, UDP is so simple it's best understood by what it lacks from TCP.
Sometimes this can be its power. Simple is fast for example.
This is the Protocol that powers most of the higher level protocols. Basically, if you're not transmitting 100s of megabytes of video in realtime, you use TCP.
TCP Provides:
- Ordered Data
- Reliable Transmission
- Flow Control
For example, you can send messages that wouldn't fit in one ip packet. And tcp makes sure they would get the destination. and get there in the correct order. Also it can try to adjust the pressure on the network, so that the network doesn't go down once too many people connect it.
Implementation consists of many mechanisms, basically a sequence number attached to each message and a state machine to keep track of which messages got delivered.
Here is couple of examples. It's not expected of you to understand them in great detail. They are meant as an introduction to common terminology in network programming.
First, we have servers, which is the component that sits there and waits for incoming connections. Then, we have clients, which is the party that initiates the connection with a listening server. And the communication between this two parties happen through sockets.
The lifecycle of this sockets is basically, creation, interaction and closing. Each of these is handled with different events. And you react to events. How you react to it can be complicated. Hence, we'll starts with a simple echo server.
// Load the TCP Library
const net = require('net');
// pick a port to connect to
const PORT = 3333
// Handle connecting sockets
function connectionHandler(socket) {
// Identify this client
socket.name = socket.remoteAddress + ":" + socket.remotePort;
// Send a nice welcome message and announce
socket.write("Welcome " + socket.name + "\n");
// Handle incoming messages from clients.
socket.on('data', (data) => {
socket.write(data)
process.stdout.write(socket.name + "> " + data.toString() + "\n");
});
// write to the console, if we encounter an error.
socket.on('error', (error) => {
process.stdout.write(socket.name + " throws " + error.toString() + "\n");
});
// write to the console, if we encounter an error.
socket.on('end', () => {
process.stdout.write(socket.name + " disconnected.\n");
});
}
// Start a TCP Server
const server = net.createServer(connectionHandler)
server.listen(PORT);
process.stdout.write("Echo server running at port " + PORT + "\n");We can use the telnet <server_ip> <server_port> to interact with this server. in this case:
$ telnet localhost 3333or we can write our own client:
// utilities
const _ = require('lodash');
// same tcp library that we used to create the server
const net = require('net');
// utility for reading the lines user types.
const readline = require('readline');
const PORT = process.env.PORT || 3333;
const HOST = process.env.HOST || 'localhost';
// SETUP the interactions
// we specify we want to read lines from the terminal.
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
})
// SETUP the connection socket.
var socket = new net.Socket();
// Socket can read data.
socket.on('data', (data) => {
process.stdout.write(data.toString() + '\n');
});
// Socket can emit errors.
socket.on('error', (error) => {
process.stdout.write(error.toString() + '\n');
});
// Socket can be closed by the server
socket.on('close', () => {
rl.close();
socket.destroy();
process.stdout.write('Connection closed');
});
// Finally, connect to the server.
socket.connect(PORT, HOST, () => {
console.log('Connected');
});
// connect the terminal to the socket.
rl.on('line', (input) => {
socket.write(input + '\n');
});We can extend our echo server to keep a list of connected clients. At this point it turns into a chat server.
const _ = require('lodash');
// Load the TCP Library
const net = require('net');
// pick a port to connect to
const PORT = 3333
// Keep track of the chat clients
const state = {
'clients': []
};
// send a message to all connected clients except one who sends it.
function broadcast(message, sender) {
const socketsToWrite = _.reject(state.clients, ['name', sender.name])
_.forEach(socketsToWrite, (socket) => {
socket.write(message);
});
process.stdout.write(message + "\n");
}
// Handle connecting sockets
function connectionHandler(socket) {
// Identify this client
socket.name = socket.remoteAddress + ":" + socket.remotePort;
// Put this new client in our list.
state.clients = _.concat(state.clients, socket);
// Send a nice welcome message and announce
socket.write("Welcome " + socket.name + "\n");
// Notify everyone else connected.
broadcast(socket.name + " joined the chat\n", socket);
// Handle incoming messages from clients.
socket.on('data', (data) => {
message = data.toString();
// we have one little command that specifies the intention to exit out of the chat room.
if (message.startsWith('quit')) {
state.clients = _.reject(state.clients, ['name', socket.name]);
socket.end();
return;
}
broadcast(socket.name + "> " + message, socket);
});
socket.on('end', () => {
// if the connection ends, we remove the client from our list of connected clients.
state.clients = _.reject(state.clients, ['name', socket.name]);
// Notify everyone else connected.
broadcast(socket.name + " left the chat.\n", socket);
});
// log what just happened.
socket.on('error', (error) => {
process.stdout.write(socket.name + " throws " + error.toString() + "\n");
});
} // connectionHandler
// Start a TCP Server
const server = net.createServer(connectionHandler)
server.listen(PORT);
console.log("Chat server running at port " + PORT + "\n");it would be a good example to try and extend this to multiple chat rooms.
At this point that is just a high level overview of the components of the internet. This is the infrastructure, that supports many many applications. Bitcoin, Torrent, Financial services, Internet of things, Your coffee machine set to be controlled from your watch, Mail Servers, Some medical device designed to monitor your hearth rate. They all use the internet. We will ignore all of them and focus on what constitutes World Wide Web.
Around 1992, world already heavily uses news groups and e-mail, a scientist at CERN labrotories realizes that internet is pretty amazing and it can be used to distribute scientific papers. So it sets himself a pretty home page with bunch of links to other pages containing the actual scientific papers. Since these pages consist of text and some additional presentation material, for example headers, images, more images, animated images; they decide to call it Hyper Text.
The transfer protocol for these documents gets called HTTP(Hyper Text Transfer Protocol).
Since this hyper text contains presentational material other than text. like headers and images.
sharing one big document. people start marking different parts of the document with special words
between angular brackets < >. And this gets called HTML(Hyper Text Markup Language).
<h1>This is a top level header<h1>
<h4>This is a smaller header<h4>
<a href="wikipedia.org">link to wikipedia<a>
<img src="pisano.co/logo.png" />Then, to be able to personalize the pages further comes Style Sheets.
h1 {
font-size: 30pt;
}and, to add interactivity to these pages, comes a programming language JavaScript.
<span onclick="alert('you clicked me!')">click me</span>So like our previous echo and chat example, we need two parties for this communication. One client and one server. In this case, clients are called web browsers; servers are called web servers. They communicate through this protocol called http.
So, since http is built on top of tcp, we can still use our tools.
If we run our client.js script using HOST=www.google.com.tr PORT=80 node client.js
in this case we can speak the most simple words. extra line at the end is important.
GET / HTTP/1.1
HOST: www.google.com.tr
google in this case will respond to us with the following.
HTTP/1.1 200 OK
Date: Wed, 29 Jun 2016 11:01:27 GMT
... more headers ...
8000
<!doctype html>.....Most of the time, that's the underlying communication pattern between your browser and the google's web servers. or any other website.
Remember, HTTP is, at first, designed as a document sharing protocol. So this GET / part of the
message means i want to get the document at the /.
Changing that GET word, notating a HTTP Method, to POST allows us to pass extra data from
client to server. which is the message produced by your browser when you fill a form on a web page.
POST /login HTTP/1.1
Content-Type: application/x-www-form-urlencoded; charset=utf-8
Content-Length: 27
username=user&password=passAnd this form functionality is the next missing part for a lot of web applications.
I would advise you to install httpie and play around with it. especially after the Web APIs lecture. Watch what goes on inside the messages.
It is a lot to digest. The main takeaway should be that Web is a complex structure with layers and layers of components. Somehow, we need to navigate not just within a layer but up and down the ladder of abstraction to build the products end users will love. And learn to structure our own applications in a way that creates imperfect but friendly abstractions.