How websites and HTTP work? Web Basics Crash Course

hey everyone welcome to the brand new course which is named as internet or web or web basics whatever it is named but this course is a fundamental course for starting off your backend development learning path irrespective of what technology you will be learning because we will be discussing about http protocol the rest api the dns and basically how the internet things work in this whole small mini course so this is a sort of an introductory video to the course we're going to be covering all these topics which i mentioned and let's get started i'm super excited to give this information to you because i am pretty sure that there are a lot of developers out there who are back in developers who are sort of okayish with code as well but a lot of them do not really know the nuts and bolts of the things so if you want to stand apart from the crowd if you want to become somebody who is best of the best these are some of the things which would really set you apart from somebody who's just coding things who's just mugging up how to code because these are nuts and bolts of the technology which you'll be working with let's start with the tech part from the next video real soon i'm gonna see you in the next one hey everyone welcome back and in this video let's just discuss about dns lookup for the websites now what is dns first of all let's just talk about that so you see that you usually visit sites like google.com facebook.com you know codedam.com as a matter of fact and you do not have to remember anything except for the domain name right but the way computers work is they do not understand what codem.com is when you're writing codem.com and you hit enter your computer this computer right here has to communicate with other other computer on the planet right now this computer might be just in the vicinity of your laptop which is sort of like an intranet or it might be on the other side of planet that is using internet now how does this work how does this domain name how does your computer know that in order to get to this particular domain name in order to fetch this page this content which computer should talk to that resolution that thing is done using something known as dns which stands for domain name system now the simple job of domain name is to convert this codem.com into a number a number which is called an ip address an internet protocol address now you might be thinking hey uh where's this number i mean i just see codedam.com right so let's just go ahead and open inspector right here and let me just go ahead and bring this up a little bit switch to networks increase the font size and refresh the page and once we do that you can see that right now we are on the image filter let's switch it to document and there's only a single document in this page that is the main page itself and let's click on this the moment we do this you can see we see that hey this browser made a request to request urlscodenum.com which is fine but what is more interesting is this part right here you see suddenly magically somehow goddamn.com is mapped to this domain name right here how does that work this what this conversion is this is basically done by dns right dns just like i said is domain name system it's basically a technology which allows computers to resolve host names that is uh you know google.com to something more like which is an ip address could be anything right so this is an invalid ip address but i'm just saying so well how does dns work and what is the first step when you hit write something like this and hit enter the first step is to talk to your actual dns resolver which is sitting in your computer every computer has got one which you are through which you are browsing the internet now this here you have two options either your isp provides the dns resolver or you manually provide a dns resolver now dns resolver what is this this is basically a ip address a hard coded ip address which your computer uses to ask a question like this hey what is codedown.coms ip address right so let's take a look at a very common dns which is 1.1.1.1 this is basically cloudflare dns ip address and a lot of computers are configured to use cloudflare's dns in fact my own computer uses cloudflare dns as far as i remember so let's go to advance let's see inside the dns settings right so you can see if i just show you my router we have the primary dns setup as 1.1.1.1 and the alternate dns is this thing so this means that my computer is gonna use this ip address to ask what is the ip address of codedown.com this ip address it's a standalone dns server it is not an http server although you can visit 1.1 like this on an http and it will render a website but most dns servers do not have an http website for example this 8.8.8.8 it is a server it is a dns public dns from google and i hope this does not have a website right so yeah it does not has a website so you can see it's just just standing there waiting for some request or whatever but the point is that this ip address takes in my question now dns follows a certain um format in order to ask a question it takes a certain question from any client like this browser or like this operating system and it tries to give you back an ip address for that particular domain now this ip address this tns 1.1.1.1 might not really know the ip address of so what's going to happen is my os is going to say to 1.1.1.1 it's going to ask the question hey what is the ip address of codedam.com right and my dns is going to say 1.1.1.1 says or maybe let's just break this like this client right and 1.1 says that i do not know but i know a guy who knows a guy right and as a client you say hmm okay and you just wait for a while then behind the scenes what 1.1.1 does is that it asks other dns servers in a specific order now i'm not really going into depth of how dns works but for example 1.1 this guy asks another guy for example 65.24.9 right a random just a made up ip address it asks this hey do you know where the ip addresses of domains with dot-com are stored because you see this is coded.com so we can infer that obviously we are looking for ipaddresses4.com and this guy says this guy says to 1.1 that yup they are with a 77 point you know let's say this one right then you go as 1.1 and you ask this particular resolver hey do you have the ip address of codedown.com then this guy says that i do not but i know this guy should have right and it gives you another another ip address right and then finally this guy when you ask this guy this guy actually says that hey codename.com is actually you know this ip address and once this 1.1.1 has the ip then you just go ahead and basically connect to goddamn.com using this particular ip address right so yeah i mean once this guy once our client gets the ip address um what might happen is that you know 1.1 might cache this ip address right so the next time when you request this it can directly say that hey this was the ip address which you used just to save some time so this is how your computer make sure that it gets coded.com's ip address right all these servers are involved this dns technology basically allows you to not remember these ip address individually in a lot of cases it's not even possible right because these ip addresses might be changing so this is the crux of how the domain resolution works when you're browsing any website so hopefully you understood how dns works in an overall way you don't really have to get into depth of this but yeah it never hurts to learn a little bit about things like these so that's all for this video i'm gonna see you in the next one really soon hey everyone welcome back and in this video let's start with http protocol a little bit so you must be familiar with http protocol in the sense that you always write http or https in front of domains so that really means that you are working with http protocol right that is hyper text transfer protocol um now i should not say http protocol because that's you know p is already covering the protocol but anyway what happens in http is that there is a certain set of rules which the client and server both have to follow in order to establish a successful communication and what those rules are we're going to be discussing them in this particular section but starting off with what http is it is a plain text protocol by that what i mean is that let me just go ahead and hard refresh this page and i'm going to show you what a plain text protocol really means so if i click on this uh i do not really get the request headers like that so let me just go ahead and go to example.com and right here you can see i get a view source button i am not sure when we get this and when we don't get this in chrome devtools but you can see that this is an actual response this is actually the whole response which your client needs in order to get this particular page back right so in order to get this particular page back from the server this is all your client says after establishing a connection with the ip address right so this is a plain text request that is you can read this it is an actual english request a lot of communications work on the binary protocol as well where you are not able to read in a human readable format what the message is but http is a plain text protocol that means you can all the time read what is going on over the wire that means over the network right so once you have this plain text protocol in place you're gonna see it follows a certain format we're gonna get to this format later on but right now i want to talk about how the server responds to us right so this section this general section contains a bunch of things um this request url which we already covered remote address is something which we covered but not this part we're going to cover this soon this thing status code is an interesting thing because you might be familiar with it with a lot of jokes and everything which are around status code especially the 404 not found but the status quo what they really mean is that what is the um you know what what sort of message the server wants to say to the requested resource for example let's visit example.com something which is random you see that the status code this time is 404 right it's not 200 it's 404 and we automatically get this not found from the browser but this 404 means that it says to this client it says to my browser the server is saying to my browser that something was not found on the server right so that means you as a client requested a resource which was not on the server 200 a status code of 200 means that everything went just fine right so 200 means that hey everything went fine we found that resource on the server it's all cool similarly there are a bunch of status quotes all along the way so for example if i just visit codedam.com but without http that is without https that is using http you can see that we get a 307 status code right and it is basically an internal redirect it's not a status code which is returned by the server but you get the idea right so we get another status code and once we are on https you can see here there's a difference once we are in https we get a 200 okay and we get an ip address obviously so status quotes is just a way of servers to inform if something went right if something went wrong in a very uh sort of concise way right not really giving a lot of information just letting know letting you know about the status of the request which is exactly what it is named as well so you can go ahead and check out this uh extremely useful documentation page from mozilla mdn and it contains a bunch of information about status codes you can see status codes in the range of 100 are informational responses right for the most part i have only seen to use this 101 which is switching protocol in case of a websocket connection i'm not very familiar with these other three but yeah they are there then 200 okay is like the most common status code you're going to see there are a bunch of other status codes in the 200 range as well 300 is for the redirection sort of thing or maybe like you know something is not yet completed then 300 after 300 we get 400 range 400 is basically client has done something wrong that is my browser has done something wrong and that is why server was unable to process that request and 500 means that server has done something wrong that means something crashed on server or maybe you know something for which the server is responsible not the client so this is this is in a nutshell what the status codes mean and look like in an http protocol in the next video we're going to be discussing more about http protocol and you know all these request methods and stuff so that you have a better understanding of how this all works and we're going to also be finally working towards making an actual request an actual raw request ourselves using command line right so yeah that's that's pretty much it for this video i'm gonna see you in the next one really soon hey everyone welcome back and in this video let's just talk a little bit about this port number which we have now just like http just let's just understand this first of all just like http we also have a protocol called ftp right file transfer protocol for example if you want to add a new page to this website for example page2.html and you hit enter and you should be able to see a different content so how would you upload this page too you can use ftp file transfer protocol as one of the ways now ftp is also a protocol http is also protocol but when you run a particular service on a computer how should let's say a server know when there's an incoming connection which particular service to redirect that communication to right now your computer is running multiple programs let's understand it this way for example you're running browser or maybe a game application all the time together right it's not that you're running a single application at a single time so similarly a computer can run an http server which allows you to serve something like this and an ftp server at the same time right so an ftp server would be running as a different program an http server would be running as a different program and both of them would be working just fine but how would the server know when there is an incoming connection request that which particular service should handle this this particular thing that particular thing is done by the port number right so what happens with port number is that by default http has a port number of 80 right so when you are visiting example.com in reality you are not visiting example.com you are actually visiting example.com port 80 like this right so if i go ahead and copy paste this in the browser you're going to see that chrome automatically gets rid of that 80 because it is useless because chrome already knows that http by default works on port number 80. similarly if you visit an https domain what you're going to see is https by default works on port number 443 right so these are some common port numbers 80 443 um 21 22 so these are some predefined services there are some predefined services like http https ftp ssh which have their own assigned ports but this in no way means that you cannot run https on a different port or in no means it can it does not mean that you cannot run http on let's say port number 500 right you can do it but it's not a convention so this port number right here just denotes the service which should communicate which should respond to a particular request what does that mean is that whichever computer this from which we got this response on that particular computer a service is listening to port number 443 which is an https server right which is responsible for responding to whenever i visit this particular domain similarly if i visit try to visit for example https 333 you can see in this case it does not omit that particular thing that particular port number from my request url whereas here it does omit that because it is not required but if we had a server if suppose we had a server running at port number 333 we're going to get a response from that particular port right here this is why when you write local host and then 3000 for a react a typical react application this is why it works right your local host your is your own computer and react is your project is listening on one specific port 3000 that is right your computer can host multiple other services as well for example if you are running mongodb it's probably running on 27017 you can see i'm actually running mongodb and it saves me that you're trying to access mongodb over http so that's the wrong thing if you're running my sequel you're going to have a port number for that particular thing so you know you get the idea that you can use these ports and port numbers to run multiple services on a single machine and communicate with all of them differently right in a clean way so this is the importance of port numbers this is why we use port numbers and yeah that's pretty much it for this video i'm gonna see you in the next one really soon hey everyone welcome back and in this video let's just go ahead and take a look at what request method means in http and it is super important it's one of the fundamental things of the http protocol and once we have connected to the ib address to the port by resolving the domain name to ip address the next thing the client does it sends the request method as the very first thing if you go actually go to the request headers and click on view source you can see get right here is just because of uh we see this request method get is because of this get keyword right so it is the very first thing which is sent when a connection is established between a client and a server so why does this matter why this kit is important and how does it you know fit in the picture well you see when you are browsing the web there are multiple ways you can interact with a web page some web pages you just visit like this one example.com that's it i visited this i clicked on this link and then i moved on so these sort of web pages these sort of interactions are placed using the request method get right where you pretty much do not send any data to the server by data i mean send any custom data right you're not uploading files you're not logging in to the server you're doing nothing you're just browsing the web so most of the way when you browse it you can browse that using request method git and in fact that is what the default method is right so if you go to google.com you're going to see this is request method get this is get everything is get right everything is good there's another method there are actually multiple methods i'm going to list some of them here so there's a method called get obviously we discussed that there's a method called post there's a method called put there's a method called patch there's a method called delete and there might be other methods as well which i just forgot but these two are the most important methods because you can pretty much do anything you want on the internet on the web with these two right um in terms of http protocol so get really just means that you are visiting web pages and post right here means that you want to interact or send some data to the server and where does this fit in for example if you are logging into a website you never see your username and password inside of a url of course i'm just talking about that you're not really on a some dummy or shady website but if you're for example if you're going to google.com or even if you are going to codedum.com as a matter of fact let me just go ahead and log in here and just write any dummy username and password if i click on login if i take a look at this xhr you can see that this request right here is made as a post request right it's not a get request and why it is not a get request is because you can see down here i actually want to send some data to the server that is this username and password and how can i communicate this data to the server is using a post request why not get request because get request does not allow you to send this payload as a body to the server that is how the specification is right so if you read the http specification you're going to understand that get request should not have a body it could have but it should not have now white does it could have because just like i said http is a plain text protocol and in the further videos we're going to see how to actually send a body as well but yeah this is this is the reason why we use post method right here right because we want to send some data now you can also upload huge files using post request you can um you know log in the user by sending their username and passwords on the wire stuff like that now if you think about it you're going to see that him if we have already have the get and post methods why do we need these three or maybe all the others as well well you see this is because these three are more or less denoting the meaning of your request right so for example instead of post if you try to use put the functionality is going to be same the only difference is your server now knows that you want to create a resource and your client also knows that if you are seeing a put request your client knows that you want to create a resource that is first reason right because it provides semantic it provides meaning to your request second thing is that you can have multiple methods on a single path right so you can have a get method on this url as well you can have a different handler for post method you can have a different handler for patch method you can have a different handler for um delete method as well so for example let's say your api was something like this uh you know infra.dumb.com posts slash or maybe just like api slash pose right so now what i can do is probably define something like if i hit it with a get i need a json of all posts if i hit it with put with us appropriate payload i want to create new post right if i hit it with delete i want to delete delete a particular post if i hit it with a patch it means i want to edit a post right so you see that even though now i have a single api i can do so much work with that right because i can define different different handlers depending on what sort of call am i making so this is how the the request method works in the http protocol this is how it all works but just like i said for the most part functionality wise post is the most superior because it can act as a get thing as well it can act as a put patch anything right because it supports body it supports you to send data inside the payload right as a payload so yeah that's pretty much it for this video i'm gonna see you in the next one really soon hey everyone welcome back and in this video let's just go ahead and take a look at what is the request headers section right so we have already seen um request url method remote address and everything but if i switch it from view parsed to view source this actually shows us the whole request but the request headers really start from i would say this point well host is also a header but you can say that the headers start from this particular thing so these are all request headers you can see they all follow a common pattern that is a name of the header then a colon which denotes the separation and then the value of that particular header right name column value name column value so http headers some of the headers mean special things to the browser like this cache control or connection or um you know except as a matter of fact but a lot of headers you can also send as your custom headers right so you can pretty much go ahead and write x dash my name dash is and then a colon and then say meihul mohan right here and it will just work fine it is not a you know a breach of protocol or anything for example this x cache hit is not a standard or you know it's not a special header content type is a special header right which the browser would actually use internally so http headers is just a way to provide more information to the request what you want to do with the request for example this connection keeper live what this means is that please keep the underlying tcp connection alive for this particular request now tcp we can talk tcp in another video but i don't want to get into depths of networking right now but tcp just just to give you an idea that tcp is another protocol on which http is built on right so hyper text transfer protocol is actually built on a protocol called tcp which is transmission control protocol which is a lower level protocol which handles a bunch of other lower level important stuff which http does not have to worry about right http just worries about that the message is in this format where you have the post uh the the method name first of all in the requested path then the http 1.1 protocol or whatever protocol which you are using then a bunch of headers then separated by two lines and then the actual post request so anyway what i was talking about is that connection keeper live requests that even though this http communication is over the underlying tcp connection should be kept alive for other communications right so this is a special header then we have cache control we have other headers as well we can talk about them all day but the crux of the thing is that these http headers allow you to communicate more information as a as a node on the network to the other computer now this is not really restricted to just client you can see the server also responds us with response headers and it follows a different format right so if you're writing an http server you have to first of all pass this request and then you have to send this appropriate response as well because if you do not send it like this in a particular sort of protocol the client will not understand your response and it will reject your response right so the browser will show you some sort of error that you know malformed response or something like that so anyway this is how response headers work first of all you're gonna have uh the statement like this the first line should be like this and then you're gonna have key values separated headers with a colon which contain additional information about your request so that is pretty much it for the http headers part in the next video we're going to be actually writing our http request in netcat from the very scratch so let's just go ahead and take a look at that thing and then we'll be done with the basics of things here hey everyone welcome back and in this final video we're going to be doing a netcat request to this example server now for those of you who don't know netcat what netcad is netcat is basically a utility which allows you to open tcp and udp connections to different services now just like i had mentioned tcp is basically tcp and udp are two layer four protocols which are lower level protocols handling how the connections should be established and maintains to remote machines right http and ftp and other similar protocols are built on top of tcp to you know just just not reinvent the wheel again because tcp handles a lot of good stuff within itself and provides you a surface to build other protocols so let's first of all establish a tcp connection to this particular website that is example.com so i'm going to start an example.com i'm going to establish a connection on port 80 right and if we enable the word boost logging what you're going to see is that you can see that it saves me that connection to example.com port 80 has been established now at this particular point the remote server is waiting for us the client to say something now if i say any garbage thing you're gonna see that we get some error response from the server in this case we get 501 not implemented right so this is because this garbage is passed as a method to the particular http client so let's just go ahead and just enter a number let's say 100 and hit enter and you can see we still get 501 not implemented so let's just do a valid response instead i'm going to say get then i'm going to say a slash which is basically the path which you want to request and then i'm going to say http 1.1 which is the current protocol which we are working on there's also http 2 and http 3 but we are not going to get into them right now once we do that now it is the time to actually specify the headers now with http 1.1 what you have to remember is that host header is mandate mandatory to specify by that what i mean is that if you were running http 1 as a client you could get away by not including the host header but from http 1.1 we do need the host header in the communication why is that because at the end of the day your connection is established with an ip address and this ip address could literally just go ahead and host multiple websites not just example.com maybe just abc.com and def.com as well so host would allow the backend server to know what domain has been requested and that way a single computer can solve multiple websites so i think it timed out us timed us out so let's just try this again get slash http 1.1 and then i'm going to say get not really get host example.com and now we have to start all these headers right and the good thing is none of these headers is actually required so i'm just going to hit enter to end the request that is it ends with an empty line and you can see that the backend server responds us with http 1.1 200 okay it gives us a bunch of headers and then after a end of line it gives us a body as well so you can see this is the body of the request which is basically our web page which has been sent and basically is the same thing as what we get as a response and yeah this is how uh you know the server responds to us so for example let's just go ahead and do a get request out of 404 which has got a 404 us so http 1.1 again host is let's say example.com and hit enter and you can see right here that instead of 200 okay now we get 404 not found so all of this information this information is parsed by your browser right here and in chrome it is visible to us like this right inside response headers and some of the headers like you know not really headers but some of the information like 404 not found is used right here at the top to denote status codes and you know any sort of errors which have appeared so this is how you're going to use netcat to perform simple request to servers running on port 80 don't try to do this with port 443 because you might think that you might be able to make the same request with https as well because https literally is just http with a security wrapper around it but it's not going to work now the reason it's not gonna work is because you need to send encrypted response right here right it should be encrypted because um that's what https is right it's it is a plain text protocol but only for the http layer under the hood right https this s right here means secure so this means your plain text http is actually encrypted with cryptography and the response the request which you send is not plain text in that case right it's not something which you can write with your keyboard with printable characters so this is why for performing netcat sort of thing on um secure sites you have different tools like openssl but we're not gonna get into that because i think this should be enough knowledge for you to start back and learning path on code damn so that's pretty much it for this video and best of luck to you go ahead and start with the back and learning path on code damn and let's get started with the next course with the next things now

Channel: codedamn

Views: 8,932

Keywords: http, internet, web, http basics, http tutorial, server, express, express js, http status, request response, http request, http response, http status code, hypertext transfer protocol, web development, how does the web work, how does the internet work, request, response, client, html

Id: iD2fgC74ZtA

Length: 37min 54sec (2274 seconds)

Published: Fri Oct 02 2020