How node JS works | Engineering side

here is something very very interesting I made this video not this exactly but this very similar topic video that how does node.js work turns out it was made two years ago and I forgot to post it yeah literally I never posted it I made the video and I just forgot about it and yesterday I got to remind it by a student that hey this do you have any video like this I said I have a video searched it and couldn't found it and then I found it in my backup and I I realized this is a good time to remake that video instead of posting that old one so hey there everyone my name is Satish and let's get started and in this video I'll walk you through behind the scene basics of how node.js work usually you use node.js for a lot of things maybe you are learning react maybe you're learning node or maybe a learning view whatever you're learning uh chances are high that you are using node.js or any similar library for that matters and even these days if you are learning the code JavaScript you are not learning the JavaScript in the browser itself you are using node.js to execute that so what makes node.js node.js how it works the internal model behind it and this video will help you to understand the behind the scene working of node.js you'll appreciate it so much and how powerful that is not only that once you understand this behind the scene mental model of node.js you'll be able to write so much better code with the node.js whether that's a file system access or a network access the functionality is repeatable it just repeats everything exactly same with different libraries but behind the scene working always remains same so welcome to the video where we are going to explore behind the scene of node.js how it works we'll be writing some portion of the code you don't need to understand that code you just need to see okay this is the code this is the portion where it works and then I'll break it down step by step in a diagrammatic manner that how the each steps are being done it will help you to understand a lot about the computer Basics as well in case you are not familiar with that but yes I'm still laughing that why I didn't posted that video I need to dig up with that anyway so let me share my screen first of all when video to actually get you understand all that so I'll just move this one a little bit all right so let me first share that how first understand the scenario what we're trying to build is we are trying to build a web server with node.js this is the most easiest thing that you will be doing and a lot of people use libraries like Express and all these other things we are not using that I just want to show you how coldly it is done and what are the important parts and pieces of that and later on you can use those libraries and those Basics and all of that but I want to show you that how it is being done in the world of node.js so that later on we can break it down and understand how it happens behind the scene and in the computer networks and Basics how it is all done so that's that's the portion which always excites me the most all right so first of all let me share the screen uh so this is our screen this is the code file I was working on some other stuff so then I realized I should make this video so let me walk you through that how it is being done that's the most important easiest thing so node.js you might have seen uh that hey you have these Library access so I can just go ahead and actually require these libraries so if I go ahead and say Hey I want to require these libraries a lot of libraries are available for me so require and there is one special Library known as HTTP there is very similar Library which does almost similar kind of thing we'll talk about that which is a net Library we're not interested in that and I'll just go ahead and store a reference of it so I'll just go ahead and say hey const and let's call this one as HTTP feel free to call it whatever you like uh why and how I'm able to do this because node.js allows me to have the access of this Library I'll walk you through with the node documentation that is super important for that but give me a minute on to this part first all right so let's just say node.js gives me access to this it would be difficult to say that these are modules purely but let's just say these are labels because uh this is something which actually can control your network of your computer and JavaScript trust me has no capability of controlling either your network or your file system this language doesn't do that for that you need languages like C or C plus plus so this HTTP label is actually holding the references of the capabilities of networking yeah that's already crazy for me I'm I'm very fascinated about this science all right so what this HTTP allows you is it gives you a lot of methods and lot of capability so notice here it also says the HTTP interface in node.js so yes it's a label interface it's not your ordinary terrible it has so much of more functionality that you can actually invoke do so much of the powerful things in that one of the method that you'll see a lot is a create a server okay server it's nothing it's just a computer somebody else's computer which is up and running all the time and is listening for some request somebody is sending me some request and I'll serve that that's why it's called as server it serves literally so that's what it has and what you'll see inside this is usually there is a request listener and a response so usually this is how you actually kind of memorize this and they would memorize it but this is how it looks like so this is a callback in it we go ahead and say this is request and this is a response I'll walk you through what these request and response literally are I'll walk you through with the documentation as well this is so much of an exciting stuff and then you'll say that hey I'll actually go ahead and store this entire thing into a variable so this is going to be const we usually call this as server that's how we do it and obviously this server actually lists sent to some Port so you can directly go ahead and say that port or you can just go ahead and say hey Port is Port is either from environment variables whatever that is or maybe four thousand whatever works for you in that case once this variable Port is defined this is just a numbers the magic actually comes from the server which actually go ahead and listen to it and inside this you go ahead and pass on this port this is it optionally you can pass on a callback that hey once it is listening or for the first time it actually ups and running is up and running then you go ahead and fire up a callback just like this and go ahead and say either in the curly braces or directly a console log that says something like this server is running so this is your very basic of it what's more interesting is when we created this server we can actually have the access of this request and response which is the most interesting thing in the entire node.js this file system as well but hey let's come back so into this one you'll notice that this response has a lot of methods and this request has a lot of properties for example from the request I can grab what path user is trying to access even with a little bit configuration I can access its cookies uh who is sending me at what time it was sent Header information browser information there's literally so much of information into this which I absolutely love about it but that's all about request the first parameter but the response is actually a collection of methods and there's a little more of it there's actually streams that are coming in I'll walk you through the exact technicalities about it which you'll appreciate in a minute but you can go ahead and find so much of this for example I can go ahead and change my status code that status codes becomes 200 something like this you can set the headers there's so much more you can do but mostly at the end of it you are going to go ahead and say end and at the end of it you can go ahead and say buy something like this so this end is actually an end of a stream that is being sent from the server to whoever is making a request so that's the basic all right so I'm not going to run this code this is really basic obviously it's going to work but there's nothing too interesting here you could have done this in one line or probably less or more better you can say better syntax by using the libraries like Express but that's what not the behind the scene understanding of the node.js this is all what you have done so far now I'll take you back on what is happening and how you should be learning the behind the scene basics of it we have learned that how we can create a basic server that's all given to us but now let's go ahead and walk you through with how you can actually take all of this into the documentation part so just give me a second I'll bring this one up all right so this is what we have okay so this is the node.js page and this is a lib UV we'll come up come back on to this one we call this as I actually not coined this term it was given in a conference libuv and node.js are onion layers the more you'll peel it the more you'll cry so again not my words taken from a conference okay so in the node.js most of the time what people does is either they choose the long long term stability version LTS or the current version they always go into the API docs no matter where your API docs you goes you just see that hey this is buffer cryptos and http module you usually store and learn from it okay hitesh talked about HTTP this is where I should be looking up for that like what can I do okay I can access request dot method request.path great way great way I'm not saying that you're doing wrong this is one of the great way of learning but what you missed nobody told you is how you should be learning the node documentation first before learning about its modules that are given to you yes these are modules so you should not be learning from them like you should be learning from them but not from the very basic go on to the home page look at the top this is the docs this is the docs you should be exploring first let me walk you through how to explore this so click on the docs and this is where all of the references guide es features and everything is available this guide is the most like not most least read by most people even I've seen the level one level two Engineers don't read about it we are blessed we are on YouTube and I have my own time to read about it that's why I go into it so go into the guide section and this is where you see the general guide the node.js core concept and the module related guide what I have found is there are some people who learn and study about it they want to learn about the node.js core concept so they'll go into this one again going directly into this one I found that it's not really ideal to learn there are little higher level concept if you request me in the comment section below I'll try to make separate video for each one of them which will be again super long but very very fun but we are not gonna go into the core concept the core concept still it is core concept is the module related guide one the guide which I actually love is the this anatomy of HTTP transaction so they literally dissect each part of what happens in the HTTP request which I'll also do but notice here there is so much packed knowledge here once you understand this beat this piece of information then you understand everything how file systems are accessed how crypto modules Works everything in node.js because trust me your JavaScript is not capable JavaScript is is not at all a capable language but it's the node.js apis that makes it so much capable of doing so many things so if I go ahead click on this they actually break it line by line and couple of points which I would like to mention here is first of all notice here the language which says we also assume you have a bit familiarity with the node.js event emitters and streams yes you should have but don't worry I'll explain you what that is in a brief so you don't have that much of requirement notice here they break it line by line that how the create server works and all of this how the methods URL headers and all of that work will not go into the request body because that's also another Beast to actually discuss about errors we are probably not going to talk about in this video but I'll talk about the basics of how the response headers are being said or how the general connections and everything is being done and where does this live UV actually uh takes part into this one so this is really something very very interesting that I'll walk you through so okay so this is the portion where I'm taking all of my references and notes this definitely takes a little bit of interpretation I'll probably discuss them in some another video but here let's go ahead and understand this not from the notes perspective but from the interpretation perspective so I'll go back here so this is my iPad and we'll try to see that how this is all being done all right now one of the things you need to understand that how the server client relation works and where your computer comes in where the node.js comes in and whole bunch of other things so I'll use a basic diagram for this one so let's go ahead and work on with this one so let's just say this is your client all right again please excuse me my bad drawings and all of that I'm not really crafted in that okay these days every request comes from mobile so we'll just go ahead and say this is the mobile and this entire thing is your server so yep that big of a stuff this is your server okay why this big office server no servers are not really that big they are just a computer which is up and running all the time but since there's so much of concept going on that's why I need a bigger diagram for this one so let's just say there is a request that comes up so usually the request that comes up to any computer or any kind of server like that is going to come up on Port 80. it's usual but you can obviously go ahead and change that here in this server there is a long request of there's a lot of ports more than 64 000 and we have to make sure that on one port we are addressing the situation that hey there might be one request that might comes up and we are listening on that and that's why we say that we are listening on the port number uh 4000 or something that's actually happening in the development segment only you never ever go into the 4000 because all the requests by default comes upon port 80. and when you actually spin up your servers like you might have heard about pm2 or nginx or these kinds of servers Apache it's their job to actually listen to the request and then redirect into your application and you can configure them that hey my application listens on eight thousand four thousand whatever that is but the ideal interface on the top always listens to Port 80 that's that's the basic computer science okay now what happens after that is uh we'll change the pen uh we'll go with the blue one okay so this portion is actually a computer okay uh I'll just call this one as literally as computer okay really bad but this is computer and another one we need another pen the green one because that's the node and this is the portion where your node.js works and we need another one probably yellow one this is the portion where your JavaScript Works happen yep ah that is how the entire system is divided and if you'll read this this entire thing happens like that okay so your node.js doesn't have the default capability to have the networking access because hey end of the day this is Javascript JavaScript don't listen to the ports it has not the capabilities to have an access to your interface card which is known as your network card so your Nic or your network card is your core computer thing it is not going to be read by that and this is exactly where a new thing comes into the picture which is known as your lib UV so your lib UV library is majorly in C or I'll just walk you through with that majorly in the C but this is where the live UV comes into the picture so this is a multi-threaded platform support and blah blah stuff but if you look at the documentation of this one and I just want to see the design overview there's so much on to this and when people actually say that hey can you explain me live UV they always go here live UV is a library which is in the node.js has a thread pool has a e-pole and all of that event port words yeah that's great that's great you understand about the i o operation and all of that but what you don't understand is where does it comes into the picture because this is a library which is an interface between your computer and your node.js your computer understands the networking requests that are all going to come in but somebody needs to pass on these requests to node.js that's where your libuv comes into the picture that's the first part of behind the scene in the node.js now you understand that part will not peel this a layer more because as I said earlier libuv and node.js onion layers the more you peel the more you cry so this is the basics you should you should know basics of it now whenever there's a request that comes up this request is not into the text format it is not something like hey uh Hey slash hitesh or slash login this is not how it works it actually comes up into streams of data and this data is a bytes of data so bytes and streams these kinds of words you are going to hear a lot about and now if you want to have an access of this you can technically that's a stream of data and for that net library is available in the node.js but we are not interested in that so that is why we use another Library which can actually talk to there and in the node.js we saw that there is a library which was written to me and that's where your HTTP Library comes up this actually converts this into literal strings that hey I don't want to deal up with the strings and bytes and all of that I'll just go ahead and work with the strings so I'll just go ahead and say string so to understand the strings we actually go ahead and directly talk to the string and HTTP Library that's the part of it okay now a couple of more things which are there so whenever a request comes up this lib UV Library actually has some interesting things going on it actually go ahead and convert your will go with the yellow pen it actually is going to convert into two boxes the box number one and the box number two this box is going to be majorly utilized for an incoming section and this box 2 is majorly utilized in the outgoing section and in case you haven't got hint yet yes these are literally your request and response the thing that we saw in the code part where is my code part here is my code part the one we said ideally is request and response yep these things these exactly are just boxes and yes their boxes literally boxes the first box and the second box literally they are big JavaScript object yes and this guy also is a big JavaScript object that's it and the entire request that you have consists of a lot of properties from where it is coming up what are the headers and a lot of information and most of this is actually inside this one and they are props or also known as properties because we have to just extract the information that's why they are in the request and there is no such thing in the entire node.js which calls them always as a request they actually have different names for it that how this actually is being done or call if you'll study this a little bit you'll find uh that hey create servers and event handlers and if you listen to this uh there it is is an instance of incoming messages so they are calling it as literally as incoming messages not a request they call it as incoming messages which you can study more that how this actually works uh remove headers send dates and whatnot and all of that but will not go into that much of death just wanted to prove the point that all of your methods URL they we call it as request but we can call it Superman there is no such thing which is stopping us they are an instance of incoming messages and in this incoming messages what we have is majorly properties yeah that's what it is on the other hand the next thing that you see is the response that you send this is majorly comprises of methods yeah they are mentioning this in the documentation itself that first one is majorly property the second one is majorly method it's not like they don't have any methods or properties in each other but yeah majorly constituted of methods and while the core method that actually everybody needs to take care of is the dot end we saw that in the in the one as well this is the method which is actually at the very end of your stream yes that's why they say you should know about the stream because your response is a stream chain you first send the status code then you send the Json response and then you end it so that the other guy knows the the stream has ended there that's why this is most important and that's why you end it at the very end since majority of the people work in libraries like Express they have never seen that end because Express does it automatically behind the scene probably will someday talk about the express behind the scene that's also very fun but coming back now uh the most important part is coming back onto the part now you understand bits and pieces of it coming back now we understand that okay this is all is happening but when we create the server this request and response all of that how does that triggers that's also a very big big mystery that how it works so what happens in the JavaScript uh there are event emitters which actually we saw that hey you need to understand about the event emitters and the stream that's where the portion of event emitters comes into the picture so in the JavaScript all you do is you define different routes and what happens so for example there is a route uh let's just write it so you wrote a route of Slash hitesh and you also wrote a route for hitesh and then insta and of course how can we forget YouTube so you wrote a route and on this a different thing happens so here probably you uh show profile uh show profile and here you actually talk to database talk to DB and get some data and here you talk to DB and then you post some data in your database so whatever that happens that's actually your core JavaScript capabilities and you never run those functions because if you take a moment and think about how JavaScript executes the thing it's actually single threaded so and it's a constant execution it keeps on executing the moment you save the file if you have put some parenthesis around it that will execute the code but you never do that you you don't execute your file you execute your file when a request comes to you so yes this is another job of this live UV to find out out of the properties that hey you know what somebody is requesting a URL of hitesh so what should I do this is where it emits an event and it's the job of node.js to actually look into your code file is through the JavaScript that hey somebody is requesting this URL and based on that it actually executes your function so it keeps everything in the memory sort of that's very fascinating that you don't execute your code you execute your code on some event that happens already so much is happening behind the scene this is what event event emitters actually comes into the picture so you go ahead and execute this code not only you execute this code the moment you actually take all of this code uh the sitesh all this code so I'll just use another pen something yeah Green would be good so here also you execute this so execute this by using parenthesis yes these squiggly lines which I'm not able to put as parenthesis these are execution and it's the node.js who execute that that's why they are in green node.js execute this node.js execute this and node.js execute this but not only execute this here's the here's the most crazy part that it took me so much time to understand this node.js actually literally uh take these objects and gives you access in each one of them directly here so how should I do this so they actually go ahead and inject your request and your response into this and that's why in each of this you'll be able to see request and response yeah so much crazy thing so imagine what is happening here a request comes into your uh servers it's in the bytes and it is being converted into string by libuv it actually tricks all the request headers that are coming in then it's crafting an object out of it yeah behind the scene and it insert all the objects into it a properties like your request headers your browser information your IPS and whatnot all this information into this object and then it also emits an event that hey you are looking for a slash route out of this object picks it up then node.js looks for an equivalent function that should be executed adds this request and response this object up here into this places and then once you are done it actually streams that response back you have no idea about it gets into the lib UV again and then it transmits it back to the user yeah behind the scene of node.js this still bothers me so much that so much superpower and imagine JavaScript is doing nothing it just is an interface you're literally doing majority of work in your libuv so it's majorly C C plus plus all you're doing is writing what should happen and if this would not be JS it could be any language it could be any language but since JavaScript is one of the simplest one that's why we write majority for code in JavaScript and call it a day but hard work is done by the node.js to imagine so much of injection into the knowledge so much of the things that is being done so Ah that's that's so much already uh an intense software work but I'm glad that I didn't uploaded that video two years ago because I think I'm able to explain it much more easily now and probably I'll do that uh probably more more explanation is required but again you get the idea how the node.js works behind the scene now and as so much is happening now once you actually understand this so let me walk you through again uh once actually you understand uh this portion of it that hey this is how it is being done and you have the idea about this mental model that okay anytime I try to access so this is actually an interface right now we talked so much about the HTTP interface only but this could be a file system interface but hey the mental model Remains the Same in the file system there might be another Library which gives you access of the file system node.js does ton of work for you and you just calls your JavaScript add some methods into it probably will talk more about all these things if you wish let me know in the comments but this is the basics of how node.js works now let me also walk you through with a little bit of behind find the scene into the docs as well so hey we need to study about that and now you'll be able to understand a lot about this talk so let me walk you through so first of all it says create server so now you understand why it says event emitters and streams because hey that's what the code Running part is so node.js server application by using the create server we know that the function that passes create server is called once every HTTP request that's made against the server and so it's called request Handler yeah that's why it is request Handler it serves on the request okay when HTTP request hits the server nodes calls the request Handler function with a few handy object for dealing with the transaction request and response literally takes those object inject them into those methods that's what they are saying in order to actually serve requests the listen methods needs to be called on the server in the most cases you'll need to pass a listen port number blah blah we understand that part this is where they actually mentioned that node.js make this relatively Painless by putting a handy properties into the request object this is the Line This is what makes your life easy so much of the so much and again request is an instance of an incoming message that's what they're saying uh the method here will always be normal HTTP method verb get WordPress whatever that is and request and headers so there is so much into your request that you can just keep on extracting keep on digging yes you can extract your raw headers and all of that I have deliberately skipped the part of request body and errors because that's a different beast and probably we need a separate video to talk about it but this is what is happening and then when sending a response body notice here it says this is the portion I would like to bring your attention to notice here it says response.n this is a stream of response that keeps on coming you can keep on writing keep on writing keep on writing and that's where libraries like Express makes your life easier because this is not programmer friendly keep on writing this much of this syntax is is really bad idea that's why that Library comes up and allows you to send response in Json and all of that but all in all look at this since the response object is a writable stream we'll probably talk about more if you wish into streams more but it's just a stream just like a water stream it keeps on coming until you hit the end the end function on stream can also take some optional data to send the last bit of the data but it is sure it is last bit but you can send more off to or to it but given the fact this is the end of it again there's a whole different talk that can be done about the errors and Echo servers and whatnot but you get the idea that what I'm talking about and why I call this as a gigantic behind the scene of node.js I hope you have enjoyed this video and now you'll be able to appreciate more when there is a node.js whether that's in react or node.js or wherever that is that how much work actually you are doing with the node.js I'll probably make more such videos of behind the scene if you enjoy them please please do share them I only get the notification through the LinkedIn that you are sharing them you are appreciating the work that motivates me to create more such behind the videos if you like them otherwise I'll stop doing it I have no intent of making videos which nobody is watching so please do support and share subscribe if you are enjoying this I'm enjoying this so let's go ahead and catch up in the next video

Channel: Hitesh Choudhary

Views: 49,512

Keywords: Programming, javascript, nodejs, engineering, how nodejs works

Id: ooBxSg1Cl1w

Length: 29min 14sec (1754 seconds)

Published: Mon Jul 24 2023