When is NodeJS Single-Threaded and when is it Multi-Threaded?

so guys node.js is branded as the single threaded asynchronous non-blocking javascript runtime right but we've also been told that that's not entirely true because node.js does use threads in certain situations so in this video i want to kind of demystify this thing a little bit and understand when node.js is a single threaded this event loop that we talked about all the time and when does it really use threads and what does that mean for our applications and can i utilize node.js threading model if i really understand it how about we jump into and understand that so we can start with the event loop real quick the event loop is literally an infinite loop that runs as long as the process is running it's a single thread that is literally does multiple checks it checks certain things like let's say there is a compute that needs to be done by node.js i don't know adding two numbers this is a compute right and then this is an execution so it will check hey do i have any compute that you know to execute do i have any functions i need to execute let me execute it and the another thing is part of the compute is essentially sending a request to say hey do are there a request that need to be sent right to to another server and here's the interesting part you can compile that get request the act of compiling that git request putting the parameters building the payload and then sending that payload that's compute that get execute but once you sent that you're effectively done right you're not gonna be blocked and waiting for that response from the server because we we said that stuff we send it we sent that resource oh no we need to wait for the server to respond meanwhile node.js actually sets a callback say okay by the way when we get a response from this particular function for that particular call here's a call back and let's put it in this shelf this is a list of bunch of callbacks callbacks from timeouts uh it's set timer the timeouts callback from functions callback from other operations as well all of them let's set them here okay certain operations not all of them the next thing the event loop goes okay do i have any callback that is ready to be executed did i get a response from this server well no but there's another thing here there's another call back from a timeout that we did there is a timer is this timer already ready for us to be called uh no there's like a three more seconds done okay let's move on is there any more compute that i have to do well yeah there is like a some operation that you have to do oh okay let's do that operation okay done so while you're actually computing this that is blocking operation you the single for that's eventually that executing that compute you're gonna be blocked so if you're if you have in your code a while one while true you're not just gonna stuck forever in the event loop because hey you just hey i'm i'm computing there's nothing asynchronous about this you gave me a compute that happens to be an infinite loop i'm gonna be stuck in the infinite loop the main thread the main event level is going to thread so be careful with your compute that's why people just like hey let's be careful with this event loop thing that how much computer you put down all right so now we execute this computer we move on and then hey there is another network network request that we need to send uh sure thing let's do it let's do an uh this asynchronous network request send it and then move on right that's asynchronous non-blocking so there is no blocking in certain situations all right so that's that's essentially the event loop very simple and very elegant we love it absolutely love it but there is this thing obviously guys this design as we talked about imagine we don't have threads imagine just we have only the event loop we're reading from the file reading we're doing a lot of queries that is i o we're doing a lot of cpu intensive operation not just like out of the box does all the stuff compression encryption this is cpu intensive so they take time to execute it's really hurtful to put this in the event loop because if you put in the event loop to compress a file then the your beautiful app will suffer if i do a lot of read requests to fo to the file system or another popular example is dns queries because not just doing dns all the time because you you i don't know you're trying to do a fashion network query you're trying to ping something you're trying to essentially resolve the units whether whether they are packages that you import or there are just things that you actually do that require dns resolution to get the ipads from host names these are expensive we're going to end up with a lot of callback even if they are asynchronous right we can end up with a lot of callback we don't want to overwhelm the the main thread with this kind of operation that's why node.js started using a library called uvlib all right and this is a c library that you can just use it it's just important and start using it it's an application that supports threading and here's what node.js uses for it uses it for two things i o intensive operation and cpu intensive operations so let's take example of i o intensive operations so here's the only these four things are the only things that we know of that node.js use the lib uv threading pool with okay and these are dns squares so anytime there is a need to resolve a host name to an ip node.js not gonna use the main loop it's then a user thread and it's gonna use basically this lib uv library the built-in library that is being imported to do this resolution another thing is file system anytime you do an asynchronous file system read and mined that i said asynchronous because the built-in file system you can do to both you can do synchronous and and asynchronous the synchronous will use the event loop the asynchronous will spin up a thread to do the read so that it's just it's it's it's it's an own thing it's not bothering essentially it's not bothering anything and it's essentially you can you can spread them as much as we want right it's not bothering us and the other part is the crypto library the famous crypto library any encryption then it uses the lab uv uh the uv lab library uses the threads so crypto encryption hashing all that stuff uses the threads so you can bump up the number of threads that you want to take advantage of that compression the zip library you just uses the tripod and you can use this environment variable to bump up the thread pool number so that you can take advantage of these if you know you're using them okay that is what i want to talk about here if you just bump up this number and your code doesn't utilize threads then it's useless and that's what i i seen i've been seeing some people make this mistake they say oh i'm just gonna build an http server and my code in the middle they just put their code in the middle say my computer is gonna scale because i'm gonna just bump up my threads to 200. now you're really gonna you really need to know when the threads are getting trigger and i'm gonna give you now a bunch of examples and uh let's make it like a fancy quiz how about that all right guys let's take some examples so here's the first example example number one we're running an http server very simple server that just returned hello to the server when we make a get request we immediately return hello and matter of fact any request that makes in the server we're going to return hello what is this going to use is this going to is is this going to use the main thread to execute this or is going to use the thread pull the answer is it's only going to use the main thread because hey what this is networking networking operations always use the main thread it never never uses the thread pull okay network operation except for the dns right so this is a part of the network we're not going to use the thread pull we're just going to use the main thread to get to execute this part all right so if i run this node example1.js and then i'm going to curl let's do curl dash v http localhost 1337 lead all right you can see that we established the tcp connection nicely we sent the get request and we got hello back nicely done this is simple this is not rocket science let's take another example here okay let's do example two now if i run example two node.js and i do this i want to pay attention to what happened i want to do dash v curl what do you think will happen here will we be able to establish tcp connection let's find out if i do that look at that it says connected so we can't technically connect to that we send the get a request and if i check in wireshark we even acknowledge the server let's be very specific the kernel's server responds with an acknowledgement to our request but the request sits in the queue of the application so we're stuck so we did get the request in this case but we couldn't even respond so let's prove this by creating another request here do a curl dash v uh localhost can i let's give him http there you go now i established another core request that means another tcp connection which worked right the connection we we connected to the server we sent the request but this time we acknowledged it but that request never even got delivered to that to begin with right so let's let's let's let's close these two requests and and add something here so you can essentially see right console.log request delivered delivered and then all right let's do requests delivered let's do the same thing again close clean boom run again boom localhost you can send a run request we did a curl here we got request delivered that's the first request that's the first infinity request right so the request got delivered to the app right and we got stuck here now another one let's prove that this is actually a single thread right if i do another call see that i established the tcp connection i sent the request but the request never got delivered to the app why because i did not get another request ever so be very very careful what you put in your http request whether this is express or this is just normal http server if you put a huge heavy compute here right here your app is going to suffer and no worker thread pull can save you from that it really depends what you do here that's why that's when you need to use cluster and i have i don't have time to explain all that stuff right we need to go through the video real quick all right jump another example three example number three dd okay for example number three we're it's a simple http server but it reads from a file http.js just raise a file and then calls back a function which is the console.log so what do we what what are we gonna do in this case well the networking aspect of this is definitely an event main loop but remember the execution of that work the file reading the file is asynchronous and we said any file operations that is asynchronous we're going to use the thread pool so now if you have it like you're reading i don't know if you're building a web server and you're reading all these static files to serve it to your client then thread pool's definitely useful here example number four example number four didi diddy so same exact thing but we're using the we're printing immediately the read of file synchronous what are we going to do here in this case synchronous operations goes into the event main loop so this is a blocking operation right so now when the event with the event loop goes into the part where anyone reads the file it's going to use the synchronous operation so it's going to be put in the event main loop okay all right example number five we're doing a networking operation to read from google and print the text that we comes from back from google what node.js is going to use is it threading is going to use the event main loop or the thread it's actually a combinator of both because this part is a dns query before we call fetch fetch will call the dns package in node.js to do the dns resolution to get google.com that uses threading so if you have a lot of domains that is safe you can do all kinds of dns operations there however the network aspect of that the actual call to fetch to make the tcp request to make the tcp segment to send the that request to get request to get the back the result that's in the event main loop because it's a networking operation and as a result goes into event main loop it says asynchronous non-blocking because we're going to put it as a callback because of beautiful promises right final example here is this is how you basically utilize the thread pole you do a uv underscore thread pull underscore size okay and this is the uv lib library by default is four okay and you can increase it to any number you want again based on what kind of operation you're doing so if you're saying do if you're seeing yourself doing crypto or doing encryption or doing compression a lot uh then thread pools are definitely very very useful right and uh you might say finally you're saying how can i make like how can i make my node.js let's end up this video by the by saying this how can i make my node.js actually scalable because node.js is a single threaded when it comes to building a web server or a service right so in that case you can use clustering i'm going to what you're going to do is require the cluster and with the first node.js that runs checks hey am i the master am i their primary node.js app yes if you are which is the first one you are then you are the master process then i am going to that's a very um that's a best practice to use the number of threads as the number of cpu so you can essentially execute them in parallel right there is also the cpu itself there's a physical cpu and there's physical threads on the cpu so like as of course if you can have multiple threads those are like again these are like time slicing essentially not real threats they give you the ability to give you the concurrency right but it's not really true parallelism and then what we do here is just loop through all the number of uh might as well just use the actual that variable so we're going to loop through a number of cpu and spin up n number of forks essentially essentially literally different processes right and then that process will use obviously the same code when it comes that it knows it's not the master right that it will go to the else and we're gonna listen to the same part one three three seven right and okay that's what the first process listens to one three three seven ready to take accept stuff right and we're gonna explain what happened what happened if i spin up another process another process we're gonna go with the same code and again listen the same server you might say saying that that's just that's a bind error right there no no no node.js have all sorts of hacky stuff in this function that's called listen what they do is this function actually have knowledge whether you are a fork or a primary if you are a fork it never listens on itself it asks its parent to listen for it so the first process the first fork process will ask the master process to listen on port 13337 on that shirt socket get a file descriptor and then the second process will do oh hey master do i have this listen you go okay let's just use the same thing it's gonna start hooking on the same all of them will share the same socket essentially that's a very powerful concept right so they essentially this master process will will manage the mutexes and then all that stuff so you don't get locking and all that jazz right uh there there is a lot going on here obviously so there's in the uv library here there's an option called except simultaneous option uh sockets this has to be enabled and it's enabled by default to properly do whatever you're doing right here to accept multiple uh connection on behalf of multiple sockets all right guys that's it for me today i'm gonna see you in the next one hope you enjoyed this video i'm gonna see you in the next one you guys stay awesome goodbye

Channel: Hussein Nasser

Views: 70,852

Keywords: hussein nasser, backend engineering, nodejs, nodejs threading, node js thread pool, ndoejs libuv, nodejs libuv, UV_THREADPOOL_SIZE, thread pool, thread pool node js, networking node js threading, threading node js, node js

Id: gMtchRodC2I

Length: 18min 42sec (1122 seconds)

Published: Wed Mar 24 2021