Node.js Worker Threads & PM2 Tutorial

hey everyone today I want to take a look at using node.js worker threads in the pm2 library to see how we can scale node.js applications Beyond a single thread especially when we need to execute CPU intensive synchronous code now if you haven't seen my last video on node.js and how the event Loop actually works I recommend you watch that first I'll leave a link to the video up above now where you can check it out otherwise let's Jump Right In and see how we can get our hands dirty with no JS worker threads and the pm2 library I'll see you there all right so let's go ahead and get started by using the nest CLI to initialize our new project so if you don't have the CLI installed yet you can simply run npm install DG to install it globally and you want the nestjs CLI at latest so once you have the nest C we can then run Nest new and then I'm going to call this project worker - pm2 so you can run Nest new and then choose the package manager you'd like to use in my case I'll use pnpm so now that we've created the project I'll CD into the directory and we can open it up in our code editor so we have the default nestjs application with our main.ts file being called which is bootstrapping the application creating our app and exposing our h HTTP server on Port 3000 by calling app. listen to accept incoming HTTP traffic and then we have our app controller which is right now simply exposing the route get path so we have a get route here and this is simply calling our app service get hello to return some stub string data so we can go back to our terminal and Run pnpm Run start Dev to start up the application in development mode so it will watch for our changes to our code and then I've gone ahead and opened up Postman to execute some requests against our API server so I've opened up a new request a get request and we'll launch this at HTTP Local Host 3000 and we just Target the root path and so we can see here we're targeting our app controller and returning that hello world string so we can see our app server is running as intended let's go ahead and now start taking a look at using worker threads let's first go ahead and refactor our root get route and I want to call this path blocking so we're going to have two examples we're going to have some blocking synchronous code that's going to run in our main thread to begin with and I want to show you the effects of running this code and then we're going to refactor it to then use workers so that we can move this synchronous blocking code to a worker thread and no longer block the main thread and we'll see the effects of doing this so let's go ahead and start implementing this blocking code the first thing I'm going to do is change the method to blocking we can get rid of the return type and let's accept a query parameter so to do this we'll use at query from nestjs common and I want to accept a query parameter called CPU time milliseconds so this is essentially how long we're going to ask the given request to wait inside of our blocking code so call this CPU time Ms and by default this will come in as a string so then I want to call app service. blocking and then pass in pars int which will convert a string into a number so we'll pass in the CPU time milliseconds to pars it into a number and now let's go ahead and actually implement this blocking code in our app controller so it's going to be quite easy to do this we'll simply rename the function to blocking and now it's going to take in as we know that CPU time Ms and this will be a number so let's go ahead and add this and then we can get rid of our return type and now inside of the actual function I want to create a const and we'll go ahead and call this start time and set it equal to the current date. now which is the current milliseconds value of the current code time and then I want to execute a while loop and so we know that executing a synchronous while loop in nodejs is going to block our main thread which in this case is exactly what we want to do so let's go ahead and while date. now minus start time is less than CPU time millisecond then I simply want to keep running this Loop so what this is actually doing is it's running the loop until we've reached CPU time milliseconds and and then it's exiting the loop so by doing this we're essentially blocking the main thread in node.js for this given amount of time and of course we know that in OJs since it's single threaded it's not going to be able to do any other work like serve other requests so let's go ahead and actually see this in practice I'll go ahead and Now launch another get request and this will now be at/ blocking and we'll pass in a query parameter so we use question mark CP U time Ms and set this equal to 10,000 so for 10,000 milliseconds or 10 seconds I want the CPU to just sleep and I'm going to also copy this request and paste it into a new tab so firstly let's just go off and send it one time so we can see the request run and of course we can see the request hanging because it's in that while loop right now and now it's broken out so exactly at 10.02 seconds we've returned a stat stat is 200 now the key thing about this right now is if we send this request and then immediately in our second tab execute another request we can see that it's hanging right now and we can go back to the first one and wait until it completes so this one completes in 10 seconds and the second request here didn't even finish up until 18 seconds or so even though we executed it pretty much exactly at the same time as the first request it still took much longer to actually complete and that's because while this first request was actually running and blocking the CPU this second request couldn't get serviced yet and enter into our actual while loop because this first request is blocking the main nodejs thread so this one completes and as soon as it completes the second request can now start processing now this is the danger of using synchronous blocking code in node.js in many applications this is going to be asynchronous I IO so for example if we're talking to a database or we're making some sort of network request well node.js uses the event Loop to free itself up from that task and do other work like service other requests in the meantime however since we're executing the synchronous code here the event Loop can't do its job and that's what we want to now try fixing with worker threads we want to offload this heavy synchronous work that's blocking our nodejs thread and the event loop from doing other work like servicing other requests we want to offload that work to a new thread using a worker thread let's go ahead and see how we can do that next now before we move on another thing I want to show you is if we open up a new terminal window and enter in top we were going to get metrics back to see the CPU usage for each of our processes running on our system so with top running let's go ahead and send another blocking request at our API server and as soon as I send this you can see our node process here using a 100% of the CPU allocated to it now the reason this is happening is because we're executing our while loop which is synchronously running this code on the main thread and occupying 100% of the CPU on that thread now we only have a single thread no JS is single threaded by default so we can only use up to 100% CPU and that's what we're doing we're using all of the CPU on that one while loop and that's why we can't serve any other requests in the meantime in our app controller I'm going to go ahead and create a new get route and this is going to be called worker so we'll create a async worker function and this will again take in that same query parameter of CPU time Ms we'll call it CPU time MS and it will be a string value and now I want to return this.a service. worker and again pass in that pars int call to turn the CPU time Ms into an actual number and then we'll go to the app service to actually implement this function so let's go ahead and do that we'll have a async worker function that takes in the CPU time of type number now and this is an async function because I want to actually return a new promise so we're going to return a new promise from this worker function that's going to actually resolve once the worker thread has completed its processing so we still need a way to wait for the request to complete and by wrapping this operation and a promise we can do just that so inside of the promise call back we get the resolve function and the reject function which we can call depending on the outcome of our worker thread so now inside of This Promise let's go go ahead and actually create the worker we'll create a new con called worker and set it equal to new worker now we're going to import worker from worker threads and you can see the import up here and this now takes in a file name and so the file name is going to be the actual JavaScript code that's going to get executed on the worker so to do this I want to create a new directory in our source folder called worker where we're going to keep all of this codee and the first thing I'm going want to do is create a new constant to actually Host this worker thread file path so let's create a config.sys that we can use in node to get get the actual directory name that we're in and then now that we're in this folder we're simply going to Target worker sljs and this is going to be the Javascript file we're going to manually create to host our worker logic so let's create this file now we're going to create this worker. JS so this is going to be the node.js code that the worker threads actually executing inside of the new thread so let's create this async function called run so when this file is actually run by the worker it's immediately going to call our run function and now inside of here we can Implement our synchronous work now I made this function async because I want to show you how we can actually get access to the nestjs dependency system with inside of this worker thread to do this we can take advantage of nestjs Standalone application so we're going to create the nest actually outside of our running app which we can do perfectly fine thanks to the nest Factory so let's create this app and this is going to be an await called to Nest factory. create application context so now we can pass in our root app module so let's firstly go ahead and refactor our import to the nest Factory to be from nestjs core and then we're going to manually import ort the app module from Just One Directory up and reference the root app module so this is going to actually create our nestjs application outside of our other running app and then we can get access to for example our app service by simply calling app.get and by calling app.get we can actually get any provider in our application that you need to actually implement this worker logic so in our case let's actually get access to to the app service and then we'll simply log out that we're in the worker thread so the worker thread got data and I want to show you how we can actually receive data from the main thread so to do this we'll simply reference the worker data variable that we import from worker threads so this worker data is going to actually be the data that was passed over from the main thread so let's log it out next up let's actually call apps service. blocking on this worker thread and then I want to make sure that we pass the CPU time milliseconds in this worker data object so that's what it's going to be it's going to be an object that we're going to provide so let's simply reference the CPU time Ms on it and then finally in order to pass data back from this worker to the main thread we take advantage of the parent Port so we can call Post message on parent port and then pass in whatever data we want it can be an object it'll be serialized or in our case I just want to pass the worker data back to the main thread so you can do your processing here receive data from the main thread thanks to this worker data and then pass information back to the main thread using parent Port so so let's go back to the app service where now we need to actually provide the worker we created with the worker thread file path and let's go ahead and correct this to make sure it's a lowercase a in path and then we'll do the same in our application and update the import so this is all we need to do to actually create the worker and spin up a new thread to do work on it however we also want to pass the worker data so to do this we have an options object where we have the worker data object that we can populate in this case we just pass in the CPU time Ms on it however you can pass in whatever data your worker needs now we're going to go ahead and attach callbacks on this worker which is how we can actually listen for events that happen in the worker thread so we can respond and of course eventually resolve this promise to the main caller so let's call worker on where we can listen from for an event so in this case we have access to the message event so whenever we receive a new message from the worker we get it in this call back so now that I have the message I'll simply log out main thread. message and let's include the message that we got sent back so we know this is going to be the parent port. poost message call that sends it and we're simply passing in the worker data so that worker data will be passed to this call back so after we log it out we're then going to resolve this function so after we've executed the blocking code inside of the worker sent the data back we're in this call back and then we can finally resolve this promise to complete the request I'll also pass in the message to the resolve function so we actually send back the worker data to the response when we send it back in our HTTP route cuz remember this is simply resolving the promise that we're returning back to our nestjs route Handler now let's also see how we can listen on errors so if an error occurs in our worker we can simply log it out and we'll say console. error and say worker through an error and we can log the error out and we can also reject the promise with the error so if an error occurs This Promise will be rejected and the call will know that error occurred in the worker finally we can listen to the exit command on the worker to specify the code the exit code from the worker and we'll simply log out that the worker did exit with code and pass in that code so now we have this completed worker function and we're returning a promise that resolves when the worker thread actually completes its processing importantly we need to make a change to our TS config so that this JavaScript code is actually compiled and included in our disc directory so simply go to your TS config and we need to add a new property called allowjs and set this to true now if we go ahead and restart our development server we should see our code gets recompiled and now if we look inside of our disc directory in your worker directory you should have a worker. Js okay so now back in post span I've updated all of our actual route calls to point to slash worker now so I have slash worker same CPU time Ms of 10,000 so 10 seconds and I have three requests occurring here so let's go ahead and Trigger each request one after the other and see the effects now so the request just finished and if we actually look at the response times for each request we can see that they've each completed in around exactly 10 seconds and this differs greatly from our blocking method so remember if we change the call back to blocking for example and we execute the same exact example we will see a completely different result so let's go ahead and execute this now so you see for the blocking example the first request completes in 10 seconds the second request about 20 and the third request gets up to 27 so you can see this cumulative effect of the request stacking on each other essentially getting processed one after the other even though we requested them all at the same time well we only have one thread that's completely blocked in each request so we can't do work simultaneously however in the new update example of using the actual worker the story is completely different since each time we EX execute a new request we're actually creating a new worker underneath the hood and doing this work in completely separate threads so we have three separate threads going here with doing this heavy synchronous work and they're able to be executed in parallel and that's why they return in around the same exact time now even more interesting to look at is if we execute these requests one after the other and then switch back to our top you can actually see the CPU for our our node application is now close to 300% and that's because well we have three threads that are actually running simultaneously each using 100% CPU so the cumulative total of all the threads is going to be 300% so you can see the CPU usage and how it correlates to our actual throughput since this work is now being executed in parallel on these three threads so hopefully you've learned a lot about how worker threads can help to pair parallelize your heavy synchronous work in a node.js app now most of the time in your application again most of this work is going to be asynchronous IO like Network requests reaching out to some server so you don't need to really worry about worker threads when it's asynchronous IO and that's because when you execute asynchronous IO node.js will simply register the call back associated with that call and do other work in the meantime so it doesn't actually take up CPU and that's how node.js is able to scale so well on just one thread it can do other work while it waits however when you have heavy synchronous code like this while loop for example then it makes a lot of sense to actually use different threads on your app so you don't block this one all important nodejs thread so it can serve other requests so next up I want to show you pm2 so pm2 is a dam process manager that helps us to keep our nodejs application online and utilize multicore architectures so as we know nodejs is only going to run on a single thread however pm2 is a thirdparty library that utilizes the node.js cluster module now the node.js cluster module I'll include a link in the description where you can read more about it is essentially an API that allows us to spawn multiple instances of node.js over multiple threads and distribute work between those threads now they're a little bit different between worker threads because they're not actually spinning up a whole new thread we're actually spinning up an entire new process so it's a bit more heavy weight but it allows us to distribute work between these processes and so thanks to pm2 we can use its cluster mode to actually create a networked nodejs app or in our case an HTTP server that's going to scale across all CPUs available without any code modifications so it's going to allow us to increase our throughput and performance depending on the number of CPUs so let's go ahead and take a look at how we can use pm2 take advantage of the hardware that our system is running even though nodejs a single threaded we can still scale it to all available cores so let's go ahead and firstly inst install pm2 so I'm going to use npm install and install pm2 at latest and this is going to be dasg to install it globally on so with pm2 actually now installed it's super easy to actually start up an underlying node.js app with it all you have to do is run pm2 start and then we provide the path to the actual application so in our case we know in our disc directory and the main.js is where our application is actually living and then we can specify the dasi option to enable the cluster mode so what that's going to do is it's going to tell it how many actual instances of our app we want to run and this is obviously going to depend on the number of cores we have so we can pass in the max option so that pm2 will automatically detect the number of available CPUs and run as many processes as possible one for each CPU so I've gone ahead and run run this command and now you can see that it's actually started our app main.js in cluster mode and you can see it's spun up 16 different processes of our node app and that's because I have 16 cores running on this Macbook so it's actually spun up a nodejs server well in our case the same node.js server 16 times one for each core and pm2 will automatically listen on the same exact Port so in our case it's still Local Host 3000 but it will then load balance those requests it receives to the available processes so if we send 100 requests at our server it will evenly distribute them between all of these available processes now we can see this if we simply run another get request at Local Host 3000 I'm still able to execute the request and nothing has actually changed so I still get a response back after 10 seconds and that's because our pm2 process is actually running the app and if you want you can run pm2 LS at any time to see the state of our application so I actually want to show you these processes in action so to do this we need to generate some sufficient load in our application so I want to send a lot of HTTP requests concurrently at the app and show them being load balanced between these available processes so to do this I'm going to take advantage of artillery which is a simple nodejs base load testing tool so let's create a new artillery. yaml file in the root of our directory which is essentially just going to allow us to define a test in yaml format so we're going to have a config key and we're have a Target so the target is essentially just the endpoint we're going to reach out to so in our case we know it's Local Host 3000 SL blocking and specify a CPU time Ms of something more conservative so in our case just 100 milliseconds since we're we're going to send a lot of requests so then we have the phases section where we specify the duration of the tests in our case I'll say 60 seconds and then the arrival rate so this is how many concurrent requests are being sent at once each second in my case I'll just do 50 requests so now we have the basic test set up we need to actually Define some scenarios which is essentially just the different sections of the test so we have a flow block here and within this one I'll add a loop and in the loop I'll add the actual get request we're making in this case we're going to access the root of this URL so since we're just calling this URL itself we don't need to provide anything else this scenario section just a basic block here to get the test to actually run and finally don't make and finally don't forget that the URL block here should actually be indented one more now now to make sure you have artillery on your system make sure you run npm install dasg artillery at latest and finally to actually run the test I'll run artillery run and specify the artillery. yaml so now you can see that our test has actually kicked off and we're now launching requests at our back end so now we can actually see the statistics for our artillery test including the median and mean response times for our given requests so while the test is running I want to go ahead and run top in another application Tab and the reason why I want to do this is because you can actually see all of our node processes that pm2 is actually running and the underlying CPU usage for them so you see how they're actually very similar for each process and that's because pm2 is Distributing these incoming requests from artillery and load balancing them to all of these node processes so hopefully you've learned in this example of how to scale our node.js application natively so even though it's single threaded we can still take advantage of multiple core machines by using something like pm2 and the underlying node.js cluster module which allows us to scale our application horizontally and distribute and load balance these requests between them now a lot of people may ask how is this different from using kubernetes and for example running a single nodejs pod and then scaling that pod horizontally so you have many of these nodejs pods and the answer is it's really not any different it's the same process we're just scaling out the app horizontally and load balancing the requests to them in that case it's a simple load balancer that's doing the load balancing so it's just two different techniques to get this kind of horizontal scaling of the underlying compute so I hope these two examples have been very helpful for you to see how we can actually scale node.js Beyond a single thread thank you so much for watching and I'll see you in the next one

Channel: Michael Guay

Views: 6,338

Keywords:

Id: UaAz27D0Lj4

Length: 29min 11sec (1751 seconds)

Published: Thu Dec 14 2023