Node.js Performance and Highly Scalable Micro-Services - Chris Bailey, IBM

say good morning today I'm going to talk about building and deploying highly scalable scalable no jess micro services and how to verify the performance of those micro services and do Diagnostics for performance when they're deployed I'm Chris Bailey I'm the chief architect for node.js and Swifter IBM and I'm joined by my colleague Chris Vignola Chris works on cloud native node.js so that's making no js' ready to work in cloud deployments so before we get started we're going to talk about some key technologies for cloud native so the cloud native computing foundation which is part of the Linux Foundation which is where the node foundation sits as well they talk about a number of key technologies one of those is docker so who has using docker today okay that's actually the vast majority of you okay so how about kubernetes is a way of deploying clustering docker okay that's a lot less possibly about 20% okay so how about helm anyone using helm okay a small handful how about Prometheus as a monitoring system for metrics okay a few more than helm but not many and finally what about open tracing in Zipkin anybody here to to people okay so there may be two people in the audience you don't learn anything in this session sorry but for everybody else hopefully there's going to be a lot of new stuff and hopefully it's going to be interesting and informative so bit so those are the key technologies we're going to be talking about but first of all this talk about micro services performance so what's the key characteristics for a micro service when it comes to performance well one of them is actually about io performance so when you're deploying micro services the idea is micro services small single task focused capabilities and you build up an application using several micro services so that means there's gonna be a lot of connections for any user responsiveness anything happens in the browser it's going to involve talking to an API that has several micro services behind it so actually IO performance becomes really key for the overall performance of your application and for being able to do large-scale so this is how Express j/s does on a on a benchmark that takes a request does a database fetch dessert and an update to the database and does a response it conover aykroyd operation in nodejs and here's the equivalent from spring boot which is a micro services framework in Java so for this IO based benchmark Express is actually faster than spring boot Java now actually if you move to really highly computational tasks things that use a lot of CPU Java tends to be faster but you don't do highly cpu-based workloads in nodejs because that blocks the event loop and we all know we shouldn't block the event loop so for i/o based workloads nodejs is actually faster than Java ok a second key characteristic is startup speed so when you're deploying microservices if you want availability if there's something wrong with the application you shut it down you restart it so that time to restart actually affects how fast you can react to to an antigen get availability it also applies to scaling if you have additional load you want to add new instances of your application right you want to have not just one node Express application you want to have five and if there's a lot more load you want to grow to six seven and eight now your ability to react to incoming load depends on how fast you can add new instances so your ability to scale it's kind of dictated to how fast you start up so looking at the same things again this is the style top truck time for Express so around zero point nine in set zero point nine seconds to start a node Express application the same in spring boot takes almost fourteen seconds so on star top speed nodejs significantly better than micro-services frameworks in Java and finally when you deploy applications to a cloud most cloud providers charge you according to the amount of memory that you're using when you deploy your application so the more memory you use the more you pay and secondly if you're running this yourself on your own machines the amount of memory and application uses depend it takes the amount of density you can have how many applications you can deploy onto a machine so if we compare memory we can see that nodejs uses something like 23 megabytes we look at the same thing in spring boot 422 Meg so use a lot less memory using node versus Java as well so all in all nodes actually fundamentally a good language to be deploying microservices not only that but if you look at the works been done by the performance benchmarking workgroup so they are looking at a set of around 10 or so benchmarks and some of those unsurprisingly align very well to IO performance startup and memory and this is how they've been doing from release to release so in no version fall we had a performance on a throughput through put on a benchmark called AK V air of 1866 in node 8 we're up to two thousand nine hundred and fourteen so released by release there's been improvements in i/o performance in no Jess and you saw some of that information in the session about v8 this morning the work that's been done in the v8 JavaScript engine owned under nodejs is enabling this startup there was a big drop when we started to focus on startup performance and it's been roughly the same released release since then and memory footprint that's almost static across releases and to achieve this improvement in performance at zero additional memory cost is actually really quite impressive engineering so node is good for micro services and the performance benchmarking workgroup is focused on keeping those characteristics good so how do we build scalable Micra services so first of all you build let's say an express app and when you built an express app you probably have these files right you have the public directory of storing for serving static files you probably have some tests you have a server JS file with your express application in it a package.json read me maybe ignore file and your server file will look something like this if you're doing just a hello world app now you to make it ready for microcircuits the first thing you need to do is add a docker file and a docker file is going to contain something like this so this says I need to have a base image that has no js' in it I need to open a port I need to install my dependencies I need to copy that into a directory and then I need to start it and expose the port you might have a docker file for tools so that's be able to run the same application in a debug mode or to run tests rather than your production mode and probably a docker ignore file and that says when I'm building my docker image don't copy this stuff over and one of the reasons you'll do that is your local development environment might be Windows or Mac your deployment environment might be Linux so you don't want to copy over you'll note modules in case there's native modules in there ok once you've done that docker build docker run and yes I have node running in docker and I have a micro service now the problem with this is I don't want one I want several and I'm gonna need to do that if I have to be able to scale line and deal with large levels of load so how do I go from one to many and that's where helm charts come in so helm charts are a bunch of additional configuration files that you need to add to your project the first is just a basic chart this is kind of like a a name and a package so helm charts they're kind of like your package management system and deploy system for kubernetes so I have a helmet art inside that I have a very complicated deployment llaman file but to explain how many replicas I wants that's how many instances and what ports I'm exposing whether it has any dependencies I have an HPA file this is horizontal pod auto scaling which means how do I add and remove instances so what this will do is this says I'm going to start off with five instances if the CPU usage for all five instances goes over 70% I'm going to add in an additional instance and I'm happy to grow up to ten instances so this is the configuration file you need add in order to be able to dynamically scale in kubernetes then like I have a service channel this means I'm exposing a service for other people to use if I wanted to use some services I would put that in there as well and there's also a values camel so there's a way of doing templating and sharing configuration across all of these files so I build a psalm charts once I've done that I do a he'll package and a helm install I get my five instances because I started off with five and through auto scaling I can grow up to ten if I need to so between docker and helm charts I can now deploy a scaled micro service okay so but once it's there how do I know that the performance I had for my single Express application on my laptop matches when I've deployed it to this cloud cluster and there's ten of them running and it's in an application that has dependent services so the challenge for micro services performance is this is what a micro service might look like you start off with a browser or a mobile device and that's how users interact with your overall application so a browser will make a request it will go through some kind of load balancer and it will come to your your web delivery tier now that's going to be no GS express or loopback or happy or Kraken from there in order to handle that web request it might need to make use of some back-end services so let's say we've got a user who's going to to order something from an online catalog so that web request needs to use the order service and that mate make a request of a database to I don't know look at previous orders get an order template something like that then the next thing we're going to do is we're going to check to see whether we've got sufficient inventory to be able to actually satisfy the order so we go to the inventory service and then that goes to a MySQL database to find out how much inventory we've got and then that's going to return back to the order if it's ok the order might write to the database and then we have an end transaction so what this looks like in terms of the timeline is from the browser the browser sees it takes this long for the whole thing to complete now inside that the load balancer sees the transaction taking a similar amount of time the difference is a bit of network latency under that you have your your web app behind that you have an order service you have the MongoDB call you have the inventory service call which then calls MySQL and you have the second MongoDB cool so I need to be able to understand where in this series of transactions all of the actual time took to understand where my performance bottleneck might be so this is our challenge it's where I have this interconnected mesh of services where in amongst this mesh is my performance problem so this is where open tracing and Zipkin comes in that thing that two of you have heard of and that it's hopefully new to everybody else what Zipkin does is it does request tracking it follows a transaction actually you can do it from the browser right the way through all of your transactions right the way back to the browser and it does this by adding some HTTP headers into your request and when that goes to your web app you need to take those HTTP headers and propagate them into anything you make a downstream call for so it's all about propagating these headers and every part of the system then making a data record which it then sends to a Zipkin server and this gives you a visualization of what your performance looks like which hopefully is going to look like this so that's quite complicated I have to get some headers I have to read them I need to put them into Mayank bound requests and I need to send the data to a Zipkin server now luckily my team creates something called app metrics which is a monitoring system for node it's open source and we have just created an extension to add metrics called and metrics Zipkin and this does it all for you with just one line of code we enable Zipkin support in a node application to do that head of propagation and to send the data to the Zipkin server so if what that one line of code this is what Zipkin open tracing gives you so this gives you a view of where your elapsed time is and which of the services it's actually taking those time so from this I should hopefully be able to identify which service I believe is causing my performance problem so that gets me to the point that I have one service but I still need to understand what that service is doing enter Prometheus so Prometheus is a monitoring system that works on a-slash metrics endpoint so your application hosts an extra URL that contains a whole load of metrics data about what your application does now the way you do that again is we have an app metrics package called at metrics Prometheus and this will register that additional endpoint and collect a whole load of data for you it collects data about the machines CPU usage and your processes CPU usage it context information about memory usage event loop latency how long each HTTP call took on each URL how many status code four hundredths occurred right it generates all of this data and it exposes that slash metrics endpoint that looks like this now that in itself is not particularly usable it's human readable but it's not usable but what Prometheus does is it provides a data store and a visualization and that visualization uses things like graphite and graph on ax which is a dashboarding technology to build during dashboards and it allows you to build things like this so I can build a latency graph for each of my individual services that collects data from all 10 or 15 instances of that service and publishes that data back to the server and allows me to build custom charts including doing things like alerts so if I have a system that goes down or something that starts throwing 404s then I can have an alert to say that this is happening so that allows me to say okay maybe I've realized that for whatever reason is running at 100% CPU I scale - 15 instances it's still running at 100% CPU and I need to understand why my node application is using CPU and whether there's anything I can do to optimize it so the next step on that is another capability which is flame charts who has used flame charts for node performance analysis okay so about a third of you so again we've got another tool at metrics - this is an inbuilt dashboard that you can host inside your application and it shows you a whole bunch of data about the single instance of your application that you connect to again adding it is very simple it's one line of code require app mixture at metrics - and attach it and that exposes a web dashboard directly inside your application and that looks like this so this tells you about every HTTP incoming requests every ank bound HTTP requests it tells you about event loop latency CPU and memory usage and so on it also as you saw in Michael Dawson's presentation this morning at the keynote lets you generate a node report or a heap snapshot and there's a hamburger menu that lets you turn on put profiling and when you do that you get this you get in that dashboard a whole bunch of flame charts and that allows you to understand where the CPU time is being spent at any point in time so the higher the peak the deeper the stack trace the wider it is the more time is being spent inside that function and you can select any part of the flame chart and it will give you a stack trace that's showing you what's using all of that CPU now one of the problems is in just about 20 minutes I've talked about five technologies all of which are new to the majority of people here and there was a huge amount of configuration that you had to do now wouldn't it be nice if there was a way that all of this was done for you and you wouldn't have to think about it and there is this is what Michael Dawson announced it's our open source generator for node applications could yono server and chris is now going to talk about that and show you a demo I'm doing a little test drive a little test drive of yo note server and all the cool things that we can do with it can you all hear me out there out in the farthest reaches all right we do a little experiment would have set this down and ask if you can still hear me all right okay so you know node server we're gonna roll up my sleeves all right let's dig in all right let's put this away and let's start out here all right first of all it's a yeoman generator npm install - g generator node server brings it down don't forget to install yeoman we've got oh a fair amount of useful documentation out here that explains the goods but we're not gonna read through that I'm gonna show you instead all right so with node server installed we simply fire it up it's gonna ask you a couple of questions nothing too taxing project name optional open API document aka swagger what this is good for is if you are doing top-down style development and you've defined some of your api's in a swagger doc you can put that in his input I've got one here for my pet store he pet owners in the audience dog up dog owners cat owners goldfish all right okay so now we also this is an open source tool and it generates a scaffolded project with all that configuration that Chris was talking about good for any kubernetes platform we even throw in a configuration for Cloud Foundry but we also throw it a little bit to help those who might be using the IBM cloud there's some service enablement completely optional if you use it we pick various services and it will scaffold a little bit of code to make it easy to use those as well but we're not going to do that today we're going to just generate out basic project and what that looks like over here we see various things first talking about docker files and helm charts so here we see it a docker file that you saw previously over here is a helm chart and the pieces that make it up so there's our deployment yeah mol and our HP am on all the rest okay all pre-configured for you okay since I passed in a swagger doc we get just a little bit of scaffolding of the routes for my pet store okay that's just to get it started all right and and then from there what we can do is we can go out here and we can play it around a little bit okay so we've got some scripts in our let's take a look at those scripts what's in our list of things in our package JSON we've got some NPM scripts in here and there's a bunch of them right there so you know some of your usual things install start and so forth but we also include a tool called IDT that's the IBM cloud developer tool it's also free a free tool and it will bring down to your development machine some handy tool that that you might need for doing things with doctor and helm and kubernetes okay and so if you run the install for that it just quickly checks if you've got these tools installed it I happen to have them all installed I wasn't going to make you sit through that okay so it's just checking that a got them all in there so it's it's uh lets me know that yep and I'm all up-to-date and I've got the tools that that I don't have to have they're optional but they're really handy so we make it easy for you to pull them down now once we've got the tools pulled down we can do things like over here we can do an NPM run IDT run too many runs but that's how it goes so why do you tell let you do a number of things it's about doing your work in containers okay so if you're gonna deploy in container it's good to kind of take it for a test drive you know before you jump into your DevOps pipeline and start sticking stuff in your you know your staging your integration your QA and your production systems alright so let's run this a little bit locally of course I can do NPM install NPM start that's a given but if I want to do the docker stuff that's what I DT helps me to do okay and so here it's gonna spin up a container and it's gonna run my app there we go we're we're running okay we are running and if we check docker what does docker say doctor says yep here we're we're running okay so it did the build and it did the run for me knew all the right parameters to do these apps come scaffold it with a few endpoints that are handy okay like since we passed in a swagger doc we rolled in the Explorer for you so you can get to your tea goodies there okay so this is your your swagger browser okay and as well as your spec so that was our yeah Mille that we passed in for our pet store and also the apps metrics - that Chris showed you before let's see I've got no data here but we can fix that so let's just give us something to chew on there there we go we've got some live action Chris talked about enabling profiling it's really easy to do we just enabled it up there in the hamburger menu and if we flip over here to our profiling view what's interesting it's uh there we go all right so here's her flame graph I think it's showing what the Explorer was doing ok and let's give it a few things to think about all right and so like Chris said you know you can look at your call stacks and you can see where you spend your time and this sort of thing alright and that's all good ok and so so let's jump out of there let's see where'd my doctor containers Oh yep yeah he stopped - ok good all right so now we've tested this out a little bit in our docker container and it's time to use helm and go to kubernetes so I can go back to my IDE and it started using vs code a while ago I rather like it is anybody using vs code yes not bad right so um let's go to a run task so that that IDT tool that one that we we did the quick install of right it lets me do a number of things we did we saw install but I had nothing to install because it's all ready we did an IDT run that's how we got the app up in the container and we were poking around looking at it and and deploy is another one we can do so deploy will deploy it out using the helm chart to a coupe to my kubernetes environment my kubernetes environment is set up to be mini cube because I wasn't brave enough to do a demo out to any any cloud always here at the conference is the Wi-Fi is sketchy you know so what we see in the terminal window down here is what helm has to say about what it did took my helmet art and it ran it and it set us up on kubernetes something that it calls a deployment and the helm calls a release so we were to go out here then and bring up our type here we go bring up my kubernetes dashboard and sure enough there's my my node server deployment my pod service definition it's all there if I go to the pod and drill down there's my container I look at my logs sure it's the same stuff we saw before when we were running it directly on my machine not not in my mini coop VM all right so that's all there now I wanted to hit a few of those services can do so easily you just type a little better the angles weird up here let's see let's bring up the explorer for instance and sure enough now you'll notice just an oddity here the the URL the the IP address specifically is to my mini coop VM it's not it's not to my my Mac here alright and the port is a funny port number you see that it's called a node port it's just uh it's a kubernetes thing you can use load balancing with kubernetes or another way to access your app is through something called node port and it's a 30,000 port number ranging above mini mini coop does node port I don't really have a load balancer on it so that's why it's they're not at 3,000 even though you notice in here inside the kubernetes network ya there to see it's that it is at 3,000 on the internal network on my VM but but my external network is looks like this ok and if we dug through kubernetes we'd find that stuff but that command I was using in the for the mini cube environment makes it easy to find these things including like my apt metrics - sure there it is ok and so there we are out in kubernetes so we went quite quickly from from 0 to few few short questions to project to running locally with docker to deploying out to kubernetes and and all the monitoring tools that Chris talked about pre-wired into this whole experience it's open tooling it's free to use and in the generated projects are a good start it's the culmination of a couple of years of our experience using these tools so we've kind of applied some best practices as convention but that's just to bring the bar down a little lower so that the entry is easier for you you can change it around how you like but we think it's a good start and you can start by going to NPM and pulling down our generator and trying it out yourself if you have more questions about this you want to hear more see more stop at our booth and talk to us we'd love to chat with you thank you very much [Applause]

Channel: node.js

Views: 45,156

Rating: 4.9267116 out of 5

Keywords:

Id: Fbhhc4jtGW4

Length: 30min 41sec (1841 seconds)

Published: Tue Oct 31 2017

Reddit Comments

[u/koprulu_sector]

This was amazing! Thanks for sharing!

[u/BertnFTW]

IBM has some fantastic Node.js stuff!