How CPU Efficient is your App?

when building back an application one of the most critical things to identify is what kind of workload is your backend application use and and basically there are two types is it is your back end mainly i o bound that means is it mostly relying on uh you know IO that whether that is to desk or network or is it more you know CPU heavy it needs the precious CPU time for effective execution you know and and this comes down to scaling because how do you actually scale your backend application and when I say say a skill I mean like vertical scaling like how much juice if you will resources do I need for one box so that my application runs efficiently and it doesn't die and I spend as as little amount of money as possible so that you can kick in the latter part of things where you can scale you know horizontally so in this video I try to explore the two types of workload i o bound I'm going to show you an example uh using postgres because it is an i o bound app and I'm going to show you another app that I wrote in C that is essentially CPU Bond it it sucks all the CPU out of your machine and you can use that with this beautiful top command which is available in every Linux machine Let's jump into it alright so now I'm looking at the top command which you can literally execute by by running the command top I know there is like another command called HTA which is like a little bit prettier but top will give you all your processes all the CPUs how much memory is each using but I'm really interested in this line essentially that's pretty much it I'm going to explain that and even in that line I'm really interested in this guy and this guy and this guy right the rest maybe are more relevant for you know system you know Engineers or Hardware Engineers is more it's more appropriate for this parameter all right so let's get started in explaining the different type of CPU metrics that we have here right so the first metric is this which stands for us or user processes this is a percentage of all the cores right in your machine of how much uh CPU usage user processes have used so this is also a percentage that's a 0.1 percent and I've seen this different depends on the app you're using so top will show this as a percentage from zero to a hundred zero means you're not using anything 100 means you're using 100 of all cores right so if you see this as 100 them and you have like four core machine like in my Raspberry Pi that means uh you're using hundred percent on the first score 100 on the second and 100 a third and so on okay so that's critical sometimes like h-top I think if I'm not mistaken we'll show you that as 400 which is to be honest it's like nicer right not a nicer view like so you see 400 percent means oh you're losing 400 percent of you you have four cores and all of them are essentially occupied right so that's the first one user the second portion here is the system so this is how much time all the cores again the CPU has spent on system operations right so that's the when you Swip swipe from user mode to Kernel mode and you start executing system calls the kernel needs it's the CPU as well but that's a different metric we count that such that we don't pollute your percentage would tell you that hey the kernel U is this much so if you see this uh using hired amount I don't know what to do you need a kernel developer to start debugging that stuff right so it means there is somewhere inefficiency in the kernel that we're using or maybe you're doing a lot of system calls perhaps so yeah so that's that's what it is essentially uh ni or nice um this one to be honest I don't have a lot of it you know examples for it but it seems like you can tag a processor with a specific priorities and these are cool for some reason nice right values and anything that was above zero is an will be counted here as its own percentages so maybe you have like a specific process with certain Affinity right and you can assign certain affinity for processes and this will increment that counter so that it gives you like a nice specific custom percentage of CPU usage for them ID which stands for idle this is how much time we stayed idle essentially so at this moment we're looking at top we are 99.7 percent idle on my Raspberry Pi I'm not using any almost no CPU right so that's it and this is the one of the important one that we need to talk about is which is weight this is all right the percentage of CPU time that is spent waiting the CPU is actually waiting for an i o specifically from disk this does not count network i o as far as as I'm when I say Network I don't mean I mean socket TCP or UDP you know socket it does not count as that if you send if you send a request in HTTP and your CPU is just waiting that doesn't increment that at all right this only get incremented when you're like the CPU is making a file read to disk or if that disk is mounted to like a network mounted device that network is just you know uh that causes just eaten as if you're doing a file disc i o right so what does that mean this is essentially a way to distinguish idleness because technically the CPU Wireless is waiting doesn't mean it's blocked the kernel can technically switch another process to to uh to do other tasks on it right so it is it is technically idle but we want to identify that the idleness such that this particular CPU has issued an i o and it's waiting for and a result before it can continue other things so that time until we get the i o back that counter essentially stops right so I think if I'm not mistaken the counter always gets incremented like if you see a CPU idle which is this case like the kernel of this season a CPU is Idle or a horiz idle and it just has issued an i o then it starts incrementing this time or this counter essentially okay the higher the value here the that means your desk IO is it's slow right essentially and I'm gonna demonstrate this with postgres because this Raspberry Pi has an emmc you know Drive card I suppose it's not really a drive and and that is really slow and I'm gonna issue a lot of rides and you can see that the CPU is is is just waiting most of the time spent waiting for Io bound workloads right so so essentially that's what it's doing if you if you see that in that area then if you see that value high that means your disk reads are slow essentially or maybe you're issuing so much so many reads right maybe you can do like a little bit more efficient in your i o right to desk and that becomes like are you building a database or building a like something in that in that regard right all right so hi High Hardware interrupts the CPU has spent some time servicing Hardware interrupts like I suppose a keyboard a mouse is a hardware interrupt right there's a software interrupt as well I suppose this is the software the service interrupt routines right and there's the St which is the steel and this is essentially if you're in a virtual environment you have multiple operating systems right sharing that course that you have and what will happen in this case like the core the CPUs will be stolen from One processor from process to another right and when it's being stolen for to execute other things this counter gets increased right but we're really interested in is really the weight and idle and the user okay so you can easily know that if you're a CPU Bond or i o bound right in this particular case I O bond to disk right how about we jump into it and do some examples all right with that all the way now that we know this how about we try our first workload which is uh doing a CPU bound workload right so I'm going to go here and I have a basically a c program here very simple that just literally loops bazillion times and then just create a sound so if I run this program it's going to stay running but that is absolutely almost using the CPU all the time yeah there is like uh traps to the memory of course to to set the variable the local variable that is the sum but you can see that now there's a program called a DOT out and it is using some part of the CPU right it's using one hundred percent of one core of the CPU because it's a single threaded uh process right so we're using 25 makes sense because uh this is a four core Raspberry Pi and so you can see that now we are jumping to 50 something happened right other other processes got kicked postgres is interfering but we're we're averaging 25 which is nice now what I'm gonna do is I'm gonna kill this process and before we do that see the idle time is around 75 right so it's like almost like a full sum game here so I'm gonna kill that process and uh you can see we're back to normal right so we're no longer CPU bound all right so we're gonna do is I'm gonna start running the processes essentially a multi-process of the same job for the same process I'm gonna spin up multiple processes on the background so we Sprint on process number 38 39 and 4 should essentially max out my CPU anyway as we can see now my CPU is fully maxed out 99.8 user space you know the user processes and there's no idle time and you can see that all of these guys are fully fully using the CPU now guys what I want to talk about here is for a brief second is while true we have we have we're seeing hundred percent of CPU it doesn't tell us that other processes has been waiting for CPU time it doesn't tell us that metric unfortunately that metric is missing right while you can technically you can have a hundred percent all maybe 98 let's say utilization of your CPU and you'll be absolutely fine no processes are starving for CPU time for but so just seeing 100 or 98 doesn't mean I am lacking CPU and I need to add more CPUs that that's not always the case for that you actually need another parameter that is not available here as I was saying 100 percent CPU or let's say 98 doesn't always mean other processors are starving right to actually see their starving or stalling processes you need something called performing stole information or PSI and Linux is an extension or or an add-in is it called an add-in something you enable in Linux you enable it in Linux and that will show you hey these processors are are stalling you do not want stalling processes because that's when you're doing context switches left and right you are starving other processes and you don't want to do that and that's actually how you can you know figure out how many threads right how how much can you push your back end how many threads can I you know kind of spin up before my backend is fully saturated like almost fully saturated because again you want to stay efficient and by efficient means I want to use everything that I have to for maximel uh you know Effectiveness right back to the screen all right so now let's kill this f all right all right we killed all the four processes nice we're back to normal ish right good stuff now let's do an i o process you guys right I'm gonna do an i o process all right so now I have logged into my postgres instance on the Spy I'm gonna do the most i o bound that it is I'm gonna start and insert a workload that inserts like I don't know a billion rows right because because I'm gonna talk a lot so I don't wanna I wanna I don't want it to finish because I want to see what's happening to the CPU meanwhile so let's do a insert in two tests there's a table I have called test uh from no select store from generate series yeah whatever it's gonna take a while but while it's taking a while let's take a look at what's happening here now guys see there's some CPUs being taken 20 comes back up and down but look at this guy this is what we want if you ever see this guy stuck at 50 40 then whatever workload you're running is probably i o bound right that means most of the operation is being uh the CPU is barely doing any work right into disk or and just waiting for the result and and this guy essentially this is a uh the the pause using an emmc as type of a storage and that is not really um that's how do I say it's not really fast per se right so you can see this goes up to 24 and it depends on the operations that I'm doing I've seen this go up to 80 to be honest right and it it really depends like what are we doing on the background process so like when postgres writes all this stuff it actually writes it right so if I even if I cancel this operation right well postgres will try to cancel it but that the data that you wrote is not gonna go away right because it's still there but then it's subsequent processes will start to now kick in to clean up right the auto vacuum will start to kick in there's another process called check pointer will start to kick in so the more we wait the more this goes up essentially because you're going to do more and more iOS so let's do that one more time right and you can see that watch watch that go up again like you can see post because she does use the CPU right efficiently in certain operations but most of the time and we can't really tell here what is what is what post what process is actually that but there is tricks we can play I suppose we can do that in another window right while this runs how about we do this grip four screws good anything that's possible there you go that's what I want to do so now that we have more details about what what is these things now we can see this there's the check pointer there's the background writer and I talked about this in another video guys where I talk about all the processes and postgres and what are what does each one do so we have uh you know we're up to 50 nice nice so we're way we have a weight events and CPU 50 of the CPU that is the entire CPU so two cores fully just waiting on i o yeah I was still running this operation there is a postgres I want to take this guy right two six was that was that it two six one four two right and we'll tell us two six one four two is actually the postgres and it tells oh my God it tells you even what is it doing I think this is the background uh there's the back end process itself that's the process that you that posca spins up for each connection and it's actually that's what it's doing right so this is not a system process that's doing that's the back end user process right but yeah that's what I wanted to explain actually essentially guys right so we talked about the CPU bound every time you see this as a you know closing to the high percentage that means your your application is essentially uh CPU bound if you see this guy goes high that means your applications mostly doing IO and rarely using the CPU are actually waiting for CPU time right so I hope you enjoyed this video and see you in the next one goodbye

Channel: Hussein Nasser

Views: 13,904

Keywords: hussein nasser, backend engineering, linux top, cpu backend

Id: BTD5I1BMx2Q

Length: 19min 50sec (1190 seconds)

Published: Sat Aug 12 2023