Getting started with API Load Testing (Stress, Spike, Load, Soak)

hello everybody i'm nick in this video i'm going to show you how you can get started very quickly and easily with api load testing this approach can also be used for your websites but it might be a bit more elaborate while api load testing is a bit more straightforward so in this video i'm going to focus only on api load testing however if you want me to make one for websites as well please leave a comment down below and i'll do that in a future video this video will cover all four major approaches in load testing and that is spike stress load and soak and after this video you should be able to actually take exactly the code you see and use it for your own apis maybe with a bit of tweaking to integrate load testing in your development pipeline and maybe also your ci pipeline if you like a type of content and you want to see more make sure you subscribe in this notification bell to get outlets when i upload a new video so what do we have here well first i have an api which is running locally on my pc it has three endpoints one uh youtube one twitter and one github and all it really does it's it returns the subscribers and the followers and the github followers and that's it and this will demonstrate the api that we would normally load this so any load tests i write will actually target this now like i said i'm going to use a tool called k6 and it is actually a very popular very widely used tool which also offers a managed version in this video we're only going to focus on the self-hosted one and you can easily install it on linux windows or mac in my scenario i'm using the windows subsystem for linux to execute that so if you want to install it you have to follow these instructions it's very very easy once that's installed all you need to do is create your first test and it's just a jes file and it's actually a very simple js file so let's just create our first sample test to see how easy it is so i'm going to say sample test js that's it and then i'm going to import the k6htp module so http from k6.htp and then i need to export two things first export led options and also export default function here and really that's it you're ready to get started this function is what is basically going to be executed during the test and then this options object will hold all the options for the tests themselves i need to add two of them just because i'm running this locally and because i don't really have a certificate and i want to be ignored so those would be insecure skip tls verify internal connection reuse that's for the sockets you don't need to worry about those and then i need two more just for the very basic test i need vus which represent virtual users let's say one here one user calling our api and then i'm gonna say duration and the duration as the name implies is how long you want this to actually run for let's do it for 10 seconds so we can fit it in the video and now this function all it's really going to do is it's going to actually call the endpoint we want to test and here you can do anything you want authentication you want to stick a user agent add the headers you can write anything you want in that function know that this is what will be executed from every virtual user every time so the simplest thing you can do is say http.get and do this and now this will call the youtube endpoint as fast as possible for 10 seconds so that's it that's the simplest load test you can run by using k6 then to execute the tests all i need to do is be on that directory and say k6 assuming you installed the tool and then run and point to the js file in our scenario sample test.js and i'm going to run it and now for the next 10 seconds you're going to start to hammer that api endpoint as fast as possible and you can see the nice ui finishing up now and then it's going to give you some metrics back so data received data sent uh request block request connecting the duration which is what you want this is the full duration of the actual uh request you can see that there's 191 microseconds here and you have how many iterations happen in 10 seconds and then you have the average duration so this is awesome just to get a rough idea but in my opinion you might want to have a bit more control over your tests so what you might want to do is you might want to import slip from k6 and sleep will allow your code to wait basically after it's calling of the method for each a virtual user for a given amount of time so in our scenario we want to sleep for one second which means that each user will call the api once a second so if you want to see how many requests per second you can handle that's what you would do so i'm gonna go back here and i'm gonna call that test again for 10 seconds and now because we only have one virtual user and we are sleeping for one second what i'm expecting is to see basically 10 requests um so as you can see http requests 10 and you can see the um duration a bit higher than before one user everything is here for you to collect now this is the simplest way you can use k6 for load testing however this doesn't really represent any real life scenario low testing you will do at your job what normally happens is you have four major types of load testing you have stress testing load testing for performance spike testing and soak testing and i'm gonna explain what each one of them are and how you can use k6 to execute those first let's start with stress test so stress testing is a type of flow test used to determine the limits of the system so how far can we push it the purpose of it is to verify stability and reliability when the system is under extreme conditions and by the way if you grab the code each test has a description so i'm going to be reading them as i go you want to run a stress test to determine how your system will behave under extreme conditions determining what's the maximum capacity of your system in terms of users or throughput determining the breaking point so after what amount of uses or requests per second the system will fail and then determine whether the system will actually recover without you manually going and restarting the server or doing something like this and it goes something like this so usually what you have in these types of load testing are stages and actually k6 does natively support stages what does that mean well it means that for example if you start from zero users and you have a target of a hundred users for two minutes then k6 will slowly ramp up in that course of two minutes it will interpolate the target users meaning that in one minute you will have 50 users in one and a half you're gonna have 75 and by two minutes you should be at the handling user so it's ramping up the uses that will hammer the system with the test and then for five minutes it will stay stable on that target to see okay can we handle this in a stable condition and then for two more minutes assuming nothing failed it will go to 200 and then on 200 it's gonna stay there for five minutes and go on all the way to 400 in this scenario which is probably around or beyond the breaking point where you're gonna see exactly how your application will perform and then after that it's going to slowly for 10 minutes ramp down where you're actually going to test the recovery stage to see if any recovery mechanism any circuit breaking and retry policy kicks in here and saves your system and in this example as well i'm sleeping for one second and i'm batch calling all the endpoints i'm not going to execute any of those tests the execution would be the same all you need to do is do k6 run and then target the name which is stress test.js and as you can see um you can see the views here slowly ramping up you can see the requests increasing as the vu's are ramping up and this is a 38 minute test obviously the stages depend on you but usually in tests like this you want to have um a slow and steady approach with them now another test you might want to run is a spike test and a spike test is a variation of the stress as we just saw but instead of gradually increasing you just very fastly spiking to a very high number this might happen for example when you have a blog and then it blows up on reddit or a very big influencer retweets it and then many many people go to your website and you have a spike this is what you might wanna stress test for and usually you have expectations on how the system will perform and you have four ways to determine how well your system did excellent where performance did not degrade good where the response time was slower but it did not produce any errors for when the system actually does produce errors but then recovers normally and but where the system actually completely crashes and the stages for this test are actually very straightforward you have a short climb duration to a relatively low normal load number and then you stay there for a bit so basically you're warming up the service and then in a very short period of time in the scenario 10 seconds you climb to an insane number like 1400 users and you stay there for some time maybe three minutes not a long time and then you climb down to normal load again and then to zero and this if it was a graph would look something like this where you have this small climb and then this huge spike and then this small climb again and this will tell you how the system will perform during spike now a more traditional test that you might want to run is normal load testing for performance this type of load testing is primarily concerned with assessing the current performance of your system in terms of how far you can push it effectively how many users you can handle how many requests per second you can handle it is probably the test you want to run first to get benchmarks and determine how far you can push your system in other tests for example stress and spy it is also a very good test to actually have as part of your ci pipeline continuous integration because as you're developing a system it might become slower or faster depending on your code and what it needs and it might be good to have a benchmark to compare previous commit with new commit and say oh the load tests on this commit responded in two milliseconds previous is one millisecond so the system is twice a slow why and then you can maybe fail a build as well and this type of load testing is actually incredibly easy you have three stages the initial ramp up which is slow let's say five minutes to uh some normal amount of users usually what you're expecting and then you stay there for some time maybe 10 minutes and then you ramp down again and if this is just runnable in 20 minutes it could be part of your ci or cd process for production this type of testing as well should usually use thresholds and thresholds as you can see here and is effectively some checks you can have during the test to determine whether it fits some criteria in this scenario we want to make sure that 99 of the requests respond in less than 150 milliseconds you can have this in other tests as well however it is way more important in my opinion in low testing which should be more regular type of testing that should by no means have slow responses or errors than the other tests that are meant to kind of break the system last but not least i have soaked test and it differs by other tests by being boring because they take a huge amount of time to run so soap testing is usually there to validate the reliability of your system over a long period of time and they can be great tools as well if you want to find a memory leak that you can't detect um during a short load test or if you want to see some unexpected restarts without losing requests uh database connection exhaustion allocated space to make sure you don't have any logs that just bloat your storage and that type of thing and usually the way you run it is you determine using the stress test how far you can push the system and then you get around a 75 percent or 80 offset from the original um amount so if it was a thousand you want to hit it with 800 and then you set the virtual users to that value and you run in three stages a small ramp up to the users and then you stay there for a long period of time let's say four hours you might wanna go further five six eight twelve however long you need to validate what you're trying to validate and then you slowly run down again to zero that's optional of course um the important thing is how long you stay in that amount of time it's the test i hate running just because i like to see results as i go and this usually will only give you results during the end hours of minutes of the test just because of its nature now as you might have noticed the code for this is incredibly minimal and this is why i'm showing off this tool because i've used other tools as well like jmeter which are awesome by the way and if you want me to make a video on those i'm more than happy to do so however i have found that k6 is the easiest for any level of expertise and the position and whether you're a software engineer qa engineer anything at any level in any position it's so easy to get started so i just want to show you that this exists and you can just integrate with it right now and it's totally free there is a managed version and this video is not sponsored by k6 by the way but i don't think that you need it of course if you need to have someone to full time manage this then maybe paying that salary and paying the managed price might not worth it so that's completely up to you but you don't need to pay for any of this this is all free and open source well that's all i have for you for this video thank you very much for watching especially next my patreon for making videos possible if you want to support me as well you're going to find the link description down below leave a like if you like this video subscribe for more like this and ring the bell as well and i'll see you the next video keep coding

Channel: Nick Chapsas

Views: 70,673

Keywords: Elfocrash, elfo, coding, how to code, tutorial, development, software engineering, microsoft, microsoft mvp, nick chapsas, chapsas, Getting started with API Load Testing, Getting started with API Performance Testing, spike testing, load testing, jmeter, k6, testing with k6, testing with jmeter, soak testing, performance testing, stress testing, dotnet, .net

Id: r-Jte8Y8zag

Length: 14min 20sec (860 seconds)

Published: Tue Jul 06 2021