Can You Use Postman for Load Testing?

all you need to do in postman to do performance testing is to export your collection write a very simple script that will take this collection and run it in parallel say for example for 1000 times run the script oh what's this hi there and thanks for joining me for this tutorial today i wanted to talk about performance testing or load testing in postman what we are trying to do with such tests is to see how the application behaves when many clients use the api at the same time now i won't get into the various types of performance testing as they all have something in common the ability to send a significant number of requests during a sustained period now if you don't have a lot of time let me give you the gist of this video right now while postmate is a great tool for functional testing it is not really the best tool for bombing an api with tons of requests while it would be great to have a single tool for all our api needs testing the performance of an api is really not an easy task alone the fact that there are tools dedicated to this type of testing should give you an idea of how complex this topic really is so using a dedicated tool is the only way to go if you're testing an enterprise level api so if you're serious about performance testing you may want to look into tools like meter catling low cast or k6 which i'll briefly demonstrate in this tutorial so let me show you what i mean [Music] okay so let's jump into postman i have developed an api and i have deployed it to aws and i've used postman throughout the entire process for testing it on my local machine to see if it works and also against the aws infrastructure where this api is deployed as you can see i have here various endpoint and for each individual endpoint i have created tests i know that the functionality of this api is sound so it works exactly as it's supposed to work now i'm at the stage where i'm wondering how much traffic can this api or how much traffic can my application handle i don't know anything about this at this point i know that i don't have a great design and one of the reasons is i'm using a file based database so every time i'm trying to write something that will cause problems on the server and also the server is the cheapest virtual machine that you can get on amazon it doesn't have a lot of ram doesn't have a very powerful cpu so it's going to be a great testing scenario to break this api let's begin like with the basics so i know that writing to this api is a problem so for that reason i have isolated here only one endpoint and this endpoint creates an order with some random data and as you can see locally when i'm trying to to run this it's providing a pretty decent response time could be better i have here postman tests that ensure that the request has been successful that i'm getting back instead to store one created so everything is working fine now the problem is how do i know how many users my application can handle or at least how many orders can i take at the same time for that we can run for example the collection using the collection runner and let's say what happens here if we run 1000 iterations this will take a bit to run right now we have run this collection which essentially includes a single request 1000 times we can take a look at the results we'll see here past 1000 times so what does this mean well it only means that our api was able to handle 1000 requests but these requests weren't sent in parallel essentially we have sent one request we have waited for for the response and then sent the next request and so on for that reason this is not any kind of performance or load test because it puts no load on the system the load itself is very very small it just sends one request after the other but we don't really know a lot about this api and how it's working so for that reason simply using the collection runner to simulate load is not the way to go so what i'm going to do is i'm going to go ahead and export this collection and we're going to use newman to try to run this request in parallel see how that works all right so i have here a very small project i have exported a collection here and i also have this script which is also available in the github repository from postman it is a simple script that allows you to run a collection in parallel for example let's stay here give a size of one so i'm going to try to run this once i'm going to open here node parallel collection runs dot js essentially we have now tested our script is working and we have executed the collection only once any tests that were inside were also executed so you know if you run it once it's working so that's a good start okay so let's now try to run it 100 times so i'm changing here the number to 100 and what's happening right now is essentially we're starting 100 instances of newman essentially where newman is running this collection is sending all these requests out and then at the end it's collecting all the results and you will see here we get back a list of all the collection runs and all were successful we don't see any errors here so far so good now we know that this api can handle like 100 requests at the same time or which were sent almost at the same time but we're not really sure about the limits and as i was saying in the beginning i know that this is a very crappy api you know on a very crappy infrastructure so i really want to get at a level where i can see what is the limit of this api let's try to increase that number so instead of 100 what about 1000 i'm going to run the command again this time i'm going to also open the performance monitor on my computer and as you can see the cpu for node is already pretty high and also the memory for the note process is continuously increasing so memory and cpu still pretty pretty high and going up okay and finally node will crash because it will reach the allocated memory and essentially we will not get any information about our runs because the entire script failed node essentially failed on the computer and of course we could go ahead and increase node increase the memory allowance for node but you can kind of see that we're getting at a limit in terms of using node using newman in order to run this let's say performance or load test against the api so so far this hasn't been very successful and even if it worked the way this test was structured is not something that really replicates a realistic scenario because we want to be able to send requests over a sustained amount of time not just send 1000 requests at the same time maybe you want to start with 100 requests and then increase that over time so in a way to simulate it like real users would use the api and on a production server for all these reasons it seems that using newman in order to load test our application is not really the best way to do it essentially postman and newman they have been designed for functional and integration testing they haven't really been designed for load testing so i wanted to introduce you to a proper load testing tool and this tool is called case six it is an open source tool it's pretty easy to install it doesn't have a graphical interface like postman does but if you're familiar with javascript and you're not afraid of coding and you will see that it's quite easy to get started with this tool and i just want to do a quick demo so that you can see like how a proper load testing tool works and what kind of an information we're getting back now if you want to install this that should be pretty easy i'm going to go here to the documentation and you will see here in the installation part you can essentially install it on any operating system including windows or mac os for mac os i've used brew to install it for windows there's also an official package or you can use this package manager i will find a link to this in the video description and also if you're using docker there's also docker image available for that so in order to test if this is properly installed i'm going to simply run k6 version and you will get back the version how do we run k6 against this essentially i only have one endpoint but how do we run it against this endpoint this is the javascript and this defines essentially the test our testing scenario and in this case it's very very simple but still kind of realistic so it's not a simple get request that i'm showing you here this is actually an http post request that goes to this order's endpoint i'm providing some headers also an authorization token here in the address i'm providing here a body also added some randomness and most importantly what k6 allows us to do is to specify virtual users so in this case let's go ahead with one virtual user and we're gonna run this test for a duration of one second and you'll see here that every time this endpoint is called there will also be a sleep so essentially this is in seconds so when one virtual user runs this end point it waits for one second and then it rises again additionally what we have specified here are also some thresholds which typically are very important for your apis so if you're really serious about deploying this api to a lot of people most of the time you will need to ensure some kind of requirements in terms of response time for example so we have one virtual user and we're running this test for one second so let's see how it goes so i'm gonna run here k6 run script and now k6 is starting and it will also give us here an overview of what's going on we only had one virtual user and in total there was only one request that has been executed there was essentially not enough time to run multiple requests in this time frame we are now sure that this is working properly and we'll see here that everything is working let's say within expected parameters and there are a lot of information there's a lot of information that's coming back here for example here we have established that the maximum duration that we expect to be below 1.5 seconds so we're getting here like the average value the minimal value the median value the maximum value that someone encountered but also like a percentile of the people that tried it out this will make more sense once we increase that number so let's go back to the script and let's say we want to have 100 virtual users and they will be calling the api over a period of 30 seconds run it again and the nice thing about k6 is that it will show you like in real time what is happening you know it gives you a progress bar you'll have an idea when this will complete so it's not just executing a script and then watching your cpu and memory go up and not knowing if this will crash or not so now we already have more data that we can compare and we can see here that this expectation that we have set here in regards to the duration of the request now the data has changed so the average for this api is now at three seconds and something but fortunately we're looking here how many requests have failed so zero requests failed so the api with 100 users was still able to take these orders it took a bit longer but it's still working so that's kind of like an important thing that we should know about but i said my objective for this one is to figure out like how many users are required to make this fail so let's increase that even more so i'm gonna go here with 1000 virtual users i'm going to keep the duration of 30 seconds i'm gonna run it again and now you'll see here we're getting some uh some warnings because some requests are failing essentially when this happens it simply means that the machine itself is overwhelmed it's not taking any requests or there no essentially the tool is not getting any responses back so essentially we're reaching now a level we're really at the limits of this so i'm curious to see how many of these requests were still successful but obviously this api like from the number of errors that i'm seeing here this api cannot handle so many virtual users all right so now we have the report and we can now notice that we're just looking at how many requests failed 85 of the requests have failed so only a fraction of the request we have sent out were successful in this case even looking at the request duration is not really that relevant because it hasn't really worked properly and you have seen essentially my system has not been under stress i was able to get this report so now i know okay this api can handle 100 virtual users given this execution time frame which of course you can make longer but i know it cannot handle 1000 virtual users so i kind of like have an idea of the system and i can also go a bit deeper look at the server see which logs have been generated see what the problem is try to find out where the bottleneck is but from this performance load testing point of view it has been a very very useful tool for me it has allowed me with minimal amount of code to go deeper and to understand what my api is doing and how my api will behave if i'm sending lots of traffic all of a sudden to it i hope this tutorial was helpful and that has given you an understanding of what to expect from a load testing tool and why post many numeric are not necessarily the best choices out there if you need to do something like this and if you're really serious about apis if you want to learn more about k6 let me know in the comment section below if i'm getting a lot of positive responses from you i will continue creating tutorials around case six and i will show you how to use k6 to do performance tests on your apis thank you for watching and see you next time

Channel: Valentin Despa

Views: 2,368

Keywords: postman, k6, postman load test, postman load testing tutorial, load testing using postman, k6 postman, postman performance testing, postman stress test api, postman stress test, postman stress, k6 introduction, k6 load testing, k6 load testing example

Id: XUZHVYSdYdg

Length: 15min 52sec (952 seconds)

Published: Thu Oct 21 2021