Automate your Testing on Devices with Flutter Integration Testing

hello and welcome back to another episode of flutter explained today we want to talk about integration testing which is on the tip of our testing pyramid right underneath of manual testing as you already know the testing pyramid shows us the amount of tests that we want to have compared to its time that it takes to run also our how much expensive it is so integration testing is relatively expensive but it should be sprinkled into our testing mix and today we want to have a quick look how everything works and now without further ado let's get started alright so what are the benefits of integration testing and why should we do it in the first place well integration testing shows the most amount of automation and also gives us the most powerful insights in how the application works and if it is actually correct how it works because what it does is it mimics user behavior it clicks through your app it scrolls inside of it and it tries to recreate the user interaction for a specific use case so one idea is that you for example have a login screen and the user interaction test or the integration test would be that your integration test runs through the login screen selects every text input field enters some values tries out if the correct ones work but also if the wrong ones work so if you have an error case that you show a validation message this should be triggered by an integration test next we can now click on login for example and switch to the next screen so all of that can be tested with an integration test and this is what it should do so you take one complete use case from your application like login for example or if you have a payment system you could also test that and then you can run your application with an integration test you can run it on multiple devices also on aws device forms or on firebase which also allows to run it on multiple devices so you can make sure that your app is running on different devices exactly the same for the same use case so if you have several integration tests you are very sure that your app behaves the right way so this is one very big and high benefit of it that you can test it on real devices on emulators alike you can have a lot of confidence that your app is now working correctly and it shows the right things and last but not least that you don't have to do it manually right it just works out of the box so i know some testing teams who really go through their apps and scroll and click the different parts with integration testing you have all that covered but of course there are also some downsides for example flutter don't allow us out of the box to create multiple test files for it so you will have to use a little bit of a trick i write for myself a script that you will see also later in the video when we develop a little bit and additionally what is also a downside is that it needs to run and start up the application always because you test it really on a real device it takes time to start up everything and to bootstrap everything additionally it leads to a little bit more boilerplate because you have to create new test files you have to integrate the flutter integration dependency and so on and so forth and this could lead to a little bit more code so this are the downsides and of course it takes time and it runs longer one thing that you should consider on integration tests you can now also make smoke tests so you can test the surroundings of your app so if your app is dependent on an api you can make with that integration test a call to this api and see if you get the response one downside there it could be that the debug message that you get so the error message is not completely concise and clear so to find out some problems it could take time and also if you integrate your integration tests in your build pipeline which you should then it could be that your overnight tests for example break and with that you are not able to deploy anymore or something like that so make sure that you have that sprinkled on top of it so integration tests is nothing that should be everywhere and should test everything for that you have unit tests you have widget tests which are way better in fast running tests besides of the integration tests all right so and now that we know all that let's try to bring that into our code we will now um integrate the integration tests into a test project that i created for that we will create some boilerplate with new folders and some new files we will integrate the integration testing package and last but not least i will show you a trick how you can wrap with a script multiple testing files and start up your application multiple times all right but now let's get started by the way this is the middle of the video so this is the time where you can like the video subscribe to this channel if you haven't yet now let's get further thank you right so we want to start by showing you the integration test and what the app actually does and what the integration test will test so i started in the background the integration test so you see the app will shut down completely and you see that xcode is running and the build process have begin or begun and now after some seconds it will start the application in the device and run our app i added some seconds to wait between the steps or else it would be way too fast to actually see what happens here so you can see the first thing that we do is entering the email we wait a second entering the password go to the login and then we select one of the check boxes and this is what we recreate so i didn't do anything as you can see my hands are always here so that means the computer the integration tester was going through our app and click every element and enters text where we need it so the first thing that you would probably recognize if you go to the integration testing branch of the project or flutter testing tutorial which you find down in the video description below is that we have two new folders the first one is test driver and the other one integration test and this is very important that you created these two files because they are essential for that what we are doing inside of this branch i already created you the app.test and also the integration test driver if you are working with intellij i highly recommend you also to tag these directories as source root and with that the folder is getting green and you have a bit more overview and intellij knows how to treat them okay but now go to step one add the integration test dependency to our project as you see i already added a link here and you know that we go to pub.dev where we go to installing and we can take this dependency here into our application all right with that we have added that and finished already our first to do our second to do is also pretty interesting that means that we have to create a main function here in this integration test driver file and you can also just call it integration test and inside of here we want to have a void main function which directly executed the function integration driver and as you see now it's all red so we have to import the new package that we got and i have to get the package yet and now inside of here you can see that we can import the integration test driver alright so much we have to do in the test driver documentation now we can switch over to the integration test and if we check here you can see we have a lot of different to do's but this is now actually the test now you're already set up to directly work with your integration testing we want to create one test case that just goes one by one through our app enters a valid email address enters a password and clicks on the login button and after that we have a several check boxes and we want to select one of them and check if this one is selected so how will it look here so here we would enter a valid email address we would give a password and we press login and after that we have this list and we would select amazing and after that we want to check if this checkbox here is selected good but how do we do that so we are now at to do number three and the first thing that we have to do is we have to import the integration uh test widgets flutter binding and i should maybe right underneath of the to-do which makes it easier and then we have to ensure that it is initialized that just means that is our build our flutter driver which then executes one command after another alright now we have to create our first test case and you can see already that we did that here as you see this is a test widgets like in widget testing and it is very important because out of two reasons first we have to have access to the tester and secondly the integration test package requires these test widgets in order to avoid mistakes so also this is done the next part is that we have to import the app itself so as you can see the lib folder contains our main.dart so this app here we want to start so everything of it so how can we do that we have to import it first so if we search for lip or better import main.dart you can see we have that and we say as app and this is now the import where we can now access the main function and with that we have already done to do number five and six so remove that remove that now we can wait till or now we have to wait till the app has settled what does settled means that means that the frame from before does not change anything so no animation is running no entering of some data or anything like that so that is very important here so we use now the widget tester and say that we want to pump and settle and with that we are sure that the app state keeps the same and we don't update any ui elements anymore so if we take a look at our login screen we can identify two text form fields that we have to find so you can see we create finders for them this works exactly the same like in widget testing so i reference it to there but we will discuss two options that you can use number one you can search for the widget itself or the type itself in our case that will be the text form fields so what we will do is we create two variables so the first one is email form field where we use the function find by say the type and that is text form field we don't have to execute that and now we have a first and we have a last this is maybe not the best solution because that could change over the time right so email don't has to be always the first one what happens if the form wheel field elements change in any way and then you would get here some problems but for our solution at the moment it is perfectly fine and the last thing that we want to search is the login button we can find by type which is a raised button at the moment and we will find that one and we can also just take the first one just to make sure so this is by type we have also by widget right so we would create a widget that would be exactly the same and the other thing is that we can use this by key so if we want to find a specific element on our screen we can use by key and what does that mean if we go for example to this text form field of the email we could pass in a key inside and say we want to create a new key with the text email now we could search for this email key and be specific here because this key needs to be unique for the application so that means we could also instead of doing this we could say we want to take that by key and give inside email and then we don't have to search for the first now like this so we search for a specific key in this case email and we find one and of course the variable is now already used but this is exactly the same thing we would find the right text form field so just keep that in mind you can use any matcher that you want any finder that you want and it will work correctly so we can remove that in order to enter a text into our email address form field what we have to do is using the tester and call enter text now we use the finder of our email form field and the text that we want to enter in our case flutterexplained gmail.com and now this text has been entered but make sure that you always if you change something in the ui you pump and settle if you want to test something you don't have to do it always so if you have multiple texts like we do we can also change first all the texts that we want to change so instead of this we could also use the password form field give here the text one two three four five six or any password that you have uh in your mind we can remove these to-do's and now what we can do is after we have entered that we just wait a second and now we use the tester.tab and what we want to do is want to tap on the login button all right so and now i will pump and settle once more and what i want to do before we go on is i want to execute that once and at the beginning of the video you probably saw already this fantastic [Music] terminal command that we have here so what i will do is i will copy that over into a script file so that you can easily read that so here inside you can see that we execute flutter so we use the flutter framework drive this is the command and we have to pass in the driver and we have to pass in the target the target is actually your test file that you want to test and the driver is the integration test driver so that was just the line of code that we created with this um this guy here with the integration driver all right so what happens now if we execute this command line let's do that and let's wait a second for our app to start oh we get a problem because as you see i here executes still login.test but what we use is app test and now try that once more we run the file so and as you see we got the error that the test driver was failing and what is the reason because we don't wait for one of the methods so of course whenever we use the tester we have to wait for it to happen happen so what we will do is just adding everywhere we needed the await function we don't need that for the finders we don't need for the execute here but for all the test widgets so of course we should fix that and try it again so i executed again our shell script let it run in the xcode let's have a look into our simulator what happens all right so you could see that we enter the email address we enter the password clicking on login and we are now on the second screen and with that you have nearly all control over your application right so you can click something and so on and so forth and you can use the expect function if you want to expect something so that we want to do in the next step what we want to do is now using the finder to find the first checkbox so find dot by type and we just search for checkbox and we take the first one this will return us the very first checkbox here so amazing is the text of course we have to pass in a variable so this will be our first checkbox and with that we can now do the rest of the things so what we can do at this point is we can expect function and we want to expect if this one is not selected at the beginning so what we will do is using the tester and here we can get the semantics of the finder first checkbox and here we receive a list of different things that we can check for so i copied now some of the semantics over but as you can see matches semantics is the user at the matcher that we want to use and as you can see it has a tab action it has a checked state is checked is enabled and so on and important is is checked is the only thing that is false at that point all right so the next thing is check the semantics of the uh if it is not checked we have done that and the next thing is we want to tap that so we use again the tester the weight tester dot tab we find the first checkbox after that we want to pump and settle again that we wait until the animation of the selected checkbox is done and now at the end we can copy that expect here and expect the same thing at the end but is checked should be true this time because now it is selected so if we remove all of that and execute the command again here run file you can see that we enter now we click on the thing but you can see at the moment we get an error so what happened when the first method expected sync ah probably we again forgot somewhere the await method as you can see we did here so we have to await here and did we do it somewhere else wrong no i think that's okay now so let's execute it again so and now it should enter our email password select one and at the end we can see that all the test has been passed out of some reasons it just recreate everything once more perfect okay but what would happen if we have two test files that we want to test and for that i will just copy that over and call it app to test all right so now we only can execute with this command as you know one target file so the second file could be executed if we just copy that over and call that app to test but what happens now if we execute this file is that we restart the application twice we have to really build everything twice and so on so now with that you can split multiple tests into multiple files but keep in mind the more files you will generate the longer the tests will take all right so we saw how we can implement integration testing into our framework now you know how you can run multiple files we have seen how we create the boilerplate that is necessary we introduced a lot of stuff into the integration testing and you understood now the whole package of course the benefits are that you can now use all the things that you know from widget testing right away into integration testing make sure that you always use test widgets rather than just tests and with all that in mind you should be perfectly prepared to create your own integration tests for your projects here on this channel we will have the next video in a week i hope i will see you soon there and now thank you very much for watching and enjoy the rest of your day see ya guys

Channel: Flutter Explained

Views: 14,599

Keywords: integration testing dart, integration testing vs unit testing, integration testing flutter, integration testing example, integration testing tools, system integration testing, integration testing types, flutter testing, dart testing, test, dart, flutter, flutter unit testing, flutter test, flutter test package, flutter widget testing, flutter test examples, flutter ui testing, flutter test setup, flutter test single widget, software testing, integration testing

Id: WPEsnJgW99M

Length: 20min 32sec (1232 seconds)

Published: Thu Jan 21 2021

Reddit Comments

[u/Akimotoh]

Any ideas on how to do unit testing / integration tests with voice recognition? :)