ESP8266 Unit Testing - Part 1/2: Unity Test Framework - Day 32 of #100DaysOfCode​ in IoT

hi everyone and welcome to day 32 of 100 days of code in iot challenge today we're going to explore the subject of unit testing in embedded environment we're going to do it with esp8266 platform io and visual studio code and my plan is to try two different testing frameworks where the first one that is a default on platform io is called unity test framework which is a c unit testing meant for embedded code like low level stuff mainly and the second for more abstract code is called google test very popular testing framework in c word so let's get started okay let's start from creating new project so platform io quick access open and a new project here i've got a name for my framework however as i said yesterday this is still going to be published as a library not as a framework like with arduino because that's going to be much more work and it's something that i will do in the future so for now we're going to keep it as a library and to create a library with platform io you just create a platform i o project um i've got the name which is uni uno that's how i'm gonna call my that's how i'm gonna call my library my framework iot framework right we're going to start from nodemcu 1.0 that's our esp8266 and the framework arduino because that's going to be the base even if i turn this into a framework at some point in the future arduino is still going to be the base of of of of union okay and the default location for the location so finish and we should have our project created in a few seconds there we go there we go cool so yeah because this is just a library i am going to remove a leap directory since we're not gonna be using any private libraries this whole thing is going to be library right in terms of platform io.ini configuration there is a one thing that we're gonna need for testing on embedded environment in embedded environment and that is monitor speed which is a default one for esp8266 actually you need to experiment with higher values because it might be possible to to put a higher value so we could increase the speed of deployment but anyway let's stick to what we used to do and what we usually do right cool uh right so as for the testing what i'm going to start with is to look into a readme inside the test directory and here we can find the link to documentation on platformio.org and i've got this already opened here and here we can find all the informations about unit testing on platform io they have a very nice example and a demo application so if if if you click and go ahead to local number that calculator this is what is going to open so it's a project on github an example calculator and all the tests like all the configuration can be found here on platform io.ini right you can you can see there is a native and uh nodemcu i'm gonna tell you in a second what's native what native is and src which is the actual library i think it's not this one actually because what they do they use a different they use a different directory structure so they have just a private library calculator instead of uh we're gonna do it in in the other way cool uh calculator and uh cvpnh right but yeah like in this example what we're going to need first is to have a production code right something that we would like to test i mean not if you do test driven development but we're not going to do test event development today i'm gonna do test driven development for the uh developing the the futures so that's gonna be in the future right tomorrow today i will focus on testing the code after implementation so for that this is quite important i'm going to remove i mean the rename essentially the main.cpp and remove those two functions because this is meant to be library right so there's no main.cpp it's just the library files and i'm gonna actually copy paste the all the files from the core library here in a second but let me start with an example so example.cpp and we will have two functions here so one function is going to be sort of independent function function that can be tested on a native environment and let's just call it add so that's just gonna add two ins the simplest example right uh it's actually not void it's gonna return hint as well the simplest you could do right um let me rename this to hpp so we don't need to define any headers cool so that's that's one function and the second function is gonna use esp8266 native functions right so for that um let's maybe do something like um let me think about this set set build in set built in diode and that's gonna take the state and what we're gonna do is just digital right on the on the build in led and the state is going to be what we pass right i could make it the enum but you know just for the sake of simplicity right so if you pass high in here or one or like actually any value except zero this is gonna write to the built-in led and um just you know if it's if it's if it's one i think that's also high state so that's going to be switched off it's uh for the built-in led on esp8266 is actually contrary right so low status is diode switched on and high state is is switched off as far as i remember but yeah we're gonna we're gonna be able to test it with the unit test right so that's that's our example that we have and now let me explain what is the actual native environment right because that can be a little bit misleading right so so for that let me go to the example and copy paste what we've got in there so yeah native environment and for the nodemcu we need to ignore test desktop because this is what really a native environment is it is a desktop so that's that's a a specific environment on platform io that allows you to run the code on your machine right on your local machine on this computer so um you don't need to deploy it beforehand to esp8266 and that can be used for any sort of code that doesn't use esp8266 native like microcontroller native functions and this is the case for the add function here right this doesn't use anything from arduino okay but if it does if it uses something it is not really and it's not really possible to make the test i mean like you know i i can imagine an example where you could uh take the native esp8266 native functionalities out of the the the equation right out of the extract it out of some class or function then pass as the argument only so that that could allow you to use mocking right but i'm going to show you this in a minute how how this is done but yeah essentially the point here is if a function or class doesn't use native arduino functions something that needs to run on the board then you can use the native environment which means like native on this computer right and if it uses esp266 native functions so so like you know arduino framework functions and it needs embedded environment then you're gonna have to use the actual environment for the particular board and in our case it's not mcu v2 okay and that's why this configuration here with the test ignore entry that means for the node mcu v2 it is going to ignore folder test desktop and this is the one we're going to start with right so for that and for that environment all the tests inside this directory are not gonna run right and yeah so let me show you how example test looks like also like we're gonna still we're gonna still look into calculator the example from platform io and for test desktop test calculator i'm mainly interested in the include right so as i mentioned at the beginning of the video the unity is the framework that is used as a default one let me just close those and open the test desktop let's just have something like test example it's a default framework that is used with platform io unity test framework right it's a bit confusing with the unity for the game development right and from here i need the main this is how you initialize everything and i'm going to pick a random function i will rename it in a second so so this is still sort of like you know test i'm going to do test add we can have test art ads to values right that's what we can do and i'm gonna i'm gonna have to put this as an argument to run test macro right this one for now i'm going to remove cool you know what maybe instead of test example let's call it test add it's gonna be better cool so yeah so essentially that's how you initialize the unity test framework you need a main function this runs right natively again on my computer so um there is there are no setup and loop functions right this is just main function this is how you this is how you uh would write a c or c plus plus code um on the on the on the desktop okay so yeah so main and unity begin unity end and in between a run test on each test that you have in here at the moment this test is is empty um but yeah what what we really want to test what is the subject of the test so the subject is really we test if art adds two values okay so we need to have those two values and then set some sort of assertion on that right but before i do that i need to include our example hpp so we have access to the code from here and now here i could do something like that right at one and three something like that right and then i have an assertion here to check if the sum of one and three is four right but for that let me for a few seconds go to uh official documentation of unity test framework because this might come useful okay so what is unity this is a very low level just pursue testing library as you can see it follows unsee standards and it's portable expressive quick simple and extensible you can you can actually use you can do mocks as well here with cmok that i'm going to show you in a second when we think about using mox and here is just on a single page it shows how to use that framework which is pretty straightforward because like it's just few macros you can use um it all starts with the test asset the basic macro where you are supposed to pass comparison okay the only problem with this one though is that it doesn't show you explicitly why the code fade i mean it just shows you that you know those two values that you compared were not equal that's it right so what you could do is to add a message that's one option right or even better use different macros for comparison of different data types right so if you compare int then um if there was an an error and they were not equal you're gonna get more detailed error message right so as here it says expected 2 and was 1 right for the hex 8 it shows you in the in the hexa decimal format and also that works for strings and rise and i think this is it there's not there's not many of those macros in visual studio code let me go back to it in visual studio code actually you are able to use the code completion to check what the word macros are available so you see the the asset is the the the base one right here but there is many more right we have char double i think it's for most of the standard c types okay okay we even have something like less or equal greater than uh greater than for all the the basic c types right data types cool smaller than and so on right so this library is pretty straightforward if your code if you have a small code base for example and it's mainly c i would recommend that library let me just do this one so because we we know we operate on int we're gonna do a test asset equal in 32 and the expected value in our case is four and actual is this one right so we can write our test even better we could do something like that and then have expected actual right so that's how we've write the test cool and to run the tests again super simple super super simple you just open the the the terminal right using this button in here and you can use either platform io test or even shorter pio test and that's gonna build everything for you and run the tests um there is one issue here and that issue is due to the fact that we include arduino yeah um yeah let me just fix that quickly because on the native environment there is no arduino dot h header and what we could do is either separate that into two different fires which i'm thinking actually about doing because that's that's the shortest solution right so i'm gonna put the add into add here so there is no headers included and then for the set build in diode because we actually need arduino dot h we're gonna we're gonna keep it like that and maybe add a name example to set build build in diode okay so it's quite important because yeah if you include arduino arduino.h it's not gonna work with the native right so now we just got that and this needs to be add.hpp and let me run the test okay and that passed right that passed it works um this is how the output looks like right so it shows us what tests um has been uh what tests the test rather went went through right with the line which is uh actually this just shows us the line with the run test so that's the line 12 okay so one test zero failure zero ignored and success okay so now i'm gonna break this test on purpose so let me try to change that maybe one uh maybe here one and five uh one plus three plus five so that should obviously fail and let's see what the output is going to be cool yeah okay so we got something and that shows us you see testart cpp line 7 which is here expected 9 was 4 right so we expected that is nine but it was four but this is obviously incorrect now this is correct one additional thing this is actually better i try to break it here so we add one to everything and try again okay so that's what we got right so now it's the other way run so we expected four but we got five because of the issue here so let me remove that and i'm going to run it again and there we go right so that works so this is how you run tests on a native environment on your computer for the code that doesn't use um esp8266 arduino specific code okay um so that's for the desktop one additional thing i would like to mention which i think is important and that is an issue with having multiple files here so if you have multiple files in the desktop for the unity test framework you cannot have multiple mains like that because the code is not going to build it's not going to be there is an there is an issue with that and i think they are explaining how to fix that but um yeah there is this please note that you will need to use unit test guide if to hide non-test related source code for example own main or setup loop functions there is something mentioned about that but to be honest i did a quick test and um i couldn't get it to work even with with this approach with the guards if you have a multiple files here and multiple mains you get an error also if you try to put this main into a separate file like a bootstrap and then have those tests separately this is also not working i guess you would have to put them into src which doesn't really look nice because tests in my opinion shouldn't be like i mean maybe for someone that could make sense right because in a javascript world we have a spec files they usually they usually sit next to src so maybe that could be one option you know maybe we could give it a go quickly so what if i do something like let's try it so i do i move this file here just quickly to try to solve this issue so i move this file here and i'm going to call it art test instead okay that's what we got and the main is going to be moved to something like a bootstrap bootstrap dot cpp and that's what we go what we do we just put the main in here and then what we're gonna need to do is to include we just need to include unity we actually don't even need unity to be honest but yeah let's do it and the test right so that would be art ah that needs to be hpp now so at test hpp and now that should actually work so if i run this yeah that works okay okay so yeah this is apparently the the way to do it so you just keep a bootstrap.cpp in the in the desktop in the test desktop folder inside the tests and and then you just use the sort of like javascript spec approach right where you keep the the file the test files next to the actual file so you know it's not too bad however i see one disadvantage of this approach and that is on the bootstrap.cpp the problem here is um you need to add all the run test macros for all tests right so imagine you have like 10 files all of these 10 files have like 10 tests and you need to put the run test macro for all of them so this is this is the the moment this is this is something that is not very convenient with unity unity test framework however if there is a way and you know about it let me know in the comments because there might be a way to avoid this sort of situation where you need to put run test for all of the tests i don't know maybe based on prefix or something else anyway that's how you uh you can do test with the multiple files so i'm pretty sure that allows me to you know that that would allow me to add more of the the files here and uh um you know you know what you know i'm going to speed it up and just try as a just to double check i'm going to speed up and try with the division or multiplication maybe i try with multiplication so speeding up now okay done and i'm going to run the test now okay that works let me break it on purpose so we're gonna do something like that and let's see if the test fails and that failed okay and that failed because 10 was 20 which is not correct cool so yeah essentially this is how you do it this is how you can have multiple files and even you know your test sitting next to the actual implementation which is not too bad but i'm still for for you know adding all of those tests in the bootstrap.cpp that could be a little bit problematic maybe there is a workaround with um with a macro it could be some sort of macro but um yeah i i just leave it to you and let me know in the comments cool so that works nice that works nicely and let's move to the nodemcuv2 environment now right so how to test in embedded environment what what if you need to test a piece of functionality that uses internal esp-266 functions okay and this can be done with uh embedded testing for which i need to create another directory test embedded and inside test embedded we're going to create set built in built in diode test cpp okay for now i'm just going to keep the test here and maybe i i move it to src and just follow the same approach right we we had with the test desktop but for now let's just focus on the test only okay and for this test for this particular one we actually need arduino.h we're gonna need a setup function and loop function even though we're not really using it probably loop function is not really required okay but anyway let's have a look how this is done with platform io example so let's go to test test embedded test calculator how they have done it so they still use unity and they okay and this is the setup file okay this is very similar um we don't really need a loop function okay so let's do it something like that they say it's two seconds delay but to be honest just one second is enough we're just gonna have a single test in here and also something that is missing would be the serial monitor to be set to the same value we set the monitor speed to okay and now we're gonna need to have um something like a test that i'm going to called set test set built in diode switches maybe changes to high if we if um if the if high value is passed something like that right this is going to be pretty straightforward it doesn't actually add any any functionality to an existing built-in uh hang on actually what we could do change this to high if that's said maybe set a set built in diode voltage too high okay that's a bit long but it's it's at least you know it's it's expressive that's how the test should be should be expressing what what the the what the test actually does right so we test if set build in diode sets build in diode voltage too high if high value has been passed i had to do it longer great and let me define this test here now so void this one takes void and inside what we really want to do is to call our set build built-in diode but we need to include it first so set built in diode set build in diode and the state is going to be high and now to test it this is just going to be a simple asset we don't have unity here which is required and not this one this one test asset test asset equal yeah an expected is high and the actual is actually going to be digital read because this is more like integration testing right on the on the esp8266 so we we checking if the state has changed right and that's gonna happen on the build in led so that's what we're gonna do digital edit is on that okay so that's what we do this one is already set cool i think that's it let me open the set build in diode hpp as well to make sure cool yeah okay and right so net so now the next step is to make sure your esp8266 is connected via usb to your computer because that needs to upload the code okay i've got this connected and the next step is just running the tests we just need to run the tests and this is going to run the the native ones on my computer and after that the embedded ones right um yeah that's this there is just some warning at the beginning and yeah it takes a while because it needs to upload the code to esp266 so that's that's taking a while there we go there we go okay so that run the tests and what we got is passed right so that was successful nice the embedded tested uh passed right but we can add another one to check if we set it to low is actually gonna result in low okay so i'm gonna copy this paste but change if we set it to low if low value has been passed so that that's what we're gonna do right so we're gonna check on low now okay and we have to run the use the run test macro and pass it to this macro and let me run them again oh i see the warning is about oh okay now we got now we got failed we expected zero but we got one okay and this is why the tests are useful okay i know what the issue is this is why i prepared this particular example and let me fix that okay so what's wrong what do you think is wrong i can tell you what's wrong to do digital grid digital write on build in led you need to set the pin mode to output first on the build in led that should fix the issue so this is why the tests are useful yeah it is complaining i checked the warning messages it is complaining about the wrong constant i used the built-in led is deprecated so we can change it later to something else okay now it passed right so that that resolves the resolve the issue again as you can see this is why the tests are actually important and now following the approach from the test desktop what i could actually do is to just move this to src change the extension to hpp right and here create a bootstrap dot cpp and copy the this code to here and also i think unity would be required and actually arduino as well just to be safe and we need to include the actual test right so that we said built in diode test so now this has access to those okay yeah and i think this is this is how this is done okay yeah let me just quickly make sure this works okay past yeah so that's how you can have a multiple test fires regardless whether it's for the desktop or embedded environment um uh however i would suggest to indicate it somehow right at the moment this can be a little bit confusing because like they all have a test suffix so ideally you would have something maybe for the for the embed that you would have test embedded or integration or something like that right so that would be different suffix than just test but i leave it up to developer up to you if you are interested in this subject to do it yourself and and yeah um and the last thing about this unity test framework that i would like to talk about just quickly i'm not really gonna show you any examples is on the cmoc library that is recommended it is recommended here i think pull in cmok for full mocking support okay so if you need a simple mox so mox is essentially you need a function that you can control the behavior of right that is a sort of fake function right this is this smok i mean you know there are many definitions because you can also distinguish mocks and stops where um usually on the mogs you set some sort of expectations but i'm not really gonna go into this discussion right now so we're just gonna uh think of mock as like any uh any test double that that's that's the actual name for all of them okay and uh the point of the mox in cmoc library is um is to um is is actually it is a bit i would say it's interesting approach not sure if it's ideal maybe that's the only one if you really code in pure c and that works with the headers right so you need a header file and for that header file that framework that sorry that cmoc library generates a code an implementation that is the actual mock okay so that's that's that's what happens um let me just show you how this is done in this example okay so we have we have does something as defined here a function this is just a header does something that takes a and b here we can set the behavior of this does something what it does right and what it does is it expects to to receive 1 and 2 as an argument for a and b and that's gonna return three we make it return three okay so we stop it to return three something like that for for four and five we're gonna return six but we also expect four and five two to be to be given to this to this does something right so we expect does something to be called first with one and two and that's gonna written three then it express expects does fun does something function to be called with four and five and that's to return six and then it expects the that does something to receive seven and eight and it's going to throw an error status error ops and then we we actually have what is under the test here and under the test we have a function called calls does something so we expect that this function inside calls does something three times right first time with one and two second time with four and five and third time with seven and eight so that's essentially how this works it uses a ruby underneath as a language to generate the code i i suppose to generate all the codes for for for your uh for your mocks so that's that's like you know it relies on the code generation um but yeah you know the point of this cmoc uh with the with unity is really the simplicity and uh there is even c plus plus support but uh i see it as more meant for a very low level c code um it doesn't mean it couldn't be used with with higher level code right it does it does have c plus plus support as well but honestly my subjective opinion i'm not really sure about this code generating thing and also with the uh this is so the forces developer to go with the approach where you need to specify uh separate header files right so you have h files and c files this this is the approach that actually forces me to do so my hpp approach where i have everything i'm just in a single file that unfortunately wouldn't work right but anyway um this is this is just a tool and i see i'm pretty sure um that there is there are definitely cases where that would be the best tool to be used okay so that is for the unity and that's how you that's how you do all of this probably all of these files require also something like pragma once so we make sure so we make sure this is just included once okay this is actually gonna be included once but yeah just to be safe we could do that as well and we're gonna do one last run and that's gonna be the code that is going to my repository so you can download it play around yourself maybe build on top of it cool yeah the test passed right having this all done i'm gonna switch to google test right for google test what i would like to add is uh that i'm planning to use google test as a default framework for my for my iot framework the reason simply is uh google test is more is working better with c plus plus right so unity as i showed you is really nice i kind of like it for a very simple code or i i think like you know in in c code that would be the best solution but for c plus plus where i would like to uh have uh mocks on the classes quite a lot and also use callbacks uh i think google test is is going to be the better solution but let's prove it let's prove it however i'm afraid for today this is it because um i think the video is already too long and i don't really wanna end up with two hour video again so um i'm gonna end now and we continue tomorrow on google test only right we're gonna start clean and i will do google test i'm gonna start with a simple example like this one and then we're gonna move to the actual iot library with all the tools right so out of the core library we're gonna try to test maybe even dispatcher okay right if you want to stay up to date with my channel don't forget to subscribe right so um as soon as i upload the next video tomorrow you will get notification thanks for watching and see ya

Channel: Tomasz Tarnowski

Views: 1,130

Keywords:

Id: KPesyRp8qqo

Length: 43min 27sec (2607 seconds)

Published: Fri Jun 25 2021