droidcon SF 2017 - Mocks, Stubs, and Spies, Oh My!

hello everybody sorry for the little delay there my name is Brian Gardner thank you for being here to listen to my talk today so today I'd like to talk a little bit about test doubles now I've been doing testing on Android since I started I originally was a Ruby on Rails developer now if you know anything about Ruby on Rails you know they really really like testing so when I came over to Android I realized well people don't really seem to test their apps that much which was honestly kind of scary so I was basically trying to convert whatever knowledge I had over to Android so I enjoyed coming out and sharing a little bit about what I know this talk is meant to be kind of an introductory talk to test doubles mostly covering what test doubles are what different kinds of test doubles there are what tests you actually use those doubles in and last but not least how some testing libraries can promote some confusion in the community because they can mess with the names a little bit so that's what it's about also if you would like to follow along with these slides that bitly link at the bottom will take you to the speaker deck for this particular talk so you can kind of follow along on your own so test doubles test doubles are basically any object that you want to replace in tests now it's a very broad definition but basically that is what they are you'll see at the bottom there are five different test doubles I'm going to be talking about in this talk these are not things I made up on my own so don't worry about that these are actually coming from the X unit testing patterns book by Girard Meszaros I would definitely recommend reading through that book if you want a very deep dive not only into test doubles but into testing in general just be warned it is not really bedtime reading it's basically a textbook on testing it's somewhere around 900 pages long and it's very very dense so these are the five I'm going to be covering for this talk basically just what they are what they do where you're going to be using them and kind of just know you can mix and match these two meet whatever purposes you have so before I get into what all these different things are going to talk a little bit about why you would want to use test doubles I suppose I should start by saying where you wouldn't want to use test doubles if you have like a very plain model object that doesn't take any objects in as parameters or dependencies and the logic on them is very simple for their methods you don't need test doubles there you can just test those in unit tests it's very easy to do test doubles come into play first of all when your object has any indirect inputs now its indirect meaning you are not passing these things into the method call so maybe it takes in some kind of object in the constructor and then uses that object in the method you want to test to may be query some data from it that is a place where you would want a test double because a test double gives you a point of control in that method so you can basically specify this is the data I want to pass in so you know exactly what's going to happen in your test now on the flip side you also will want to use test elbows if your method has any kind of indirect outputs now this is anything other than the return value for that method because it's very easy to test a return value you just throw in assert and say hey this should return this particular data indirect outputs are more like maybe you're parsing some data from a web server and as a side effect you are sending the user a notification or something like that you will want to use a test double for whatever that notify component is so you can say hey verify when I parse this data correctly I want this object to receive this other call so you'll use a test double for that and last but not least slow tests now notice the asterisk here and I really should have made this a lot bigger because if you are noticing your tester slow don't just start throwing test doubles at it to try and make them faster there are very specific cases where you will want to use test doubles to speed up your tests and I will talk about those in a little bit but testicles are not a one-stop fix for that kind of issue so next are the types of tests I'm going to be talking about here so I have unit functional and integration tests now unit tests are when you are testing a single object in isolation from anything else it should not be talking to any other actual objects in your system integration tests are basically the polar opposite of that integration test should be as close to your final production code as possible you want to try and keep it as realistic as possible so you know your code actually works and just for me I say functional tests are kind of somewhere in the middle usually I think of functional tests as you have some subsystem inside of your application maybe a few objects that are working together and you just want to verify that those objects work together correctly so it's just kind of in the middle of those two so as I go through each of the objects I'll be talking about where that particular mock object applies to these tests and for the mocking library I will be using mockito for this because that's what I'm familiar with if you use power mock or easy mock or anything like that it's probably fairly similar but mockito is one of the largest mocking libraries for Java not just for Android so it is probably the most familiar to the most people so first off we will talk about mocks now mocks are probably the most commonly known name for a type of testable but they're probably also sort of missed named in some cases so again I am using the X unit testing pattern definition of these so for mocks you say a what is a mock well basically a mock is an object that you can configure with it's expected method calls and parameters so when I create a mock email service you'll say hey email service expect the method call with send email to this address with this content and then the mock object will basically say hey during this test I either did or did not receive this call now one thing about mocks is that they do the verification themselves so if the mock object receives a different method call the mock will fail your test it'll say hey I was not expecting this call so I'm just gonna throw and you're gonna get a test failure because of it now a secondary effect or secondary functionality of mock objects is that they can return values for the methods of that object now I say this is secondary because it's not really its main role mock to do this basically so your test can run so if you are expecting an indirect input from a particular object you still need to provide that or else your test is probably gonna fail because it's gonna say hey I didn't actually get any data from this object and then you'll get nullpointerexception somewhere so you can just return data just to make sure that your test can continue now with mocks there are two sort of subtypes two mocks we have strict and lenient and these names should be kind of self-explanatory strict mocks are very particular about what they verify so if the verification methods you pass in if they're out of order it'll typically fail for that if they get they call it does not expect it will fail for that so you have to be very careful when using strict mocks lenient mocks are as their name implies more laid-back about these kinds of things if the order is wrong they typically don't care about that they will just say hey I don't care what order I have I just got the method calls so it'll pass for that they may tolerate unexpected calls so if you didn't configure it to expect a certain method it might let that pass just fine so you just kind of have to pick which one you want to use for your particular test case so mocks where should you use them so I'll go ahead and start with unit tests well unit tests you are testing one object in isolation from everything else and mocks are made to test those indirect outputs so this is a great fit for your unit test so I give it the green light here you basically will pass in a mock object to your system under test and if it has that output correctly then it's going to pass your test now functional tests are a little bit different from unit tests because it is a group of objects it is a subsystem in your application but I will also give the green light for the functional test because you can typically use mocks at the boundary of that subsystem so if that system needs to interact with some other system in your application you will typically want to verify that it's calling the method correctly on that other system now for integration test remember I said that it needs to be as realistic as possible or as close to your application code as possible so you're not gonna want to use mock objects for your integration tests if you need to provide any other data for your integration test you're gonna want to use some other mechanism in order to achieve that so yes for unit and functional and a No for integration so this leads me into how mockito declares its mocks and with mockito since mock is in the name it does in fact have a mock method this is a static method on the mockito object basically how this works is you pass in the class that you want it to mock it creates an empty implementation of that object and basically puts in some logic to count how many times your methods were called it records what parameters were passed to that method so then at the end of your test you can verify that in this case the add attendee method was called on my attendee store now this deviates a little bit from the definition of a mock because by definition you're typically supposed to add your validations at the beginning of the test so right after you create your mock you would typically add those validations but mockito deviates from that a little bit and just has it do all of that at the end so next up we have Stubbs now Stubbs are also one of the most commonly used test doubles but they are probably one of the most missed a m-- and you'll see why in just a minute so Stubbs are responsible for returning values in a test that is their whole job you basically will create a stub object and they are responsible for handling those indirect inputs into your application so if I am using an object that uses my attendee store I will want to create a stub attendee store that returns some data that I can use in my test class now Stubbs cannot return anything that you don't specify because they're not that smart they basically just kind of return any values you create but Stubbs will also not record the number of interactions that they receive because that's not their job they don't care how many times a method was called they just want to return their data now like mocks Stubbs also have several subtypes so Stubbs we have responder saboteur and temporaries now the responder type is basically used to test the happy path in your application so you're giving back valid data and you want to test that path in your code The Saboteur is on the other side so you want to test the unhappy path so you want to throw an exception you want to return some invalid data so a saboteur is the name for that type of stub temporary test stubs not super relevant for testing itself temporary test stubs are more relevant to a test driven design type of environment where you know there's some kind of role in your application that hasn't been fulfilled yet but you don't have a name for that role so a temporary test stub is used in your actual production code to return the data you need until you figure out that role once that role is figured out and you have a class for it you basically delete the temporary stub because it's not needed anymore so just like the mocks we'll see where we can use our stubs so same way with the unit tests if your component has any sort of indirect inputs into it you're going to need to use a stub for that so we give it the green light their functional tests the same way if your system or subsystem in this case needs any kind of information to be passed in from that boundary you're also going to need to use a stub for that because it's very easy to control so we'll give it the green light there and again with the integration tests since this is supposed to be as close to realistic as possible stubs aren't very realistic so we go ahead and give that one at the red light so next we have mockito stubs and this is where some of the naming issues come into play because mockito does not have a stub method mockito has a mock method so you can create a mock but in this case it functions very similarly to these stub objects so not only can you verify that a method was called on it but you can also say hey I want to return this particular bit of data now the exact syntax for that they use a when then return syntax so when this method is called then return this data that is for the responder type of stub but if you want to Saboteur there's also a then throw so you can say hey when this method is called let's throw this exception so we can test that error condition next up is spies so spies honestly probably not one of the more commonly used test stubs but it is useful to know about them just in case you ever run into a use case for them throughout my like five or six years of Android development I've had to use spies like maybe three or four times so not very often but they are very useful when you need them so what is a spy well the easiest definition that I could come up with a spy is there kind of a combination of a mock and a stub a spy will basically allow you to configure what data you want to return and it will also record the method calls for you the main difference is that spies do not only record the method calls but they also typically record other information one common example I hear is that if you have a spy email service inside of your test if you want to say hey I should send ten emails with this email service when this thing happens instead of adding ten verify calls where it says verify send email was called with this data and then verify again and again and again the spy will basically give you access to say verify 10 emails were sent and verify it was sent to these addresses so it just makes it a little bit easier to verify that data instead of having to basically declare everything in order so yeah also allows you to verify - Lee calls so very similar so this leads to where should we use it and if you've been following along you probably know where this is going already so since it is a combination of a mock and stub you can test the indirect inputs and outputs in your unit test so we give that a green light same way for functional tests since you can test at those boundary layers we give it a green light for that and we give it the red light for integration test because it is not a production object so you shouldn't really use it there so this leads me to mockito spies which is one of the weirder things for mockito personally I think so if you've been following along I said mock a spy is a stub and a mock and you could say well this mock thing is basically a spy then because I can create this I can dictate what data I want to return and I can verify the method calls coming out and you are technically right in that case the weird thing with mockito is that it also has a spy method and this deviates quite a bit from the definition of a spy instead of passing in the class that you want to spy you have to pass in the actual object you are trying to spy on you can still basically override the return values for the methods here so when the get attendee method is called I can say hey don't call through to the actual implementation of this but returned my test attendees instead you can also verify whatever methods are called on it with whatever parameters now the reason mockito does this is because they want to allow you to test things with side effects now when you're using a mock objects there are no side effects at all so if you mock a arraylist for example if you call add on the arraylist several times when you call like get items from the ArrayList it will still be empty or if you get the size it will be empty because it's not actually doing anything to the mock object spas on the other hand at least mockito spy's in this way they do allow for side effects so if i mocked a ArrayList when i added items to it if i got the size it would tell me exactly how many items i added so this is very useful particularly in functional tests if you want to verify that subsystem is working but you also want to verify the state and you want side effects to be in play you should definitely use mockito spies so it deviates a little bit from the definition but it is still very useful yes the original methods also called like if there is a get it there is a get it in these method and you're faking the response when it is called is it calling it's a thinking response and also calling the real one I don't believe the actual method gets called there I'm pretty sure it just uses whatever pre-configured value you have methods of the real object you want leaving the other one exactly yeah so in this case for my spy store I would basically override the data that I wanted to return so I don't want to have to go in and say pull down data from a database and configure all of that data because that would slow down my tests a lot so I can instead say hey just override this one method and then leave all the other methods alone so it just makes things a little bit faster in that way next probably my favorite name for a mock object and this is dummies and dummies are probably the easiest to understand mock objects and that is because well what is a dummy well they don't actually do anything in your tests they just kind of sit there and basically their role is they fill parameter lists so if you can't pass in say no for a parameter or something like that or if it's required you basically make a dummy object and pass that in instead so we should use it again unit tests if whatever code path you are testing does not use that parameter then it is fine to use dummies in that case so I'd give that the green light functional tests are a little trickier because functional tests are more subsystem related so you're saying hey when these objects are working together they should do these things so I give this one kind of a maybe if you're doing a functional test most likely you won't run into a case where you need dummy objects you should probably pass in the real implementation but if you do run into it that's perfectly fine to pass in a dummy object and then finally for integration tests gonna give that one a read again just because again you are supposed to be testing your app from top to bottom from your UI all the way down to your business law business logic models so dummies don't really fit into that area now for mockito dummies it doesn't really have a dummy method either so again we use the lovely mock method typically what you'll do in this case is you'll just name it dummy so dummy attendee or attendee dummy one of the two just so it's like hey this is a dummy object don't actually use this one other way I've seen of doing this is by not using a mocking library but by creating your own actual dummy class I've seen people basically throw exceptions for all of you make the method calls there that's a little extreme to me because it's basically saying hey if this object is ever interacted with your test is going to fail I guess if you want to have that kind of confidence that it's not actually used that's fine but in most cases you're probably just going to want to pass in the mockito version instead and last but not least we have fakes now fakes are interesting because they don't really fit like the other test doubles we've seen so far and you'll see what I mean in just a minute so fake objects are basically they are real objects they are actually a Kotlin class or a Java class but they are a real object that you want to use in your tests now the job of a fake is to replace one of your real objects with another implementation so your fake version will typically implement the same interface as your real version but it will basically just give you the speed benefits to it the most common fake you'll typically hear about is a database fake where you have some database component that hits the database it's reading and writing to the file you want to pass in a fake implementation that does it all in memory so it basically will give you a huge speed boost upwards of like 10 times as fast so this is very important when you say have a large integration testing suite for instance and you want those to run faster so you can run them more often you'll want to provide a fake for that so where should you use it well it's not so good for unit tests because you'll typically want to use something like a stub or a mock for that because then you can control the inputs and outputs you can't have that much control for fakes really so typically you're not going to use fakes for unit tests you will however use it for functional tests so if whatever subsystem you're testing is hitting that database layer you can provide the fake version and it will give you a speed boost there same way with the integration tests you do have to be a little bit careful with integration tests because if you start implementing too many fakes in your integration test you're not really testing your entire system you're testing kind of a fake version of your system in that case so you just have to be kind of careful the way I've seen people typically use fakes is they will have some of their integration tests use fakes those are the ones that they will run the most often and then they will have other integration tests which truly test their entire stack but they aren't run as often maybe those are run overnight since they're going to take so long so if you have a nightly build like that you can run your entire suite with those real integration tests just to make sure that everything is truly working as you expect which leads me to mockito fakes and well mockito doesn't have any kind of concept of fakes because again they are a real object you're gonna need a real Java or Kotlin class for that so mockito does not have that bit of functionality for you so recap most important three things number one make sure you use the right terminology when describing what test doubles you're using even in your tests do not call everything a mock even though mockito tends to this is one of the big reasons that I think there is so much confusion over test doubles is because a lot of people when they come to testing they just hop in with something like mockito and then to them everything is a mock because that's what mockito calls it so if you are talking with your co-workers if you're naming your variables make sure you use the right words because a mock is much different than a stub just so you can get your point on what you're actually trying to test and what kind of behavior you're expecting make sure you match your test doubles for the appropriate tests don't start sticking mocks and stubs into your integration test because when something breaks you're gonna say well my test case is past so I don't really know what happens but just make sure again if you're using your test doubles in the right tests you will have more faith in your tests for being correct and last but not least do not overuse them which kind of leads me into my next point in that there can be some issues with test doubles particularly with overuse now what happens when you start overusing test doubles which is very easy to start doing I know I've done it and honestly still do it at some points you get what is called over specified tests this basically means that instead of testing what your system under test is trying to do you start testing the implementation that you are basically using in your test so you were basically saying first my object has to do this then this then this so this can lead to a lot of fragility in your tests you can change something in your object under test which can break a completely unrelated test case so when you see something like that happen it's going to be very painful but you're gonna have to go into your tests and start refactoring them to actually test what that object is doing rather than how it's doing it so test doubles can be good but like anything doing it too much is probably bad for you so just be careful so last but not least the Who am I so as I said before my name is Brian Gardner so that is my Twitter handle there Brian Gardner ATL I am from Atlanta if anybody else from Atlanta is here I kind of doubt it because it's a little far away again that bitly link is for the slide deck so if you want to check out the slides after the talk by all means but as for myself I work for a company called big nerd ranch some of you may have heard us from our technical trainings or our books that we put out so I am an Android developer and technical author for our advanced Android class but I also teach our essentials class but we have a bunch of classes and things we also have a Kotlin class coming out so we're very excited about that so if any of you would like to talk about test doubles afterwards or talk about technical training please hit me up I am open to that so does anybody have any questions about testable spicy integration test so imagine sometimes we want to do integration that's also called flow and we usually like this it's a very last mile of it it will be something deep in the Android or when we can to like extra guarantees that we can against something we can actually redefine actual measure value what we spice and spice an action site for it we can at least verify that we call proper method post Pro provided to the system and set like we can only your local system to be working right so using SPICE an integration test just if you are verifying that method calls are made I think that's probably fine just that like you said if you're calling to the system and you want to verify that you're making the correct system calls there that is good but the reason I said to normally not use spies and integration tests is because a big part of spies is to basically change what it returns and if you're changing what the values get returned then that can be a little dangerous integration tests do something or stuff we can't move like to verify final classes you could possibly do that one big thing about test doubles is typically you don't want to mock things that you don't really own you can get into a little bit of danger there but if you do find a good use case for it then I def wouldn't complain about mocking the flower class to verify yeah so for mockito how do you specify if a mock is how do you configure it to be either strict or lenient I've been using it and I think by default is lenient yes marquita marks are by definition lenient I don't believe there's a way to make them strict the only thing I've seen is they do have a custom rule you can do where if a mock is unused it can throw an exception for that but as far as I know you can't make them actually strict if you wanted to do that you'd probably have to write your own custom mock objects hey thanks for the great presentation one question that I had was you mentioned mock objects can sometimes return values in order to sort of make the the code under test run can you that sort of makes it sound a little bit like what you described in the case of a stub so can you speak a little bit how to think about returning values from mock objects versus using stubs and how to differentiate the two effectively yeah so I think the way mockito typically differentiates that is when you mock an object it creates an empty implementation of that class so it adds some counting for how many times each particular method was called and it checks the parameters like that as well now for returning values mocks will typically only return the default value for whatever your return type is so if you return an int it should return zero if you return a boolean value it should return false so anytime you get away from returning those default values that's where you kind of get it two more a stub type of object because again what mocks are just returning things to make sure that the class keeps going stubs are more for actually providing that necessary data I was just wondering in unit tests so normally how often we use a mock because recently I'm searching the topic and I found that there are two groups in debate how often right so there are kind of two classes too especially with test-driven development like the classical test-driven development where you don't use mocks at all there's also the ma kissed version where you basically use mocks for everything personally I'm more on the mock side so I will typically reach for mocks instead of other actual objects they just make it easier to verify hey this particular value is returned especially if you want to check what kind of parameters were passed in mocks make that extremely easy to do but again it is very easy to overdo it so I do have to sometimes rein myself in and say hey I probably shouldn't use a mock here because again I start to over specify my tests so thanks for the presentation quick question is considering and spiced a cup on a real implementation in order for it to really kind of wrap around it and as far as it can remember you can even call real objects on that what happened would you consider basically instantiated in spy on top of a one of the fakes that you mentioned just just in the SN areas of you know the real implementations being like - depending on Android components so it wouldn't really run on unit tests I think that's a good case especially if you notice whatever real object you are trying to wrap is slow like the database implementation for example I think spying on that fake version would be just fine just to say hey verify that this particular method is called and then as long as you have like really good unit tests around whatever object you're replacing I don't see any issues with just wrapping the fake with a spy okay yeah wasn't sure if it was the limitation of the Mojito framework potentially now yeah mockito can mock and spy pretty much any class or object you give it quick question were using Robo electric currently and trying to get away from it especially for testing logic in our activities in theory we should be able to like use mocks to inject all of the like inflated views we have and then run a test on an activity using only mocks or is that gonna be limited because you can mock final classes so actually in my Kido 2.10 I believe or my Kido to whenever that came out you can actually mock final classes now that was kind of the big thing they added for the next release so you can mock those final classes now if you need to but I have definitely gone down the same path I started using robolectric when I first started testing and testing your activities is quite a pain so I would do basically that I would have like my custom view objects I would mock those out pass them into the activity and then just verify the right calls are being made on whatever actions just to make things a lot easier so you can you can do that without robolectric if you use mocks properly right I believe so yeah having your activities run though can be a little bit difficult without something like robolectric but I've definitely seen it work that way yeah all right well if there's no other questions thank you all for coming to my talk I hope it was helpful and if you have any other questions about tests Devils feel free to hit me up during the conference thank you very much

Channel: droidcon SF

Views: 5,170

Keywords:

Id: tVCSKsMtXn0

Length: 35min 36sec (2136 seconds)

Published: Tue Dec 05 2017