iOS Automated Testing with PhotoKit, Vision, and other frameworks | iOS Dev Live Mentoring

hello gordon hello mike hi gordon hi kyle hey guys hi doing welcome back gordon welcome back yes thank you thank you so how can we help you today um so um i've been spending some time recently just looking at other frameworks that we get within um the the apple sort of ecosystem um and one of the things that i'm kind of trying to understand is from a testable perspective um where we should kind of once was kind of integrating with these frameworks where we should kind of draw the line in uh you know i'm i'm not going to test this part of the code because it's it's either code that i don't own or how to architect the system in a way where we minimize the amount of code that we're kind of executing that we don't own if that makes sense um to kind of push those uh those kind of those areas where you kind of take your hand off the wheel and say i trust apple to have tested this framework or i trust you know whoever and you kind of just expect that to work each time um i found with some frameworks things like for example like we learn in the academy with um url session things like that even though that's code we don't own because of the the the protocol we're very much able to um intercept those requests so we can still test our networking layer and have complete control over it with things like core data we're able to seed data early on or see data in tests so we have control over that environment but some frameworks in in this example would be like photo kit and and vision um right they they um they don't you're not given the same kind of advantages as you are with like the url protocol stuff and things like mechanisms right that's exactly yeah like for data you can add some data like persist some data in the file system and then you can query the data and see like the data is the data i added is still there right so exactly exactly you can get some feedback from that yeah and you can you can you can configure that data depending on the you know the test case if you want to test to test for a certain example you can see the database that you can see core data with that data so you can control that environment but some things are um some some frameworks just don't kind of give you those allowances yeah so when you're dealing with frameworks that were designed with testability in mind or they have any kind of testing facilities like store kit now does beforehand we didn't have testing sandbox and that we could run unit test but now we do and we can create like all this mock data from xcode and run starkid so when a framework is built with testability in mind it's much easier it makes our lives much easier yeah yeah so url session has all these mechanisms that you can use the url protocol for the url loading system and you can kind of intercept any requests i don't think that was made for testability it was made for allowing new types of network protocols to be added easily like https ftp and so on but we can use that it can take advantage of that mechanism to intercept requests and facilitate testing yes but yeah frameworks like photo keeps like vision it's a bit more like a black box you don't know what's going on you don't know what happens internally and there's no way you can investigate what's going on yeah you find a lot of those frameworks as well kind of um the apis to access them are kind of like static methods on a class so it's very difficult to um it's very difficult to like represent the type that's kind of being created or or understand how you can inject those dependencies or at least i've struggled with that up until this point because we're kind of just working with static values as opposed to like a a class that conforms to something that we can kind of build our own spy on on top of yeah so usually when dealing with static methods since with functions are first class citizens we can pass functions as parameters for example right so we can inject a function that have the same signature as the vision framework for example and this is a way you can enable testability but there are two kinds of tests there right there are unit tests there's more like developer tests that you're testing like that your code does what's supposed to do and there's more like behavior tests more integration tests it will actually check that vision is doing what you expect it to do right so usually you don't want to test third party code you don't want to test code that apple should have tested so what you do is you test your interaction with the framework as long as you're sending the right messages at the right time with the right values the framework should do its job all right so in a design you can have your code and you can have some kind of boundary it can be a protocol like a closure an abstract interface then you have to the framework implementing that boundary now you can test this in isolation without the framework right because you can test that your code is sending the right messages to this interface because you can create i a mock a stub something that will pretend to be the framework yeah but it's not so can have a test double for example when testing here's the real framework well here we're not actually testing the behavior of the framework we are testing that we are sending the right messages at the right time the the right amount of messages with the right amount of data and so on however this can be tested but we're not testing that actually vision is doing what we expect it to do yeah right for example when we mock or we replace a url session with a test double we're not testing that the request will actually hit your back end and return what you expect so your test may pass because you are sending the right messages as set up in the contract with the back end but the back-end may misbehave it returns something completely unrelated they can break the contract right so it depends on how much you trust those frameworks to see if you need to create this integration test that will actually exercise the framework or not and sometimes like record data for example is such a complex framework they have so many settings so many buttons you can click that change completely the behavior so many performance improvements that could break some expectations that you usually want to write some tests with core data in the middle there to make sure that core data is doing what they expect because there's threading in the middle there are queues there are all those settings for performance there's a lot of hidden behavior so you actually use core data during in your tests to make sure it does what you expect yeah and usually you you need to decide here the cost benefit let's say we have cost and we have benefit so here is high cost here is low cost here is high benefit here is low benefit so something that is high cost with high benefits so writing a test that it's going to take you know it's going to be expensive because it takes a long time to write or it takes a long time to run because it needs to be a ui test but if it has a high benefit yes let's test it right if it's high benefit low cost absolutely let's test now low cost low benefit why not is low cost doesn't cost a lot now high cost and low benefit usually you don't want to test it if it costs too much like oh do i need to run a test that takes an hour but it doesn't give me a lot of benefit because even if it passes i don't trust that test don't write this test yeah and this is the hard part this defining something that is high cost and low benefit and maybe the benefit is here is no benefit you know like no benefit then absolutely you shouldn't you shouldn't write this test yeah but there's a lot of points here right in this quadrant that something might be not a lot of benefit but still have benefits yeah so you need to find out good trade-off here yeah what's going to work for your case exactly and various strategies as well like something that comes to mind right now is for example you know like we're talking a lot about ui tests right and how the cost is very high but maybe it's something that is worth you know like having a ui tester so there's a higher benefit but the cost is very high as well and then perhaps you know like you create a new uh scheme for that right so you you don't make it part of your development process right so you're trying always to improve this uh cost benefit uh pair right even though it seems you know initially that hey like this is definitely you know it's going to take a lot of time to run right so you're always trying to find ways to to facilitate that and make sure guarantee that what you're making is you know is doing what it's supposed to do yeah and some features only work on the device for example so you cannot run the tests on the simulator even ui tests wouldn't work then you need to have a whole infrastructure for running tests like a ci infrastructure that can run tests on devices and some companies do it it's very expensive but they see a benefit in doing so so they have a stack of all kinds of devices all kind of ios versions running tests on actual devices running ui tests unit tests integration tests and that's very expensive but they see a benefit in doing so but other maybe smaller projects would not benefit of all these costs so that's why it's there's no definitive answer right so when the cost is low like using your session card data it's very easy to write those tests now let's have a look at photo kit and vision do you have some examples here for us um in in the project that um yeah so this project is just a it's a simple app that basically will fetch um up to a thousand photos from the photo albums on your device um it sorts them in in the most recent kind of order and then you're able to select one of these photos and basically what will happen is vision will yeah run an analysis on the image and then decide what's what's um like coordinates within that image you're most likely to have your attention drawn by and then i just draw a red bounding box basically around that to it to indicate that um so there are some um there's some services that we kind of use in this in this example to kind of do that so if if we had a look in the image image analyzer um yeah the attention-based one yeah so this is is essentially where we kind of run that that query against the given image um and and kind of get some results back and you can kind of see the the um the kind of setup that vision requires or or what it requires to run um it's um it's kind of very specific to vision if you know what i mean it's kind of like um one of the things i kind of thought about was about how to kind of abstract this stuff away or or how to integrate with this stuff in a in a more testable way but it's very much like they have their own set of like results and their own set of kind of classes that they return based on on the outcome and um yeah it it's kind of difficult for example as a simple example it's kind of difficult to force this service to fail if that makes sense right just to just to see how how the output of this service would be you know just to cover in a fail it you know it had it passes the correct error and things like that but it can fail right it throws yes does it say why you would throw um it it can throw for a number of reasons if if the if the image analysis fails if if the if you've kind of not set up the request and the handler properly um there's um there's kind of a various reasons why it might fail um it doesn't it it's so it won't fail in the it couldn't find anything in the image it will just return an empty array in that event you know there's no observations for this image um so it's not like uh so you can't feed it like uh bad data and that's gone right i tried that have you tried an empty image zero by zero pixels uh it just returns it it will just return uh um that there was no observations in that image yeah it doesn't okay yeah so you would have a path here that you couldn't test that's right yeah so this is our case that the benefit is quite a bit low because the code is quite simple like do catch and they're complete you're just not testing this specific path one line right and maybe the cost for simulating a failure here of mocking a bunch of methods and injecting behavior just to be able to throw and catch here may be too high this is one of those cases that you may decide you know what one line of code here can this go bad yes can i refactor in here maybe break this behavior because we remove somehow the the catch and it never completes it can happen so it's a cost analysis here what is the cost of making this fail for testing purposes it's too high you have to mark all those classes then probably yeah so there are tests around some components um and the only way i could find to reliably kind of create a spy or a mock was to to subclass some of the components and ad add methods to like capture when they were called and obviously that's um that's kind of quite high risk because that could break at any point like a new new version could come out and the internals could break or there could be an edge case i guess that i'm not aware of that means the code works but the tests fail and the tests fail because it's not intended for me to be kind of you know overriding certain methods yes yes exactly and it was also you i mean we'll see if we kind of look at some of the tests it was also quite that this like used kind of subclass you know the observation class and then there's a property that you expect from that so you have to subclass that property and then there's a value on there that you want you kind of have to override that but that returns another property on a model so you now need to subclass that model and it's like uh feels very flaky you know it doesn't feel very elegant and it doesn't feel it feels like it could go wrong at any time at any point and it might be difficult to work out where that happened that's always just a concern for me with subclassing code you don't own for testing yes absolutely we need to avoid that yeah that's why you did exactly what i said here you create an image analyzer like interface here a protocol in this case with yeah just a simple interface right that yeah doesn't expose any of the internals although it leaks here a little bit the observation and what do you need from this observation um so the observation has things like like say they call them salient objects and the bounding box where we kind of start to get the the properties um the properties that we want to try and calculate how that observation should um how that bounding box should be drawn um so i guess just thinking about that now it might have been it might have made more sense if i'm gonna be abstracting away like if i'm gonna get this array of observations but eventually i just kind of want like a cg rect that's like a foundation right that i can stub and and and mock myself you know with with minimal effort um so that i guess the analyzer could have done a bit more work to kind of give you the interface that you want um give you the output that you want rather than just stopping at the first output if that makes sense yes yeah it could be something like this so instead of leaking vision details here you could have your own structure what you expect or you just say maybe it's just a cg rect right something like this yeah you hide all the implementation details of how to get the c direct maybe the c direct is coming from vision maybe you're actually making an api request to a back-end service that we analyze the image and return it to you right yeah there are different implementations of this image analyzer could be on device or it could be on the web and so on yeah that makes sense because something i did consider was um so for example in in the app when you click the icon at the top it changes the aspect ratio of the image and we we reanalyze the image yeah so then if you click it back again it goes back to the previous image now of course these images are static nothing changed so the calculation and the perform the work's already been done so there could be a layer in between that caches those previous operations right and and of course with ctrex that kind of thing would be much easier than me trying to store like vision observations and things yeah it makes a lot of sense yes you have like a decorator right something like that yeah in the middle of that and then you just need to scale this pretty much is it's a cg-rec dot scale right yeah yeah then you could just test that your framework your code is sending the right messages to this interface we're guessing that your code is doing the right thing not that vision works as expected right because then you can use a test double when testing your code you don't need vision in the mix yeah and this could be more like a unit test style a developer test if you want a behavior test you can also write now a test they're calling your implementation that uses vision does what you expect and you can have a an image in your test bundle that you load they already expect a specific bound in the bounds there you know a specific cg react and you call the real framework and see if it returns something close or within those bounds yeah because one thing that happens with real frameworks like vision anything related to ai is that every new ios version that will have different models different implementations and those behavior this behavior will change the body box you get on ios 15 is probably different than what you're gonna get on ios 15.1 maybe it's like just a couple of points but we break your test if your test is not flexible enough so to say hey it's if it's something with a margin of error of 10 points 100 points whatever makes sense it's acceptable and you can write a test using the real framework now not directly but you will call your implementation the heights that all this usage of the framework right in the unit test you will call it with an image that is bundled and if it returns already a c direct you just check that c direct is between the bounds you expect for that specific test image you can also have an array of test images that you run yeah uh your code and make sure that even on new ios versions it's still within the boundaries you expect um yeah that makes sense yeah now how how slow would be those tests i don't know would depend on the machine running it but i guess that goes back to what you were saying mike right about pushing we can push things like this to a scheme that will handle this in in absolutely the ci pipeline for example so we as developers can continue to get quick feedback and and let the um kind of let the sea ice kind of catch those things as well or a separate scheme that we can kind of run before pushing to the repository that's it you can either add a new target for those slow tests you can call it like integration tests or vision integration tests and you don't run them as part of your development process because they take a long time and and in here you're just hedging against like vision breaking and expectation right so this is very unlikely to happen so you need to run it less times because you cannot break a vision only apple can break region yeah so you only need to run it eventually when there are new os versions and and so on yeah and you keep running tests against your code and every now and then when there's a new ios version you run those is lower test to make sure that everything still works as you expect it to work another way as you said we can configure the scheme to run only specific tests and then you can have two different schemes you have one target but two different schemes one with only fast test another with slower tests i find it harder to manage in the scheme target here yeah but it can be done as well for example you can disable some tests from the scheme and you can add a new scheme with those disabled tests but i find it harder to manage here because every time we add a new test you need to remember to come here and and edit what can be done with schemes or separate targets for those frameworks yeah that makes a lot of sense i think um especially in terms of vision i i kind of see now how how those abstractions can kind of be done and how we can kind of just test you kind of architect architect the system in a way that you're only dealing with the interfaces that you control between like the modules and and kind of you know push those um push those integrations with the frameworks to two separate set of tests where it's like yeah you're testing your code and then you're testing everything in integration as a secondary kind of process yes that makes a lot of sense yeah that's when we say that you should separate infrastructure from your code because then you have all your code here and some interfaces similar to ports and adapters you have your code we think there's boundaries here you know and you communicate with external world with those ports right you plug in vision but maybe you plug in a web service that will analyze the image yeah or you plug a test double when you're testing so you can test all your code all your logic and saying like oh if the real framework returns this specific c direct i want this to happen in my ui image right you can test all of this without those frameworks that can be slow or fragile and then you're free to change those frameworks and that makes a lot of sense yeah it does especially because if we're talking in terms of vision vision is really good but vision when you use it in combination with something like core ml and some like saliency models it becomes even more accurate than it is now um and yeah i can see how by following this kind of design it's very simple very simple as a oversimplification but it's it's much more straightforward to swap you know swap services in and out there's a saliency attention-based service and then there's the one that uses core ml as well you know it's i see that that does make sense yeah you can easily use a third-party library as well you know you can get at cocoapods and bring it in put it here without changing your code yeah and you can test all of this in isolation that's why we usually recommend separating infrastructure and hiding infrastructure from your code with a tiny boundary yeah very very tiny boundary here will give you much more flexibility the other things you could use to test uh [Music] like vision is getting this generated image take a snapshot and keep in your snapshot testing suite as well the problem is that if there is one tiny difference here of one point in the box it will fail unless you give it some uh percentage of tolerance yeah yeah yeah especially as we move between ios versions that's obviously we still haven't seen that recently that that can happen so that's a real a real thing um i found with photo kit um that was that was much harder to kind of to kind of test because um or not to test that i did try and abstract things away um if you have a look at the um image and so in the scenes if you kind of go into the image uh image grids yeah if you kind of going to the composer for yeah that one um so i kind of kind of added some adapters and and try to only inject like the method signatures and things like that to give me a bit more control um but a lot of the um a lot of the operations that kind of happen with with photo kit are all kind of asynchronous anyway so there's kind of like completion handlers on completion handlers so um it was kind of like trying to abstract things away i guess it felt like a lot of a lot of work but that could just be a symptom of of working with something with a particular framework i guess that's just not developed with that in mind um those sort of behaviors right yeah so do we have an example here yes sorry so um if you have a look at um if you go into the um there's an adapter in this component um if we start with the view controller sorry so if you're going to the image grid view controller um that one yeah so this is just a collection view um and i was able to kind of keep keep vision keep photo kit everything like that kind of out of this component and was quite happy with with that approach by just yeah very small file yeah just working just working with what kind of what ui kit expects it it kind of worked um but this this calls on load on an adapter um this that's in the composer um okay so here you create obstructions as well but instead of protocols you're using closures yes yeah download and you can preload images you can cancel preload images yeah um i did think about abstracting those into into protocols but i kind of thought the first one was a was a first one would have only had one method so it works as a um worked as a closure and then i guess the prefetch i kind of could have done that as well that that that could have been a protocol because i guess the decision was if if you start pre-fetching should it should the clients be forced to cancel prefetching and i didn't kind of think that much into it um but they would in definitely if i was making a production kind of app i would definitely expect those two to kind of be like if you're responsible for starting it you should stop it as well because we should be good citizens um yeah absolutely um so yeah so there's um the uh there no sorry the bottom one there image loader presentation adapter the one just below yeah so this basically accepts um this basically gets called when the view controller loads um and here is where we kind of do the work to talk to uh photo kit um and we basically create a request here with these fetch options and then we get some assets returned to us those assets are like of a kind of weird type um they're a like a fetch result with assets which kind of look like an array but you can't interact with them in in in the form of a collection um so you can see yeah that fetch results has it right yeah um so once we kind of get that result back here we we kind of pass that out by that closure um to another component and that so this is like an array of acids right yeah you have to enumerate over the that but it's um it's a bit it's it's a bit it's just a bit strange because it's not like you you can call for each or anything on it like it's a collection and it has a count and you can kind of access values by the index but you can't work with it like an array which um just was a little bit of her just to surprise me a little bit when i started using it um but this is we also in this adapter we handle the prefetching so if you kind of scroll down a bit you'll kind of see those here we have the preload and the cancel as well so it's prefetching is a bit different in the same way that we would for example prefetch from a remote service or something we kind of just tell this caching manager to start caching these images and obviously to cancel that operation as well um yeah so um what we do yeah so now what we do once we kind of get those assets um this is why we kind of hold a reference to the assets in this component because to pre-load and and to cancel preloading we need to get the assets we need to get the ass the asset off of that collection um as opposed to just like passing it straight out as a onto a callback or a closure but you'll see in the composer now once we kind of get this result and start to map it to cell controllers yeah so here's where we kind of that so that output there is that collection and that that fetch result um the asset type on it and now we kind of enumerate over that to create uh eventually what would be a ui image which is kind of what we do in the cell controller delegate adapter right so each cell controller which represents a cell yeah so here we have the each cell has a cell controller which is created with the image delegate yeah so the images are also loaded asynchronously that's right yes because we have to because we have to map them then from a ph asset to a to a ui image and to do that we have to we kind of have to request that um like give me the give me the the url or give me the image for this asset and then that operation has a completion handler it's not like a synchronous where it just returns us um it just returns us the value so it kind of forces this to be a an asynchronous kind of flow where we kind of request and then provide a mechanism to receive the image to receive the image back which can be a bonus in a way in in this way in that if if we were to show images from a remote source for example um and we weren't kind of abstracting all that synchronicity away um like this cell controller i could use for remote images as well very easily it's um right it doesn't depend on photo kit exactly yeah exactly like it needs an image that will be loaded at some point by someone you just tell it the size you need yeah yeah all right so you can test this again this is your code you can test this yeah very easily so this is tested 100 no problems here now so um you can see if you go into the the controller delegate adapter the the class just the adapter class um on 40 that's the one yeah so here we kind of we get the asset um and we're able to create a task to make it cancelable which we can kind of we can kind of um track as well um but then we interact with this caching image manager which is kind of what fetches the images for us and when we request the image and and this is kind of where we start to touch code that we obviously we don't own um and to test this i was kind of like i kind of created a subclass for the image manager um but that kind of forced um just forced me to do some things in tests that didn't that probably aren't representative of what really happens if that makes sense right yes so this this the photo kit kind of view the grid the images the kind of the selection of images the correct image being rendered all tested but tested by subclassing ph asset image request id caching image manager too and what if you use a real one would it work so the the thing is that i found using a real one because to access the photo album on the device you obviously need permission to access that and there's some like settings that you would do you know select photos every photo no photos and i was like confused about how to handle that edge case and then also because i couldn't see the photo album in test or i wasn't aware of a way to do that i was never it was never clear how i would know like every time i run my tests on every simulator for every version i'm going to get the same the same output if that makes sense yeah you would have to set your simulator always in the right state before starting the test which is to give permission yep and make sure that all the images there are the same yeah can be done it can be done yeah but usually these kind of things will be done with a ui test yeah yeah it's simpler to give permissions and things like this um that makes sense in these cases and you see that no there's a benefit of doing it because there's some quite complex logic here that i want to be sure that i'm doing it correctly then you can use you can subclass it as long as you don't change a lot of the behavior because you can create a mock that changes all the behavior and then you're not testing the framework anymore but if you're changing it to do that's a tiny bit thing different that might be okay we have a specific subclass here yeah if you have a look in oh it'll be in the image grid folder sorry that you're just in um yep and then in in the test helpers um so you can see in the the loader spy test is is kind of where uh the one at the bottom sorry yep that one kind of has the subclasses for for for what i was doing i probably should have renamed that to be honest um but you can kind of see the extent of of what i was doing with subclassing was literally just trying to spy on what took place and and where there was a completion handler kind of capturing that to try and uh to allow me to invoke it kind of a bit later on um but to do that because because there's expected return types aren't necessarily the same as what you would um they're not kind of like you're returning an asset and the asset has a property on it but that property happens to be another class so there's kind of like stubs of the all of these properties um which it just feels like it complicates the test um for example you can see um in this test here or in this override on online 42 kind of capture the capture the completion handle so it can be invoked but then we just return the default we just create a ph image request id because there's a requirement to return that um but that's not necessarily representative of what happens you know in the real system and there's another one a bit further down which if you keep going there's a bunch of overrides here right yeah exactly there's one as well where there's like um where we kind of enumerate over swear to we're expected to enumerate over some stuff and there's some values that we we have to return and i kind of here we are so this kind of line 74 this kind of four each um there's an enumerate objects method that is kind of responsible for for returning the assets so i step set the assets up with a stub and then use a four each but there's additional values here you can see this unsafe mutual pointer and an integer and i'm kind of providing a default value so i can compile and run the test but again it's like that's not what happens in in the real world and i'm like the test pass but it's how much value is there in that because is that is that is that that's not what really happens do you know what you know what i mean yes yeah so here you're not testing what really happens but you're testing how your code will react if this happened right yeah yeah so here it's probably fine because you're not changing the actual behavior you you're stubbing you're just replacing hey when you call this method just return this thing that i'm giving to you up front right so you create your stub with some specific assets and that's it it's just an enumeration like enumerations shouldn't have side effects in this framework right they could they could they have side effects that you don't expect that's why ideally you should use the real framework as i said the cost here is high because you need to set up your simulator it always needs to have the same same images in there you should always have permission to access the photo album so maybe the cost is too high so then you go to the second step using stubs or mocking would it change too much the behavior in a way that i'm not actually testing the framework the answer is yes then you may even decide to not test it the code then you look at the code like how much complexity there is in the code you're actually testing it looks very simple right like that this is all set up like there's no behavior here correct that's correct yeah what actually happens here so this is what you're actually testing right so you you need to create so many mocks to test like three lines of code right i think so i think i made the like the fatal error of like you know we're like running the tests and then seeing like the red bars down the side about stuff that isn't kind of covered and then be like oh this is a disaster like i can't leave this whole you know these these 10 lines here with a red block like but then the side effect of that is i've written like 85 90 lines of code to test you know two things that an image comes through and that a task is created um yeah so it's kind of like high that's that's kind of high cost definitely i think yes because if there's a breaking change in the framework your test will not tell you that because you are mocking the behavior right so maybe there's not a lot of benefit there are benefits for sure there are benefits but maybe not a lot so in the quadrant we were drawing before we are somewhere in here right where the cost is high here and the benefit is quite low so you prove that it can be done we you can also try to actually investigate a lot how to do it without mocking anything and see what is the cost of that as well maybe it is way more right maybe it's even more because of actually using photo kit in the tests and what is the cost of doing this with ui tests as well [Music] right compare the options you don't need to actually implement all of them but like bring all those options to the table and also have the option like we're not going to test it because only it's only four lines of code they're quite fairly simple just need to call a method yeah i found that is something i've started to adopt with with swift ui stuff like one of the things that we kind of lost in the transition from ui kit to swift ui is for example being able to trigger a button in in code programmatically in the same way we can with ui kit and it's kind of like i've kind of adopted the mindset well if i build my views simple enough then i can see in the preview that that behavior works i can have a ui test that kind of does the flow end to end and that's okay like until apple provide us a different mechanism and a way to do it i i'm like i'm okay with that like it it doesn't keep me awake at night anymore but this like you know this um this should be applied in the same way it makes sense there's the humble object pattern okay remove all the behavior from the object make it just like it's very close to getters and setters right it's so simple they're like why would a test get as in setters just like in swift ui if you design your ui there is just actually getting the data from the model and putting in specific places and there's not a lot of behavior going on when you tap buttons you just call a function you can say that oh it's the humble object here he doesn't have behavior anything crazy there's still a risk that you will not call the right method at the right time and not actually testing what's going on there but when you're dealing with something that is very hard to test and the cost is too high you move all the logic outside the component right you move all the logic outside the component hide the component from your logic and this may be the tiny adapter that you create here you have an adapter here well the real framework is tested by whoever created a framework you test your code and then there's a tiny tiny as small as possible here adapter which is a humble object here that it's so simple so simple that the the risk is so low that the cost of trying to test it would not make it worth it yeah yeah but don't resort to the humble objects and just always right always always wait the cost benefit because otherwise you can we can become lazy you know yeah and just say i'm not going to test it because oh my god it's so simple and then the bugs start coming back in and then you start losing the benefit of having tests so this is for extreme cases when you're dealing with frameworks are very hard to test very very weird apis everything is optional everything you know you cannot simulate errors like daniel you make an informed decision by waiting options cost benefit and you move all the logic that you can test outside the infrastructure and you may have to create a tiny adapter here as you did with only four lines of code and then you say i'm making an informed decision that i will not test these four lines of code because the cost to do so and the maintenance burden and the benefits i will get is not enough to justify yeah that's actually it's actually really interesting as well because something that that makes me think about is when you kind of take that approach when you're discussing code with another developer if there's more than one of you working on a project you it's very easy to be aware of where those risks are when you're talking about an adapter at the boundary of the system you're not talking about like remember in this class you have to do this and remember in that service oh we didn't test that because of this it's like we know that that component is the way it is for a reason and it's very easy to to reason about the rest of the system you know in a collaborative way um because you're not thinking of you're not having to think of a bunch of different things it's it's like that's why that component behaves that way or that's why it is engineered that way um and it kind of gives everyone a point of reference when they're talking about these things which is is really useful as well yeah absolutely it's very important that you are on the same page with everyone in your team right everyone understands that as well so the communication should be very clear because i know a lot of things you say we don't test view controllers because we should move all the logic to the view model but it's like the cost for testing view controller is very low and you cannot move all the logic there there's still some stuff in there that you may you know break by mistake it happens all the time i think there's some things that the view model shouldn't be responsible for as well and just moving it there to make it testable is um you're kind of like which like what bit what what problem do you want do you know what i mean like which is the challenge that you want um so if the team is on board okay we will test the controllers as much as possible when the cost benefit is is going to help us right it's going to be acceptable yeah yeah it's a better i don't know standard i think than just giving up in the whole series like doesn't matter the complexity of the view controller we don't do it no we do it when it makes sense to do it and we don't do it when it doesn't make sense yeah yeah and i think that's like talking about view controllers we do that with tap gestures and things like that right they're incredibly hard to test in code so we kind of either try and use a button or something to to give us that testability but if for whatever reason we're just not able to do that if the requirements need us to use that component you kind of just make the decision as a team that like we can't reliably test that gesture in code during a unit test so we test what we can around it or if we have a very complex application that uses many gestures like it's completely just your base what you're gonna do is you create a subclass of the the gestures to enable testability because then you have a big benefit because your application is fully dependent on gestures it's like a game you know that everything is gesture based then you need to have the testability for most applications they have one view that you want to have a double tap gesture you're not going to bring sub classes and create all this complexity for one tiny double tap gesture that doesn't even change anything yeah you know it's not even like a a feature that is critical in the application yeah that makes sense that goes back to what you're saying about cost cost versus benefit right if it's not if it's not the the you know if it's not the core thing that that is ultimately the most important thing in the app like think about the benefit and the cost yeah and there are solutions it's just that you can test anything anything can be tested either manually ui tests you can create some mocks so it's just a matter of understanding what you should test in your specific case right everything can be tested but there's a cost are you willing to pay the cost if the answer is no you might test manually right because you may decide that okay we're not gonna test this but we are making as a team an informed decision that we will always test this manually before shipping the app on all versions it's going to be very expensive as well to run everything manually on all different ios versions you support on all different devices you support yeah and then if that's the case and you have a critical feature that you cannot test easily in code but it costs a lot to do manually maybe it's a case that you will use some more expensive solutions here because you need it it's it has high benefit in your in your application in your project otherwise move all the logic away from the infrastructure test all the logic separate it from the framework create a smallest adapter you can humble object and you don't test this small object here this tiny adapter that communicates with a real framework this can be a struct a protocol implementation it can be i don't know a closure implementation right because this could be a closure it could be a struct could be a class any kind of abstraction and you have an implementation of that and i guess you can reduce the now i'm thinking about it you can reduce the um the opportunity for for those uh for those things to for for bugs to develop by testing early with um like alphas and betas of ios you know when we're jumping from 14 to 15 and things like that testing early with the tools that were given that you know access to the early the early versions you might pick up those differences between like the api has changed in the vision framework for example um and rather than find out on the day you kind of like yeah hit release you go okay well like we've got three months to fix it or something like that and that can reduce that that that risk yeah when xcode13 was out i was running some test suites and i got three failing tests and i'm like oh my god what happened remember mike yeah absolutely and we found out was the behavior on the differable data source that changed the method on ios 15 does something different so we had to call another method if i didn't have the test we would have a bug yeah you helped us find and you can run it with any beta version of xcode and find that bug so it's important to have regression testing yeah mandatory i think yeah that's what i was thinking regression testing you know i agree 100 yeah and using the real framework yeah when the cost is low and if it's too slow you move it to a separate suite that you run for those regression testing suites they run less often much less often maybe it's not even part of ci it's part of a release pipeline yeah for example all right so we found out that we can test revision by creating the right interface and if you expect a c direct in the interface we'll set us a direct and then you can test that c direct is within the boundaries you expect yeah and with photo keys and those very complex frameworks and they're not very complex because they're bad they're very complex because they're doing hard things we want to load thousands of images of very high resolution images from the photo album into your application so there's permission involved there's threading involved there is cropping involved no downsizing involved there's so many things that i think they did an amazing job creating those apis yeah they're really performant they support like asynchronous loading and canceling at the same time but it's a complex framework because it does complex things yeah i mean it's called photo kit right but you can load videos you can load different media sources it's like it does so much um in a it's very and it's what i like is that requesting videos requesting images different different media types the apis are very familiar it's not like a completely alien api getting a video as it is to getting an image that so it's definitely like it's definitely well thought out how it works it's just yeah um the testing is just what confused can concern me a little bit but this is this is that has kind of answered a bunch of questions i had which is really useful and if it's very critical to your application and let's say sometimes there's a bug in photo kit on ios 14.3 and you found out and there's a workaround they need to create around it you know with some swizzling let's say and you don't want to have that code coding there so you have you write a test proving that the bug exists and it was fixed and every time there's a new ios version or you drop support for ios 14.3 as well where the bug was you run the test again everything is running on our ios versions you can remove your test and yeah and this whistling right it was just a temporary thing and you have a testing place even if it's a slow ui test just to prove in the regression testing that you didn't break anything and you leave the test there because you found a bug that maybe is going to come back because there are bugs on apple code as well their bugs on third party code there's bug their bugs don't combine async await everywhere it's the nature of of software isn't it it's not um it's not perfect i mean if it was we would be out of out of jobs because they you know it it's constantly iterated on and built upon and um it's just how it is yeah that's it when the cost is low like i think it's pretty low to test vision because you can just call that the apis and expect something yeah but it might be a bit slow then you move it to another suite now photo kit and this is more complex apis you have to mock a bunch of things maybe not that beneficial for most applications as long as you move all the logic away from the infrastructure and you end up with this and then you don't need to test for lines of code yeah yeah because like you say this behavior could could be covered with a ui test which is um you know that actually would provide more value because um i can set the simulator up in a way for for for the test that i that i want to i want to reset i want to check these things because there's certain things in code i'm not able to test properly the if for example the in the scene delegate there's we cast the the image url that we get we cast that to an image and that obviously returns an optional but i don't want to work with optionals so i have if you scroll down um there's um a little bit further sorry keep going here so you can see here from line 41 this make image grid ui um when when you select an image and we pass it for analysis we get the url for the image and we just map that to to an image now obviously if that operation fails i just show an alert to say that that couldn't happen but i can't really test that properly um i can't test that if there's not um if you don't have permissions how that performs but with the ui test like i can i can i can kind of do all that that's um that's pretty straightforward um right so there's there's more benefit there's more benefit in that case of using a ui test to cover that behavior than trying to kind of stub and mock everything out to force that behavior to happen when i could just give it a bad image and and make it break in a ui test yeah you can also move this into a function and just call that function right [Music] it's another way of doing it which is breaking a little bit the the guidelines of don't expose private methods but when there is a benefit you do it so all the when things are guidelines it's like this don't do this usually yeah don't do this don't resort to this as your first approach but if it's the only way then you do it and there's a benefit then you do it for example you could extract it into a function i think that's the one thing or one of the things i'm still i find myself kind of learning every day is is that whole concept of like these are the best practices until they're not and then it's like okay and it's like understanding like when to stick to this kind of when when to like this is what we need to do and follow these kind of like principles to deliver it and when it's okay to say like this is going to be a blocker or the cost and the benefit of kind of like sidestepping this one a little bit is um absolutely it's worthwhile that's like what what good dreams what good teams do right like they adapt all the time like they don't see there's a static view of you know the code base of just one stamp so just no it evolves all the time and you need to be aware of it yeah like you have a guidelines and some people are very dogmatic and they follow the guidelines all the time or they just give up on the guidelines very easily yeah you want to be somewhere around here right yeah in the middle you end up with like analysis paralysis where you think i i don't know how to satisfy all of these principles like you haven't even written any code yet and you're already like i don't know how i'm supposed to do this but it's because like they're a guide right and like they should steer you but you should you should kind of use them as a way to inform the decisions you make not as like a hard rule that just shouldn't be violated on every single occasion because you just won't have any velocity in what you're doing that's it absolutely and what i like to do is like okay i cannot find a solution to this so i will resort to this less ideal solution but when i have a free time i'm gonna find a final way yeah and sometimes i find a way of doing it but i still think okay i can do it but it's still too costly and i think what i did was the best solution the simpler one that was less ideal yeah because the one that actually does what they want is too expensive yeah until i find a better solution so we do this less ideal solution until we find a better one because maybe it's just that we don't have the information we just someone else might have this info and that's why we have a community we go there and say i don't know how to do this and someone comes and oh i had this problem two years ago and this is how i fix it yeah that's it yeah so here uh what i was saying is let's say something like image or a url will you extract the book image does it return anything no no it doesn't return anything all right so that path something like this now you can call this method right and test it and here you just inject that method if you wanted to rectify it that's really good actually because now of course in test i can pass it an invalid url and i can pass it one um that exists it's orchid um that's it right so if oh i cannot get in testing i cannot get to that block because i need to be able to create these ph assets with fake urls that don't work well well less ideal solution if you cannot set up the scenario to execute a code in a cost beneficial way this is exactly what we talk about in the academy broke with the scene delegate testing the scene delegate it's it it's very difficult to create a scene delegate i'm kind of all of the things it requires so you can you can trigger that what is it did the did launching method i can't remember the name of the method so instead to test if the window is configured yeah see it's very difficult to set that method up in test with all its dependencies so instead yeah because we cannot create a ui scene exactly yeah so i have the configure window method below which i do call in test and that checks that the window was created and set up as i expect um so that makes sense i see how that that principle can be applied again yeah again this should not be the first solution yeah maybe solution 10 if you tried a bunch of things you didn't find a solution then you should have options right okay this is the best thing that could happen i cannot do it what is the next best thing the next bad thing and then you know yeah you decide and if none on the list satisfies and you think oh the cost is going to be too high you may even decide not to test it but you will test as much as you can and move all the code that you cannot test into a tiny adapter and then you don't need to test that adapter all right there you go make sense now we can test this logic which is not ideal again but at least it's tested you don't need to write a ui test they will press photo x for permission press another three things and yeah and that yeah because i guess what's nice about that as well is we're testing this behavior we're not testing the behavior that's like either side of that operation happening as well we can be very specific about what we're testing here which is faster and it means there's less dependencies here um should something else change you know the the tests aren't if something goes wrong with this then we kind of know here it doesn't necessarily mean it went wrong further down the chain and it broke a bunch of tests um absolutely you can also extract this into a class right this function can be into a class or it can be a free function it can be i don't know a static function it can be a function or a method on a struct and you have an instance of this object and you pass the method here as well you can have different implementations that does different things like it could be a router right get an image and i should either show something or show a neighbor like a coordinator people like coordinators yeah so extracting this logic into another object makes it testable because it don't depend on all the infrastructure state yeah and you can change the behavior as well right it's much easier to change the behavior if you had a a strategy or something that could then decide which one of those if you had different components for different reasons um yeah i see it and again you're only dealing with small components that are only responsible for their operations and everything's kind of like responsible for its own thing which again is more testable is um makes sense makes a lot of sense absolutely could create here a analysis flow maybe this push analysis is all in here yeah that's really interesting actually because there's different types of analysis so you you could um quite easily extend that that with an enum with an associated value that had like attention based or object-based and the component can be just be responsible for saying like someone requested object-based analysis on this image um yeah make analysis ui now you have an object i just parsed the knife there three the nav oh yeah fantastic object can extract it into another file and you have this behavior tested and here is just a composition right yeah and it's just reducing the amount that's not tested like just because you can't test just because one thing is difficult to test everything that happens around it shouldn't it's not an excuse to to not test that or you shouldn't avoid testing that it makes sense how easy it is to test this now yeah no mocks or anything awesome and i think that's it do you have any other questions no as always super helpful super useful um i'm probably gonna spend my evening now rewriting shouting at my partner asking if she can just look at this camera so i can take a picture of her face um but yeah no cool brilliant really helpful thank you again let us know in the community the results as well yes yeah findings i will awesome yeah i will awesome thanks a lot thanks a lot guys thank you so much gordon thank you gordon take care take care bye [Music]

Channel: Essential Developer

Views: 684

Keywords: ios, swift, professionalism, ios development, ios engineering, ios app development, xcode, tdd, modular design, architecture, advanced ios development, iphone, ipad, advanced swift, clean code, unit testing, testing, xctest, framework, solid principles, uikit, ios testing, advanced ios, advanced testing, mvc, design patterns, how to, how to become, coaching, senior ios developer, iOS lead essentials, podcast, coredata, mvvm, mvp, viper, clean iOS architecture, iOS architecture

Id: WKvOmzKVoYA

Length: 71min 52sec (4312 seconds)

Published: Thu Dec 02 2021