iOS XCUITest Network Layer Mocking

as eugene mention my name is igor and this is boris both of us actually employed at tinder currently I am an engineering manager responsible for release management for the entire tinder stack including the mobile iOS and Android applications previously I lived in Silicon Valley in Bay Area worked in companies such as Barnes & Noble and Expedia for more than a decade my background I started as a regular QA manual tester whatever you used to call Quality Assurance engineer and in 2011 I shifted to test automation I worked in a web mobile through entire career majority mobile experience and transition to management back in 2012 since mobile became my expertise I actually started teaching in that area in 2013 and formed the company called code Fitness we actually teach boot camps on the test automation majority I'm actually in Silicon Valley because the audience for that was a bigger back there and now the high tech growing in here in Los Angeles and Orange County which is great I enjoy the trend and the thing is going to instill going wow and Boris Boris actually is engineer he is the USC graduate master's degree in computer science and he is currently working at tinder on identity team so pretty much the whole authentication process from this guy for the is product today we're actually going to do a little bit different that you guys have seen before on the meetups we try to do actually more hands-on kind of nations to show you what things are in reality rather than gives you like a lot of you know slides and speeches so I'll slide that is very short and you're gonna see more code so this is well I would say technicals need up let's get started so before we jump to the first topic and we actually shuffle that you've seen the agenda for the day with shuffle topics a little bit because they depend on each other and complement each other so we're actually going to present with the marking network services on network layer on is applications as a first topic and analytics we're going to do after Q&A session so but before we're going to jump to the first topic a lot to ask you guys a question because this is the interactive meetup right I'd like to understand who you are what the audience please raise your hand if you currently had mobile testers testing mobile applications manually in automated ways okay and who is doing the web interesting interesting I don't root people who actually doing well how many people do mobile web has the majority of their work okay so if you notice for the past few years the transformation goes towards the mobile now is right look at yourself every day everything that we do we use the phone right booking hotel right scheduling any appointments even for haircuts everything purchasing goods Amazon eBay pretty much your whole life on a mobile device therefore they say that in a few years moving on forward the entire high-tech an inclusive more towards mobile web is not gonna go anywhere don't freak out people because the hello people sitting here in the web it's gonna go towards mobile web right almost responsive or right now emerging platforms for such as react native right you can build cross-platform applications which is very similar to mobile so the the web is not going to go anywhere which is gonna transform so what we're going to show you today it's gonna give all is native solutions this is built for iPhone or any air like iPad Apple TV or even the air or sex for their max but we're gonna give you examples today specifically for iPhone because the right phone market is very large it's very big right so a lot of people using iPhones nowadays is it voice 50/50 right now with android market share feel like in u.s. probably yeah we're going towards that song before only rich people could afford iPhones right nowadays is actually equalizing slowly so today we're going to talk to so talk about topics which is very sensitive in test automation world and why I say sensitive because now this is not my words this is the coming from the what we're going to show that this is not the solution we baked in the tinder this is the solutions that actually used across large organizations such as Google LinkedIn there is amazing articles and blogs written about this we're going to show you the best practices around those topics and why explain why they are important nowadays especially if you're thinking to trans into test automation world everything I'm going to show today is not only to the mobile from all the folks who sits in here from the web world it's applicable there as well and it's important so all the aspects you can actually inherit into your web world if necessary let's get started I've ever heard about box and stops don't matter web people and mobile people anyway first manager Bob Watson stops fakes there that's awesome great so let's talk about this so let me try to explain in a simple language using my hand gestures what is all about ready there's I'm just I'm assuming there's a lot of people who never heard about mock stubs or never work with the web services before so we're gonna basically try to explain into plain English so think about this when you're using any application right let's say Facebook you go talk to your friends on a facebook chat or your called messenger right or you are putting a post on Facebook your application talking to the back end right use a Facebook Graph API very famous one so obviously what happens is that when we do that as a tester right we're hitting the entire staff the whole layers right away from the clicking or tapping on the bottom down to the databases on the face servers right the problem was this approach that we are going to talk about automation is obviously I'm going to describe that in details is we don't know what happens in the whole staff right if something goes wrong and that's what we are tested we try to discover defect bugs it's very hard to understand which layer the failure is right if we see the Box we like okay we found the bug go figure it out right however in automation world right think about this when we hit the holes tab we run to a lot of different issues which we're actually going to describe this presentation I'm going to show you why and test automation is important to cut off from the backend think about this when you call yourself a client tester a functional tester you try to mimic user behaviors of the application right you try to isolate the application in a hermetic environment right so therefore you are not the backend tester you don't test the back-end services you definitely not the testing like some third-party dependencies like for instance if your application use a Facebook rule again definitely not so what we're trying to do to try to eliminate a lot of unnecessary thing that you do need to even touch and think about you are testing a functionality of a mobile application in a hermetic world right in itself so therefore you eliminating a lot of issues that can be pertain with that such as a/b tests right for us to talk about think about this there's right now at least ten verses of the same screen right but there is a ten versions on the back end as well and you don't know it's randomly drawn like a 10% 20% depends on how the server configured you have no control over this you have no control on the user data you know what sort of going to respond to you you definitely have no control on the server errors right if you need to test how the client will behave in negative scenarios all of your testers here I'm assuming you do a lot of negative tests how would you make the client to fail something right you cannot send bacon please throw me like five hundred it's not gonna happen but you need to simulate this to see how the client gradually will fail with a proper error think about all these things any other yeah what all the users can do when you do like life with test application gets live services what are the issues you can run into anybody can think about on your day to day testing raise your hand we can help you shoot any each other's life testing absolutely very important one we cannot control this server response based on the law you are the tests are testing a production there's millions of user testing exactly your application at the same time if you're testing a midnight hopefully traffic with the lower unless the application is global right we don't know so it's very hard to understand how the application behavior was networks timing with completely eliminate latency zero because we don't keep the lie back and right you basically it's called hermetic testing you can adhere that word a lot of different presentation and that's what it does it actually removed latency anything else don't get us wrong back-end testing is extremely important as well we just say that it should be completely dependent busy isolated right like somebody else could test back and completely isolated or even you can test it with different technologies with different CI probably right with different programming language doesn't really matter so both things are important but here right now we talk about client testing so we don't care about in if there are some errors in the back exactly the spy you remember Michael cones the test period remember that is famous one if you test your probably like should not like a Bible right or something so there is different layers but usually Michael cone shows three layers shows the tip of pyramid which is you have functional test integration test in the middle and unit test but this is 2012 right this is like I'm sorry Michael Khan pyramid a little bit all dated if you've seen the modern pyramids they have much more layers of you said database right it could be like very bottom of it Union test integration test is crazy when you say word integration test people get like you're good skin like what it means it could mean anything some people think about integration test is the rest API test but some people think what integration test is some sort of micro services layer test to how the different micro services talk to each other Boris thing about integration tests how different modules on is are working together but many people think it's a union test so there's a lot of confusion what the actual testing means nowadays right it's crazy so that pyramid has different layers what Boris tried to say that we are talking about client think about the chief of pyramid of Michael Kahn we're talking about functionality - everything else and it's worth mention I'm gonna repeat these words it doesn't mean we don't have to test them we must test them but in isolation so we're know exactly what's failing and we can respond to this quickly and help developers to fix right issues right away and so I spent hours - debugging like I don't know what's going on either the rest API layer fail the database wasn't properly populated my great services went offline or was under the new deployment we have no clue this DAC Dastak could be endless that's why we have testers us testing different layers there are people who testing the UI layer the people who testing the you know intermediate layer love rest API layer there are people with testing microservices for entire career and there are people who testing all the databases and which is fine right and that gives us opportunity to have jobs and the jobs become much smarter and this is good right so today today we're going to focus on optimation and testing that functional layer of the market now this is this is what the person said butcher named green ravine process is absolutely true there are startups for the matter of fact a method person who was original developer of selenium api's for ruby the person who actually developed a whole selenium api Scooby original guide and he is right now city owned the company asked him why are you using what you've been reading for many years no said why I have time for this we have to deliver fast we have time for testing I work in the startup thing changes so rapidly I have no testicles no time for testing at all the guy who actually invented this doesn't use his true however it changes it changes and it's all depends on the company in addition it also depends on the management if the merit I mean he is nodding the head you guys didn't see cuz I'm I see all of you if the management believe in quality right and it changes I've seen a lot of companies before who were completely relaxed about what it is that I don't care about poetry to move fast but then they hit at that end right the user reviews get worse customers not happy technical death loading application sucks that's the reality right so what happened in tinder thanks God right I went through three layers of management three different like you know CEOs and and CEOs until we actually get to this it takes time right and it also depends on us to teach our leadership teams to tell them what they need to do and what they not right it's also not we're not supposed to sit silently and hoping they're gonna come to us and ask to do this we have to educate them we have to educate our leadership teams and tell me what's good or bad and why and there's always ROI and you have to explain how many people we need and have to come to this wisely again today we're going to focus on IES and we're gonna pour show you things that work very well and we're gonna show you only two topics there's more than two topics right we'll get a live for more meetups later right so we can meet all of you again so again let's come back to the topic of network layer marking let's go get a cop yes so first test set up think about that stood out was going to talk about this if you need your optominute test right to actually mimic some use case scenario how the mimic was that was live services right very hard right if I want certain outcome if I want my test always to be talking to Boris it's hinder how can I do this right it's impossible I will go to a lie of data going to get random people maybe I'm gonna get some sort of you know some AP test popping up on the middle of this how I'm gonna orchestrate it was the proper word my test to actually give you what I want us again we are testing the granny style I don't care about the back of the charts I want to see how the client and the random things on the front end right so that's set up a test orchestration is a very important aspect for the network's timing because we can tell our face were to give us back my test always been returned the same data all the time so my expected result will be always the same and we eliminate what's the most important thing flakiness right because this always has to be the same data next one right so again number one we want to control our data and we want to be able to set up our data for our tests so continues back-end deployments how many people if you know raisin have how many people know how many deployments actually done birthday you're competent back anybody else in this room good how many men have it three 2.4 miles per day seamless you have no idea the room happened they just forget to tell you that can ruin your desk if you have automated tasks written against five services somebody to deploy something doing it back then you're done it's not basically you basically don't know what's happening right the whole day that change you do you name it scheme of you Jason can change blow up everything if you want depend on it I don't know I'm just give you example we don't know so that's that's the problem because data nowadays and back-end even services and logic changes few times a day when I started courier was few times at work so we have to respond to this so now we are completely eliminating this because again our data is preset it's fake right which is since tell our test what kind of data it should return all the time next one network latency the biggest issue right now adays tasks have to be very fast in order to qualify for CI system you probably heard a lot of times do you want automated test its run in 15 minutes like you kidding me might want test runs for three minutes whatever five tests yes and that's the number one there is amazing article written 2016 by LinkedIn company 3x3 3x3 really amazing article it talks about speed call speed of wide right or need for speed I think so the reality of the test does not only have to be stable to be fast how you expect functional test to be fast right and that's why we are talking about stabbing because stabbing remove latency completely wouldn't have wait like you know 500 millisecond network response anymore Oh thousand sometimes 3,000 rate depends on the how the load on the server we're completely made that part and we have one separate test that tests how your app behaves if there is evidence right if there is some error as a result of subway dancing we can check it separately with one test we don't need to experiment on our like hundreds tests yep this is the probably very important for negative test cases how would you stimulate the application the we talked about is the back end errors right there's a lot of them right there's three there's custom back and errors very hard I mean you have to you have to let go out of your skin and somehow tell your server please fail like I mean it's kind of dumb it's like reverse engineering it's much easier to do with fakes and stops marks all the synonym words food cooking a call today because we can tell right away please return you know 404 that's it and I think it goes soon there are it's like me right there is no errors oh users god scenarios why do we need to you Emily similarly don't errors that's not true on the scale we have millions of users you don't even imagine how big percentages of users who got some failures errors different errors yes so we need to we need to make sure that we know how our app behaves in case of error that's what we want to test here right will simulate the error on the back end and will test how our app behaves direct should be good they react should be in such a way that users like that does not become angry on us right and so as a force angry is a good word what people would say oh my god cannot log into the tinder for school there's no more day for me to believe that's it I'm done right my personal life ruined so yeah a lot of people get frustrated with that and we get a lot of this but again how back-end testing is different story it's a back-end problem right there as their owners but how the application give you a right error and there is a lot of different flows in application right authentication has at least couple thousand of flows that actually brings different errors and this is very important and it's impossible to do it with the live back-end services you have to mock them great let's move the next slide so now we're going to talk about how the mock environment look what so think about this what we do we inject into our application code right and again with native test automation with everything going to show today is going to be X UI test x-ray test is nothing more than just simple library is part of X call when you download Xcode is there ready zero installation it's part of the Xcode and it's part of the exit test which is unit test framework used by all the developers so again everything comes out with the box we're not going to talk about X GUI test as a framework or library or about exit tests we're all gonna say that the whole code lives in the same code base as the application set so basically developers and test automation engineers work on the same repository same for the same github or get laughs whatever repo right and that means that every time you commit anything it goes to the same code review sample request process as a developer so pretty much you're a developer right but the good thing about this we can inject a very lightweight web server it's literally very lightweight and there's a more than one alternative we're going to show you one example today but there's more than one and what it does basically that server intercepts all the outgoing requests and instead of going to the back end is going to return the responses we got a predetermined ahead of time but basically let's say that always logging on the Boris and the Boris is only one man all the time so we tell them so basically your server is not going to go anywhere outside is actually will be inside your application it's called mock server and very common in iOS and Android world there's a ulti HTTP mock web server on Android out of the box very common what every developer knows this and there's few exists we're going to talk about ambassador today but there's a xcb GUI best analyst very long word there's more than one exists on IES and it's up to you which one you choose we can write your own even rival servers it's very light exactly it's basically what it does can it click through the view yeah so I'm gonna show you boys can actually talk more about it so we need to percept the data is incremented and also it's important that every test should be able to present its own data right one test we want to login with Boris and have a much bigger and the second test wants to login with somebody else and the third test wants to try to log in but the server should return error that there is no such account so the flu idea is every test is able to configure its own setup manual testers who do write just write test cases a regular basis call precondition write exactly the same thing now to make the tests each tests will have a precondition the form says hey please create me a response that always will look like this before even that starts so this way you are preparing to take data to the return full orchestration right let's walk through our other ones so as we talked about already full test arbitration every test can continue its own setup we are not dependent on the web server updates we configure our own responses they're fast and reliable if we need to mock the wait at the mall the majority of the web servers will allow it to like specify that I want like one point five seconds on this particular response we are able to test analytics who have a separate topic today for analytics we'll talk about that later and we can cover negative escapes okay force this observer to return some parents a 404 there is no such account or whatever if we need let's go to the demo yay so I'm on demo wake up everyone the fun part can you change the presentation mode oh yeah oh yeah like I mentioned we're going to show code and good thing about good code is very well written since we the boats like playing English I have to do like a super is developer and senior levels under standards here at all okay everybody see that from the classroom awesome okay very quick recap of the app what I was showing here this is our application under test specifically for this purpose so this is Charles Charles it just intercepts everything that comes out of client and you can even see all the requests and all the responses and some tools even allow it to replace some of them or put breakpoints but the basic functionality is basically show me what action is happening under is there right the app somehow communicates with the server with the back so I would like to know what exactly are there HTTP responses and HTTP requests and responses and by the way there are multiple other protocols right WebSockets you name it but today we are will focus on the HTTP unrest okay so basically the app is very similar as simple it allows us to see the list of restaurants and you can go to some particular restaurant you can make a reservation here you can detect the table on the simulator you will be able to put the table number but on the real device it will open the camera and we're looking forward QR code and then you are able to do some order right you have the mail and you can order it right here okay so as you can see right now I just run this app so this is a real data and I chose the loading later of somebody oak can also okay with the Facebook so those are restaurants that are stored where commander server what is restaurants come from sir obviously right any Co an app will request the data the actual data from the back the backend knows how to get it from it from the database or probably from the third party API it doesn't really matter today somehow server returns these restaurants and we can interact with them Charles will show us how to house happening right so we have this IP address and port from their address of server right and we can see that there was one HTTP request it was a get request to endpoint places and the response what the Jason what else what is Jason good so basically ministration contains that necessary data right this is an array of restaurants but you see that replicates the whole drop of the app is to render the JSON we give from the server right you can see the router was for a bunch of restaurants we can see it here now our idea is what to cut off the reel back right to set up the white weight book server inside the app and whenever Apple try to send HTTP request to our back-end this wide wait server move intercept it and we'll pre setup it hey right now I don't need 10 restaurants return me to restaurants this is the name of the third this is the name of the second this is the address of the tours this is an address of the second that's what is the manual inside the first restaurant and now the test knows exactly the data that should be returned and in those exactly what to test now let me give you example if you look at the application right here the border shows that we don't know which restaurant is server gonna return you can change anytime right if anybody can update that's that that's the problem was the test especially the test automation we want to expect this sudden like you know lay out certain data format certain way because we are testing at the back end with testing on client work we assume that is correct and returns correct Jason that's our assumption so what were the words gonna show you is like how completely cut off from back end and tell it exactly which restaurant to return on the fly was changing all these fields if you want to okay I prepared one test here this is reservation test so basically we go to the restaurant and we open the reservation menu and we provide some date and phone number and number of people and try to reserve first thing we need to understand here if I'll try to do it right now with reservation let's provide my phone and let's provide number of people everybody I'll try to book it first of all app will ask us to login and Facebook organ is something don't trigger on iOS and then mobile in general right Facebook runs a lot of different tasks and it's always who could try to automate Facebook audience in this in any way when you open the webview and provide email and password right there this is a nightmare really it just doesn't work so we want to somehow to provide like to fool our app you are all in already and you cannot do it with the real server but with mock server you can mock server will not cheat well it will not I'll try to authenticate it it will believe that you have the permission to ask this request okay so let's try to run this test I will suspect of its model you probably know what is this same thing from mobile who knows it from birth okay so you see only two restaurants because we talk to do that and now we try to do it without number of people [Music] okay and we fail somehow Murphy's Law Yammer photo of course during the presentation always fails you try ten times before I don't matter so it passed here what can we do here first of all people did yeah it's my job right and right here Chow's right now does not solve a sloka post request but you can see that the last request to the our actual IP address was like two minutes ago right 1936 and it's already in 1930 so it was like two minutes ago we did not hit the real back-end when we were running this test doesn't make sense any questions this for now or we'd have a Q&A right do you want to basically maybe how many people interested to see how its implemented quickly yeah okay so as you mentioned there are a lot of different alternatives here basically you need to keep one simple idea you need to intercept the network and be able to replace it with your own stuff and like droplet or tunnel stuff which is not actually the server at all it intercepts the ural session of iOS and replaces data there but ambasador is more readable the embassy actually yeah embassy ambassador only get coffee can find it yes believe it implementation is pretty simple we have this router of ambassador ambassador is open-source project by the way so you can like download it play around with it for kid if you need your own functionality and they have instructions how to embed and you are existing the application yeah on their hip hop step by step it's pretty simple just cocoapod depends yeah on our consulting actually open a bunch of issues right there because we we needed some functionality it does not exist there so basically we have this stopped Jason's right here and we construct this Jason's on the fly and you just override the endpoints right you say hey for this and point places please return this response if you need there are a bunch of configurable parameters there you can say return it after one second or whatever but basically it is very simple you just Merrill it construct the jason on the 40 and then so you parameter isin you can actually tell the json schema what actually values you want to return let someone say the test setup before tester precondition can say always return restaurant name like mcdonald's and it will inject it on Jason return so it's very very life so you construct your jason responsible way yeah and there is one caveat before I forgot there's one problem with this only one if the backend changes the schema how actually JSON format looks it will fail your client because client already using this team however if you talk to your back-end data automation engineer or the backend engineers or they tell you they are anticipating to change it or you will know hard way when are you test failing and then ideally back-end should not change very long right because all the all the previous clients will die basically so but sometimes it happens and there are one there is one tweak actually that the real app right the real app that is under your test doesn't know that you are intercepting or whatever so you need somehow to specify what IP address to point your or all your requests right so you need to set up whenever you run your test you need to send a signal to your app that hey right now this is not a user this is a test so point all your requests not to their whatever IP addresses for your back-end but for for example localhost for your local webserver I think we have still one more topic sorry I want to do Q&A here how's the queue name for this topic for studying feel free to ask any question stupid persons almost [Music] what does it self-service a master proxy I can use it for efficacy I know that is ambassadorial occipital it built and sweet for the IES but there are times oh there's a web there's a lot of web proxies built for web of summation and you can i've actually for web world a lot of people actually write just the separate mock server and they just like maintenance you have one he comes for free nothing comes for free they just so much yeah a lot of people what yeah what's the name Praveen thank you he mentioned amazing thing and maintenance is number one pain in testing if the thing is automation Jan you're gonna write test you're wrong eighty percent of the time you will maintain them and twenty percent I'm gonna write new ones that's the reality however when you think about this right implementation that's why I talk about orchestration that is important people are looking for the easy way out for silver bullet they use a process server which is basically intercept live events and always recorded returns the reported data but there is no cell so you don't control the data you get kind of cache data all the time same thing but the problem with this you have no orchestration which actually paid the price for this so what's this solution it's a little bit harder implementation the beginning but then you're enjoying the full orchestration you can tell the test what to do was the return one can add a work with so attempts it has become very configurable and like I said on the web world there's the solutions like this existed then a messenger helps with the formatting all that Jason or wait manually ideally you store it in the file and then you read from the file a buzzer has nothing to do with this ambassador just allows you to say this is endpoint this is what I need to return and how intercept or this is your particular thing and this is not ideal ideally we'll start to file and like shares the data so you can configure from every test so there are many improvements this is just a demo yeah but it's nothing but yeah it's up to you how you are castrate your Jason's different file the face absolutely absolutely you need to first of all intercept this data with Charles or whatever you understand what is returned to your app then you need to walk every single endpoint that is somehow touched with your tests so you can no back-end that's the ideal you come to this a key miss Wagner right you're gonna see all the documentation that's the only way that's why there is no such thing that I am manual tester I'm automation engineer is a back-end it's a one team its share knowledge guys the in the testing world everybody has to be on the same you know both if not it's a failure everybody doing and help each other that's not necessarily there's a lot of shared data absolutely because just like in I mean the same response can be reused across multiple tasks but in terms of gender we have about two hundred functional tests and the reason we don't grow them only cover p1 test cases just kind of the most important one the rest we'll try to do differently we have manual testers for instance right but the the most important thing about this approach it allows you to put your tests in CI so the wrong every merge so speed right that's the most important thing scalability and empower speak today by the way yeah a laboratory would have set of tools to achieve the speed we'll talk about this capability here how it contributes to speeds and the most important thing your stability our failure rate is less than one percent on functional tests it's remarkable and we are in the same boat as LinkedIn all the LinkedIn has like thousands of Engineers but we are less than 1% because of this sorry once our one little follow-up question for this money I know how many apps do you have but each app will have this car you only have one purpose of Torah teach people so it's isolated it's chromatic it's like at least think about them as a unit test unit test also going to get mocked he writes in your test every day it also gets marked in a different way because the unit tests when we test the code here you test the Moloch integration part right but each repo in our case is it has a wonderful support for us have put in there and has all the mundane Android has completely different solution and separate repo last question um I was just wondering what's the application like you know do you want to do you guys use attender test while mock service and deployed or you just test the build quality and do the more function test involving the data there because you know once it goes into the real world that thing that you have taken out is going to come and those procedures or you know data synchronization is manager because it asked a very good question yeah from what Ron is actually asking is in terms of this is not the solution is gonna replace manual testing order by end-to-end testing we have to understand this this is the what we call gatekeeper think about this every developer makes pool requests one per day at least there's multiple like at least 20 turret polarize get merged daily at least on a DC application think about how many times we can fail this is the first guard we have multiple gates right but this is the first enter gate of the unit test swiftly and write code static analysis and this after that obviously there should be back and if the I present hours all functionality from API layer because we don't test live services this and 100% manual testing manual testing is not going to go anywhere you have to do this but this one can catch inside the hair like hey man you broke something 99% this test actually don't break because of the bug the bring the cost of the change of UI which is okay that's the way it's supposed to do but they catch really good box and I mean you have a store here's a story that after this presentation words can tell you one-on-one like what kind of bugs this test case it even possible to catch manual alright thank you very much for equal force

Channel: TestPro

Views: 932

Keywords: testing, automation, it job, it career, information technology, qa engineer, it courses online, it course, python, java automation testing tutorial, test pro, qa jobs, qa automation, testing courses, testing automation, software testing, selenium tutorial, testing for beginners, selenium for beginners, qa тестировщик, qa обучение, qa тестирование для начинающих, работа в it, тестирвщик по, ios, iostesting, xcui

Id: 5ReuZmaSrFM

Length: 45min 19sec (2719 seconds)

Published: Thu Feb 06 2020