Appium is Dead by Igor Dorovskikh (Engineering Manager at Tinder)

please welcome team from tinder swipe right I guess thank you so much for coming tonight on Wednesday and we do this presentation many times before but like negative things about a deal we basically didn't help you out to make decision both of these frameworks are phenomenal and we basically here to help you out to make a hard decision your company just FYI and previous presentation from as I write was amazing thank you so much for preparing we're not even close to you guys all right let's start with a formal presentation so today what I'm gonna go speak with Boris Boris is is developer at tinder they wanted me to move any camera but so alright so Boris is IES developer tinder he is amazing guy amazing developer and he's also in extreme sports he likes like snowboarding kite surfing you name it so he is pretty cool not like me and my name is Igor I've been in this industry for 11 years now I work at companies that burns in noble and if it has anybody used note before yeah that's my baby I actually delivered the first one in Shenzhen China in 2011 and the work at the Expedia has a first in an engineering manager to build a test automation from mobile and now I have a tinder as an engineering manager and charge of mobile releases some more like an DevOps dance automation they have automation but in mobile context not the backend so another thing to do is Boris we actually also founded company 2013 called code Fitness we teach people how to code for death automation engineers but more like on mobile so we are focusing only on iOS and Android mobile so we also do that as a you know fun part of our lives let's move on one more awesome so the industry testing industry in my era shifted so fast when I started in 2008 it's nothing like it is today right now who remembered the j2me phones remember those like Nokia and 75 and 95 340 that means you old welcome to my era so yes and today everything is so different whatever you were used to watching the movies as a kid it's actually true it's happening today so in QA a poet assurance world things also shift it so quickly so you have to be very agile you have to be up-to-date and understand what's going on so before we're going to present the difference between the two amazing platforms we want to talk about where we at right now what's going on in our industry in test automation in particular so today mobile develops process requires fast and reliable feedback so we are in era of speed everything has to go fast right and very stable that means that next one more testing activities have moved from post merger facts right when the code to merge build and develop and deployed to the pre merge in the development process right and you feel it it's a painful process but it's happening as we speak and that shift in my opinion started someone on 2012-13 so it's not like overnight we'll just continue moving faster and we have to respond to that it's very important let's move on so shift left how many people here in the room heard about the term shift left good that means you're gonna keep the job yeah so she left is actually means that the whole process of testing moving from right hand side when we actually did build is delivered to you already that the process the parts already been introduced to the development process before bugs are introduced before the code was merged before they built happen this local shift lab in a simple term test earlier that's what the shiplap means for that reason we need a certain criteria for this task we're talking about right now but automated test just FYI scalability right we need to know how to scale right in height and weights both directions real ability we cannot afford this flakiness at all because if we move into development process does have to be rock-solid how we can achieve that right speed who is ever done development here before as a developer besides Boris good so we have three developers that's amazing for somebody like maybe used to yep so for those with developers you live in continuous integration system you make a blue request and usually you want to get a quick result right you want unit test code stuck in the code static analysis check around winter and you want like in five minutes feedback so you can understand this Europe you are qualified for visual review by other developer right who want to wait two hours right so you for functional test the depart of a brain so you need speed speed is so important right now and our frustration is like how we can enable/disable tests without commits how we can actually make our test to be very configurable within the blue request right so we're talking about development process right now right that's what the shiplap is let's move on so continuous integration system can you stop for a second right here where is this fast is developer you see that like quickly cleveland done all right so see i was a CI continuous integration system right means that continuously caught code that is committed by developer been merging tested right unit tested you name it continuously right so continues integration is happening in this way I'm a developer I wrote like a very small subset of feature few lines of code and I open the pool request right and usually my pool requests have to go to stolen checks like unit tests and link right if those checks are checked out and I'll give her a bit pocket so then what happens Boris Boris look at my pool request and says yes the code looks good so we're ready to merge it and that's when we have their bill different like flavors of our app into the release learn deployment branch or hotfix branch or develop branch and then then we deliver a bill to manual tester for validation right that's still what most companies are working today right this is the process the flow and then a lot of testing including optimation happens here it's called post merge right after things happen so by the way at this point developer might have already introduced a bug so the bot end up in a bill and we are finding the defect and we say hey developer I found the effects okay yeah I'm gonna fix it like next release thank you thank you great job QA person so in the shift laughs we're shifting the testing activity from here to here before right before we merge the code right how many people done this before besides myself one two three four five five that means we are playing catch-up game and a lot of companies and I will give you a few names who already done this linked in 2016 there's amazing blog post a by three right very good I recommend to read they actually change the entire engineering department to adjust to this process Expedia did that in 2016 as well till turn we're almost there right it took me like two years to get to that point right there is a Fitbit bracelet you know Fitbit company they already also doing this it does the companies that I know but a lot of companies still go through the whole process okay let's move on okay so in order to qualify for the speed and we're talking about speed right now right and stability native instrumentation frameworks came into the picture and those are XE test which is actually people called X GUI test it's not it's exit test for every x QA test is just few classes it's been there forever and they just added to do functional UI testing and espresso by the way Express who do not exist anymore we just used the term because people familiar with this it's the cold jet pack right now package is called Android testing library that unifies just few libraries for espresso and you I have tomato it's just the whole name by the way doesn't exist anymore so those frameworks were built for developers so why we all here that means we are shifting towards development if you want to be comply with shift left movement it happens right now okay let's move on Anna I want to actually if what does mic microphone to Boris tell you that we all try to explain you and make decision what's the major difference between the a view and those native frameworks and they're not I would say you shouldn't think about it you should watch use one or another his one is complementing another and you have to understand it very well crystal clear is not in the I've seen companies to use both and this is very successful right and then Boris right now we're gonna draw the line for you so when you're gonna make a decision right you'll be crystal clear about expectations from your management okay Boris thanks here everybody so yeah as I mentioned ships left is having ink already we can argue that we can think well we can do air but it's happening correctly so we need understand how to meet the criteria so how to do this so Adam amazing presentation most benefits and drawbacks actually aboard total ready today so let's just sum up really quickly yes it's open source it's of course lots of different programming languages you can start almost immediately to some testing with this framework it's really easy to write some easy test Sarah I mean basically just a subset of webdriver protocol over HTTP so if you're familiar with selenium and webdriver it will be super easy to you to understand what's happening there and how to actually call something in I'd known it worse awesomely for hybrid apps which is huge benefit it's I would say the biggest one eventually it was developed to like the main feature of Vibram is to support both iOS Android and Windows right but unfortunately in the real life if in your company both iOS and Android apps exists you know that wait there is no moment when they are completely the same some features are done the Android and Google developed in iOS like in half a year and vice versa so apps are kind of different if you are using native so that's M me that's why but again for hybrid it works awesome cuz it's really it's doing its job correctly and if you have things and I own incurred all all that stuff so it's really easy to start many programming languages but do we need many programming languages well yeah as presa supports only drama coffee and this is a part of Android but maybe it's time to learn something new right to sheet forward development so it's more espresso yeah thanks espresso much harder favor really this is unrelated framework and really developed it other it supports it right now officially so this is part of jetpack SDK now and you need to be a developer to do the real job there like it's junior death but things that it allows you to do they're awesome there first of all we know that we need speed and scalability now right so we will have a tip for speed which will show the difference in terms of speed and due to scalability nowadays to do the real first of all you need to separate back-end and client testing right it just in case you don't wake we run all tests against stop nowadays because you don't want to test production and production is slow production I can can change responses production back-end is doing some maybe testing you need to be able to do some error cases you need to be able to test analytic so you need to first of all work the things in the app and you need to do those changes in the real app itself not the testing framework you change the app so for to do that you need to be in the same code base with developers you need to have an access to source code first of all right you want developers to seal poor request to entertain with you to see we really do there you can ask for help them you can do whatever you want with the app itself in terms of Apple you can just do a joking the testing framework totally separated doesn't have anything in common with the real app so yeah the AP is really small so and there is a recording feature so people who do the first time might think well it's took me like 5 seconds to record my first staff but if you really want to achieve shift to act it should be scalable it should be stable as a rock you don't have a right to have a flaky test now because developer makes a pull request and if it fails and he's bored because of some flaky test well the whole concept does not work whoo-hoo freak out obviously right because we're we've done our job and we won't like to merge it in and suddenly some testers for IKEA and it just fails my poor request and fells my CI so it should be super stable and obviously it should be fast so generally it should be faster than unit tests check and link link check I would say on average it's 30 minutes it's a maxim so in general we have 200 tests and we're on it in 20 minutes five minutes is built so like 15 minutes for 200 tests and five devices and it's really scalable so you can like add 20 more devices and improves like there is no need for 20 wait yeah we've done some experiments 10 devices is the most optimal which is something like 15 but that's a unit test for certain minutes anyway right so anyway between 30 is okay the opera still waits for uniqueness so it's okay okay let's see our demo so we prepared a couple of tests same test in Apple and espresso for same app let's see both framework connection I just see which one is faster much faster I think it's retest here we we will use widows retest here yeah I will show the time for the first course except this one is Apple right we set up the node.js server it please some time to watch okay this a special app for testing purposes we use it on our bootcamp in cold sickness okay first test second test opens the map we'll talk about this a little bit in more details and clear test we'll open the dial number and verifiably call the restaurant and in tuku 42 cents 42 seconds now we'll run stain three tests in espresso as first as say this guy's saying that we're done is to I think in nine seconds because Butte is is obviously same right it took half of it so 45 with nine as you can see five times faster again we don't want to say that we don't need to film anymore with this presentation does that but that's not true it's still valuable for such thing as again hybrid things and all the stuff and we saw the companies that use both is true but if you're doing automation tests for native in a developer and you want to shift to add you want to include those tests and say all right you can achieve much faster test and much stable test much more stable with espresso in our opinion yes and I want to emphasize sorry in in a summary few things like for instance that it was a demo with the map and the phone number was a reason there's such things called intense and Android how many Android developers here could you know was intense so we actually not asserting a physical phone number because every Android phone has different UI native UI phone number that's where actually IBM will fail big time and we actually assert that the the right phone number was Sam as a Nintendo intercepting them it's called int and stubbing and same thing with the maps so your test when you are testing against multiple devices it actually that's where espresso is going to be much more beneficial when you want to fight those things another thing is parallel execution is a I will call des sharding farting is actually not only thermal execution but actually all the tests shot em across devices for the speed there's framework called fork there's spoon fork all utensils right the bottom is fork is actually produced as well spoon it's helping to do shutting out the Box give you amazing result it's done by Chism and supported versions are also you can fork the fork and implement your own stuff on it so but here's the interesting summer that will help you to make decision so expressway used by developers that's the key it's it's not easy to write good test Indian Express I'll tell you right away you have to be junior deaf or have true as that I mean we're talking about Google a stats like you have to know like how to read the code base of Android it's a fast and reliable it's definitely scalable because you work parent pair with developers and best use for pre can meet that's the keyword NCI feedback validation okay that's important used by QA people it's much black box automation used by QA folks and automation teams it's mostly full coverage was very validation is basically good for regression testing that ones post-merge a few day a few times a day not every commit but let's say you can automate like the most important business use cases in this press around every PR and every commit any PR and everything else you can do with AP home right so you that's why I said it's actually complementing each other it's good for like I said NIDA execution and best useful floor regression testing right so that's what you have to always think about it if you need speed and be part of the development process and you have intention to become Android developer espresso overcoat automation test library is your choice as if you want to become Android developer in the future if you are like quality assurance if you want to stay in QA if you want to help manual testers to automate regression a peon is amazing choice as well okay it's all about choices so we just try to help you out with choices and next slide I just QA I'm sure you have questions feel free raise your hand and tinder yes and it's he like Morissette inside the codebase same repo and the benefit is very simple when the work was the same repository as developers when you write a test it has to go through two checks means the mm it if they don't because we're we're not the best day of right and they can recommend you say he were I'll tell you how to write it in a better way so we're in the same codebase yes both Android and iOS they're very different very different I would say that x UI test close to pay him the Moloch black box nature of iOS platform espresso is actually embedded you have to use with the dagger if you use dependency injection and you can actually there is no such thing as weight in espresso they have idling resources this wipes in the speed there's no wait that long exist if it goes with it was the UI that speed but if network threat thrown like a separate light you can actually slow down on it so it's completely opposite you're not waiting you slow down they have to and to go was this so it's very fast on xu8 there's there's no such thing so it is very very different frameworks the balls native so whenever the new SDK arrives you get the latest and greatest stuff right and I have to wait for him you know to get right away but they're different yes as Tinder's we do not support APO we do only espresso and xui test we have developers and they're like for Mario desistance but they get the same like the devs yeah that's why was yeah no like for instance for exterior test we have team of three as that writing test but their actual death they also do other things just to just pass the dev ops right there also wasn't just a fancy word there for instance the Boris can tell about what I asked him doing on a dev outside yes so nowadays automation teams do not do only automate testing it sounds really funny about that it is healthy so we're with automation team in iOS for example but in order to at the same thing we're doing different interesting solutions with Devils as they go mentioned so we automated many clear processes many have processes we cut branches automatically we protect them we back merge release branches to develop daily we like know many things that dream translations with the way automate processes in the company we all do wait wait everything that can build tomato well tomato them usually you need DevOps knowledge for that you used to it champions use AWS is screens these shell scripts with Swift scripts for that so many scripting things yes so this colder to Devils them to a step yes multi had obvious 200 instrumented test and then you take like two hours and they have 27,000 scene at this and they take like 20 seconds the question is do know what's happening with the unit bests my container if they take 30 minutes yeah yeah how wait how much will be created we have three tests and ours unit tests that takes 30 minutes in circles here right now we do unit testing in circle side and we do UI testing objectives and then pull it west like grabs reports from there from there for what's to 3 k3 Sonic's yeah dependent really depends on the nature of unit tests if unit tests are asserting one function like to post pause it's super fast but sometimes in your tests are doing much or deep links is actually integration test it yeah it's so it it's kind of similar I went to UI test of course you had this cumbersome in more UX perspective yeah but unit is notice also is a can can be kinda hard to be one unit test can take time yeah it's all depends on legacy code it also depends on the architecture and because of the like I have three companies been more than five years in business they have legacy and dealing was this is not easy as you have to quickly a so that's why I like princes on is the biggest problem is pluralization of the test that's why we get that much time we cannot paralyze them at least for now it's a big problem yeah but we're working on it yeah usually unit tests for mobile apps for mobile apps I'm not that as quick as for like I said Java back-end right there are a little bit more brittle so you talk about improvisation boys presser so we have as we speak right now we have five devices per each Jenkins node so whenever we have three pr's table requests in parallel running so issue of them executing this and it's about twenty minutes we can scale we don't have to because we're quicker than other checks and once we need to speed up which is control devices however as with people move to the darker kubernetes implementation will work on it to use scalable architecture we're going to emulators for those tests because we can scale pretty quickly it's about Android about iOS so we all know that extra tensile force protesting however unfortunately this is fordwick so yeah whenever you write a test right and the guy right over there is out looking and like you not guaranteed the test actually Iran stable your thing is gonna run locally bright thing CI who can nobody can guarantee it how it's going to behave out there only the boy s device form God knows right so for that reason would way created quarantine basically the task has to run in CI ten times successful in the role then automatically gets into the suite if it's failing it will keep running so quarantine helps us to mitigate flakiness on both the test new coming test and the desk we fix right as it's a basically internal web service that we build what is called quarantine just like a regular quarantine right and we're also the same tool called test Orchestrator it helps us to go would say i make its assets Ithaca happens in CI but it fails not because the developer introduced the bug but because the feature changed so I don't want a block developer what Boris will get so pissed at me he's gonna run with a baseball bat right the keyboard I cannot put to the minister cannot like merge might be armed so what we do without making consecutive commit to disable tasks we disable tests in the web service and removes right away so developer get unblocked so we build all this like I said as that is not only writing tests it's a solutions right it's architectural solutions so you like even I would say it's a true engineering job just let's be transparent I do have to be a true engineer any more questions last question you just to be an actual black box test let's press on absolutely not as I mentioned there is no such thing as a straw so anymore it's called instrumentation fasting and it's definitely integration test definitely and cui test yes XUI test yes that's why I said x-ray test the consider a blackbox automation but asbestos integration test activation yeah it's a black box however it's not that black box as a drone right because you still have source code right here in your project and there is an actual way to something in the source code of the app delegate in one child oh man so you can mattifies the app somehow that hey I'm testing you so please don't go to production server don't want the authentication process just mock your already authenticated to this to this to this because I'm your casing that's even if it's black box it still has a small hole you can pass some arguments and that helps thank you guys thank you very much our next before my favorite part from networking we have several events our next meetup will be at company acorns anybody heard about them yeah yeah so it's in Orange County they invited us to host their the next Meetup yeah as you know you can sign up to the middle about us you can find in test product I know we have Facebook page with a LinkedIn profile and we also to to boot camps one is for QA basic for people who work to transition from different career to QA and the one is for evaluation but not mobile

Channel: TestPro

Views: 2,533

Keywords: testing, manual testing, automation, silicon valley, success story, it job, it career, information technology, qa engineer, it courses online, it course, python, qa jobs, qa automation, selenium, testing courses, software testing, selenium tutorial, testing for beginners, what is selenium, selenium for beginners, qa testing qa уроки, qa тестировщик, qa обучение, qa тестирование для начинающих, работа в it, тестирвщик по, Работа в сша, Qa вопросы на интервью

Id: eEoiZKd7AqI

Length: 33min 53sec (2033 seconds)

Published: Thu Apr 18 2019