End to End tests with Google's Puppeteer - Tyler Clark

[Music] all right so yeah today I'm gonna be speaking about Indian testing with puppeteer did the introduction already for me but yeah I'm a father husband teacher like learning about new technologies and teaching them I've got a lot of blogs and videos of intros and stuff that I do I work for Perl site my Twitter I am Tyler W Clark when I'm done with this I'm going to post all the all the slides and links and stuff so that's where I'll post it I've broken up my talk today in three main points I want to first talk about what into end testing is just kind of the principles behind that talk about what puppeteer is the mechanics of how you use it and then at the end we're gonna use create react app just and puppeteer to walk through some implementations just a fair warning though 30 minutes is not a lot of time and especially to cover three main topics like testing and jest and puppeteer and how to implement them all so I'm moving kind of quickly through everything and I'm giving some high-level examples and stuff but you can spend an hour on just the principles behind testing so I'm gonna go over just some high-level stuff but definitely check that check out my Twitter later on I'm gonna post these slides and I'll also throw them in Utah Jaya slack as well okay so everyone is developer has probably heard this at least once or at least said in their life I'm sure even today there's somebody in this room who's put it and slacked it and said let's wait till Monday before we push that is there somebody I guarantee it I like this guy tweet I saw from Eric Elliott once we said you've all heard never released on a Friday or a night right well my answer released every time I merged passage of test leads wouldn't that be ideal when you have the confidence your test Suites your CI passes that all right I feel good let's go ahead and push the prod right but it seems like a lot of times when we push changes to through you kind of just stare at that loading green bar praying that it doesn't go red right who's been there come on raise your hand there you go I know everyone's been there actually it was me two days ago so and/or my all my favorite type of testing is the hand waving I'm gonna click on everything in the front end after I did something and make sure everything still works right we've all done that just two days ago a co-worker and I we some reason couldn't get a MPM package to pass through teamcity or CI but it would work locally and so we did some magic and we got to go through and we were that little girl staring hoping that it wouldn't break and then once it finally went through teamcity our CI and it was deployed out to our staging environment we you know clicked around does this link still work does this button still do what it wants am i still making HTTP calls well that's not the best way of going about it right I mean that's not very failsafe you're not you're gonna miss things so there's got to be a better way right but how like how do we how do we get to that state where we know that once it goes through when it passes everything we can go to prod right how do we get to that point well unfortunately as you hopefully it can be imagined that's probably not 100% possible you're never gonna cover everything or catch all the bugs right especially when we've got browsers and we've got versions of browsers to go through we've got databases we've got back in languages we've got front-end frameworks and typescript flow and CSS and jas we won't go there but it can get overwhelming to feel confident like Eric said in this tweet where it passed there see I passed our test suite we can go to product you know I feel pretty good about it everyone in this room probably understands testing is important who uses some type of test mentality or methodology like TDD BDD test first all right how many people do that okay awesome a lot of people how many people want to do test but they can't convince their boss there's always a good amount of people for that as well okay well the testing obviously is a good thing that's not the point I'm trying to get to today but you probably if you are testing look so most people are you're writing probably unit tests and integration tests and unit testing is the testing individual components right an isolated manner integration is testing component collaboration right probably to two units talking to each other maybe it's a database in the backend you're testing your REST API in point and making sure that that integration worked well even after you write all your tests and you go through TDD at a plural site we have a TDD our team uses TDD but two days ago I was still staring at our CI praying that it wouldn't break our front-end and our client so even after these mentalities of testing you still miss things you're still not confident about it and you kind of feel like this right where you feel like I've tested everything but I don't know it just doesn't work like a human supposed to work right but you've tested the feet work right the the the mid-level works the head works everything works right but some reason the end product still doesn't you know fit together right and that's kind of where Indian tests come and come together and solve that the issue of testing kind of the whole system from the from the very end one point that all the way to the other end so Indian testing it tests that make sure the whole app works correctly when all components are connected and tied together so in other words you're running a test that replicates actual user interaction with your website okay so you're not testing just in your jest and or whatever your your dome if you're using ensign or something some type of like fake Dom like you're actually writing a test that's testing in the browser where your users are actually going and in that end result that's where you're testing if your application works correctly so what does that mean what does that look like so this is where you can test things that kind of utilize many parts of the stack right forms is a great example links where you depend on routing that calls more HTTP calls I'd you know if there's a redox or mob whatever widgets HTTP calls normalizing your data on the front-end there's so many there's so many things that it all connect to each other that it's hard to test everything so a good example would be it forms right how many people are using react probably most people all right awesome let's say for example we're gonna test the form the the hands that gets passed through the chain is a lot right if usually react if you're on a big app or whatever using Redux you've got your react components your inputs they type you probably keep in Sates and we're either inside react or redux if you're using redux you might be you're probably using a middleware like funk or sagas to handle the asynchronous submission of the form you've got your back-end that has that endpoints node whatever it is and it grabs that stores in the database hey the database works send it back to your your API send it back to the redux - it's just this there's tons of things and there's a lot of places where something can go wrong right at any point in here this is where we look like the girl that's just staring a team your CI waiting that doesn't break right because you're you at any point in this handoff of information something can break and it causes an explosion on the front-end so let's talk about puppets here now let's move into just talking about puppeteer as a whole what that is and then we'll we'll talk about how we can use puppeteer to create these into end tests it's important to start off with what is puppeteer as to like how to use it because as I'll get to in a minute it's just basically a straight from the docks it's just an old library that provides a high level API to control Chrome or chromium so it's specifically chromium which is the open source part of Chrome but more or less it's it's just Chrome so Puppets here is pretty close to what chrome is the browser today and puppeteer gives us the gives us simple API is that we can tie into to utilize and do stuff with it so because it's not specifically designed to be used with testing you can do stuff out kind of testing right it's just a browser you can generate screenshots you can scrape text and images like you might have done with phantom j/s or something else you can work with service workers and aggregate information from search engines there's there's tons of stuff that puppeteer is used for today that doesn't include the testing environment because it's just a browser which when tied to a testing library like gest or mocha or in chai you can write these Indian tests so it's important to say what it's not and I've said this a couple of times a couple times now it's not a testing framework I guarantee because I get asked this question a lot and anyone that talks about puppets here and testing has probably been asked this but number one question I get a good-ass this house it's like Cypress how is this like selenium or a Nightwatch like I've heard of those I've used them in the past how is it like that well again puppeteer is not a testing framework it's not something that's designed for testing you can use it for testing but it's chromium it's an API that you can tie into so of course if you use testing frameworks and the like these other ones you can get little more features but a lot of people get turned off when it comes to writing into end tests my previous job we use selenium to do some stuff and it ends up spending tons of time because tests are brittle they break in IE and they when they work in chrome and you have to figure out why can I this one test just pass across all our browsers it usually takes a lot of effort to get up and going they're heavy and the biggest thing is that they're brittle right they changed between things changed in your app between browsers so people give up on them what I like about puppets here is that it's pretty straightforward as you see it takes only five lines of code to to start it and to make your first assertion and it's just one package that you just bring right into your testing environment and go nuts with it so the elephant in the room is that yes this is testing only in Chrome all right so it's chromium right so you write your test and with puppeteer you're not testing the test might break and ie or Firefox or what-have-you it is only gonna write your tester your stuff in chrome which obviously you need to understand your your user your users and say you know is all of my users on ie maybe it's not the best case for you still might give you value for things that you do but there's trade-offs right you can go with selenium or Cypress or Nightwatch whatever it is you're gonna get a lot more built-in features that can test more browsers but I like puppets here just because it's really simple really easy to use it's a native browser that you can actually use and integrate with your test and it's a lot easier to get up and running and it kind of accomplishes the the goal which is a test the end result and then make sure it works on the flip side the fact that it is a browser kind of gives you some cool tools that you can use inside your app right it's a browser so open up chrome dev tools there's a network there's a there's a performance tab there's a whole list of things that you can do with this performance so like with Google's performance tab you can and and and just as a side note every time chrome releases something new you can upgrade your puppets here and get that and add that into your test which is pretty cool but you can if you're a if you're a team that hates console logs in errors which everyone hates those right but you'll see that a lot of times they stack up over times people miss them they they don't get checked when you you and you merge PRS but you can actually use puppeteer to query your console and then you know push it on an array and assert that you don't have any errors so this could be kind of an implementation detail but it's still something you can use and add to your your testing is style you can actually write a test that tests the page speed of your of your website so if you if you have as a team as a goal you say I don't want learn to our page to ever take longer than this amount of time you can throttle the network to I mean I you can imagine everything you can do in the dev tools for the most part in Chrome you can write and be creative and write a test around making sure your website handles or does what you want it to do with by just getting information with the use of puppets here and wrapping it around and test page weight there's also another great one so this is cool story and all right like I kind of want to tie just kind of like get up and going right let's just text test one thing and then before we go crazy and test PageSpeed and first paint and all that other jazz so how many people that are using react are on 15 right now all right good amount there's a lot of people on 15 still I know for when I was upgrading our team from 15 to 16 I would have loved to have Indian tests that test just basic stuff like routing of links calling networks submitting forms drag-and-drop that kind of stuff I would have loved the head in test because as we know 15 to 16 wasn't breaking change but a lot of people didn't experience that many breaking changes to their site but if your app is built entirely around react and you upgrade from 15 to 16 it's hard to check every single you know nook and crevice of your app you're you don't want to miss anything right so you're that grimacing face that's clicking yeah I think it's good right hand waving a thumbs up to your team lead saying we can we can deploy I think it's good if you had some minute in test you've built a long that's checking those links or those simple things that's gonna give you put you in a better spot so you can upgrade react 15 to 16 and hey you've checked you have this test suite against your basic features of your app and you're you feel pretty confident so let's look at some implementations of combining this puppeteer browser with jest and and again you don't have to use Jess if you're using mocha Jasmine whatever you're using you can use that alongside a puppeteer as you'll see you just switch out all the jess stuff with your mocha and chai stuff 30 minutes is not a lot of time again to go into everything so if you don't have Jess installed or moco installed or any kind of testing suite go through the docs it's pretty easy to get up and going with that there's lots of videos and and and basic setup stuff so get that working first it's pretty easy but to work with puppeteer again it's just it's just one package right it's a big package because it comes with chromium so you want to be aware of that but it's one NPM package that you can download within P&A yarn or whatever and it'll download all of chromium with you so you want to manage that right you don't want to ship that with your node modules that'll be bad it's also because you're working with a browser or you use this weather I know other frameworks like Cypress I think unwraps the promises for you but since you're working with a browser it's really helpful than understand a single wait because obviously the loading of a page is an asynchronous thing so brush up with async/await and you'll see here we use a weight a lot of stuff just because when you interact with a browser it's gonna be asynchronous so let's just choose creat rack DEP right pretty simple it's just the basic loading page when you install let's just write one simple test tesla's of the h1 the welcome to react actually says welcome to react right let's just get started here and look at one simple thing well to get up and running with this it's easy as 1 2 3 4 5 it's pretty it's pretty quick so the first thing you need to do is launch a browser so inside of your testing framework you're just going to do a wait puppeteered launch so you launch a physical browser instance of Puppets here and then you create a new page just like you would on a tab when you launch a browser you tell it to navigate to your app so in our case of creepy act app we're just gonna do localhost 3000 this is telling it to navigate to our running app fourth thing is this is where we're actually do something this is where we actually assert that that h1 says what it says we use we're gonna talk about the the method very vow in a sec but we use just a a method on the page object which was the part of opening a new page on our browser and we passed through a selector so that could be a class in HTM whatever type of valid selector to grab HTML there's a callback function there to grab that that text and then using Jess we do expect that our HTML is welcome to react so pretty simple right it's pretty easy and the five thing is to clean up we want to close our browser instead of our test so if we put it all together this is what it looks like so we're gonna import Puppets here instead of our file we launch a new browser so just public tear out launch we open a new page we navigate to our local host running app we assert that our h1 says we grab the h1 text from that page we're at a quick Jess assertion and then close the browser when our test is done so pretty simple right get up and running pretty easy you open a browser you navigate you find whatever you want to do and you write a simple assertion quickly doesn't work every time I click all right so some tips that you can do to make this even easier we want to utilize before all and after our hooks Jasmine mocha everyone it has these you can open up your browser inside the before all you can close it in the after all it's helpful to understand a single wait again because it's that's both Brown promises you can utilize environment variables which is pretty cool publish here has an option to be in headless mode or not so if it's not a headless mode it means you actually see a browser open up when you run when you run your test as you'll see I've got a couple examples it'll show you that it's loaded up your your website checks whatever you told it to check and then closes when you when it's closed so and this is to make it headless let's go back here we just passed through headless as false as an option when we launch the browser so this is going to tell it to actually physically and not physically right open up a browser and so you can see what's going on the do stuff part again is really the fun part you know I've mentioned thing like performance testing and we'll see some basic setup for that in a sec but this is really the fun part of using puppets here as we as we saw there was that dollar sign eval this is a method that does basically the same thing as document a query selector so this is the Dom element method there the double dollar sign eval that's going to return an array of all of the selectors that match on the HTML and there's many other ways to do this as well as you can see we were using it instead of our testing file there you can interact with a page this is the part that makes you feel like a wizard because you can tell puppeteer to do stuff with your app so you can tell it to actually click on things so you say page that click and you pass through so this is good for buttons for links from forms you tell it to click on something and you can tell or if you're in mobile view you can tell the tap on something you can tell the type so you pass through a selector tap on this input box you're gonna find it by this selector that I'm giving you and then you have a text or that you want I want you to type so the puppets here will open find the selector and then type where you want the type and so here's a code example of us filling out form so we've we've the first arrow there at the top we're grabbing all of our input boxes by selectors we're naming them we've got last name email password submit and then once we've assigned those two variables we tap so tap on an input form and then type then tap on the next input form and then type so forth typing in first name last name user email finally a password and then we the last arrow is we're tapping on the submit button so we wrote a test that steps through a form one input box at a time typing in an input and then once it's done at the end it taps the submit button this is a quick little video of that in motion here we've got in put a forum where we're running a test and this isn't headless this isn't headless goat falls so you can see puppeteer open up and there's a little bar on top says chrome is being controlled by automated test software it'll open it up it'll type into those input boxes you told it type in one at a time and in the order you told it to because you're using weights and then you press a submit so then you can write tests around did my six my my cool submission icon show up because that means it was successful did it send a request with this information you can capture those from puppeteer and then write cool creative tests around what everything what's all supposed to happen when people submit a form and you can write assertions around each each point of that if I was taught this by smart man but in in tests are it's best to have them act like real-life users right so it's it's cool that when you submit that form that actually goes to your database saves that and it returns back like that complete roundtrip that's gonna give you the most exact test and to know that something worked if that doesn't work for you in your wherever you're working your environment you can't filter out these dummy test requests there are ways just with any browser to intercept requests so there's methods that are built on where you can say hey capture all the requests if it says something then abort it don't let it actually submit your form you can still test write test around it that make sure that the responder request is correct at all you can emulate mobile on this so it's a browser you can emulate different phones just like you can in the console when you click on the icon I change it to different formats iPad or whatever you can do that here you when you create your page you say open up with this viewport with this user agent with all the other options that you have you can test your app in different styles and on different devices as I said you can capture a console messages so you just keep a you just keep a watcher whenever you open your browser you say whenever you see a console message the page error warnings whatever or it is that you care about or your team cares about that you go on the right tests around you just you can write an assertion simple dot length equals zero there wasn't any logs or errors in our console if you work with John Sethe indication local storage service workers guess what you can you have all that power at your fingertips with puppets here because I can't stress enough it's just a browser so anything that you can do with that Chrome browser you can do so you can set inside your testing environment cookies or local storage values that way you know that your app works correctly when it comes to in that testing environment so that's how you can get around some of those issues you might have with authentication something that's pretty cool I've actually heard been talked about last couple of time is something called visual differing tests if you don't know what that is it's also I've also heard it called visual regression test where you have screenshots of what your website is supposed to look like when usually to interact with it you can have a screen shot my site when I throw 100 users at the same page at the same time click on the Save button it should be able to hand that I'll handle that in the front end it shouldn't blow up shouldn't get errors you can have screenshots and compare it to what your app actually looks like and that's simple because puppeteer has a screen shot option so I've got another implementation of what that looks like there's a lot to it but I've got Mike's assertion they're expect that the number of different pixels is zero that it matches pretty closely to a screen shot that I'm using that I'm taking with puppeteer on the left side there that's the screen shot on the right side that's my running create react app and as you can see they're identical right they're the same thing when I write MPM test and I run my test it'll open up and headless made false and hey the screenshot to your running app is identical but if I can go back and break it now so if I go to just my app and I add a P tag says break the pixel text you'll see it pops up on the right there now if I rerun my test my screenshot it's not going to match right and it fails there's a difference in pixels so you can change the intensity of the pixels to beat that it tests against repeat and extra P tags okay you can increase the amount of pixels you want it to be off by it doesn't have to be you know 1 P tag breaks it but you can write a little bit less sensitive test but the power is there right you can make sure your app looks the way it's supposed to look throughout the whole time so you're keeping yourself accountable right and it really ties back to what Eric said in this tweet if you have if your app your changes that you've done react 15 to 16 changing something on an on your on your upgrading to done that core or something from from standard if you have these tests especially a visual differing test that says your my website supposed to look like this and even after these changes it passed my test suite I'm pretty confident that everything's gonna work its gonna make it through so this is another another pillar or another triangle another step in your testing tool belt that you can add that really makes sure that everything works and then the really the fun part that I recommend you go and check out is once you get comfortable with it and working with it there's cool things you can do like in this example I'm testing the first paint time of my website and seeing how long that takes you can have these accountability test to begin with that make sure that you're not you don't slowly slow down your site without even knowing it that you're keeping yourself and your team accountable all right and there's more right there's always more I would say the best for the last this is my favorite part - this just came out like three months ago some of them made a really cool Chrome extension called puppeteer recorder and I'm gonna I'm gonna we're gonna see it in action here but what this does is you can pull up your app whatever you want to test push record in the Chrome extension and then type and then interact with your website however you want it to interact with submitting those forms sending those links whatever interaction you want typing clicking touching it'll record everything you do and then spit out the code for you so literally all you have to do is copy paste in your test suite write your assertions around it because CSS modules you might have a little bit of problem with a selected part but a waiver hand on that as well but at least the heavy work of launching the browser nabbing to the page it's super fuzzy there which is unfortunate but it it does a lot of the dirty work for you so we're gonna we're going to navigate to the Utah GS site because it we know that site definitely could use some testing around it so I this is a Chrome extension up here in the right I've already got it installed if it doesn't work I've already done it so I've got some code there so you so you can believe me that it works but we're gonna let's act like we're gonna buy a ticket right so let's we'll restart this press record this is recording we're gonna click on buy tickets the inner internet works that's great oh they're all sold out you know what we're gonna stop this and go to Google so that I can actually feel something when I did it yesterday and let me still select one so it's good to just Google all right so if we restart this and start recording let's just click on the input box type whatever you want and then Google search it oh yeah there's a result for that we can stop the recorder and there's our code that we need so it's still super blurry but it says basically navigates to a page set the viewport that my laptops currently in and navigated to Google calm it typed on that input box it gave it it gave it the text I did right there everything that I typed in there waited for the next page which is the wait for selector and and so forth and then it closes the browser for you at the end so this just came out three months ago or six somewhere somewhere between that so there's only on version three zero point three so there's probably some bugs here there but there's really no excuse of like the hard part it's this isn't this shouldn't be shouldn't be scary like selenium was in the old days you can walk through your site do whatever you want to do copy that and then throw it in your testing page and write some assertions around it it makes it super easy and that I'll go back so that we can look at that so this was just called puppets here bird quarter again this is gonna be my slides you can just google it it's a Chrome extension and it'll make life super easy I'm out of time here but really go through and check the the puppets here docs there's I hit on like six think methods and there's like fifty so there's a lot of stuff you can do a lot of smarter people than me have created some some stuff that like the first paint time stuff that you can use and implement all my examples that I've used so far have have been are gonna be posted in my github and then as well as these these slides if you have any questions you could catch me after or or and I'll try to ask them as much as I can thanks [Applause]

Channel: UtahJS

Views: 23,916

Keywords: Puppeteer, Chrome, Google, Jest, End to End tests, utahjs18, JavaScript, E2E

Id: xwiWqEkrtyQ

Length: 32min 28sec (1948 seconds)

Published: Sun Sep 16 2018