How to use TDD to build a web app that queries a REST API with Jest and Puppeteer

in this video we're gonna look at how to use test-driven development to build a web front end that queries a REST API I'm Bill sewer from dev mastery and you're watching mastery Monday the weekly show that helps you write better code and advance your career please subscribe and hit the notification bell so you never miss an episode before we begin I just wanted to give you a quick heads-up about what to expect in this video this is not a step-by-step beginner tutorial on web development or even TDD this video is a more advanced look at front-end development where I show you a specific approach to using tests to drive your UI development and we'll also explore some of the tools that are used in this approach so if you've just started learning to code this might not be the best video for you but if you've already had some experience building apps I think you'll really enjoy it okay let's jump in okay so we're gonna build this little app right here we enter in a title of a movie like Batman press the search button and we get a bunch of posters for Batman movies now what's going on with this app is behind the scenes we're calling an API from the online or the started the open movie database so this is a free API that you can use there's a couple of gotchas so if you get a free API key you're only allowed a thousand requests per day which might be troublesome if we're building a real app so that would be something that we'd have to take care of especially if we're doing test-driven development and calling the API over and over and over again that might that may not be so practical the other thing about this is obviously we don't own or control this API this is a third party API that we're just consuming so the responsibility for testing the API is not ours but also there are no guarantees about the state of the API nor can we control the responses from the API so we're gonna have to deal with that the other thing is this is gonna be a modern web app so things like typing in the typing in our movie title and when we hit enter or press the search button we're gonna want to see some kind of spinner or indication that it's searching basically the UI is gonna have two Update parts of the screen at different times without refreshing the entire page because that's how modern web apps work and you'll see that's quite tricky to do from a test-driven development point of view but I'll show you how to do it alright so what we're gonna use to build this app is we're going to use something called puppeteer so this is a product by Google and what it allows us to do is it allows us to drive a browser using code so we can basically code instructions to a Chrome browser and have the browser perform different actions like typing in a text box or pressing a button we're also gonna use jests so just is a tool for testing so whereas puppeteer is just a tool to control a browser just actually allows you to perform tests so when you combine jest and puppeteer you get a really interesting framework that allows you to test your app as if it was being used by a user in other words you're gonna test your app from the user perspective but I'm getting ahead of myself the first thing we need to do is start to understand a little bit about how this API works so anytime you're dealing with an API that you don't control the first thing you want to do is get to know the API so what I have here is a tool called postman and that allows me to query an API allows me to send requests to an API with different parameters and see the response the JSON response that comes from the API and so this is a cool way to sort of interact with an API and get to know how it works so here I've sent the same query that I showed you earlier Batman and you can see what the results look like in the JSON format that they're returned in I can also come in here and test out how the API behaves under different circumstances so let's say I just put in a bunch of gibberish so I wouldn't expect this to find anything so we'll hit this end and we see this is what an error response look like looks like for no movies found we can also come here and just enter a single character and you see we get a different error too many results so this tool allows us to play with the API and get to know how it works of course you can use many different tools to do this but this is the one I like it's free it's called postman they don't pay me anything to say that I just it's one that I use all the time and so with this now we can get a sense of how the API works and we're going to be able to create test fixtures so test fixtures are like static dummy data that we can use to return in place of real values in order to save the first of all the time required to do round trips to the API and second we've only got a thousand requests per day so we don't want to burn through those too quickly so we're using the real responses to generate fixtures which are going to be dummy data that we're going to use in place of actually calling the API so let's jump into the code and I can show you what I'm talking about so I've set up some boilerplate code for us which I'll link to in the description this is an app that I created using create react app and then I added puppeteer and some configuration to make puppeteer work with jest and create react app I'll put in the description links to information on how you can do that and also to this code so that you can download it the other thing I've done is I've created a folder called tests and in here I've added our fixtures so this is what I was talking about before so under the fixtures folder I have this API and I've got a few different fixtures which we'll look at later that simulate or look like the responses from our API I also have a search spec here which we'll go over in a minute in the source folder I have another folder called poster search and that's going to have our components or main component that does the searching and this is just a mock-up right now so there is no active code in here there's nothing interactive about it it's just basically the HTML markup ok let's start coding so on the left hand side we have our test suite and on the right hand side we have our component now as I mentioned this happens to be a react app but what I'm gonna show you actually will work with any framework you want to use so this will work with view with angular or even with server-side frameworks like dotnet PHP it doesn't matter the reason is because we're using puppeteer to drive our tests from the perspective of the browser anything that runs in a browser can effectively be tested using this method so yes this is a react app but don't worry if you're not a reactive out / that's not necessary you can do this technique with any framework that you choose to use you'll notice this setup is really cool you've got a test on the Left code on the right and a terminal on the bottom and this is the way that I like to set myself up all the time when I'm coding I find this makes me the most productive so at the bottom in the terminal we're gonna do two things first we're gonna start our app so that we're running a development version of our app that will automatically refresh as we change files and then we're gonna start our test suite and have it continuously running so that it can watch our files and as we change files the tests will rerun you'll notice at the bottom we have a bunch of skipped tests that's because my test descriptions are written with an X in front so the X causes them to be skipped I like to do this to give myself a kind of to-do list that will tell me what I intend to implement in terms of the behavior of this particular feature of my app in the future with gest we'll actually get a dot to do keyword which will give us a prettier interface that will actually say instead of skipped it'll say to do at the bottom in a nice purple color so if you're using a modern version of just instead of putting an X in front you can just do it dot to do and that will give you your little to-do list and so you'll notice I'm describing everything from the perspective of the user with myself being the user so I have things like it tells me when we're searching it tells me when there are no results it tells me how many results were found and how many are being displayed and I like to use kind of natural language and just give myself this kind of to-do list to tell me what it is that I'm gonna implement so let's start by implementing the first one which is doesn't let me search until I've typed at least three characters and that comes from playing with the API I noticed that anytime you try to search for something with only one or two characters you usually got an error that said too many results so there's no point in letting the user type one or two characters hit the search button and just get frustrated because they're getting an error okay so with puppeteer there's a little bit of ceremony before we begin you'll notice that I've got this before all block up here and what this is saying is when the browser disconnects closed the browser and that's because in watch mode at the end of each watch cycle the browser is gonna disconnect and I want to actually close the browser so that I have a fresh new browser each time and then before each test I reset the current page and I navigate to my localhost my local running dev instance of the app and the reason that I do that is because before each test I want a brand new page so it's as if the user just landed on the page and we're starting with an own sort of virgin State okay so let me show you what a puppeteer test for it doesn't let me search until I've typed at least three characters let me show you what a puppeteer test for that looks like okay so here's what our test looks like so the first thing we do is we go and grab this search button by its ID from the page and then we interrogate the disabled property and we want that to be true and you'll notice I'm using async/await here because puppeteer is promise based so everything is going to be promises that's why we use async await so when we go and interrogate this button make sure that it's disabled then we go and fill in the movie name text box with the characters ABC and then we check the search button again and make sure that it is no longer disabled okay and so if you're wondering where these come from if you look inside my component here here's the button and it has an ID of search button and here's the input and it has an ID of movie name now in general when I'm on a project I sort of have a rule that JavaScript owns the IDs and CSS owns the classes the class names and so that makes it safe for me to use IDs in my tests but if you're working in an environment where you have designers or where people are accustomed to using IDs in there stylesheets then it might not be safe to use an ID to do your test in which case you may want to use a data test ID so you would add a custom property here called data test ID and you would use a selector for the data test ID attribute so bracket data test ID equals whatever they did a test ID you gave it okay and so I'm gonna remove the Skip here and we're gonna watch this test fail so when I save this we can see we now have a failed test and it's failing because our search button is not disabled you can see it right here it's a regular button there's no disabled property and so to make this work we're gonna need a little bit of react magic and voila we have a passing test I'm not gonna bore you with all the details of all of the code but basically the main bit that's working here is I've got this handle input function that gets triggered when I type in the textbox and it sets a disabled search based on the length of the value typed in and then it also sets the movie name to the value typed in which we're gonna need for later if I introduce a delay here when I fill out this form you can actually see it working in a real browser here I'll show you we add a delay run our tests and puppeteer does it's magic so this is that dance that we all do right where we write a bit of code try it in the browser see what's broken go back change a bit of code try it in the browser again except that now it's all completely automated for us so we just wrote a few lines of code and we have this working for us automatically so that's gonna save us time over the lifetime of our application it's also gonna give us a repeatable way to test the application from the perspective of the user so it's very very valuable and that's part of the reason why I absolutely love doing it this way okay so let's move on to the next one we'll get rid of the delay for now so that it doesn't slow down our tests and then we'll collapse this test so that we can see what we're doing and we'll start on it tells me when we're searching so the idea here is to provide some indication to the user that a search is happening prior to the results returning and this is something that can be very very true right because you've got to write a test that is going to interrogate your UI after the request has been made but before the response has been received and when you're dealing with network requests it's often difficult to predict exactly when the response is going to come back so there are a lot of hacky solutions to kind of testing this kind of thing where we rely on delays but that creates very flaky tests the good thing is puppeteer has a great way of testing exactly this kind of thing so let me show you okay so here's our test let's walk through it so we'll get rid of this skip as you can see puppeteer allows us to intercept requests so by turning this on set request interception on our page what we're saying is that we want to intercede in every potentially every Network request that the page makes and then we can register an event handler for the request event so whenever our page makes a network request puppeteer will run this callback function that's here so the nice thing now is we know exactly when a request has been initiated and before responding to the request which is what we're going to do here we can assert that we've got a label with an idea of message and that it is displaying text searching so that means as soon as the network request happens we're going to show the user some text that says searching inside of a element with an ID of message and then after we've done that what we can do is we can send a response using the respond method of the request that gets passed into this callback and we can control exactly what that response is going to be so here we're sending back a dummy posters array and this dummy posters array is coming from the fixture that we created based on a real response from the actual API so we're not calling the API we're sending a dummy response but it's structured exactly as if it was the real response because we created it based on that and importantly we have this done function here that tells us that the test is over so because this code is running asynchronously inside of a callback we want to make sure that the test doesn't until we've actually done the work of testing what we're trying to test so this is a registered callback so now we need to manipulate the page so that we fill in some text into the movie name field and then we click the search button and another precaution that we've taken here is we've put this expect assertions at the top and what this is going to do is it's going to ensure that there are three assertions in this test so we don't want to have this assertion get missed because it's in a callback in so it's happening asynchronously so we want to make absolutely sure that all three assertions have run within the scope of this test so now if we save this it should fail and there we go we've got a failing test and you'll notice the reason that it's failing right now is that we're expecting three assertions and we're only getting two assertions so what's happening is this assertion here is never being called because our component never makes a request to this URL right so because the request to this URL never happens we never enter this code and everything just times out so let's fix that part first and now we've got a passing test again so what's happening in this code is we're simply setting a message to searching when the user clicks the button and this message is appearing down here in a paragraph tag that has an ID of MSG which is what our test is looking for we go out we get the element with an idea of message and we expect it to have text matching searching so now that we've got that working let's collapse it and head down and implement it displays all results so this is what we call the happy path it's the experience that most users of our app will have most of the time so it's important to get to implementing that as soon as possible because this is where we're going to discover some of the gotchas and it's also the main thing that our app is meant to do so it makes sense to get it out of the way before we handle exceptions or alternate paths all right so let's walk through our tests so as you can see again we're intercepting all of the network requests from our page and then we've registered a handler for the request event so when our page makes a network request we inspect the URL and if it's a request to our API we send a response containing our test fixture dummy posters so this is just a JSON file with an array of poster objects similar to what the real API would return to us but one that we've made and we can control then we handle the response by registering an on response Handler and here we receive our response we inspect its URL to see that it's a response coming from our API we capture our dummy posters into a variable that we can then loop through and check on the page if we have an image for each element in our dummy posters so it's helpful to go look at dummy posters at this point and see what it looks like so you can understand this code a little better so here's the JSON file that we're using for dummy posters and as you can see there is a post or property in each of these movie objects that's a pointer to an image for the movie poster so back in our test we're looping through the results and we're checking that our page contains an element that's an image and the source of the image is the poster property for that movie object once we've done asserting all of that we turn off request interception and we're done our test and of course now that we've registered the handlers we have to make sure that we fill out the form and press the button so when we save this we'll have a failing test and you can see our test fails because it's not finding any images with our posters and in fact it just exits on the first one so it can't even find the first poster because we haven't implemented it yet so let's head over into our component and implement this functionality so we can make the test pass okay so now we have a passing test so you can see we've captured the JSON from our fetch request extracted the results and stored them into a posters variable then on our component we loop through the posters we map over them we get a bunch of movie objects as I showed you from the JSON file and then we just insert an image with the source equal to the poster property of the movie object so for each movie object in our search results we're going to display an image with the source of movie poster so now that we have a passing test for the main path through our application where a user has entered a movie title and has received a set of results in the form of movie posters it's important to make sure that this code will actually work against the real API whenever we're working with fake data or dummy data there's always a risk that even though we have a passing test something about the fake scenario we've created is going to be different than the real API and so it's important to have a way to test against the real API especially before we push any code to production so I'm gonna show you how to do that in a moment luckily with puppeteer it's pretty easy we've also done a number of things to sort of make sure that we're as close as possible to the real thing so we're actually intercepting requests at the browser level rather than trying to mock out some kind of library like fetch or Axios and that's helpful because it's going to behave in a manner that's much more consistent with how things would behave in real life we're also basing our test fixture on a real response that we got from the API so that should be as close to real as possible nevertheless it's important to test against the real API to make sure that our code still works so if we look at our package JSON file you'll see we have a build script here that's gonna generate a production build of our app I've added a post build task that will run automatically after each build and what it does is it sets an environment variable node end to the value build and then runs our UI tests this means we can use this environment variable inside of our tests to switch to the real API instead of using our dummy data so now in our tests we can read the node n very and then depending on whether or not this is set to build will navigate to either the prod built version of our app or the dev version of our app and then in our display all results test we read the environment variable and if it's set to build we just continue with the request so we let the request pass through to the real API and when we process the response instead of looping through our dummy results we loop through the real responses JSON so here's our results and here's this looping through them and it's going to be different depending on whether or not the node envel is set to build so now this gives us a way to run our tests in production mode where we're actually hitting the real API and seeing if it works so let's do that down in our terminal we'll just run a build and there we go we've got passing tests hitting the real API and this is going to happen every time we do our continuous integration or our deployment where we build a production build of our app it's actually gonna go touch the real API and make sure things are still working okay let's get back to developing okay so now we can use the same techniques we've been using to implement all of the exception paths like handles API error handles Network error and we're gonna do that using these test fixtures up here which again are just JSON files that were generated from the real API okay so I've gone ahead and implemented all of the tests now using similar techniques to what we've just learned and everything is passing and it's all working so let's take a quick look at a couple so let's say for example we want to look at handles API errors so as you would expect same pattern we request we intercept the request make sure we're calling our API and then send across an API error response which is just a test fixture with an error similarly we have Network errors so in here we actually go ahead and abort the request with a failed message and make sure that we tell the user that something went wrong so like I said I'll put a link to all of this code in the description and you can go through it yourself and take a look and examine the details but hopefully this has been instructive let's think about what we've learned so the good news is everything works the bad news is if we look at our code for our component here this code has gotten pretty messy it's basically a big ball of mud but that's okay because our approach is gonna be first we need to make it work and then we're gonna make it better so getting to the point where we have a functioning app or a functioning feature that does exactly what we expect it to do is a huge win and now we've got a test harness that gives us a safety net and allows us to go ahead and refactor our code into much cleaner much better separated much more elegant code and that's what we're gonna do in the next video hey so after seeing that I figure you might have a couple of questions so I wanted to address them right now you may be wondering why not use a component testing framework like enzyme or react testing library to test react components in isolation and the truth is the more your tests know about the UI framework that you're using or the component hierarchy that you've built the more brittle they become so by testing from the perspective of the browser you free yourself from the dependence on the specifics of your UI framework or your component hierarchy that means when the UI framework comes out with new features you're free to use those features without potentially breaking your tests and it also means that if you want to refactor your component hierarchy extract some components create some higher-order components you can do that again without affecting your tests having said that there are times where your page becomes really really complex or the part of your app that you're working on takes many many steps to get to so you've got to log in do lots of clicks and that can be a little bit of a pain in the butt to do with something like puppeteer what I recommend in those scenarios is to use a product called storybook jazz so what storybook does is it allows you to focus a single component onto a single webpage and then you can use puppeteer to hit the web page of storybook that just has the component that you're interested in testing so you get the best of both worlds you get component isolation and in the meantime you get really resilient tests that don't depend on any knowledge of the UI that you're using or the specific component hierarchy of your app the other thing you may be wondering about is what about unit testing so this may not feel like the kind of test-driven development that you're used to usually when we talk about test-driven development we're really looking at what some people call unit tests the problem with the term unit test is it's a very very misunderstood term in my opinion and it's something that can cause a lot of trouble in fact you'll find that you get the most bang for your buck out of integration type tests so yes unit tests are important and absolutely we will do some unit testing in the next video as we start to look at how to refactor this app and clean up the code but it's better to actually start from the outside in especially when we're talking about UI so I've found it's much better to start with integration / UI kind of tests and drive your app that way and then once you've got a solid test harness that proves to you that your app is working in the way that a user would expect it to work then you can go back start to refactor and for complex bits of code you might drive those complex complex bits of code with unit tests the other thing to note here is we are working with the front-end only so if I was building the API to go along with this front-end then you would see what might look like a lot more traditional sort of quote-unquote unit testing and taking place if you enjoyed this video I highly recommend you check out dev mastery comm and sign up for my newsletter I send information about upcoming videos as well as links to articles and other interesting insights about how to improve your code and advance your career mastery Monday is here on YouTube every Monday so please subscribe and hit the notification bell so you never miss a video thanks so much catch you next time [Music]

Channel: Dev Mastery

Views: 9,102

Keywords: devmastery, dev mastery, TDD, Jest, puppeteer, Bill Sourour, UI Development, Front end code, front end development, javascript, react, create react app, javascript testing, software development, jest testing, web development, test javascript, tdd in javascript, puppeteer google, javascript jest, ui development projects, rest api, rest api concepts and examples, javascript tdd, front end developer 2019, javascript web development, front end coder, unit testing

Id: Hw522TDxMZk

Length: 28min 39sec (1719 seconds)

Published: Mon Feb 18 2019