Complete Code Coverage with Cypress

very cool so let's go ahead and get started shall we yes awesome well thank you everyone for joining us today for another webcast this is just part of our you know ongoing series of webcasts that we're going to be doing from now on we had one of our first ones of this nature last month it was called Cypress to the net show and if you're interested to get up and running with Cypress quickly and you're not fully acquainted you can go search for that on YouTube and and get going with that but today we're going to be talking about code coverage I'm joined by my friend and colleague CLEP bomba Tov how you doing Club I'm doing axel hi I'm here I'm doing great - thanks alrighty so let's go ahead and jump right into it let's see yeah there we go awesome so today's topic is code coverage and this is going to be our agenda for the day we're first going to talk about why you even need code coverage in the first place and if you're not familiar with the topic then you'll definitely get acquainted with us today and then we're going to look at you know the state of you know how code coverage is done in the JavaScript world then we'll see how we can apply code coverage or end-to-end tests and see how a unit tests kind of add to that story and then we'll see how we can accomplish full stack code coverage as well and throughout the talk will not just be talking about concepts but will also look at real examples and I think you guys will find that really helpful another thing to note is that we're going to be accepting questions on sliggoo so you can actually go to SLI do and then you can type in this event code cyc see and here's actual link for it there you go and you can ask your questions here and people can upload questions and at the end of the talk we'll go ahead and review the questions and get them answered and then we can ask you something so we're doing this for a first time right using slide ooh yes anything specific about slider what you want to mention should mention that slide ooh is tested with Cypress there are suckers users so we're very happy to use them today so hopefully everything works well hopefully it's a well-tested app as to that awesome let's go ahead and I continue so y code coverage right why should you even care about code coverage well the thing is you know you're gonna start a new app where you already have an app and you know you've begun you begin to write tests and as you write more tests you know things get more complicated right complexity adds adds up but I'm here can I interrupt and say yeah that's very unusual thing right the claims at least to do an usual thing yes and what's what unusual thing does many things but the thing that it really does is it makes it really easy to write tests right Cypress it helps do that it's got a nice API that allows you to easily and quickly write very big tests and to end tests especially so you know you ran your app and you're writing more and more tests and you're start to wonder it am i am i testing my app enough do I have enough tests or am i over testing do I have redundant tests in my test suite that could be adding to the build time right you'll have these you have these constant questions going on and and how can you go about you know getting answers to those and this is where a code coverage comes in so you know the state the state looks looks kind of kind of like this if I can get my mouse going here so you know you write your first test you're happy and then you know you're gonna write some more tests you got I don't think things are things are viable still and then you got a whole bunch of tests and then you're you're living life like this fisherman just completely overwhelmed don't know how to handle the situation right so code coverage is gonna be your friend here I'm so glib do you want to elaborate on you know on how code coverage kind of kind of works so let's say you ask yourself are we testing all the features of a particular application in this case let's take a very simple to do and this application you can find many online so what's the feature number one for two I'm here like what do you think I want to add it to do exactly first feature is users should be able to add it to do a second future you probably want to complete it to do and the third feature would be you probably want to delete to do so okay let's start writing tests and at first we can keep track of what features we have tested manually so we can name our tests the same as a feature and then if we do everything correctly right we say oh we're actually lacking any tasks because none of a test actually is named Aaliyah to do's and we'll write one and we've completely tested our application but of course that requires a lot of discipline right but requires manually keep track of the tasks and features so it doesn't really scale I think so instead we're trying to do something else we're trying to use the code coverage as a proxy to features and it works like this when you implement feature a adding to do this you probably write some code and then you implement future B and you probably add more code maybe change some original code but now we have more code implement last feature as a sound code so instead of kind of trying to separate test by feature you say okay if I have two tests and if I somehow know which lines of code they actually execute when they run let's say mark them in green then I know that Vera miss lines of code I don't know where we coming from I can look at the code try to determine what features miss code implements and then I know this is not tested so I'll add one more test and then I'll skeet all the lines and I know that all my features tested with us so the code coverage allows us to indirectly measure all the features where we are testing and of course I'm here you can ask me this question yeah so I mean are you promising me that if I have a hundred percent code coverage that I will have no bugs absolutely not unfortunately I would say so Enver two sources of a problem here because there are two interactions one is the our code might not actually implement the feature correctly virological back bugs errors in our judgment errors in our code the way we actually wrote the code right so our features are not nestled in flamenco and second our test might be unrealistic you know for example you might we might use a particular email as an example and never test other types of emails and that leads to a user discovering a bug later in the system even if it's covered at 100% of code all right I'm here can you walk us kind of through like what's the code coverage how do we do it gotcha so you know setting up code coverage is going to take three steps first we need to figure out a way to instrument our code so we can know which statements in our code actually get executed then we need to run that instrumented code to actually you know execute those statements and then finally we need a way to figure out how to gather the results of all the code that got executed you know we need to generate a report and see how everything got executed or if something got executed so let's go and see how we can do this well the first step is to figure out instrumentation let's look at this simple example we got just you know a bunch of console logs and let's look at what this statement is actually going to do right and since you're all geniuses you've already figured out that first it's going to log out a then B then D it's going to skip seat because if equals true you know it's always true after that so let's see how it actually gets its executed okay go ahead of myself there a little bit so you know constantly console.log a then B then C will not get executed and then D but I mirror you you did this all in your head right because you're super smart exactly this is this is exactly why I'm here it's for these specific skills what about more complex examples can you actually like do it well let's see that's for instrumentations for right so how can we go about programmatically tracking the execution of these statements well we can assign owners to each statement and as the code gets executed we can increment those counters so you know for console.log a we can assign it this the counter at index zero of this index of this array which we're using to store all the counters and and so on so as the code is getting executed we're incrementing the associated counter for that line and then after we're done you know we can execute out the whole array and see what the final result was and this is you know kind of like a rudimentary coverage report and then you know we can we can run it like this and from this we can infer that we have 80% coverage of our statements because four out of the five lines we execute it actually ran and we know that because they're associated counters got incremented cool so we got some some of the initial building blocks for reporting down but we can build upon this so we can you know build a nice HTML template around this and then we can you know include our code in that inch into HTML and then we can highlight the lines that actually got executed so for example you know we can show a little nice green numbers for the ones that got executed and read for things I didn't get executed awesome and then you know the thing is you don't want to do this by yourself right you don't want to reinvent the wheel there's already tools out there that can help you with instrumenting your code one popular one is called Istanbul Jas and it's a code instrumentation library they can use now to to you be productive with it you want to pair it with another library called NYC which is a nice CLI tool you know client really for NYC and you kinda you know seeing a trend here of projects named after cities so let's see how we can use my C and Istanbul to instrument or code and generate reports cool so we go back to our console log example here and then let's go ahead and install my senior project cool and then once we run you know our project with M by C it's going to create this nice little table in our terminal giving us that same report that we saw earlier so we got that 80% of statements here it's kind of nice right it is the same numbers from like battle-tested tool exam annually it's nice it's it's good conformation good compact cool so we got this that's great but we can also you know use NYC to generate those nice HTML reports right so we can use NYC in this manner by using the reporter flag and saying a1 HTML and so on like this and then then it's going to generate this you know coverage index.html for us so we can go ahead and open that up and then we're gonna get a nice low report like this and it's going to highlight the lines in green that we didn't get executed and it's gonna tell us how many times that line got executed and then it's going to highlight the ones we didn't in red cool so now we go back to our example and then let's look at how the instrumentation instrumented code actually looks like if we go in and print out you know how Istanbul does the instrumentation will see this you know this global coverage right here basis yes can highlight it there we go this coverage global so and you can also see some other interesting things Jerry remember those counters we were talking about Istanbul is doing the same thing it's assigning counters to every statement in our code and we can also see it incrementing those counters as the code is getting executed so this is the same console.log you know if statement situation we had up there but and the bottom is the instrumented version and then if you pull up dev tools you can see this coverage global and if you see this this means that your code was instrumented and you can go ahead and look at this global coverage variable and it will give you all the instrumentation information so it's kind of cool so it always helps to know how your tools work and then and there's that awesome so Club can you tell us now how we can use you know code coverage now that we know how it fundamentally works how we can use it in our applications especially with regards your end-to-end testing absolutely and thank you Amir because what you just showed instrumentation running into a code and then generating report is exactly what you would do for your web application you would instrument your application and I'll show how you would run Cyprus test instead of executing an old program and then you know you'd have to collect with coverage save it somehow and then generate reports so how do you instrument your web application source code well the easiest thing is if you use popular transpile a bad Babel is to add one more plugin it's called Babel plugin Istanbul not surprisingly and all you have to do add it to your bubble or C file like this just one more plugin and the new code will be instrumented you probably don't around instrumented code in production though right so you probably want to insert that plugin only if you're running in test environment so Babel or C allows you to add plugins only in environment for example name test so when you start your web application when you start your bundler you just have to set node environment equals test so here's a realistic example that I'm about to show called Cypress example to do MVC Redux this is the bubble RC file it has normal presets normal plugins would apply in all environments but in the test environment we're going to instrument our code and if you have babel or sieve and it works you've pretty much any bundler right because all of them load bubble or sieve the same way so if you have a parcel and bundle a combination of a web pack and Bible preserver like it's all gonna work the same way and if you use something else let's say you're not using babel but maybe using some specific loaders from web pack but it's probably an istanbul loader right and we collected a couple examples a couple links to actual projects in our code coverage guide you can go into with section and find an example and but if example doesn't exist well let's figure out how to do it and we'll add the link there for everyone to see now this is about to change in the next couple months this manual instrumentation probably will not be necessary because once we finish full network stabbing issue 687 Cyprus will be able to instrument your code on the fly so it's gonna be great feeling because you'll no longer have to modify anything massive plugins it will become a core feature of Cyprus oh we are also experimenting with getting the native code coverage through chrome debugger protocol we've done some proof of concept but this probably will come after this so I should also add you know full networks dubbing is one of the many features we're working on that's in our roadmap and we often get you know asked what's what are you guys working on and next week we're going to be releasing a public road map for our test Runner and you all can see everything that we're working on including full Network stepping we have a very nice road map and lots of exciting things exactly so at this point I want to show how this all works in practice right so I'm gonna switch to my terminal and I'm gonna run a little to-do MVC application so as you can see if know the environment test so my code will be instrumented so once we store it I can reload as I have my to do MVC application I can add things so how do I check but my code coverage is where if I open death tolls I'm gonna zoom in a little bit I'm gonna check if where this variable coverage and it's present right I can see all the source files source files in the single object if I click on individual one I can see the statements exactly all the counters like we've seen before so if my window has this underscore coverage object but means my code has been instrumented correctly a good thing about bubble plug in Istanbul is that it generates source Maps so for actual work and debugging map it doesn't really matter if it has been instrumented or not let me switch to Cyprus so I have the Cyprus test right here right I hope everyone can see it so this does just checks if I want to add if I can add to this I'm going to visit the page it's a local page I'm gonna find input box with class need to do and I'm gonna type free to those and then I'm going to check if we're really are free to do it when I run this test I want ensure that mines code has been instrumented inside Cyprus correctly right but I haven't lost something so the way you check you open dev tools just like before you look at the console but if you check a coverage object it's not gonna exist and the thing is cyprus runs your website in an iframe so you cannot check the top environment you have to switch to your application and then the coverage will be there this is a little technical detail but who knows usually stops a lot of people from finding the right thing switching the context so you can actually expecting inspecting your application okay let me so we checked whether application has been instrumented and we're gonna have all the data right but here is a problem right how do we like we're actually getting this coverage information back to generate reports right what do we do physically so I'm gonna switch to representation right so we have our coverage information inside the application in order for us to actually save it and generate HTML reports or any kind of reports we have to use a plugin so we've written a plugin called Cypress code coverage okay so this plugin takes what object right but gets updated during the test that has all the latest counters and it saves it correctly you know because Cypress resets the window object so you have to save it and then merge it so code coverage plugin does it all for it so the way you use added to your side you add the plugin you add NYC because we're using NYC to generate reports and you add one more peer dependency to actually merge coverage reports and then you have to add code coverage plugins to your support file and to your plugins file if this is kind of standard thing for adding any type of custom commands okay so once you do that all right and I have done this in my application notice when when I'm running this for a couple of additional tasks reset coverage combine coverage combined coverage and coverage report so this tells you that there is something going on right that you know various coverage information passing back and forth so this comes from the plugin and I ran this test and the coverage information has been sent to the backend has been saved so what does get generated so if I see next to my Cypress folder but is also a coverage fault that generated so where a couple files were so for example this clover XML it's a coverage file but this is actually information for each statement of files covered by the test this clobber and this al-kahf are two machine formats that many third-party coverage services expect so these two reports are good for sending reports somewhere the final coverage numbers are saved in coverage final JSON so this file can be used by NYC tool itself to merge you've more you know reports for example the most interesting one is this alcove report which is the actual static HTML alright so let me open this one open coverage alcove index so this is HTML port of all the source code but has been executed by our single test right here okay so you can see the folders the subfolders and so on now notice a very interesting thing even though we ran only a single test right it's actually covered already 72% of all our statements all of them awesome I know it's pretty interesting how effective end-to-end tests are at exercising all your components all the sources because you're not marking anything usually if you run in full end-to-end test so in this case you can you know kind of drill down into like where with beings are missing right and well let's kind of see how we can use this report by the way before I can continue let me just show you an example of a single file so this file for example was only covered at 30 percent you can see the counter of each statement and how many times it was executed by that one test in yellow you see the statements and branches not covered in this case the switch statement deleted Oh edit were not executed so we never actually tested those things the read other statements not covered by our tests so in at this point we can ask ourselves what can we do with a coverage and the easiest thing for you to start with is to take that coverage information and use it as a guide for writing tests so let's say complete to do wasn't covered so let me go back to my test write take the first element fine toggle button I have to toggle and check it so as soon as I change my test Cypress test rerun so it checked where free items found found the first item in my list found the toggle element and checked it so I can see what my application did the right thing and if I look at the coverage and reload this file boom I just hit both source lines so I tested had to do three times right because I added three items and I completed one of them that's why the counter is one and so my coverage tells me exactly just find the statement but wasn't but thread figure out what end-to-end test would actually reach for a statement and just add the test so how does it look in practice well instead of this one test I can run the full spec which is kind of a standard to do MVC a set of tests move very test for adding to dues for editing to do marking them completed deleting to dues so pretty much all the features so let's see what the end run tests give us in this case I mean do you think I'm gonna hit 80% we were at 72 right I think you're gonna do a lot better well let's see I'm gonna go back to old files I've barely missed it nah that's not good enough that's that's unbelievably bad okay so at this point I just want to say but we covered you know what the code coverage is and the end-to-end code coverage but you saw but line right like so what is my coverage report yes so but everything is at 100% except for this one particular line or edge case right here you always have those always right so what is this right so if I look at my application and if I add an item where are all these selectors for what you want to see right so if I have a couple like one item right now I'm looking at all to do items I can filter and only show active items or complete right so my UI only is wired to three different states and these states are covered by test but this one each case cannot default can liquor no sanity check accord is unreachable from my UI right this is just a good code so it frozen error when you're trying to you know show invalid state but my end-to-end test cannot reach it so what do we do in this case so we can write unit tests and we can write any test get code coverage and hit that line so there is the hard way in an easy way so the hard way is to take a unit testing tool like just Abba maka add code courage there you know it comes built in with just for example but then you run the unit test saving it as coverage ran your end-to-end tests with cypress you get end-to-end coverage and then any AC tool has an OEM Norwich command where you can take several JSON files with coverage and combined into a single one and I have an example in in this particular repo but that's a hard way like who wants to do it I mean do you want to do that not at all sounds like wait that's not that's not the safest way not the cypress way so the cypress way would be run unit tests and end-to-end tests with cypress right so you will get combined code coverage so here's how it looks in practice in this particular case is that you want to take your spec files any one instrument them as well not just your production code but gets loaded but also your spec files but will run unit tests and all you have to do add this one line but comes from Cypress code coverage plug in into your plugins file so let's see how the test looks my little test right but I've written for this particular case is this so this is the unit test for Cypress notice how it actually imports with what function get visible to do this right from selectors from application service code and then it just instead of loading the page just execute a single expectation so it's purely unit test so I have my Cypress running here and I'm gonna run this particular task and the interesting thing about this but test runs but there is no no application right we're not loading any you know we're not visiting any website I can confirm that my source code has been instrumented so if I switch to a iframe where my tests live because they're isolated bring the coverage object so we did instrument our spec files correctly so our new tests will provide code coverage information and now if I switch to my code coverage report I will see only the selectors file and if I drill down I hit just well particular line right my unit test is very very very well targeted at that particular line and the good thing is that I can go back and run all the specs right and so I have a combination of end-to-end tests and but one unit test what I really needed to reach that particular edge case deep inside my code and when I run all them together I mean what do you think are we gonna get to nine nine nine like six nines I believe in your globe I think we can do even better than nine nine I think we're gonna hundred we did get it yeah but it was actually beautiful so we did hit 100 and this is the cypress way and once we have a way to collect code coverage information from end-to-end test or combination of end-to-end and unit tests it's not just the map it also functions as a card so what do I mean by this you can fail you built if code coverage drops below a certain threshold and it's built into NYC report command in this case I'm saying MSU report give me a tech summary and also check the coverage and if it drops below two hundred percent for statements which is impossible standard right because public have one hundred it will just exit with error code so this will stop your build so if you for example said whatever threshold you want ninety percent seventy one percent is too low ninety percent 80 percent your build will guarantee that your source code has been covered by certain percentage before it's passing a good idea is to take your code coverage report that HTML folder and store it and send it to store it on CI so for example we ran our continuous integration in this case on circle CI so after the end of a test I'm sending this whole coverage folder to go as a static artifact and then anyone can drill down and see if we covered or uncover something so we can discuss it as a team and at some point you will reach a limit of just like keeping track of all the coverage information a cell so in this case we can run our end-to-end tests on four machines using our dashboard precision service so which means that each of the CIO machines will only run a subset of tests right and that means that each machine will get only a partial code coverage information right so what it in this case right how do you combine it to get the complete picture best well you know third-party services like coveralls code curve code as you come into play you can send all the partial results to host services and hopefully they can merge those results correctly now I had some mixed successful vet so in this Cyprus example real world link what you see below I actually show a way to combine cough coverage results yourself but good thing about what services they can show you but coverage results really really well so that the whole team can see it and especially how it changes over time so example in this case recovered Rose I can click on this build or is it just not let's look at this one right I can see where the coverage dropped right which particular file right and drill down and see what this is a new line that hasn't been covered with a test so I can see easily with changes over time to my coverage information and another thing well third-party coverage services allow is that they can be a pull request commit checks which means that let's say you add a new feature you make a pull request before party service like coveralls can't comment on that pull request and say yeah coverage remain the same so you know but you like your safe you did not change something so poor requests with the same code coverage or higher are really safe to merge but let's say somehow the code coverage dropped this is a fail check on my pull request which says the code coverage decreased by almost 9% to 91 and in this case you don't merge your pull request and instead you ask yourself why did it go down did I add new features and I did not write the test did I accidentally skip some test for whatever reason right so what happened did they delete some test and so investigate and you know you determine if it's a real cause or if you have to actually update your test common error or common source of drug code coverage II accidentally leave it that only in your source code and so we wrote a little utility call stop only where is also easily in plugins but stops you from competing of pushing tests but have exclusive tests but to me what code coverage provides is this nice badge but it's not next to my CI badge but says 100 percent or 90 percent right and as long as I know that I covered the most important features of my application and it doesn't accidentally go down like green CI and green code coverage gives me a lot of confidence in my code as I add new features as a refactor code but we have another surprise right Amir we have some so you know so far you've been showing us how to generate all of our coverage reports for our front-end right but um but you know that's not our whole app we also got a whole nother side tour to our service which is you know our our note backends you know if we have a node so how about you know combining both front-end reports with our back-end notice Amir can you come in right yes yes I lost sound for a second gotcha gotcha but we were talking about how to combine our front-end and back-end code reports and let's coverage reports and you do it the same way as you do it for front and you have to instrument the server code and if you use an OSC just like we did before it's really easy to do so usually you start your server saying node and server write some just file if you want to instrument your code and please do it separately so it's not in production code you say NYC node server so you literally just add and we see in front of your server and they use silent flag so doesn't know to the report because we're gonna grab a report separately what's it so now when your server runs it actually has a global coverage information and but information is just sitting there so you have to extract it somehow so in this case I have a server so the only thing I have to do is add one more middleware and this middleware again comes from Cypress code garbage plug-in and this middleware just as an endpoint so but the code coverage plug-in can grab results from the server from side so let me show how this works in practice I'm gonna start this is a Indian brand their coverage so I have this real world app application it's an example of a blogging platform this has a real back-end real data base authentication writing blog post and so on so this server server started with Cypress everything is instrumented the backend and the front-end so I'm gonna run one of the tests let's say like new posts spec so this logs into a user the user can write blog post as you can see can comment tag things it's a lot more realistic application than gesture to doing this year I'm just gonna wait for it because I actually like watching Cypress how it drives the app it's very soothing mesmerizing sometimes yes ok so we just finished bunch of tests but exercises adding an article so I don't need this thing I'm gonna open the same folder but now look at the report and look at the file names it has client which is our front end and it hole also has server all the routes all the models all the files that the server actually executed right in response to the front-end adding new items following a user and so on so one of the things that I wanna show in this case is this server models folder it has an article right so when we were adding a new boss we're adding an article so everything as you can see actually executed on the server and this of a typical edge cases that you will see in your server code for example we did not really exercise an edge case when somehow the title of a post is undefined right because our front end did not allow us to add we also missed an edge case where the slug for an article was determined from a title he's already taken so I have to add a test probably that adds an article with the same title and checks that a new slug is generated and doesn't override the old one this is a typical thing word I can't even write a unit test for this case because this is completely back-end code right so this would be something else but you have to think separately if you want more information about everything we've seen from today we've written by code coverage guide is part of our documentation it explains literally everything we went through today from how to instrument your code to how to use code coverage as a guide how to combine end-to-end code coverage and unit test coverage and full stack and all the examples that I talked about they are all are there so go to Cypress that are your code coverage and the plugin is open source it's on github if you want and if you have any feature requests please or issues please open the issues in the repository and we'll be happy to do more and if you want to stay up to date with our latest features and releases follow us on Twitter go to our blog and so on and as we get more examples we're gonna put links to our code coverage guide we especially interested in user submitted examples that show for example angle and typescript or you know swell tail or a rollup bundle up if you have examples where you successfully instrumentally to your application code and got useful information please send us a link or open a pull request we'll be happy to add the link so other people can see it so I think before we go to questions and answers I mean we covered pretty much everything right but we set out to cover yes we did you know we talked about what code coverage is and why you should care about it we saw have works underneath you know underneath the hood then we saw how javascript kind of answers the code coverage questions with tools like Istanbul and NYC and then we saw how we can use those methods in our end-to-end tests and you know trickling some unit tests whenever necessary and then we also saw how we could utilize the reports of our back-end and merge them in with our overall coverage report and to get the bigger picture and we saw that using code coverage services like coveralls and coda C really helped with that and so I think we we told a pretty good story about what what code coverage is and how you can use it today to get better insights into the coverage of your apps if you have any questions now let's go ahead and get them into slide ooh so again you can go to Slide do /cy cc or just go to SL IDEO and type in this event code and then dropping your questions and you can vote on other people's questions as well and then I Club and I can start answering them for you so I'm here do you are you ready for me to stop sharing and you taking over sure thing sure thing I'm gonna stop sharing all righty so clever are you seeing the questions that are coming in I do see the questions Wow we have some good questions yeah I think so alright well you know you guys are voting on all this stuff so let's just you know use democracy to our power let's look at the first one so Damien hopefully I said that correctly is there any way to use this to see redundant coverage like if there are too many tests that are wasting time testing similar or overlapping functionality it's excellent question so the code coverage plugin grabs the coverage after each test because Cyprus clears your window context right so the coverage you get from end-to-end tests is actually ties to particular tests which is excellent because I have a PR open but actually has a test to coverage mapping where you can see which particular lines of source code is each test heating so we're still exploring how to actually show it and how to use that but you can definitely see how the tests overlap and how much of redundancy varies so I'm will open to any kind idea of how we can take advantage of it information great alrighty let's go to the next one maybe you can answer this one can we use this mechanism to measure code coverage for single page applications or a system with multiple micro-services well you know what we showed you is yes it of course does apply to single page applications and it's completely you know framework agnostic it doesn't matter if it's view react angular whatever you know we're just testing JavaScript over here now with your point about micro services I mean I'm assuming these micro services are distinct individual systems and apps so you would need to you know applied them discreetly to every single every single one of them and so because their code bases if they've they don't cross over in their code base right I'm assuming these are distinct separate repos and don't have any overlapping code which is how micro services typically work cool all right Gleb so what's the typescript story for the code coverage plug-in let me see something so code coverage I'm trying to check if I actually ship typescript definitions with my source code and I don't think I do I'm really sorry about that there are no types right so the thing about code coverage plug-in it it it doesn't provide any user-facing commands it's not like it adds side that cover or side that code coverage so that's why I didn't write type script definitions but I will because it's a good ideas about your target compiler knows about all the Cypress code and it doesn't scream at you so I have to add that yeah and and we're big fans of typescript here at Cypress and you know this plug-in is in its early phase and it's only gonna get better from here okay so what happens when I end to end tests are written in different repo from the actual code with Cypress code coverage plugin I guess apply in that case Club right it's excellent question so what happens in this case is that your source code in your deployed application doesn't have instrumentation right it doesn't have a coverage object at all so in this case you have to either wait for full Network proxy supports but so with Cypress can instrument your source code on a fly or I've written code coverage proxy as a third-party NPM module but you can put in front between your application and Cypress and that proxy can instrument your code on the fly that project is pretty old I've done a long time ago but it should work so actually should add a link to it to this as an example because it's actually a cool example it's excellent question awesome so we'll drop in a link for you all yes cool all righty so and I've seen this kind of questions come up in the comments too and you know it is you know with all this stuff regarding to our back-end reports applied to backends that are not written in JavaScript so for example Python and what we show today was just for node right that's right so Island I don't know what would actually let me go back as long as you can produce the code coverage information from Python in Elko format I believe third-party Services can show that can merge it and can merge it right so as long as you know how to instrument and grab a code coverage information from your language you'll be able to just send it to your code coverage or merge it with Cypress provided code coverage and you should just work and this is another one of the benefits of those cover coverage report services right they kind of abstract this extra level of complexity away I'm so you don't have to worry about it it's all look the same like how many statements did you hit and did you hit this one statement so exactly all right so do you recommend using Cypress as a separate project to test an application or add it as a depth event dependency and use it within your application now the way we recommend to use Cypress is to add it as a dev dependency to your you know current code base so if Cypress lives with the rest of your code and it gets committed to to your version control with the rest of your application and this also makes it a lot easier to import your code in whenever you need it for you know maybe a specific unit tests or so on and this is especially helpful if you use our component testing libraries that we have out there where you need to you know import in this you know your actual component code for them to be tested so we recommend you include your tests with the rest of your code base so we do see people have a completely separate repos for their code base as well especially if you try it out right yeah yeah okay so is there a way to use code coverage to identify which api's are getting fired as the end-to-end tests run and I'm assuming they're referring to Network API HTTP API I don't know what this question might entail because what is an API endpoint so 18 points and keeping methods like get past yeah so dear anonymous maybe open a github issue for code coverage yeah it may be tweeted us we've and we can talk about more details it's interesting right yeah it's interesting to know which which API endpoints you're hitting but remember we're talking about code coverage right so you were you were now looking at although all the API endpoints you have for your system and seeing which ones got hit or not you know that's a completely different coverage right story so we're supposed to be talking about coverage you know where we're looking at which statements in your code actually got executed right yeah okay so let's find the next one so does the Cypress Co coverage plugin cost or add to the runtime I guess the question moves around a little bit we're gonna add to the runtime and I saw this question kind of being redundant a little bit so let me just rephrase it so we we can kill a couple of hours of one stone so you know we have to instrument our code which that's going to add some additional code to our code right if people see that as overhead is that going to slow down or code glove it will unfortunately there is no way around that you're executing additional statements these are very simple statements but we still add more JavaScript more processing time in addition after each test but code coverage plug-in has to actually save that information was it will get destroyed so it has to extend it to disk merge it with existing code coverage that adds time ultimately it's a single request before and after the test so how much can it add right unless your application has a giant code base so but coverage information is ginormous but you know we usually advise to paralyze your test and then you will not notice yeah I I don't think I I think the benefits you get from code coverage completely outweigh the you know the minimal overhead that it adds to the execution of your code I remember you're only instrumenting your code during test time we're not saying you do this in production right so you're still going to get the actual performance that you've always had in your production systems all right and I really like this next question well the coverage reports be visible on the Cypress dashboard well you're kind of reading our minds right now and this is a little bit of foreshadowing of what's to come so be on the lookout yeah at this point we might as well do it I mean since you asked we'll do it just for anonymous yeah ability to turn on and off code coverage with flag to run tests faster I mean it's you you you decide to run NYC right when you're executing your code so you can you know conditionally in CI or whatever mechanism you have in place choose when you want to run code coverage or not so this is completely up to you right can you integrate unit test coverage with end-to-end test coverage yes that's one of the things you know Club touched on was you know he wrote you know a unit test that got him really far in the coverage report right you know initially you got like the 76% coverage and then God found that there was one route in the logic which was untouchable by the applications UI so he went and wrote a unit test for that and that gave him increased coverage so yes you can definitely do this art glyph so I think we already answered this one so we can skip that will you share the video presentation too in the docs as well yes we will everybody will get links to everything you saw the slides and the video recording so no worries is it common to merge test coverage reports from end-to-end testing with ciphers and test coverage reports generated from units and integration test Suites yes so this is kind of similar to the question that was asked previously you can totally cover the end-to-end test coverage reports with whatever other unit testing library using like Jess or Ava or mocha independently and so on we can easily merge that in with NYC and also those code coverage services helped a lot alright let's let's get that one checked off so is it common to merge test which reports okay we are you read that one huh have you tried this coverage code code coverage plug-in with minified je s there's a work is expected so I have not tried to minified the thing it should work minified javascript is still valid JavaScript so it should be instrumental but the information you get back unless you have source maps is meaningless right it doesn't map back to original source code so we right now looking at all the different examples like for example instrumenting typescript as long as your typescript transpiler provides code kalloch source map when the ýstanbul should also correct get word source map and merge it so that it can correctly merge back to original source with code coverage information right so the same should apply to minified as long as you have code coverage information but we can only prove it by actual example not just by words gotcha so source maps are the key there also so let's check that one off all right so best practices for testing a client-side only app example an app that uses third-party api's and no back in locally well hey your life is a lot easier because you don't have to worry about merging in unit tests from your back-end you can just use the Cypress code coverage plug-in and only apply it to whatever your you know client side app is will angular be support it of course we are completely no framework agnostic when it comes to the front-end totally supported automatic Network stuffing is just all kinds of cool I completely agree what is your current guests on a timeline for that to be available well I can't share timeframe at this time but it is currently being worked on as we speak please check out that roadmap I mentioned earlier that we will be releasing on next week and you can actually give us feedback on that roadmap site too so please share your thoughts there as well alright so how would you go about testing of components go out I think we know what the answer to this one gonna be code which is reused a lot and time and testing in each place brings overhead mocking its inputs might miss bugs club you want to touch on this one we we do have a way to run your component from different frameworks for specific adapter in Cypress which is kind of nice I actually like it a lot instead of visiting a page you just say give me this component and it appears live and you can use the same API I haven't created an example where you actually do a code coverage for a component like that but I don't see why it wouldn't work so follow us on Twitter and on what fall of a block boss and we probably will tweet the link as soon as we have an example yeah and one thing I should also mention is that you know we're actually big believers in component testing and so you know one of the things you would like to add to Cypress the core of Cypress is component testing itself where you would have this ability to mount components from any framework and then will automatically figure out how to bootstrap that component and allow you to test it in isolation cool so let's go into the next one so how to add stubs for any page or new link we want to visit I'm not sure I completely understand that question ah but if you're talking stuffing service can handle it is there a way to identify routes there aren't actually getting mocked in the code coverage report would be helpful to get rid of those JSON files so no because we only talk about code coverage right this is like we have no idea of what API is where are we have no idea we take care outs are called it's unfortunately not okay does unit tests does unit test can be created within an application and without changing the plugins can we have another session regarding unit tests rather than end to end well the thing is you know the same concepts apply regardless of its end to end or unit test right we're just executing JavaScript you know we're gonna list of what type of tests you have they you know the methods that we just showed today using Istanbul NYC the Cyprus code coverage plugin it all still applies remember you can completely use Cyprus to do your unit testing no problem again it's just JavaScript I I just want to add one asterisk as long as the code you're testing is actually browser code right if it is in the browser if you're doing file system operating system database request that code has to run in node environment and from that you can use side tasks which is not as easy as unit tests yeah so you know yeah with this all within the context of you know things that get loaded in the browser so your friends from foreign code alright I think we're almost out of time right almost at a time alrighty so let's just do a quick look so we only have a few questions left and don't worry we're gonna we know who submitted what and we can we follow up afterwards so we'll be we'll be in touch and again you can reach me or gleb on twitter or feel free to you know email us at Cypress we're always glad glad to hear from you all and to help you out with that said I think that concludes our our webcast for today club well everyone thank you for joining and I'm you it was a pleasure pleasure is always a club thank you so much and thanks to everyone that joined the call today appreciate it take care everyone have a good one everyone

Channel: Cypress.io

Views: 25,171

Rating: 4.9670782 out of 5

Keywords: Code coverage, Cypress

Id: C8g5X4vCZJA

Length: 60min 43sec (3643 seconds)

Published: Wed Jul 31 2019