Automated A11y Testing with Cypress - Rachel Noccioli - Angular Lunch Oct 2020

welcome to automated excel accessibility testing with cypress i'm rachel nacholi i'm a software developer at oasis digital and an angular bootcamp instructor so today we are going to cover um a little bit about what automated accessibility tests are why you would want to run them how to set them up with cypress the the underlying tool that we're going to use is something called ax core and that's not unique to cypress but i'm going to show how you can use it bundled along with cypress because that's probably one of the most common ways that it would be used with angular talk a little bit about what kinds of issues the tests will and won't find and then some resources for fixing the problems flagged by tests all right so a little about um just accessibility in general this isn't a general talk um but um accessibility really is just about making sure that the the products we're building are able to be used by everyone and it it usually calls out disability but it really just means like being inclusive of um the the way that people will interact with the products and so there are a lot of different like keywords that have to do with accessibility um the like these they're like there's like a spec and um a bunch of different things that you might want to test and different um like assistive devices but we're gonna focus just on the the tests that we can use um that will be automated so the specific ones that are going to be able to be run in the browser to get those um those easily um like the ones that can be checked um easily but that's not going to make sure that your whole app is truly accessible it's just going to be one piece so there are a few methods of automated of doing automated accessibility testing and the first two on here browser extensions and built-in tools those are kind of the more manual ways of doing automated testing where you have to actually spin up the browser open an extension or just like kind of open your dev tools use something like lighthouse to run some tests and then you get a report and then you can go um and update your um your code accordingly and then the last two are using those automated tests but in your unit tests or your end-to-end tests so um if you do it that way then you can make sure to run your your automated tests every time you run all of your other tests you just kind of build that into your normal ci cd pipeline and um you can be assured that those are going to run um like however often you're running all of your other tests like hopefully before you're gonna merge something into production you're running all of your your unit tests or your end-to-end tests and then you're also gonna be running these automated accessibility tests okay so um why would we want to do that um of course this testing is um really similar to all the other kinds of testing that we do we want to catch errors early before we push them to production um and so we can make sure that the users of the application are going to be able to use each new feature that we push we can it can help us to catch errors often and so you can write tests and like um if you're doing it manually you might check manually that everything is working when you push a feature but if someone comes along and fixes a bug let's say they might not realize that they're missing a particular accessibility attribute for an element in in your future and so having this like incorporated into the ci cd process is going to to make sure that these actually get run over and over without needing to rely on every individual developer um running them and then the um the last thing is that it can help you to learn about errors that you weren't aware of so if you're adding in this tooling because you haven't really been doing a lot of accessibility testing or you're onboarding a new developer who is not um doesn't have a lot of background with like any with manual accessibility testing then if you run the tests it's gonna show you some errors that you may not have known and then you can kind of go research those errors and um hopefully it'll help you to just write code better the next time so if you didn't know that you needed to include a particular attribute on an element then the next time hopefully you'll just remember to do it so it can help you as a kind of a training tool as well okay so the um the uh the underlying tool that we're gonna be um seeing today is called ax core and this is kind of the way that the organization that created it describes it so they say diy accessibility testing with the ax core api and i mentioned that we can do accessibility automated testing with unit tests um so there's like a a way to use this with jest and we're gonna see it with cypress but it's not um unique to any of those particular testing tools it's kind of its own um underlying thing and it also has its own browser extension so there are multiple ways that you can use this ax core tool and we'll see one of those today okay some limitations are that there is a narrow scope of problems that can be detected um so for example the um the texts might show that you didn't include alt text on an image but they're not going to tell you whether that alt text is appropriate for the image right so you will need to pair your automated tests with some manual testing we've also hit a few issues with third-party libraries uh and in general if you're hitting a lot of issues with a third-party like component that you're using you might want to consider um like looking for other alternatives but if there's you know just one particular issue there are ways to kind of override particular tests for particular components so we'll see how you can do that and then just lastly the automated testing is not going to ensure that your site is accessible it's going to do one part and it's going to do that part over and over to make sure that um it's it's checking um as much as it can but you still have to do manual testing and specifically um you'll want to be testing like with assistive devices so maybe with a screen reader the automated tools are just going to run in your browser okay so when should you add automated testing so like many uh code quality or app quality tools it's going to be easiest if you start at the beginning of your project before you've written any features before you have a whole bunch of mistakes um and um usability problems but we're gonna see some strategies for adding the tooling after you've already started so just because you're not at the very beginning of the creating the app there's there are ways to gradually introduce the automated testing without feeling like oh well if i add this it's going to break all my tests or i have to spend like six weeks figuring out all of the all of the errors before i can even start adding the tests okay so we're going to see a demo it's going to be an angular app that was built with nx um using cypress for the end-to-end text end-to-end end-to-end tests um ax core that's the main part again of the automated testing and then cyprus acts is a wrapper for axe core that just lets it um work well with cyprus so um when we want to add cyber's acts to an app there's going to be about four lines of code to get started um you're going to need to install your dependencies so that's going to be ax core cyprus x and app types cypress x then in the support index ts file you're going to import cypress axe and that's going to get get access to the cypress axe library it's going to add some additional methods to the cypress object and then in your test files you can call psi.injectax so you'll have to do that after you've visited a page so after you call sci.visit you can call psy.injectx and that will bring in the axe core functionality into the into the browser and then you can call psi.check accessibility so this is the the most basic way that you can add these tests and very likely if you add this to an existing project you will find quite a few errors so those errors are going to look something like this which is really just any cyprus error so if you've written cyprus tests this should look familiar and we can see here that we have an accessibility error and then it gives the id of that error in this case it's image alt and then on one node but what does that mean um if i click on the name of the error in the um in the like cypress output it will log a lot of information about that error to the console so in here um we'll get a slightly longer description of what the the problem is so in this case it says ensures image elements have alternative text or a rule of none or presentation so that's just a little bit um a little bit more helpful um but if you still aren't sure what what that means or how you can fix it you can click on this help url and that will bring you to the documentation from axcore okay so let's um jump over to the code and i have already run the yarn ad so i installed the dependencies and then i'm already running i'm already running the end to end tests so in an nx project the typical way that you run those is just yarn e2e dash dash watch so that we have the end-to-end test running here so this is the cypress app i clicked run all specs and so i have the um the end-to-end tests running and we just have one and it's just checking that the header gets displayed okay so those four lines of code um we did the first here the yarn add then in here in the index ts of support we're going to say import cypress x so this is the cypress accessibility etv that's just my my nx um project and then source support and then index ts and then in my spec file i have my one test and then in here i'm going to add another describe and i'm going to say this is just going to be called accessibility and then in here i can make a new before each so before each i'm going to side dot inject x so now we have in we have acts loaded on the page and we can call um psi dot check accessibility inside a test so we can say it should have node detectable accessibility errors on page load all right so then we call psi dot check accessibility and let's come over and see the app okay so this app um is really i it's something that i had for a an angular material workshop that i taught um so it's mostly angular material components and angular material the angular material team takes accessible accessibility really seriously so they test all of their components with screen readers and with other assistive devices but there are a lot of places where you can still make mistakes so i've put a few mistakes in here and um oh no gotta undo that i fixed my mistakes okay let's let's rerun these so yeah so i added a few um accessibility mistakes to this app so that there are things for us to fix and to talk about um cool i don't know why it's not showing up there should be six mistakes okay now we got it so there are six accessibility violations the the test is expecting six to equal zero so it shows us all of the accessibility errors um and then again i can click on one so i can click on button name and then i can see some more information about that so we see that the impact is critical the this rule is supposed to ensure that buttons have discernable text and if i need more information about that i can click on help url so this is going to bring me to the the ax core documentation and it's going to show the all of the the um the rules for this um for this accessibility check so it's showing here what the correct markup solutions are so a button can have some some text content here and then that will tell a screen reader what to um what to read when the user is active or when the user focuses on the button and um if the if the button doesn't have any um text content we have a few different ways of alerting the screen reader about what it should um what it should say so here we have aria label and so um arya is an abbreviation for accessible rich internet applications and that's a spec from the w3c about creating accessible websites and so we can just pass the aria label attribute and pass a string here so we have a bunch of other documentation about different ways that you can handle different situations and um some more like narrative stuff about how to make your apps more accessible um but for just a moment we're we're not going to fix this one right just yet we'll fix it um in a minute but i wanted to point out this impact critical so the each of the um the acts the acts core rules has an impact and we can we might say at the beginning okay so we have six errors that might not be too bad um we could probably fix those fairly quickly but if we had say 600 errors and we had a really big app and we weren't going to be able to get all those errors fixed we might want to focus on the most important ones so in this case we want to actually fix this critical error but we might be less concerned about about the rest so let's just go over to our spac file and um in our check accessibility method we can pass some configuration so by default check accessibility is going to check the um the whole dom and so the first parameter is um lets us override that so in this case we just want to say null and then we want to pass a configuration object so we can say included impacts and in this case we're going to say critical okay so that's going to rerun our tests and we should find that we have fewer errors because we're only going to be checking the critical ones it usually goes faster than this um i'm just gonna oh okay i think with the screen share and the video and everything it's taking a long time all right so now we're down to two accessibility violations so we have we still have that button name violation and we have the image alt violation so we've narrowed it down to just the worst um most crucial bugs to fix so let's go over to the html and fix those problems so on the button i can add an aria label here and the button is this button so i'm going to call it the user um user settings menu or user profile menu all right so i give it the aria label and if we notice the the content of that button is just an icon so that doesn't really give the screen reader very much information about what to what to read out but now we've told the screen reader to read user profile menu then down at the bottom of the page we have this floating action button and it just has a plus icon and so again that's not really going to give the screen reader very much information so we can say aria label equals add astronaut um ad astronaut so we don't have to say that it's a button because it's going to it's already going to have told the user that it's a button all right so now if we come back over to our cypress tests the the app is reloading but if you've used cypress much you know that the tests aren't going to get rerun so we just have to click the reload button so we'll let that go and now we should have just one accessibility um one type of accessibility error so the other error that we saw was the alt text on the images so if we come into the code and we see our image tags or our image elements we have a source binding but no alt text so in this case we need to bind to the alt attribute and we're going to bind astronaut dot name so here um it's we're not binding to the the alt property but we're we're actually binding to the alt attribute so we use the square brackets adder.alt all right so we'll save that rerun those tests notice it it had said image alt on 50 nodes and so if i run the tests again then we should see no accessibility errors all right so all the tests pass but of course we've we've scoped it to just the most serious ones okay so this app um also has this floating action button to add an astronaut and if i click on it then it's going to pop up this modal and checking um on the modal we kind of have two options we can scope our tests to just check the the modal itself or we can have the have cyprus x or x core run all of the tests on the whole dom so let's see what it would look like to have it run on just the um just the modal so in that case um we're gonna improve the performance of our tests because we're not rechecking all of the dom ever for every single test it will help um to make the test run faster but you may be um potentially not catching all of the errors so it's kind of a balance there all right so it should not have detectable errors on the ad astronaut moodle oh no all right so we have our test so we're going to click the button and then we're going to make sure that the astronaut form exists and then we're going to check check accessibility but if we just run it this way it's going to check the entire app like we've seen before so we can pass it an object with an include property and then the include property is going to take a nested array of the css selectors so in this case it's the app add astronaut so now we're just going to check the modal itself to make sure that it doesn't have any errors okay and so the modal itself doesn't have any errors so let's try um adding a test that does actually check the whole page so it should not have detectable errors after opening the add astronaut modal and then we'll take this configuration out and actually i'm gonna just run this one test okay so it's checking the whole dom to see if there are errors so it turns out there are two accessibility violations so um in here we have this aria hidden focus and then landmark one main and the landmark one main was um a violation on the the page as a whole that we've um that on when we tested the the page as a whole um we told it okay only test the the critical bugs so we might want to make that a little more generic so we don't have to say it every single time we we run the test so we'll see how we can do that um but this aria hidden focus is a problem with the angular material modal so unfortunately angular material does have an open um accessibility bug with this aria hidden focus because they hide the content of the page from the screen reader but it allows it it doesn't um it it allows the focusable elements to remain focusable so um we can see how we can um in here we can override that particular test so in this case we're going to pass null because we still want to check the whole page and then we're going to pass a configuration object for rules and rules is going to take that id so arya hidden focus and then whoa enabled false oops all right so we have a few uh nested objects here so we've got this object with the rules property that's an object with an aria hidden focus property and then we have an object with the enabled false property so that's just going to be how we can handle if we have a third party tool that's not that might have some sort of issue we might have to um come in and disable a particular rule for um in a particular case but again um if you have a component and you have like 17 rules that it's breaking that might be a reason to consider looking for other options okay and then one last thing that we can do is set some global configuration and so in the support directory i can make a an axe configuration dot yes file and in here we can create this default ax configuration and we can write the rules that we specifically want to check or not check so in this case we've um or i've uh written a list of just one rule and so i said the image alt rule is the only one that i want to check and so i set it to enabled true and then i disable the other rules so if you're adding this to a project that has 600 errors you can kind of pick which ones you want to tackle first and enable the ones that you've that you've caught and then disable the rest as so you can like kind of build up um your list and then once you get close to being done you might switch it to um a disabled list so in that case you would obviously just take off this property and put um disabled on the ones that you did not want so the default axe configuration we would just bring that in after we inject ax we can call psi dot configurax and pass it the default configuration okay all right so every time psi gets injected or sorry every time x gets injected onto the page we configure it with the rules that we want to apply okay so um just a recap of the demo um getting started with cypress acts is just those four lines of code that we saw um you can kind of phase in your rules with either the global config with configurax or for each test with check accessibility um you can override rules if you really need to um but just be kind of judicious about that and then i've um uh you can check out the repo and i have a um uh try it yourself branch and you can um try it's like installing axe um and cypress acts and like fixing the accessibility problems um yourself and like you know you might not um want to disable the rules but you can like check out um try it different ways all right and then i just have some resources um i gotta i have to actually push so just hold on just a minute for the repo um and these are links to the the two libraries and then just some general resources about accessibility that um are useful if you have just more general questions okay so that's all i've got any questions thank you that that was great uh as always i have like 50 questions but i'm going to try to let other people go first so um go ahead everybody okay people aren't fast so i'm gonna jump in so like that seems like such a huge win to use this like it kind of makes me ask like should basically every angular project use this as just the first cut at getting some basics right i mean i would i would think that was probably a good idea yeah i mean like this is and again like this is one tool there could be others um but but yeah like doing this kind of testing i would say there's not that high there's not that much to to learn to be able to to get it in there um and because you can kind of phase in the rules you can tackle um tackle the ones that are going to be the most severe and then kind of um keep going from there but especially like a project that's just being started it's it's like a really good idea because then you can i mean you have to be disciplined about actually calling check accessibility so there's that but um but yeah um in the examples that you saw and then fixed live it looked like a lot of the things it finds are just like a few minutes to fix like is is that is that true where like a lot of these things you just like kind of follow the path and fix it and it's pretty straightforward or was that just like for the demo you intentionally set it up so they'd be easy i think a lot of them and the ones i kind of went through like um some public-facing sites to see what accessibility problems they had and um a lot of them are just like ones that i had in there like you don't have um a main section so one of the fixes is just wrapping that um i think it's like the sidebar container um in a main element and that's going to tell a screen reader where the main content is um and that was a really common thing that i saw but if you kind of just like become aware of that then it's it's a just like kind of one fix in your app and then you can kind of just like be more like cognizant of creating a main element on your page i'm going to keep asking until somebody else uh unmutes and jumps in so one thing that i've seen and i just stepped over zachary sorry uh one thing that i have seen is that sometimes there are problems with third-party libraries that you know if a g grid or material or something else doesn't necessarily do things with sufficient contrast in their colors that can run into trouble do you have any advice on how to deal with that um so i mean i we kind of went over that with like you can disable the roles um you mentioned color contrast that's one of the other like really common things that that i saw as i was going through like public facing sites um and to the extent that you can like customize the colors um like if you can like maybe write your own theme for that specific one that might be helpful um but yeah i don't know about like how exactly to to deal with it i mean open an issue ah their issue tracker um but yeah i don't know if that helps yeah that's about what i was expecting yeah i would also say like again that if you're if you're testing out if you're like trying to pick between a few different components like try them out with the accessibility tools um and that might like sway you in one direction or another with like something like a g grid it's like okay well there's like a few different options but um there's not like a ton of options and that's kind of one of the big ones um so if they have a ton of accessibility issues i guess that's like harder to to deal with but picking libraries where they actually are committed to making sure that the components that they're writing are accessible um i think is is one of the keys um i have a question so i was wondering um how much more of a gain something like axe core is with like e2 or unit testing or whatever is versus something like sonar cube which has like static analysis um because currently like one of my projects uses sonar cube for some stuff but i don't know how much of a gain it would be to swap over something like this i only have like a little bit of experience with sonar cube um is it is it checking access like accessibility rules do you know if it has like yeah so it has a decent amount for like just html files um the only word there i guess would be something like if you're doing like angularism since it's so dynamic there might be issues that crop up with like libraries use or something like that so i just didn't know if it would be better to swap over something like this versus right because the e2e is like going to be testing your actual page yeah as well as an integration test right yeah yeah so i imagine that it probably will catch more things um but and that's kind of a difference too with if you used ax core or something similar with your unit tests is that it's really just going to be checking like one component at a time and it's not going to check like your page level things like it won't be able to tell you that you don't have an h1 um on your page or that like your headings are out of order like you jump from h1 to h4 to h then take okay yeah so it looks more like html architecture too from like a broader like i guess page wise website wise that's cool good job yeah more of a comment you know i really liked the example that you gave related to zachary's question actually of how you ran the the checker against the entire dom and it came back clean and then the modal and that came back clean but on the entire dom after you opened up the modal like that one failed like that was an interesting use case i wouldn't have thought of or expected to see so i guess a question related to that is uh are there other common kind of like ui ux patterns that you've seen that kind of walk you into this scenario like oh when you use these things together now you suddenly have a accessibility problem i not not that i can like think of off the top of my head um but i would say like that that would be a good thing to kind of keep in mind um especially like kind of trying to decide where you're gonna put your tests um because like again if you if you're going to run it on the full dom every single time that might be um overkill but but i think specifically in the case of the modal um that's sort of the way that cdk it's an it's a cdk overlay so it's going to be adding it like in a completely different section and then disabling the rest of your um content so i think that's probably a pretty like that probably is like if if you have something like that where it is modifying a large part of the dom um but yeah i don't know of like other specific things i have a question is there like would i expect to find the same things if i run this like via cyprus like does that replace the need for me to like you know go interactively click a tool to check the accessibility and explore my page like or is it i really still need to do kind of both sometimes um okay so in theory it's all it's all ax core right so if if you're talking about like the axe um plug-in in practice i was finding slightly different things so that was like maybe a little frustrating um but if you're talking about like it was that kind of what you were asking or were you talking about like other types of tests where you had to click through am i covered for using x-core if i run this or i'm like well you really need to also run x-core by hand occasionally so yeah so that's kind of related like i was finding slightly different thing so i'm not ex like i'm not a hundred percent sure why why that would be um so i should probably do a little more investigation on that but um so i'm not i'm not totally sure like if it like how beneficial that would be but i would say um running these kinds of tests isn't like that's not all the accessibility testing that you're gonna do obviously you're still gonna have to do other things um but in terms of running both kinds of accessibility or of of axe core tools i mean you might you might want to do it and like especially do like spot checks and things great yeah thanks for coming everyone

Channel: Oasis Digital

Views: 914

Keywords:

Id: 0IBprY9wfYI

Length: 46min 18sec (2778 seconds)

Published: Thu Oct 22 2020