Platform Foundation Academy Session #2: ATF - Best/Good practices

just so you know this session is being recorded as you can see and we will post um this session on youtube later on so um don't forget to post your questions to get them answered and uh to have them documented on youtube as well and can still send us more questions for upcoming sessions we're going to cover all kinds of topics um so well we could just about start re-dive in all right so welcome everybody i am so glad you joined this is our second ever platform foundation academy um two weeks ago we started this thing with our lovely colleague andrew barnes we talked about features that came in quebec we will have a rome session at some point in the future because you may have heard already today is rome ea day so very much exciting we're excited for the new release and to get the info out there and um show you what's new and there are a couple of really great um features for atf coming in rome as well so without further ado today's session is about atf good and best practices and your presenter today is my colleague daniel who is amazing by the way and daniel take it away all right thank you lisa yeah so today will not be much about the rome right and we'll we'll spare that for a later session but if you have a question around that let us know well we do have the q a time at the end there should be plenty of time to answer your questions and if not we'll definitely take them back home and answer you as a follow-up you'll see that in the community post where we and address all the questions we couldn't address today so welcome good morning good afternoon good evening wherever you are let's get started with atf good and bad good and bad best practices around that we don't want to do the bad practice no not because we don't want bad practices no that's not a good thing we have all done that in one form or another i guess but definitely not what we hear today so atf um as we dive in and get started i would want to start with a little poll to understand who is the audience who do we have here today what's your your knowledge background and atf let me launch that poll as we get started so that should be the right one you should see the poll now move that to a site and while we let that run for a couple minutes to give you chances to answer it atf is our automated test framework um we kind of expect for today's session that you know what servicenow is what uh atf is maybe you have used it already i i would hope so we are very much interested to see where and how you use it and where your struggles are we will talk about a bunch of practices that we learned in the engagements and journeys together with customers what good looks like how should we do that and we can also look a little further if in ap and what's up next where do we head with atf right atf is automated testing for servicenow on series now we are not there to automate end-to-end testing across different platforms or different systems if you talk about integrations to your erp system something like that we are not the platform to test what happens on the erp site right we can test our site that's what it's there for it's a low code environment um to to enable um also non-techy users to create test cases and that's kind of what our target here is you see the answers coming in just a few more that's good um let's yeah have to save harbor up from here if you have been working with series now you probably have seen that before right uh we just put that out there we might be talking about things that come out in future releases depending on the questions where the discussions go we can now talk about rome as that is now on the docs page that's great yeah we're happy much uh happy on that but there might be things forward-looking please don't make any purchasing decisions based on that what i've been saying here right don't hold us accountable there might be things changing plans change releases change and things like that i'm pretty sure you know that so with that um let's start and think about when do we want to use atf what are good use cases what are bad uses or maybe not so good use cases to use atf right so we do recommend to use atf for a functional testing of your business logic and i'll get to a bit more detail in that in in talking about how to set up a test and what you should be testing as you go through the practices right um through the good practices it is really testing the functions does it work and it is your business logic you're not you don't need to test service now baseline that's what we do before we deliver the software and i hope we do a good job and you don't get any a lot of issues around that um but you need to test what you change what you amend on the platform that is support that you need to test you can test browser compatibility as you can run the same tests with different browsers it might be an uh if you have to use opera as one of maybe the ones that not very famous or firefox or chrome you can test the same tests in different browsers and see how they react is it the same result does it work the same way so it's really a good use case for that regression testing and you've seen that i guess in one of the questions there is another sweet spot for atf and with regression testing i actually we mean test before a change and after a change so you have to delta the difference what happens in between right if you do an upgrade of the platform you you might be planning a roam upgrade zone so do it run your tests before the roam upgrade do the upgrade and run the tests again that gives you a clear picture what has changed and does fail due to the roam upgrade if you miss the first test you might miss that a test was broken before you did the upgrade or anything right so that is what we see with regression testing tests can be integrated with just me so if you happen to be a proficient just mean user you can use the same techniques within atf if you do your scripted tests right there um what are the areas we don't see a really strong fit for atf one thing for sure we don't expect you we don't recommend you to run a test on a production instance why is that um first of all tests does have a performance impact right while the test runs there is a transaction being blocked maybe multiple transactions if you run parallel testing so there would be a significant performance impact to that instance potentially but the bigger issue is actually if you test for example raising a priority one incident ticket that would raise all kind of alarm bells in your organization right and you would test raising these alarm bells they would go off it is a productive system same with integrations if you happen to talk your a to your hr system trying to change user profiles things like that that is just on a productive world difficult we do see some customers want to run like a smoke test on production yes it can be done there's no technical limitation but you need to be really very clear what is the scope of the test what is my test data how does it react on other areas of the platform so our best practices don't run into production um if you're testing ui components think about if really every ui element a component needs to be tested right it's it's probably and i'll get to that also as uh in one of uh the further slides um if you have a form with a lot of fields change request comes into mind typically has like 30 40 50 fields on that form um it's probably not a good idea to test in each and every field each and every choice list it's just a lot of work to test condense that and test what is critical to your business right um and we'll get to that in a second um if we talk about unit testing and there was another question that we had in the poll at all that i think we can close now um unit testing as part during the development yes you can do it um but it is now a little bit different uh than it might be with other areas like certain things you can't really test that easily while you're developing it they change a lot as you do it so we kind of recommend to wait till you have a almost stable functionality before you start building your test cases and last but not least on the not recommended side atf is not intended to do a load testing or performance testing of the instance or application so it's not designed to run like 200 concurrent users against the platform and see how it behaves that's just another use case for atf what you can do is kind of similar to a regression test run it before and after a change or an upgrade and the delta that you can see gives you some indication if the performance was better or worse right so within the delta you can see some how it changes so if if a test suddenly takes twice the time than before then that is a three indication something has changed to the back but it's not really designed to for the scalability piece did you actually share the results lisa or you said from the poll yes i just shared them so everybody should see the results as well and we can speak about them okay so let's take them i wanted to see who do we have here so i see the majority of you actually has atf experience either as a designer or administrator that's great we have a smaller number of not really uh being exposed to atf yet so you might find this session maybe not too useful as we talk about the best practices we have a lot of other materials out there um and we'll link that on the community post and kind of what is atf when should you use it what's the business case behind that so you might find that useful as well as a follow up on that um we see also a lot of you are using to validating upgrades that's great i hope it helps you to speed up your upgrade sign that you can actually get faster to the newer releases that's one of the main purposes why you would want to do an atf that's great um deployments unit testing is there great you have some others oh i'd be interested to see what your other use cases are please put them in the chat that's good information was to see when are you using atf great stuff one notice um one note about the validating your upgrades if you're on a i think quebec or later instance um you have access to the wonderful upgrade center which now features the option to check on any um test runs that have been running in the past 30 days so the upgrade center will evaluate um for each of the test suites the last run that was executed for any given test suite so you will get a result on your upgrade and when you preview the next upgrade you will know which of your or how many of your test suites were successful or unsuccessful before you go into your next um release and in the next upgrade thanks so looking at who's responsible for writing test cases that's an interesting split we get there like an even split it looks like between developers business owners dedicated teams uh slightly in the lead here um and that's kind of what we see with customers it's i think a fair representation if we see with customers where do test cases being written and i guess there is no right or wrong in that it's just a different um setup that you have i personally from my point of view i think the developer should write test cases like if they have test cases in mind while they develop a change script or something like that they completely develop differently right they have different mindset and and see what's happening so my theory is a business owner processor should define what the test case is but the implementation should be with the developer just my view but then there's different views on that absolutely and quick start tests also there we have a split of like let's say 50 using it and 50 never use them and some of them don't know what it is okay so maybe we'll we'll cover quick start tests a little bit later um for the ones who never use them i might be interested to heal here and listen why are you not using them right but maybe put them put the comments in the chat and let us know because that's something we can use to learn and see is it just because you maybe didn't know or they didn't fit your needs or something like that put that in we're happy to learn on that okay so let's put the poll away and i saw in the chat there was a question why is it not recommended to test every single ui step for mark right if that's just a timing thing or is that the effort to set up um so from our understanding yes the effort to set up and to maintain that is really growing significantly if you do every element timing is one thing yes but atf's you could schedule them and run them nightly and somewhere in the background uh so you might not be too concerned with how long the test runs but it's really just a maintenance effort if you change a field that is not business critical or process critical you always need to update your test cases so that's kind of the background to that why we would say not but no need to actually do that with every single ui component um with atf recorder from one of the mvps it's gone where's the question one with fdf recorder can you easily create a slim htf test and read yeah atf recorder is a tool with uh it's out on i think a chair if i'm not mistaken share servicenow.com you can find that uh and i believe it's even a free tool that allows you to record your test steps so yes with that one it is probably easier to to create and maintain these tests like um i've never used it myself it's a permanent solution if i'm not mistaken but yeah um it's out there you maybe want to talk about that in uh towards the end of the session awesome let's move forward oh should i run atf in production yeah andrew you see you checking that yeah do not run in production that's not recommended in the top right corner here not recommended don't do that we don't want don't don't run but still promote them so we we encourage you to promote them upstream so that they're available for cloning um there are very many um artifacts involved in tests and test suites so you while you could go the clone exclusion list route it is very difficult to um to find all of the all of the tables the right tables to exclude so it's it's a lot easier to just promote the um tests upstream and then clone them downwards from prod onto your subprime instances there is actually a knowledge based article on our high support system that lists all the tables relevant for atf um i but as lisa said please promote them and have them in the clone bag what i would rather like to pre um preserve during a clone might be your test results right that's much simpler to preserve the findings and test results you had from previous runs so let's move forward um i grouped these best practices around some umbrella topics let's say the first one i want to talk is about designing your tests while you design tests and test cases test suites um you should start with a plan like any good developer when you develop something have a plan have something like what you want to test how detailed should it be we already spoke about how many should you test all in each and every field of that is that really required um who will maintain your tests uh going forward it's not a one-time effort to create them we hear customers um jubilee saying oh we don't use atf yet and it's a big effort to get atf implemented because we have that backlog of things that we need to actually create tests for i think that yes that is true there is a backlog but if you start adding tests with everything new you build that backlog at least doesn't grow right and you start improving your work as you go forward anything you you add or you change start building these tests as you go forward you don't necessarily need that big project at the beginning to create tests for everything you have and some places as you do tests you might find a functionality that is actually difficult to test like for example you might some features might use a browser pop-up window asking for confirmation are you really want to sure you close delete configure them whatever right if they use that browser pop-up window that's something we can't really test but if they use a glide modal api dialog that is testable so if you see that kind of scenario the question is what you do do you refactor your code to make it testable or do you accept that this part is just not testable at the time being and put that on a parking lot maybe for later something like that have a guideline for your developers how they should act right um how do you want to test things that are common across multiple features approvals is a good example you have surely approvals in service catalog and in change requests and probably in a lot of other places does every department needs to create test cases for approvals or would it be good enough you test at one place and everybody else should be fine right so these kind of questions uh have answers to them have guidelines for them when it comes to have the tests um or have the test being built out i reckon they just keep them short and simple and somebody else said stupid and simple or short and stupid something like that so i like short and simple what i mean by that if you if you want to test the service catalog for it to come back to that example you could have one large end-to-end test uh creating a user and a and an end user go to the service block order it creating a manager of that or impersonating the manager approving it creating an agent assigning it working and that all in one large test you can do that but these tests get relatively long and if something fails it's hard to find out what was wrong right because it's a lengthy test and every time you need to run it you need to wait till you get the pair and stuff like that so make these tests more granular shorter uh more bespoke like have approvals in a separate one half assignment in a separate one have sla tests and a separate one you can have these tests still all roll up to the to a test suite and run them as one test suite and that's fine but have them smaller and more bespoke makes it much more easier to to troubleshoot and develop a question that comes to that and i'll get to that in the next minute or is what happens to the text data that you create going from one test to another test in the same test suite for example things like that how do you retain that information if you need to um when you design a test think about execution time how long does it take to execute uis is always slower right if you have a ui site testing that means we need to talk to that client-side browser window that you might notice let's get your test runner we need to wait for the browser to load maybe take a screenshot so that is just taking time and extending the duration of your test so avoid repeating steps in the ui similar to service catalog again if you want to test different items to be ordered from the service below maybe the first one you do on the render the ui get to screenshots and everybody else you can just run on server side that's fine the ui will work right if you had one item that had all the complex elements of drop down list choice this reference list whatever then it will work in the next one you don't need to test iphone 5 6 7 8 nine and 10 on the ui side iphone 10 would be enough something like that right um and lastly here i'm designing tests tests for the outcomes not for configuration i had a request from a customer a while ago saying oh i need to read the condition field of a business rule to see if it's being triggered why would you read the transition table just create a record that fires that business rule and see if it worked right test for the outcomes not for the configuration makes it a lot simpler and actually you're testing that something's happening on the platform rather than reading text fields from configuration tables um how do you choose test on ui site versus server side when you add test steps for example you have test uh you see these test steps if it's client-side or not right so for example open a record that opens it on a ui if you make read record a lookup record i think it's called that is server-side so you can see that on the category uh where you select from i think visa as adding some some comments there as well i'm thinking the uh the question is mostly on how do you choose when to use which so when to use ui over service site and that is a very good question i think you will hit that in a second too so okay cool so that's why this one we use quite commonly when we talk about how does a test suite be executed what's happening and how should you be using that um maybe that's common knowledge for if you have been using atf a long time already then it's probably common knowledge but we still see that this has some unclarity so we have test suite as our overarching object that you use to start a series of tests and the test suite can contain other test suites and other text waves and eventually it will have some tests and these tests will all execute um either parallel or sequentially that depends a bit on the setup we can talk about that but they could run in parallel these tests and in these tests you have a set of steps that do something right so if we look at one test one of the recommendations we have now we'll see that in the next slide again is creating a user when you start a test one of your first step if not the very first one should be creating a user with the profile the authorization that you need to execute that test you shouldn't rely on that there is a user on the platform called lisa or andrew or daniel that has to roll because that user might change position leave the company or something like that happens and suddenly your test phase just because of that so create that user as one of the first steps assigned as a appropriate roles in that and then impersonate it that's one step we have create and impersonate in one step and that kind of starts in terms of having your test self-contained make sure that everything you need in that test is either master data that's out there always right might be departments or organizational structures or something like that or is created on the fly so your text is not dependent on cis or on assignment groups or something like that that might change and cause your test to fail later on create that self-contained as part of your tests that already answers also what do i need if i take that service code service catalog example of splitting an approval on from an assignment if i have two tests one is doing the approval the other one is the assignment every test needs to create that test data on the fly they need to create it from scratch and you can do that with server-side steps right create records in the state that you want them then impersonate the user that should be working on that and do your steps what you need to do that's kind of how you do that at the end of the test there's an automatic unimpersonate so we automatically un-impersonate the session and after that we do a roll back and delete everything that the test has created so we roll back the changes that happen to that instance to the instance in in that context so we don't leave any results and artifacts back and once all tests are executed all the results will basically roll back together so we can see them from the test suite entry so we can see all results from all the tests within this week from one place let's move forward when you implement tests there's one thing and i think i saw a question popping up there somewhere about step configurations a step configuration is the going back to that slide basically the definition of what a test step is doing right if you have a look up a record it would do the glide record created to the database fetch that record and get it back that is a configuration and you can create custom step configurations for your own use cases we we had a create a toolbox session i think a week or two weeks ago on how to do that and when to use them a good example is testing flows or actions within flow designer they are tough to test sometimes as you need to create records and wait for the flow to execute and wait for the result and things like that but using a custom test step you can actually invoke these flow actions or flows and and have that running as part of your tests and get immediate feedback so that's a really good use case for that and there are plenty of these use cases out there creating test data might be one if you always happen to need the same set of text data create a step configuration it's easier to reuse so your test designers test implements can just use that config and they know they have the test data they need yes makes it much simpler and speeds up your test development right um we talked about great user impersonate i think i don't need to add that on this one we we think it's really important to have that in the beginning against the self-contained test piece and you can be sure that it works um we have a one step you might be using frequently as a record update right so you look up a record and change a value maybe a state to process in your flow uh and send that back to the database and that record update will return yes true all good but it could have happened that there was a data policy or another business rule avoiding blocking that update right the record update will still say yes i did it and you don't know if it happened so when you do record update we recommend you use a record validation afterwards to validate if your update actually happened because we've seen customers failing there like oh why doesn't it do it right and there was just a simple business rule or data policy blocking that change which was correct so the test was formed but uh without the validation they didn't really see that when you have um elements that you want to reuse in your test right so you might want to look up an assignment you you want to have an assignment group or something like that you can use a record query to find that entry and have that as a rare variable available as a data pin basically going through your test and reuse that object moving plans like a property if you develop code somewhere right so use that to reduce hard coded values in in your tests forward um testing security as you create an impersonator user in the beginning of your test you have only a bespoke set of privileges as you execute the steps right creating records updating records sometimes you need to step outside that security boundary to be able to process in your flow right and get forward now you could either impersonate somebody else that has admin privileges and do the stuff or a lot of our steps actually have a field called infor security the default is it's tick so it is respecting acls but you can untick that to say this particular action needs to run with system context use that wisely it helps you to make your test run a bit smoother and quicker right dynamic selection we have an option to dynamically select the value out of a pool of values a good example that i typically use is you want to test assignment of an incident from a group to a person within that group now you could hard code that member or find a record that is member of a group and assign directly to that one member but you can also use a dynamic selection say pick anyone in that group that has the role of ideal or whatever role you need right that randomly selects a user and that has a couple of advantages for you first it will find the record along as long as there is one right so we'll at least have one and it adds a kind of random and say randomization to your test as it's every time it might pick a different one or selecting a ci an effective ci in a change something like that right so by randomization putting that as part of your test you're expanding the test scope right and you might find an object that just behaves differently um so that that is a good thing and it also helps you not breaking references right i see you deploy your tests from a test this development system to test the prod the ref the records you use might not be there but dynamics will actually help you to find one that matches it right here that you actually need when executing tests you mentioned already you should validate these validate tests before an upgrade like the regression testing do the before and after so you have the delta so you know exactly what the up what was causing the issue you don't know what the issue is but you at least know what the cost was the upgrade the deployment or whatever you have changed um when you run the client-side tests we do screenshots of that browser window right and i think since quite a while the default is that we only do that for fail steps in the past some somewhere there was a time when we had that for all steps in the meantime we say only for fail steps and the simple reason for that is taking these screenshots is a taking time the storage as we need to storage on the instance and see most of the time you don't look at them because always good who cares about the screenshots so turn that off put it if you have it still on turn it off and make sure you only have it for failed stats and if something gives peer-shaped you can still enable it on a test-specific basis and say i owe for this test i don't need it because something is wrong and i have no idea what it is you can enable it for that one and then turn it off afterwards running tests in parallel as mentioned if you have a test suite with multiple tests inside we have a logic in atf to figure out if these tests can run in parallel or not and can run in parallel depends on a few factors like which user is it using if you're impersonating always the same user we don't run in parallel right the same user needs to wait for to become available again if you create them as we mentioned no problem there are certain system tables if you work on certain system tables we figure out oh better wait for one test to complete before the second one starts so there is some logic inherent logic in it but you can also set a mutually exclusive flag on your text to say oh this one has to run exclusively alone because it's maybe so resource intense or does things that nobody else should be seeing or whatever it is right so you can't set that flag manually the default is we try to figure out by our own logic what it means if we need that so if you can't have multiple tests in one test suite that's great because they tend to run in parallel take a time out and i see a commentary from eric wait one second um whenever we run tests on client side you probably have seen that once the form is loaded we're waiting like 60 seconds as a default for that form to calm down right to run hx false and whatever it is to have that pause and wait for it to happen on the server side we don't if you have lots and lots of server side steps creating record updating records and so forth we don't wait we fire these entries as fast as we can to the server and see what happens on some areas it does help though to give our server a little way to breathe and think about what you just told him right if you want to see a notification being triggered a workflow to being started that might take a few seconds to act as that goes maybe through mq to schedule or something like that but think about them and add these wait seconds if needed um there is use as an automated test for wait one second that is preferred over gs sleep because she asleep puts the whole thread at peace whereas our weight for a second is a little more resource friendly for all the other parallel tests that might run right so you use that feature it's one second each so maybe you need to have a few of them depending how quickly your server catches up with what you've told them and i think last but not least last slide before we just dive deeply into your questions that are still out there have some naming standards as you create a lot of these tests test suites and test cases think about how to name them so you find them you know who owns them you know which area have its m in their csm whatever you use have have some convention you probably have already naming conventions for all the other objects you create on servicenow like business rules and flows and stuff have the same principle for atf tests as well i think that is what i have prepared as best and quick practice that we still have 27 odd minutes answer we have so many awesome questions um we've been um answering some of them while we um as we went along but maybe we just um start with the one that is in chat currently because we have some info uh rohit asks we need um they need more information about scheduling the client runner [Laughter] that is the way before rome rome will bring some uh wonderful enhancements and one of the biggest ones for our atf is the uh the awesome feature called headless browser testing so they're in rome there will be no more need to keep a client test runner somewhere active in the foreground on some server or whatever um with the um help of the docker spoke and integration hub you can spin up a docker image which will open up the browser and take care of all of your testing needs so that is like the biggest thing we have in roman that will definitely um solve that problem yeah and then that will actually be we will be supporting the chrome browser as the main browser in the headless runner you you'll get these docker images from us from github or if you enjoy the boxes out there so you'll be able to find them and we support a linux host and windows hosts with chrome browser and if you're running a linux linux host you can even run a firefox browser that's what's currently in there and yes we will basically start and stop these client test runners as we need what you need on your site is a docker host where we can place that buffer image all right let's go through some of the great questions pulling up rome days mark asking a question there as well um quick oh gone okay now there it is um screenshot quality enhancement yes so we have re-implemented let's say the screenshot features uh one of the new added features is we can finally capture the full page right so far you have that client test window in a defined size like 800 pixel height or something like that now we can dynamically resize that window depending on the page content that's a great enhancement um the quality of the images should be better we use a different engine to generate these images and we will be supporting um the new user interfaces the now ux framework better in terms of screenshots right in the past these screenshots look pretty ugly but they should be coming out just fine now with rome however all these features are turned off on upgrades so you will need to activate them look at the release notes how to do that there are a couple of properties you want to set to get the new features can i stop sharing yeah yes stop sharing so if you can see us i can't do that awesome so we have more questions we had a couple already in the beginning paula wanted to ask is it possible to create ats for portals for example resource workbench investment portal etc those are based on the service portal technology so you can test them with the regular service portal test steps they're usually indicated by an sp in brackets um new um portal technology like everything that is um currently based on the now experience framework is as as of today not yet testable but we're working hard on that so that is the biggest push that we're doing on the atf side is to support now experience page testing that includes configurable workspaces like the new csm configurable workspace but also the starter portal experience that can be created through aes so maybe to add to that with the new user ui experiences service now ux ux framework that we have brought out and with ui building quebec so atf is lagging behind a bit on the testing there right but this is a top priority so every coming release you will see new features being added to allow testing of this ux framework it's unfortunately not like one big bang and with rome everything is there but it will take a couple releases to catch up with all the features that the ux framework has so you'll see it coming there's more all right um the second question by paula was already answered during the presentation so um best practices and and step configuration when when creating that from scratch there are uh quick start tests um there are some some more uh best practices on how to start um and um anything you want to add to that no that's good all right mark's question was already answered during your presentation um should i promote ietf to production of course as i said before make it available for cloning um there's also um the info that the test execution tables are excluded by default so you don't have to worry that your test results would get overwritten awesome great there's more so by any chance can we parameterize custom ui steps i've put some in there but do you have more experience with parameterized tests daniel i actually don't awesome yes but it should be possible so i haven't tested that myself um try it out with the documentation i posted in there um it should be possible since the parameterized um testing is uh its own feature and not necessarily uh tailored to custom ui steps so it's all of the tests can be parameterized should you run them in production no we covered that there's a new one coming in from when bandana sorry not your name wrongly can we expect the copy step functionality in any atf we miss that so i understand copying one step out of a test right and put that into another test that's my understanding of your question please please correct me if that's wrong um i'm not aware that we have that on the roadmap but feel free to check our idea portal uh post it there if it's not there and use the community force to vote it up realistic you're reading that the roadmap for for san diego is pretty close uh but for the ones after that we we have not fully nailed out yet what all the details will be i've also linked the idea portal in there so you can um check it out all right more questions is there a best practice to schedule runs so run them on a daily basis or weekly or do we have any recommendations around that i was reading ashley's question sorry [Laughter] all good all good yes so the question is is there any best practice does it have impact on the performance um and andrew's answer is is quite straightforward um since it runs on the subroute as it should or usually should it shouldn't impact your performance on your product instances um there's a case to be made for any of your intervals so make sure that it covers all your cases and that you get current test results before starting into an upgrade so whatever fits your upgrading or where release cycles i guess i see andrew andre i'm answering tap tapos on if eddie gray tests atf re-test management 2.0 well this is at least an idea in our roadmap uh we're um you should have also on the idea portal you should be able to see that load it up put your comment in there it is on the backlog no commitment currently yet even when it will happen but it is something that we look at brandon will there be a better edit this a better way to edit steps with an atf currently i have to delete all the steps add in the one i want i would be going to have a click track click and track function so you can rearrange steps i have a tip for that you have a tip okay have first your tip and knock i'll come back two tips two tips one is when you create a new step in the bottom right corner there's a selection on where do you want to insert this new step so you can say at the beginning after step one after step two after step three and so on so you can pick the point where it's supposed to go if you have already created your test steps and you want to rearrange them just pick one of them in the list and edit the order number that will auto rearrange the other numbers it's not very intuitive but it's there so you can actually reorder your test steps and now i can come on the back of that in a few minutes before we close that section i'll have another poll asking you about what would help you to bring atf forward and one of the things we have on our backlog thinking about prioritization of it is creating a like flow designer user experience for managing tests so you would get these nice as you know from flow designer with the branches and things where you could drag and drop things and have input outputs data pills these kind of things to maintain your tests that's one on the roadmap we have another one i'll explain when we get to the poll but you might vote for this one if that's your desire yeah drag and drop moving test steps yeah actually i just answered that yes there are plans again it's not yet defined which release and we happen to take some of your input today to see where we should have priorities there's also when it comes to um copying a subset of test steps you can always go ahead and create a test template from your tests so you could create a test template for a specific set of tests that that you can then use in other tests and insert them um when wherever you need them so you if you find you have the same order of three or four steps to do a specific thing for example setting up the test data for one of your test um larger conglomerate of test cases then maybe create a test template for that initial demo uh test data setup and then reuse that all right there was a question about integrating test management 2.0 with atf test results that is like i think one of the most common asks that i have seen there's uh it's been on the um on the on the idea portal it would be awesome if we could get it currently on the backlog right not saying when um but it it has some intricities right typically you run your test management in the productive environment and your tests run in the modern crop so we need to have the instances talk to each other to make sure that they get the right information over and things like that so we need to think a bit more about how to do it what should it be doing like raising defects and things like that um but yeah it is on the backlog and check the idea portal upload it to set your priority priority to us great i think we answered almost all of them i'm checking them out just now we forgot any there's two more coming from one from paula hints and tips on how to maintain a test suite just thinking out on that can you elaborate what you mean by maintaining tests with a test suite contains of other tests another test suite so the thing that you need to maintain is basically what is part of your test suite and and i think we have that filter option there we say can i want to filter all tests starting with a given prefix so if you have your naming templates done right it should more or less auto maintain itself kind of please correct me if you have something else in mind and eric is asking where's the dynamic selection for field values i think we'll follow up with the link there's bound to be some documentation on that awesome cool so maybe you can still type in your question we might address them let me bring up the last poll and explain you what what i want to know from you let me bring well it's not the last one it's my last one we'll have a final poll to see how you like the session so i was talking a little bit about the road map and priorities two things that on our on our backlog which we haven't decided which one should be first um and i want to hear your feedback i just say first of all it's coming in awesome first do i record it we spoke earlier about the atf recorder thing that's on the share platform somewhere out there which works to a certain extent i'm not sure if it does everything but it does just a good trick so would that be the one that you say hey yes something like recording the steps as you go through to create your test cases would be really really helpful or you say actually the user interface of atf is a little outdated have more that flow designer risk interface with drag and drop features data pills going in and out which one would you prefer what is the software you say work on that one first that's more important um i'll take that back to our inbound team not promising anything that this will happen but at least it gives us a sign of where we should be going keep that open a few more minutes for you to vote and also i didn't mention earlier but you we can totally let you ask your questions out loud and i can unmute you because for example a question that that ashley just posted if you want to ashley would you please elaborate on what your question is is that a feature request or um something you wish for for it to go through or how did we help you that was um a response to someone asking how they maintain their tests awesome all right that is very good thank you so much we get more questions i made maintaining the actual test part of the suite rather than have to review them before upgrades i think that was what what ashley answered okay thanks and for the field value dynamic selection we'll follow up with the link yes good so far so good all right what how does the poll look do we have uh this is awesome i switched the window and it's 50 50. i love it that tells us why we have struggles to figure out which one is more important okay that is amazing thank you very much for that kind of feedback we'll take that back and probably we cast a dice somewhere to see where we land but we will take some additional series and additional sessions to see to what kind of conclusion we come back uh agreeing both yeah we know bose is good but there is a kind of a limit in resources when we can do things it's not saying we don't do it it's just when so yeah um and the highest priority at the moment for atf is really making the ux framework testable in its entirety so that's taking uh the majority of the resources but we try to get other things in as well your recorder would boost adoption first ui would be a nice to have second thank you ashley yeah okay so there's uh there's one last question before we wrap this up let's say i i'm still confused on how where to choose to run your test in the ui or server side so most of the time you will have to you will have to test the client side once so you will go through your test scenarios and figure out what are the ui steps that you definitely need to have covered are there ui policies you want to have checked or make sure that specific fields are filled out or any anything that is ui um ui relevant but after that you would encapsulate all of your tests in their own suites and then collect the suites in parent suites and up and up so think about your app structure or your configuration structure and then probably have one set of test suites that covers the ui and anything that is important to the ui and all subsequent tests that are on the same process will use server steps uh because they're faster because they don't need the client test runner and uh because they're they're easier to write and easier to evaluate so that is basically it if you have to check anything that happens on the ui use the ui steps once everything else that is business logic process business rules flows approvals anything that happens in the background and doesn't require user interaction you would test in the back with server steps just there was one question from tapos about issues and quebec with atf thanks andrew for taking that to the support yes absolutely if you have have issues specifically after an upgrade before it was working enough it works differently please raise that with the support team in terms of issues i haven't seen a lot out there one that i have been made aware of is that might be true if you have a heavy client-side testing test suites or tests uh length of steps and things like that the client test runner does have a memory problem if it's open too long it consumes a lot of memory and so that could be maybe one of the reasons you run into that um but please raise the support here they'll be able to drill into deep if that is your issue if there's something else all right i'll kick off the last poll that we have for today and that is just some general feedback on this academy so please give us your worst rate how you like this session would you attend again was it too too long too short whatever give us your feedback we look forward to seeing it if you like this session or you want to read up on any of the questions that we have answered and also any follow-up uh we will put this um session on youtube there's a um there's a collective community article on um that houses all of these sessions but also the creator toolbox sessions that we mentioned earlier so make sure to check out that article maybe subscribe so you get notified about updates when these recordings go live there's a youtube playlist we put add this recording to so you can also just uh hit the whatever subscribe to the playlist and then you'll get notified if there's a new entry for that um the next sessions are in this academy looking forward will be let me just quickly check my file um we have another session planned in two weeks time so this one will is scheduled to run every other week thursday at the same time the next one is in two weeks and i will be covering how to get started with quickstart tests and will be supported by one of my wonderful colleagues again um we're planning more sessions going forward uh we're covering topics like instant scan maybe we will um add some system clone best practices that just came up the other day that um it would be good to to put some content out there on that and then of course once we near the general availability of rome we'll also have another session on rome features um similar to the session we did two weeks ago um i think it would be useful and then we're kind of planning to also do a session on headless browser testing on how to get started with the headless browser but be sure to also follow and check out our awesome developer advocates we're covering chrome features starting today so live coding happy hour um this afternoon for the americas this evening for the european uh time zone and very early morning for apj is live coding happy hour with brad tilton with chris johnson and me so we will speak about rome features don't um so don't miss that um thank you so much for being here any further closing words from you daniel well thanks everyone for being here and making this interactive posting your questions really like so many questions coming up that is great that really shows your interest in that and i i hope you find that useful and we'll see you again in one of the future sessions that is awesome thank you so much andrew for helping us and thank you everybody for being here bye bye you

Channel: ServiceNow - Now Community

Views: 595

Rating: 5 out of 5

Keywords: ServiceNow, ServiceNow Community

Id: XfErr0gce_g

Length: 55min 39sec (3339 seconds)

Published: Mon Jul 26 2021