What's new in the Automated Test Framework (ATF)

all right good afternoon everybody so we are going to go over ATF lab before we start the lab I have few things to talk about the first thing is safe harbor slide which is basically you might have seen multiple times what it says is do not make any purchase decisions on the forward-looking statement towards the end I will be demoing what is new in New York and that's why this safe harbor slide is there the lab is based on made version it doesn't have New York features so you won't see it in your instance that so here are the lab instruction I I hope everybody is into an instance and they have lab guide and everything open anybody still lagging behind cool so a little bit about me I am shriti Varma and I am senior product manager at ServiceNow in the platform group I own multiple products and automated test framework is one of the product that I own other than that I have made tree base and other app deployment related products and upgrades related stuff and with me I have sure I'll let him introduce himself everyone hi there can you all right there we go Joe Fisher I've been with ServiceNow for five years I'm a manager of platform development or my team's based out of San Francisco and where our team is the owner of the automated test framework all right so indexed one-and-half are what we will be doing is I'll give a brief overview of what a TF is if if at all you are new to ATF and then we'll start the lab the first lab is setup and then we have a bunch of lab exercise that's we will do and as I said towards the end I will demo what's new in New York and then we'll wrap up okay so how many of you are new to ATF don't know what's all right couple of you good so let me give you a little bit of what what is ATF ATF goal was to help you with upgrade testing to help to help automate all the manual tests that you do do during upgrade and to speed up that upgrade testing time that was the main goal of ATF now with Madrid we also come up with a bunch of QuickStart tests so that will help you also to generate all those automated tests quickly and faster we have heard from customers we have heard from many customers that ATF has helped them like reduce that upgrade cycle time testing time from weeks to days so it does have that power you can I mean it depends upon what you are how your instance is what all you are using but - we have I think we have covered almost 90% of the stuff manual tests that you do today on our instance can be automated using ATF or with Madrid release definitely when we introduced all this custom UI so we will have a lab on that as well so the goal of ATF is of course the end goal is to make sure that all the manual tests that you do on our platform you should be able to automate those tests and the other one is it should help you upgrade or faster productivity wise atf reduces upgrade cycle time as i said it it reduces the manual effort that and then it is available or in an instance and you can move like test you can create test on one instance and move it around to different instance it is designed for both technical and non-technical users so if you are like script savvy person you can just write script and you use ATF and run those scripts using ATF framework it it is a homegrown build product so you don't have to have a third party tool installed in order to use this product it is integrated with the platform itself feature wise you can schedule test yeah there is parametrize testing that we added in Madrid so you can run same tests with different set of data you can create a really reusable step test steps so that is also we will go in the lab where you can create your own test step and use that while creating ATF tests you can may write scripts and and the good thing about ATF is after your test runs it will roll it back every every changes that it did so then that means you will get your system in a clean state it will never be like you have to go go back and do cleanup if you have to do that then that is a bug that you should report it to us so it should never be a case where you have to go and do any kind of cleanup you want to the original reasons why we 80s so here's an engineer so he like he is the Guru of ATF you can say he's the mastermind behind it yet I wouldn't say he really put a lot of time today yeah back in 2016 we realized or it was very clear that we had a lot of issues with helping customers get through their upgrades help help them speed it up and turn me down just a little bit so yeah so we realized what we're missing is not I made a test framework and we had some tools in the backend this early as a Helsinki but nothing as as complete as a test remote so it really started out with how can we interact with a forum how can we get some script validation and since then oh yeah so I'll just talk louder so yeah so we originally started with how can we is there anything we can do to help our customers automate their tests or automate flows through their customizations and one really important component that was missing before Istanbul was the rollback engine so this is really a critical this is really a critical point without the rollback engine this mostly wouldn't be possible because they you need to be able to run a test more than once so since Istanbul we've been then adding lots of other categories and then we have the next slide that shows some of those yes so this is how our ATF evolved over the period of time in Istanbul we release this product and as Joel said we just had form and UI testing and server-side testing server-side testing is very powerful like you can test business rule workflows you can test any kind of business logic using server-side testing yeah the ability to create test steps was also added air from the start so it was in the Stangl itself it supports Jasmine assertion engine so you can write script and run tests as well and it supports both out-of-the-box and custom applications so if you have your own application built you can use ATF to create tests for that as well and if or if you are using our application like ITSM CSM and all you can use ATF to test that as well and any kind of customizations you do you can use ATF starting like it was like from day one we added a capability in Jakarta then we added Service Catalog testing we added REST API testing which is only for inbound rest outbound is still on our roadmap it's not there yet then we added scheduling scheduling is basically you can schedule ATF test any time of the day and run it and it so you can like run it in the night and then come back and look at the results kind of that was added in Jakarta before that it was like you have to manually click run tests to run the tests and then pick a browser is a feature where we added a way where you can route your tests to a particular browser before that it was like it used to randomly pick one browser and run the test with this feature you can say okay I want to test with Chrome browser or ie or Firefox or whatever and then in Kingston we didn't do much we added one one feature which was rerunning of failed tests in a suite the reason why why we didn't do much in Kingston because we were focused on making a TFW cat compliant and our most of the time find they're making a TFW complying so that's why we came out with a very light feature set in Kingston in London then we had lot of new features we added service portal testing this is only for out-of-the-box widget for custom widgets still it doesn't support London won't support that you know then we added left navigator testing so basically application navigator where you can just start your test so you can say click open incident and you start your incident related tests through that or you can also look for module visibility not visibly visible or not on menu item is visible or not those kind of tests and then there was few negative testing that was added at our site so you can say if the roads are returned or not kind of deleted or not inserted or not those kind of stuff and then ignore browser console error this is something that we added to reduce our red line because we got lot of incidents saying that ATF is failing because of this client-side JavaScript error and we don't know what to do that we also don't know what to do that because we don't know which scripts are failing so that the thing is with this particular feature what you could do is for the time being you can say whitelist this error and still continue with business process testing and then come back fix that error and remove that that from the whitelist at right so that is what it was so one of the asks from customers was I get that there's a javascript error but I need to get I need to ignore it for the moment and come back to it later so you can whitelist it in your test but then you can go to a different module and see where they're all coming from which tests were failing on them and you can link that back to which scripts they were probably running on the forms that caused the problem so it comes down to the this step it'll be like I was trying to click a UI action on this form and I found it client error so then you can say oh must be a UI action or something that's occurring on that that forms table all right so in Madrid we added custom UI testing this is where you can test your custom service portal widget you can cast you can test any UI page you can test any for example in scepters all those things you can test using this custom UI testing you I have heard that there's a risk management page that comes up and you have to write a test to test that during what creating some kind of change request or something then you can do that is a custom page so that kind of you can test even that starting Maitre D' and then we write so some things we've been hearing is I've got a my own glide modal and it's got a bunch of fields and I need to fill out some of those fields so with custom UI testing you can now interact with those fields we couldn't do it before that because it needed to be on a form and a glide modal wasn't a form also people want to test widgets service portal widgets that aren't actual forms so this the the API we designed will generate metadata off of any kind of page pages that we can support and then you'll get an interface you'll see that in the lab but yeah also so like the change interceptor that was a big one any interceptor and wit and Wizards are supported yeah and then we added parameterized testing which is basically you can run the same state test with different set of data so you don't have to create multiple tests just because the data is different right before Madrid you had to do that and that causes manageability problem it is very difficult to manage now if you want to change one test you have to change ten of that test because they are the same just that big data is the difference so this is this was a big Oscar as well from most of you who are using ATF and then starting Madrid every a bo business unit every application they are packaging some prepackaged automated tests with their application and that that is basically that is dependent on demo data and that is testing out of the box functionality so you cannot use the same test as is you have to basically go modify to test your customization or at least point to the data that you want to test but it is a starter right you don't have to write the whole test or buy like from scratch to just test a functionality which for a particular app that we roll out so that is how product evolved and I will talk about New York towards the end so let's get started with the lab our first lab is general setup we will have five minutes for this particularly loved Soleil and I'll go over lab with you so how do I all right so I I hope everybody is having an instance right we you want this no other instance one all right setup so this this particular lab is basically just making you familiar with what a TF how you run a TF tests and where to look for results and so forth right so the first thing is you will navigate to URL and I hope you all did that and you will log in using admin user for this particular lab for now make sure that your scope is global so you just go to click on this developer cog and then go to the Developer tab and then select application as a global here oh alright a little bit before you go to the setup lab just make sure you do all this clean up so that your lab doesn't run into any problems which is basically close and restart your browser and then make sure only one browser window is open yeah it'll be easier to find your client test Runner so when you start a test it'll actually open a client test runner and then it will leave it open but you need to have that tab in front when you're actually running a test otherwise the browser itself will throttle the seat or the CPU is throttled to that browser tab unless it's visible so you want to make sure that you know where that tab is and if you can't find it close all your tabs and open a new one again all right then you make sure that you the ability to run tests is enabled so we go to automated tests and then click on properties this to check ma you need to do that in order to enable the run test button because the reason why we have this one is we don't want you to run tests on production so when you enable this you are making a conscious deliberate decision that you want to run tests on this particular instance and the reason why we don't want you to run tests on your production instance is because these tests will make some database changes so if you have a notification enabled and you have a business process which says that oh if there is a p1 incident open I need to send an email the the email should go out to cio of the company what will happen is if you have a similar such kind of test which says that or create a p1 incident and do something that email will go out to CI of the company if you run tests on production so that's the main reason why we say don't run atf tests on production there is some ways you can do that so if you really really want to run on production then what we say is do not commit do not submit form to the database just cancel it in the browser that way it at least it will not send out the notification but I mean if you ask me my recommendation is do not run tests or production at all right and if you ask support running a TF in production is not supported so please don't file a p0 another point some people have asked well how do I do my email testing so if you're not familiar with you know configuration there's a way to set up your sub prod to send all the emails to one account so there's there is a way to do email testing on survive alright so then you go to the test module and you can pick up basic UI test so this is a sample test that we have and just run it so if you look here it will ask you to open a client test runner this is what Joel was that whenever you run a test you need to have this running in the foreground otherwise your test will take too long and sometimes it may timeout because of the browser throttling problem so once you run the test this is how you go to the results and you look at all right oh I didn't know it makes a nice that's your lane okay all right so everybody done with the setup anybody behind no okay so now let's go to our next lab which is basically creating yeah that will be there because it is now it is open and it is now waiting for another test to run so yeah just keep that window open yeah we realize it's not very clear but it basically it's always saying waiting for a test run it's constantly waiting for tests we're gonna change that all right so the next lab is about creating a custom step this is where you create your own steps so we provide out-of-the-box bunch of steps and you may find that oh I don't find a step which can do something like this so what you can do is you can create your own step and this is only for the server side not the UI steps but any server-side thing that you want to do you can create your own steps and we will go over creating one step in this particular lab so and what is that step about job you know the custom step is passed through oh yeah so we have some limitations with our some limitations with our back reference angles so some of our parameters are running glide variables and they don't quite map to certain kinds of fields so this path through pasture lets you pass in a reference and output a string so there if people are coming across situation where they can't pass data from one step to another this is one option where you can bypass they it's a date it's a data type limitation this allows you to pass a specific kind of value so this step is yeah not out of box but it's something you can build all right we'll have ten minutes to go over that lab it should be a quick map so the thing is what you do is you first thing you go is you we have a module called step configuration this is a module where you go and click say I want to create a new step right so the name the step has passed through reference yeah and you can pass daytime right [Music] Oh that's any kind of data types in between but it's you want to have control over your passing so without this step like this if you're trying to Kasich add a time to a time those two field types don't not like pass data to chai so here if you see we are adding adding like we are taking input variable of all these different parameters and we are passing it to to an output variables and we will define that input and output variables once you create this step click on that back then create a new input variable I type as reference Nicholas yeah so what we are trying to do here the lab is all about custom robot application and in this particular lab what we are making sure is you are able to order custom robot and also making sure that you can order different parts of custom robots and we'll be using parametrize testing to do that so here the first step is to create this particular step that we will be using in our parametrize testing right and now it's arms and now I'll have a legs for male robot so here's and then here I'm referencing a part table where all these parts are stored to make sure it is type as legs thanks pops the last one is body so this is how I created my input variable which which is a reference of reference type and now I'll create output variable so remember we are now taking a ref a data from a reference type and passing it to a string data type so I'll create an output variable of string datatype for all these different reference variables that I created input variables that I created make sure to use the string reference type not the string utf-8 just a couple notes about this part so like the four types are you got a service catalog and you say you want a a custom robot you can set the head body arms legs depending on each of those has a price and if you exceed the price that will generate an approval request you'll have your approval manager approve that note they will then generate an order record the again I mention a little bit earlier but the reason why we're doing the pass through is because reference types don't map to the the string types that we need in this case so in Jakarta the the mapping values were kind of limited down to the types that they could support so we're looking at opening that loosening the back up again probably in our next release all right so this is this is my pass through step that I created where the what we do it in description is this is where this is the description that you will see when you click the step while creating tests and then here you add all your code for that particular what that particular steps needs to do and then these are the input and output variables so let's just take a look at the script real quick a script so we've got an inputs variable and an outputs variable providing the closure and you can see we're setting the inputs reference value to the outputs head value and then below that for every step config you would want to set an output message that's what shows up in the step result record so it tells you if you were successful or not and why and then you set success or you set failure depending on the case so you can do a condition and if something went wrong you say set failure and then you put a message along with that for the output message and your step result say I failed because input value etc this is just a really contrived version of that so now this is going to make a noise oh and the interesting so in the in the in the jar the comment there there's a GS sleep 1000 so that's not supported GS sleep is not supported in scope taps so I think it was Madrid may have been London we added a new function for ATF or you can say SN ATF sleep one second I'm pretty sure that's documented if it's not documented call us well well make sure it is but we support the sleep one second so you can do an iteration I'm waiting for something to happen wait until assertion or condition but GS does sleep does not work in scope tabs all right so we are out of time for this particular lab anybody lagging behind all right we will wait for few more seconds so but that by the time you guys are finishing up your this lab let me just go or what our next lab is all about this is parametrized testing as I said you can run the same tests with different sets of data you will see there are there are different options to create parameters parameters as your variables and then you have data set we will see when we create a parameter s test and I'll go over that but we have two types of parameters that you can create one is shade and one is exclusive so shade is you can share that parameters across different tests the caveat is you cannot share the data of that parameter across different tests that is something that we it is on our backlog to fix in future release but for now you can just share the parameters so the you don't have to create multiple parameters for different tests so that is what shared parameters is all about exclusive parameters is just for your own tests and then it will run with will with different sets of data and it will produce results for all different data set and we will see when we do do that lab an example of a parameter that people might use a lot maybe like a shared parameter would be a see you want to pass different approvers through your tests so you'd add a shared parameter for approver and then all of your tests will be able to use a parameter called approver and we're going to show you how you populate that during each of your separate tests and side note that step config we just built we're going to use it so it's not just sitting there all right who who else is lagging behind in our previous lab exercise oh this one you can help all right let's start our next slab which is doing parametrized testing we have 15 minutes for that so let's go create a new test and name that test as and test for different robot parts so here we will be running a same test and we'll make sure that different robot parts can be ordered and make sure you check this animal parameterised testing checkbox to make sure and once you create you will see all these new tabs that got created one is parameter definition and the other one is run a test run dataset right so parameter definition is the parameters or the variables that you you define and the test run dataset is the data that you will add for which the different test run should happen so let's create a parameter definition so in this lab we will create exclusive parameter and will not do the shape parameter and this is a reference type parameter so we will do pretty much the same thing that we did for when we created our output variables so I created a body type and then we have head and arms will have an arm let's type is reference sorry so now I have like my four parameter definition created now Ness next is I will add some test data to to that to run the tests right so here we have two option one is import and another one is new so input is basically will let you import data from a file instead of you manually adding it so if you have a large set of data you can use import option otherwise you can just create and manually enter some data here so import is basically you will pick an excel file and you can like we have a template to guide you how the excel file should look like so that it works with ATF so you can download that template and look at it you know but for this particular lab we'll just create two data set manually let me just go over this template quickly so this is what it will it will tell you what to do and let me see if I [Music] can den this is this is where you can add your data and then you can just import it through the in Excel but for this lab let's add new let's add the data manually and through the new option so for [Music] so just for some context the debt that did the test run datasets basically it's you're running the test with a set of this data so you got those four inputs and you can use those four inputs throughout your test however you want and then the test will actually run for each iteration of data set so that's good we're gonna do two runs to the same test so I just created two data set for for this particular test now let's go create the test itself so the first step first step is this in the step that we created now we will be using that step remember we created this step you are we added description so that description will show up here so you when you create a step you can add your own description which will show up here and then I select next so here using this particular icon which is we call it glide and glide element mapping jump in you can so this is how you are now using parameters into the tests you use this particular jump in to map it to the column and this is where the test will take the data and then it will run multiple times a test now what we are doing is we are using Service Catalog and we are we will be using the same thing the the data that we created in order to order that particular robot so so the next one is we are just adding a step to search for the catalog item select robotics because that is what our application is and then cert on customized robot here so this is where we are searching for that particular catalog item then the next step is to set variable values right so we will be setting up the the variable values basically what what exactly we are ordering right so here when you are setting variable values make sure you use the data from which you have passed it in the step 1 so we will again use the same jump in in order to reference back it to the step 1 and we'll get the data from there so this is this is very very good feature in this this will help you your chain your test steps together and you can build complex tests using that particular feature and in between if you have any questions feel free to ask we are here to answer questions as well we have an expert also here so if you have any kind of questions and then the next step is to order the catalog item oh I say ok now I have created my test so I created a parameter definition I added data set and my tests are now ready now I can say okay run this particular test when you run the test you go to the client estranja that is already open hmm for some reason my test fit let's go and look at it oh I'm missing one step opening of catalog item thank you yes alright let me say ng for not define is not the greatest error message but just means that you're not you don't have the form open and actually why don't you show them if you close the closed the step real quick just create one more time so in the drop-down there you can just say after insert after the search step it so don't put it I mean that is cool and then all or you can just go and read I'm like number here execution number that will also reorder that that test steps for you all right now let's see folks or not there it is yeah yeah yeah yeah yeah that is because that's a reference column and in that particular table we already have yes sir on there yeah in this one I think it had set D for not default but it just selected the first one I think so at the double check pretty sure that was part of the custom at the he load the if you're the catalog item in service catalog it should be all right so test pass now if you go to the results you will see we have like two test results for each data set you can drill down into that and then you can look at the test results for every data set so if you have 10 you will see 10 test results and then if you go to the test here you can also see parameterize test results so this is like for every different run when you click a run test do you mind is hitting back a few times just go back to that parameterize result just hit the back like the browser back button this one so yeah see here's the parameterised results so it's saying for all the executions of this test here's the overall results status success and then each one is an individual test result so the success of each parameter set test all right anybody still working on this lab okay there's one over there can we help all right so our next lab is a page inspector so we added a capability to test custom UI right so but whenever you build a custom UI as a developer you would want to make sure that your UI is testable by ATF and the whole purpose of page inspector is to make sure that the UI is testable by ATF so you can you can see whether a component is visible to ATF or not and when that under the lab is all about that the first one and it's in two parts so the lab for is in two parts a is using page inspector and B is where we will create a custom UI tests so let's have what the test will be using the components that we verified we can test with the patient's vector so now let's go to the custom UI test the first one is making sure which will verify the components that can be used by ATF and which cannot be used by doea right so the thing is you go to the again the develop first thing is impersonate as a as a user who is an approver basically so the idea is you're gonna go into the interface and look at it from the user that you want to test it from and then we can just turn on the page inspector and it will surround the UI that you're looking at so now we are turning on the page inspector so you just go to the settings and then the developer here you will see automated test of framework paid inspector so this is where you will enable the page inspector so we go to the approval I'm not gonna take the paintings picture so there's a crosshairs icon that will turn it back on all right so now what you can do is you can use this little that what do you call that crosshair processor you can drag and drop to a particular component and you can see if it is testable and all the properties of that particular component so this is how you can use page inspector so here I drop and drag and drop to the approval so first step is that then you can drag and drop through something else so let's drop to a something that is not testable by ATF right so where can we go that click on this it's take in a lab useful state you click the request item so you go into the approval request form for that request and then we're just trying to get pick the attachment icon there yeah now if you drag and drop on that you will get a message saying this element is not as accessible by ATF so you can just make sure your which n elements are accessible and not and then you can go and fix your page as well so there are certain things that you could do to make component available to ATF for testing as a developer so so what is that list of things are supported on supported in the lab there's a link right below that recommend you guys open it and probably bookmark that link it's it's right there and like this thing so I talks about things that we don't support the reason why attachments icons not supported is it needs to access to the operating system and the ATF since this in JavaScript doesn't have access to that all right so this is page inspector let's go turn that back off and switch back to this is some administrator now in the next lab we will be creating a custom UI tests so basically it will test a service portal page for an approval right oh yeah the same order approvals page yeah it's now create a new test and I say okay does this so the first step is here is to clear all the approvals for the particular approver this is again a custom step that we added in order to run through this lab we didn't create in the lab but this is also not available out of the box this is that we add it just for this particular lab yeah we added this as a shortcut to save time in the lab this is a contrived step to clear up a certain interface before you test it don't recommend really doing this in general in production or in your own tests and the next step is impersonate as the the approver oh wait sorry no it's the user who is submit Westerner requested make sure you select here complete robot instead of customized robot because that will trigger the approval process that we are trying to test and then you set the variable values select that as a classified so configuration is a classified here when you set the variable values and then the next step is to order the catalog so this is use a requester has ordered the catalog item now we go and we Traverse through the approval process right so now we go and impersonate as an approver and we will be opening we will be testing the same approval page that we use page inspector for right so now we will go and say okay now then the next step is I want to click on a particular module in that particular page so the first step here is you will have to navigate to that particular page so you will go to application navigator you this is this is a very important step in order to test any custom UI the first step you have to do is to add and open that UI page if you don't have that on that particular step it will you will not be able to write add another steps through test that particular custom UI page so the first step is always open their UI page either you can do it by navigating to a module or you can also open service portal page so we will add a new step in new york that will be directly you can give a service portal link and it will open a service portal page yes i matured you want to go you want to get to your portal page from module so in this case we're going to our custom robot app portal page and has the approvals form widget in it or they actually the approvals widget is excuse me but in new york we can look at something other than just navigate the module so now here are what i'm doing is this is a new test step category that was added in madrid which is custom UI test step category and there's all these different steps that were added so the thing is now I'm doing is I'm going to click a component on that particular approval page right so I do this now it'll ask me to retrieve component for that particular page so what it will do is basically the navigate module step it will it will go to that particular page whichever you are trying to or navigate to and then it will crawl that page and get all the components in that page and it will get all the same components that you have just saw with when using page inspector so it will identify this component and get the list here for you to add a click step make sure you have your client test runner in the foreground because behind the scene it is running that particular step to get the list of components and if you don't have that it will timeout just sign know so this the retrieve components buttons a little bit different than run test button the run test button will open or you see a window and you can run the browser it'll actually open a new window and run it but if you have an existing browser like a client they're already running the pic a browser modal lets you choose it or pick like a different one but the retrieved components button only just goes to the the current active client test runner on your session so if you click it and nothing happens you probably have another tab already open with a client test running on it so click the request item here that you see so I have this particular number your number request number may change I oppose yes you're looking for a link type in our EQ that's that first alright yeah so that's so yeah there's a lot going on in that page a lot of it you don't care about so that's why we added a search filter on that drop-down so try and see if you can find it by the search filter first everything in that drop-down is part of the search filter so your step should say click the component link a request yeah wait order now I'm trying to what is it said component values right so you have to do retrieve component as long as you are navigating to the page but if you are staying on the same page then you don't have to do the chief component you can just click Next the whole purpose of retrieve component it is getting the list of component on that page right once it got the list you don't have to retrieve over and over again yeah so that's the form the approvals form widget that was the page we're looking for and now we're gonna try and set something on that so step 29 you know so there's this tent oh oh so one point is that you'll notice that these components don't read very well in this step this is because the the widgets as they were designed didn't have a lot of information behind it they just expected the users to look at the page not inspect and read it so it would make it easier for you if you put information like the title or the name or a proper ID on your component that'll be easier for you to find it when you're building your test against it yeah so here you make sure you pick text area and not text input so you can see if you open that drop-down one more time you can see the label says post input that's because there was no label on it even though it's supposed to say like submit comment so it'd be a good idea when you're building your pages to have actual titles on them it'll be easier to find what you're testing and this time see we we are will be on the same page so we don't have to retrieve the component you can just do next yeah this is another great button it's called button input empty this submits the comment here make sure you pick this button particular button which is button input and not they're a bunch of buttons that you will see make sure you you pick this one so here what we did was we added we impersonated as a as an approver we navigated to the model we clicked on that particular request we said okay as an approver that request looks good to me I clicked a button saying approve the next is making sure that particular approval went through and that particular text appears so we we are doing assert on that particular page which is basically the whole thing yeah wait wait no I did button click and this yeah this is the one so cert text on pre page is pretty pretty open I had someone asked me earlier today can I do this on notifications on the page and yes so you can do it on like field annotations info messages error messages it just reads the entire page for that text it'll strip out all the HTML so if you add later on in this in this lab we're gonna in the exercise we're gonna read something has a link in it but you can just write the strictly the text and it will find it that's fast it depends on a bigger page if your pages ten megabytes I might take a little bit yeah but there's only type of proof yeah you want you want the approved button in step 12 so basically we clicked from the approvals widget into the approval form we set a comment and the approval we submitted the comment after we submitted the comment we're gonna go approve this request so it's set set text area submit button the input button input empty and then click approve button and then a certain day and it said that order yeah and plus when you approve the request an order is generated and an info message is submitted so that's what we're checking here and now we run the test so you submitted the request open the requests we're gonna set looks good and you notice that will actually scroll down to the type your message here field here in a second it all scroll down automatically so you can see and then assets look good send so the send button is the button input empty you can see the text is there it asserts that text approves and then search that the info message showed up and all that way to fast yeah so now this this just fast and you were able to test a a custom widget on a service portal using custom UI testing and then reading messages on the widget all right so we are almost to the last lap which is basically how to use quick starters as I said these are the tests that will come out with every application there will be a bunch of test tree if you have an application installed you will see that test in your in the ATF and how you can make use of that particular test the only thing the thing that yeah you have to remember this test cannot be run as is because these are dependent on demo data and this is testing out of the box functionality and not your customization so you may have to change the customization but what to test your customization basically but what you want to do is you can go here you can group by package so this is what the the lab is all about so you can say okay I want to group by package and when you do that you will see all this different application here so far for example in on this particular instance I have changed management and incident application installed so that's why I have see all this QuickStart test for that particular application so you can click on that one pick one and then if you see there is no Run button here because we don't want you to run this test as is it will never pass it will fail if you don't have demo data it will never pass and if you have customization done it will not pass but what you can do is you can make a copy of it so here it wants it it tells it that okay these are all demo data dependent and these are not meant to test your customization but you can make a copy of it and then you can just make changes here whatever you want to if you want to add a new new step here if you have a customization done nearby you can add a step but this is this is like a QuickStart you don't have to start from scratch for example how to change how to test a normal change right so there is a quick start for writing normal change request type so you can make a copy of that if you have a customization you just plug in those steps in between remember you can reorder so you can reorder the steps or while creating the steps as well so this is a last and final lap that we have how to basically use QuickStart test so what we have done today has a recap of the lab is basically you created your own step the next thing thing you did was you you use that owns you create the step that you created in parametric testing and then you use page inspector to inspect a service portal a page for on custom widget on that page then you used another test to test that particular page which we inspected and then we saw how to use like QuickStart tests what you can do with that now the final and the last piece of this particular in the lab is what's new in New York right so what happens when it there's a single lane traffic right and what will what happens if this particular car goes slow and you are over here towards the end you feel like getting down off your car and start pushing all the cars that's in front of you right you feel really bad but in order to fix that water what we do is we have multi lane traffic right and then everything goes smooth but then there's a problem what what happens if both of both the cars in the different lane try to go to the same place there's a crash right so that's the same analogy I mean I failed here in my creativity to depict parallel testing and what the problem is with parallel testing but this is something like that right if you if you run tests in a sequence for the tests that is running is very slow all other tests are suffering because that the test is running so slow when you run in parallel you have a problem where if two tests are trying to modify or use the same resource you may get some unexpected results right so in New York we added a capability to run tests in parallel but we also added capability to identify those conflicting tests and run them in sequence so that you don't have to go back and debug and see why unexpected results is it because there was another test which was trying to modify the same reserves that this particular test was trying to do so we did support so from New York you will have parallel testing support let me show you a little bit about what that is I have to get the password you see what so while she's doing that a little bit else about parallel testing so the ideas that we're hearing a lot from customers that to test designers want to run their build test and debug them at the same time this in New York that's gonna give them the capability to do that there's there's a lot of issues with overlapping conflicting issues like two tests want to touch the same incident so one wants to change it to close while the other one wants to like add a comment or or change of maybe to reopen or something and that will definitely break one of the tests will always fail and when you try to roll that back it might get confused about what to rollback so we built a solution where we track what we're interacting with in a test and then we record a bunch of metadata for that and the next time you run that test let's say I'm gonna go lock all the things that I interacted with in the system so that if someone else tries to run and touch those it has to wait in line to run so if they're touching the same record the wait in line and we realized that that will confuse people wondering why their test still sits around and waits like why doesn't it run in parallel why isn't it why am i waiting for someone else for five minutes for an eye test so in test design we put some messaging on records to say oh it's probably not a good idea to change metadata while you're running lots of different tests like - to test changing Syst properties or to tests touched the same incident 0:01 so only we'll let you know hey it's not a good idea to to open and modify an existing record why don't you try creating record so that's something that will be well documented and be kind of well displayed in the test forms only it's New York so now I have this to test and if I run I can just show how it will run in parallel I have like open in two different browser if you see this test is running this test is also allowed to start running so the two tests can now run in parallel the other thing that I wanted to also show is that it is what we will do if we identify the tests that are conflicting with each other right so so here if you see there will be a mutually exclusive tab which will list all the conflicting tests you can add add your test as well if you if you don't want two tests to run in parallel you can add here I this will also show like what kind of record they are conflicting with each other and then you can also look at what test ran in all that one I have to see the test results of these eyes right to see the test that ran in parallel at the same time so that is something that you can see here so when you go to a test result you will have a parallel test so you can this is this is just in case system is not able to identify conflicting test and you still find one then you can go and troubleshoot here we we try hard to find it out but there could be cases where we couldn't like the systems couldn't at all due to some other scenarios then you can just come here and debug so this is what we added in New York apart from that we added two new steps which is basically a steps to actually the add attachments is two steps this one for record server side one person yeah yeah okay so we added like a step to upload attachment and that there's two steps to do that one is from the server side and the other is to the form so there are cases where opening if there is a business logic which says that opening an incident require an attachment or closing an incident require an attachment those kind of scenarios who would not be able to achieve without this particular step and we got those risk requests so we added those steps to upload attachment and the third one we added is to create user assigned role to that user and then use that user as an impersonate Ted step so what happens is when you clone down data and data the Society of the data changes so if you have a hard code value for example a hard code user or a hard code incident number and a society changes you will see not referenced kind of so you have to go back and you have to modify your test if you have a self-contained test if you use this particular creative test and improve impersonate this one you will not face that problem so you will have a self-contained test even the cloning down it will not impact your test kind of so it is it is a good practice since now that we have this particular step it is a good practice to use that I'll just point out that those are convenient steps but you can still create a user through server steps as well and add rules to it and then impersonate as that user so it's possible to do in Madrid right now this is just a single step where you set all of it and you can start parameterizing roles on it so and then there are all this resources available for error for you for your health and we are also launching an e-learning training - today it's today right May 7th this is the coldest I've seen free so this is this is it and this is yeah definitely a very good training and if you want to get started on ATF you can use this particular train yeah they should be posted online it's a graph that many zeros you alright correct yeah I'll give that feedback to our training for you that's a good catch say again yeah tag to you you are tiny URL no I don't have that this is what I got from eLearning but to your question I think so yeah the presentation should also be available I exactly don't know where but I'll ask marketing to communicate that if it is available oh you want you want to do that yes we have like three more minutes if anybody has any questions should it right now yeah [Music] that is because as I said if they are using a hard coat request item number then and the society changes when you flown it down yeah yeah yeah so the thing is we say you you have to you have to build a self-contained test so you need to generate that request in a previous step and then use back reference to use that instead of using a hard code value and taking a request already existing in a table so that way that's just the test itself will create the request and then you can use that request atom in the strap itself so you have to modify your test and make it a self-contained yeah that's a problem that's why we have there now that create user step results yeah yeah hold on all right can you yeah go ahead Arvind Joe can you create a tiny URL please so hold on one second I'll stick around after the lab I can stay for a while just copy the URL training URL so can you create a tiny URL sorry yeah go ahead so you have it but as new items developed what is that transition to now update ETF with now the new stuff that we created on the platform so we just copy our 2.0 test scripts in and make an automated yeah yeah because today so 2.0 test is you're managing test application right so and this is you create the test so then today there is no integration between those two products so you have to create it we have a backlog item to create that integration to build that integration between the two products or you did create one okay so eightieth now thank you it depends upon your instance so how many worker threads you have on your instance - to our little test you will run and number of nodes so this is worker threads x number of nodes - - it is in New York yeah so for anyone else who has questions for me but I don't have time I'll be over in the now platform like near creator con there's an now platform pod and there's a section on upgrade faster I'll be there like late morning or late yeah that every Morrow a Stanley's testing will be is planned for our land or not committed yet but we are thinking of doing it in in Orlando next reference icon not today but that is something is on our backlog to do yeah

Channel: ServiceNow Events

Views: 2,417

Rating: 4.5294118 out of 5

Keywords: Knowledge, ServiceNow, CreatorCon, Now Platform, ATF

Id: Ktdj0WuN5fc

Length: 88min 31sec (5311 seconds)

Published: Wed Jul 24 2019