Basics of Application Modeller - Part 4 Win32 AA UIA Modes (Blue Prism)

[Music] hey everybody Dave here this video is part four of the series that I'm calling basics of application modeler if you haven't watched part 1 2 & 3 I'd suggest to watch those first otherwise we're gonna jump right into part 4 where we talk about the spy modes win32 AAA and uia now that we have looked at HTML elements and a little bit about spying them and some of the best practices that I would suggest whenever you are choosing attributes for them and then remembering to change the element type sometimes some of those same things apply whenever you're working with win32 active accessibility and uia mode but I want to go ahead and and actually go through the motions of creating attributes for those as well so I'm going to start here with win32 and then move into the other two the mayor may not be in a separate video so I'm going to go ahead and take the same application model that we've already defined and I'm going to clear these and Reese PI each of them in a different mode before we get into active accessibility and uia mode I want to cover win32 so I'm gonna create another element and we're gonna actually no we're gonna call this we're gonna do it up here let's see screen home page identify okay and notice that we've got HTML mode press the Alt key and now we've got win32 mode you can't see it there it is so I can't put my mouse over it otherwise it'll disappear on me so it's going to the other side of the screen you just you just can't see it in your view of my desktop so on the left side you see that it has win32 mode and if I move around you can see that it identifies a lot of pieces on the in the browser but none of which is going to be inside of the web page the HTML part of the page is not accessible to win32 mode however it is still very useful and one of the things I very commonly do is I'll spy the entire window and in win32 mode and I'll use that for activate application so I've just spied the whole window sometimes I'll just do like you know an inner frame or whatever but the whole window should generally work so it's an element type of window and I'll briefly mention this I'm not even gonna go gonna go over these others because I think they're very self-explanatory if you watch the HTML video you saw me change the element types and that can affect the actions available in navigate stage so let's just do that for one of them here so this is a window type element I'll click OK go to the navigate stage add the screen homepage in and then you can see I have activated application click window click windows Center close drag drop got a couple little mouse click things hide window maximized move resize whatever I can hole bunch of stuff very useful things and then if I click ok go back to application modeler and I changed this to something like a button click ok go back and navigate stage and change see the action you'll see that there's a limited set of actions available to me and that's because buttons can't do all the same things no windows can do so I'm gonna make sure that you recognize that that is absolutely a thing and even to a larger degree with windows with win32 spy mode same thing with the rest of these and it doesn't even matter that the element isn't a tree view I can select tree view click OK go in the navigate stage change the action you can see collapse item expand item whatever else and so it's important to you know choose the right element type but those will not work if I were to run it on anything in this web page right now or I guess on the the window itself because that's what I justify ok so back to this element types are important I'm gonna leave it as window and then I'm gonna do my best practice ok looks like there's none that are selected that have an empty value even when 30 to the next thing I'll do is just verify do any of these look like they might change while the application is running and if the the answer is probably because visible and screen visible I believe can change I've tested it some seems like it happens right because it's screen visible if it's not visible on the screen it's not so you know as a possibility of changing but the bigger important best practice that I suggested in the HTML video or section in this video however I organized it I don't have a good reason for using visible in screen visible there are no windows that are invisible that matched these parameters so I'm gonna deselect these and I'm going to real reorder all right we've got window text Google - Internet Explorer enabled and this is a case when enabled is actually very important when 32 enabled this is definitely used so if I had a pop-up let's say let's see can I cause like a modal pop-up to happen over here print maybe okay so in this case you see I want to try to click this background over here this window is not enabled right now so if I try to spy this or highlight it you'll see unable to match the window with the query terms if I click cancel now it will be able to find it because it's enabled again so the enabled attribute is actually useful and the way that you know whether it's enabled or disabled is what's called a modal dialog pop-up if you do something that causes you to be an unable to click the window below this is considered a modal dialog which is now the in enabled window and then it disables the window beneath ok so in this case I do want it to be like this because if something happens and it caused me to be am unable to find this window and I'm trying to work with it I actually want it to throw an error and not work and not pretend like it works right like if it's disabled it could still find it as disabled back there matter of fact let's just explore that right while we're here ok so we've got the modal dialog up and I can't click this back here it's disabled but if I change either I just deselect this click highlight it'll find it I can also check the enabled and change to false and it'll hide it'll highlight it I can also change to true and say not equal to true and it'll find it there's a number of different ways to do that and then but I'm gonna leave it here as enabled equals true so it should not be able to find it it'll throw an error for me whenever I try to work with it when that's disabled class name this is a time when actually I think that it is fine to leave the class because this ie frame this is not going to change this will always be ie frame and I think that it does help narrow it down because if I deselect AddThis I could have a window open like notepad plus plus that has the same name and it would conflict with finding it so if I have the class name as ie frame then it'll narrow it down to specifically something in Internet Explorer rather than you know conflicting with other applications I already talked about with HTML talked about X&Y and how those aren't usually good to use unless you're using them for a reason same thing for width if there was another window that's conflicting with this one that's invisible let's say then maybe you could use this to go like with greater than but you don't want to select with equal to 779 for example 7 7 9 pixels unable to match the creators okay wait wait okay so it'll find it because I haven't resized the window since I spied it but if I resize the window even slightly now it's not going to be able to find it anymore if I say that the window is greater than 10 it'll find it again because the pixels is greater than 10 so I'm going to deselect that though and not use it ordinal this maybe this was true in the past that ordinal was a useful attribute I have found it to be extremely unreliable and I would suggest you just absolutely never use ordinal the ordinal being order like it's kind of like match index I imagine except that ordinal maybe is something that the application actually defines itself but I found various apps to change what the ordinal numbers for certain elements I have no explanation for that all I know is that that is like I confirmed that to be the case and I have since then and never gone back to ordinal especially because it's only useful in win32 I can't remember but I don't think it's an act of accessibility in UI a hajikko see but match index is always going to be your best friend and you can pretty much always choose it unless again you want it to throw an error if there are two windows with the same well in this case so right so it's still finding it and you could not have match index selected in the case of win32 it'll still find it really fast because usually when 32 is extremely fast okay so let me show you now what I primarily use that for and that is activate application so I have screen selected activate application this is what you need to use whenever you use send keys so you know I I'm there's more you need to normally do with this but I'm gonna click into that box let's see if this works then I'll add here do you like scurs mm-hmm oh that's oh here hmm I'm crazy so do the root node and then global send keys and then we'll just send some random text and we'll put pause after each step of 0.5 seconds reset and do this so what it did was it activated the application so it brings it to the foreground and you saw really quickly I'll do it one more time I don't know why Cho I don't know why it typed of ROM that time whatever but anyway it it typed random into the blank now granted recognized that I would need something in between here to actually like click into this blank I felt like you know I kind of simulated that by clicking into it ahead of time so that when it activated this application and this element was already selected but this is the pretty much this is the main reason or the main way that I use this this type of an element that's win32 there's another way that I use it too that's less often and I only do this when I really need it but that is minimizing and in maximizing a window that if at some times there will be other windows on the desktop that will just want to take control of everything and the best way that I found to bring an application to the foreground is using minimize maximize or maybe it's just a second way that I'll do it so I'll put a put like a two-second wait in between cheese so that it should minimize wait two seconds maximize way too seconds and then let the process flow continue so let's play it so it minimizes it and it maximizes it not only is it at a stable size of a screen but it's also you know in the foreground it's like now I can send keys to it or whatever else all right now we're back in application modeler and I'm just scrolling through this list to see if there's anything else that may be useful to use match reverses also here I suspect that that's not usually going to be necessary when 32 is so fast and typically match reverse is used to search from the bottom if it's if there's a lot of elements so if you are using an application that when 32 works with has a lot of elements you could use match reverse typically don't use most of these other attributes sometimes ancestor text I feel like might have text in it but that often also might just equal window text and I think that's about it for the basics of working with win32 some things to recognize with win32 is that because it can go so much faster that's one of those times when you really notice the need for intelligent wait stages every single thing you change on the screen for a win 30 you know for win32 spy mode like when you're working with that way always always have an intelligent wait after every click to change the screen you know you enter a value into a field that should then be read by the app and update the screen somehow moving from one screen to another opening the app closing the app whatever you should always have an intelligent wait after it just to make sure that something is the case or is not the case is a window here yes it is or is the window gone now okay good I can move on because it runs so fast we're gonna compare HTML uia and active accessibility and it's really just gonna come down to you testing how things work in each app and get familiar with the different types of attributes I think the information I've given so far is a good start and we can definitely get more advanced but but I'm finding then I'm making this this video extremely long so I'm gonna try to limit what more I talk about so I'm gonna spy HTML for this this image okay so let's do that three times we're gonna do I'm gonna make these children four active ik soaps I'll make these children four active accessibility there's an Erica so I'm just doing it to fast so kind of go a little bit slower sometimes when you're changing the spy mode so here's UI a mode and active accessibility have to cycle through these by pressing alt active accessibility I actually tend to use more than UI a mode but something to recognize is that Microsoft did come out with UI a as opposed to active accessibility so UI a is newer and it's supposed to replace AAA mode but what I found at least in blue prism is that it doesn't have the same capabilities exactly and one works better than the other in some cases so I find that you kind of just go back and forth between the two active accessibility at this point may not be in Chrome I think that it's not in chrome automation so you would only have UI a as an alternative to HTML and you just be coming back forth between those two but in Internet Explorer a mode still works great so I use it let's look at this oh okay right so let's do let's just remove these blanks you already see it's not looking great for UI a mode active accessibility remove the blanks and now that this has come up I'll treat any value that says self as blank because self is not a descriptive you know limiter it's not gonna help me narrow down to a certain element okay all right you can see that between UI a and AAA there's a lot more active accessibility attributes than there are UI a don't let that fool you though because most of them are not useful if I sort by value and then you'll see that a whole bunch of them are boolean values true or false and I would imagine that many of these you went through and you checked them all that and it's impossible to get some of these to actually change by no matter how you interact with the window you can't get these to change to true or change to false so what you want to look for when you are picking elements are picking attributes to help you identify a certain element is to again narrow down to the smallest list possible of reliable attributes parent role link that might be useful parent default action could be useful but I'm gonna remove it for now class name internet explorer server I don't see anything wrong with that that might help us and then the value we're gonna do the same thing here as I did with HTML and change it to wild card like this we'll go ahead and put our star here in case we have other environments that we're working with I'm gonna say since these are the same that we probably don't need this in this case I'm actually going to use parent value and changes the wild card remove value and then role graphic because that's helpful because that narrows to an image and let's get a match index in here ok so we can find the image it's working that doesn't mean that it's going to continue to be reliable here's ancestor text that's actually a one that I often use that can be useful and then you don't necessarily need this URL because I can do this but maybe I should explain actually what I just did - I typically don't like when there's extra spaces at the beginning or the end of a value and so I'll usually change that to stars and then the match type to route to wildcard and I have done this for years now and this has worked well for me it's not always perfect - there are times when this part can change so you know you can even do this if you know there's only going to be one window of Internet Explorer open but you know sometimes you need to have multiple open at the same time so element count zero could be useful but I'm going to deselect it and force myself to determine I need it later parent role link will remove that enabled we will remove that and [Music] highlight okay so it looks like we're good on this and we followed a similar pattern that I did for HTML where you're just kind of limiting to as few as possible and you know why you're using them so if I look through these I know that this is sort of like the window text that's up here and I'm matching to the window but I also kind of want to match to the URL that's up here that helps me to know that I'm on the right page too so maybe I could just have one of these but for now I'm gonna use them both because they they are true values class name I'm using actually I don't need class names so I'm glad that I looked over that I don't need class name Wow look how much slumber that was so this is what we can learn through testing if I check that back again what was it class time let's organize by name and a class name sort on match type or a match it's finding it faster with Internet Explorer server selected so I don't have an explanation for you on that let me know in the comments if you know why that is but the fact that I'm seeing the important the you know an improved performance by having it selected I'm gonna leave it there let's jump over to UI a and try to see if we can do the same thing what I often find is that UI a just does not have enough specific identifiers to be able to narrow it down without using match index 5 or matching x6 or some crazy thing like that so I can't use the name of of the element Imageworks will probably want match index I guess if it's a hyperlink then that helps us narrow it down further wait what is this this is control type okay UI a height okay we'll probably want to do this because we know that there's that other image that's on the page so we'll go like height greater than 40 and it's still it identifies it let's sort on match so we have match index control type UI a control type so we got the parent control type and the UI a control type so it's an image and its parent is a hyperlink so when you click it it's using the parents link to go somewhere so if I click on it yep it takes me somewhere all right good to know and so I would have to say this is UI is not what I would use in this case I'd use HTML or AAA because I feel like those are a little bit better one thing I really don't like about UI a is that it it frequently doesn't give the media ability to identify what window it's working with sometimes that can be good because maybe it just doesn't care what window I'm working with but I like to have a positive you know specific recognition of this is the window I'm dealing with because it has this window title and I know it has that window title if it has anything else I want it to fail because one of the important things for me is that the robot only does exactly what it's supposed to do and and cannot have the possibility of doing anything else and that if it would it would just fail we got the same thing here again I'm gonna go look at the navigate stage for this element and see what kind of actions we got drag drop focus verify global stuff and then if I go back to application model change this to be like a menu item and then navigate focus okay so it gave us it like I think it gave us some different elements or different actions let's try try tab control that should be different yeah select tab so it'll change it based upon the type of element that it is as well for UI a let's try hey now for this okay so we've got default drag drop focus some of the same stuff if I go into application model so I think that you've likely seen that this is very similar you know Wow just verifying okay that UI a and AAA are very similar and the reason for that is that UI is a replacement for a it's just a bad replacement it has its purpose but it's not it's not great loot prism has not always had UI a mode I remember when it came out and everybody was like oh my goodness this is amazing I'm gonna be able to identify things I've never been able to identify before and in some cases true like in various apps UI a will help you to identify like a contextual pop up that win32 couldn't identify and obviously if it's a desktop app HTML it's not gonna help you hey might have a hard time with it and UI a was able to do it so there are times when it's very useful but in my opinion it is it should be your your last choice before using surface automation should first try to use win32 because it's so fast and reliable then if it's in a webpage try to use HTML next active accessibility and then after that UI a mode and when I mention that another thing that is better about UAA mode compared to uia is the AAA has match reverse and UI a does not I don't know why that is I think blue prism could include it I should probably ask for it to be included a min but I'll do I don't know so if you're trying to spy web page using UI ara a and an element is way down at the bottom of the page I'm going to use a everytime because it I can do match index 1 or match index 2 or whatever it is I need and match reverse true so that it looks from the bottom up and that's all I have for you for those other types of spy modes I know that I have really focused on HTML and describing that but I wrote what I really feel like is that the types of things I described to kind of apply to all of them and the only differences between them is some of the types of controls on the page some types of elements and then some of the actions you can do with them but the ways that you identify elements the attributes you select and the types of values you put in them it is very much it can have a commonality too to make sure you're limiting the selectors make sure you're limiting the attributes and that you are choosing them on purpose you don't use blank values unless you're doing it on purpose and you got your various methods to identify elements as opposed to invisible ones by using greater than for height and stuff like that thanks for watching part 4 of this series of basics of application modeler I don't yet have any other videos for this series but if you have any questions or comments please leave a comment on that particular video and I'll be sure to go back and note those and make a list and then if I have enough of those that I'll make a part 5 where I will do a follow-up and I'll say here's the things I forgot or look at this I said this completely wrong thanks for watching and happy application modelling

Channel: Dave The RPA Guy

Views: 2,020

Rating: 5 out of 5

Keywords: BP, Blue Prism, BluePrism, RPA, Robotic Process Automation, Robotic Processing Automation, IPA, ML, Machine Learning, AI, Artificial Intelligence, Intelligent Automation, Automation, Technology, Innovation, Application Development, Dave The RPA Guy, Tutorial

Id: hO0CX0-roBY

Length: 25min 11sec (1511 seconds)

Published: Sun Feb 16 2020