Solve reCAPTCHA challenge with Power Automate Desktop - no human intervention! (new version)

hello there it's graciella on the side of the screen in this video we're going to learn how to solve recapture validation using power animate for desktop and this process can get a little bit tricky because sometimes you don't only need to click on I'm not a robot but also you get this visual challenge in which you have to select images from the screen so to make this easier what we're going to do is use a browser extension we are going to be using the baser capture solver extension and this example is based and oriented in Crum but this extension is also available for Firefox and for Edge so whatever browser you want to use you just need to uh Google the extension name and then find the right link for the browser you're using and then just click on it and then just just install the extension what this is going to do is whenever you click on I am not about checkbox and then you get the challenge you're going to get added this small icon and once you click on it that is going to solve your Challenge and this works for both visual challenges and also audio challenges and we're going to see how to work with both of them throughout this video and just before jumping into it I would like to thank my amazing colleague anony ernandez who was the one who came up with this brilliant idea of combining a browser extension with our automate desktop to solve this challenge so now I already created an empty power automate workflow and let's go directly to the record order for this example we are going to be using this web page so I'm just going to copy the link then I am going to use the three dots and then go to launch new browser and then we're going to do Chrome and I will just select my link here and then also add it on this section now what we want to do with this web page is just uh add here a number and then validate that word not about finally we just want to click on this on this blue button on this area and that's it so let's go ahead and refresh this and then record you can see part automate is recognizing what I'm doing I populated this input text here and it's recognizing it also recognize some other stuff so I'm just going to delete it then let's move to the bot validation and you can see it kind of recognizes our checkbox and then we move to the next action which is just clicking on the extension icon and now we are validated and then we just click on this blue button finally we click on done and I will for now close this I will also go back to my part automate flow and we're going to see how part automate behaves now and we're going to see which changes we need to make to make this work so let's go ahead and run it and we're going to see it's going to fail at some point so you can see that once it reaches the section or the step in which we are supposed to click on the I am not a robot action or I am not a robot checkbox you see that now it fails and it is because sometimes power automate doesn't recognize this bot validation as part of the browser that means that power automate can't communicate with the browser and give it commands or instructions on what to do so the way to fix that is instead of using browser automation we are going to be using UI Automation and that way power automate won't communicate through the browser but through the actual interface even though we are here in the web page interface power automate makes a difference between what a browser automation is and what a UI automation is so I will have to delete these two steps because we know that the but wouldn't be able to recognize that as part of a web page and I'm just going to open this UI automation section and I'm just going to use this action that says click UI element in window and I'm just going to add it right after inputting my field into the text field and here is asking me what do you want to click and what we want to click we still don't have anything added because this is a UI element and we were previously working with web elements so let's just click on ADD UI element this is going to open this picker and now we can pick or I am not a checkbox we do that by using control and then left click you can see that now it is automatically recognizing my checkbox I am not a robot and then let's click on Save we know that sometimes after we click I am not a robot we get an additional challenge other than just clicking on the checkbox so we also need to account for that in our flow so I will just go ahead um to add another action to click on a UI element and I am going to click on ADD UI element what this is going to do is just add our picker here and then I can just go ahead and select my icon for this first section we are going to work with the audio Challenge and then we're going to show how we can switch to the visual Challenge and solve it by using the extension icon so we're going to click on here doing again Control Plus right click and again this is going to automatically recognize my element and then let's just click on Save now something important to mention is that sometimes the I am not red validation takes some seconds to show up so it's important to always wait for the element to show in the screen for our butd to try to click on it so I will use this action that is called wait for window content and put it right before I am not a robot here I will will select that I want to wait for this checkbox I am not a robot to be present for my but to continue and then we also know that sometimes we click here and that automatically validates that we're not a robot because that depends on the number of times that you have used the capture validation in the last 24 hours so sometimes we're going to get this other window and some other times we won't so we need to also account for that in our workflow so I will just go back to my animation and you can see here that I selected twice this solve the challenge button so I am going to select the first one and do select save and I am going to delete uh this number two because it's the same item so I just captured it twice let's close this and now after we wait for the content we need to check if this other challenge is showing up so where I'm going to do is just um use this conditional that says if window contains so after we check on the I am not a robot we are going to do this if window contains and if this contains the solve the challenge icon then we are going to continue to the next to the next steps which are uh clicking on this solve the challenge button and then if we don't have that button then we just continue with the next step which is just clicking on the blue button that says check and something else that I'm going to do is just add a little bit of a delay in between some steps because sometimes it can take a second or two for all these B validations to show up so I will add here a weight of 2 seconds 3 seconds and then I am also going to add a weight here of two seconds okay so now let's see how ourf flow is doing and let's go ahead and run it but before that I will change this uh my Crum to open in a maximized mode so let's go ahead and run it and see what happens you can see that it solved it and then it tried to check our policy number but of course it's not a valid one so it's not found so you can see that it's initially working so something that happens sometimes with the capture validation is that the challenge shows more than once so we also need to account for that in our workflow so let's go back to part automate desktop and here we are going to add uh something that we call labels in part automate for desktop so I am going to create a label here I'm going to let me capture steps and then let's click on Save what we are going to do is once we check if the window exists and then we solve the challenge and then we wait we want the flow to go back to that label to try again and see if there's a again another solve the challenge icon and if it's present after we have clicked on it that means that the validation is asking for another challenge to be solved and we're going to do this until this stops showing up so for that the only thing that we need to do is to go to the flow control section and then use this go to action so we drag it right after we're waiting here is going to ask us where do we want to go when we reach this step and what we want to do is go back to the capture step validation so whenever we click on the checkbox and if the challenge is showing that we need to solve it then the bot will solve it and then if there's another challenge it's going to go back to step number seven and check again if the challenge needs to be solved again and this is going to happen uh for every time that this icon is being shown in the screen so every time that we are requested to solve a challenge debot is just going to solve it something that I like to add here on this click UI element in window that is solving the challenge is I like adding this on error section and then I click on continue to run flow because sometimes it takes really a couple of seconds for this um check to happen so the icon keeps showing even though it really just loading or validation so I like to add that just in case by mistake the automation recognizes that it still exists in the screen and then tries to click it when it's just actually trying to finish the validation so we're going to add this here to avoid any error and now let's run this animation again okay and now you can see that it properly added the text field then it added the I'm not robit validation and finally it clicked on our button here in the section of the screen now what happens if your audio is not working or if you have issues with the audio validation you can always use the visual validation as well so for that we are going to also use the click UI element in window so let's go ahead and add that step into a canvas so I will add it just here in the middle and then we are going to create a new UI element into our flow this element is this little eye that we get here right next to our solve the challenge icon remember it's control and left click and that will add it as a UI element into our workflow then let's click on save so sometimes this validation will either show us the audio validation or the visual validation so what I'm going to do is to check again if this visual challenge is showing in my screen and if it's showing I want to click it so let's do this and then select the get a visual challenge button we're going to put this right here finally I will also add a little weight in between because you know that moving between validations can take some seconds so in this case I'm just going to put it to two seconds click on Save the object that part automate will recognize as the button to solve the challenge will depend a little bit on if we are trying to solve an audio challenge or a visual challenge so if we go to this section to the UI elements in our power automate interface and then we double click on our solve the challenge button you're going to see here that there's some code that is making reference to how part automate will recognize each of our icons saying that it's going to work with the audio and not with the image so what we're going to do is we are going to see how will power automate recognize if we are working with a visual challenge instead of the audio challenge so let's go back to click here into this in into any of the click UI element in window and then we are going to capture a new element so just we can see how part automate will recognize this element instead of the audio one so let's just uh click on it so it can be recognized okay I'm just going to cancel it because I don't want to save it and now you can see that power automate has brought a new U element and if I double click on it you're going to see that the code is pretty similar except that here it says say instead of audio it says image select so what we are going to do is to trick per automate a little bit so we can uh work with any of those two that we want to work so I will go back to the original solve the challenge and I will delete the one that we just had that was for visual Challenge and I will go back to this one and let's go back um let's select this area that is the one that was different so that is in this UI custom section and you can see here that the ID is equal to rc audio so we're going to do is just change that to contains and I will remove the audio word so this way part automate will know that whatever starts with RC regardless if it says audio or image it can work with both of them so let's go ahead and save and then click on run and now we got the scenario in which it's asking us to do double validation so in this case you can see that it went back to step number 7even to complete the validation and now it's doing again and we're going to get this until the validation is happy that we are not a robot even though we are a robot but um this extension really helps in fixing this and now you can see that it's happy and now it's going back to um The Next Step which is just clicking on this button and of course we get this error because this is not a valid number for this system we're just using it to validate the robot you can see that we have combined browser atation with UI automation to build this solution and to be able to complete these validations so that's it for now I hope that this was useful for you see you next time

Channel: Power GI

Views: 285

Keywords:

Id: 0jUYyZcRFvw

Length: 15min 19sec (919 seconds)

Published: Fri Feb 16 2024