Multi Pages Web Scrap (extract) with Power Automate Desktop #45

here and in today's episode we'll be talking about paging about Automation in power automated desktop web scrapping web scrap because this is a very important topic you will encounter some form of pagination in most of the types you will need to do the work scrapping and the they are very different one from each other the developers attempt to code them differently so we will be talking about the example a simple example CSS selector will go into the next page and after that I will just show you the tools you have to work with different forms for example tool Loop and try to get the element value based on the page num number the text value because this may be important in some cases so stay tuned in and let's start [Music] okay so I'm on this crap this sitecom and I'm using the the okay forms you can copy this website address and from this page I want to scrub team name the year the win percent ratio and I want to use a pager which will transfer me to the next page until I'm after 25 pages and I'm already and I've already scrap all the data okay so how this simple flow looks first of all I'm mounting new Chrome instance I'm launching the the site as you can see there is there is page number exceptions I'm using the first page I'm using the maximize window to avoid any mistakes and errors regarding resolution Etc okay and Main action of of my flow is extract work data from the web page and I'm using the the input which was the variable created on the launching Chrome I'm extracting all the available data I'm sending the physical clicks for the next page usually that may cause some issues if you are not using this not all sites are working without the physical clicks select the next page and I'm just creating an extra spreadsheet with the data so when I'm using this action I will open the the web live helper when I'm on the the when I'm on the web page which I want to subscribe I mean it will open on the any web pages but if I have this this pop-up open then the web helper will will open as well okay so here I've already told the web Scrapper that I want to scrub the data from this this and this let's click finish now sorry let's go let's open again okay now it's working you have to confirm that the red border is is visible it's I've clicked here with the right Mouse button I extract the text here and the in-person ratio and I've also used the second row just just for power argument let's go to know that I'm scrapping the table now the the handpick data okay I need to go to advanced settings instead this double to yes to use the paging click OK and find the page or the paging button which will be visible here so this is my button I can answer here and set this element as a pager so let's do this now let's go let's go back to to my flow and in here with regular extract data from web page you won't see these three rows these three options I want to extract all the available data then I want to send physical click for the next page so it will iterate until it cannot until the iteration is over because the element will not be visible the power atom in the desktop is that smart to recognize this I click save I click finish here and let's run it open page correctly it iterating through the pages as you can see I would bother you with this so let's meet after 25 pages are already scrapped and as you can see my book already was created or if all the results from the pages let's go let's go all the results for here great okay so this was the the first example this is how you usually work with paging okay but let's assume for a moment that we have a offer a harder example here if for first I we've used this button which iterate us through each page but sometimes in some examples this button won't even exist and you would have to iterate on those buttons with the number so you have to grab this number and you have to know how many pages are there so in most cases you would have the last page visible as well so in order to do this you would have to extract data from web page and extract the last number which you would see here and with this number you would know how many iterations you have so I've already created an example showing you how to iterate two pages first we have the launch new gram this is this the same as it was then I'm setting the variable which will take care of iteration number and I'm setting the value for one and I'm looping I looked for four types but you can look for 24 times because you have you have 24 pages or as I already told you you can extract the last page number and assign variable with the cell variable and you will have the number to iterate based on the variable because you can use the variable here but let's let's keep let's keep four for now just see how it's working and you will extract this data also but you will extract this data I'm using for now a variable you'll extract this data one by one so you have to copy the actions we already had and extract this the same data which we already extracted then we're going to iterate the iteration number and added plus one and then by getting details on the UI element in window let's look at this action now UI um details cui up we will use the the button here so what you would like to do is to use the URL element which you have and if you don't then you will need to add this UI element you see that pop-up window the element picker and we will just just use this um this element with the control and as you can see we have which is called hyperlink 2. okay so where I have this element I have hyperlink to which is element I've created I can go to the to the right part of the screen to the UI elements click on my hyperlink as you can see there are there this is the selector and we have elements from this but watch I would like really to do is to create it with the text editor and as you can see I have index editor I have the number two so what I need to do is to change this for iteration number which is my variable I've I want to do it again I've already created this in in my hyperlink monetary which use the iteration number and this is the indication that this is a variable with the double quota so I've just changed the two or this this number you can save this and then you have a working selector so this will click the the button number two and then I'm clicking this this button scrapping extracting the data again and again and again and again and after a loop is ending I'm closing the browser so let's see how it's ironic page opened correctly it's scrapping right now hopefully and as you can see the button towards pressed now the button number three the button number four the button number five and it's ended okay so after this iteration can you answer yourself does it work correctly you can pause the video now and the answer is no because you can see the data from the web page has 25 rows and three columns which means that it only iterates once and why is that because the variable data from web page was overwritten over and over and over again with each iteration so what we can do I propose we just keep this in Excel which will append to the Excel to the to the end of the accent so let's go if launch Excel let's go with the blank document let's make instance visible this and that's another new Excel process now very purpose that will be an Excel instance now we want to do right to Excel worksheet the data which we have the value to write we are using the data from the web page and the column let's not go on specify itself first of all yeah yeah I get the error but let's go with the get first First Column and first row let's only use the the row cool so that means that we will be writing to the first through your free row we will use the first three row here and we'll use the a here cool and let's see how it's running as you can see we are iterating correctly so it means that we don't have any crucial error already let's see how the books is working as you can see we got the four pages so it iterated correctly uh four times which means that we are done here but we can do uh we can also in advance of the blank document we can also go and copy this to write the next worksheet and write the column names if we would like to but this is not necessary for for our our iteration to work because it already does so this this was the hard example of how to web scribe on the data we have from the website by using the the selectors and iterating through each selector based on the name you can also have other attributes like ID I like like a ref or something like this so you can iterate for many of different attributes but you now know how it's working so so it's great I hope that this is something useful and you'll learn something new and if you do if you did so you can like subscribe Etc and you can look at my other videos because I have many power automate desktop at Power Platform our apps content generally Microsoft content on my channel so it was a pleasure and see you soon

Channel: Zbigniew Lukowski - Tutorials

Views: 4,099

Keywords: power apps, powerapps, power platform, powerplatform, zbigniew lukowski, web scrap, web extract, extract data, scrap data, how to web scrape in power automate desktop, web scraping power automate, scraping power automate desktop, power automate save web page, power automate desktop tutorial, web extract power automate desktop, web extraction, web extraction power automate desktop, paging, pagination, paging power automate desktop, multiple pages extract power automate

Id: oJ9RuMhmNI4

Length: 14min 41sec (881 seconds)

Published: Mon Jul 31 2023