Extract tabular Data from PDF using AI Builder Form Processing in Power apps

hello everyone welcome to this video and in this video we will discuss how we can use ai builder to page tabular data or tabular information from a pdf file and we can use it in our excel file let's say there is a pdf file and i want to extract the table that is presented on that pdf format it may be a invoice pdf or it may be a form 16 or whatever right if any pda form that contains some tabular the tabular data and how you can extract from that data from that pdf file and how that pdf file extracted tabular data can be inserted into a excel sheet that i store in a onedrive that we can see right also i'll show you how i can use that ai builder that will paste the data items from the pdf will be used in canvas apps and also in power automate so let's get started so you i hope you can see my screen right and this is the screen uh which is nothing but the powerapp ai builder right when you land to this powerapp screen you will see that in the left side you will find there is a ai builder and in the ai builder you will find there is option college build in the bill you will find different types of ai builder model that you can create right if you have created any model you can see here in the model section okay and if you want to create a new build you can a new ai model you can go to build and create new build okay so today's session is devoted to create you know an ai model uh that will read some pdf files like invoices and all other formats that you can think of so use form processing so i will use this form processing and i will give a name like extract data from pdf for example right now when you give this name you can see here it is telling that at least you should have five plus documents layouts with you because when you are giving a training to you know ai builder artificial intelligence you have to have minimum information with you so that you can train so the requirement for this is five plus document so i have five plus documents layouts with me if i go to my desktop i have already saved it so here this is five pdf uh formats that i have with me if i will open uh the pdf you can see the pdf looks like this so this is a typical invoice format where there is a inverse date inverse number you know uh the company and inverse 2 sales person and some line items with their line price unit price description quantity and subtotal sales tax and total price of the invoice okay so i have different you know the same layout but different data items but i want to train i want to you know build the ai builder so that it will read any you know template any pdf file and it will retrieve data okay so i have this minimum information with me now i will go here and click on create as soon as you click create it will open the ai builder designer screen and here everything starts right the first step is you have to choose what items you want to read normally if you click on this item the drop down you will see field and single page table is there but if you click this add button you will see three things at present you can only face fill information from a pdf or from a form but single page table which is in a preview mode this is used to fetch a tabular information and the multiplex table is also in experimental uh condition and that is not publicly available uh so this option you can use to fetch the table uh which is spanned from one page to another page let's say you have a you know multiple uh row uh of the table which is spanned to next page so that you can also use but anyway today we will discuss how to retrieve single page table right i have already shown you in other video how to fetch fields and today i will show you uh how to paste tabular formats and field also okay so let us first use field okay click on next give a name let's say i want to fetch invoice invoice date okay so you have to keep adding multiple fields that you want to page inverse date and invoice uh total price okay right now this this is the field but i want to read table so you have to click on this ad and go to single page table and click next here you have to give the table name what table you want to read from that pdf i want to read invoice line items right and you have to specify what columns you want to specify for this table that you want to read so just the default column one is there you just rename this as the first column is a quantity okay and okay and next column i will add for description okay uh description then next column i want to add unit price so this is the information that i want to read from the pdf so keep adding those columns that you want to read so next is line item price okay now my table is done now i have declared so at this step you have to choose that information you want to extract i am trying to extract two field values from that pdf and one tabular value so click on next when you click next it will ask you to provide the collection of documents because this is required so click on the new collection give a name here my collection image invoices you can also add new collections for your bot for your ai builder so in this invoices i want to add some documents so click on this plus button and click add documents here you have to you can choose sharepoint to upload uh data or you can choose azure blob storage but i want to use upload from local because i have stored some forms in local file so this five pdf files i have saved for training purpose for this ai builder because ai builder requires some minimum five minimum five uh formats or layouts to process so click on upload five document now it will try to upload the documents as soon as it is uploaded we can use it for ai builder training now click on close all are uploaded once you click close now you can click on analyze this is the area which we which is most important because in this analysis step we are trying to tag specific fields that we have already selected in the first step with the pdf specific items okay we have to map which section of your pdf points to invoice date or which section of your pdf points to invoice amount or which section of your pdf points to a tabular format this is important okay so it will take some a few seconds and it will open the designer second next step of tagging documents so add collection of documents is done now it is trying to open the tag documents for me so you will just keep waiting for some seconds now you can see this screen is telling me to tag the specific items from this pdf with the fields or tables that we have declared in first stage you can see these are the five documents i have uploaded and every document has a red symbol red symbol means you understand there is some error or some tagging has not been done if you click the first document you can see the two fields inverse data and inverse total price and tabular format is there uh fields are there but these are not mapped so what i have to do i have to map it so go to the pdf and select the section which should be mapped as inverse date in this pdf this date is the inverse date so i will just drag and release up to the date so i can map it inverse date like that invoice total price i want to use so i will just drag this one and map it to invoice total price now i want to map the table right this is the table so to do to map the table to tag the table you have to just uh drag from the left right left top corner and end it at the right bottom corner of the table and map it to invoice line items as soon as you tag to table it will ask you what items you want to associate with different columns so click on the first column because this is these are the columns that i have defined for the table right and i have to map corresponding items so click on this column and click on the quantity so quantity is mapped like description you just drag this area description edit you need price click on this and line item price right this is the line item price right so first row is there is done now i will do the second so keep adding okay now there is no third column so i will just delete this row and click done now you see the red symbol is gone and all are tick marked that means you have mapped correctly now you have to repeat this step for all four documents so let me do it quickly inverse data and inverse total price is already mapped you have to just map the table so i'll just select this again do the same thing quantity description right unit price and item price like that quantity description okay so you have to repeat for all rows this row is not there delete row click done now this is also done so quickly i will do the next three so i'll just select this and done done i am trying to map all these informations with the table items this is not required delete row click done and the third one the fourth one is there here also the i the table is not tagged so i'll just tag it then map it so this is very easy so it's no coding is required right now this is not required delete this row and the last one so in the last one i will also do the same thing because the table is not mapped map it okay now once it is done what you have to do everything is captured right everything looks good now i can click on next so if you want to go to go back and you want to check the data items the table is properly formatted or not you can click on this option click on preview table it will give you all the tabular information that has been captured right now if you want to edit you can also edit table right now click on next now click on train once you click on train button it your model is going to a training phase that means whatever data we have given to the ai builder that data will be taken into consideration and the ai builder will process the source files so that whenever you publish the ai builder that will taken into consideration so click on go to model now it will let me go to the list where all the models are stored and you can see here the status is training right i have to wait till the status gets changed to trend okay so it will just take two uh two seconds or two minutes so let us keep refreshing or just wait for some seconds [Music] and it will not take much time it depends on your server and you know the internet speed so once it is trained our next thing next purpose is to publish this ai builder until unless you publish this ai builder you cannot use this ai builder in your powerapps canvas apps or in power automate so publishing in ai builder is important and this for processing ai builder you just to read some pdf documents and it fetches some data items tabular items and field items from the pdf and we can do we can extract that data and we can do whatever we want with that data that is extracted from that ai builder okay so let us wait uh let me go to again refresh this and go to models now the status is trained right now i can click on this model and i can publish this as soon as i publish this i am okay to use this ai builder in any application like power apps or canvas apps or ai or power automatic so click on this publish it will take some seconds to publish it now once it is published published extract data from pdf which is the name of the model will be available to use okay if you want to test this ai model you can also use this quick test option so that it will give all you can test this ai builder whether it is retrieving data or not okay you can click on quick text click on upload from device i want to test this pdf so upload this so it will try to analyze the mappings automatically are tagging automatically that you have trained in the initial stage and it will fetch all the data so it is populated if you mark it here inverse that is populated and confident score is 99 percent this is the tabular data inverse line items right so this is detected this is the table that is detected and the total value is detected right now i can go and so once it is published i can i have tested it now i want to use it in power automate so i will go here and open power automate and also i will open uh i will create a canvas apps to face data first let us see how i can use this ai builder in power automate right so this is my power automate i will go to create so i will go to create a instant cloud flow i will give a name let's say fetch data from pdf i will use manual trigger flow click on create as soon as you click create it will ask you to you know we have to pass the input parameter because the ai builder expects a pdf file to process right so i have to add a input parameter of type file and let me give the parameter image target file and once it is done i will add a step and this time i will use a connector called as ai builder so choose the connectors and choose ai builder if it is not found here you can just search here ai builder so this is ai builder i can choose now you have to choose one of the connector or trigger or action my action is to extract information from forms right so click on this extract information from form you have to choose which ai model you want to use i want to use my ai model called as extract data from pdf now you have to use which form type you want to use i want to use pdf document you can also use jpg png other values now i want to use pdf so pdf is selected and which form you want process you want to process the parameter that has been passed from button click right and i want to associate that target file so this is the parameter from manually triggered a flow step so click on this now once it is returned i want to retrieve this data and fill it in excel sheet right let me go to one drive and let me check if i have a excel sheet or not to kill the data yes i have excel sheet i have already created line items book and in this line item book i have already taken four columns quantity description and unit price and line price right and i want to fill all these data items all the stable information whenever the the ai builder will fetch the data from the pdf so to fill this excel sheet data we have to use another step let's say excel add table this is the action so i can use add arrow into table and i can choose the location the location is onedrive for business because i have kept my excel file in one drive and i will use the document library called as onedrive and i have to choose the file the file name is line item books right once it has been selected you have to define which table you want to insert i want to use table one because if you go here this is the table one right if you go to the table design you can see the name of the table table one now i can go here and try to now once it is done i have to assign quantity description unit price line price all the columns that is displaying quantity description unit price line price now i have to assign one by one right so i will just search here quantity right now when you search quantity you will see there is a quantity value invoice line items quantity value as soon as you click this it will and circulate the add row into a table within a loop because there might because we are processing a table and we are inserting the row one by one so it it will go through a loop with all row that is created in this table from this ai builder and it will try to add rows in the table right like that i will add descriptions also so description value like that i want to add unit price and i want to add just search unit price and list price just go down here this is a line price right so just short line line item line item you need price right sorry we have to just check which data we have created uh you price unit price is done line price so line item price let me go here and check what i have taken for the line item price uh yes this one so inverse line item line item price so i have done done this now i am okay with this power automate and let's check it if it is working fine or not i have just taken manual trigger flow and i will test it so let us test it click on manual and click text now whenever you test it will ask you to import a file because it requires a pdf file to process right so this is my pdf i can run flow and as soon as i run flow click on done it will try to run the power automate and try to insert data here let us wait for some second yes the data is returned right but you can see in the line price item it is stored something or data right it is something it is not that line price item it is something else so i have mapped wrong field so i have to edit this and i have to change properly so i have to check what data items i have taken here let me click on quick test you have to understand what data what is the field name of that you know what is the column name of that table you have to properly map otherwise it will not work so i have to check here so this is the table so this is line item price right so i'll copy this and close it go here i'll just delete this because the last column was not proper i'll delete this two row now i can go here and this time i will search line item price so line item display name line item unit price and what is the column i forgot let me go back here so this is the inverse line item okay so i just want to see what let me again quick test because i want to see uh what is the last column name so once it is done you have so you have to properly remember line item price okay so i'll copy this and go here and search here line item price so this is called as inverse line item line item price value so choose this now save it and test it now this time it will pick all the uh table table that contains in the so let me choose import and click on run flow now this time i am expecting all the data will be picked properly without any mesh up right now the first quantity and second quantity description unit price so it is already done with me right now it is updated the success the power automate also successfully run now i have shown you how i created this power build powerapp ai builder and also i have shown you how to publish and use it in power automate now i will show you how i can call this power auto or ai builder in canvas apps so let let me go and create a new uh new app let me close this and go to powerapps click on new app this time i will use a canvas app let me use a fetch data tablet click on create [Music] so like that in power automatic edge we have used ai builder in canvas app also we can call that as a ai builder component right so click on skip if you go up in the insert you will see there is option ai builder right this is my canvas app now i can use form processor as a option of ai builder as soon as you click on the form processor it will insert the font processor from form processor ai builder control and it will ask you to associate a ai builder that you have created so this i have created this extract data from pdf build model so click on this now this will assign to you this is this will assign to the ai builder right now i can choose i can upload analyze this by clicking and choosing a pdf it will try to analyze and page all the detected tags that i have already given to the ai builder right now let us wait so it is see it is already identified everything now although it is identified where to extract that value you want to display somewhere right so you can use some labels to display let me drag and drop some labels this is a text level and this is another text level let's say here i want to display invoice date and i will take another set of level to display total price so invoice total and here in this level let me rename this or it is not required so you just click this and go to text property and write down form processor one this is the id of this form processing control dot fails and dot invoice date when you click on dot all the items the all the fields that we have defined in the ai builder that will display here so like that you can use the same thing for second level but this time i will use invoice total price right so it will automatically display here right and if you want to display also the data item you can choose any data item here right if you want to display a data table control you can display here and this time you want to associate the table that is returning from here right so you have to go to this item property of this data table and select form processor dot tables and dot the table name invoice line items so if you choose invoice line items you have to click which data you want to display you want to display the items click on add fields i want to display quantity description unit price line item click add now it will display all the items for you right now i can run it right now i can choose another one let's say let me go back and choose another this one i want to choose now it will try to refresh and the data will refresh here so it is trying to calculate or trying to analyze the pda form to extract the data items and it will display respective data items labels right it will display inverse date total price and the line items here right you can see here so three uh three line items are there and also here the inverse date is october 26 and price is there right okay so thank you guys i hope it helped you how to retrieve tabular data and display in canvas apps and use it in power automatic

Channel: Softchief Learn

Views: 8,832

Keywords: Dynamics, CRM, Dynamics 365, Power Apps, d365, power platform, dynamics 365 tutorials, dynamics crm training, Extract tabular Data from PDF using AI Builder Form Processing in Power apps, ai builder, invoice processing ai builder, how to use ai builder, ai builder power platrform, power platform ai builder, call ai builder from canvas app, call ai builder from power autoamt, call ai builder from flow, extract data from pdf power apps

Id: KJyWaqFjteI

Length: 29min 10sec (1750 seconds)

Published: Tue Jul 27 2021