AI Builder Invoice Form Processing with Approval using Power Apps and Power Automate

hey everyone this is druveen and in this session we will talk about how we can use ai builder invoice form processing with approval process in power apps and power automate so now let's get started [Music] before we get started let me give you a little bit introduction about myself myself travinsha and i'm working as power platform specialist and office 365 developer in one of the reputed company at gujarat and about india i am very active blogger and youtuber having 120 plus articles and 2 320 plus tech videos i am rewarded three times c sharp conor mvp for my constant contribution towards the technology you can connect me on all of my social media handles my email linkedin twitter and facebook so now let's get started and let me talk about the agenda for today's session first of all i am going to give you some overview about form processing ai model once we complete with the overview part we will take some form example and we will train our model once our model is trained we will publish our model once our model is being trained we will use our model inside the powerapps to scan the invoices and once the invoices are scanned we will insert those data to the sharepoint list and after that we will trigger one invoice approval flow for that so whenever user upload any invoice it will automatically create invoice approval process for them and once the manager or approver approves the invoice at that time that request is considered as to be approved we are going to cover all of this thing in detail throughout the session so stay tuned with me throughout end of the session now let me get started with our first topic that is what is form processing ai model right form processing ai model allows us to read and save the information from standard documents such as invoices or text documents when we automate this process we can save valuable time by reviewing extracting organizing and saving the data automatically by using power automate and powerapps so in short if i summarize this form processing it is something which scans our document and extract the useful information from our form automatically so that doesn't require any human interaction for scanning or reading the form now let me talk about some real like example of form processing system let's consider an example of invoice approval system with the help of form processing we can easily scan our invoices and store the data in a structured manner we can pre-populate the fields based on our scan data once we have our data with us we can also integrate approval process for our invoices so this type of real life example we are going to cover during this session so will we are going to upload some invoices we are going to train our model after that we will build our powerapps which will scan the documents after scanning the document we will pre-populate the data into sharepoint list and once the data is pre-populated we are storing that data to the sharepoint list and on save we are going to trigger one approval flow stay tuned with this end-to-end scenario now let me talk about some other real-life example where we can use form processing let's say for an example if we want to scan rent receipt at that time also this concept is really helpful to us we can also automate hr interview process using this form processing what happen here is initially if any candidate came they are filling one hand written form so what we can do is we can scan that handwritten form to our system and that will pre-populate the data in our form right so those type of requirements we can achieve with that now the third example is inquiry form scanning sometime what happen here is if someone came for any inquiry in any shop they are basically fill up one form so those type of form we can also scan with the form processing ai model so these are some use cases where we can use form processing concept effectively now let me show you one of the example that we are going to use in this session so consider this sample example where i have one invoice where i have invoice number its date due date and balance due information his bill to address and his ship to address and these are the different line items that are being used for the purchase and at last there is a subtotal tax shipping and total information now what i need to do is i need to scan this particular form with my form processing ai model and based on the scanned model based on the train model it will detect the field automatically from there so now let's get started now before we proceed further with the development process let me show you the demo which we are going to perform at the end of the session so we are going to build this type of power apps at the end of the session where we have one scanner over here and we have one form over here and we have save button over here so let's say for an example whenever i am uploading any of the document any of the invoices it is going to perform scan on those invoices and once the model is being scanned it will pre-populate all the information into these fields so you can see that it has detected all the information from the form and it is public populating the information over here and once we click on save it will store this information into our sharepoint list and once our information is being saved it will automatically trigger one approval email for the approver and here it is we have received one email the email is looking like this it is asking for reviewing the invoice and let's say whenever i am clicking on my invoice number it will open that invoice item over here like this now as the approver i am checking everything from here and let's say i want to approve this item from here so what i need to do here is i just need to click on approve and i need to provide any reason if i want to do otherwise we can simply submit this form and once the form is approved which means that that has been approved by the approver and the invoice will be released for the payment once the item has been approved it will trigger an email that your request has been approved and so this type of approval process we can perform with the help of powerapps and power automate right now let me show you the step-by-step procedure for developing this particular app over here before we get started with the development process let me open my presentation dac over here and let me explain you few things that we need to consider before we proceeding for the form processing ai model so first thing is that at this moment it only supports jpg png or pdf format if pdf are password protected we must need to remove the lock before submitting them file size the combined file size of the document used for training must not be exiting 50 mb and the pdf document should not have more than 500 pages for images dimension must be in between 530 cross 100 and 4200 cross 4200 pixel for pdf files dimension must be at most 17 cross 17 inches corresponding to legal or a3 paper size or smaller if scanned from the paper document scan should be high quality image and it should be latent alphabet means at this moment it only supports the english as a language now let's talk about some of the limitations of this ai builder form processing model ai builder doesn't currently support the following type of form processing input data complex table so if you have some nested tables merge headers or cell and so on it will not support checkboxes and radio buttons are not supported signatures are not detected fillable pdfs are not supported and at this moment only supports english language so if you have invoices other than english language it will not support now it's a demo time now in order to start with form processing ai model first of all we need to train our model for that what we need to do is let me explain you that step in detail first of all open make.powerapps.com now here you can see ai builder section now under that you have the build option over here just click on that option now here you have different type of ai model available now click on form processing ai model before we proceeding further let me explain you one more thing if you are not able to see this ai builder which means that it doesn't support your region or maybe you need to activate your license check if your region is supported ai builder i am going to provide one link in the description box just check on that so you will get an idea like if your region is supported with ai builder or not right now from here let's click on form processing now here it is showing some get started procedure example and best practices to us so now let's check those things so as as you can see here it is showing step by step procedure the first step is to upload the documents second one is select the important areas and train your ai model and the third step is publish and use this new ai model right so these are the steps that we are going to perform some of the example it is showing it is showing some of the invoices like this you can see that different type of invoices that are provided over here like let's say for an example this is the invoice example this is the mortgage statement and this is the school of finance example right so this type of examples we can use with our system then there are some best practices provided the first base practices is use the document with the same layout means the whatever the document that we are going to use for the scanning it should be having same layout but the different data don't use the document with the different layout in the same model that is correct use the document with primarily text don't use example with lots of images or checkboxes correct and each document should be a separate file don't use file containing multiple documents correct so these are some consideration point that we need to keep in our mind before we proceeding for the form processing right now let me click on create button and let's create the form processing model so first of all we need to provide the name of our model so i am providing my form processing ai model name is form processing new now we need to upload at least five document with the same layout over here right just click on create so first of all it will ask us to add the document examples over here so let's click on add documents and upload from my local storage so i have some documents with the same layout so i'm uploading those documents over here click on upload 5 documents over here our document has been uploaded successfully now click on close so you can see that our document has been uploaded here now we need to analyze our document so select any of the document and click on analyze so it will start to analyze your documents and this may take a few minutes now our form processing model has detected some fields over here and and you can see that this dotted rectangular border around each of the field which has been detected now we need to add that field in our invoice so let's click on that this is the invoice number just select that so it has added that field over here this is the date this is the due date this is the balanced view and this entire thing is considered as table correct now this is subtotal this is tax this is shipping and this is total correct now this is this has detected some wrong information you can see that this is the name and this is this fill should be detected as build to date right so let's consider this one like this and now let me just rename this field as build to date correct so if something is detected wrong we can correct it using this thing right so here i also have one address let's just drag and add that field as ship to address and just click on select right now i confirmed all the fields from here now let me just click on confirm fields so we have trained our form now let's move over to our third form that has been detected the field like this here also we just need to train ship two and build two right now same way we will go with other two documents over here so ship to address is this one and bill to address is this one and one last document from here my bill to address is this one and my ship to address is this one right now once we are done we can click on done and we can click on next so it is showing me it so it is showing me all the detected fields from my invoices and now it's time to train our model so just click on train model from here it is going to train your model like this now click on go to models so it will redirecting you to this model section and here you can see that it is going to train our model like this so let it to be performed properly and here it is my model has been trained successfully now click on this model and here we have two different option publish and quick test so let's click on quick test now here we need to upload any document so let's say here we have trained these five different images let's upload any of the images apart from this five images i am uploading this image and here it is it is detecting all the fields properly over here just click on start over and click on close now from here what we need to do here is we need to publish our model so publishing is the most important step if you do not publish this model you are not going to use that particular thing into our powerapps or power automate so it is going to be publish and here it is your model has been published successfully now we can use this particular thing into our power apps right so you can directly use this use model option from here or you can create the canvas from scratch but we are going to create the canvas app from scratch so let me show you that thing in detail go to the app section and from here click on new app and click on canvas app select the blank canvas app for tablet layout so we are going to build this type of application for our app so this has been created one blank canvas for your application now let's create one scrollable screen over here and let's just give a name as invoice approval now let me just add elements into my data card so first of all go to the insert menu and first of all let's add our form processor model so we will add the form processing object from here go to the ai builder section and it will add the section like this right now it has added this section now what we need to do here is we need to connect the ai model with our form processor control so for that just click on a select model and now from here let's select form processing dave model so it will once the model is connected this will look like this right now let's try to scan any of the document over here so let me scan this document over here so we'll get an idea like how it will look like so you can see that all the fills detected successfully over here right now what we need to do here is we need to pre-populate one form over here right so before we proceed further let me show you my sharepoint list structure i have two different lists one is invoice list and one is invoice product list so in invoice list i have different field of invoices like invoice number date due date bill to date shipping date shipping address subtotal tax shipping total and approval status and with that we are also going to capture approval date and comments by the approval and in invoice product we are storing the line item so what happen here is let's say for an example this is my invoice so what it will do is it will show the parent items to my parent list that is invoices and it will store all of this line item with respect to parent id of invoice into this invoice product list so here i taken two different lists to store the data right now let me do one thing let me just establish the connection for my sharepoint list over here and after that we are going to create one form so go to the connectors and select sharepoint now from here just add connection and let's provide the url so my url is healthcare and from there i need to select invoice and invoice product from here and just click on connect so my sharepoint connection has been established successfully over here now what i need to do here is i need to add the form over here in order to do that click on insert go to the forms and click on edit form so that will add one form over here now let me just connect my data source over here so here i am going to use invoices over here so let me just use invoices from here and it will show all the appropriate fields from here and once the filth has been added we need to make some correction in our form so you can see that that has been added successfully now here i don't need this attachment control also i want to arrange some field like this approval status approval date and comments by approver or something which is not part of the form so only these eight fields should be here right now let me just rename this fields over here right so in order to rename this field once we rename this field this will look like this so title will be considered as invoice number then we have date due date bill to date shipping address subtotal tax shipping and total right so this will look like this at the end so basically you just need to rename the field and this will look like this right now one more thing we need to do here now let's say we have added this section now we need to show the line item right in order to show line item what we need to do here is we need to make one collection on change event of this particular invoice sample right in order to do that let's click select on change event and on on change event first of all we are resetting our form we are initializing our new form so it will open automatically open up the form and then we are going to use one clear collect on clear collect and this is mine the name of my collection and this is the name of my form processor object so we are storing form processor one's table value inside the collection and we are going to use one of the update context to show and hide the grade variable right so this is something that we are going to do on change event of the form processor model right now what we need to do here is once we are going to once we are done with that part let's upload any of the invoice over here and let's check the thing like whether it's initialize the form properly or not and whether it is adding the values inside the collection or not right so what we need to do is we are done with this part and now let's check the collection value so on change we have the invoice info so let's check that collection and let's check out the value so you can see that the line items are available here right so now what we need to do here is to show the line item to the add end user we are going to use one data table control over here in order to do that what we need to do here is just go to the insert menu just click on this expand and select the data table now it add that data table over here like this and now we need to provide the data source to our data table that is my collection and once we do that we just need to add the fields from my collection so basically i just want to show item then its quantity then its rate and then its amount and let me just add those fill to my collection so it will add that line items over here right now this is the line item thing now what i want to do here is i want to show the detected pre-populated fields over here in each individual field so how can i do that in order to do that what we need to do here is we just need to select this field we just need to go to the default event and we just need to add we just need to unlock our card and let's add the default value is form processor one so that is the id of my control from processor 1 dot what i want to have here is invoice number right so form processor 1 dot fields dot it will show when you uh when you just click on it will show all the available fields over here right so basically i need this invoice number so i just need to type invoice over here right now same way you can see that that has been pre-populated now let's say i want here shipping address so let's unlock the card and let me go to the default event so here same way form processor one dot fails dot shift to address correct so that is showing the ship to address to us over here right now let's proceed the everything for each individual field over here and at the end of this thing we are able to see all the fields detected over here we just need to go to every field and click on default item over here so let me again scan the document you can see here it detected all the field perfectly over here and the same thing we have applied in all the default event over here so let me just select all the data card here and open its default event so you can see that invoice number is added over here this is the date field this is the due date field this is bill to date this is shipping date shipping address this is subtotal this is tax this is shipping and this is total so we have added default field everywhere right now the important thing is save function right now how to manage that so save function will store the master detail inside the form so we just need to use submit form and inside that just provide the form id right now what what to do for this line items right so once this item is added to the form after that it will also add those line items to the child list right this is my child list right so in order to do that what we need to do here is we just need to open the form success event from here so in order to do that let's do one thing from here let's select our form and now let's open its own success event so on success event we are going to use we are going to perform a loop on our code and on that we are going to add each line item to our list right in order to do that lets open form success event again so this is the form success event so what we are going to do here is we are going to use for all inside the for all we just need to pass the collection name so the collection that we have created that was invoice info and it will scan each line item and then click on patch patch function is something which will insert the value inside the table so patch into which table so i want to add the data into invoice product so just provide the name of the list then we want to add that as a new item so just add this line item that's default and inside that list name so it will perform the looping on this collection so this item this record dot item same rate this record dot rate amount this record dot amount and quantity this record dot quantity and at last we need the last submitted id as a title column or as an invoice id because the internal name is title so here we need to provide the internal name so form1 dot last submit dot id that is something which would be the last submitted item that we just submitted so whatever the item that has been submitted over here that has been taken as a id over here and it will be create the other items based on that id right so this is something that is the code of the for all right and once the form has been submitted we will just notify form has been submitted successfully right same way you can reset your form you just need to apply some reset code reset form reset form and clear the things and don't worry i'm going to provide this all the code to my github account you can directly download it from there and practice more so let me just save this form over here and here it is the form has been stored successfully now let's check the output in invoice table you can see that this line item the first master data has been added approval status is pending by default and let's see the line items so you can see that for that particular invoice the line items has been added successfully and it is showing like this right now what we need to do here is we need to trigger one approval flow whenever the new item is created to the invoices right for that what we need to do that let me explain you that thing in detail in order to do that we need to create one flow one approval flow over there right so in order to do that let me show you the step-by-step procedure for that so first of all we will add an action when a new item is created inside that we will provide the list name and site address and we will add an action start and wait for the approval right so basically for that you just need to search here start end wait for an approval just use this action right once you add the section it will ask for the approval type you just need to select custom response wait for one response now here we need to still add the response option item so my response should be either approve or reject so i just need to name as approve and second button should be reject right now we just need to provide title assign to details and item link right i already did that let me show you so we have added the response option that are approved our reject right in title we have added please review invoice number and then we are providing the title assigned to this item is uh assigned to the account department whose head is draven so it's email address has been so his email address has been added here detail this is created by and created on item link that is linked to item and item link description that is to be name of the title right so this is something that we need to configure first let me just remove this unwanted one from here once we do that we need to initialize one variable that is comments so whatever the comments that we have received we just need to need that field in the string format then when uh what we are going to do is whatever the responses that will receive we are appending those responses to this response field right so the main reason is that it is only supported string format and whenever we are adding this particular field as a outcome from here as a response is it is automatically add one applied to each loop and in order to remove this entire loop we are appending that value as a string over here and that's the main reason we have added that action over here right now for status as well we are just by default before any approval has been initiated we are storing that value as a pending right now what happened here is uh this action is something which will send one trigger email to the approver right so based on the approvals uh result it will decide like whether it is approved or reject right so in order to do that what we need to do here is we just need to select outcome from here if outcome equal to approve then it will send an approval email and it will set the variable as approved and if the outcome is reject it will send the rejection email and send set the status as rejected and once we are going out of this particular condition it will update an item with the status like whatever the approval status that has been set like approved or rejected that has been updated from here as approval status value then approval date is something which is set as utc now and comments by approval is set as that particular variable that we have taken right once the item has been approved or reject based on that it will update the sharepoint list item correspondingly and it will automatically set its approval status approval date and comment by the approver using the variable that we have defined status and command and approval date will be added as a utc time now this tab is optional if my flow has been filled during the above stage it will automatically send an email to me in order to do that what we need to do here is we just need to configure run after and whenever the flow has been filled only at that time this action will be triggered now let's test the entire outcome so let me just scan my form so i'm just scanning this form it will pre-populate the field over here and store the show the line items over here it will show all the pre-populated fields over here and showing all the line items and now click on save so the item has been submitted successfully over here now let's check the flow rank over here so the flow is being running over here and it automatically goes to the waiting stage to get the response from the approver i received one email so let me just approve that and let me just add comment and click on submit so it is submitting my approval now let's check that thing so let me just check what response we are getting from the approval so we are getting the approval as what would be the outcome so user press the approve item so it is showing approve right now it goes from all the step it goes to the condition approve is true right and we are getting update list item and it stay it set the approval status as approved over here so let's check that thing this is the list and here you can see that approved status has been added now let me add one more item over here and let's check the rejection functionality let me add some other forms so we have submitted this invoice and you can see that all the fields has been pre-populated over here now let's hit save so that item has been submitted and now let's wait for starting an approval process over here and here it is the approval process is being started over here now let me just reject that thing from the email that i received let me click on reject and click on submit so the item has been rejected so let's see in outcome what we are getting in outcome we are getting reject so it will execute the else part and set the value as rejected and let's check the sharepoint list item so you can see that it has been rejected right so this is how the approval process with invoice scan processing will work with the help of ai builder powerapps and power automate hope you enjoyed this session so this is how form processing ai model work in powerapps and in additionally we can give the power using power automate if you have any question related to this topic you can add your questions in the comment section this is the social media handles detail where you can follow me if you are for the first time to my channel make sure you hit subscribe and press the bell notification to never miss any updates from my channel stay tuned with me on facebook linked in twitter and whatsapp if you wish to receive the daily notification about my new videos from my channel click on this link it will be redirecting you to this form fill the form and you will be receiving daily notification on your whatsapp number stay connected with me have a great day good bye

Channel: Dhruvin Shah

Views: 5,619

Rating: 4.891892 out of 5

Keywords: Power Apps AI Builder, Form Processing AI Builder, AI Builder and Power Apps, AI Builder Form Processing, AI Builder Form Processing for Invoices, Invoice Form Processing using AI Builder, Automated Form Processing with Power Apps and AI Builder, Invoice Approval with AI Builder Power Apps and Power Automate, Processing and Approval of Invoices using Power Apps AI Builder, AI Builder, Invoice Form Processing AI Builder, Power Automate, Power Platform, Microsoft Flow, MS Flow

Id: YefcO4dmJDI

Length: 38min 11sec (2291 seconds)

Published: Thu Sep 10 2020