Invoice Extraction Bot - Langchain || LLAMA 2 || OpenAI

invoice extraction bot will be implementing this particular application using line chain and an llm which can be open AI or laminate right here I'll just walk you through the working of this particular application it is asking for some files to be uploaded we can browse them by clicking on this particular button right here you can select any of this let me select all of this and click on that once this is done we can click on the button right here which says extract data so this is running right now once this is done we should be able to see the results which is a table containing all the extracted data like this the invoice number description quantity date unit price and all these things along with the phone number and address and you can also download the CSV file yeah this is how the file will look like with all the values being extracted let's go ahead and try to implement this let's try to go ahead and implement the application that we have just now saw the first step for that would be to implement the files let's keep it as app dot py here we will implement the front end of the application and for the backend let's use another file called utils.py and as we know we will be communicating with the llms and friend Services move forward we'll create a DOT environment file and Dot environment example file so that we can store all those particular credentials as I already have them let me paste it right here this is there let me go back to this as you can see we have the values available in this particular file the schema of this will be available in dot env.example file we'll be using two Services here one is open Ai and the other one replicate dot AI this I'll soon let you know as you are already aware of this so I am not going into this particular thing this I'll just explain it to you as we go towards the implementation let's concentrate on the UI of the application this is the code that we will need for the implementation of the UI I'll walk you through each one of this please make sure that you have selected the appropriate environment for me this is 3.10.8 64-bit I am saving this let me go to the terminal and run the app streamlit run app Dot py then run it make sure that you have all the dependencies that you need so this is how the UI will look like this is the header and this is the part where you'll be uploading the documents and as soon as we click on this you should have a response the original response but here we have a hard-coded value for now this is how we were able to build the UI of it as this indicates the title of the page and this the title of the body and then a sub header we are able to get all these things from streamed it Library we have a component by using which we can take the inputs from the user which are PDF files SD dot file uploader and then the type of this should be PDF and it is giving the access to select multiple files once this is done we'll click on the extract data button if this is clicked that is what we have right here we are using sd.spinner which actually gives a waiting progress kind of option on the screen till this particular part of code is being executed and once this is executed we will have a success case as you can see this one and here we are invoking the main function because we want this to be executed whenever this itself is being executed as a project rather than this particular app.py being used as a module this is done this will load all the keys available right here into the environment that's it so we'll go ahead and try to implement the backend logic of this particular application so we have already seen how to run the lamba 2 model locally on your CPU will also have a look at replicate service which is a kind of Hosting which will help you host your ml models and try to use this for your prediction and there is some pricing associated with it these are few of these options you can host your own model and then use it as an API because Lama model needs a lot of infrastructure for it to run most of the use cases that we see over YouTube or anywhere else are on Google collab so I try to replicate the same thing in replicate.com by using this service few of this initial predictions that you will actually use up will be free if I just go to this right they have already told that this will cost some money but few of your initial calls will be free okay that free version should be more than enough for you to at least experiment with this platform you can also use different AWS Sage maker and all this particular platforms but for this we will limit our experimentation to replicate platform I'll be going ahead and using the models that are already available right here which is llama 270 billion chart I know this will consume a lot of infrastructure so we'll just experiment with this model in our project if you are interested it please go ahead read the documentation and try to upload your own model and then experiment with it let's go ahead with the implementation let's move on to the back end implementation of the code which is utils.py here I'll just try to paste all the code and the functions that I have with me and I'll try to give a good walkthrough of this all functions moving to the top you have to import Lang chain by PDF pandas Library replicate and all these things for this whole code to work please go ahead and do it if you don't have this installed on your system once this is done then we can just import them and their respective modules first thing that we will try to do when we have an application like this is I just close this extra tab we'll try to upload the files and then we'll try to extract the data out of it for that particular task we need data extractor this is what we are doing with this function extract information from PDF files this is the function right here I've given the name of get PDF text it just takes up a file which can be something that we have uploaded the invoice itself let me copy the invoices into the current project that we have once we click on this this is how the invoice will look like it has the build to address company's name invoice number date description of the item the quantity unit price amount and total these are some of the fields that we will try to extract from this PDF going back to this and then this this is what will be passed on to this from here we are just initializing a text variable with empty string and then by using this library that we have just now imported we will use this module which is taking up the file of that particular PDF and then from each page available in that particular PDF we'll try to read it and append that particular text to text variable that we have and then it just returns it back this is what this function is doing right now this has a very simple task assigned to it then comes extracted data what is this exactly once we have the data with us we need an llm large language model to give this particular output which is the invoice data right for that we will need some intelligence which can be open AI Lama 2 hugging face models or anything so this function will help us with exactly that it is taking up the data that is coming from the PDF page which is the output of this particular function and then we have a template written which says extract all the following values what are the values invoice number description quantity date all these things are coming from the PDF that we have just now seen this things exactly we are advising the large language model to do that particular thing once we have told all these things we are also giving the context from where it has to pick this values which would be this data and then the expected output we are defining please remove any dollar symbols because this sometimes causes issues and this should be the format of this this is what we are giving an example of how exactly the output should look like once this is done we have the prompt template module which we can use which is coming from the land chain it takes up the input variables which is the data this thing resembles this placeholder right here in the template variable and here the template represents this whole template into which this variable is going into once we have the prompt template available we can either use open AI as the llm for us which can be an intelligence which will be used to extract the data out of the invoices and also use the llm model and then use this llm model to extract the invoices by supplying the data coming from the PDF file to this particular template please refer to the earlier videos that I have uploaded which will exactly explain you how this prompt template works this is one part of it this is something that you might have already implemented or you are aware of but then we have an another code which is available here which helps us use the Llama 2 model which has been released recently this thing I have already shown you how you can use it locally but right now I'm going ahead and trying to use a technique where we will be hosting our model which is Lama 2 model on a ml hosting service and then through the API we'll be accessing the prediction coming out of that particular web service let me show you how exactly this works as it says the below code will be used when we want to use lamba 2 model we will use replicate for hosting our model as you can see replicate.com gives us the ability to host our ml models which can later on be used in production ready applications here I have chosen this for this particular project there are lot of different services available which you can opt for I have tried to experiment a little with replicate services so there are a lot of models that are already available and one among this is 70 billion chart this is what we'll be using as you can see 70 billion parameter language from this is used for chart completions you can also go ahead and try to learn through documentation where you'll be able to push your own model to the replicate and try running your application on top of it there are different steps right here which you can go through and upload your own model to the portal back to this there is a demo right here which says this is the input prompt that we are giving and this is a system prompt which is just an instruction to the llm and then there are few of this parameters Max new token maximum number of tokens to generate a word is generally two to three tokens and then the temperature top K values top P values and repetition penalty on all these things you can go through the description of this to understand it better and once we are passing all these things to this repository we'll be able to get the output like this by clicking the submit button right here and there are few of these instructions available right here which says this model cost approximately this to run on replicate but this varies on the input that you are actually passing on and this will be using Nvidia 880 GB GPU Hardware which costs around this per second and as you can see a typical prediction takes this much time so please consider different Alternatives available in the market like AWS sagemaker and all these things you can also look at the pricings of this if your model uses CPU then the charges for that would be this for us it is charging this particular infrastructure for us it can change based on your model coming back to this we'll be going ahead and using this model as our intelligence which would be the llm to use this we have few instructions written right here under API section this code can be used for that we will be using this model for now do we don't need 70 billion rather we can go ahead and use llama27 billion chart for us but that is not there in this particular portal for now so I'll just use up this thing I can't find it here so I'm just going ahead and just for the experimentation I'll be using this particular model I know it will cost a lot for me rather compared to 7 billion chart model which is this model that we are talking about we are using this model for now but this would be more than enough for our experimentation though going back to the code here as you can see I'll just comment this out and then enable this code right here I'm using control forward slash for this this is a comment so let me comment it yeah so here for this to work we have to install the replicate library and import it into the file that we are currently working on CE this is done on this particular class we are trying to run a function called run which is taking a few of these parameters the model address which is coming from this particular page in API tab you will have something like this this is what you have to copy and paste it into your code once this is done it is expecting another parameter called input into which you have to pass your prompt which is this whole thing which includes the PDF data also and there are few other parameters that you can fine tune your model with temperature top P values max length repetition penalty and all these things once you have all of them you have to form your final response because this generates a generator object type to get the whole information out of it we'll be using this code right here for each item in this particular thing we are concatenating it to form a complete string this is what we are returning back awesome we have this understanding clear I guess so once we have this moving back to this we have the final thing which is I take over files in that user uploaded PDF files one by one there is a main function that we will try to call from right here because once we have the PDF files from the user next we'll try to pass them into this function which is create Docs here we'll just take up all these things the files list and then creating a data frame with necessary column names as you have seen in the application that we have created we have a tabular structure which is displaying the extracted information of the invoices for that we'll definitely need a data frame that's where we are trying to create a data frame with specified column names and their data types this is one thing and then because we have the PDF list right here which is coming from the user we are using it for each Loop to get each file and do or perform some actions that we actually need first thing just to confirm if we are able to get that particular file I'm printing it right here once we have this then we are using get PDF text function looks like we have seen it somewhere this is the one we are using this function to get all the information being extracted from that particular PDF file once we have this we are assigning it to a variable called raw data you can also have a look at it while running The Code by using print raw data and then you can also have a print statement stating this is done then comes extracted data what is the use of having the raw data with us the main agenda here is to extract the information from the PDF that is what we are trying to do by using extracted data function as we have already seen this takes up the data from the PDF the raw data and uses an intelligence which could be lamba to model are open AI model and then gives us the output this output is something that we can use up right yeah so that's where we have the llm extracted data being stored into this variable but the problem here is the data coming from the llm will have some extra data in it we'll have a look at it I'll just uncomment this for you to understand it better when we run this particular function because we are expecting some junk data coming along with whatever is expected which is this format we have to perform some text processing to clear up all that unnecessary data so this is what I'm writing here which is using a regular expression here we are defining the pattern for the regular expression saying anything that comes between this flower bracket should be considered please search it with this pattern into this particular llm extracted data this gives us the match if there is a match we are going into this if there is no match we are saying no match found this is what we are doing right here capturing just the required data and excluding any unwanted test there should be text from the llm response once we have this obviously we know that we want to display it on the screen for that we have to append all this data that is coming up from the llm to the data frame that we have here to this data frame that we have created for each Loop we are appending some data which is the data dictionary which is nothing but this after applying the regular expressions and all that and then I have kept a print just to understand if each one is being executed or not once this is done we are returning the whole DF the data frame that we have right here to the app.py so that is what we have to implement here to do so I am just pasting the code that I already have with me here as you can see we are trying to call the create docs which is coming from this this one but it is giving an error because we have to import that module from utils import everything let's save this and this is gone we are calling this function which is doing all this tasks of extracting the data then using llm to extract the entities and then formatting them then adding it to the data frame this data frame is being retrieved and being stored into this particular file then we are using sd.write df.hack to display it on the screen this is one part of it and then we want the user to have an option of downloading the extracted data this is where we are using DF dot to CSV function which will actually convert this to CSP format once we have this we are using SD dot download button component that comes from the streamlit to give a download option to the user the file name of this would be invoice extracted data and then we are saying download data as CSV the option of that particular button the label of that particular button here it should be C here yeah so this option would help us download the file as CSV this is all done another thing that I want to highlight here is as I was using this particular service for my application it was billing me and the bill I can just show you right here is still loading I have kept a limit of five dollars right here and it has costed me around 3.4 dollars for this particular application or the experimentation this is quite huge because I'm using the top model of this llama 2 which is 70 billion parameter model if I might be using this it would have costed me less I've actually added my credit card here for the experimentation but most of my initial calls were free so you can just utilize that to at least understand or get started with the Llama model coming back to this as we know that we are already ready with our implementation everything is there we have kept a print statement here and this is the raw data let me print that too so that you understand what exactly is happening and yeah this should be more than fine let's try to go ahead and run this particular application that we have just now implemented stream late run app Dot pyth yeah the application is ready let's go ahead and try to browse some of these files these are the files I'm selecting all of them and clicking on open this all are loaded right here let me click on extract data will take up some time let's go back to the code as you can see it has extracted the data from the first file this is the raw data right here the llm is currently working on top of it to get the data extracted this is the data that is being extracted from that and then this is being pushed to the data frame this is the second file and this is the raw data the llm is working on top of it looks like that is also done right here this is something that you can see right here is llm extracted data after applying some text processing this is also done let's go back to the application once we have the results ready so there are three files that we have uploaded and we have got the results for them and we are also having an option to download the result if I just click on this it should be able to do the same thing for me here is the data So based on your requirement you can just uncomment this open AI and have that key right here uh if you are interested to use the Llama 2 using replicate service you can just use this it's completely based on you I'm just commenting out this for now and enabling this so that for the initial runs you can start using open AI as your lldm just to save some cost this is completed hope you have learned a good amount of knowledge through this particular experimentation stay tuned I'll be posting more such projects based on this llms that are coming up in the market

Channel: Sharath Raju

Views: 6,869

Keywords: Langchain, Langchain projects, LLAMA 2, LLM, OpenAI, ChatGPT, Generative AI, Build AI apps

Id: hFqIa-mX4GM

Length: 25min 48sec (1548 seconds)

Published: Wed Aug 02 2023