Structured Data Extraction from ChatGPT with LangChain

[Music] I'm getting sick of all large language models ouch sorry what is wrong with you it's not wrong with me it's Richard GPD and all these models they are not stable at all in their output for example I'm trying hard to get a structured output out of these models like Json format but each time it's keep adding some random stuff to the answer like extra comma dot quotation marks I'm done man there's no way to get a stable output out of these these models but you don't need to manually all time keep changing your prompt there are some solutions developed that will help you to get a structured output from these large language models and you don't need to manually parse the output of these models or I I need to know more about this and did I really hit your leg I added these pillows to pretend in the video I am throwing the laptop away and hit the pillow not your leg sorry let's go hello my friends and welcome to this video well you might have already dealt with the challenge of getting unstructured output from large language models like chat GPT so getting some text as an output is sometimes not good enough for our llm application for example you want to get a structured output from these models like Json format that will help you to grab that output of your large language model to call an API or call your other tools so you need that structured format as an output of these large angles models and you might have tried to do so or enforce that structured output by telling that in your prompt which is sometimes not really that stable so in this video we're going to check that out that what are better Solutions developed some of them by opening ions some of them wrapped by Lang chain to implement that structured output from large language models then you can take that output to again call your apis or call other tools in your large language model based applications then let's check it out before we start make sure you subscribe and hit the Bell icon so you will get notified for the next video alright welcome back everyone and let's start talking challenge that we're going to propose a solution for this in this video which is how we can get a structured output from large language models like chat activity instead of just always getting texts which is unstructured by a native so there are multiple ways actually to build that challenge and we need to actually find a solution for that challenge because let's say you want to grab an outcome from Chachi BT to call an API so if you want to call your internal API you should have the input of that API call something like a Json and a structure having a schema but if you get the output of the large angles model as a text as a sentence you cannot use that to call an API or let's say the output of the large language model you want to save it in a database so you need to have that output is structured like Json like as you can see here this is actually pasted by Lang chain in order to have that data insert inserted to the to the database or let's say you want to extract a specific part of the output to do some specific queries for example to keyword search Point by the end of the day you have to have your answer in an instruction manual like Json so now how we can enforce the llm based Solutions in a way that we always get the output of our let's say charge GPT model answer in a structure that we want or in a schema that we want well there are multiple ways the first way it's called functions some of the large language models can call functions to extract those specific entities that we are getting from the response of large language model why some llms well mainly a natively open AI models which is a type of large language model support calling functions if I click on that it actually navigate me to open Aid recently not recently that's a I would call it a recent capability added to open AI models that you can now do function calling what does that mean I'm gonna come to that surely and we'll see in the code how can I utilize it to enforce the outcome of Los Angeles modernist schema is specified in like Json format the second baby do so is output parsing this is also wrapped by Lang chain that means we let large language model generate the output but through this parsing and using element by itself we can enforce the output in a schema that we are specified to call it an a structured answer by large language model foreign between these two is that first functions just some of the large language models support let's say if you go with some go hugging face and grab some open source large language models they don't have maybe that function natively developed by similar to what open a has done and that's one thing the second thing and and they're by the way more General but they do understand the context so if for example you want to ask something in your structured answer that you haven't provided any specific info it understand that and it will add it too I'll go through the code that will make much more sense it might be a little bit confusing for you and the other difference compared to parsing is that parsing will exactly extract what you specify nothing more nothing less conversely functions can also figure out more stuff Beyond just what you have specified if you're going to come to that again shortly so I have utilized some samples developed by Lang chains and they executed the codes before you call the video to show you both and see how they work in action so let me go to my vs code all right the only thing that you need is just making sure that you have an environment that has python installed you can run it everywhere on cloud doesn't matter really I'm just wanted to test the code so I'm running it on my local machine and here's my vs code environment first of all make sure that you have installed nine chain and open AI of course these are the main two things we need for executing this and definitely you need to have your opening IQ also added to the operating system environment that you want to execute these codes so here the first option that we talked about was option one which is we're gonna call open your functions which is also wrapped by Lang chain so what we're going to do here we are going to actually specify that hey this is the schema that I'm expecting the answer of large language model be like this so if you ship that question which is for input here's the input that we have as an example I am using chargpt which is GPT 3.5 you can use gpt4 if you have access to to get the answer or the information out of this context through this schema I want to get the name of people here the the height the hair color and I'm saying that I require name and height and I execute that and there you go you see that it captured two people with different height different hair color and I got them all added here properly so I can be even more specific and by the way I'm using Create extraction chain that on back can use open your functions I cannot even get multiple entities out of the same context again using the first option which is open your function so what what do I mean by multiple entities for example instead of just saying I grab give me the name I can add prefix I'm saying that given the name of a dog and give me the name of a person and because I'm same person and dog and I told you that of open AI functions can understand the semantic so they will do what I'm talking about and then shipping the same sort of information and there you go now I got this all information back based on the schema I specified so you do not need to write down a prompt to tell that hey GPD model before you answer make sure you answer like this and sometimes you say that oh there is no comma here or there's extra comma it happened to me a lot but now it is open AI functions it's sort of more structured and more reliable and you don't get in stock in trial and test process of changing your prompt or keep adding guideline to The Prompt which is not always data stable or nothing guaranteed gonna be stable um not only that in case for example let's say for entities that you want to let's say here I I only want to return person attributes or only dog attributes for single queries then we can use that required option we talked about that so even in case there are some stuff that you need to grab information out of the context but it's not a specified here you can grab them to how in this code example you're saying that I want to have the person name height hair color so on and so forth but any information that you can get regarding dog that we call the extra info I want to grab it as a type of string here in this schema there you go it captured something based on the specific information we provided like it likes to play with other dogs this is not a hair color this is not name this is extra information that was available in the context right so before I go to the second option that we discussed about and to wrap up what's happening at the backend sign when we use these open year functions by launching so I was actually checking Lang chain documentation and they said that this is what's happening when you use that create extraction chain which we just used in the video and I figured out it is actually using sort of a prompt it says that extract and say relevant entities mentioned the following passage together with the properties and now what are those coming from these are those schema that you are defining so my schema sort of goes all the way to that as you can see and it's shipping back to the backing code and then grab the information for me so that information extraction which is specified here is a function on the backend side and if I click here it will navigate me to the line chain implementation and there you go that's the function that extract information based on entities that you provide so now going back to the code let me open up my vs code there you go so the second way that we are going to actually Implement that sort of structured algorithm of large language model we call it parsing now what's the difference here again as we discussed it's not using open-air function that means it is going to parse and get exactly what you ask in your schema that's it and what langchain does and use it is using actually open source python Library called pedantic I hope I'm pronouncing the name correctly which seems to be paidonic is a data validation and City management Library so it will allow us to create a data class with all those entities and attributes that we want to validate them when we initiate an object so for example here I'm defining it pedantic data class these are the attributes that I'm going to grab and then instead of calling create extraction chain I'm calling create extract chain pedantic now building option two and then we specify that this is the properties or or attributes of the data class I Define that I grab it from my context that I specify here so I run is very similar outcome that we got for using open AI properties I apologize I need to correct myself I mentioned that this is option two actually it is not if you're still in option one that means we are still using open UI functions but we are adding this data validation layer on the top of that using pedantic but because by pedantic is good enough to be used by itself without open AI functions for having a structured output out of large language model answers then we can use pedantic by itself this is now option two so what we need to do again with data class that we Define in pedantic we can have all these attributes as specified but as you can see what I'm doing here I'm sort of creating a specific prompt to say to large language model that hey I need to answer the question of mine with the given a schema that I'm specifying that I have developed at using the class of pedantic and then I pass it as the prompt template to my large language model that means not only I can specify the schema with this matter but also with by the article I can't then check the validation of this schema making sure it is working properly without me manually parsing and figuring what's going on and there you go the answer is back for me very similar to what we have on the top so if you have or if you're using open your large language models then open your functions is something natively developed and I think that introduced by open AI to several months ago it's not really that old it considered sort of recent and you have sort of enough flexibility and it have the capability of extracting more information that more entities that you haven't necessarily specified because it understands the semantic but if you want to use with open source large language models that they do not necessarily have a function capabilities like open AI models then something like printed can be helpful to implement and validate it and it will make the job easier for you if you go through line chain implementation because it is well integrated with other large language model capabilities and here was a quick example to just showcase you the art of possibility if you have been in the pain of not having always a structured a schema or a specified framework from the answers of your launch language models and manually parsing or developing cleaning functions for yourself maybe that's a good movement to pause and think how you can leverage this to save your time and have it more reliable instead of manually parsing the outputs and I hope you enjoyed the video that's all how big you would have dreamed if you knew you couldn't fail [Music] dream B my friends believing yourself and take action next video take care [Music]

Channel: MG

Views: 8,930

Keywords: SDK, OpenAI, embeddings, freshnessfunction, langchain, openai chatbot, chat gpt explained technical, Open AI in Azure, chat gpt, chatgpt advanced guide, retrieval augmented generation, DataExtraction, Langchain, Pydantic, DataProcessing, LanguageModel, StructuredData, DataScience, ArtificialIntelligence, DataAnalysis, DataValidation, Coding, SoftwareDevelopment, TechTutorial, LearningPython, DataManagement, AI, ML, NLP, Open AI functions, Structured output from ChatGPT, Langchain parser for Open AI models

Id: q1lYg8JISpQ

Length: 15min 37sec (937 seconds)

Published: Tue Aug 29 2023