OpenAI Function Calling Explained: Chat Completions & Assistants API

hi everyone in today's video we're going to look at function calling which is you know one of the feature of open AIG GPT models using which you can connect your large language model to external or let's say internal apis or the tools including your database or you know whatever the aps we have so we're going to see some handson you know some real practical example so what I'm going to do actually so the first thing I'm going to show you how the function calling is working with an existing let's say you know chart completions API so we have this chart completions API how does function calling uh currently working with this chart completion API and then we're going to eventually see how does the function calling is actually working with the assistant so in the assistant you can treat function calling as one of the tools for the assistant right here and you can see so we're going to see both the examples okay and I'm going to share the code also with you so that you can try uh you know yourself and if you're not familiar with you know assistant API you know what we're going to see today you can go and check my you earlier video where we you know have this beginner tutorial where we go through the assistant API understand its flow and you know all the beginner level details you can go and check so let's start with so the first thing definitely we're going to require the open AI library and the you know let's say uh we're going to initiate the open a client okay and to make it real what I did actually so I have created one API which kind of you know simulate the order details things I have a database where I have a lot of orders customers and all of those details so basically I have a let's say function which is nothing but the get order details which takes the order calls our internal API and simply Returns the order information so if I execute this function you can see if I pass the order ID is equal to one that function return me some information about the delivery status where it is currently that order what is it expected delivery and that kind of stuff data might be old because I I just generated the data so look at the concept rather than the actual the you know factual correctness of the data similarly if I pass some different order ID I should get the uh you know order ID for details so our job is to teach GPT to use this particular function whenever customer or user going to ask something related to the order if customer wants to know their order stator this is the function that they needs to call right and I just created one dictionary here which represents the mapping between the name of the function and the actual function uh you know what we have because we're going to require this particular mapping later right now if you want uh just I told you right we want to make sure GPT understand that we have a such a function so we need to create get some specification here for each function So currently I have only one function so here is only one object in this particular list right and this function I'm telling about the name of my function which is the gate order details here is the description so that GPT understand when to call this particular function so I put that you know retrieve the details of an a given order and some kind of detail and then I talk about kind of parameters this particular function going to take I'm going to say it is going to take order ID which is going to be integer and uh you know I can specify if I have multiple parameters I can specify which parameter is required so I don't need to this is pretty easy I don't remember this syntax I simply take one of the example from the open AI documentation and I use simply chart GP and ask it to generate that particular specification for me right but I understand what exactly that particular specification right now let's see how we can pass these functions to our chart completions API so here is a function a typical chart completion API we have seen in many tutorials you know where we take our Opera client and we call the chart completion API specifying the model that we want to call the message array that we want to pass the here is the new thing is the functions so we can pass actually the list of functions here and then the function call is equal to Auto auto means the GPT will decide if there are multiple function which one to call or something like that okay so this is simple utility function that we have seen it at many places right then I have one more utility function and we will see how we're going to use them actually which something nothing but simply execute the function so here I'm going to pass hey here is my function name called get order details here is the order ID it is going to Simply call our function it going to take the arguments and simply unpack those and call we're going to see where we're going to use it so this function uh job is to just execute the function that we're going to give now Let's test our uh you know uh this chart completions API and see whether it is able to identify the functions right let's do these things so uh first I'm going to test for the query called tell me a joke so let's say I'm going to ask chart GPD completion API tell me a joke right and here is the messages array that I'm creating I'm putting role as a user and the query is what I'm just going to ask right and we know that we created one utility function which will take our message array so at the moment message array has only this one object which is like whatever something the user is asking now let's say what response we get from the uh you know chart completion here okay the functions is not defined maybe we didn't run that particular sale okay so let's run that sale and come back here again and let's execute the chart completions API okay so we got something chart completion message and which is we see the content which is nothing but the actual GPT reply it is saying that you know I'm glad you ask how does a computer get drunk okay some kind of a joke it has say with the role of assistant right and you can see function call is equal to None So based on all the details we provided the GPT think that it doesn't need to call actually the function to you know reply for this kind of uh you know stuff right now let's try to ask something else so instead of this we will ask maybe you know I want to know my order status this is what we're going to ask and let's see what we get so we're going to focus on what we get at the content level and what we're going to get at the function call level so let's run again our chart completions API and let's see we got the message we still got the content message saying that sure I can help you with that please provide me the order ID you know to call this function and there is still not any function calling because GPT knows that before calling the function it needs to have all the required parameter which requires the order ID that is the reason it is asking for order ID before it is actually making function call is equal to true what will happen if I give all the information which GPT requires okay now I'm saying like this what is my order status of order ID is for or order ID whatever it will understand that I'm asking about this thing and see what happens now now if you see the content is equal to none now GPD I haven't reped me anything like a text message rather it is telling me that I need to call a function call because now I got all the information and GP is telling me you should call a function name called gate order details and here is the argument that you should pass to your function that is the you know use of the specification what we have given to the GPD this is what happens so when you get response from the GPT we can check whether we need to call a function or not not right let's say in our case in this particular criteria we know that we need to call the function so how can we get that function details so whenever we get the response message we just saw here it has a property call function call and that has two things the name and the arguments so let's retrieve that information here we could see the gate order details is the name of the function and this is the order details right now as I told you we need to get that information right so GPD has suggested us that hey you should C your gate order details and here is the parameter so let's execute the function so this is where our utility function is uh you know getting used we are passing the function name which is the get order details and the arguments and this will call our API or whatever the function we have and this is the response that we got from our utility function that we had earlier seen it here gate order details now since we got the rip reply from our own API right we need to give it back to GP because GPT has asked us to run that function we need to give that information back to GPT so that it can generate response for the user right if you look at our current message array what we have it has only this thing you know first only one message what is my order status of this order ID so first thing we will do we will append in that message array whatever the response we had got for the function call because GP should know that it was about the function call so we will first insert that response related to the function call then what we will insert then we will insert the response that we got from our own function so now GPD knows that it has to be uh you know uh you know we had called let say the function and here is the response that we got from the function so we appended two messages inside this message array and it will look now something like this original message function call and the output of that particular function call now let's GPT generate the response for the user and if I now send back this is what GPT has generated that this is what now user going to see or the customer going to see the order ID with this particular for you know is actually shipped and it is in warehouse and all of this right you could write better prompt you know system role to Define what messages what we want to see here but that's not the purpose the purpose here to show you this is how we're going to uh give GPT or internal function detail so that the chat GPT can communicate with our internal apis so imagine if you're on e-commerce website and you have the chat bot you can deploy this thing function there so that if user ask anything related to the order ID GPT will decide which function to call then we're going to call that function and automatically the user is going to get the response that what is the status of the order so this is what with respect to our current chat completions API okay I hope you understand now let's go back to the assistant API which is the new API and we know that it also supports the function calling as one of the tool because assistant is capable of tools like taking a tools like the code interpreter where it can interprete it can generate the code knowledge retrieval where you can upload the files and then it can answer from that particular document this is what actually tutorial what we saw last time we actually upload the document of my website like HTML page and then we ask U you know assistant a questions based on that particular document and the good thing is that we didn't need to worry about the vector database or nothing we simply uploaded the file and then assistant API took care of chunking that file finding the semantic chunk and giving me answer and everything right that's the beauty of let's say the assistant API but today we're going to explore the other tool which is called the function calling right so let's look at how the assistant API is actually you know capable of calling the function just like we Define the specification for the functions assistant API is also we need to define something similar it is the same thing but it's like a list of tools kind of a thing we are passing so here is the for assistant we Define that same function like this that hey I'm giving you the list of tools here is the one tool which type is equal to function and here is the same thing what we saw on the top right so let's declare these tools for the assistant now again if you're not familiar you haven't watched that video let me quickly in one minute tell you what is the flow of creating the assistant API or how do we use it actually so here is the you know the notebook that I had uh given with that particular tutorial so basically as I told you we created that assistant which is capable of answering question question based on the uh uploaded document so if I go here you could see the first thing we did actually to upload the file we uploaded file on the uh you know open AI platform then we created the assistant so when you create the assistant I'm not sure maybe I zoom it a bit yeah so when we create an assistant we can of give some name to the assistant we can give some instruction we can give the set of tools which model we want to use and if any file we want the assistant should have the access right so the first thing is the creating assistant then what we do then we create the threade threade represent the conversation maybe uh every time you click on the new conversation here new chat is actually new when I click here it's actually the new thread creation and then you can you know exchange messages between them so every time a user interact with the chat bot on your website you're going to create actually the new thread for each user or each session you can think of right and once you create the thread which represent the conversation session then you can actually create messages inside that thread like you know whatever the message that we going to put and eventually you're going to execute or run that particular assistant right and once you run the assistant you know it will go to the different status whether it is completed whether it is spending and when it is completed you're going to get the respons from the assistant so this is the typical flow okay now let's come back to the tutorial what we had and we're going to perform the same thing again here so first thing let's create the assistant now here I'm telling the name of my assistant is going to be the e-commerce bot here is instruction that you are an e-commerce bot use the provided functions to answer questions and maybe I just say that you know synthesize the answer based on the provided function output and be anyway there could be spelling mistake because I didn't bother about you know all of this things GP is enough smart to deal with all of those things right now since we got the assistant created I created one more utility function because we know that whenever we want to get some output from the um you know assistant we had to actually create the message so I just declar one utility function okay here what it does actually it takes our assistant the user query and then it checks do we have already a threade if we don't have the thread it creates a thread okay and then finally it's going to create the message with whatever the query that user has say which is the creation of the you know the whatever the thread. message. create and we're going to Simply run that particular message and we're going to get the run run represent whenever we execute something actually all right and this is what this utility function we will see in detail when we're going to actually run actually the assistant right this is what this particular function does and then we have one more utility function what it actually does we know that right we just saw here so whenever GPD want us wants us to call a particular function it has some way of indicating it so in the chart completion thing whenever the function call has some value it means we need to call the function and then this is the syntax we use actually to extract the name of the function and the arguments that we need to pass which actually we got from the open or let's say from the GPT itself right similarly for the assistant API I actually also created one function because the thing is that every object is so much nested or hierarchical you know to get something you need to actually access like three four levels down right so I simply created one utility functions which take the run that we got from here you know when we're going to run the message we're going to get that run and that run if it required an action just like a calling a function or something like that this way I can access actually the name of the function okay for now just don't bother about it the moment we will print that object then it will be much more clear the only thing you can remember here this is the utility that is giving me the name of the function and the argument that I need to call right that's the job of this particular function now since we got whenever we just saw in the chat completion whenever we got response from the function that we call we need to submit back that response to the chat GPT so that it will generate response from the user for assistant API it is the same thing so I also created one utility function whose job is to actually take the function response that I got from our own API which is the gate order details API and simply call the submit output end point or that particular function of the you know open a library which will take the function ID because because it could be the multiple function so it takes the function ID that we actually called and then the function respond now this way GPT knows what response we got from our own API right this is how we submit the response okay enough of this all of those things now let's actually run these things here I did something better than what we just saw earlier here I try to simulate the chatbot kind of scenario so that we can ask some followup questions uh you know and we don't need to tell everything in one particular thing now let's run the thing so I'm going to run the query called I want to know my order status right and as I told you we already created the utility function which will take our assistant and the query and it will simply create message and run our query now let me print run also what we got this is how a typical run looks like actually if okay so run is nothing but whenever you run something you know some message on that assistant let's look at the status you say the status is cued means it is still in the in progress or kind of a status right and um you can see all these function details which is a part of this particular run I guess right right and the thing is that we have also seen in the previous tutorial that since assistant goes through the different state we need to keep track of does the status completed so we have to use some kind of a v Loop and keep retrieving the keep retrieving the our uh run information to see whether it got completed or not for that purpose we actually have this while loop what this while loop do it simply retrieve that it takes our run ID and the thread ID that we just had on the top and it will retrieve that run it will print that status of that particular run and now we are checking through status if we see that the Run status is completed means GPD directly has some output that we can directly show to the user Okay then if it is completed this is the way we can access so threads has the message list because since we are putting messages inside the thread open AI assistant also puts the response inside the same thread so using this uh you know list we can get all those messages and the first message is actually the latest message and we simply print that message okay here let me run this thing so when I run this thing you know if you see the Run status was completed means GPT had something for user what GPT says to help you with your order status I will need the order ID because it knows that we have given the tool specification that hey you're going to require the order ID to do something now let's see what happens let me subit saying that you know order ID is one now it again go to the V Loop and it see what happened it again went to the Run status cued okay it's in the v Loop right then see what happens then it went to the status called the require Action Now require action is where we have to call our function so instead of completed GPT went uh the assistant went into the requir action and what action we need to require let's see what's happening inside the code so when ever it went to insert this required action this is where we want to know our function name so we are using that utility function called get function details which will take the run which is in the required action stage and going to return the function name arguments and the function so if you look at our utility function which actually you know get function details we have printed the run. required action object so let's look at what that object has so this is how that required action object looks like this is like this big and you could see it will have the details of all these things so it has a tools calls this is the ID this is the call that we want to make this is the ID of the function here is the function details and you can see this argument we need to pass the function and this is the name of the function so the required action has all the details that is needed to call the function now next what happened when we got this D details from our utility function we use our execute function which is nothing but executing our a you know that particular function we got the response and as I told you once we got the response we need to submit that response back inside the threader for the assistant right and that's what here happens so we submitted and we again continued our conversation so let's go back to again that submit tool what happened nothing but it is submitting the response what response exactly we are submitting so if you look at what response we got so okay I haven't printed here response I guess but whatever the response we got here we actually pass it to that submit tool function so this is nothing but it again called our threade run submit tool and now here we are passing the function ID that we just called and the response that we got now GPT knows actually what you know and once it no then the loop again continued right and then it went into the let's see where it went once we submit the action the function detail it again went to the cued state it went to the in progress in progress and then it completed then it came here because now it got the response because we have submitted the detail now what this is what it displayed for the user user it is saying that the delivery status is shipped it is in the warehouse and all of this things right the order consist of the laptop and all of these detail it has given for the user then I might still ask you okay when can I expect the delivery and see what happened when I asked when can I expect the delivery it first went into the cued status then it again went to the progress progress and finally completed because it has all the information and it is saying that that I can expect by January I know the dates are old the data is old but you can see this is what the correct thing what the say so we are able to ask the followup question because it it knows okay so I I I can understand since this is not a proper chatbot UI but still I try to explain you know we try to uh simulate that thing here in the while loop and you don't need to worry about that you know you don't remember how can I get the function name and argument even I don't remember I don't know this you just need to uh take this code code or if you have any doubt you can go to the you know the API reference and there you can simply see the hierarchy you know here is the required action and then actually you're going to get the list of tool calls if you have multiple function you can get those multiple function call you will get the name of the function you will get the argument so we don't need to remember anything right and this is um the what I'm going to share the code with you and I made one condition that if the user input is a stop I need to exit this for Loop so I'm just going to do the Stop and I'm going to exit this wi Loop okay this is the finish so I hope you found this video useful you got both the code for the chart completion API and also for the assistant API try this thing and if you face an issue let me know inside the you know comment and I should be able to respond you thank you very much

Channel: Pradip Nichite

Views: 13,925

Keywords: openai function calling, function calling, assistent api, chagpt api, chagpt function calling, assistent function calling, chabot, openai chatbot

Id: pI1yUiNKyDA

Length: 25min 51sec (1551 seconds)

Published: Mon Nov 20 2023