Function Calling via ChatGPT API - First Look With LangChain

what's going on crew there was an exciting announcement today around open AI introducing function calling within their API which is pretty cool because up until this point we've been kind of wondering and I've been wondering how are they actually choosing which tools to use for their plugins and they're starting to peel back the onion layers and let us know how to do this so what I want to do in this video tutorial is go through their announcement blog post show you some of the key points that were really exciting to me and then I want to run through a Jupiter notebook where we do an easy intermediate and more advanced option for how to use function calling within your own applications one of the interesting waves that we're seeing with language models is increasing their capabilities outside of just normal text generation while poems about your cat were really fun to do back in October when chat gbt first came out there's a lot of systems that need to make decisions as well and when you use a language model as a reasoning engine well freeform text isn't the best way to talk to other computers it's better if you can do it in a Json format and that's what function calling starts to step us towards let's jump into the blog post awesome so here there's blog posts they came out with function calling and other API updates so the important parts here that I wanted to call out was they have a new 16k context window for GPT 3.5 that's not bad now they said they weren't going to do it for four because it's too expensive and they have GPU costs right now but we'll get there eventually now the interesting part is the function calling functionality that they described this is only going to be available in two static models and so you can't just use this with regular gpt4 or 3.5 you actually need to call out the June 13th version which is uh what they had today here the interesting part is it's going to be able to convert queries so email Anya to see if she wants to get coffee next Friday well you can specify the output of that natural language request that you wanted to return back to previously you wouldn't need to do some fancy prompting for this so if there's any way that we can offload this format to open AI who will guarantee its output and format then I'm happy with doing that so we can also do with API calls or database queries and we can extract structured data from text so this is going to be pretty cool now a function calling example what's the weather like in Boston right now so call the model with the functions uh with the functions and the user input so you're actually going to call openai and you're going to say hey here's what the user just said and here are the functions that you have available to you okay and then what it's going to respond with is it's going to respond with the format that was specified within the function itself all right now with that format that you specified in the function that should be the same format that you need to go and pass to your other systems so what you would do is you would go and pass that response from the language model over to your API so now you don't have to do any parsing of the two in between and then with that response that you got back from your API so in this case they got the weather back from the API there send the response back to the model to summarize so what we have here is we have a chat history message now this is a series of messages there is a user message which is what the user asked in the first place there is the AI message that said hey you need to go pass this information over to uh this is the function you're going to use and then we have information around what was returned from the function so we have a new role which we hadn't seen before which is the role function it's the name it's the get current weather function which is corresponding to this function right here and then we have the response which is going to be the response you got from your API and then we have the specified function down here and then what open I will respond with is the weather in Boston is currently Sunny with a temperature of 22 degrees Celsius all right so that's interesting there let's go try this out ourselves and let's see if we can have some fun with it awesome so I spun up a Jupiter notebook that we can go and work through here I want to work through those two examples and then show how we're going to introduce this within Lang chain itself now we're going to do the vanilla open AI example first and then we're going to move on to Lang chain and then a complicated example so we're going to import our packages here make sure our API keys are all set and then the first one we're going to do is the vanilla open AI example so what I'm going to do is I'm going to exactly just do what we had beforehand but I'm going to define a function that's called get current weather and so we're going to give it a name which is get current weather we're going to give it a description which is kind of uh this is the instructions that the model will know how to which tool to pick in the first place get the current weather in a given location okay cool then we're going to give it some parameters and then we're going to give it properties so here we're going to give it a location and a unit and you're going to define the type the description and in this case you can even find the enum values if you only want a specified number of values to come and return here and then your return parameters that shouldn't come back well in this case we want location and we want unit to come back let's go ahead and run that and so for the user query what's the weather like in Boston let's mix this up because this was just what was in there before what's the weather like in San Francisco and so keep in mind again this is vanilla open AI we're not using Lang chain yet we're going to call openai chat completion create we're going to use gpt4 and remember this is the June 13th model that we need to use and we're going to give a series of messages so in this case we're going to give it just one message which is a user message and the content is going to be the user query which we selected up above here we're going to pass in our functions parameter which is our function descriptions up above we only have one tool that's selected I hear but this is a list you can do multiple tools we'll get to that in a second here and then for the function call we're going to do auto now this function call it specifies whether or not the bot should return by using a function or not or should it choose in the first place and when you have Auto it's going to Auto pick for you if you were to put this To None then you don't want the function to be used then the function won't be used so what we get here is we have our AI response message now the content is null because there's not a message per se it's not actual content message but we do get a function call response back to us and here we have in our arguments we have the location and we have San Francisco which has been parsed out and then we have Fahrenheit which has been parsed out for us too in the unit and the name is the get current weather function so let's go ahead and clean this up here and we'll get our user location and user unit and I made a simple function here that's just the static function it's going to return some data for us it's going to give us the weather in this location and for this unit let's go ahead and run this and then for our function response let's get this all right so what we have here is this is the message that mimics what we would get back from our API call now this will usually be a confirmation of a change that has been updated or maybe a success message or something like that but either way this is the response from the API and then what we're going to do is I'm actually going to replace that one we're going to get a second response and so we're actually going to send our response back to open Ai and so we're going to use gpt4 and for this user this is going to be the message history that we had beforehand so the role we had a user message with content and it was a user query which is what's the weather in San Francisco and then we're going to pass it our AI response message so this is representing what it sent back to us in the first place and then we're going to give it the response from the API so we're saying hey you told us to go call this function and we just did call this function because we call this function get current weather and here's the response that we got from the function it's going to be all this really good information and it's going to notice that we got the weather and we got the forecast and then with that let's print out the second let's print out the next message here currently the weather in San Francisco is a sunny and windy with a temperature of 72 Fahrenheit now this is cool because what it did was is it got that API response from us it knew what format it was working with and it gave us the natural language response back okay here's the vanilla example about how to do this in open AI let's go and check this how to do this in line chain right now so the key part to keep in mind here is that with any new technology that is introduced on the open AI side it's going to take just a little bit for us to understand what's the best way for us to work with it and it's no different for Lang chain either here so I've done a few workarounds but kind of utilized what they've done so far and the team has been amazing they actually just put out an update just earlier today within an hour of this new framework coming out and they already had support for it so as we continue on I imagine there's going to be more and more support for it all right so we're going to import chat open AI because we're going to use gpt4 we're going to import a human message which is going to represent what the human is saying the AI message now in order to mimic the function message which we saw up above I'm just going to do the generic chat message here and then we're going to give it a custom roll we're going to import some tools that we might have some fun with let's go ahead and update these gpt4 all right so we're going to do the move tool and so this move tool is just hey how do you want to move some files around and this is actually the example that Harrison showed beforehand and then we're going to format each one of those tools into the function schema or the function calling schema that open AI wants to see let's do this and then let's take a look at these functions that we have right now so you can see we have our name we have our description and we have our parameters as well which is going to be the different information that it needs to know how to use those tools and then what we're going to do is we're going to pass a human message and we're going to say hey please move file Foo to bar but the important part is that we're going to pass in the functions right here and this functions is going to be the list of tools that it has available to us so let's go ahead and run this and then let's take a look what we have here so on the additional keyword arguments we have our function call one which is going to be the function call that we're going to get back from openai and this is the Lang chain way to do it it's saying that we need to use the move file tool and hear the arguments Source path is Foo destination path is bar and that is expected for us now I want to do a little bit more of a complicated example so I want to do multiple tools but then I also also want to do multiple requests within one user query okay so now let's do a function descriptions again so this is going to be a list of tools you notice that I have two tools right here and we'll get to the second one the first one is going to be an edit Financial forecast so make an edit to a user's Financial forecast model the reason why I did this one is because Financial forecasts are littered with parameters there's a lot of different ways you can go and update it all right so what we're going to do here is we're going to specify the parameters and we're going to get properties I'm going to have three properties year which is the year that you want to edit the category meaning the category of the thing that you want to edit and then the amount that you want to edit by this isn't by dollars we're just going to use units to make it simple and then for required which is what we want on the output on the way back we're going to ask for all three of them because we don't want to skip anything now the second function that we're going to do is going to be a print Financial forecast send the financial forecast to the printer and we're going to have properties again we're just gonna have one property which is the printer name so this would be my expectation for hey I'm about to make an API call I need to call a printer name please help me figure out which prints are I want to do we're going to say it's a string and it's the name of the printer that the forecast should be printed to and then the cool part that is well I only have two printers at my house I have a home printer and I have an office printer so I just want you to pick one of these it's pretty cool that we can specify this on this level and not need to go deal with prompts or making sure that it's super strict or anything like that so the enum will be these two printers and then record printer name let's run that now for the user request this is where I thought it got pretty interesting please do three things please do three things add 40 units to 2023 headcount subtract 23 units from 2022 Opex and then print out more my forecast at home now this is three requests baked into one and I thought it was pretty cool to see how they're dealing with this which I'm excited for you to check out as well let's run this user request now what we're first going to do is we're going to take that user query or user request and we're going to pass it through our language model now this is where I talked about it takes just a little bit of massaging on these early days until we get more support on it but I'm going to string these messages together so our very first one is we're going to have a human message and the content is going to be user request so it's literally just a human message with these three things on it and then we're also going to pass it our functions with our function descriptions which says our two tools above so let's run this and let's see what the AI comes back to us with all right so for the content remember there's no content necessarily because it's not really a message that is they're sending back it's more of a function call that it's doing so for this function call it wants us to do the edit Financial forecast which is great and for the year 2023 uh head count and uh the amount in the 40 which is what we wanted uh 2023 head count and 40. so that looks pretty good let's take a look at what this was again and we'll take a look at the name that it comes back with here it's just at a financial forecast and there's that same information we can clean this up if we want to we can go ahead and parse this and we can see our year head count and the amount that comes through there now what I want to do is I want to pass this information back to the language model itself and let's see what it wants to do for a second time so now I'm doing predict messages again but we have our human message which is the original request with all three different requests in there and then we have the a we're going to construct an AI message and the content is going to be the first response additional keyword arguments now that is going to be this information that's right here so what we're doing is we're telling the language model hey you already responded back to us on this first request and then we need to make a chat message which is where we're going to specify our functions that come in from here because this is the response from the API that we got so in Make Believe world we just updated our system and we got a response back from our system that says hey this was updated so this is the function response that we got back the additional keywords is this is going to be the edit forecast name and this is just a phony response that I made up but just updated the financial forecast for year 2023 category head count amount 40. and so what the language model is going to do is it's going to say oh we just got this API response back what should we do next well let's see what it thinks it should do next oh cool so we just ran this and it saw that this was successful and then it moved on to the second request so year 2022 category Opex and then negative 23. if we go back up to our messages right here subtract 23 units from 2022 Opex that's exactly correct which is I think that's really cool because it just made the decision that it could move on from that certain request all right so let's get this function name and yep again it did edit Financial forecast which is nice so let's do this a third time all right so we're going to pass in our original user request that had three different requests on it or three different queries we have the first response this is the updating then 2023 we had the second response which is subtracting in 2022. and then what we're going to say is hey we just did both of these so this function name it was the edit it was the edit Financial forecast and then the content hey we just made the following updates 2022 Opex 23 year 23 headcount and 40. now I purposely made this sloppy just to show you that it can interpret some pretty dirty data right here and it knows what to do so once these two updates have been successfully made well there's only a third one that's really ready to go it's to go print it out and if we're going to go check this out we say what's the third response oh we need to go print the financial forecast and then the argument is going to be the printer name because I said up here then please print out my forecast at home so it just saw that the first two requests are done and it just printed it for me or it's in printed format but it told me to go print it for me so it knew what to do now what's cool about this is I'm kind of bringing my own tools to the table and I'm doing my own printer work and I'm doing my own model update work but this is the reasoning engine that's kind of telling my systems what to do so I'm really Outsourcing the intelligence to open Ai and really it's intelligent through an API which is nice and if we look at this last line of function name it tells us to print the financial forecast all right so let's close this out we have no more function calls to make so what's it going to do for us now well here's the original response here's the um here's the response that it gave us for the first call the second call the third call about how to print the thing and then I'm going to go ahead and tell it that let me see if I can move this just a little bit for you I'm going to go ahead and tell it just printed the document at home this is my response that I got from my own system that I'm going and telling this um the language model and let's see what the response is for us here um great I've updated the financial forecast per instructions it has also been printed at your home that is pretty pretty cool because we just strung together a whole bunch of different commands and it shows which tool to use well not only did it choose which tool to use but it also chose when it didn't need to use a tool and it needed to give us a natural language response back now now I'm really stoked to see how y'all are going to be using this for your chat models I think there's going to be some really cool functionality that comes in here now it's nothing revolutionary per se but it's offloading a really complicated and kind of messy part of the prediction process that we hadn't had beforehand so please share your work with me I love seeing it on Twitter please email me any comments whatever you want to do let's go have some fun we'll see you later

Channel: Greg Kamradt (Data Indy)

Views: 34,777

Keywords:

Id: 0-zlUy7VUjg

Length: 16min 18sec (978 seconds)

Published: Tue Jun 13 2023