NEW GPT-4 Function Calling Model!

open AI have just released a new functionality for two new models those models are the updated GPT 4 and gbt 3.5 models this new feature is called function calling and it essentially allows us to pass to gpt4 and GT 3.5 a description of a a python function and based on that description ask the model a question and if the model sees that this question needs to use one of the function descriptions that you've passed to it it will go ahead and return you the parameters that you need in order to use that function in the way that you have described to GT4 GT 3.5 so it is essentially the tool usage that you would get from Lang chain but directly within the open AI API and it seems just from some initial testing to work pretty well so let's have a quick look at what this is and also dive into the code and how we can actually implement this ourselves so here we can say we can see that describing what this function calling is essentially what I just said right so we can describe functions to GP 3.5 gp4 and the model will choose whether it should output a Json object containing arguments to call those functions and then this is an important thing here so the chat completions API does not call the function it provides you with those parameters in Json format so let's go ahead and actually see how this works so I'm going to be using collab we don't need all of these prerequisite libraries just understand how this works but I wanted to go with a kind of interesting examples so later on we are going to be using an image generation model from hugging face diffusers so for that component of this notebook again you don't need to get to that point in order to understand this but for that component you do need basically all of these prerequisites or libraries and sold and you should also make sure that you are using a Cuda enabled GPU because otherwise it's going to take a very long time so you can switch to a GPU runtime within collab by again to runtime at the top into your change runtime type and making sure you have GPU in here and you can use the T4 GPU on standard which I believe is available on the free version of collab as well so let's go ahead and run this so one thing that we will need here and we will need to pay for unless you are within the like the free usage of open AI is a open AI API key so you can get one of those over at platform.openai.com there'll be a link to that top of the video right now yep once you have that all you need to do is just enter your API key in here run this and you should see this and this will tell us that you are authenticated to use the open AI API okay it looks good so what we're going to do is first create a function that we will be using gbt4 to call also if you still don't have access to GT4 that's fine we can also I'll show you where you can switch this out for gbt 3.5 as well and you should get pretty similar results okay so this function that we're going to create is a page builder for a a product right so imagine um your no you want to generate a ton of product pages you're going to use this this function here and what you're going to want to do is sgpt4 or 3.5 to give you a title and some copy text to like the marketing text for that product and we're just going to take that into our HTML code here we're going to write that to file and then just for us in the notebook we're going to view that file as well so we run that run it again and you should see in a moment we get this okay so I've just manually put in these two um parameters here so the title of the product and the copy tapes for that product okay now what we want to do is give GPT 4 GT 3.5 instructions on how to use this function okay so to do that we need these three items at high level so we have the name of this function that gbt will use it's going to be called page builder um it doesn't it doesn't necessarily need to align to the function name here you can put anything out anything you want in there but basically you'll be using this to identify which function a GPT wants you to use we need a short description so creates product web pages you can make that longer or shorter depending on on what you need and then parameters so this is the probably the most important part so these describe the inputs into your your function right so we those actual inputs are with then the properties here okay and we have two we have title and copy text right so both of these are strings and I'm just describing what they actually are okay so the name of the product and then here marketing copy that describes and sells the product that's all it is and then after that so within parameters we say what is required and that is all of the parameters so title and copy text okay so this is basically the description on how GPT can use our function so with that what we're going to do is go to the check completion endpoint we're going to create our prompt so create a web page for a new Cutting Edge mango cutting machine that's what I want gbt to create for us and then as usual we use check completion create so these lines I've highlighted here we would always run those okay the only difference now is one we're changing the model so using this gpt40613 if again you don't have access to T4 yet you should use this model here so GT 3.5 turbo zero six thirteen and then the other difference so the difference allows us to use this function calling feature is we add functions and then we pass in a list of functions that we would like GPT to be able to use and when I say functions I mean these kind of function descriptor items right so we let's run that and this okay and in the response from GT4 we can see we we get all of this so we have um what do we have here so basically we can see in here we have the message we have function call okay so this is the function code that GPT is creating for us and we can also see that we have this new finish reason so programmatically how are we going to identify that GPT for wants us to call our function well we're going to come down to here we're going to say okay if the Finish reason is equal to function call that means we need to call a function okay so we run that and okay yes we should call a function and then based on that if it says we should call a function we need to extract the name of that function so if we have multiple functions this will be useful so we know which function to use and we also need to extract the arguments that we should pass to that function okay so we do that also note that this is actually a string so we convert it into a dictionary okay and we get this right so with that we can use the keyword arguments syntax here and we can just pass all those straight into our page builder function okay we run that and we get this so the manga master and we get some copy that is is definitely better than anything I could write myself so that is pretty cool but I think we can probably do better especially with all these other models um that are available to us all right so this is yeah it's a nice text but I don't I don't know of any product pages where it's just takes there's always images as well so can we also get this GPT for LGBT 3.5 function to also generate an image for us and yes we can just not with GT forward GT3 1.5 we need to use a different model so we're going to go ahead and do that so we're going to use stability ai's stable diffusion model so this is disabled Fusion 2.1 and this is just how we initialize it okay so using hook and face diffusers here again this is this is actually where we need to use accurate enabled GPU you can use CPU it's just going to take a long time um for it to generate the image for you and all of this up to all this I've highlighted is what we need in order to initialize the model or the the fusion pipeline then here I'm just running a test okay so my test prompt is a photo of astronaut riding a horse on Mars and then we're going to generate that image with this and I'm going to display that image to make sure it is actually working so that will take the first time we run this it's going to take a long time to just download the model so you'll just have to wait a moment for that okay so I've just switched across to this notebook which I've already pre-run But continuing from where we just were after loading everything you can just run the same thing and it will be very quick so this is 20 seconds on the T4 GPU I'll see if you have something bigger like an a100 it can also go faster so so this is all you get so it's pretty decent a photo of an astronaut riding a horse on Mars Works relatively well now the thing that we need to do now is integrate this image generation logic into the function that we were giving to GT4 before so our page builder function so that's actually really simple all we're going to do is we're going to provide an image description so this is going to be the prompt to generate the image and we're going to take that we're going to pass it into the pipe that will generate the image or extract it and then we're going to save it to file right and then we're going to use this image on file as the image in our our product web page Okay so that means we're gonna have some slight differences in the HTML so I think the actual HTML here is we just add an image and a source we don't need to make this dynamic because we're just going to save the image file so it's going to be product.jpg all the time and yeah I mean that's basically it we just have some CSS to make sure things are styled kind of correctly and that's that's all we need so from there we would just as before we're going to save our index HTML to file and that is our function modified we didn't really change much we decided that image generation step now as well as modifying our actual python function we also need to modify the function description that we're passing to the GPT for 3.5 so in this we just need to add an extra parameter into our properties and that is going to be the image description parameter again it's a string and what we do is just give a description of what we want so I want a concise description of the product image using descriptive language but also no more than two sentences long okay so I actually added this in later because if we don't add that in it's going to give us a very long piece of text to describe the image which is fine but there is a token limit in the clip model that is used by stable diffusion of 77 tokens which is relatively short so it means that things are being truncated and we were not getting all the detail that we actually wanted within the image prompt so I added that low no more than two sentences long into the end then it works well and then we just add the image description to our list of required parameters and yeah that's it so we're just gonna with this we just run our this whole logic again uh the only difference here is I've wrapped all into a single function so we make our request to open AI we pass in our page builder function description and what it's going to do is we're going to check for that finish reason if it's function call then we extract the name and the arguments if the name is equal to page builder then we're going to call the page builder function with those arguments and then from there we return the HTML of that page right and we get this okay so this is actually just a loading bar it just doesn't display properly but below that we have our HTML here we have the title The Copy Text and then we have the image here I couldn't find a good way of showing the image from colab so what you actually just need to do is you go to your files over here and download the index.html and product.jpg and then just open it locally and I will show you so if I open this the HTML file locally we'll get this so we get this I don't know mango cutting machine like thing in the background here we get our title and our our copy there as well which I think is kind of cool it was super quick to put that together and you know it's decent so yeah that's it for this video I just wanted to have a quick look at this new feature from openai the the function calling seems really interesting and it I I suppose realistically we're already able to do this with a lang chain but I think having it as within the chat completion endpoint is pretty useful for when we just want to kind of keep things simple and lighter in our code so I think that will be pretty useful and from what I've seen it seems to work really well as well so I think this is I think it's pretty interesting now that's it for this video I hope all has been useful and interesting so thank you very much for watching and I will see you again in the next one bye

Channel: James Briggs

Views: 44,517

Keywords: python, machine learning, artificial intelligence, natural language processing, nlp, gpt4, gpt 4, gpt 4 api, gpt 4 python, gpt-4-0613, gpt-3.5, gpt-3.5-turbo, gpt-3.5-turbo-0613, gpt-4 functions, chat gpt 4, openai functions, openai functions example, james briggs, langchain agents, langchain tools, openai agents, openai tools, gpt 4 tools, gpt 4 agents, gpt 3.5 tools, gpt 3.5 agents, gpt 4 function calls, gpt 3.5 function calling, gpt 3.5 function calls, gpt 4 function calling

Id: 7oZKIwz2wk8

Length: 16min 26sec (986 seconds)

Published: Wed Jun 14 2023