OpenAI Assistants API to Build AI Agent SWARMS. Better than AutoGen?

so the open ey developer conference happened not too long ago and the recordings that were available to the people who attended are now beginning to roll out to the rest of us and some of them are pretty cool for example this one this is where the presenter goes over how to create assistance by doing API calls now the two biggest announcements for me during the developer day were one gpts and two assistants they're both similar in that they allow you to customize the opening eyes model the the new GPT 4 with vision whatever they're going to end up calling it right now they're referring to it as GPT 4 1106 preview but you're able to take that model and build your own sort of Agents or chat Bots or your own applications with it now gpts the custom gpts are very powerful and I think we're going to see amazing things emerge out of them but they are purposely set up to be limited in certain important ways and that is they can't be autonomous they can't go out there and work on your behalf 24/7 each time it runs you have have to prompt it if you're doing an API call you have to click I agree can only do one call at a time you can't chain those together as far as I know as if this recording it's limited in certain ways which I think was part of the plan don't dismiss them because there's still a lot going on there there's a lot of opportunity and a lot of applications that can be done just with gpts alone however if you want to build your own autonomous agents using all the functionality that openi provides right now creating assistance with API calls seems like the way to go it seems ungated unlimited there's no barriers to what you can do with it or at least so far from what we've seen the only problem is to really truly be able to use it effectively you do need to be a developer be a software developer have have some coding skill have some coding skills and abilities Etc now from talking to some of you who watch this channel from doing various polls Etc I would estimate that probably less than onethird of the people watching have really the highle coding skills like that's their main job that they've been doing for the last 5 10 years plus the rest of the people that are coming coming in they might be a little bit new maybe they've worked with code maybe they haven't coded at all and so one of my goals on this channel is to try to bridge the gap between people who let's say haven't really coded before and being able to do something like this because you no longer have to be an engineer or a software developer or coder to be able to have access to something like this so in this video we're going to do two things one we're going to go through her presentation and see what she's talking about because I think it's very interesting we're going to see exactly what kind of power these assistants have and two I'll show you you how to jump in and start coding something like this even if you haven't ever coded before if you're already a god tier developer some of these might be a little bit basic for you but there still might be some things you might be interested in as we move forward in our next videos we will be working on more and more advanced subjects regarding to this but I wanted to do just a quick primer for everybody for whom all this just looks like gibberish all right let's get started all right so if you've never coded before and you want to follow along I think this will be great I'll show you how to build your very own AI app in under 2 minutes right ready let's start you're going to need two things the first one is Google collab so just Google Google collab should be the first one and it looks a little bit like this we're going to start a new notebook now you might have to sign in with your Google account or whatever other account you want but you know this is sort of under the Google umbrella and you need to it's free but you do need to sign in and this is what that looks like there's a lot of things going on here and you'll discover more of it over time but really what you need to know is that it has code blocks like this and you can add new code blocks like this and you can delete them like this and you can paste code in there and then hit play to run that code this saves you the trouble of creating a development environment and worrying about what code you're running on your computer this is all kind of an assimilation it's safer it's not going to break anything everything's on Google servers so it's easier it's faster you have to think about less things you can just jump in here and start coding hitting play and seeing what works so this made the coding environment a lot simpler the only other thing you need is the code where do we get the code well how about Chad GPT now Chad GPT just by itself you know using GPT for the pro version of Chad GPT it can do a lot you can ask it to create code for you and it would just spit it out and you just put it into Google collab and it will work what I'm trying to cook up right now is so far I'm calling it assistant coder so this is there's a lot of words you can't use it to name these things because if you're using certain words it doesn't allow you to publish it so I can't say open AI or whatever but this is my GPT that I'm building that's going to help me code these assistants so the thing that she's doing right here I want to be able to tell this GPT what I want to do what I want that assistant to do and have it give me the code I need to create something like this this is work in progress you're you're more than welcome to use it please jump in there try it out but it is work in progress I am going to be improving it there's a lot of things to be worked out but over time I'm hoping to develop this into something that will make creating these assistants and these API calls it'll make it very effortless and simple simple and easy but before we go there let me just show you what you can do without assistance and just by using openai API calls so my assistant coder I'm going to say let's use Google cab and open AI API to create the following and so I'm next I'm going to say what I wanted to create let's plot a chart of Nvidia and Tesla stock price change year-to date so that was the example that was given in autogen so using multiple agents to do that so let's see how well this can generate the code for us and so it's going to walk us through exactly how to do that so step one set up a Google collab notebook that's exactly what we have right here step two install required libraries so with a lot of this stuff when we're writing with python which is the language that we're going to be using here which is the language that's becoming kind of the go-to for a lot of these AI applications usually on the first few lines what you do or first few commands is you install all the various libraries and modules that you're going to need which is basically like the functionality that you're going to need to do whatever it is you're doing in this case since we're talking about stocks we need Yahoo finance we're going to be using open AI so we're going to import their libraries and the way we install that stuff is we do this little Pip install you're going to see a lot for various things pip install this pip install that and so here we're pip installing Yahoo finance this thing this is for plotting it's like charts and graphs Etc and opening ey you can also do this on three different lines by saying pip install this then pip install that and third line pip install open AI so we're going to copy that bit of code we're going to go to our Google collab we're going to paste it into that first code block and we're going to hit play boom and so what it's going to do it's going to go ahead and install those libraries that we need in this environment in this notebook in this project that we're working on so as you can see here it's like saying well we already have some of these things and then it doesn't have this thing so it's downloading it a lot of this you're not going to you're not going to need to understand it you just need to think of these as little Lego pieces that you're putting together into some functionality over time the more and more you use it the more and more you'll understand what the different pieces mean but don't get too bogged down with trying to figure out what each line means that will come over time but now because we have these coding AI assistants can be a little bit easier to to start building and then learn as you go you might have heard this referred to as just in time learning right so you're not learning you're not pre-learning a bunch of Concepts You're Just in Time learning it to build the very next thing so this is the output where it kind of tells us what happened we don't need that let's just clear that this has a check box next to it saying okay we we did that part all right so the next thing that our assistant coder is telling us is we need to import libraries so some of the stuff that we've installed let's go ahead and import them into this project right so what we're going to do is we're going to add another block of code and we're just going to paste that in there and we're going to hit run and so what this does is it brings in all those modules that we need all those tools that we need to make this run all right next we need to get our open AI API key so an API key it's your secret key that you get along with your open AI account and basically it's like the keys to your car right you put it in and makes the car drive so any piece of code that uses open AI API calls it needs a key for it to happen it's also works a little bit like a credit card and that if you if you put in your API key and then you run functions that cost money so for example when you're generating certain responses they cost money per token right your open AI account gets charged that amount so you got be a little bit careful you got to make sure to keep it safe you don't want to give it out to anybody like you don't want to do a YouTube video and show everybody your code when tens of thousands of people are going to see it that's a bad idea but yeah so we're going to copy this and we're going to put that into we're going to create another code block and we're going to plop that right in there and then we're going to get our API key in the open account so you got to log into openai so platform. open.com here on the left we have our API Keys all right and then we're going to click create new key we can call it whatever I'm going to call it test because I'm going to end up deleting it afterwards and here's that key we're going to copy it and we're going to replace this your API key with the actual API key so that goes between the two single quotes like that so we took the openi key and we set it to this which is our API key we click run and so that saves it so now this file will remember that for whenever we have to use next all right next fetch the stock data and this is the little command that it gives us to to do that and so we're going to put that in there we're going to hit run so I got an error message because there's no symbol ticker Tesla it's TSLA weird so it's weird that made that mistake but there you go so here we can just go ahead and you know delete the E because the stock ticker name is TSLA let's try it again yeah so it's completed okay stuff like that happens every once in a while but it's an easy fix in fact if we tell it that it made that mistake it might actually fix it once we're done with this I'm going to try to see if uh if we can do that all right and next step is PL data so we just copy this code create a new block put that in there and hit play and bingo as if by Magic AI magic here is that year-to dat stock price change for NVIDIA and Tesla so from January to we're in November now what's interesting so we didn't even need open AI for any of this none of this actually used openai for anything so we used uh Yahoo finance to fetch the data and we used this this matap plot library to plot the thing but the next thing we're going to do is if we wanted to have a text analysis of the stock performance using GPT we can do something like this Mito gives us the openi API call we need to get the response back as you can see here we're going to use model Tex D Vinci so it's one of the older ones and the prompt is wrer brief analysis comparing the year-to-date performance of Nvidia and Tesla stocks let's try that so we're going to create a new code plop that in there and hit play now what happens is we get an error message because just now the open AI upgraded a lot of these commands because they're rolling out all the new models and stuff like that so this is an outdated command but as you can see it gives you a read me here that tells you how to upgrade it that tells you how to solve that problem here it gives you the code that you would need to use instead so I'm going to go back here I actually create a brand new block here post that in there we'll take the prompt of Write a brief analysis comparing the year- to-day performance of those two stocks we're going to use this and we're going to put that as the content here all right and we're going to put our API key here and we're going to hit play so what this did is sent this request to open Ai and open AI completed that request with the GPT 3.5 tripo Now by default notice it didn't show the answer here so but we can do this print but we can say print and then chat completion as you can see here we have the content that it gave us the year- to-day performance of Nvidia and Tesla stocks can be briefly analyze as follows and it gives us this whole long answer about the analysis all right so that might have looked a little bit weird right because right we sent this completion request and then we had to print the completion request but this is important to understand because that's exactly what we need to be doing when we're dealing with these assistants you'll see why in just a second but let me just say this if you noticed we built this whole app if you know how to do this you can probably do it in a minute or two we used Yahoo finance API to get information from them we used a library that is able to create these graphical charts and graphs that's able to show it to us in these graphics and we use the openai API to write a brief analysis comparing the stock prices I just wanted to show you what you can do with just collab and Chad GPT or in this case this custom GPT notice I didn't write any of this code other than saying print here at the very end I didn't write any of it Chad GPT wrote it in its entirety I told it what I wanted to do here and it did all this and explained to me how to do it now the goal is to make this into something that's able to code assistance for me so with that let's go back to what this lady here is talking about so this little presentation that she's giving she she's saying a couple important things one is that you can create a new new thing let's say called assistants and an assistant can be thought of an agent that you've trained to do something specific you gave it instructions you gave it various documents to read you gave it basically everything that you can do that you can do to a GPT you've trained it to do a particular task whether that's fetch stock prices or check the news or edit video or help you write or create images whatever you can think of you can create an assistant that does that but here's the big difference that assistant will be persistent meaning it won't just disappear it'll stay there and you can call upon it at any given time with an API call and this is where that idea of a swarm comes in we've talked about earlier this idea of creating autonomous agents and really swarms of these autonomous agents many of them working together to do the things that you want them to do and each individual agent in this case is referred to as an assistant so you can have a 100 of them a thousand a million and the code like this you call on each one with an API call and then a code can have you know a million different calls as you're calling on one of them at a time to do whatever different task you want them to do my apologize if this sounds confusing but keep in mind this is one video of many we're going to get through this but the first step is to understand what she did in this presentation and how you can do the same so let's get started so now we're going to go back here we're going to ask our assistant coder help me create an assistant and it answers with this now this isn't the answer is going to give you normally this is the new correct answer this is the updated answer and this took a long time to figure out how to do correctly because otherwise it would just default to using its knowledge which is out of date and and this is why I think this assistant coder will be so useful because it will have it will have all the upto-date stuffff it'll tell you exactly what you need to know and it'll make this whole thing easier but so this is the correct code to do that so let's copy and paste it over I had to get really emot tionally manipulative to get it to give me that answer instead of what it thought was the correct answer so I posted on in here so create an assistant we're going to name it math tutor instructions you're a personal math tutor write and run code to answer math question and for tools we're going to use code interpreter we're all we're going to allowed to use code interpreter and the model is this latest model gbt 4 1106 so let's run that boom so what it did here is it created that assistant in my openingi account so if I go back and I take a look at my assistants boom as you can see there it was created on November 15 7:50 p.m. I've been doing this so long I just realized all right and the ID is that so let me take that and we'll copy it and just like we did up here with our open AI key we're going to say assistant _ ID equals 2 and we'll put that go here like this we'll save that so now in the future if we need to talk to that assistant that code will be save we just need to call it using assistant ID anyways in the future I'm hoping that the assistant coder will have all the answers how to do all of the stuff right now I have to slowly one at a time like just coers it into giving the correct answers but this is the new and updated documentations that they have here this is the correct way of doing it and so the next step will be to create a thread a thread is like a conversation like a back and forth conversation all right so we'll copy this we'll throw that into our Google collab like that and we'll hit uh run so that's done it created a thread by the way when you run stuff there's this little variables tab here so you click on the box that you ran and it will show you the various variables so it's basically all the information that gets saved and created is in there so in this case you know the thread ID is that in case we needed so I'll do thread and so as I'm typing it it it thinks that it should be this which I think that is correct and so we put the thread ID in there we'll just run that and save that and so then we can add messages to that thread and any documents that the user might might be able to upload so for example here we're going to ask our assistant I need to solve the equation whatever can you help me so we'll post that in here all right now that that ran that thread is Creed with the different messages and now we have to print those messages so remember that's part of how the system is set up so you send a request it generates the request and then but nothing else happens it doesn't automatically spit it out with the new assistance API I don't have to store any messages in my own database open AI handles truncating messages to fit the context window for me the output is generated even if I'm disconnected from the API and finally I can always get the messages that the model output later they're always saved to the thread I think that's pretty cool then you can have the information go wherever you want you can push it somewhere else you can display on screen whatever I asked Chad PT how to print it after this happens and it gave me this code all right so what this does is it adds that message to the thread so the user says Hey help me with this next we run the assistant to answer the user how do we do that well it's this right here we p P that in there and it says please address the user as Jane Doe the user has a premium account so we're going to hit play so what that did is now the AI answer the question so remember everything's getting saved the whole conversation is getting saved if we want to see it we just print it but until we say print it doesn't show up or we can send it somewhere else so it's visible elsewhere Etc and then this is how you display the assistant response all right so I ran this code to get the back and forth messages so as you can see here the user asks I need help with this equation and the assistant and anwers the solution to the equation is this all right so I know that might have been a lot it might have been a little bit confusing you're like what what is going on but the point is those are the building blocks of building your own back and forth assistant here's how to ask it questions here's how to give it certain tools here's how to extract answers they are your basic building blocks of how to build an assistant let's take a look at this presentation I think now that you've seen how to do it yourself it might make a lot more sense and then after she'll also show you an application of how to do it by the time we have the next video out I'm hoping to have this assistant coder a lot more developed so you can just tell it in plain English what functionality you want the assistant to have and it will go ahead and it'll create those assistants give them roles create the threads Etc now if you've played around with autogen as you might imagine you can create multiple AI assistants with different roles fow them all in one thread and have them talk back and forth to each other you can do that with all these tools that we just went over so I hope that made sense I know it might have been this is a little bit of an experiment that I'm doing so hopefully I was able to bridge that Gap a little bit if you have any questions let me know if people have a lot of the same questions I'll try to answer them in the next video and check out this assistant coder again it's going to be getting better and better every single day I hope and in the meantime here's the rest of that clip from the open aai Development Day Conference thank you for watching now let's get started building our very own assistant so I'm building a geography fre tutor app to teach my users about world capitals I can get started in just a few API calls and since you all are developers I'm going to start out in my terminal with some curl requests can see my terminal here awesome so we're going to start with the very first curl request so now we want to create this assistant to power my geography tutor app you can see we're posting to the assistant endpoint and we're passing two key pieces of information the first is the model we're using the newest gp4 turbo model the latest heat that just dropped today next we're passing the instructions to the model you can see I'm asking the assistant to be helpful and concise and letting know that it's an expert when I send that request I get an assistant back and I can store that assistant ID for use later before the assistant's API if I wanted to use these instructions repeatedly I would have to send them on every single API request to open AI now with the new stateful API I can create my assistant once and the instructions are stored there forever now let's move on to the next primitive that Olivia mentioned which is threads threads are kind of like one session between your user and your application so now I've got a user on my website and they're starting to type out a geography question so let's create a thread you can see here we're posting to the threads endpoint and there's nothing in the body because the thread is empty for now we've got a thread ID so just like before let's save it for later cool now my user has finished typing and I want to add their message to the thread so let's do that you can see here I'm posting to the threads thread ID messages endpoint and I'm passing in the data about the message so the role is from the user because the user's typed out this message and the content is their question so they're curious and they want to know what the capital of France is cool so now you can see we've got a response back we've got a message ID and the message has been added to the thread now you're probably wondering how can I get my assistant to start acting on this thread so we've introduced a new primitive called a run which is how we've packaged up one invocation of your assistant so let's cick off a run now and get my assistant going you can see here I'm posting to the threads red ID runs endpoint and I'm passing in the assistant ID this is pretty cool you can actually have multiple different assistants work on the same thread but for for now let's use my geography assistant I'm going to kick off the thread and tell you a little bit about what's happening in the background you can see I've got a run ID and the run is CED so run is how we've packaged up all of the work of loading your models all the messages in the thread truncating to fit the model context window calling the model and then saving any resulting messages back to the thread so let's see what happened we can fetch the messages on the thread to see how assistant has replied so I'm going to issue a get request the threads thread ID messages endpoint to see what there is you can see we have the first message from our user and then the assistant has replied saying the capital of France is Paris so this is a super simple example but we can talk about why this is better than the previous API with the new assistant API I don't have to store any messages in my own database open AI handles truncating messages to fit the context window for me the model output is generated even if I'm disconnected from the API and finally I can always get the messages that the model output later they're always saved to the thread I think that's pretty cool great so these are the basics of the API and now let's move to what I'm most excited about which is how we can power your application with tools one of the most useful parts of assistance is their ability to leverage tools to perform actions autonomously code interpreter is a tool that is hosted and executed by open aai when I enable code interpreter in my assistant I can expand its capabilities to include accurate math processing files data analysis and even generating images to optimize the quality and performance of these hosted tools we fine-tune our models to best determine when to call these tools what inputs are most effective and how best to integrate the outputs into the assistant's response so now let's get started by using Code interpreter so I'm actually building a personal finance app that I want to ship to my users to let them analyze their transactions and we've already been in the terminal so let's move over to the open AI playground here you can see the playground that you know in love it's super useful for testing chat completions we've actually refreshed it and you can see there's a drop down in the top left to pick the assistants tab here in the playground you can see you can create assistants change their instructions and you can start threads super useful for testing you can actually see the assistant I just created in the API all of that information is loaded here as well so now let's create a new assistant to show off the power of code interpreter I'm creating a personal finance assistant so let's give it a name I'm going to call it the personal finance genius and I'm going to tell it you help users with their personal finance questions now I'm going to select a model and I'm going to select the newest gb4 turbo there it is and I'm going to flip on code interpreter with just one click or one line of code you can enable code interpreter for yours system so let's save that great so now a user has come to my application and they want to analyze a spreadsheet of their transactions let's take a look at what it looks like you can see there's a bunch of info in here bunch of numbers the dates aren't even sorted it's pretty messy let's upload that to the thread and now my user is asking for a chart so generate a chart showing which day of the week I spend the most money awesome so now I've done a compound action to create a thread add a message and kick off the run you can see things are happening in the background let's let's talk about what's going on when we've kicked off The Run we will get all of the messages on the thread summarize them for the model to fit the context window terine if it's called any tools execute the tools and then give that information back to the model so let's take a look oh here we are we've got the output actually it's actually pretty surprising I did not think I'd be spending the most money on Sunday let's look at how we got here you can see that our personal finance genius has started by telling us what's going on it's actually written some code we can click into the code here and then it kept writing some more messages and finally generated a chart which we were able to see so we can actually look a little more deeply to see how this happened we can look at what we've called steps steps are basically the logs for your run and they're super useful so you can render your own Rich uis to show your users what's happening you can see the logs tab here I can open it scroll down and find the steps for this run this is reverse cron so I'm scrolling to the bottom and I can show you how we got here so first you can see we've created a step and it has type message so that corresponds with this first message we've created next you can see there's a run set that is a tool call and it's for code interpreter so this is how the playground was able to render this snippet the inputs and the outputs are directly in the API next you can see another message creation step corresponding to this message and then there's a few more code interpreter and message Snippets so all of this information is sufficient for you to render the same UI the playground is actually built entirely off of our public apis so you can do the same and speaking of this is exactly what chat GPT does under the hood when they're using Code interpreter so you can skin your application to look however you like great now let's move on to retrieval the retrieval tool is incredibly useful when you want to expand the knowledge of your assistant maybe you have some information about your app or business that you want to imbue your assistant with rather than having to implement custom retrieval system on your end openi has built a tool that you can use with one line of code or one click in the playground the retrieval tool does all of the document parsing chunking generates embeddings determines when to use them so you don't have to let's get started so I'm going to hide the logs clear this thread and create a new assistant so now I'm actually building an assistant to help my users you know use the open AI API better so I'm going to create the open AI API wizard that looks pretty good now I'm going to tell it that it's a helpful assistant and then use the attached docs to answer questions about the open AI API so I'm actually going to upload I have a full dump of the open AI dots just to markdown file didn't process it all I'm going to attach it to the assistant and I'm going to flip retrieval on just one click finally let's pick the new gp4 turbo model and Save Our Ass so while this is being saved we're actually doing the work on the back end to make this data accessible to the assistant so let's see what it looks like when my user asks a question about the openai API so my user is curious uh and they're wondering how do I use the embeddings API and so now we've klicked off a run again what's happening in the background here is we're fetching all of the messages truncating them however needed calling the model determining if the model has called the retrieval tool executing retrieval for you and then giving that back to the model to summarize you can see here we've actually grabbed a snippet from the docs and we even have a citation giving a direct quote from our docs so we could render that for the user I think that's pretty cool just like last time we can take a look at the steps and see how we got here this one's a little simpler first the tool call to the retrieval tool so you can let your user know that's happening while it's going and then we have a step for message creation so you can render this UI awesome so now back to Olivier to explain how you can use the retrieval tool with different levels of file scope not that retrieval is extremely useful to augment the assistant the knowledge of your assistant with data from outside of the models concretely retrial can work in two different ways number one you can upload and pass files at the assistant level that's useful if you want your assistant to leverage that knowledge in every single interaction in every single threads so for instance in Miss example on the customer support and the API docs you should likely pass that information at the level the second option is to pass files at a thread level that's useful if you only want that specific to be aware of that content so so if I'm uploading my personal bank statements I should likely do it at the thread level behind the scenes it will takes care of sometimes it even like do not do like vector search and instead um stop the context um so you don't have to essentially handle that logic um on your end and we're excited to launch several new features over the coming month for instance like we want to allow you to pick between thisable strategies to find the right balance between cost accuracy and lency one final reminder open AI never trains on any files or data that you pass to the and so that's the same for retrial files all right let's move on to the last category of tools function calling so again function calling are custom functions that you define to the models and the model selects those functions on your behalf we are excited to release two new features to function coding starting today the first one is Json mode with Json mode the model will always return valid Json behind the scenes we made model improvements and improvements to our internal infer stack in order to constrain the sampling of the model and to make sure that every time the put complies with Json syntax so that's pretty useful that means that you can trust that the model generate Jon that you can directly execute on your end that's pretty cool and by the way function Json mode will also work outside of function calling so if you use for instance the chat completions API and you have like a very simple application that just like text in data extract like some Fields convert it to Json Json mode will work as well the second Improvement to function calling is parallel function calling with parallel function calling the model can call multiple functions at a time we SE in many applications users giving multiple instructions at once to an assistant so let's say for instance you know I'm building a car voice assistant and I tell the assistant raise the windows and turn on the radio before parallel function calling that meant that you had to do two different model calls um to open AI which of course result in extra latency and cost with parallel function coding such a use case can be handled with one single call which is pretty awesome all right let's do a demo now of function coling awesome back to the demos so I'm building off Olivia's example and I'm building a new type of car and I want a voice activated assistant to be able to use some of the features of the car so I have a bunch of functions that I've implemented so let's pull up an assistant I've previously created here it is so I previously created this car assistant and I've told it it's a helpful incar assistant and to please call the appropriate function based on the user's request so with function calling I have an assistant with all of my functions and my assistant will tell me when to call my functions based on the user's input and what the most appropriate arguments are this is super helpful so I can figure out how to use my system to answers the user request so let's take a look at some of the functions I have here so I have a lot of the stuff you might expect to see so honor start phone call send text message so let's try it out there's a user in my car and they just said ah that guy just cut me off let's see which function the assistant determines makes the most sense to call there it is our assistants decided to honk the horn to let that guy know it wasn't cool you can see here that we've called the Hong horn function with no arguments so now I know how to pass it to my system we also o have a way for you to give the function output back to the assistant so it can keep going so here I'm going to enter the output it was successful oops I have a typo there but that's all right we're going to keep going and the assistant says I've honked the horn for you please stay calm so actually I'm realizing that there's a function here that's missing that I want to add so let's show you the add function flow you can see there's a button here where I can add a function we have some helpful presets to get you started so let's use the get stock app and change it a little bit to be my function so I actually want a function to be able to change the volume in the car so I'm going to call it set audio volume and I'm going to give the description to the assistant so sets the set the car audio volume now we need to tell the assistant how to call this function so we can tell it about the parameters and whether or not they're required so the main parameter for my function is the volume and actually the volume isn't a string it's a number and so this is the volume to set and it's 0 to 100 finally we'll tell the assistant that this is required when you're calling this function great this function looks good so let's say save it and now I can save my assistant you can see here the new set audio volume function is listed on the left let's start a new thread and now my user is asking for a different query they're saying play wonder wall and Crank It Up in the background my assistant is determining which of these functions makes the most sense to call you can see here we've got the output and there's actually two functions this is showing you the power of parallel function calling the first function starts the music with the query wonder wall makes sense and the second sets the audio volume to 100 pretty cranked up now I can execute these in parallel and then get back to the assistant awesome we're super excited about parallel function calling and let's recap everything we launched today we're super excited to see what you build with the new assistant API we've added three new staple Primitives the assistant for storing instructions models and tools threads for keeping track of all your conversations and messages which are between the user and the assistant we've also added the runs primitive for every time you want to invoke an assistant and we've added steps which are useful logs for rendering your UI based on what the assistant is doing finally we've launched two new tools code interpreter retrieval and we've made two huge improvements to function calling Json mode and parallel functions we're super excited to see what you build and now over to Olivier to tell you about what's next thank you Michelle so that was a lot but there is a lot more coming soon we plan to ship many new improvements to the assistance API in the coming month number one we want to make the API multimodal by default it should be able to accept and generate images and audio file number two we want to allow you to be able to bring your own code execution so you can execute on your end the code that was generated by the assistant a third big um feature that we want to ship soon is to um ship a synchronous support with websockets and web hooks to make it easier for real-time applications to use um the API it's a new feature it's a new products it's in better uh we would love to hear what you'll build if you have any feature requests any wish lists please tweet at us at openi and show us what you build we would love to hear all about it thank you so much and enjoy that day

Channel: Wes Roth

Views: 43,390

Keywords:

Id: hdHtIdJSWuQ

Length: 34min 52sec (2092 seconds)

Published: Thu Nov 16 2023