Vapi AI Function Calling Explained | Complete tutorial

trying to enhance your vappy AI assistant with functions is like going for a treasure hunt you need to fight your way through the jungle or better the Swagger documentation until you maybe find what you're looking for at least that was the case when I started out building AI voice assistance on VY and luckily since you found this video you don't have to go through the same pain I did today I'm going to show you how you can Implement custom functions and tools directly into vapi without spending hours inside of their documentation you will also learn how to debug functions work with custom parameters and I will also give you access to a couple of goodies that I learned along the way if you don't know what function calling is here's a brief explanation function calling is nothing else than giving your AI assistants access to tools and functions that they can use during a conversation this allows your AI assistant to directly interact with Ds while it has a conversation with your actual users this is incredibly powerful as you can share live data directly during the call while the user is on the phone while this may sound complex in theory here's an example imagine you have an AI caller for a real estate company and you have another person on the phone who would like to schedule meeting with an available agent tomorrow at 9:00 p.m. to explain it even better to you I created something for you visually so this caller is basically the one that is calling a real estate AI agent and vapi is the one that obviously host the AI agent if you don't know vapi it is a AI calling infrastructure provider I have mentioned it many times in my previous videos and I definitely suggest you to check it out if you don't know it but let's just quickly go over the flow so let's assume a caller is basically there and calls our real estate agent with VY or on vapy then VY would basically understand aha there is a call happening and let's assume that user asks in that moment is there an agent available then bapi can schedule a function call and send it to a custom logic platform which can be anything like your server make.com any other no code or low Cod automation platform and this then basically is able to figure out what to do with this request so VAP send a functional basically that says something like agent availability then our custom logic platform decides what to do and decides to ask our calendar or CRM if that agent is available then this calendar can either say yes no or anything else you would like or maybe even give some alternative dates and then sends that information back to the logic platform and the logic platform then basically sends the information back to vapi and vapi tells that to the caller all of that happens in a blink of an eye directly inside of the conversation with a user if someone asks hey are you available tomorrow at 9:00 p.m. or is any agent available at 9:00 p.m. then this whole process runs through in a couple of seconds or milliseconds even until the caller gets an answer which says something like yeah Alex is available tomorrow at 9:00 p.m. does it work for you this is incredibly powerful especially since you don't need to touch the prompt all the time if you usually create a prompt in the beginning you can make that completely Dynamic by using those functions even without adding those information right inside of the prompt because otherwise imagine you would need to update the prompt all the time based on the availability of your assistants or of your agents and in that case you don't need to because you have the function that can take that life data inside of the call and actually use it and give it back to the user or the caller and with that out of the way let's dive right into it the first thing I'm going to explain you is obviously how you can create a functions and once I'm going to show you that inside of a static assistant so how you can create a function within a static assistant inside of the VAP dashboard and secondly how you can create a function inside of a dynamic transion based assistant so I'd say let's dive right into it here we are on the VAP website and what you need to do is obviously have an account so I'm already logged in you simply click on dashboard you end up inside of their dashboard so obviously you would like to have a number or you would like to use web calling and the first thing we need to do here is creating an assistant so it already brings the pop up if you don't see that you simply click on create new assistant under the overview assistant Tab and I'm just going to use the customer support agent as an example I don't like the name Ava as it's always complex and the AI sometimes mistakes that I actually say AA or Eva or able I had literally all kinds of cases with that so we just name her Lisa I click on create Lisa and we just continue so once you have created the system you will see the screen like that you're probably familiar if you have worked with it before so let's just make it short you simply head over to the functions tab which is where you can create those custom functions so right within here you have a couple of settings and the bottom settings here they don't directly count as the functions that you basically create by yourself those are predefined functions that vapi adds to your prompt if you want that with the features that they allow you here so you have an end call function which basically is another custom function that they programmed for you so all you need to do is toggle that on which then allows your Bot to actually cut the call in case it sees the need for it so usually I don't advise using that if you do some call calling stuff because it just decreases the rate of actually having a successful end of a conversation so better just have a higher overall time for the conversation and just leave that function deactivated dial kead is another function that can basically dial numbers on the keypad while the assistant is on it that usually makes sense if you are calling hold calling a number that has a predefined menu where stuff has to be selected so you can use that to automate some cues that then can basically transfer the call to you once the queue is ending so you don't need to sit all the time by yourself in that queue and you have also something for a forwarding phone number and end call phrases so we are not going into into those because for us it's important to actually have custom functions so we're looking right into one right here so as you can see I can basically delete it but there's none anyways cuz it's just an empty one so let's for example say we take the same example that I showed you earlier with the real estate agent we would like to check the availability of an agent so you can define a function name which should usually be something that the AI can understand of what the purpose of that function is so to be very precise I would always call it something like view agent availability it is very clear you know that we want to look at the availability of an agent and as you can see here as well I basically wrote it all together with always the first letter of the next word in a capital because that is just an easy way to understand for the ey it is all written together and it is very easy to interpret so I always suggest to write in a similar style and inside the function you can be a little bit more precise of what that function should do and again this can be completely optimized for a prompt and I will try to keep that always as brief as possible to just keep the token length flow while the information is actually as accurate as possible so in that case I would just write something like checks the availability of an agent very precise in short so you can obviously extend that as well in case you would have multiple agents or whatever it needs to be inside of your functions because that also can depend on the parameters you send along so we can also say something like for a given time because obviously the user will probably ask for a specific time where he would like to schedule a meeting and we can even add here when scheduling a Mee meeting when scheduling a viewing so for the properties which is basically the variables that we are going to send along in the inside the custom functions so if I talk about variables or parameters you can just imagine of additional information that we are passing along inside of the request that can then be accessed from our from our logic platform as described inside of miror right here so those can be used to identify or determine what needs to be done after the information has been passed along so in our case we obviously want to send along the time and the date so all I do is I click property I add something like time date as a parameter name I also write the some same thing together and as a description we describe what we basically want so we say the time and and date of scheduling time of date of the preferred viewing so we can keep it as a string they don't support date yet but string is a perfect alternative because we can later on anyways use it inside of make.com for example to format that information right and now we need to click save very important otherwise it's not saving that whole thing and you also need to later on click add because otherwise it doesn't save your function so lastly we need to have a server URL which is basically the call B URL whenever that function is actually called inside of a call so let's say this Lisa AI assistant for the real estate agency is called and someone actually asks for scheduling a viewing in that case the function is called and it calls this server URL with specific information and I'm going to demonstrate how this looks by using a site called web hook. site where we can basically see response data from a response that is sent to a URL it is completely for free so you can simply go to web hook. site and create your very own Dynamic URL and just quickly going to do that right here so when you enter the first time it looks something like this you have a unique URL on which you can click to copy it which is the one that we are going to use as a call URL inside of our function I'll just paste it here you can send along a server URL token which usually should be something that is secret that you can keep for yourself so that you can check against that specific token if that is actually something that you sent or that maybe some Intruder sent cuz if you don't set any token and you don't validate it maybe anyone that has access to the URL that you would add in here could basically send the data to and you don't want that in my case I keep it simple and I just literally add it to 1 2 3 4 5 and all I'm going to do now is I simply click on ADD so as you can see we have our first function added yay and you can basically add multiple functions if you want to for that example to demonstrate it I'll just keep it simple and I'll just keep it at the view agent availability function and now I click on publish so this function is published and all I'm going to do is I literally just call the that number now let me just see what I set as a provider it set to Sarah okay perfect so now I'm basically just using the web calling to actually call my agent so we can see what response comes back once this information is called once this function is called for that we are doing two things number one is obviously we talk with Lisa so we start the call and then we are going to check within web hook. site what the information is that actually comes along so to do that let's just quickly click on talk with Lea and get the call going hello this is Ava how may I assist you today hey I'd like to schedule a viewing for tomorrow at 9:00 p.m. is there anyone available one moment hold on a sec it seems that there was an un yeah so obviously we didn't send anything back so this case Eva or Lisa basically didn't know what else to answer but you can see that the function call was successful and it seems like it actually called it twice with a 4 second difference might be something from testing I'm not exactly sure but as you can see it called the function and you can see that this respon or this message request is a from the type function call we can see the function call key right here and we can see that it calls the view agent availability function including a parameter called time date and then a time so the problem is right now that we obviously haven't set any kind of dynamic parameters inside of our assistant which you would usually not do inside of static assistance anyway with inv vapi so this time and date is basically just something that it came up with because we didn't give him any reference of the current time but obviously if you have seen my previous videos you know how to create Dynamic text directly inside of of vapi so in that case this would be obviously your actual and real time so this is the way how vapi basically does function calling and once you receive that information from within your logic platform that what EAS either is make zappier or your very own server or anything else you can use as a logic platform you can use that information and connect it to your CRM to your Google calendar to your c.com whatever it is that you would like to integrate with and check the availability and then send that information back to send the information back it is always sent back in a Json format with a key called response and you can also find more information about that inside of the documentation of papy by clicking on documentation and within here you can scroll down to server URL and you can scroll down to server events and if you look here you will find something about function calling as seen here and if you scroll down a little bit more you can see that you can send over a result in Json format with the actual information that basically should be available to the assistant afterwards so in that case it said your email has been sent in our case and based on the little board or the little graph that I created here earlier or graphic you can send back yes no or you can basically send something back that is a bit more meaningful something like the assistant is available tomorrow at 7 but maybe not at 9:00 something like that right and you can also send the whole you can also send a Json string back inside of that result key in case you would would like to have more structured data that you would pass along that is another alternative and possibility obviously in my case I always try to keep it more conversational as it's just easier for the AI to understand all right but that's basically the whole Magic if it comes to creating static functions inside of AP so now for the dynamic way there is a lot you can check within the vapi documentation so if you would like to start from scratch from scratch which I don't recommend you can head over to the API reference scroll down here and click on Swagger and go through their whole Swagger doation to figure out how these function calls work so in case you don't want to go through the complete documentation there is another way that you can check at least like a very very basic version which is by just going through the assistant endpoints or you can also just head into the documentation within the API reference and just check other parameters or other end points right within here so something like G assistant inside the G assistant you can see inside of the response as well that they basically send along a couple of functions which give you a basic idea of how you can configure them directly inside of your assistant you can see here you have the name which we discussed earlier asnc is another value that you can set which basically means if the function is executed asynchronously which means the AI agent during the call doesn't wait for a response so it will never give anything back but what you usually want if you want to return something you want to set asnc to false so that your assistant or your agent in that call can actually retrieve information back which is exactly what you would like if you would check the availability then again we have the descriptions and we have the parameters that you can Define right within here inside of the properties key and you have another required key where you can Define which of those parameters are required lastly you can also Define the server URL and the server URL secret as you have seen it earlier too so this gives you the guidance but it's still not the complete function there is another way that is even simpler than that and doing that and how that works is by simply creating a static assistant and creating the function directly inside of the static assistant and then using a tool like Postman to retrieve that assistant from the database so you can just simply copy and paste it so if that sounds weird I'm going to explain it to you now because as you have seen earlier we basically created already a function inside of our Lisa assistant right here that is called VI agent availability so what we are doing now is we are using another tool called Postman which is also free which allows you to test apis and web books and right within here once youve created an account you can click on the plus up here inside of your workspace and you can create a new request we're going to keep it as a get request and now simply we're heading inside of the APA documentation of the vapy documentation we're going to get assistant and we are copying this your L right here this one we are going to paste inside of the request inside of the URL field of Postman we're going to remove the dynamic tag now we're going to head back into the assistant inside the dashboard and we are copying the ID of our assistant so this ID is basically just appended at the end of the URL and you can try it now already by clicking on send but we will run in an issue because we need to authenticate our request as you can see here missing authorization header to do that what we're going to do is we click on authorization we select the barer token authentication and once done all you need to do is head inside of vapi go to organization and you're going to copy the vapy API key right here this is the one that you are then going to paste right within this field and then you can again send the request to actually fetch that assistant so as you can see now we are getting back that whole assistant Json which is basically the Json ised version of that static assistant that you have created on VY so if you scroll down within here you can as well see the functions and you can see our view agent availability function completely filled out including all of the values that we set like server Eur secret as well as the server URL so if you're creating a dynamic assistant you can simply now copy that part of that function and you can basically paste this whole part inside of your Dynamic assistant without writing that whole thing by yourself so since we already inside of the functions I'd like to also share a little bit more details about the parameters that go into a function because that is something that seems a bit complex or maybe a little bit weird to some people that don't work with functions in the first place so for that I'm simply just extracting those functions from within here and I paste them into another tool that is called Json editor online it's again free website where you simply click on it and you will be able to create and edit Chason right on it so in that case I'm simply pasting those functions right within here I'm going to format them so we can manually edit them here manipulate them which again just helps me to explain to you what I'm going to show you now so as mentioned earlier we have parameters and you can see that parameters are defined in the type of object which again just tells open AI or whatever llm you use that you would like to get a Json back that can then be used within further processing so this is usually something that vapy requires so I would not change that in that case right now and I think you can actually only have it at object right now in the first place so this is usually something you can just leave you don't need to adjust anything and I'm just mentioning that because probably most people don't even know what that is for then the properties is where your actual variables or your actual parameters are and you can see that the key of each of those properties is the actual variable name so this is always very important because those variable names will then be used inside of the assistant to actually determine what information you want to use and within it you have basically two more details which is usually the most common ones which is obviously the type so in that case it's a string you can also send over an integer all of that is as well described inside the Swagger UI when you scroll down here and you go to the function parameters properties and I think right within here you can check inside the Json schema the type which is defined here so as you can see it's pretty complex but just to show you you can Define it as a string number integer Boolean array or object so if you use an array or object it would basically give you back a Json which is more complex and it's more likely that you get an arrow so I would always suggest to stick to the more common values if possible obviously in sumar you can take them as well but I usually recommend keeping it as simple as possible for the calls to just to reduce the arror rate so now heading back into my Json editor we have another value which is the description which again we have described earlier visually inside of our assistant right within the actual function parameter right here so if you want to extend it you can also do that inside of the Json editor and all you need to do is literally just copy that whole part with the field with a parameter name you add a comma at the end and you paste that whole thing again and now you can change that if you say you have something like the time and the date separate so you can even split those values or just Define them differently and the AI will understand that and put those informations properly just don't forget to adjust the description as well so if you're working with transient based assistants that do code calling or outbound calling you might also run into some other issues with function callings that they just don't work and maybe you have no idea why exactly they don't work or why your call is cut so if you want to debug those kind of cold calling agents that are outbound there are a couple of ways of doing that obviously if you use a tool like me.com you will be able to see inside of the response within me.com so in the basically Bubble at the top right of that element that you're basically trying to execute you will see more information about it I'm going to demonstrate that how that looks with the broken functions inside of Postman so I just quickly prepared another API call inside of Postman that basically allows me to create a outbound call with that agent that we just created obviously doesn't make sense cu the prompt is something completely different but I want to show you how it looks like when you get an actual Arrow message back cuz inside of the system that I created right here there is a mistake and this mistake basically causes the cold caller not to make the call and by clicking on send you will now see that it sends back a status 400 which means the call has not been made with the assistant because something is wrong and thanks to tools like Postman or with a response inside of make.com or any other no code or low code automation tools you will be able to understand what is actually Wrong by checking the payload inside of the body so as you can see here at the response from byy we get back a message that contains two values right within here and it says that inside assistant model functions zero and property Dame should not exist so what I did is I basically renamed the name inside of the function to Dame which then causes again the function to not work and as you can see here you basically just follow the path right here by checking assistant dot so assistant do model do functions if you scroll down here functions then zero which is the first element inside of the array and property which is basically the definition for the value then you can see that D name is defined right here and it should not exist so by just removing that this Arrow will be gone the other one will still be there but I'm going to keep it so I can actually show you that the arrow will disappear if I fix it after sending it you can see the arrow is gone and now we are only left with the last one that you can then fix as well so this is a very simple way of how you can debug transient based assistants that have issues with function lastly I would like to share with you a little goodie on how you can build better functions that are just optimized for specific use cases so that you can make sure those functions are called in the right way at the right time whenever you call this assistant or whenever the assistant basically sees the opportunity of actually using that functions in the first place so if I Define functions especially for outbound call cold calling agents where I can check availability where I can schedule a viewing where I can cancel that viewing as well in case the agent or the caller calls again there are a couple of things that you would consider so you usually you can Define three functions for that for checking the availability of an agent for scheduling the meeting for that agent as well as for canceling the meeting what I usually do is I try to combine them in a smart way where I can basically just reuse and leverage those features in the minimalist in the most minimal way possible and in that specific example I just gave you now I would for example combine the viewability and the schedule meeting functions because in the end your goal is anyways to schedule a meeting with a user right and it's better to schedule a meeting than not scheduling a meeting so what I would do is I would combine both of those functions and inside of the function call of scheduling a meeting I can also check if that agent is available and if it is not available I basically just tell the user and ask him for alternative times you're basically just able to get the information to the user in a more concise way while already booking the meeting based on the time the user requested and if it books a meeting he also has more pressure and probably most likely commits to it in the first place this is a little goodie that I figured out while actually implementing code calling for some of our customers so that again helped them to just book more meetings that they actually could take and there's another nice thing that we usually did which is nothing related to that but you can send them a reminders as well with SMS or with another call before their scheduled meeting out of the way I hope you now have a good foundation of understanding what functions are how you can leverage them how you can debug them and how you can just build better AI voice callers using functions in general and I really hope you enjoyed this video so far if you have any further questions or you'd like to see me cover anything else drop me a message down below and I'm very happy to help you out until next time thanks for watching

Channel: Jannis Moore | AI Automation

Views: 4,336

Keywords: vapi ai tutorial, tutorial, full ai caller tutorial, function calling via chatgpt api, function calling, vapi tutorial, function calling openai, function calling gpt, groq function calling, function calling groq, groc function calling, function calling via chatgpt, function calling chatgpt, full tutorial, automated cold calling, voice ai tutorial, function calling another function python, cold calling, ai voice clone tutorial, ai cold calling, cold calling ai

Id: rc4PlH9BRbA

Length: 22min 29sec (1349 seconds)

Published: Mon May 13 2024