How To Build a $5,000 AI Voice Assistant For FREE With Vapi

today I'm going to show you how we can build an AI Voice Assistant from start to finish that will blow your mind right now ai voice assistants are the hot new AI topic and they are honestly scary good they can do anything from onboard people answer questions to book meetings I'm going to show you my favorite way of How I build these and how you can get started hello guys I'm Mike and I'm here to help you make more money and save more time with all these a eye tools right now my agency focuses on making voice assistants much like the one you're going to see in this video so if you want a fully custom voice assistant to do anything you can dream of make sure to book a time with me down in the description below and we can talk about it there before I get into it let me show you just a sneak peek little demo of what we're going to be building in today's video Welcome to Mike's Diner this is Mary how can I help you today hey Mary do you guys accept reservations at likes Diner we don't accept reservations due to the Casual nature of the diner however our friendly staff will do their best to accommodate your party as quickly as possible is there anything else I can help you with today um yeah what time you guys open Mike's Diner is open seven days a week Monday to Friday from 6:00 a.m. to 9:00 p.m. and Saturday and Sunday from 7:00 a.m. to 10 p.m. p.m. is there anything else I can help you with today um no that'll be all okay thanks for choosing Mike's Diner have a good day awesome so enough talking let's get into building so the tool that we're using to build all of these crazy cool AI phone Bots is called VY and VY in my opinion is the state-of-the-art the best way to go about building foam Bots right now as of May 2nd 2024 they have a super clean dashboard in a really flexible building structure to allow you to build these voice Bots to do whatever you want you can give it a demo just by heading on over to the vapi website and clicking on this button right here and they also mentioned on their website they can handle a wide variety of tasks and then one of the big things that I commend vampy for working on is the latency if you've ever seen those demos online where it sounds like you're talking to a real person that's because the latency of the phone bot from when it gets transcribed by the AI is just so low and vy's number one priority is to get this time down as much as possible so it just sounds like a natural person when you're talking to it and what's amazing is you can get started for free I'll have a link down description below it's where you can follow along with this tutorial and let's head on over to the dashboard and get started with VY so once you're inside the VY dashboard all we're going to want to do is head on over to the create new assistant button right here and we can pick from some of the predefined template assistants they have for us I'm going to go with this appointment setter here and click on continue we can name her if we would like I'll keep it on Mary and click on create Mary and this will just set up a very quick dummy demo bot to kind of show you what these foone Bots can do so just to get started talking with Mary all we have to do is click on talk with Mary I'll click on allow to use my microphone in my browser hello this is Mary from Mary's Dental how can I assist you today uh hey Mary what time are you guys open hey there we're open from 8:00 a.m. to 5:00 p.m. Monday through Saturday closed on Sundays though cuz even dental offices need a day off am I right all right Mary that that's great all right we'll end the call for now but yeah you can see there's some fine tuning we need to do to make this a little bit better but you get the idea so how this works basically is we have a couple different tabs here in the first tab we have the model Tab and this is where we're going to have our prompt and our kind of AI framework of how this is all going to work we have our first message here which will just say this message exactly as it's spelled out every time someone connects with our phone poot and then below it we have our system impr prompt and this is going to be all of the guts and juicy bits of our AI phone bot which will explain how it functions how it responds to people and ultimately the workflow of how the user interacts with the bot we can pick from a ton of different models here for our bot but I'm just going to stick with open AI as I found it to be the most reliable and then also we can change it from GPT 3.5 or we can click on GPT 4 if we want personally I've had much better success with GPT 4 but 3.5 also works for simpler Voice Assistant you'll notice if we click on GPT 4 it changes the total cost per minute here of our buy and right here is where we can kind of see the different stacks of our bot to see the time and the cost of what it takes to run marry so you can see we have the VY fix cost which is just at 5 cents per minute and then we have the stt deam model which is the speech to text model then we have the vapy gbt for Turbo cluster and then we also have the 11 Labs turbo which is our text to speech so this stack right here allows to basically say something into the bot the speech DET text then transcribes that audio then that gets passed through the large language model which is something like open Ai and then the response gets passed through 11 labs and then spoken out to us from the phone bot and then we also have our latency here so this tells us how long it takes in between each response from the user it takes in order for the bot to respond so when the last word is said by me it'll take roughly 1,500 milliseconds for the bot to respond and you can see here there's like a little chart broken down of how long it takes based on an average milliseconds of time response and you'll notice as we play around with the different models here it changes how fast the bot responds so you're going to want to play around with this in order to get the right settings for your voice assistant next let's head on over to the transcriber tab here I like to keep it on deep gram I've had no issues with this and it's relatively low cost and then we head on over to the voice tab here and as I mentioned before this is running on 11 Labs which if you're not familiar is the state of of the art textas speech model that is out right now you can pick from all of the different voices here inside of 11 labs and you'll even be able to add your own custom voice which I'll show you inside the voice Library later and you'll be able to do that with adding The Voice ID manually then there's also some other audio settings like adding background noise which I really like you can change between the default and the office or just leave it off if you want you can select between the 11 Labs model here and also change the option parameters for the model I don't like to touch this stuff at all but you're welcome to change it if you want to fine-tune it a little bit more but you don't have to use 11 laps if you don't want there is a ton of other text to speech models here that you can choose from and play around with so I would recommend spending some time and playing around with this and just seeing what works for you then we head on over to the functions tab here and this is where things get really really fun I'm not going to dive too much into this video but basically functions allow you to call certain API web hooks inside of your Pham bot to do whatever you want it is actually so busted and I'm going to have a lot more videos kind of breaking down all this stuff in the future but basically you can name functions describe functions here and then add end points to send over the information that your Bot gets so you can do things like send text messages book meetings or literally anything it is actually wild and then you also have some predefined functions here which is for ending the call if something is said by the bot so in this case if we kind of scroll down a little bit more we have the end call phrases right here so if goodbye or talk to you soon is ever said within the voice bot it'll just just immediately hang up so it won't have to like linger there on like a dead phone call for instance we also have the dial keypad function here so if we want our assistant to type in maybe some numbers or something inside of our phone bot you can allow it here and turn on this function and we can also use the forwarding phone number function so in our prompt if we want we can say you know if user says X Y and Z forward it over to this phone number and you can just put in the phone number right in here and you'll be all ready to go and then lastly we head on over to the advanced dashboard and this is where we can just see a couple more fine-tuning options to get our voice bot exactly the way we want what I love is they have hipa compliance so if you're working with any kind of medical offices you can turn this on then also we have the option to enable recording here I like to keep this one on just so I can see the recording after the fact in my call logs and then there's also some more options down here that you can play around with like the silence timeout response delay the LM request delay number of words to interrupt in maximum duration these are all kind of self-explanatory and you can kind of fine-tune it so that if you're kind of waiting for someone to give maybe a longer response you can kind of edit it so it waits maybe a little bit longer than usual to capture that response also for messaging they have their own server URL so if you want to put in your own endpoint here you can do so to send over web hooks to your server and you can Define what's going to be sent over based on the client messages and server messages then also you can add a voicemail message so if you are doing outbound because yes VY does allow you to do outbound messaging you can add in your voicemail message right here and then lastly we have the end call message this is pretty similar to the functions over here and call phrases I like to keep them just the same but you can kind of play around with this and have a standardized way of saying goodbye if you would like that's kind of the guts of how VY Works before we get into building out a custom bot I want to kind of go over a couple more things that being the phone numbers here so the first way you can add in a phone number is by buying a number boy has a giant pool of numbers and it will give you a number for like two bucks I think it is and it makes it really easy cuz they host all the numbers right inside here so you can either buy a number or you can import a number you can choose either twilio or Vonage put in your number your Sid your off token and label it and you can import your number from twilio and also similar to Vonage same thing your API key secret and your number and you just import it right into vapi and then also we have the voice Library so this is where I said that we can make custom voices if we want to put them inside of our AI Voice Assistant we can search from all of the preloaded voices inside of 11 Labs we can even search for the other voices inside of some of the other text to speech models right here but what we would want to do in order to sync up our voice that we made with VY is first of all create a voice over on some like 11 Labs sync this with 11 labs and then you'll be able to see your voice come right up in here and you can take that code that's made for your custom voice and you can just pop it right into your assistance right into your voice add manually your manual ID here I copied the idea for Gupta but you would just copy in the ID for your trained voice based on your synced 11 Labs account and then lastly I want to go over the call log so this is a super super super powerful feature that I am so glad that VY has a backend for and that is basically anytime someone calls they record everything they record the audio they record the transcript they give you a summary they give you a total cost breakdown it tells you every kind of function that was ran and even how long the call was ran for so you can see here we have a call that I just made at 4:57 p.m. and we can play the recording right here hello this is Mary from Mary's Dental how can I assist you today uh hey Mary what time are you guys open right so that was me from earlier and we can also see the logs if there is any the summary of the conversation and then also the exact transcript of what we were talking about in our conversation which is just mental there is so many different ways you can take this it is actually insane and then also on the right here we have some more information about our call things like the ID the total cost breakdown the reason why the call was ended in this case I ended it so customer call ended any metadata the assistant that was used who it was calling to the type when it started and when it ended and you can really really get super particular about some of the data that gets shown based on your phone bot usage and now let me show you some tips and tricks on how you can build a $5,000 AI Voice Assistant that you can actually sell to local businesses I'm going to head back over to the assistance tab here and then I'm going to select create new assistant and I'm just going to start with a blank template and for this example we're going to make this a restaurant assistant so I'm going to name this Mike's Diner assistant and we'll click on create Mike's Diner assistant and once we're inside our assistant we're going to want to start on off with a first message so I'm going to have a link down description below to where you can join my free school community and there will be an exact notepad template of this exact first message and first prompt that you can download to just pop inside of the system and get started playing with VY with a real life demo of one of these voice assistants so if you want to follow along and not have to type all this out yourself you can join the school down download that information and then just pop it right into Vy but I'm going to start off with this first line here and that's welcome to Mike's Diner with two exclamation points two periods a comma this is Mary how can I help you today so this is kind of a edit of the original Mary vot that we made in our first example here and I want to explain with this first message here is that we kind of have to add in some different exclamation points and periods here to kind of correct the Cadence and the natural tone of how Mary is going to speak to us over the phone this is using 11 labs and if you're familiar with using 11 Labs you'll know sometimes it just kind of run through things really fast and that's not what you want at all you really want a slowed control conversation that sounds natural and normal which is why we add in these periods here with some of the exclamation points to make this sound as normal as possible so I'll slap this in and let's give it a listen welcome to Mike's Diner this is Mary how can I help you today right but obviously you can do some fine tuning to get this to work exactly how how you want and now let's move on to the system prompt here so first what I'm going to do is start off by explaining to The Prompt exactly what this bot is so I'm going to say you're merry a helpful assistant for Mike steer you're polite and sweet and straightforward you start by asking the user how can you help and use the Q&A below to answer any questions they may have about Mike Steiner be concise brief and informative so we're just explaining to the bot exactly what it's supposed to do and then as soon as we're done we're going to follow up with this line here we're going to say after you've answer the question make sure to respond with is there anything else I can help you with today and this will make sure that as soon as they're done with this line and the bot response with an answer it follows up with the question I kind of put this in front of the entire rest of the prompt here just so it has that context when it goes into querying the entire prompt and now we can add in some information about Mike's Diner so I just used Claude to come up with this quick sheet here of some questions and answers about yours truly's new Diner where is Mike Steiner's located what's the operating hours what kind of quisine does Mike Steiner serve and right here we have a really solid start on an AI voice assistant so all we can do is just make sure to publish and click on talk welcome to Mike's Diner this is Mary how can I help you today hey Mary where is Mike's Diner located Mike's Diner is located at 123 Main Street in the heart of downtown Millersville a Charming small town in Upstate New York is there anything else I can help you with today right you get the idea so you'll see it actually responds with the exact message that was put inside of our Q&A sheet right here so you're going to want to fine-tune and tweak this sheet so the bot will just say whatever kind of answer is associated with the question but let's say for instance they ask a question that's out of the scope of what's inside of the Q&A what I like to do is add in a line like this and that's going to be if you cannot find the answer to the question then say I'm sorry but I do not have enough information to answer that is there anything else I can help you with today so now if it runs through the chat bot and it doesn't know what to say then it will just say this so it kind of gives it like a backup because I've had sometimes where people call this thing and they just ask wacko questions so it's like a kind of a safeguard so it doesn't just go off the rails and not know what to do and then we can add in some final wrap up props to get this to stay on task and then close out the call if everything is good after you've answered their question then say is there anything else I can help you with today respond to their question if they have another one repeat this until you have answered all questions and then afterward if they say no say okay thanks for choosing Mike steer have a good day and hang up now we have our full prompt let's give this a try I'm going to click on publish and talk with Mike's assistant welcome to Mike's Diner this is Mary how can I help you today hey Mary um what time you guys open Mike's Diner is open 7 days a week Monday to Friday from 6:00 a.m. to to 900 p.m. and Saturday and Sunday from 7:00 a.m. to 10:00 p.m. is there anything else I can help you with today nope that's all okay thanks for choosing Mike's Diner have a good day so you'll notice it said the exact phrase when I said no I'm done with all my questions but it didn't hang up so let's fix that we're going to end the call here and we're going to just copy over this have a good day text right here head on over to functions and just slap in the end call frame and allow the end call function so now when it says have a good day it'll end the call same thing too what I like to do is at advaned just say for the end call message just have a good day and publish so let's publish this and give it one more try welcome to Mike's Diner this is Mary how can I help you today hey Mary do you have a liquor menu Mike's Diner does not have a liquor menu but we do offer a selection of bottled beers and wines to compliment your meal is there anything else I can help you with today nope that's all okay thanks for choosing Mike's Diner have a good day and there you go you see it ended the call without me having to do anything so that right there just gives us some more realism so it acts like a normal person and hangs up the call when the user is done with their request and that's really the bulk of it after this you've made your first real simple AI Voice Assistant all using VY the biggest thing I want to stress here is this prompt right here is every everything you're going to want to play around with things like the model here the voices and also the functions if you would like in order to fine-tune this and get it to work the exact way you want I can make a 3-hour long video going into all this stuff but I'm not going to have enough time to cover it here so if you want join my school Community where I'm going to be sharing a lot more tips hacks and secrets about how you can make these voice assistants as crazy good as possible I haven't even touched the functions yet also like I mentioned if you're interested in building a phone bot and you want to work with me make sure to book a free time down description below and I can help you build out any kind of AI phone bot you like and when I mean any kind of phone bot I really mean any kind of phone bot it could be anything from the medical industry to like a simple customer service bot like the one you're seeing now there's even ways we can turn this into an outbound phone bot to do things like sales call onboard people and even recruit and one last thing I want to show you before I go you can head on over to hrzn labs.com and you can even try out some of the demos that I have right here like a front desk recruiting and even this customer support agent right here and also all of my voice assistants come with a back-end dashboard like the one you're seeing now and you can pick a time right here so we can get started on your voice assistant you'll notice I have a really simple dashboard here that I built out which is very similar to VY dashboard except you can see things like the total amount of calls the average call duration and even stuff like the call logs right here so you can see all of the past calls and even things like the number the date the duration the call type the ended reason any functions that were called called and also the transcript of the entire call once again if you're interested in working with me and any kind of AI voice assistant make sure to book a time down below also if you want to get access to all the resources I used in this video like the prompt and the starter message and get that text file in my school Community also down description below there's going to be a lot more content coming out with the AI automation Mastery Program which is going to include everything from scraping to phone Bots to chatbots and all of that in between there's a lot of great cool stuff coming and I'm excited to be at the very beginning of this wave you got any value from this video whatsoever make sure to give a fat like on this video it really really really does help me out and also leave a comment what kinds of phone Bots are you going to be building I would love to know your answers down in the comments but if you've made a great phone bot and you have no one to sell it to you can check out this video here where I show you how to get Google Maps data for free which will allow you to reach out to local businesses to sell these phone Bots too it's a great video it's free data so I would recommend going and giving that video a watch if you haven't already so I'll see you guys over there

Channel: Mike Powers

Views: 4,594

Keywords:

Id: OH8WvJC0J00

Length: 20min 36sec (1236 seconds)

Published: Fri May 03 2024