Getting Accurate AI Answers in Voiceflow

you're noticing that your AI answers just aren't good even though you've got the information in your knowledge base in fact sometimes they're just flat out wrong now you don't need magic to fix this the key actually lies in how you structure your data so I'm going to show you how to fix that in four easy steps but before we do that YouTube says that 78% of the people who watch this video aren't subscribed so if you want to know how to build the most advanced AI agents in voice flow hit subscribe below now let's dive into what you're likely doing wrong in a couple steps on how you can start to fix it so here we've got our voice assistant and I've gone ahead and loaded in a ton of data sources from our website to create our bot but if I go ahead and ask it some simple questions like how much does voice L cost it gives me an answer but this answer is actually wrong and if I really look into it it's because it's coming from some outdated blog post in January like 2 years ago if I asked another question like what are the steps in voice flow so you would expect an answer like the button step the text step all the steps you use to design your flow or your assistant and you can see here that it doesn't actually answer that either it gives me some something about user testing and drafting the design this isn't what I'm looking for now these answers aren't good but there are a couple ways that I can go ahead and improve them and rethink about how I'm structuring the data that's going to be informing this assistant the first one is understanding what types of information I want to have and where I'm going to grab those from now I want this to be a baller website so users are going to come here they're going to ask questions I'm going to eventually capture their emails and try to get demos and leads but the first thing I want to do is I want to make sure that I've got my core bases covered of what people are going to ask so I know that people are going to ask about pricing this is a big one that I want to make sure I have up toate information on so let's go pricing the second one is going to be about case studies and use cases right I know that some of the common questions people ask me are what can vool do how much support can vool automate what is the type of AI agent that I can build on voice flow so I want to add in some upto-date case studies and maybe some relevant blog posts about some of the most recent features that we've released now the third one is probably going to be the biggest bucket which is actually going to going to be our support documentation a ton of users come to our vo website and they ask questions about how vo works now we want to make sure that they've got a really good source information to get this from so in that case I'm going to write support docs now for each of these buckets I want to make sure that I've got upto-date information that's being loaded into my assistant so for my support documentation I'm actually going to go ahead and grab these specifically from zendesk I may also add our developer documentation in here this ensures that the information that people are going to be asking about is as up to date as our docs are so I'm going to say import from zendesk now upto-date case studies is going to be a bit more manual I want to understand and look through our blog and figure out what are the specific types of blog posts that I want to include in our assistant to make sure that users have the most up-to-dated information one of the biggest hazards with importing your whole website is that you have a ton of content out there that is out of date but the AI has no idea what's out of date or what isn't and so that's up to you to decide what goes into there to kind of form the brain or the base of their knowledge and the last one here is pricing now pricing is super sensitive right you don't want to give out wrong prices you don't want to give people R this information so I want to be really careful and make sure that I've got up toate information and that I only have one document around pricing so that I can go ahead and update it if the pricing ever changes or that the informational pricing I know is only going to come from one unique source that I have a lot of control my recommendation here is that when you're building your assistant for the first time you start off with a few pieces of information so in this case I'm going to just start off with pricing and then I'm going to move to the case studies and it's for documentation when you start off with the smaller number of documents in your assistant it's easier to debug and figure out how to get the optimal responses before you go ahead and add more documentation and you get lost in a c of information now we're going to go ahead and start adding the most important piece of information first which is pricing and now this goes into the second rule which is to make sure that your documents are formatted appropriately and I'm going to show you what that means so let's get rid of all of this other information in our knowledge space here so we can start fresh great now I'm going to start off with pricing so I want to add in information from our pricing page this is going to be the most upto-date source of voice s information and so you can see here on our pricing page we've got our actual prices which is 50 bucks a month for our Pro Plan 500 for our teams plan and more for our Enterprise but the first problem here is that this information isn't actually organized well for an AI to understand now for a human it's easy because I can understand that this is in a kind of form that these icons mean something but a aren't as good as doing that on top of that the other thing is that the web scraper inside of voice flow is actually built for text documents so it's not going to be able to recognize that on this web page these UI elements actually indicate different parts it's just going to scrape all the text from this page and jumble it together so if I go ahead and try to add this page directly to the knowledge base this isn't going to go that well so let's add data source URL and import this so one way you can check is that when you actually upload a document to the knowledge base if you click on not the URL but this white part here it'll open up a side panel and you can see how voice flow has chunk that information so right off the bat you can see that I've got a bunch of URLs here this doesn't really make much sense I'm not seeing any of the columns and that's because a lot of the information on this page was actually in images so they didn't get scraped by the web scrier so by now try to go and ask something like how much does voice will cost it gives me an answer but it's not actually what I want right it says editors have the ability to make changes voice has different pricing plans check the website that's not why I'm asking your assistant I want the actual answer and if I check the sources here it's grabbing it from the pricing page but because that document doesn't actually have any well properly formatted information in it that the AI can access it's not able to actually answer it so because this is really sensitive and I want to make sure this is right this is where I can add a different data source with more information that's structured properly so let's go ahead and get rid of this and instead let's add a data source that's bit different so in terms of the order of control that you have so URLs and site map have the leas control this is using a web scraper to actually scrape the page for text and then it tries to put it into the best way it can for formatting and uploads that to voice flow an integration is pretty solid if you're using zenes this is a great one this is actually going to pull the documents from zenes and because these are text documents it's going to be formatted well the next one are uploaded file and pl Tex now this is how you're going to get the most control out of your assistant because when you upload a PDF you know that the documents are going to be structured properly but really plain text is what I'm going to go for here because this is going to allow me to inject q&as or specific pieces of information into my knowledge base that I can then go and modify later so what I'm going to do here now is I'm actually going to paste in the information I have about pricing now if you have a document that's we better you know I have just an example here but ideally you have a document with a ton of detail about these different PL plans the prices any other information you want users to know that can be fed to the AI and so I'm just going to put in a quick sample here but voice pricing is as follows Pro Plan teams plan Enterprise plan who's it for ETC so now if I go ahead and import this you can see that when this is done and I click it that the information in the chunk is actually great uh this is exactly what I put in here I have a lot of control over this and this is the thing with adding text is that you can actually go ahead and modify this afterwards but this is pretty good and now I go ahead and start asking it questions like how much is voice flow it's going to be able to answer with a much higher degree of accuracy because it's pulling from that main source so you can see here was able to summarize all the different plans and if I ask you something like I'm a new agency with one person what plan should I get it's going to be able to interpret that and recommend a plan for the user so $50 a month so you can see here even though I don't have a lot of information I just got one piece of information but it's good high quality information my AI model is able to understand that interpret it and then respond that to user questions in an efficient and accurate manner so in the case of data sources with AI less is more and more is actually not that great you want to have less higher quality sources to be able to produce higher quality answers that are just as robust as what you would expect with the large dollar space so you want to make sure you spend a lot of time making sure that your most sensitive information is formatted correctly so now that we've got our pricing data source in there let's work through those other sources that we've got over here in our diagram so we've got our pricing now now let's added some update case studies and then some support documentation we're going to import it from zendesk so if I head over to the voice website I'm going to now go to our blog and I'm going to manually pick out the documents that I want to upload because again it's going to be important that I have a high level of control so for here I'm going to add a case study with Trilogy that automated 60% of their customer support with voice flow this has got a ton of really good information in it about how they use voice flow how many their tickets they actually automated and how they were actually able to use AI to increase the number of tickets they automated from 6 to 60% in just a few weeks so let's go ahead and add that in and for this one because we know what URL is going to come from we're going to add in the URL now let's grab two more let's test this out now and ask something like what can voice flow help me do and you can see here that it says voice will helps you automate customer support provide accurate answer to customer questions Etc can you give me some examples of customers and how they used voice great and it was able to actually really pull from up to date so it says Rome must start up in the flexible car industry automated theirselves Trilogy they saw 35% resolution from the first week so it's giving us a lot of really good in-depth information here now I want to get the third piece which is all of our support documentation so this one I'm actually going to import from zenes so we've got all our support documentation in zendesk and this is live up to date and there is a ton of it that can really help our users so now that we're know we're getting a base of really good information we're going to go ahead and add our support documentation in here to really make this a robust assistant so what I want to do is rather than scraping the website I'm actually going to use the integration now Integrations are always best right now we support zenes we're going to be adding more in the future but you can actually use knowledge based API to upload documents as well so if you have a Content system somewhere that's going to be much more accurate to be able to import it and I'll show you why so I've authenticated and let's add all our docs so we're going to add the voal docs we're going to add English we don't have any user segments that's fine and we'll add our guides and our documentation so this is going to add 136 data sources here so now you can see that it's starting to add a ton of information in here but because I adding these in pieces as I'm testing it out I have a really good sense that I know my Base information is good and now it's about to sendes ones so if I go ahead and click on something you can see that the chunks are actually pretty good so if I go ahead and now ask it something like what are all the steps in voice flow this gives me a much better answer than what I originally got this is a great um answer and I'm pretty satisfied now knowing that my voice assistant can ask answer any question about voice flow it can also answer case studies and business information as well as a really sensitive information about pricing now we're going to get into Point number three which is what you can start doing after you've filled out the base of your assistant which is to start injecting frequently asked questions into your knowledge base sometimes there are questions you get that you want to give a specific answer to and you want this to be high quality similar to pricing but it may be related to other services refunds things like that that you know are pretty sensitive pieces of information that you want to have the right answers to so let's say I've got a question like does voice process refunds so let's see what it says right now and it says yes voo does process refunds customer support a refund by emailing support ato.com and if I check here it looks like this is mentioned in some of my help articles but let's say I wanted to provide more information to this so I'm going to inject an FAQ into the database and the way it works is when you actually ask a question to the voice of knowledge base it's looking through all of these different chunks and it's returning what it thinks is the most relevant so you saw in that last example when I asked the question about refunds it returned three chunks one of them had a slight mention of refunds the other ones didn't but I thought it might be relevant so what I'm going to do is I'm going to add a data source and I'm just going to add plain text now we're going to have an entire built out solution for FAQs uh coming out in the near future so if you're watching this a couple months from releasee and we've got a whole new section here about FAQ you canor this but if you're not this is also a great way to kind of add in or inject manual overrides into your knowledge space so I'm going to ask put a question here so I'm going to say question or maybe Q is does voice flow offer refunds does voice flow give out refunds and now I'm just going to add in my own answer voice FL only processes refunds in specific cases please reach out to support f.com if you require a refund great so it's import this in here and so now I've got something a bit more specific so I now ask again does voice flow offer refunds you'll notice that now it says voice only processes refunds in specific cases please reach out to voice.com if you require refund and if I look at the chunks that's because the first one it's Serv for saying that's the most important is the Q&A that I added in there because I added a piece of information to the knowledge piece that's highly relevant to this topic area it's going to service this first and it's most likely just going to use the information from here and it'll look at the other ones just to see if there's anything that it might be related to so you can see it gave my thing about pricing here and it has this document here about downgrading and can canellation so this is how you can inject an FAQ into your knowledge base to have it have a higher degree of answering accurately for that question now we're getting into tip number four which is really about modifying your knowledge based settings so you've gotten hints at how the knowledge base works here about retrieving and providing chunks and you find more information into the details of how Vector databases work in general in the links Below in our YouTube channel but let's go into some set settings that but let's go into some settings that you can tweak to try and get better or more accurate responses within your voice on knowledge base you have these settings here these are the global settings that are kind of like the foundation of your assistant now if you open these up there's a couple different things so there's the AI model so we've got a bunch here we're adding way more in the future we've got the temperature so this is going to control the randomness of the answer that the large language model provides if you're okay with your answers being a little bit more flexible you can increase the temperature here if you want them to be pretty standard you know someone ask the same question three times you want the same answer and the same way you can reduce this down to be very deterministic I'm okay I want to be a bit deterministic so I'm going to have it be 0.1 Max tokens are also important this is the max output tokens so this is the number of tokens that can be used to generate a single response so if I have this at 128 that's going to kind of snit my answers at 128 but it'll also be a bit shorter just to be safe I'm going to increase this to maybe 200 just to make sure that the AI has enough room to be able to communicate an answer now the last one here that's very important is the chunk limit so if you remember when we asked a question there are certain sources that are passed back to the knowledge base these chunks on the side here this is going to dictate how many chunks it should pass to the large language model to determine an answer from the more chunks you have the more information the large langle has to work with but the more chance that it might actually mix up conflicting pieces of information and give the user a wrong answer so a large number of chunks works really well when you have an knowledge base that you know is really consistent and high quality you're going to pass information over and the large language model is going to look at all eight of these different chunks of information and assemble an answer now the more chunks you use the more tokens it's going to cost you can imagine that 10 chunks is like 10 of these gigantic pieces here being sent to the AI it's going to use a lot of tokens may not always be best on the other hand if you use one chunk so you lower it down the AI model will only have one piece one chunk to to go from and so if you have relative information in a couple pieces it's not going to see that you'll just see one but on the good side here it's only seeing the most relevant chunk so this is a good one if you want to make sure that your answers are very highly accurate you may have more cases where an answer isn't found because the answer wasn't necessarily in the first chunk that was past but for the most part you're going to have much more accuracy on your answers and so that's what this chunk limit is the default here is three I like to keep it at three or two it's a good balance between accuracy and the number of tokens the last thing here system prompt this is pass the AI model as more of like a general set of instructions so this is great to enter a Persona and even things like keep your answers brief or answer the questions as detailed as possible and provide a lot of information so those are the four steps let's go ahead and wrap this up and actually try out some of those original queries on our new and improved voice know face spot how much is voice flow and you're going to see that we've got answers that are much more accurate and up to date what are all the steps great and done now you have a much better idea of how to use a knowledge based in voice loow and how to get better AI Answers by better structuring your data that's it my name is Daniel I'm the head of growth that voice flow and if you have any other questions feel free to hop into our Discord leave question in the comments or hit us up on any of our socials if not we'll see you in Discord and on voice flow and happy building

Channel: Voiceflow

Views: 1,561

Keywords: Voiceflow, Prototype voice apps, create chatbots, chatbot builder, ai, artificial intelligence, machine learning, webchat, automation, assistants, agents, NLU, GPT, ChatGPT, design, Design chatbots, Production, AI automation agency

Id: iQJ9ecSwHxg

Length: 17min 46sec (1066 seconds)

Published: Mon Mar 11 2024