Using Voiceflow with Mistral AI or Open Sourced AI Model

there are a ton of new open- source AI models coming out that are better faster cheaper or more specialized than GPT a great example is mistol mistol came out pretty recently it's an open source model and is much cheaper and almost just as accurate as GPT and so a number of tools like perplexity are actually starting to build on top of mistol so in this video I'm going to walk you through how you can actually use vo's knowledge base as a vector database with a thirdparty large language model like mistro so for all of you developers out there people are experimenting and don't want to spin up your own Vector database this is where you can use voice L to save a bunch of time and connect it with any third party models that you have whether that's mroll another open source model or perhaps it's a fine tune model on gpd's platform itself or a locally hosted one heads up this next section is intended for developers or people with a little bit of understanding of JavaScript it's a more advanced application of using voice flow and a way that you can use it to speed up your development of custom applications now vo flow actually has its own built-in Vector database so here I am in my voice project so I've got my actual project over here and you can click into our new CMS and in the knowledge section you can go ahead and add your data sources so I'm going to go ahead and upload a bunch of URLs here and this is going to import them and actually start vectorizing them and turning them down into chunks that I can now access via the voice flow API so you can see here that it's processing and uh parsing all the documents in chunks and ductors and once is done actually scraping the web pages and you can see some of them are done over here you can see all the different chunks that it was able to pull out from a document so now that this is all done we can run a quick test in here to see how it performs so how do I build an AI assistant and now this is going to search through the knowledge base find the chunks that are relevant and then actually synthesize it with a large language model so you've got a really good answer here and if I check below I've got my three core sources and you can see that these chunks are being passed to gpt3 Turbo to create an example response now one really interesting with thing with voice flow is that whenever we design something so just like the knowledge base you can actually access all the apis to make this possible and that's what we're going to use to be able to use this with a custom large language model so what we'll be able to do is bypass the models that are available here and we'll be able to use our own model at no additional cost and to do that what we're going to do is we're going to use the voice flow knowledge based API to send a question to our knowledge base and return chunks then we're going to take those chunks save them and actually send send them to mistol or a third party model to return an answer first I need to go ahead and grab my API key that I can get from our knowledge based do here and open our documentation if I go ahead and drop in my API key here I can actually test out this API I need to add a question here for this API to work so I'm going to say what is a vector database and you're going to be able to see what's happening behind the scenes here awesome so here is our response and so in the response you can see that a couple things are actually returned the first one is is that different chunks are returned so these are the chunks that we saw so it returned two different chunks and that's just because we've got a limit here so let's go ahead and make this maybe five so let's try it again now great so now we got five different chunks that were returned and the first thing you see here is score so this is the similarity score so if you remember in that 3D map this is how close this chunk is to the question that we ask where the vectors are and so the model determin is about 75% similar or 75 and so we've got five of them so the first one here has the actual chunk ID the document it came from and then the actual content which is the chunk itself and remember the better your document so the like the more structured the text is whatever it might be the better the chunks are going to be and so it's got the content here the chunk The Source the URL the name Etc and it does that about five times for five different ones and it looks like it found some pretty similar chunks now the next thing is that you can see it actually did come up with an answer because in this model it's actually sending it to an AI model to answer and the model is cloud B1 and so it did use tokens and it came up with an answer to say you know Vector database is this now one really cool thing about the voice API is that you can actually turn this AI synthesis off so if you see here synthesis let's turn this to false and you can see it's updated in the body here and if I try it again now you notice that it just Returns the chunks but it uses no tokens and this is because now you're just using the voice on knowledge base like a vector database you're putting this query and you're getting these chunks now this is exactly what I want if I want to use a third party model like mistol like we're about to do because what I can do is I can just take the chunks here and then I can use an API call to pass them to the model that I want I can even do some more complex stuff so for example I can see here that the confidence is pretty low so I can set a filter to say only accept a confidence above 75 or similar a similarity score above 75 and that way I can search through a bunch of different documents and then get the best chunks that are the most relevant so if I ask something like how do I build an AI assistant you'll see that some of the chunks we have have a much higher similarity score of 087 and this is because we actually have more documents related to how to build an AI assistant in the URLs that I uploaded then Deep dive documents on what a vector databas is and so this is great for being able to just induce a base similarity score to make sure that whatever answer you're going to give the user is accurate or at least is reflective of the question that they're asking so now we're going to head back into voice flow we're going to look at some of the more some of the developer tools we have to be able to work with this API and actually pull all this information so back in voil here there are two ways that I can leverage this API the first one is an easier way with something called the API step so that allows you to make an API call but something we introduced more recently that's better for tasks like this that may involve a bit more complexity is a feature called functions so functions is in the CMS here and what it does is it actually allows you to make fetch calls to an API and then write JavaScript as well to transform them so if I started a new function here I call this example we can see what it looks like so it gives me an actual code environment here to start writing my coded and I'm able to Define input and output variable now input variables are what I'm going to get from my project into the function and the output variables are what the function is going to create and so inoo here you can see I've actually created two functions already and so you can see here that I've got some input variables which is the API key for my voice project and then the question and then I've got some output variables which are the chunks uh and an error message which I'm then going to pass into my next function which is actually going to send all this to mistol so I'm sending in a chunks in an API key and then asking a question so let me show you what I've built here for the get knowledge based junkins now you can also download both of these functions uh I'll link to it in the description just to save you some time and so if you want you can import them via this import button here but I'm going to walk you through the code that I've set up just so you can understand what's going on here so in the get knowledge based chunks code I'm mainly doing two things the first one I'm doing is using the fetch request to call an API so to call the knowledge based API that we just looked through and actually get the chunks so just to walk you through each part of this the first thing I'm doing here is grabbing my input variables from the canvas and so I've got my question that a user asked in my vo API key the next is I'm just defining the U the API data so I've got my endpoint here which query end point and I've got my body which is a chunk limit of five synthesis is false and then the settings actually don't matter because I'm not really using these because I'm just returning chunks and here I've got my question the next part here is I'm just actually making the API call so loading in the API key the URL and the data that I just defined up here and I'm just putting in this try catch block so there's an error I can catch it I check for a couple errors down here and in this line I am saving the response from the API into a variable called response body and then I'm going to start parsing it so here I pull out the chunks and so if you go back to our other example you can see that in this API response inside of inside of chunks is where our chunks are so I'm just pulling that out because I don't really care about this other information and once I've got the chunks I'm filtering them based on a score so I want to make sure the score is above 0.8 and then I'm also just removing any other information so the score the chunk ID the document ID tags all the rest we can get rid of them I just want the information and then the sources that's it then finally I'm just passing that out into a variable called clean chunks and that's right over here so if I go ahead and test this out and the way I can do this is by actually dragging out this function on the canvas like this I can select get knowledge based chunks now I can start building the rest of my project so let's do hi what can I help you with we're link that up then we'll use the capture step because I want to capture a question so we're going to capture the entire user reply and we're going to capture it to a variable called question so let's just create a new variable in here called question right now whatever user says will be captured in here we're going to take our functions block that I just created connect these up and now we're just going to map these together so for question I'm going to use the variable question that I just created and then for the API key we need to grab our voice slow API key which was in our settings here and I'll just go ahead and actually put this up top here so VF API key and this way it's a bit easier for me to create so I'll just say VF API and we'll just put it in here then we can map this variable together and that way it's a bit easier for me to keep track of now outside of this the function is going to process clean chunks and an error message so I'm just going to save these in a variable called chunks that I create and the error message will go into this error one and now we should be good to go so what I'll do is just print this out to the canvas here you can actually see what it's returning so let's run a quick test here hit play hi what can I help you with how do I build an AI assistant now it's going to send this to the knowledge base you can see that it actually grabbed all the chunks here so I've got a debug message that's actually showing me everything is grabbing from the URL and you can see here that I just printed out the chunks to my step block here right so it's got a ton of info this is the content name this is the actual chunk information and it does this a couple times cuz it's got two or three chunks it looks like so now that we've got this now the next thing we want to do is we want to actually go ahead and pass this to mistol and mistol is an open- Source large language model that is a lot cheaper than GPT it is just as accurate if not it's it's pretty good I would say it is a bit slower but because it's way cheaper maybe it's something you want to use and so to do that I can now do two things right I can use the API step to make an API call to mistal or I can build a function for it but let's go ahead and see where I can access this and to do this I'm actually using a tool called together together is really cool because it has a bunch of open- source models that you can access and play around with so I use this just for testing out but here we've got uh mistol 87 8X 7B we've also got a bunch of ones other ones that are built uh and they just hosted you can use our API and it's a lot cheaper than should be use so this is one that I like playing around with but uh feel free to use your own if you have a fine tune model let's say on on open AI platform or a fine tune model hosted somewhere else you can also use those I'm just going to use this endpoint because it's a bit easier so looking at the together documentation the actual API call for this model that I'm looking at mol is pretty simple so it's you've got your url here API key content header and then basically you're just saying here's your model and here is the question and this looks like it is a system prompt for the model that's pretty much it just a post request so if I go back into boy now and I actually build this out and go back into we could use the API step to do this and just drag in the API endpoint the body headers but I wanted use functions again for this just because it's a bit easier to share so I'll go back into functions and I've got one built out here called 7B and I'll just walk you through each parts of this again the input variables we're taking from the canvas are the chunks so this is what we got from the last function we've got a myal API key that I need to load in and then the question seing those into variables in my function here because I'm going to use same thing as last time I've got the URL I've got the body for the URL that I'm going to put in here and you can see that the way I've organized this is I've got my system prompt that says answer the question using only the information provided and then in the amount sent to and then in the value sent to mistol I've got question is question and then information is Chunks so I'm going to pass the question and the chunks to mistol and it's going to come back with a response this is where I make the API call so again is just uh right here so pretty straightforward it's a post request just checking for some errors and then I am taking the answer so this API comes back with a couple of different things uh in there is the answer so I'm just pulling that out into the answer variable and then now I am pulling it into an answer variable and putting that back out to the canvas so pretty straightforward if you want to learn more about functions just hit the question mark here and that'll take you to the function stocks and this will explain everything you need to know about actually creating a function we're also going to have a landing page coming out soon with examples of different functions that you can use and there's also examples of functions within here that use different calls so now that I've created this I just added a little image and a description and if I'm going to go ahead and pull this on canvas it looks like this so in Dev I'm going to function step again I'm going to select a function mral 7B let's connect this together and now let's map these up so chunks is going to be the chunks of variable that I'm saving from this function myal API key I'm going to add it right here and then the question is going to be the question that the user asked so for the API I'm going to do the same thing I'm just going to use my m one and just cut this part out so I've got a Mr API key now loaded in and I can go ahead and test this out let's map my output error variable so answer to answer and I'll add my text here and say let's just pick the answer awesome so connecting this up together so now I've got a full flow where it's uh taking my question it's hitting the voice of knowledge base it's grabbing the chunks then sending those chunks with my question over to mistol and then it's answering it in the tool so let's go ahead and run a test and see if this works oh looks like it ended in an error so I got the error path here I do have debug messages on so let's just go turn on debug and see so you can see here that right off the bat it looks like the first one worked because I got all my chunks so chunks received uh and I was able to retrieve all of my chunks and then it looked like the function step the next function step here is missing an input value required for API key so it looks like I forgot to map these up together but super helpful when it comes to debugging so I'm going to go ahead and add in the myal API key and map it to my function variable and now let's run this one again taking a bit longer so no one's going to M this time and there we go so a little cluster here on the Prototype tool but when you share this it looks totally fine so to build an a assist follows these steps BL and that's pretty awesome so let's go ahead and hit publish and I want to see what it looks like on an actual End chat boot so I published it let's grab my widget code now you can go ahead and embed this onto your website I'm going to just render this on a web page here just as an example by pasting it into the console and it pops up here so it's rendered on the page here let's go ahead and start a new chat hi what can I help you with how do I build an AI assistant and great so you can see here that it was able to actually respond it looked properly formatted and now I've got a custom bot running on mistol an open source model that I was able to customize so that's pretty much it for the explanation and so that's an explanation of how a vector database Works how you can use voice the DOR Vector database and now you can go ahead and actually send these chunks to whatever thirdparty model that you want and build them back into vo so and actually come up with an answer so this is a great way if you're experimenting with open source models cheaper models faster models Niche models that you may not have in the tool if you want to obviously do this all inone Bo has a response AI step where you can just use one of the models that you can use through voice flow so Claud GP Etc but this is really useful if you want to build more complex apps using your kind of backend systems or fine-tune models you can do that in voice flow and just a reminder if you want to grab these functions I'm just click the link in the description you'll be able to just download the Json file and then import them to save yourself some time but then yeah just make sure you can go through you can modify the code however you want and you can start using Voice Low with third party model today that's it if you found this helpful remember to like And subscribe and join our Discord in there you'll find a developer chat Channel with a ton of developer resources my name is Daniel I'm the head of growth and voice flow and let me know in the comments what you want us to cover next and will make sure to make a Tut on it thanks and we'll see you in the next video

Channel: Voiceflow

Views: 4,043

Keywords: Voiceflow, Prototype voice apps, create chatbots, chatbot builder, ai, artificial intelligence, machine learning, webchat, automation, assistants, agents, NLU, GPT, ChatGPT, design, Design chatbots, Production, AI automation agency

Id: skel9AlZnkw

Length: 17min 11sec (1031 seconds)

Published: Wed Jan 17 2024