Flowise AI Tutorial #10- Using OpenSource Models in the Cloud with Replicate

Video Statistics and Information

Video
Captions Word Cloud
Reddit Comments
Captions
in this series we mostly look at integrating our chatbots with the incredibly powerful open AI models but there are many reasons as to why you might want to make use of an open source model instead open source models can offer bespoke specialized and uncensored features that are not available with the openai models one example of a powerful open source alternative to the open AI models is the highly popular llama 2 models by meta in the previous video we already had a look at running these open source models on our own machines and then integrating them with flow wise running these models locally can be a lot of fun and is great for testing and prototyping our applications but when it comes to building production ready apps we need a solution for deploying these models to the cloud thankfully flowwise provides several tools for integrating with services like hugging face and replicate in this video we will add a llama 2 chat model to our application by integrating our flowwise app with replicate also stick around until the end of the video where I will show you how we can generate images from our flowwise applications as well let's get started so what is replicate replicate is a service that runs machine learning models in the cloud and allows you to interact with these models through API endpoints this is not limited to language models either as we can interact with image and video generation models as well we can use the replicate services for free but when your usage picks up you might be prompted for a credit card at some point I have used the service quite a bit and have not been asked to enter my credit card details as of yet so don't worry about adding a credit card in order to follow along with this tutorial let's go ahead and create our account by clicking on get started after logging in you should be presented with a dashboard like this on this dashboard you can do a few things like view your recent predictions upload your own models Etc but what we are interested in for this tutorial is the API key we need that in order for flowwise to interact with replicate and we will also have a look at the explore option let's first go ahead and grab our API key under your profile click on API tokens you can then go ahead and create a new token or simply copy the default token after copying the token go back to flowwise for this demo I'm running flowwise locally but if you are running it in the cloud you can still follow along let's save our replicate credentials by clicking on credentials then let's add this credential by clicking on ADD credential let's search for the provider replicate and let's click on replicate API let's give the credential a name I'll just call mine replicate API Ai and let's paste in that API key and let's save this credential now let's have a look at which model we will be using for this demo replicate offers access to hundreds of models including language models image generation models video generation models Etc we can browse through all the models by clicking on explore and from the explore page we can browse all these popular models or we can use this search box to find models by name just looking at the models on screen we can use the code llama model for code completion or the stability AI sdxl model for image generation but for this demo we'll be using this llama to 70 billion parameter chat model let's go ahead and click on this model from this page we can view a demo of this model and if we scroll down we can change the prompt enter the system prompt set the max tokens and some other parameters and class to the bottom of the screen we can click on submit to run this model and on the right side of the screen we can view the output in order to view the API details we can simply click on the API Tab and this page will give us some sample code and instructions on how to use this model and we can see some sample code for JavaScript and Python and the curl API we will not be calling the apis directly all we need to integrate this model with flowwise is the name of the model and we can get the name of the model by going to the sample code and then copying this entire string value we will get back to this in a second let's switch back to flowwise in flow wise let's create this jet flow so I'll click on chat flows let's then click on add new let's give our flow a name I'll just call mine llama 2 chatbot let's click on Save let's now add an llm chain to the project let's click on ADD nodes then let's click on chains under chains let's add the llm chain then let's go ahead and add our language model let's click on ADD node then click on llms and then add the replicate node to your canvas and let's connect this llm to our chain like so right let's have a look at this replicate node first we need to add our replicate credentials because we've already done that we can simply click on this drop down and select the replicate credential that we created earlier we now need to enter that model name so from replicate let's copy the name of this model and let's add it to our node let's leave the temperature as 0.7 finally let's add the prompt let's click on nodes under prompts let's add the prompt template node and let's connect this to the chain as well let's enter a value in our prompt template I'll expand this and I'll start off by priming the model with something like you are an AI assistant answer the user's questions truthfully and do not make up your own answers let's start off with the user's question which will use a variable called question and this variable represents the question from the chat box lastly I'll just enter assistant colon and space let's go ahead and save this prompt template next let's assign a value to this question variable by clicking on prompt values for the question variable let's click on edit and for its value let's select the question go ahead and save your chat flow and let's test it let's click on chat and let's type hello and after a few seconds we do get a response back from the Llama 2 model also take note that if we go back to replicate and let's go back to our dashboard we can now see that interaction in the list of recent predictions and we can also click on this ID of the slightest prediction and that will show us additional information about The Prompt that we passed to this model this is ideal for keeping track of which brands work or troubleshooting issues with your prompts of course this is just a simple example and the real fun with working with these open source models is to play around with these uncensored models as promised I do want to show you that it is possible to generate images from flow wise let's go back to replicate and let's go to the explore page and just for fun let's click on the diffusion models and you can use any of these models but I'll just go with this stable diffusion model so I'll click on the model and here we can get a sense of what the prompt should look like and effectively The Prompt should just describe the image to get the model details let's click on API and within API let's copy the model name like so then back in flow wise let's go ahead and replace this model name and the list also change is the prompt template as a prompt template should contain the description of the image that we want to generate I'm actually going to clear all of this and simply add the question from the user let's save this and let's save this chat flow right let's test this out I'm going to clear the chat and let's enter a prompt like a cowboy riding into the sunset let's send this and after a second or two the bot responds with this link let's open this and there you go a cowboy riding into the sunset I hope you enjoyed this video if you like my content please consider subscribing to my channel and please like and share this video I'll see you in the next one bye bye
Info
Channel: Leon van Zyl
Views: 10,734
Rating: undefined out of 5
Keywords: langchain, langchain tutorial, flowise ai, flowise ai tutorial, flowise tutorial, flowise setup, flowise install, data science tips, artificial intelligence, ai, llm, openai, gpt4, flowise, chatbot, document, artificial intelligence course, embeddings, openai embdeddings, langchain tools, langchain agents, langchain memory, ai chatbot, chatbot tutorial, chat with data, ai for business, chatgpt for company data, Flowise API tutorial, llama 2, replicate ai
Id: ppST8_LiuqU
Channel Id: undefined
Length: 8min 40sec (520 seconds)
Published: Sun Sep 03 2023
Related Videos
Note
Please note that this website is currently a work in progress! Lots of interesting data and statistics to come.