Local AI Agent with ANY Open-Source LLM using LMStudio

hey and welcome back to another video and today we're going to be going over coding examples that use the open aai API to directly connect to LM studio and then we can call whatever we want on any open source model I've been reading more comments lately where people want to use open source directly and don't want to worry about paying for a chat GPT and I totally understand that so today I'm going to first show you how to directly code something that connects to any open- Source model with LM Studio we're going to quickly go over how to install LM studio and then go over a few examples let's get started well the first thing is to install LM studio is go to LM studio. and then once you come here you'll see this web page and there are three different buttons you can choose from there's a download Elm Studio for Mac windows and Linux so depending on your system you'll want to click this button I I have a Mac so I'll choose LM studio for Mac then once you click the button you'll get the file to install and then just go ahead and install it and then once you have it installed open up the application and you'll see a homepage just like this now that we're here this is considered the home screen of the software and you can know that by on the left hand side there are five different tabs we have the home there's a search there's a place where you can chat where we can start a local server and then the models that you've downloaded you can look at them now the first thing we have to do is actually download a model right so that can be done in a couple different ways you can either scroll down on the homepage and just choose a model and once you find one let's say uh the Mist 7B instruct version 0.2 okay they give you there's more uh model sizes that you can shoose choose from so what you can do is just click this download button and then this tab on the bottom kind of shows up you click here and now it's going to show you the download progress of the model you can also come up here to the top and in the kind of middle Center there's a search for models so you can also type in mistal for instance here hit go and what it's going to do is it's going to go and search for all the models on hugging face and you can see here here are a bunch of them so I can click on let's see this is version 0.2 uh this is the one that we are downloading but we can also choose others right if you want a bigger model size or even a smaller one now once that's done on the left hand side go ahead and choose the tab for local server and this is the main part how we connect our python code to LM Studio to use this model the first thing you need to do is at the top you can select a model to load so hit this drop down I have a few models but we just downloaded the mistal instruct version 0.2 so we're going to go ahead and click that and it's going to go ahead and load it into LM studio now once the model has been loaded into LM Studio there's this big green button here called start server all we do is click that and now we have a local server that we can connect to from the python code similar how we would want to connect to open AI API using chat CPT well instead of giving an API key and choosing the model well here we're just going to have it connect to the local server well what is the local server well right here this Local Host 123 34 slv1 this is the URL that we're going to use to connect to LM Studio from the python code whenever you go to my GitHub whenever you go to my GitHub this will be updated there are three examples here so let's just go through them one by one and we see the interaction with the python code and LM studio so in the first example here what we have to do is you have to import the open AI API and I say client equals open Ai and there are a few different parameters you can give here but because we're using LM studio and a l local server that has the model that we want to connect with you type in base URL give it that same URL that is in LM studio and because we're using a local llm we don't need an API key because it's local so you can actually put whatever here I just put not needed but this can be whatever you want it to all right in this next section of code this is where we're actually asking the model in LM Studio to do something for us so here we're going to tell the model that they're a fitness expert and now we just want to create a single day of workout for arms okay it's really simple but a couple things here the model here the model parameter if we were normally using chat GPT you would want to put here like GPT 4 or GPT 3.5 turbo or whatever it is but because we're connecting to LM Studio you can just put whatever here I just put local dash model what whatever you want to do but we don't have to put the actual model in here because when we connect to LM Studio the model that's loaded is what's going to respond to us and then I just made the temperature 0.7 you don't actually need this and then lastly I just print out the cont content here in the console in my IDE but we'll also see in LM studio so let's go ahead and run this all right so first thing is I had to change the model to the new 52 model which I know is a researched model but it's a lot quicker and my computer can't handle uh LM Studio very well and record so uh but the same thing applies here you can choose whatever model you want it doesn't matter but it came back and it finished and I asked it for the arm workout so the content here is all of this right this is what came back and responded with all right and so if we go back into uh py charm here it is right I cuz I printed out the content from the response uh that we asked the local model and this is it just a little bit more organized and formatted okay and that's simple that was just a simple example asking the model to do something and we get the response back okay for the next example this is essentially a chat example so we're going to keep giving input inside the IDE and then on LM Studio it's going to answer it and respond back and it's just going to keep doing that like we have here in the while loop all right so again you have to use the base URL to connect the LM Studio which is where our model is loaded again the API key doesn't matter you can put whatever string here you want okay this time when we have the create function here which is where we basically send okay so basically where we create the chat initi all right so now basically where we initiate the chat with the local model we give it the model string name it doesn't matter what you give it because it's going to know in LM Studio the messages this is really uh the roles that we are giving and telling what the model is what and what we're going to do uh the temperature set to whatever you want 0.7 is just what I said to this time we have stream equals true okay and that's important to know because instead of just having a response from the uh from LM Studio we actually get chunks because we're streaming uh data now right we're streaming the responses back and forth so for Chunk in completion if there is content uh from the stream then we want to append uh that content or we want to give it to this new message so the content starts out empty but as we keep getting responses chunks of responses back from Elm Studio we're going to keep adding it to the content uh property in this new message object and then here this last line history. append you the role the user the content okay so this input this is where we actually ask uh the local model whatever it is we want to ask let's just go through this now and it might make more sense okay so after running the example it says hi there I'm a smart assistant what can I do to help you and then as you can see here there is the right carrot which is right here in the input so in Python we say input that allows me or you as a user to give tell us something and so I did I said what is the capital of Canada and it's still responding but it's giving me the answer and so when it's done with this answer it's going to come up with another right care and I can just continue to always ask it or tell it something until I stop the application over on LM Studio side you can see that streaming the response cuz I set the stream to True um it's just continuing to stream the response until um until it's done giving me the full response okay for the last example I'm going to also show you how to use a function with with all of this okay just to give you some more ideas of what you can do so as always we start with an open AI client instance okay you give it the base URL again just remember that whenever you're connecting directly with code to llm LM Studio you just want to give it the base Ur URL and you do not need an API key uh we'll first come down here to the prompt I have an input so um at the end of this I'm just going to basically give it a word um and then the response uh this variable is going to equal we're going to call the function and then basically give it the word that I want to rhyme with right so it's going to come up here here uh I'm going to give it the prompt this will be the word so I had the formatted prompt to say hello give me three words that rhyme with and then the word that I gave it um so in the messages uh array object we're going to give it that formatted prompt we're just going to I'm just going to print out uh The Prompt and then we have the function the create function which actually uh gives it the um gives it the model the messages and everything that I wanted to so this is going to initialize the chat with the local llm um the only real thing we need here is the messages uh and then we basically just return the message object okay and then we'll see that whenever we go to print this out you'll see uh every property uh in the in the actual content uh message object that we're returning from Elm Studio okay so I'm running this and it gives me uh now I'm at now I'm at the input where I can give it I need to give it a word uh so what word let's just say time okay so then my prompt my complete prompt is hello please give me three words that rhyme with time and then over on L Studio again we're not streaming and because stream is set to false over in LM Studio it's going to show you um like every word or token that it's accumulating um as it gives us back something this will take a minute to finish and then we'll see the result all right it's done it completed the response so let's go back to the IDE and this is what I meant by here is the full uh chat completion message object okay not only does it have the content which I printed up before but also has like the roll function calls and Tool calls which is essentially allowing us to call other functions that we've created but for this simple example uh it gave us time Prime and dime and that's it okay so I just hope hope you learn now how to directly connect to LM studio with code which means that we can use any open source llm that you want locally and it's free as always this will be my GitHub so go to that if you have any questions or trouble um clone it or Fork it whatever you want to do um or just copy it directly if something is working on your side if you have any questions or comments please leave them down below I will be here to help you or if you have any suggestions also please let me know I want to help you out and also it's helping me out too right I'm also learning it's not just me teaching you I'm also learning let work this process together to learn AI as much as we can thank you for watching and I'll see you next video

Channel: Tyler AI

Views: 7,172

Keywords: ai, chatgpt, artificial intelligence, chatdev tutorial, ai agent, ai agents, autonomous ai agents, autogpt, build autonomous agent with python, chat gpt, gpt 4, python ai chatbot tutorial, ai automation agency, how to setup autonomous ai, your first software ai team, ai tools, artificial intelligence and machine learning, microsoft autogen, autogen, auto gen, ai tutorial, memgpt, mem gpt, lmstudio, lm studio, lmstudio tutorial, open ai, openai, chatgpt 4, ai agents explained

Id: Hds_fJaAu78

Length: 11min 12sec (672 seconds)

Published: Sat Jan 06 2024