Run LLM AI Model in Local machine with Zero effort (No Internet needed)⚡️

hi guys my name is Karthik and I from is automation.com and today in this video I'll be talking about how we can run our own local large language model you know all these days we have been using this chat GPT large language model or Gemini large language model to perform all the operation for example if you just ask all this information from here it just goes all the way to the Google server and then it's going to get you the response back from the server for you and the same thing happens for the chat GPT as well so whatever that you type over here like like who I am then it is going to go and ask the same thing to the server and then it's going to give the response back from the open AI server right that's how everything is working all these days but what if we wanted to do everything in a private large language model how can I even run that well the answer is GPT for all there are many different ways that you can run the large language model in your own machine I mean you can download the large language models something like the Llama Or the GPT M within your machine and then you can run it it's kind of pain if you wanted to do it all by yourself you're not an expert on doing that but if you want to do everything in a very very easier manner pretty much like an end user who just know how to download the software and then start using the llm without needing to do all these extra nuts and bols ceremony to run a llm then gbd4 all is the answer for you as you can see this gbd4 all is a very very freet to use locally running privacy aware chatbot no GPU or Internet required which is really really mind-blowing and that is the catch for me to use this software and I can see that it can run on any CPU as well you don't really need any specific CPU architecture to run this particular large language model all you have to do is you need to download the installer like Windows or OSX or eventu and then you can start using it as like a visual studio code or like a notepad that you can do with so I have my Mac operating system over here and I have already downloaded this particular GPD for all within my machine and I have installed it I'll quickly show you how this particular GPD for all client looks like so if you just go over here this is the GPD for all client and it does show you in the homepage that you need to download models so you can see that it supports mistal Falcon llama llama 2 MPT Ripple and many more and if you just go to the download models you will see that it will be presenting you the Lama 3 instruct and there are many different models and if you're going to be using the chat GPT 3.5 then you need to use the API key over here and similarly for gbd4 you need to specify the API key over here but doesn't really know about how many parameter it supports and what are the features that it really offers for you out of the box because it's all going to be coming up or updated based on the API key that you're going to be supplying but as you can see over here the Lama 3 which is from The Meta or Facebook it is trained by them so it has parameters of 8 billion and it is something that you don't really have to specify any API key or something like that you can just use it freely and the same thing goes for this particular version as well like Norris Hermos so I have already started downloading the Lama 3 model and once you start downloading it it is going to download something like this as you can see over here and because I have already downloaded it's going to start initializing it over here much much quickly and once it is ready then we can start using it pretty much like the normal chat GPT within our local machine without even having to have any internet access so I'm just going to go and close this window over here because it has already downloaded the Lama 3 instruct for me and guess what I'm going to turn off my internet this time over here it's done and I'm completely off grid right now so if I just go to the browser and then if I go type let's say chat gp.com you'll be saying that you're not connected to the internet which is cool so it's not connected and now I'm going to go and ask for some of the question for example tell me the evolution of human being so if I do that it's going to start telling me all the information over here and all these are coming because we have the llm or Lama 3 running over here and all the informations are coming up pretty much like the large language model which is going to be given to you from the chat GPT as well and similarly you can ask some automation testing question or programming question if you wanted to so for example if I'm going to ask like write a simple selenium with C code to initialize a browser and open popup window or whatever so if I do that you'll also notice that it's going to start writing the program for me behind the scen so this is pretty cool as you can see that it has generated the entire code for me over here and it's also showing me some of the coding responses you can keep on asking more questions from here on I'm not going to go deep into like how all these question and answers are going to work pretty much like how the chat GPD does but you'll also notice that it has exactly same kind of familiar UI so the history of the chat is going to be recorded over here on the left hand side and then you can also create new chats from here so this is pretty cool it is something which is doing pretty much like the chat GPD but it is all all personalized model so you're not going to go and connect to the internet but you're all doing everything in your local machine and one more thing about this gp4 all. I provides to us is the local document support so what does that really mean so if you just go over here and hit this particular database icon over here it's going to show you the local documents so what does it really mean basically it is telling you that you can use your local document as the model for you to be processed to get the responses back so if I just go and hit this add and remove it's going to show you that local document collection so this feature requires a download and text embedding model in order to index documents for later search so you need to download this SB text embedding model for performing this operation so I'm go and download that but for that I need an internet so let me go and connect it over here and once it's connected I'm going to go and hit download for this eser so this is just 43 MB model very very super super simple it's downloaded already and now I can see that this local document collection is going to show me a different screen altogether guess what I have did is like I have already downloaded a simple PDF file from Microsoft which is going to be showing me all the details of the asp.net core and it is like 205 MB so you can see that it has quite a lot of pages over here and you will notice that it is almost 208 pages so now we can use this document as a reference for your model to perform the rest of the operation so I'm going to use this model over here and we'll see how it works so I'm going to go uh browse this particular documents I'm going to go download and document and then going to go to the collection name I'm going to say that it's going to be a local as.net document and I'm going to hit add and this is going to start indexing right now so once I hit this close over here you see that it is going to be starting to do the m embedding and once the embedding is done it is also going to start doing the indexing for me behind the scene so this way I can use this document the downloaded net document for my local questioning and answering purpose and this is going to be happening based on the model that I have downloaded which is nothing the B model that I have downloaded so it's currently doing all these operation for me behind the scene while that is happening I can just go and choose this particular document over here and then I can just close this guy over here so I'm going to go and hit this new chat and now if I try to interact with this particular document that we have got I'm going to go and choose this particular database icon over here and you see that the local ASP dotnet is actually still indexing for us and if I go and choose this checkbox over here it's going to give you the warning saying the searching collections while indexing can return an incomplete result I'm still going to agree this particular warning and I'm going to go and start chatting based on that but I'm going to choose the model which is the Llama model that we have got and now if I start searching anything from here let's say what is test host and hit enter you see that it's going to go and choose the document that we have got and it's going to give you the response based on the context so you know that the indexing is still happening that's the reason why we can see that it is an very very incomplete result but once the whole indexing happens once you wait for the entire indexing to complete you will see that the response is going to be differ you can ask some follow-up question based on this as well but I would recommend you to do it after the whole operation happens because sometimes this whole client crashes while this indexing happens I have tried many times that every single time while I ask the follow-up question while the indexing is happening it always crashes so make sure that you don't do that so if I ask let's say what versions does it support and if I hit enter you look at that it's currently crashed and this particular chat quits unexpectedly comes up over here so this is fine because you don't have to do it during that particular time I mean this is one of the issue that we have got with this particular GPD for all that we have got but at least now you have got the idea of how we can make use of the uh GPT over here which is nothing but the chat GPT alternative to run in our local machine and even use the document that we have got to use as a context to perform the operation so that's about it guys hope you like this particular client and you can also use it with different models I mean I really like the matter of fact that we can use many different models with this particular client and the good thing about this particular client itself is that you can keep updating the model once there is a new model being released so once there is going to be any update from the model you can also refresh this particular model and then you can use it from there so that's about it guys let me know your thoughts and catch you in the next one

Channel: Execute Automation

Views: 654

Keywords: AI, Artificial intelligence, GPT, ChatGPT, ChatGPT alternative, LLM, Large Language Model, Bard, Google gemini, Gemini, Google, Open AI, OpenAI, Microsoft, mistral, GPT3.5, GPT4, GPTForAll, Local model, running model locally, executeautomation, Latest trend, latest, cool trick, trick, simple trick, fast way, AI in track, latest 2024, fast and easy, introduction, how, tricky

Id: 7I8igx6WAG4

Length: 10min 38sec (638 seconds)

Published: Thu Jun 13 2024