Run Local ChatGPT & AI Models on Linux with Ollama

what's up guys it's Josh from kekki and in today's tutorial we're going to explore how to run chat GPT and other large language models on your Linux system using the AMA tool now AMA makes it easy to download and interact with a variety of Open Source large language models similar to open AI chat GPT so whether you're a tech Enthusiast a developer or just curious about AI this video will guide you through setting up AMA on you bunch of 2204 so let's get [Music] started all right so I'm at the GitHub page for olama but before diving into the installation let's discuss what AMA is and its significance now like I stated earlier AMA is our open source tool that enables the setup of a wide variety of large language models using simple commands it eliminates the need for manual processes making it a game Cher for those looking to run models like llama 2 or like Orca Falcon and open chat locally that's the dope part about it you can actually run all of this locally and so that opens it up for you to store all your information any of the information that you put in it locally on your own system and you control your data you don't have to worry about an organization losing any chat information that you put up there and it's always recommend not to put any personal information up there on like chat GPT or any of these other AI tools that are out there and so AMA helped with that and so at the end of the day this approach offers greater control privacy and the ability to interact with these models directly on your Linux system now this application uses a MIT license so you guys can check it out it's currently on version 0.1.2 and it was updated 5 days ago so they're constantly developing as you can see it's got 105 contributors to the code of this application if you go down a little further you'll see how to get it install you can install it on Mac you can install it on Linux and what cool thing on Linux it's just a one one script you have to run to get it installed they also have a Docker image you can install as well but I'll just use a script that way we can get it installed directly on our server and not within a container and it also breaks down the models you can use they have a library of all the models and let me go down and go there so you guys can see that but these are the major ones now let me read this note down here I want to show you guys this you should have at least 8 gbits of RAM available to run the lb models 16 GB if you run the 13B models or 32 GB to run the 33b model so that's one thing you really have to pay attention to is your Hardware of your system and what you're running because it could be very hard to running some of these models if you don't have the proper hardware and typically what you want to do is have a dedicated system to run these models off of and majority of the time people have video cords because that can handle a lot of the processes when it comes to looking at these models and so just be weary of the hardware requirements and if you go down in here it breaks down you know the installation references to different things like the models how to download multiple models or create your own model based on the model and actually storting it and all that good stuff so you can go through the documentation of it they do a great job of putting all the information out there for you oh and one other thing I forgot to show you guys if you go over right here under the model Library you can click right here is. a and you can go to the library and this will show you the other models that are available to you it's a whole bunch in here like Orca 2 Orca mini it was another one I saw in here I think it's one of the ones created by Microsoft yeah Orca to that is orca 2 Microsoft research and this model is designed to Exel particularly in reasoning so just go through and check it out like this one wizard math so it's a model focused on math and logic problems and you can also see like which ones are being used the most as far as the pools go and I think it's in order by the amount of people most people use the Llama 2 that's the most popular it's just a general use language model and if we go back over to the main page if you look at llama 2 it tells you it's a 7B as far as the parameters so 7B can run using 8 gig G Abit of ram so I got at least 8 gbits of RAM on my virtual machine and also let me show you another application that I will go through and install because if you can't visualize it what's the point you know what I'm saying you can do they have tools for the command line but I want to show you guys how to set it up on the server and I also want to add the AMA web UI so I'm going to go through both of these and this is a dope little tool right here this is a picture of it it kind of looks just like chat GPT but you can select your models and you can run through and do whatever you need to do using a local language model you know right from the web and you're connecting to it you know from your house your internal Network which is super dope now let's go on and hop over to my virtual machine so we can go through the install and get it set up before we move forward I wanted to give a quick shout out to ciq the official partner of Rocky Linux Rocky Linux is a Linux distribution that is intended to be a downstream complete binary compatible release using the red hat Enterprise Linux operating system source code the project is led by Gregory curer who was the founder of the cosos project so check out Rocky Linux at c.co all right so I'm connected to my virtual machine via SSH and let's start with the installation process by first updating our system that's something you want to always do whenever you're installing any new software on a system is to update it so let's run SHO apps updates and press Center and this system is this server is actually already updated I know it's not going to have any updates i r through all the updates and as you can see it says all packages are up to date but you may see something different now let's switch back over to the website cuz I want to show you guys how to get it installed and it's super simple on Linux there's a script I definitely want you guys to check this script out you definitely want to look into this script make sure it doesn't you know have anything in it but I reviewed it off camera you know saying just checking to see what it's going to do but it's going to go through install the applications create a user give it permissions and all that stuff so you want to check that out make sure it's doing something that you wanted to do on your system so let me switch back over to my terminal so you guys can see me run the script but that's all I did was copy that so let go on and run it right fast but that's all you got to do this is the installation of a llama it'll go through the process of downloading and then installing everything via that script you'll see it creates a user a alarm user and it also create the service it'll enable that service and it also start that service and so one thing you can do to check if the service is running is by typing pseudo system CTL status and then it's olama do service so let's press enter and you'll see that it's active and running so our server is up and running we have Ama installed and it's good to go now it's time to download our large language model and let me go back over here to the website again just to look at them again like I said I'm going to do the AMA 2 and actually let's go back over to the library because the name is the exact name that's up here so all you have to do is type it exactly like what what's up here so L llama 2 so switch back over to the terminal and I'll show you guys the command cuz it's a you know it's a few commands for Alama you can check them out I'm not going to go through them all but I'll just at least show you guys how to pull down the LGE language model so L llama and then two and that's the one we want to download and you can download as many as you want and as you can see it's it's downloading it's going to go through the process it's like 3.8 GB in size and that's one thing I didn't point out you want to make sure you know you have enough drive space cuz these things can get pretty big and take up a lot of space on your system so that's why I recommend you you know set up a dedicated either server like a smaller server or something like that or just set up the virtual machine to where I can run this properly with enough hard drive space on it and as far as other large language models that you want to put on here you're only limited by the space that you have on the system so you can install you can pull down as many of those language models as you want and then when we get to the installation of the goey the web UI then you guys will see where you can select different language models to actually run and use on your system now we're pretty much done with o it was pretty simple want I'm going to run the UI using the docker setup and so we need Docker install so I'm going to run through this right fast I want boy boy you guys through the process I have a lot of the commands I already copied out but I'll at least tell you what I'm doing so we already ran an update on the system where it refresh the repositories let's go to install a couple dependencies for Docker so let's go on and get that installed that's basically the app transport htps CA certificates curl which we already have software properties- Comon so just go on and press y enter we can go through and install those packages and now what we're going to do is add a PPA and so the first thing we need to do is add a gpg key for Docker the docker repository so I already have that copied I'm basically curling down that gpg key and installing it on the system and then now we can add our repository and like I said I'm I may make a blog post that'll have these commands in there that way you guys can copy them out but all you got to do is search how to install Docker there plenty of tutorials out there but I create one on my website just so you guys can get the information if you need it but this will add the repository for Docker and now we can run the Pudo apps updates and this will refresh the repositories and also refresh that Docker repository which is the main one we want to do now we can install Docker and what I'm going to do is install Docker C as well as Docker C CLI and then containered IO so just go on and press enter and go through and install that all right and so Docker is installed you can verify that it's actually up and running we can type pseudo system CTL and then status of Docker and it's start service press Center and you can see that's active and running so we're good to go with that now let's go back over to the documentation on the website just want to show you guys uh a little bit more more as far as the UI but they do have some installation instructions if we go down a little further you'll see It'll it kind of walks do a lot of the same things that we just did but AMA after the initial install it actually when it starts it actually runs at 127.0.0.1 and then this port so 11434 and so we need to know that in order to get our Docker container installed and it connects to the oama back in and that way you can use your language models that we have installed or that we pulled down so we got Docker installed let me just go down and copy and paste our Docker command in there and let me switch back over to the terminal I just wanted to show you guys where I got the information from I had to modify it and I'll show you guys what I did to modify it right fast so let's go back over here paste in our Docker command and as you see it's just Docker run dasd I had to make some modifications to it so D- Network Das host and then I also put the location of our olama server which is basically a loop back or the home address for the system with that same port and then API and then the rest of the information like the name of the docker container and so let's go down and uh press enter we can run it and I forgot about that we have to run this using pseudo cuz I don't have pseudo privileges under this account so you want to make sure you do that as well if you want to if this is like a production system you want to set up a Docker you know and give your user privileges to Docker I didn't want to waste time doing that so that's why I didn't do it so we can just run it using the P sudo and that'll get it started for us and it'll pull down the image for the web UI if it's not on there already which is not and then it'll set up the site and then we can go to it in our browser in a couple minutes when this finishes all right so our Docker container is created so let's go on and just check right fast you can run PSE sudo Docker uh PS and then we could do- a uh and this will show you the actual web you want why that the container is actually running it's been up for like 3 minutes or so and so before we switch back up to our browser we need to figure out what the IP address is I already know what it is I'm just showing you guys how to do it but IP space a press enter that'll tell us what our IP address is that the server is 192.168.1 133 so actually let's copy it right fast and like I said the port is 8080 so switch back over to our browser open up a new tab let's paste in our IP address it's going to try to go to it but we have to put ports 8080 at the end of it and press Center it'll take us over there it doesn't have aert or anything so all you have to do is type sign up and this is all local don't worry about this this is not going out to any servers and storing your information anywhere and let's just type a name in here I'm going to put kit and then keep it techy at gmail.com let's just use that and then just type in a random password whatever you want so you can remember hit create account and this is all stored locally like I said it's going to stay here and now we are in the olama web UI super simple if you've been to chat GPT it's essentially the same thing you just have a whole lot more features for one you can actually talk to the thing I think you have to set a lot of this up but if you go in here under settings just want to show you guys this is how it's connecting to our llama backend the API that we created on our server like I said it's running on 11434 don't you remember I tell you guys that that's where it's actually connected to that actual language model that's running in the background or not the model but the actual application that's running in the background that allows you to specify a language model that you installed on the system and you can go through here I'm not going to go through everything you can set a default language model you can actually download from here we already have some models that model that we you know downloaded we could select that one and make that one the default so and then there's some externals uh you can do add-ons and like I said I don't want to go through everything everything chats accounts and all that stuff so let's hit save that way it'll save our model and actually we probably have to select it up here and we can set it as our default there we go so that'll work we have the one that's downloaded on our system and we can use that one for everything and so let's just run through and do a simple chat and this may take a little time because the server that is running on like I said it's not heavy performance-based it's running in a virtual machine you know what I'm saying so you'll see the text come in kind of slow but it actually works so that's why I say it's kind of best to put it on physical hardware and then also one thing I'm doing is like I said I'm I'm using the CPU for this system and so the CPU you know it's kind of slow versus having a GPU tied to this virtual machine if I had a GPU it'll probably run through a lot of this faster so it's super cool I'm going just stop it right here well actually finished so that's super cool and then we can go through we could train it a little bit more by creating chats acent certain information or remember just like on chat GPT you know tied to your account modif files you can check out this right here you can go to prompts documents you can upload documents or pictures you know all kind of stuff but the cool thing like I said it's local and then you can also search for previous chats that you had in the system and so congratulations you successfully install AMA and Lama web UI on your upine 22204 system and are now ready to explore the world of large language models locally I hope this tutorial was helpful in getting you started with ama if you have any questions or facing the issues please leave a comment below and I apologize if I sound a little crazy I I have a little bit of a cold and that's why I sound a little weird but don't forget to like share and subscribe to the keep T channel for more Linux and Tech tutorials until next time keep exploring and keep it [Music] t [Music] I

Channel: KeepItTechie

Views: 46,062

Keywords: artificial intelligence, Ollama, Linux, Ubuntu 22.04, ChatGPT, AI models, Large language models, Installation guide, Tech tutorial, Open-source AI, Llama 2, Orca2, Falcon model, OpenChat, AI programming, Python environment, Ollama WebUI, WebUI, AI interaction, Tech enthusiast, Nvidia GPU, Intel/AMD CPU, KeepItTechie, llama 2, chat gpt, open ai, gpt 4, machine learning, ai chatbot, llm tutorial

Id: 0NGMqR_e5rE

Length: 17min 11sec (1031 seconds)

Published: Thu Jan 18 2024