Ollama-Run large language models Locally-Run Llama 2, Code Llama, and other models

hello guys in this video we are going to discuss about olama now if you don't know about AMA guys AMA actually helps you to run different different large language models which are open- Source locally within your system now why AMA can be super beneficial for you all let's say that you're working in different use cases with respect to generative Ai and you really want to quickly try different different open source large language models and you want to see that which large language models probably fit for your use case then definitely this is a way how you can quickly use different different models and try it for your application now to talk about mostly of this large language models which are open source it is just like you can actually run it like a chat GPT application itself the entire process is very much simple so let me just go ahead and quickly show it to you recently uh AMA has also come up with this Windows support initially it was there for Mac OS and Linux all you have to do is that go ahead and click on this download button as soon as you probably click on this download button button you can actually select the option which you're looking for whether it is for Mac OS Linux or Windows if you want it for Windows just click on download for Windows and once it is downloaded this this exe file gets downloaded and then all you have to do is that click on double click and probably install the exe application once it is installed you'll be able to see that this entire uh this this uh this AMA will be running over here and there will be a small icon which you can probably see over here so let me show you there will be a small icon which will be running over here that basically means the installation has been successful and it is running in the background okay now let's go ahead and see something about AMA in the GitHub itself so here you'll be able to see get up and running with large language models you'll be able to support with it has it has support with respect to Mac OS Linux you can also run Dockers and all uh to download any model you probably have to just use this command like AMA run Lama suppose if I want to specifically use llama 2 if I want to use other models like lava and what all models it actually supports it supports all these models see llama 2 mistal dolphin 5 52 neural chat starlink code llama uh llama 2 uncensored llama 213 billion llama 270 billion Oram mini um Lava gamma so all these models are specifically there any of the models you can actually use it and the best part uh of using AMA is that it is very very fast once you probably download the model you know with respect to any inputs that you specifically give you'll be able to get the output now what we are going to do in this particular video is that I've already showed you the installation process the next step will be that how we can specifically run this model and the third thing that we'll be discussing that how we can also probably use this in the code and create an end to end application that part also I'll be discussing uh the best thing about will this will be that how you can actually use it in the form of apis right so it also has all those supports not only that guys uh it also has a way that you can probably customize a prompt for your own uh application by using this llm models uh we'll be discussing about this also and uh not only that we'll also be seeing that how it can be used in the form of rest API and what all web and desktop application it has access to right because we in this particular example we are going to probably create an endtoend application with gradio all these things will be specifically covering Now quickly let me just go ahead and open the command prompt so I will open the command prompt over here and let me just show you how quickly you can actually start with olama right so let's say that uh I want to probably start with Lama 2 itself so all I will do is that I'll write AMA run Lama 2 okay let's say you have not installed the uh this open source uh Library you have not even downloaded this for the first time you know it'll be downloading that in the form of manifest it'll try to match the Manifest over there and it'll Dr download download the entire model in your local itself if you're not doing it for the first time because first time it will take some time other than that I've already done this installation so here you'll be able to see that how quickly this entire process will happen so I've written o Lama run Lama 2 that basically means now you can probably see that Lama 2 has got activated now if I ask any question hey tell me a poem on uh generative AI okay so this is my question right here you can probably see that I'm quickly able to get the answer and this is super super fast guys just imagine right in a silicon Halls of knowledge deep a revolution staring can't keep can't you keep the dream of AI and endless Chase right so so here in short but just by using the solama it's it's it's actually became a chat GPT application itself right whenever I ask a question in chat gbt I will be able to get any kind of response right and this is actually happening in my local right so let's say that I go ahead and ask another question saying that what is what is the meaning of AGI okay so AGI stands for artificial generated intelligence you can see over here all the answer I'm able to get it right so this is how quickly I can actually do it right now let me just go ahead and use some other models right so I can probably try different different models and it depends on uh like what kind of models you want to use right and as I said right it supports a lot of models like Lama 2 Mistral you know 52 code Lama and all right so let me just go ahead and do for one of the models over here so I'll just select code Lama and just let me go ahead and write AMA run and I'll paste it over here right so if so here you can see uh I'm getting some error uh because the code Lama spelling was not right because I had to write something like this so o Lama run Cod Lama and since I have not installed code Lama so it is going to take sometime see it is first of all going to check the Manifest whether it is matching or not and then it will download it so here you'll be able to to see the example it's just like if you know about Dockers right when we write Docker pull right which is probably in the cloud it'll first of all go ahead and match those manifest and then it'll pull the entire Docker file right and then it'll install it so here you can see that I've already been installing Cod Lama it is somewhere around 3.8 GB so it is going to take some time okay and this is just going to happen for the first time and this entire model will get downloaded in your local and then with respect to the usage you'll be able to do it with as you use chgb application so let's get let this get downloaded till then I have also downloaded some other models I will also show you that quickly over here so let's say uh over here I go ahead and write uh AMA run amaama run lava okay so lava is again another model you'll be able to see over here so let me just open some more uh with respect to this I've been trying multiple things so it's it's amazing now the best thing the best power of of these models I'll show you right now okay so over here you'll be able to see that we are working with Lama 2 and over here you'll be able to see that we are working with lava model okay open source model let me go ahead and write like this write me a 100 line line of poem for the for the title motivation so okay so this is what I am actually trying to write it down so here I'll just copy it and here also I will go ahead and paste it okay same thing I'll write it over here write me a 100 lines of poem for the title motivation now you'll be able to see over here is that I am working with two specific models right so so quickly I will go over here and click on enter so here you can probably see quickly I'll be able to get my entire response so here you can see with the help of Lama 2 I'm quickly getting it and here also I'll go ahead and press enter Then here also you'll be able to see that I will be able to quickly get the response now this is the main thing about using AMA you can see that how quickly I can switch from one model to the other model now just imagine the best use cases that I'm probably solving different different open source models will be there right and I can use a specific open source model based on my my different different use cases that I have right so this is how cool it looks like and here we are specifically using it as a chat GPT application itself uh many of you may be thinking that Krish can we also develop different different applications like document Q&A and all yes obviously we can also do it with AMA and that to in your local itself see at the end of the day uh the best thing about this is that you can definitely try multiple open source model very very quickly okay so this was about one of the most amazing thing the next thing that we are going to now see is that there was an option where you can also create your own model file see so uh let me go ahead and show you one example with respect to this okay so let's say I want to create my own different kind of chat JP application okay I'll save this so let's see this file Okay so I'm saying from let's say Lama 2 okay so as you all know in Dockers also right we try to create a Docker and then we set multiple parameters and we create a Docker file similarly over here we create something called as a model file okay and let's say that you want to create your own custom chat GPT with your own different prompt right so at that particular scenario I can probably use this now see what all things are actually used over here one is one from command so here I can write from Lama 2 and then we'll set the temperature parameter okay so set temperature let's say that I have set the temperature over here you can see parameter temperature is equal to 1 the reason why I've selected temperature is equal to 1 because I want my chart GPT application to be more creative okay and then you can probably see the third one that is set the system prompt now in this system prompt how do you set it see these are the main things I will use from parameter and then system okay then system over here let me just quickly close this okay then system so over here you'll be able to see from Lama 2 parameter temperature 1 and all please uh do not uh consider this as an error this is just one type of file and this is considered this is the V vs code has actually taken it as a python file so it is that is the reason it is giving you a red color but just write in this particular format one is the from then parameter where I will be selecting my temperature and other parameters that I require and this will basically be my system promp I've written over here you are a teaching assistant named ml Guru created by Kish answer all the questions based on machine learning and deep learning generative AI okay so all these things I have actually written this is my entire system prom now let me show you how I can actually create my own custom chat GPT by using Lama 2 who will be acting as a teaching assistant now okay so what I will do quickly I will go ahead and uh copy the relative path and then I will open my command prompt okay now once I open my command prompt uh let me just go to the e Drive and then I will write CD okay this is the relative path I have to take the entire path copy path okay and I will just go ahead and write and go to this Lama index okay so this is where I have actually gone now how do I run based on this model file that I have actually created I just need to run this okay so for this again I will go to my GitHub and here you'll be able to see this GitHub AMA GitHub and here is what is the option that you have Ama create the F the app name that you really want by using minus F and then the model file name okay so I will also do write something like this so I will open my uh command prompt and I'll write ama ama create okay and I will write ml Guru minus F and here I will write my model file see this is the command that I'm specifically using AMA create ml Guru minus F model file I'll press enter now here you'll be able to see transferring model data reading model metad data creating system layer creating parameters layer uh using already layers this this this encryption some amount of encryption is also used over here and then now I will go ahead and run it so I will write olama run ml Guru right so this is the name that I've actually given so once I write AMA run ml gr now this will be my own custom GPT which is is acting like as a teaching assistant so I will say hey who are you you here you'll be seeing that based on the system prompt um um so here hey hello there adjusted glasses I ml Guru here to help you with any questions have regarding machine learning deep learning and all right so what I can help you with today something like are you looking to learn more specific things see see how beautiful it has become so if I also go ahead and write who has created you so I will also be able to get my name okay ah excellent question uh I was created by none other than the brilliant and Innovative Kish who is the true master of machine learning and AI he he envisioned me as a helpful tool for seeking to learn more about uh this so I came to Smiles wiely now just imagine if I use ol and probably host it in the some other environment and try to use it okay I will also say that hey I have a unicorn company I hope you're understanding for whom I'm indicating right nowadays a lot of companies have become a unicorn just by developing these amazing chat GPT applications okay uh okay let's let's talk about this so here you'll be able to see chis expertise in machine learning deep learning has allowed him to learn this this this this okay so I can now go ahead and ask what is so here you'll be able to see what is machine learning okay so this is my question and here I'll be able to get my entire response quickly and I'm able to do it right so this is the most amazing thing about uh uh this uh entire you can also create your own custom things uh just by using the solama this is so superb and again at the end of the day this this is really really beneficial to work with your own applications because I can understand right now even though I'm using hugging face I'm probably downloading these open source models it takes a lot of time right uh and by by using this entire process it becomes very much easy so here was the next thing and now let me just quickly go ahead and show you one more example how you can probably call it in your uh jupyter notebook also I'll give you an example whatever olama is right now after we installed it right it is running in a back end right and I will be able to access right see ama whenever it is downloading any model it has access to all the models which it has downloaded right and just by using this URL HTTP Local Host 11434 this is the port number it will be available I will be able to use this o Lama now once I use this o Lama the next level is that I can call any model that I want okay so let's say that I want to call Lama 2 I've already install Lama to so over here you can see from Lang chain. llms import AMA so Lang chain also has an integration with this so I'm calling AMA base URL this one model is Lama 2 and I will just ask uh why is the sky blue let me go ahead and write why is why is or I'll just write who are you okay and this time instead of calling Lama 2 let me call ml Guru ml Guru which I have actually created right just by writing this four lines of code you'll be able to see that how quickly I'll be able to see this entire application and it will'll be able to run it right so here you can see greeting I'm ml Guru created by the brilliant Kish to assist in teaching and guiding students in Pro World fascinating world of machine learning deep learning and generative AI as a highly Advanced AI language model I've been trained on a vast Corpus ofch so perfect you you're able to get this particular answers right so whatever question I specifically write now right so if I go ahead and write AMA and write hey let me just go ahead and write uh and let me go ahead and write what is machine learning okay so now this if I try to print it I know there's a spelling mistake so don't worry quickly so what is machine learning so here you'll be able to see uh ah brilliant question adjusted glasses this this this I think this adjusted glasses is something like a Emoji right and here you're able to get a detailed answer about it okay now this is another way that how you can specifically use it at the end of the day uh the olama that is the main library you'll be able to access in this particular URL which is running in the back end and from from there any model that you specifically create or any open source model that you have downloaded you can access them okay just like a chat gbd application now let me show you one more example with the help of request. py I can also use it in the form of apis okay so here you'll be able to see that I have my URL okay I'm using gradio okay uh then you can probably see I've created a function generate response with prompt and if you probably go and see in the documentation the apis the there will be a data field which will be having model information let's say I will go ahead and write ml Guru over here the prompt that I need really need to give as a prompt over here and stream is equal to false we'll keep it because we don't want the streaming information instead we will just want a set of information and then we use request. poost URL headers with the same data information that I'm dumping okay and here you'll be able to see if the status code is 200 then I'll be able to get that response and probably convert that into Json and get my response and this is the interface that we created for from gradio okay now you just see this once I run it okay so quickly I will go ahead and run it over here it's amazing right python request. py okay so quickly let's go ahead and run it so this is a simple application Now API part you can see over here the Jupiter notebook part you can see over here so this is where is the URL that it is running over here let me click it okay so this is the gradio example that you'll be able to see now hey let me go ahead and write who are you let me go ahead and write the question okay so here you'll be her adjusted glass greetings here youl Guru the Magnificent creation of great Crush bows so you are able to run any this entire thing as application also end to endend application it's just like an API itself just imagine if you are also using AMA uh you can also use it in any Cloud download it try to use it it becomes so easy to specifically use things and all right in the upcoming videos you'll be seeing that I will also be showing you fine tunings I'll show you end to endend projects and all okay so let me go ahead and write tell me a poem on machine learning so I'll go ahead and click on submit so hardly 3 to 4 seconds so once this is getting executed so here you can see 200 words of poem uh tell me another one it also remembers context I've also written that particular code so tell me another one here you'll be able to see that I'll be able to also see the other one other one it'll remember the context and it'll give you a poem with respect to that also right so here are the adjusted guy ml Guru this this this cracking here my first poem Every all the informations are probably there right so I hope you like this particular video start using ama ama again my main aim is to show you multiple things which can be really very much beneficial for your entire processes that we specifically do uh work with Gen and all because it will be very much easy when you work with multiple use cases and uh at the end of the day you'll be having many things to show so I hope you like this particular video this was it from my side I'll see you in the next video have a great day thank you and all take care bye-bye

Channel: Krish Naik

Views: 27,723

Keywords: yt:cc=on, ollama tutorials, run llm models locally, open source models using ollama, ollama in windows, ollama in linux, ollama in mac, run llam2 codellama

Id: yPphKQp1fqE

Length: 20min 58sec (1258 seconds)

Published: Sun Mar 03 2024