Running OLLAMA On Windows // Run LLMs locally on Windows W/ Ollama

in this video I will show you how to run large language models locally on a Windows 11 computer using ama ama is a free tool that allows you to run various models such as llama 2 mistol Cod Lama and even allows for customization to create your own AI assistant additionally I will guide you through creating a simple chat gbt like web application with AMA using node.js let's get started [Music] first open your favorite browser on your windows 11 computer and navigate to the AMA website then click on the download button AMA is compatible with Windows 11 Linux and Mac select the windows icon and proceed with the download once downloaded run the executable file to install AMA on your Windows computer after the installation open the command prompt type AMA and you will see the information about the command now let's download an llm to see how it works head over to this page click on models and you will find various options including jamama Lama 2 mistol code Lama and more for now let's choose llama 2 then select a model size based on your systems RAM for instance if you have 8 gigs of RAM choose the 7 billion model and if you have more RAM choose 13 billion model now copy the command for your chosen model and paste it into the command prompt to initiate the download now once completed you will notice a change in the prompt let's interact with the downloaded model you can see that it responds as intended the speed of text generation depends on your computer's Hardware next let's request the model to generate a source code in Java to determine if a number is palindrome [Music] notice that it also explains the generated code now let's ask it to tell me a story in 30 wats and see its capability in storytelling now as you can see it did fairly good now to exit the model simply press ctrl+ D type Alama list to see the installed models on your computer let's test another model named mistol by running it in warbus mode Let's Ask mistol to write a code to create an alert box with swift UI and see its response you can anytime interrupt the text Generation by pressing control+ C which returns you to the prompt now let's customize the model create a new file named model file without any extension using vs code and paste the provided lines this file serves as a blueprint for the model's Behavior here I'm going to set the temperature to one for enhanced creativity and Define the system message to react as Lisa then save the changes Now navigate to the model file using the command prompt and create a new model named Lisa using the blueprint to do so type Alama create Lisa and specify the model file using the dasf option that's it now if you type AMA list you will see the new model named Lisa now let's run the Lisa model and see how it responds let's ask what's your name and you can see it responds as Lisa it's time to create a chat gbd like web applic using AMA make sure you have installed node.js and python on your computer then execute the provided command to create a new node.js application named example change the directory to an example then navigate to the app hierarchy using fult manager navigate to app API chat and open the route. tsf using vs code I'm going to replace these two parts of the code with these code [Music] [Music] Snippets specify the model name here I'm going to leave it as LL 2 and save the changes then inside the command promt type this command to run the web application now copy the generated URL and paste it into your web browser you will see the chat GPT like prompt at the bottom I'm going to test it by tapping an arithmetic expression to see its response now as you can see it it does amazing job the back end for this application is llau but you can use any AMA model based on your requirements and that's pretty much it this is how you can run open-source large language models on a Windows computer using AMA let me know what do you think about it in the comment section down below thanks for watching this has been ksk I will see you in the next one [Music] G

Channel: Ksk Royal

Views: 1,782

Keywords: Technology, ollama windows, ollama tutorial, ollama python, ollama vscode, ollama windows install, ollama windows tutorial, ollama windows gpu, ollama windows wsl, ollama windows docker, run ollama on windows, ollama run mistral, ollama run, ollama run llama 2, run llama 2 locally, run llama locally windows, ollama custom model, run llama locally, KSK ROYAL, AI, Chat GPT Alternatives, Chat GPT Locally, Chat GPT Offline

Id: srGF8P8-8Jk

Length: 7min 20sec (440 seconds)

Published: Thu Feb 22 2024