Ollama: How To Create Custom Models From HuggingFace ( GGUF )

hello guys welcome back to at another video of AMA until now what I have been doing is creating the videos of AMA and using the models which are in the model section of ama's website right so as you can see here also if you are new to AMA please go and install AMA you can follow this video if you want to install it but the problem here is that if you go to the hugging face website there are 521 th000 more models but in the AMA there are not that many of the models right what if we want to take let's say one of the model from hugging face website and create our own model using AMA and use that to chat with documents or even create a simple simple chart GPT like UI right the final goal of this video will be taking one of the model from hugging face website using AMA and replicating the create chart UI using chain L Lang chain AMA and the model that we create by ourself let's get started this is the ama's website and the prerequisite as I showed you before is you need to have Ama installed locally right first make sure that you have Ama for that you can just type AMA and if it shows something like this it is installed and make sure that it is also running in your machine to know if it is running or not you can just go to the Local Host and if you run the port 11 1434 and if it shows AMA is running then we are good to go before creating our own model I first want to show you how to navigate or let's say what is the model file that is already present in the existing model so that you know the idea okay how to create the models right let me just go here and say AMA list we have different models let us just go with the Gemma model that I that I showed you in my last video right how to know what is inside that particular model you can just go here and run AMA so and Gemma Tob which is the model that we have and by the way before that also I can just show you this and then I will show you help we know that okay it takes system uh system message it takes the template there is the model file so model file of the model so we are creating the model using model file so I just want to know what is in this already existing model right for that what you can do is you can just go here Gemma to B and then go here and draw model file this is the model file that is shown here as you can see here model file generated by olaso to build a new model file based on this one replace the from line with as you can see it says from and you can replace the model that you want to do and here as you can see it is showing my local local path where the model is is located and this is the template here start up the turnone user if system system end do prompt end of the tone and then there is the start of the turnone model and as you can see a response end of the turnone right and there is the parameters stop when to stop and parameter stop when to start of the turn and when to end of the turn and this template depends upon different uh models just to show you this is for Gemma 2 right I also have the model mistol right I will just go here and I will say mistol and now these are the two different uh things model file here is the path it is the same but the template it takes different template and this takes a different template and also the parameter stop and parameter stop it has different things remember that when you create a model you might need to provide different template uh message and the parameter stop might be also different now we know that what is in the model file file itself now I'm on the hugging face website and in the the block let's say the page right because he has many models where uh there is the gptq and ggf file formats I'm going to show you in this video about ggf and if you are new to ggf let's say on a higher level this is the latest file format which holds all the information of the model when you have the let's say the llms there are different weights associated with it and all different things right because it contains a lot of information but this ggf is a file format which contains all the information and it's easier to use let's say on on a higher level right and I will go inside this Capa Herms 2.5 mistol 7bf right for this uh video demonstration so you can read all the informations here and there is also this prompt template we are going to use this template as you see in my and here there are different ggu file formats the good thing for you is you need to check okay what is the Quant uh method what is the bits what's the size maximum RAM requires and use cases based on your need right but I'm going to use the balanced one here so it's easier to use and it gives the balance uh balance answer and you can see it requires 6.87 GB of RAM maximum RAM requirements what you can do is you can just go inside this and download this you can use any method how you can download this but I have already downloaded this model you can just click the download and it is going to download right so now I will open my terminal so as you can see in the desktop I have downloaded that I will do LS and this is the Capa 2.5 mrol 7B Q4 I just downloaded now I need to create a model out of this GG UF file format right for that we need to create a model file I'm going to create a model file in the VM here and I will paste all the necessary things so the first thing that we need is the from right we need this from and you need to provide the location location is in the desktop in the same folder I'm saying dot slash and the name of that particular GG UF and what are the next things we need to provide are the parameters and the template right here I'm I'm also saying the parameter stop I am start and the parameter stop I am end and the template as it might be different for different and this template is exactly what it is mentioned in this website as you can see here the block has provided the chat ml promp template you just copy this and it's here so once this is done you just save the file in any editor you want and the model file is being created if to see if it is created or not cat model file so yeah all the necessary things are here now we can create a model giving our own name out of this existing model file how to do that you can just do o Lama create and you can give any name for [Music] example model something like this enter so it is going to create that particular model for us so yeah it's already being created and now you can run this model how to run the model you can just sayama run and you need to provide this name here right just copy the name go here crl V enter and now you can have the conversation with this model same as how you used to do with other models I can just say here hi and yeah hello how can I assist you today so I will say here what is Twitter and it is going to provide me what is Twitter kind of things but the good part of this particular model is also that it understands the previous or let's say remembers the previous question that we asked for example here I said what is Twitter and this can this might not be the case for all the models that you create but for this cap bar heres it remembers the previous message let's say the question we have I say here what is Twitter and it provides the information about the Twitter I can say can you uh say more about it just something and it says of course there is more information about Twitter yeah that's great now we can run this in our terminal but what if we want to create a simple chat you right for that as I said you before also we are going to use the same code of this create chart UI using chain lad Lang chain AMA and Gemma model but we can replace the Gemma with the one that we created that's what I want to show you now I can just go here and say byy so that I can just go out of this I will clear the screen and here I have that particular git repo being downloaded Lang chain Gemma AMA chain lad right so I will just just say here o Lama list to copy the model my own capy model is the one that we created I will copy this and now I will go inside this Lang chain GMA chain lit I will open this in uh let's say vs code so code dot right and it's easier for me to show you what we need to do I'm not going to go through the code again because it was already being shown in my last video but you you can just go to this model part here and now just replace this with the one that you created that's it you don't need to do any other things and if you don't need these elements because that is what I'm showing about the Gemma right you can just go here and just let's say delete this also I will save this one that's it so LS so this is the file that we need to run right so I will just say chain lead run Lang chain gma. P just run this one it is saying loaded elb your app is going through this and yeah this is the simple looking chat UI I will say here hi so yeah you can see greetings as you can it's just providing the answer I can just say okay what is Twitter again here so okay this is this Zach dors all the different things 280 characters and it is providing the answer for us here right so yeah you can just go here and ask as many questions as you want that is how you can quickly create the model and use that to create a simple looking chat UI so yeah I hope now you can use different models which are in the hugging face website create your own model using the AMA and just use that locally if there is anything else you want to know please ask in the comment section and other than that thank you for watching and see you in the next video

Channel: Data Science Basics

Views: 1,866

Keywords: openai api, code, chat ai, large language models, llm, chat, langchain, langchain tutorial, langchain explained, what is langchain, chat models, chatgpt like ui, chainlit, open source model, ollama, chat ui using ollama langchain, run llm locally, langchain ollama, gemma, ollama gemma, run gemma locally with ollama, gemma chat ui, fully local gemma, ollama create custom models, ollama plus huggingface, how to add custom models with ollama, huggingface model, huggiingface, capybara, ai

Id: TFwYvHZV6j0

Length: 10min 54sec (654 seconds)

Published: Sun Feb 25 2024