Okay. So in the previous Ollama
videos, we've looked at how to set it up and stuff like that. And we've also come in here and looked at
how to install different models in here. The challenge is what if you actually
don't see the model that you want in here? And for example, if it's a fine
tune of one of the models that are in here, but you don't actually see
the model that you want in here. So don't despair. It's actually quite easy to install
custom models into Ollama as well. So in this video, let's
look at how to do that. All right. So the custom model that I'm going to
install here is a model called Jackalope. So I've been playing with this in CoLab. I might actually make a full
video just about this model. but for the sake of this video, we're
going to install this into Ollama. So you can see that this
is basically a 7B model. it's a fine tuning of the
Mistral 7B model in here. And what we want to do is we actually
want to get the quantized version of this. So if we come down, the one
that we want is this GGUF here. so if I click into this. You can see that TheBloke has
converted the weights from this model into this GGUF format. So GGUF is used to be GGML And this
is, a quantized version of models. So this is the project called LLaMA CPP
and allows us to basically run models. It's what Ollama is using for
models underneath the hood. next up, we're going to go into and find
the files and versions of this model. And here I've got a bunch
of different choices. So basically these represent
the quality of the model. They're different quantizations in here. I'm going to go for a reasonably big one. I'm going to go for these Q6K in here, but
you might want to pick a different one. I'm just going to click download here. Now, if you're using git, and you're
quite comfortable with git, of course you can just do you can just
pull this down by a git as well. but here we can just come in
here and we can download it. So that's going to take a
little bit of time to download. but what I'm going to do is
download that and I'm going to put that in my model's folder. so that I can then start processing it. Okay. Now the model is actually downloaded. We can actually see it here. in my model's folder here which I've made. So you can see I've got some other
models that I've downloaded last week. And I made custom versions of last week. So the next step is that we
want to make a model file. So if we come in and have a look at
the Ollama model file, we can see, you know, what the parameters are. and the general thing is going to be
from, and rather than us put in sort of LLaMA-2 or something here, we're
actually going to point to the file that we downloaded, where they're going
to set up, our template, et cetera. and then we're going to use the model
file to actually create a model. So let's do that. So I'm just going to make it a
text file for a model file in here. Okay, so you can see here. I've just sorta prefilled
the model file in here. So we're basically saying from, and
we're pointing to the checkpoint that we just downloaded the model
weights that we just downloaded. So remember these are the quantized
version of the jackalope 7B model in here. we need to put in a template
for a system prompt. So this is the template
that they're using here. and then the system prompts
people can actually fill that out. when they're actually
running the model in here. So, okay, I'm going to save this. Okay, so i saved that out. we can see now i've got my model file
there Now you can see that i'm going to be making a model called Jackalope using the
model file Jackalope that we've got here. So if I run that You'll see
it passes the model file. It's looking for a model. It's going to create the various
layers it's going to create the system layer, et cetera. And then it's going to write the
weights And then finally at the end we'll see this success here. Now if I come in and have a look
at my models Under model list. You'll see that i've got the
hogwarts model which remember is just a different prompt for LLaMA-2. And I've now got this
Jackalope model in here. So if I want to use this model I can
just come in and say, okay, Ollama run Jackalope And it will start up
the model You can see it's running. And now we can use the model just like
a would any other model in here So if I want to ask it a question I can do that. I can converse with it just like a normal
If I want to see the commands of what I can do I can basically come down and
see that and you'll see that it's just like normal for these particular things. So this is basically how to set
up a custom model how to use it .There are probably are some
models that are not going to work. But certainly all the fine
tunes of like LLaMA-2. The fine tunes of Mistral 7B, the
Falcon models et cetera we'll work for this So this is something that you
can definitely use to basically start trying out different models that you
see on the hugging face hub there. Anyway as always, if you've got
any questions or any comments put them in the comment section below. Please click like and subscribe
it will help people see the video. I will talk to you in the
next video bye for now
