GPT4ALL: Install 'ChatGPT' Locally (weights & fine-tuning!) - Tutorial

hey welcome back in today's video I'm going to show you another large language model that you can install on your computer no high-end Hardware required just your CPU not only do you get the fully quantized model you also can download the weights and the raw data as well so really really appreciate the work by nomic AI so I'm going to show you how to install this on your computer step by step and then I'm going to show you some examples of it actually in use but for today this is absolutely free let's get into it and before we get into the install let's look at the actual performance a lot of people from the previous video complained that the alpaca model really wasn't performing well but what I've found from the GPT for all Laura quantize model is it actually performs really well on par in my opinion with a GPT 3.5 so here are a few examples one thing to note is that everything is done through terminal so there's no UI yet I did put in a suggestion that maybe they should integrate into Dolly I think that would be really cool but let's take a look so I typed tell me a joke and it did okay what's brown and sticky before I cleared my screen I actually said tell me a joke and it gave me the full joke here it's starting with a joke but it's actually telling me to insert the punch line which is kind of weird next I said write Ruby code to count to ten and it did puts counting down four and it actually does it this is good Ruby code not only that it actually outputs the description of what's happening in the code which is really cool next I said write a poem in the style of Shel Silverstein about llms and here it is I am an llm I have been through hell my life has not always been so Grand but now it'll never be again so I won't read this whole thing but you can see it I asked it who is Elon Musk it got most of this information right which is pretty cool next I asked it what Chachi PT is and it didn't quite understand that I was asking about the open AI version of Chachi BT but it is saying character level text generation with pre-trained Transformer which is accurate next I asked what the cutoff date for the data was and it did not have an answer that was interesting I did jump in the Discord and asked what the cutoff date was and they said it was the same as chat GPT 3.5 turbo and then last I said tell me a story about peanut butter and it output a story about peanut butter so let's test one more thing before actually going into the install what are the first three constitutional amendments first amendment protects freedom of speech the second amendment guarantees a citizen's right to bear arms and finally the third amendment prohibits quartering troops and private homes during peacetime without consent from the owner or occupy so there it is seems to do really really well the output is fast it's accurate and I've tested a bunch of stuff and it seems to work really well so let's actually get into the install here it is this is the GitHub for GPT for all binomic AI so the first thing you need to do is clone this repository and to do so come to its GitHub click this green code button copy the URL and then we're going to switch on over to our term terminal now you can put this folder anywhere you want but what you're going to do is type git clone and then paste that URL now I've already downloaded it so I'm not going to do it again but what you would do is hit enter and it's going to clone it to your local computer it really should only take a few seconds because you're not actually downloading the model itself you do that separately which I'll show you now so on the GitHub page you're going to scroll down you're going to look for the try it yourself text and then right here it says clone this repository down which we already did and download the CPU quantized GPT for all model and that's a link right here now this is a pretty big file but it's not huge it's only about four gigabytes so you can download it it'll take a few minutes so this is the GPT for all folder that we cloned we're going to open it up and you're going to place that GPT for all Laura quantize.bin file that you just downloaded in the chat folder the next thing it says is there's actually only two commands one if you're running on the M1 and one if you're running on Windows or Linux and so what we're going to do is we're going to grab this we're going to copy this is the command now I did run into a hiccup and I think they're already fixing this if that if not already fixed but you'll actually have an error loading the model if you don't navigate to the chat folder so I tried to do it from the root cloned repository and I got an error and I went in the Discord I asked how to fix it and they said just navigate to the actual chat folder and then run just this part of the command I'll show you that now okay so I'm in the chat folder as you can see right here so what I'm going to do is dot slash and then put that Command right there I'll hit enter it's loading the model and there it is we're in so this is the prompt right here and we can type anything we want let's say what is the day after Saturday Sunday and we can say how many days in February 28 days in February because it has a leap year that's cool it actually gives you much more information than just what I asked so one thing I'd like to be able to do is actually use temperatures so I see it's outputting the temperature right here but I want to be able to set that temperature when I went into the Discord it said that the only way to actually configure temperatures if you're running this on a GPU I'll dig more into that and I'll leave information in the description if that's not true and we can actually do it from here but I haven't found a way yet so one other cool thing to point out is that this is actually trained on 800 000 GPT 3.5 turbo generations to train llama so they've basically taken what alpaca did and kind of reproduced it they're using GPT 3.5 turbo based on the Llama core data set and they're using it to train it to be an assistant style large language model and it works extremely well I also again just want to give a shout out to nomic Ai and the people there because they open sourced it and I really believe in open source I think one company the keys to large language models is not the way to go so I appreciate all of these different Engineers putting out the open source models so if you see here you can actually run this yourself you can fine tune it on your own machine if you have the right GPU for it and so here it is trained Laura Waits you can download the weights from hugging face you can also download the raw data right now it says they aren't Distributing the 7B checkpoint though so that's just something to note they also give instructions for setting it up locally and training it so here's how to train it you're probably going to update this fine tune7b.yaml with your own custom data and then you're going to run this command and that should do it I'm not going to be able to do it because I don't have the right GPU for it but that's how you do it so a few videos ago I made a tutorial about how to install Dolly which allows you to use the llama and alpaca models locally on your computer a lot of people gave me really great response and feedback from that video and one of the questions that I got over and over again is how to fine tune your I did a lot of research into this and the bottom line is you need a very high-end GPU which I don't have so I think you can minimally get away with a 40 90 but a lot of people are saying the a100 which is a multi-thousand dollar GPU and I just don't have that I saw another YouTuber Martin Thiessen actually use a GPU in the cloud which was really interesting and he had a video about that so I may end up recreating that but regardless if you want to fine tune these models there doesn't seem to be a way out of having to actually Fork up some cash to do so and that's it I hope you got this working and if you didn't feel free to leave a comment below I'll try to help you out and of course you can always dive [Music] thank you [Music]

Channel: Matthew Berman

Views: 96,704

Keywords: alpaca ai, alpaca, llama ai, llama, large langage model, gpt, gpt3.5, gpt4, gpt 4, gpt 3.5, gpt 3.5 turbo, gpt4all, install gpt locally, install gpt mac, install gpt windows, llama 13b, alpaca 13, alpaca 13b, alpaca 7b, llama 7b, artificial intelligence, chatgpt, chat gpt, openai, open ai

Id: DDfUoQWnrfM

Length: 8min 5sec (485 seconds)

Published: Wed Mar 29 2023