How to run ChatGPT in your own laptop for free? | GPT4All

hi and welcome back to Dev explaining Channel today I wanted to discuss the concept of running a large language model locally on your own machine your own Hardware so I will go through the good reasons I will show you some kind of practical things I will grab one of the large language models and show you what you can do with it and then I have some usings and elaborations as always near the end and of course as always if you find Value from this video give me thumbs up along the way drop comments requests feedback below the video so let's get started for good reasons to run a large language model locally right now right today well number one is that it's your own so no matter how awesome is open a chat GPT and their apis and all the other possibilities you have these days still all of those are kind of maintained and maintained by some other party who has kind of their own ideas where they are taking those and you're kind of following along the ride but if you take ownership of something and you take something locally you are free to Tinker with it and take it to interesting directions so you have more kind of things you can play with it and I think if you're a similar person like I am that you like to play and tinker and break things and reassemble them this might be interesting for you but there is also some other good benefits that come from that one now second reason is that it's going to be free so if you take ownership you don't need to pay any subscriptions or or kind of pay for the use of API or tokens you just pay for the hardware when you're on running it so it's going to be very different thing and scalability goes crazy because if it's free you can use it quite a lot more than if you are paying for it and for example for the things where you are not making any money yeah it's more probable to play with kind of free things than than put money on the table so of course if you are monetizing something and as part of that you need an API sure go for it then it makes sense to spend money to make money but if you're just wanting to play and experiment Something Free might be really tempting now next uh thing is that they tend to be more lightweight so if you're running a model locally they are typically not a monster models people are competing how small they can squeeze them and right now you can already take them on a memory thumb drive memory stick you can put them on Raspberry Pi and have your favorite mascot talk back to you using a large language model the large language model is kind of relative the concept it's not that large anymore and final thing is that if you take ownership of these things you know where the data is stored so you don't need to worry so much about kind of sensitive data leaking from third parties as has all already happened and gdpr people like I who is living in Europe when you are creating something and it touches even remotely personally identifiable information it can be a real show stopper if you have to send it to third party website or third-party database somewhere especially if you don't have agreements to cover the aspects of gdpr or sensitive data in common so your own free lightweight and kind of sensitive data compatible it can be depends on how you manage it but at least you have a say in that so those are very good reasons to start getting excited about locally running the models right now so how do you do that I'm glad you asked let's go there I'm just picking one of the options you have but today's option is GPT for all I'm dropping the link in the description section as always with my videos so you can follow my steps if you like but this is GPT for all so of course you have other options for local models as well but this one is easy to approach and there is a lot of interesting hooks I haven't even explored already all of them so um there is nowadays a nice install program graphical user interface if you like that kind of thing but what I did I just cloned the whole repository with Git because I'm I'm a software developer I'm a tech guy so that's what I like to do so I just grabbed it and then what I did was grab the model so there is a few options here but here's the try it yourself part so I used the direct link this is about four gigabyte download so you need some kind of good internet access to be able to download it of course you can use torrent as well but once downloaded you can just run it locally and you don't need to go online anymore so using it is extremely simple it's something like this one let me show you then I get back to this page because there is more interesting information available so here I'm in my clone repository and there's the chat folder where you are instructed to put the binaries and then you have a few scripts for different platforms so I can grab PPT for all other quantized Linux sorry for butchering the pronunciation again and then I grabbed my model sorry I need to put the minus M parameter and put the model here and by the way I have here one crazy on the filtered model that doesn't know what it's not supposed to talk talk about so one benefit for the free information to have something that has a little little bit less of the safeguards or restrictions on top of that it's also more wild than crazy and doesn't give you so kind of safe answers we could really go deep on the kind of need of restrictions restraints kind of safeguards there's kind of different sides to that question but topic for another video another day let's not elaborate anyways my model is loaded right now so the model size is four gigabytes more or less it's a Marvel that people have been able to kind of fit it in such a small space it's a fraction of what I have available already on my machine so here is one of the caveats I've been speaking boldly that you would be running the model here and they say that this model is like level 3 gpt3 level well it kind of is but it's not nearly as good as Chachi PT with model 3 because they're cheap it is more than just the model and the training is just not the same and Hardware is not the same so all the benefits I told you about running a local model they turn around this model is not nearly as clever as chat GPT with all its power so this is more like your inbred causing Billy Bob from the mountains missing a few teeth that kind of level compared to what you have what you can do with GPT for jet GPT level things but let's give it a shot anyway um give me example code on getting my first buy test test going on with python yeah it's not going to do wonders but it can generate code for you it can answer questions for you based on the training it already has so it's a basis it's a platform but it's not refined of course as it's yours uh feel free to kind of grab it and take it further you could retrain it you could fine tune it you could teach it new things but it's it's doing something I know by 2023 standards this is not much if you if you ask some more interesting questions you see the difference more plainly let's say give me good instructions on how I could do clean code and refactoring for my dirty Java Legacy code base so with your GPT it would elaborate and give you kind of Larger than Life answers with this one now I'm not holding my breath but it's still something it's a basis now as I mentioned you can yourself make this a whole lot better because it's an open model it's quite easy to give it few more spins and make it do what you want how you want like for example I've been interested in playing with the simulated personalities and we have none of that available here but we could have of course our second thing is that right now the Moore's Law is of exponential growth of processing capability I think it's now pumping quite wildly in the AI scope so right now all the AI Solutions are on exponent exponential growth path So within a year the locally run large language models will be crazy mark my words remind me in 12 months time let's get back to this in 2024 and see if I'm right but I'm just saying that probably this will this scene will explode and you will see a lot more also how about putting some Auto GPT on top of that and because these are cheap and all models how about having Army of your AIS have it be orchestrated by Auto GPT kind of capability so I see a lot of growth Happening Here in upcoming months and you might be part of that as well but I wanted to wrap things up also with something relating to the data and training so first of all there's a nice technical report describing some details on how it was trained and by the way I've been a bit disappointed because um open AI is right now experiencing a lot of competition so their white papers are getting a bit vague right now they are not giving out all details like they used to do meanwhile the open models might have a little bit more details available and I think this is exciting Direction because if I want to play with something I'm very happy to know exactly what it has eaten so I can expand it and understand it and I'm afraid the commercial models these might be commercial model as well but a lot of this is very open and the model is of course Apache version 2 licensed model so core is very kind of Open Source still so final thing I wanted to show here is a link to all the training data used to make it clever and sorry here so you can actually browse The Prompt and response pairs by the way if you would like to see me fine tune this model with the data from my own website and blog uh let me know because it's a little bit of work but I have kind of I've been getting interested in that one so if I get multiple requests to do that to do a video I might just do that so let me know in the comment section let me know also if you never would like me to do that but there is a possibility to fine tune with your own data and you would get the data to the to the similar kind of structure and then you could use your company intranet or your own kind of chat logs from any any place your own video comment section or your own video transcriptions a lot of options here but I've been tempted to kind of crawl our company blog website there's years of Blocky information and use that as training material to fine tune and I might do that if there is kind of interest in that one okay but anyways I wanted to wrap up with one cool thing here the atlas map so I think this is extremely cool because there's a visualization of the training data so you can actually find the prompts and questions that have been used for the training you can similarly find the answers from the other link let's see where that was here here's the responses so you can find the responses and you can even cure them and you can just browse this kind of cloud of things historic places it in Pennsylvania so I can track that one and then I'm able to kind of see the prompt and response pair so all of those that crazy amount of training that went you can just visualize it here let's see what it knows about Java coding so we can do a search and we can find some prompts and here you can see one of the 500 that I'm browsing and I I as you can see there is a lot of kind of training material material but you can go on this level of transparency and I really enjoy kind of being able to see what has gone in to that machine especially because for example stucco overflow is kind of getting conscious about people using the material for training and I'm betting a lot of other resources so I think in the future it might be very interesting field you have transparency on exactly what sources were used and I like the models where this is clearly documented and I can I can trace it and I think in the future this is getting more and more I like to think that in the future this is going to be an interesting topic anyways uh I hope you got excited like me on capabilities of running large language model locally and by the way whatever you saw me do in command line you can easily wrap a chatbot or a web interface or you can put a speech interface on top of that so there is easy python bindings you can use to just tap into the model and do the same things I just did in the tool so that's very very interesting thing and if you are not excited like me I think you should be because this is all the rage nowadays so everybody would should should be playing with these things a little bit and figuring out their own tenses of things and this is very easy very free and very in that sense very easy to get started so as always let me know what you thought about the video there's the comment section waiting for your comments thumbs up if you didn't do it already and subscribe to my channel for more and I hope you you are enjoying this when I try to keep things short so let's end this one see you in the next one bye bye

Channel: DevXplaining

Views: 1,112

Keywords:

Id: J-LbJkexZrg

Length: 14min 54sec (894 seconds)

Published: Tue Apr 25 2023