How To Run Llama 3 8B, 70B Models On Your Laptop (Free)

hi this is faran and welcome to another video uh today meta released their latest uh llama models Lama 3 in fact this is a major leap compared to Lama 2 uh they released their model card and they uh show their benchmarks and from these benchmarks we can just see that it does absolutely amazing especially the the 8B model which is much better than the Gemma 7B model from Google so in today's video we are going to be looking at how you can run um the Llama models locally on your laptop and it is simply very easy to do so in fact I have written an entire tutorial on this uh if you can check it on our website it's called School of machine learning I will link it in the description box below so you can definitely uh check it out so to start it off um we can head over to ama.com and from here you're going to download uh AMA since I have already done this uh I'm simply just going to check out the 3B uh Lama 3 so once you have downloaded uh you will go through the entire process in fact I have written this entire process on the um on my website so you can definitely check it out there anyway once you have done this uh once you have downloaded AMA uh you're going to run AMA and then you're going to open Terminal so so uh Lama 3 comes in two variations in fact it's going to come in three varations but as of making this video we have the 7B model the sorry 8B model uh 70b and then they're releasing another model which is their most powerful one it's the 400 billion parameter model uh somewhere around the next month so in this video I'm going to show you how to download the 8B uh model because uh my laptop has 16 GB memory so that's the maximum one I can use but if you have uh 32 GB memory you can definitely uh use the 70b model so once you have done uh uh once you have installed AMA You're simply going to go on your terminal and type AMA run and this command all of these will be in the description box below and then it's going to uh basically download it so let's wait for it to be downloaded so it is almost uh downloading the 8B model and uh we are that's it and we are yep so we are pretty much done with downloading the the Lama 3 8 8 billion parameter version and here you're simply going to type let's say um why is the sky [Music] blue so as you can see here it's pretty much running locally on my laptop in fact I can just go ahead and turn off the Wi-Fi here and it will still run okay it does uh in my opinion I have not gotten to test it that extensively but on the limited testing from what I've seen online from various people it does an absolutely amazing job in fact the Llama uh 8B version is almost on par with the Llama 270b version which is absolutely amazing so we can just say like can you summarize it so yeah there you go it is as simple as that and let me know if you have any uh questions uh in the comment section below

Channel: School of Machine Learning

Views: 14,626

Keywords:

Id: ZQaUJJfio5E

Length: 4min 12sec (252 seconds)

Published: Fri Apr 19 2024