Integrate GEMMA AI on Raspberry Pi 5 | Easy Tutorial

[Applause] hey everyone welcome back to helon explanations today we have an incredibly exciting project lined up for you I mean just for you I'm going to show how you can run gamma an advanced language model developed by Google Deep Mind on a compact mini computer which can just fit in your pocket named Raspberry Pi 5 imagine the harnessing power of cutting edges on a tiny Affordable Computer it's like bringing the future right to your desk right whether you are a tech Enthusiast a DIY hobbyist or just curious about AI this video is just for you so let's get started and dive into the fascinating world of AI on Raspberry pipe before we jump into running gamma let's first understand what is AI and how it works AI or artificial intelligence or now apple is naming it Apple intelligence refers to the simulation of human intelligence in machines that are programmed to think and learn like humans AI Works using a various techniques including machine learning where systems can learn from data and improve over time machine learning models can be you know supervis unsupervised or reinforcement learning based one of the most exciting applications of machine learning is nlps natural language processing which enables machines to understand and generate human language language models like GPT or olama or gamma are part of this field and are designed to generate coherent and contextually relevant text based on the input the receive first let's talk about Gemma and why is it so impressive out there it's a family of lightweight architecture thing and uh it even runs on lower RAM based models also so that you can even run this model on your Raspberry Pi 4GB also that's not an issue because you need at least 8 GB of RAM if you want to run llama 3 or lava 53 anything out there you at least need 8 GB of RAM but this thing even runs on a 4GB which comes with 2 billion parameters tiny llama which comes with 1 billion parameter or 1 billion tokens you say We'll first test our raspberry with tiny llama check whether it is working properly and we'll also get the benchmarks of tiny Lama which is working on our raspberry we'll get this benchmarks compare your benchmarks with mine mine won't be too great but yeah it'll be at least better than the average right you compare with mine and you check whether your raspberry is compatible with running Gamma or tiny Lama or anything it should be able to run tiny Lama of course but gamma you have to check with that because you know if your raspberry has some Ram issues or memory allocation issues isues you can run into trouble because your raspberry wouldn't be able to allocate sufficient memory to run that particular code of Gemma model so that you have to swap your memory by running a script or a file I'll tell that in the future if you want you can comment on in the comment section about that I'll give you the link to download the script and they can run this script so that you'll free up or swap your memory in your p first let's talk about Gamma and Gamma is a language model developed by Google Deep Mind as we all know that designed to understand and generate humanlike text on the other hand Raspberry Pi 5 is a compact and Powerful mini computer right which is perfect for our daily life projects or know di projects like when you hear about an AI what do you think you think is it a thing which computes all your Excel data or is it a thing which completes all your assignment or homeworks which your college or School gifts or your boss of course or is it something which completes your code or help you in your research purposes it could be all of that we are just using AI to enhance our work and our productivity previously before 3 4 years we used to do Excel data analysis or you know generate a 400 words of essay with our mind used to take at least 4 to 5 hours of time right but today with this all Ai llms and all those we'll just take 2 to 3 minutes those 2 to 3 minutes also will go on editing our text into human readable thing so that our boss or you know our Superior don't catch us with playism check right so we spend our time doing those modifying the generated text by gener generative AI let's see more about this and we'll now directly jump into configuring and running our llms in our Raspberry Pi and we'll also see the use cases of it which is very important if you do something please first know the use case of that and then start your project so that you'll know where you are going otherwise you'll your journey will be like without destination you'll be moving on and moving on and moving on so first find your use case and then proceed with your journey for my use case I want to integrate all these llms or these AI models into my sensors or when I get my data from the sensors I want to use these llms to generate a file which gives me analysis of my incoming data this is my basic use case this is one of the use case we can find more right let's move ahead and then first we'll start with llama tiny llama my raspberry is 4GB model so I can't run Lama 3 and 53 and all those gamma which is the original one as which gives you at least 7 billion parameters I think that could also not be done with my raspberry so I can't run that also you but you can do it if you have an 8GB Raspberry Pi model I'll provide the links in the description to buy those models so that you can choose your own Raspberry Pi and you can buy them let's go ahead to get started we'll use AMA to integrate our required models by Linux interface so we'll be using Linux as our OS right in our raspberry so that we'll use Linux thing make sure you at least know the basics of Linux I mean how to switch through directories how to know your memory usage and uh how to manipulate your memory allocation of your raspberry these are very important because I might not know your error and uh your troubleshooting case in that case you could Approach at GPT but it it'll give you a general solution it doesn't give you your required solution right it will give you a solution which is common for everyone which might be common for everyone but in your case it could be little different so that make sure you at least know the basics of Linux Basics is enough you can ask the basics code or commands of Linux in chat gbt that could be good these are some models which they have created gamma 2 is the latest one I'll just show you an overview of that we won't be using this but you could actually you could use that 9 million tokens is there uh it is 5.4 GB you could use that that will be wonderful and there is one more thing 27 billions also this is around uh 16 GB and this one is around uh 5.4 GB that's why I've said you could use an SSD for your raspberry to boot up if you don't know how to install your raspberry or boot your raspberry with an SSD you could just view our previous video on how to boot your Raspberry Pi with SSD it'll be on the cards on the top or at the end screen at the at the end no worries we'll move ahead this is Lama 3 and this is gamma 2 they have shown the benchmarks of those and yeah two sizes 9 billion and 27 billion parameters thing we won't be using gamma 2 if you want if your model is efficient and it is capable of using gamma 2 then you can go ahead with that your at your model I mean your Raspberry Pi at least should have 8 GB Ram to run 9B but about 27b I don't think so any Raspberry Pi can run that because it's too big I think it could pull off 9B you could do that but our thing for today is this one we'll be doing 2 B1 the best model and they have given a note here about it to run you need at least AMA 0.1 26 we'll be installing the latest one of AMA itself that shouldn't be a worry about it and these are some parameters you can look at for running this model your device should be capable because the minimum RAM it does require is 4 GB and this is the command we'll be going to use to pull gamma which is of 2 billion parameters first we'll install AMA and then we'll move [Music] ahead so first you have to download AMA in our local machine so I'll provide you the link in the description below so you can download this AMA in Linux by using this command just copy that command if you want ama if to run on your Mac or your windows they have given options for those also your Mac o should be at least Mac 11 or BX later or for Windows they have given a preview they've launched a preview recently so you can download for Windows Also let's get back to this raspberry thing I'm connected to it via SSH you can use VNC or your external desktop anything anything would be fine we just need a terminal just paste that command here and then run it it'll download the AMA for you and now after that just type this command and help if you say this then you have successfully installed your AMA on your machine now let's see what models do we have with the installation these are my models they won't be available for you I've downloaded all these on my machine but if you don't get anything that's fine that should be normal and next what we'll do is we'll try to install AMA model tiny llama which uh which will be done with this command Ama pull tiny Lama this could download the tiny Lama thing after doing this it'll download this I'm already done with that to run your model just type AMA run and that that's it there you go your large language meta AI is now running locally on your raspberry that's great right so you can test this out let's say I want to know how many colors are there in the rainbow let's see that's great and let's say why is the sky blue you have to take care on how fast your result is being generated by it because if your result is delayed that means there is an issue with your Pi or if your Pi is continuously making fan noise if you have cooling system for your pie and it is continuously making that fan noise that should indicate a trouble this is the tinier version of llama thing let's say uh let's ask its own full form let's see if it does satisfy this after this I'll show you my benchmarks of this llama thing okay I don't think it is connected to the internet for latest information it is just giving me some other things [Music] fine to exit this you just click this it will automatically exit and type this command to see the benchmark or the time taken to generate a message let's repeat the [Music] [Music] command oh there was some type of yeah now it is starting the generation and after the end of this generation it will give you the benchmarks you can see my Benchmark and you can compare it with yours to see that everything's running fine fine yeah my total duration was 19 seconds and load duration was 2 milliseconds prompt tal count was 41 tokens evaluation rate was 13.62% second okay now compare it with yours and let me know in comment section how was yours running with this and there was a comment in the previous video saying that how to integrate this llama thing with our halo AI module which have discussed in the previous video how to use your Halo AI module and thing I just want to discuss about that over here because Halo is recently launched one and I'm still figuring out the commands to compile something into that the there are some few troubles with that when I've gone through this compilation and integration with this AMA thing into the Halo a module what I found was first I couldn't find a specific compilation command which I could use to compile this AMA and run locally on that Halo AI accelerator actually it's not a processor it's an accelerator so that could be some complicated and this Halo SDK kit it's not mentioned anywhere but it will come inbuilt for you when you install this sud sudo app install hello all it will automatically get installed on your computer right so to figure out that SDK and to get the commands and to then integrate your AMA into that Halo I just need some more time to work this out because I don't find commands which could run on this Linux to integrate AMA into our haloi module I just need some time I'll take at least 1 month or 3 weeks or 2 weeks maybe but I'll sort this out I'll make a separate video on that on how to run your olama or any llm on your Halo module there are many use cases with this llm I could show you some more let's say if I want to know what's in my P picture I've downloaded a nature picture for a test case you can download it from Google itself I mean your chromium thing from your Raspberry Pi OS so you need to be at least connected with your Raspberry Pi through Raspberry Pi connect you can use start r Pi connect and if it is asking to download your RPI connect and if you are using this command for the first time please refer our previous video to know how to connect your Raspberry Pi using the official Raspberry Pi connect and without using any VNC VR or that kind of let's run that llama thing again I'm still using tiny llama make a note of it we are not yet dive diving into gamma we'll first test [Music] it just type this what's in the picture and then go to your picture double click on it and then copy the path of the picture and paste it here let's see if it does gives us the info on that's great actually the image I've provided is a nature one which I've mentioned before and it clearly describes my image it contains a landscape scene with trees Hills and a lake the subject matter is nature with the overall image conveying peacefulness tranquility and serenity that's great actually it could scan my image and give me the output from that image we can use this output and we can TR transfer this output into something else and use this as a part of a huge project also so we are just building block by block and block and block so that we could someday merge up all this and build something we also have something called uh YOLO which is something out there YOLO X is the latest one it's so advanced in object detection thing we'll test this YOLO in the next video as I have said when I've have run the haloi module and uh the object detection thing example detection example it did not recognize my tube light and my airpods or if I just show a scissor or knife scissor it could recognize but knife it told my knife was a baseball bat So to avoid all this YOLO is the best one you if you try to integrate YOLO in your uh raspberry and run it with your Halo a accelerator because why I'm um mentioning YOLO here is that they have created the GitHub repository to download YOLO in your Raspberry Pi and run it with your Halo accelerator they have also given us the instructions that we need to do all this and YOLO X is the latest module which can run on your raspberry now keeping all these aside we'll talk about this more in the upcoming video the video title would be Advanced object detection with your Raspberry Pi yeah even I waiting for that now first test this gamma whether it could give us the same output as Tiny llama yeah if tiny llama does that gamma can obviously do that let's again list our models I have gamma 2 billion model I also have lava and Lama 3 I've downloaded them but it's of no use they take lot of time to generate text now let's run our most awaited model just type this you can just type that type that command and uh if you don't have GMA 2 billion thing module it will automatically download the module and then run it for you if you just want to pull the module or download the module and don't want to run it you can just use the command pull instead of run there you can substitute the word pull if I run that now let's copy the same command we've used before this can take a time like more 5 Seconds thing so it is saying that it is unable to access external websites or specific files I don't know why but yep that's the answer it is given Let's test the benchmarks of GMA and Tiny Lama yeah and before that a huge Congo that now we are running GMA on our raspberry let's try to run that again and let's test with it let's at least play with it for 5 seconds let's give it some questions let's see whether it can answer that so it is saying that my question questioning was wrong and it is saying my questioning was wrong that's fine because it is running successfully on our machine and that's it we have successfully completed running two models on our local machine one was GMA and other was Tiny Lama which was completely fine they were doing good good and I've missed one thing I've wantedly did this because I did not run the benchmarks of GMA and Tiny llama together again but you can do that you can do and let me know in the comment section below of what model is best among these two whether it is Tiny Llama Or gamma 2 billion parameters thing I definitely say it is Tiny Lama because it's just uh 670 MB to download and run run so that it'll definitely be the first one to run faster than this gamma thing on your Raspberry Pi which is a 4GB model but yeah please let me know know in the comment section below and we'll see you in the next video with more models which could run on our Raspberry Pi using Halo AI next time we'll definitely make the models run with haloi module before completing this video I would like to cover some more things and then we'll complete it this is the one I wanted to complete this video with this is this is the one the YOLO X [Music] version and this is its object detection thing it could detect orange a folk and many other things ports ball a car a bus a a person a bicycle truck stops and also that's great let's test this I'll give you my review in the next video and I'll also show you how to run this they've also given some benchmarks of this they have compared this with their previous models that they have created YOLO X is the latest one right so they have compared it with yolo V5 darket 53 YOLO V5 darket 53 and YOLO X Nano all those models Yola X tiny is also there we'll test all this let's see if YOLO X does run on our model and these are some benchmarks that they have given for all those models out there which they have created this was the one I was talking about to quick start this they have given us the installation installation guide or demo thing and reproduce our same results on Coco I'll brief about this Coco and YOLO in the next video this is just for a you know trailer thing so that you could wait till our next video come and then you you watch it with curiosity I am excited the same as you this is it this is for the video if you have any suggestions that I could improve in this video please let me know in the comment section below and until then signing off [Music] [Music]

Channel: HelEx

Views: 604

Keywords: #HAILOAIAccelerator, #RaspberryPi, #AI, #ArtificialIntelligence, #MachineLearning, #DeepLearning, #TechTutorial, #AIProjects, #AIOnRaspberryPi, #TechUnboxing, #RealTimeAI, #AISetup, #TechReview, #DIYTech, Gemma, Google AI, GemmaAI, GPT, Ollama, llama3, llama2, llama, tinyllama, Llava, ollamamodels, RaspberryPi5, OllamaAI

Id: NxNntbHI2D8

Length: 30min 41sec (1841 seconds)

Published: Mon Jul 08 2024