Local AI Just Got Easy (and Cheap)

I finally got the coral AI mini pcie accelerator last week and it totally blew my mind it's a chip built by Google designed to run AI workloads on the edge and this Nifty piece of Hardware is perfect for home automation systems that depend on computer vision like Intruder detection or building your own DIY Furbo now my objective was to get this thing running on a single board computer so allow me to introduce you to the zimma board and the zimma blade and what I like about these is that they have full p p c slots ready to be hacked on today unlike some sbc's Raspberry Pi 5 Jokes Aside it seems that Pi 5's pcie capabilities are still awaiting some peripheral adapter support so for now I'm forced to chart my own path now I did get in touch with the folks over at Pineberry and I picked up each of these hats here designed for the Raspberry Pi 5 and I'm pretty stoked about it cuz they're going to support gen 3 speeds but for now I'm still waiting on my order now this ship has an M2 form factor with an AE key so the mission was to somehow marry it with my zimma blade so after some intense office funking I Unearthed a Wi-Fi card with a pcie connection which had an M2 slot now if you don't have this set up no worries any standard Raspberry Pi with the USB TPU accelerator will work just as well but the thing about the USB version is and I actually didn't know this usb3 only has a throughput of around 600 mbes per second so not only is my set up half the price but it's technically faster it's also cooler literally thanks to the better Thermo regulation than the USB version finally this approach also leaves open the USB port which gives us more room for activities but this magnificent piece of Hardware is going to be useless if we can't get working software to wield its power so I booted her up and ran the installs this comes with an operating system called Kasa OS which is a flavor of Debian but we're just going to use a stock version of deian 12 which is like the most recent version of deian we're just going to overwrite the operating system on this so I'm just going to flash uh the the operating system to this thumb drive we'll boot it onto here and that way you can follow along because you could use that version of Debian on a Raspberry Pi a Mini PC spare laptops Etc and then this tutorial will be a little bit more agnostic for you okay so the way we get Debian 12 onto our zimma blade is by putting the iso image onto this thumb drive here flash it on the thumb drive then we plug the thumb drive into the zimma blade turn on the zimma blade press f11 until it gives us uh the system bios and then from there we can um write this operating system to the onboard storage and install it and be good to go okay so we now have our fresh install of Debian 12 and so I I could um open up the terminal here and start programming but I actually want to use warp so I'm going to connect remotely from my laptop and I'm just this interface is going to be faster for me and we will start installing our programs okay so I'm going to go over to clients right here and we selected the host name Debian so I should be able to use this IP address to connect so I'm going to pull open a warp terminal here and I'm going to do SSH and I did Tim as the username okay so this is our fresh uh Debbie and install so let's get the TPU working first so I'm going to go to Coral AI accelerator M2 so I can see the installation instructions documentation so we're going to plug in some of these commands here download the drivers all right so let's check well if we check for the TPU it's not connected right now so it's not going to show anything um and then for the PCI [Music] driver okay okay so the TPU not being connected is fine I haven't plugged it in yet but the driver here we did download the driver but it's not showing here and what I found is that we need to disable secure boot for that to work grab this command here okay so technically we disabled secure boot so what I'm going to do is plug in the TPU and then we'll reboot the system and everything should work at that point all right so let's try running these commands now so it should recognize the TPU okay there it is coral Edge TPU looks good should recognize the driver as well now so let's give that a shot there it is looks good if we just wanted to hack using the coral AI stuff then we could download pie Coral unfortunately it looks like the pie Coral library has been neglected so so if you try to run it on any recent Dro you're going to have a bad time it's basically telling me that I need an older version of python in order to get it to work which is kind of lame so I'm actually not even going to use py Coral because frigate can use the TPU out of the box so let's just go right to to using frigate so now that our system recognizes the TPU what can we actually do with this hunk of cyber black magic enter frigate and no I'm not talking about a level in Golden Eye frigate's this awesome home lab computer Vision system that works seamlessly with the coral AI Hardware score but will it actually work on our Debian 10 shimmed Wi-Fi adapter turned pci2 slot running on an x86 Tech Marvel that sounds like it escaped from a Netflix series plotline probably but we got to put that to the test so frigate uses Docker so let's go ahead and download Docker I'm going to grab this Command right here and then we are going to run this guy guy all right and then I should be able to list containers we have none running can list images we have none but at least Docker is installed and ready to go so let's go over to the frigate documentation I'm going to just use a Docker run command with a bunch of flags cuz that's what I have set up already yeah I'm I'm just going to use this guy here so one of the dependencies is a service called mosquito or mqtt uh and it's how uh messages are really and that's how events are captured so let's go ahead and set that up as well so let's do this this one all right this is the primary service and then we can enable the service and uh check the status so it is running it looks good one thing I did note though is we want to edit the comp file so um and we want to add just two flags in here okay so we just identified the port and then we said Anonymous Origins can listen not just Local Host so let's go ahead and restart the service yeah let's use this right here so let's start Let's uh run the frigate container so you can pull the commandline argument from the getting started guide on frate this is what I'm using here because uh that's the driver for the TPU and everything um also it expects a config file under a folder called frigate so actually let's go ahead and create that so go a root directory I'm going to make a folder called frigate and then I am going to make a config.yml file and this is actually where you instantiate all your different cameras zones and like protocols and things like that this gives you a good example to go off of so let me throw this in there so what I'm doing here is the mqtt server uh I enabled the true I think it's set the True by default but I put my IP address to get your IP address do just do host name I it's going to be that first one that's the port that we identified 1883 for detectors I'm using the PCI htpu if you're using a USB grab what that looks like out of the documentation and then I just enable certain things like that it can record that it can detect stuff like that so let's go ahead and give this a shot we can revisit it if it doesn't work but this is basically what you tweak to get frigate to work so to start the container we're going to [Music] run and on the first run it's going to download all the layers after that it'll be faster cuz the uh the image is going to be stored locally and Theory we have a a container running so technically it should be exposed on Local Host Port 5000 uh the web UI that is so if I go over here and I plug in this okay so this is frate this is the um the web web UI and this is where you kind of manage everything this is that config file that I created but now that we can just edit it from here and we can restart the service every time we edit it but our camera has no feed so here's the thing frate expects you to use essentially an IP camera I don't have an IP camera so I'm going to use a webcam now the trick there is IP cameras create rtsp streams that are available on their hosts uh webcam doesn't have a host it doesn't have anything like that it's just a peripheral um but what you can do is you can use some some open- Source software to stream your webcam to an rtsp so that's actually what we're going to do so it's just a standard USB webcam uh I'm going to go ahead and plug it in to my zimma blade okay so basically what we have to do is we have to set up an rtsp server so let's grab this and we're going to unzip that and then there's a command to run it and for me me the one that was working best was actually sorry it was this guy here okay so you do your IP address which is host name I and then I'm exposing it on Port 554 you could technically choose a different port if you want and we should get this feed here this logging here is basically saying frate is trying to find a stream on this path here and we haven't set that up yet so that's the problem frate is trying to connect so I'm going to leave this going you want to leave the rtsp server going but now what we have to do is we have to direct the webcam to the rtsp server we have to take the the live feed and stream it to the rtsp server ffmpeg actually can do that so you have to put your IP address in here uh Local Host IP address in there you have to put the port and you have to give it a path I'm just going to give it a path of my stream and so this should open up the webcam and stream it into our server here all right so that looks promising the frames are coming through technically and you can see okay it's publishing to My Stream So what's going on here is now we have an rtsp stream available at this location here so it's protocol IP address Port path and so that's the whole thing right like for the config file for frigate now we just have to make sure that it's hooked into this and it is rtsp 192168 0218 554 so oh now we're getting an image now this is a static image it will just take a snapshot and it will throw it in there if you want to get a live stream you go to um this section here bird's eye but it does work and I have one detection algorithm running on this person detection so anytime it sees a person it's actually going to start recording and it's it's considered a frigate event like right here it there's actually a recording in progress right now because it detects it detected me and what's really cool about this is if I move the webcam over here it actually stops recording right so now if I refresh this still says it's in progress but now it's done see so again it's running tensor flow models against the camera and then deploying logic based on the result of those models so if I refresh this yeah so now it it captured some more clips right and the inference when it runs the frames against the um computer vision model the inference happens on the TPU that inference time which is how fast it takes to to run the inference should be stored in the yeah right here right so this is our coral one Coral accelerator chip and that's showing how fast the inference is now that's actually a little bit slower than when I run it from the command line but um that might be like a cold inference when the model's not loaded into memory or something like like that so you got a whole Michigan control here and you can add as many cameras as you want if I had a facility I would totally like create a a home lab surveillance system with multiple cameras and like multiple tpus honestly you could remotely manage an entire facility and it's just uh I don't know it's just bringing like the whole home lab thing to the next level so video I wasn't even going to make a video about this but I just thought it was so cool so here we are

Channel: Data Slayer

Views: 246,843

Keywords: Data Slayer, frigate, frigate nvr, coral ai, zima blade, raspberry pi 5, sbc, single board computer, docker, security camera, webcam, rtsp server, mqtt, coral.ai, USB Accelerator, raspberry pi, M.2 Accelerator A+E key, edge tpu, pcie, ffmpeg

Id: mdOEaNV8NXw

Length: 13min 26sec (806 seconds)

Published: Sun Dec 17 2023