Using the Raspberry Pi CM4 for AI Revenge

wow see what i have to deal with redshirt jeff did that yesterday he snuck in my office grabbed the pie for i used for testing and cut the thing in half he said he was angry there was no pi 4 model a and he thinks it'll still work but that's just nonsense so i decided revenge was the best course of action i've neglected my office defenses far too long so i thought the best thing to do would be to add some sort of intruder detection system and i've always seen people building machine vision cameras with raspberry pi's but i've never done it myself i have a spare high quality camera i used for testing and look at this this little box came in the mail and it has one of the world's smallest raspberry pi compute module 4 boards it's the cm4 ext nano from harlab i've built cameras using a pi zero or model a before but they were always a little underpowered using the cm4 with this little board means i get a small footprint but i also get all the power of the fastest raspberry pi model with up to eight gigs of ram i should mention this video is sponsored by arm as part of its virtual conference arm dev summit coming up on october 19th to the 21st i'll talk more about it towards the end of the video so if you want to find out how you can learn about even more amazing projects built with arm devices stay tuned so here's the plan redshirt jeff comes sneaking in here far too often i need to discourage that next time he tries i'll have the pie set up to blast him with an air cannon aka my leaf blower and if i have a camera attached i can also have it record and broadcast the moment for real time revenge but first let's take a quick look at the board itself it's got connectors for the compute module 4 on the bottom and a whole lot of i o on the top it has usb 2 usb-c power input 2 hdmi ports an audio output jack a 5-volt power supply connector a stem qt or quick connector microsd for light compute modules and a reset button and if that's not enough it even has more in the middle there's a neopixel you can control in software mode switches a dsi and csi display and camera connector and even a full 40 pin gpio header that's a lot to cram in such a small board i should note that the 40 pin header comes with an included spacer and if you want to use it you need to solder it in manually that's fine with me though because if you don't need the gpio it's safer to not have the pins sticking out on such a tiny board assembling everything together is easy enough first you mount up the camera to the adapter plate that comes with the ext nano you basically use standoffs to separate the camera from the adapter plate and another set to separate the adapter plate from the axt nano then you connect the camera ribbon cable to the csi port on the board screw the ext nano into the standoffs and pop a compute module on it and bingo now i have the pi and camera ready to go next is the blower luckily since the hq camera has a built-in tripod screw hole it's easy to mount in this case i'll just tape a little tripod on the blower handle there a little gaff tape fixes anything next i need to wire up one gpio pin 5 volts and ground to a servo i have this trusty little servo and i'll strap it to my blower so it can trigger the switch i plugged it into the right gpa opens and now it's time for software the first thing i had to do is actually get the pi camera to work with the compute module 4 things aren't quite as plug and play as the other pies you might be used to first i wanted to use my usb keyboard and mouse but the usb ports on the cm4 aren't even enabled by default after a small addition to the boot config i was up and running for the camera to work i had to download a special device tree file and the instructions for that are on the compute module documentation site with all that out of the way i tested the camera was working with raspy still which is built into pious now that the hardware and software all seem to be working it's time to train the pie how to identify red shirt jeff i needed to find a way for the pie camera to monitor the doorway and detect if someone was present but that's not enough i also wanted to make sure it would only do the air blast if red shirt jeff tried coming in so i found a guide by caroline dunn on tom's hardware she actually has her own youtube channel so go check it out i used caroline's code as a starting point and set up the pi for real-time facial recognition using machine learning first i set up the software i'd need opencv face recognition and imutils these three libraries make it so i can build a model of someone's face then use that model to detect that face in any still image or video as with every project i do all the code is free and open source over on my github and there's a link to it below once you have the software the first step is training a model you have to teach the computer how to identify someone's face and luckily i have a lot of footage of redshirt jeff so i grabbed a bunch of still frames and put them in a folder called redshirtjeff then i ran this script to build the computer model which is stored in a pickle file wait a pickle file is that some weird rick and morty reference no it's just a special file that stores python data structures like our computed facial recognition data the next step is to use that pickle file for live facial recognition i won't go too deep here but this script basically tells the pie camera to start capturing video it grabs a frame looks for any facial matches reacts to those matches then moves on to another frame if that sounds like it might be a lot of real-time processing for a tiny little pie well it is and that's why i've been trying to get a coral tpu working with the pi over pci express because it could speed up things like face detection dramatically the arm cortex a72 on the pi is a great little processor but alas it's not built to be a specialized machine vision processor so i'm stuck at a little over two frames per second you can see what that looks like in this little test video i did of myself it's good enough though because redshirt jeff can't move fast enough to not be detected at that frame rate at least i hope and i also set up the script to record everything to a video file so i'd get redshirt just reaction so the software side is done the next step is to trigger the blower when the pie sees red shirt lucky for me i have some experience building a really simple circuit for this since i built my pi notification bell last year triggered by a servo in this brief hacker montage i slammed out the code to control the servo and make it turn on the blower for three seconds and tested it by training the script to blow me that's not never mind anyways it looks like it's working so it's time to set it up for red shirt jeff now the last thing i wanted to do before letting it loose is make sure i can watch the camera feed remotely the quickest way to do that was to set up vnc so i could just stream the pi's desktop to my re-terminal wait what's that i don't believe i've introduced you to this little guy seed studio sent me this thing to test a couple weeks ago it's the re-terminal and it's generally used for building hmi or human machine interfaces like you'd have set up in factories but it's also a really cool portable tablet style device and it runs a pi compute module 4 just like the cm4 xt nano i'll hopefully have more time to talk about it in a future video so make sure you subscribe to see it but back to the streaming i set up vnc on the ext now and started monitoring the stream so i could see redshirt jeff's reaction remotely ah revenge a dish best served cold and at high velocity now when i came back to my office i uncovered a pretty massive bug and luckily i left my camera rolling so i could show you i forgot that redshirt jeff looks like me so much in fact that the machine learning model i trained thinks that he and i are the same person i should probably also make sure the pie identifies the shirt color to make sure it only blasts red shirt jeff and not also me that might add a bit of complexity but that's why i'm always trying to expand my knowledge so i can make these projects better and better and that brings me to this video's sponsor if you want to expand your knowledge on embedded devices like my arm powered air blaster you need to check out arm dev summit it's free and you can learn from tons of industry experts like raspberry pi talk about things like embedded computing edge and even high performance clusters you can also discuss your interests and network in arms official discord channel maybe you'll see me chatting about this session on how water scope tests water quality and low income areas with a pie if you're at all interested in computing you should register for arm dev summit the event is from october 19 to 21. register online using the link in the description so yeah maybe after that i'll have the time and knowledge to make it so i don't get blasted every time i walk into my own office for now i think i'll have to shut down the blower and maybe think of a better way to get back at redshirt jeff and he told me again he thinks he can get his frankenpie working but i had to go back and buy another one to replace it because even if he does get it working he chopped off all the i o ports and i need those for my testing if you want to build a fun powerful and tiny pie based camera like i did harlab's cm4 ext nano will be 30 bucks and will hopefully be available soon like all projects its production timeline has been delayed due to the chip shortage but i'll update the link in the description as soon as it's available until next time i'm jeff gearling and it's not doing anything oh shoot it's like locked up oh no i also wanted to make it sure make a trad oops because the raspberry pi hope it works my wife tells me that i sometimes don't listen that well apparently i don't read well either

Channel: Jeff Geerling

Views: 77,485

Keywords: raspberry pi, cm4, ai, blow, blower, air blower, face, facial, red shirt jeff, defense, servo, programming, ml, artificial intelligence, machine learning, pytorch, compute module, pi 4, pi zero, coral, tpu, facial recognition, dewalt, arm, devsummit, project, diy, circuit, reterminal, cm4ext nano, baseboard, tiny, numpy, imutils, opencv, python, blowing, cut, model a

Id: Gai_w3uCtIM

Length: 10min 35sec (635 seconds)

Published: Wed Sep 29 2021