motion detection and tracking for the esp32-cam water gun

thank you [Applause] hey everyone so I'm just going to go through the code I've written for motion detection and tracking for the esp32 cam water gun so I'm just going to start by taking you to through some of the theory that I've used um and it's quite simple so what I did is I started by watching this video by um Maker's mash up esp32 cam as a directional motion sensor and what he's done is he's broken the image up into a van into a bunch of vertical strips so along the horizontal axis so use 10 strips and he's looked for the strip with the most activity in it and then that's where he's pointed his um his Servo or his stepper motor and I'm using a camera mounted on a Servo but he was using like a big I or something on a on a stepper motor um so now the way he's extracted his features is pretty good and I've used that but I've changed the processing quite a bit from what he's done um so I'm going to go through that now and discuss that um so what he's done is he's down sampled the image um into four by four squares so 16 pixels all together and he's taking the average gray level over these 16 pairs 16 pixels and he's compared the average gray level from the current image to the average grade level in the previous image for each corresponding square and if it's above a threshold then there's activity in this in the Square now um now what he does is so he's broken it up into 10 10 vertical strips and each strip has broken up into another into another eight strips and then if there's activity in one of those strips so if one of these squares has um has a previous Square which which where the difference is above the threshold then he puts a one here and then if one of these squares along here um exceeds the threshold then a one here and a one here now what I've done is instead of using a one because it is susceptible to noise if you if you activate each of these vertical strips just based on one square so I've done a count so I've counted the number of squares that are above the threshold where the difference between the previous frame and this Frame is above the threshold so that might be three that might be four that might be one and this might be two and then I've sum these all together and I've done the one with the maximum count is um is the one with the mic which is the most active that's where the motion is and the other thing I've done is also if you're breaking the image up into 10 strips um it can get a bit noisy because one strip might have a bit of noise in it which is which is not the actual which is not the actual strip so to deal with this noise what I do is I break the image up into three strips so I start out with three strips and I find the strip which has the most which has the most um which has the most Motion in it so whichever one is the most active out of these three so say it's the left strip and this one has the most activity in and then what I do is I break this I break this left strip up into a further three strips so one and I find and I do another search on these three to find which one of these three has the most motion and I found that that kind of eliminates a lot of noise and um and Smooths it out a bit so say it's this one out of these three with the most motion then this one is the is the strip where I where I point my Servo and camera to so I'll take you through the code now but before I do I'll give you a demonstration on this little test rig I've set up uh which I'll just set this up for debugging and tuning the parameters before I connect the actual esp32 to the electric water gun and this is a little bit more convenient because I can just tap this at my at my desk while I do the debugging so I've got a little test ring set up here which I'm using for debugging which is just a pan tilt mechanism that I got off AliExpress with a esp32 cam attached to it and I'm just going to press um I'm just going to press track on the controls and see if it tracks me all right so here's the controls uh this is the streaming controls and I've added a button track and track off so I have to press the track button to start the tracking all right so I'm just going to walk around and see if it tracks me okay so that's pretty good so I'm just going to go over the code really quickly just the outline of it because I have put it up on GitHub and if you've got any questions you can leave them in the comments so where I've put the motion tracking algorithm which I got the original code from maker Mash Up I put it in in the Stream Handler and the reason why I've put it in the Stream Handler is because this is where the the image gets captured and this is image gets captured by the camera that I need um for the motion tracking so now the stream Handler is like running on a separate process so it's kind of running on its own its own task basically to The Loop and in the loop is where I put the move Servo functions to move the servo left and right so the way I communicate between the stream Handler routine and the loop is through a bunch of global booleans so tracking on basically just means that the tracking button is being pressed on the web web interface page and finished tracking is true if the servos are finished moving because I don't want to mistake the motion for the servos with any outside motion that I'm trying to track so while the servos are moving we don't run this part and then first capture that's just different that's just if we're initializing it so all that does is initialize the previous frame with the current frame and we call that we set that to True after we've moved the servo because after we move the server we're kind of starting to track motion again and when we first started so it calls this update frame which just copies the current frame to the previous frame and then if we've detected motion this returns true and then we set do tracking to true and finish tracking to false and these get returned back to the loop um so I'll just show you the loop um so if do tracking and tracking on is true then we move the servo so we move left if we've detected a region of interest on the left hand side otherwise we move right and then we fire the water gun and we wait 100 milliseconds just to let the servo settle so that we don't um we don't detect the motion of the servo so like I said um any questions just leave them in the comments and thanks for watching and I'll hopefully see you in the next video

Channel: Jonathan R

Views: 10,858

Keywords: esp32, esp32-cam, motion detection, motion tracking, electric watergun

Id: NK1wj7sDLNc

Length: 9min 5sec (545 seconds)

Published: Fri Dec 23 2022