Use Facial Detection & Recognition on an ESP32 Wi-Fi Camera [Tutorial]

today we're going to set up and test the limitations of facial recognition on an ESB 32 cam on this episode of cyber weapons lab the ESP 32 cam is a really interesting little board and one of the best things about it is it is incredibly cheap and it has just enough RAM to be able to do on chip facial recognition now this is really cool because it involves two different things at once one is facial detection and the second is facial recognition and there's two burnin processes that allow you to both recognize faces and detect who they are that are built into the ESB 32 now to try this out we just need an ESB 32 camera and we'll also need a FTDI programmer in order to communicate with it because unfortunately it doesn't have a USB interface now today we're going to wire everything up program it and then we're going to test the limitations of facial recognition because as you can see this is a cheap little Wi-Fi micro controller and I don't expect it to be able to have a high degree of distinction between very similar faces in order to follow along you can check out the null byte article in the description to see all the required components but in general you'll need a computer a FTDI programmer and a ESP cam once you have all that together then we can begin now in order to get started with this project you'll need to get around the fact that this doesn't have a USB port so for that we'll need an FTDI programmer like the one you see here and you'll be able to find a link to this in the null-void article now you'll need to also take the connectors for the FTDI programmer and connect them to the ESP 32 cam and you'll be taking the ground and connecting it to the ground port you'll be taking the 5 volt and can be connecting it to the 5 volt on the opposite side you'll be taking the TX and connecting it to the uor and the Rx and connecting it to the uo t now once these are all connected then you're actually not done you can't program this board unless you short 2 pins which the ground and the io0 pin so if I go ahead and short this by putting the second pin here all I need to do now in order to program this board since I've shorted it is press the button which unfortunately I was down the underside here or just unplug it and plug it back in and it'll boot up ready to be programmed so if you get frustrated while setting this up then number one have you gone ahead and shorted this pen because if not then it will not enter programming mode and you won't be able to program it as soon as you have this wired as such then we can go into our Reno IDE and simply flash the board with the default camera sketch today we're going to go ahead and set up this ESP 32 cam so that we can look at the facial recognition capabilities and start messing around with them so this is the sketch we're going to be using and to get here we're going to need to take this JSON address copy it and we're gonna put it into our preferences in arduino ide so when we go into preferences we should see a the additional board manager URL if we click there we'll see a list of all the different board manager urls we have and in my case i already have the ESP 32 index but i can just paste that here and there it is again I'll press ok and what this has done is added the JSON link to our list of indexes that will be updated whenever we look at the available boards so if I go to the tool section and then the board board manager I should be able to now look for the EST 32 boards and this will after I install it make sure I have all the example boards which is what we will need for this demonstration so here we can see there's a bunch of example boards we'll install them and since mine already installed I won't click on remove but once yours are installed then we'll be able to go over here click on tools go to the board manager and then select the a I think re SP 32 camp and once we have that we should be able to go over to file examples and then under DSP 32 we should see the camera example and from there the camera webserver and that will bring you to the sketch here now we can go ahead and make sure that this line here is uncommented meaning there's no slashes in front of it and that will select the pins for the AI thinker model and then here go ahead and put the network name that you wanted to connect to your Wi-Fi network and the password right here so once that's all done you'll need to find which port all this is connected to alright so in a terminal window what I'm gonna do is type DME SG to find out what port this is connecting to so I can see that it's connected to see USB t2i USB 3 so I'll go to tools port tty USB 3 and this is now selected so the last remaining thing is make sure that you short the pins from the i/o 0 to ground because otherwise this won't be able to go into programming though so assuming that you've done that and it's now connected and we have the right board selected and we have the right port selected go ahead and press upload we should see that it compiles the sketch and then sends it to the board and if this is successful then we will go ahead and unsure the pin that we were shorting before and that should allow us to then use this let's go ahead and see if it works there we go and it's now uploading so I noticed that it's really hard to reach the pin are they button on this sometimes if you have it plugged into a breadboard as ideal so I just went ahead and unplugged it and plugged it back in and since I shorted it it actually won't go into programming mode until you cycle it or power cycle it unplug it plug it back in or press the reset button either way you just need to make sure that it restarts before it is able to accept programming so this is going ahead and continuing to send the file and as soon as it's done then we should be able to look at the serial output to see exactly what's going on and if we've successfully been able to join the Wi-Fi network so it looks like we are now at 100% so we'll press control-shift m to take a look at the serial output and of course we don't see anything because we are still shorted so I'm gonna unplug the pin from being shorted I'm going to unplug and plug this back in really quick there we go it should now be in the correct mode it'll start up and let's see if we can get any output there we go so we now have a camera streaming on this IP address cool so we're going to go ahead and go to this Wi-Fi network and as soon as that's complete then we'll just copy this address right here and take a look at the camera all right I'm gonna go to a web browser alright and here we go so let's go ahead and start the stream and I'm gonna go ahead and up the quality a little bit than most we can do is four hundred by two hundred ninety six and again this isn't the best quality but it should be enough to get us started so alright here we go we have created basic capabilities here the stream is not very fast and it's not a beautiful camera but if we go down to the bottom here we should be able to start face detection and this is the beginning of what I'm talking about when it comes to some cool stuff we can do with this now I now am being detected by the camera and if I go back to the Arduino IDE serial output you can see that the detection czar being logged so if I were monitoring the serial output I would be able to use these detection I could even probably just correct for this if I wanted to make a really simple bash script and by monitoring the output on the serial port I could make a program that detects faces so maybe I could tell when someone is outside my door or something like that now that's pretty cool so on a very basic level facial detection is a pretty simple process we're just identifying faces and trying to maybe figure out what is a face and what is not but the real test here is going to be facial detection so let me show you how to actually start doing facial detection enrolling a face so we're gonna click on face recognition and all right now we have facial recognition enabled and we can see we get an intruder alert so this intruder alert means that we are detecting a face and that it is not currently enrolled and if I go back to the Arduino output then I can see we're getting a negative 1 for detected so we're basically getting a warning that we're seeing someone who is not in our database that's a very flattering shot alright so now we're going to click on enroll face and it'll take several pictures of me in order to try to get a bunch of variety so I'll try to move around a little bit ok we got 4 5 samples okay and now I am subject 0 so we're doing facial recognition we basically have the ability to detect whether or not it's me I'm going to go back to the Arduino output and you can see now we have detected zero meaning it's detecting subjects zero so I'm going to go ahead now and have our production assistant come over and I'm going to enroll his face and I'm gonna see if it can detect the difference between our faces so Michael go ahead and come over and I'm going to go ahead and stay right there very good now I'm going to well it's not even okay intruder alert I'm gonna enroll your face alright look directly at it three samples four samples okay and now five samples and now we have subject one so if I look at the Arduino output I can see we've got subject zero and subject one enrolled when I turn it back I can see we've got subject zero turn it back to Michael I can see we have subject one so even programming li I can now tell who is looking at the camera whether it's subject zero or subject one now we're going to use as you can probably see from my browser that's open this person does not exist comm to see if we can fool the camera into thinking that one of these fake people is either Michael or myself so let's explore the limitations of fake facial recognition and see if we can actually trick this by presenting it with photos of people that aren't real and seeing if we can overwhelm its limited capability of understanding what it's looking at all right so I've gone ahead and set up this person does not exist comm and we're gonna go ahead and load a bunch of faces and see if any of them can falsely trigger as a known person now that we've enrolled two different people [Music] [Music] [Music] okay after running this for quite some time we've seen that even though we've had a wide variety of phases none of them so far have actually triggered a false detection now let's see if by introducing a couple different angles we can get the get this to identify Michael as me instead all right we have a registered as subject one so we have an intruder alert so we're able to get some false readings subject one all right so we can get it to M identify him as an intruder so one that's not registered but as you can see turning his head will basically break the ability to see and or even recognize his face and then when he tilts his head up it can be recognized most the time correctly but there are instances from angles such as a high angle where it's just simply not possible and because none of the enrollment photos had an angle like this it is pretty confused and can't identify Michael properly so while we haven't been able to get Michael to be identified as me by accident we have been able to get him to be identified as an intruder and also to just not be registered at all so as you can see the limitations of this microcontroller are that it can't seem to see faces when they're tilted to the side or at a different angle and it can't recognize faces when they are at an extreme angle because even tilting your head back is enough to make it think that you are suddenly an intruder as we've shown today there are a couple limitations to what you can do with a basic microcontroller when it comes to facial recognition however it is still pretty awesome that we can begin to write applications that consider someone's face as a factor and already we can use the serial output from this microcontroller to write a program that can interface with it and determine whether or not someone's face is recognized that's all we have for this episode of cyberweapons lab if you get stuck or if you need to solve any problems you can always check out the article linked in the description and if you have any ideas for future episodes you can send me a message on Twitter at Cody Kinsey because they'd love to hear from you we'll see

Channel: Null Byte

Views: 87,600

Keywords: wht, wonderhowto, nullbyte, null byte, hack, hacking, hacker, hacks, hackers, how to hack, howto, how to, tutorial, guide, cyber weapon, cyber weapons, cyber, cyber weapons lab, facial recognition, facial recognition technology, facial recognition python, facial recognition software, esp8266, esp8266 projects, esp8266 arduino, esp8266 tutorial, esp32 cam, esp32 projects, esp32 camera, esp32 tutorial, esp32 arduino, esp32 wifi, esp32 cam projects

Id: LOqVIe9cnW8

Length: 14min 51sec (891 seconds)

Published: Fri May 28 2021