OpenCV Python TUTORIAL #4 for Face Recognition and Identification

and this one we are gonna be showing you how to implement open CV with Python to do facial recognition and facial identification so what facial recognition is is really just pulling out faces in an image or a video this technique that we're gonna show you works for both of them and since open CV is so powerful it's actually really easy to implement so like we're not actually gonna have to do a whole lot of work yes there's some understanding on how to actually get it set up but once you do it's gonna be very useful and easy to get going to recognize faces now that's not the same as identifying faces identifying faces as what you see now that's a bounding box around my face with my name on it right that actually shows that we can actually identify faces in a video or image as well using open CV this takes a little bit extra training that we have to do so at the end of this we'll have two different Python modules one that's dedicated towards training and one that's dedicated towards not only recognizing but also identifying faces in a video now the training portion and the recognizing or the identifying of a face those things can absolutely be advanced but open CV has some built-in features to make these things a lot easier and Weiss when I say advanced I mean by using like a deep learning libraries such as tensorflow or PI torch those can advance our recognition feature for the identification portion now if all this stuff sounds a little confusing you might want to go back and learn a little bit more about open CV but if you know open CV and you have it ready let's go ahead and jump in and if you don't have it actually installed and ready but you want to jump in check out the description below because we actually show you how to get your system set up with open CV so without further ado let's actually jump into the code so of a new file here called faces PI and I also have some images in here that I'm going to be running at some point so to get these things the faces up hi is based off of base high which is on our repo alright so this is the repo itself and of course if you go to join CFE comm slash github it will take you to the repo and you just want to look for the open CV Python series and inside of SRC you'll see all the things in there including the final code for this once it's there and you also see that I have those images in here by default so we'll go over to those in just a moment but the thing is that we first and foremost need to do is we need to get the built in har cascade classifier so so this is something that's relevant to actually identifying a face in any given frame ok so inside of faces let's just run an example here real quick to make sure that we've got everything running I'm gonna go ahead and go into my dev folder into open CV tube and I'm gonna go ahead and activate it dot slash scripts to activate and I'll do pip freeze just to make sure I've got everything installed notice I have pillow install that's something that you will want to install if you don't have it so pip install pillow - upgrade and of course open CV and numpy are other things that you'll want ok so yeah I did a typo there but you'll want to make sure that you have pillow installed okay so now let's go ahead and just run inside of SRC we're gonna run Python faces that PI I just want to make sure that my system is actually allowing for my frame to look in work right so python faces dot pi should actually run my commands he in here so again i'll save that and run it one more time and there we go so now I've got my frame open this is the webcam that I'm using as I did with the intro same same thing right but of course now is driven by open CV and not driven by Camtasia which is the recording software I'm using so there's a cue to quit it out and now we've got our base ready now we're ready to actually bring in those Cascades now if depending on when you're watching this or actually probably by the time you're watching this the Cascades will actually be in this as our seat folder so if you have any issues with what I'm about to do you can actually go on to the opencv source and grab it from there the reason I wouldn't do that is because if open CV updates these the ones on my repo most likely won't be updated so you want to go off of the default ones and when I say default ones I'm talking about files here it's really really simple so let's jump into Python the Python shell and we're gonna import cv2 and then I'm just gonna go ahead and print out CV - period - underscore underscore file underscore underscore inter this is where my open CV is located right so it's in my site packages no no big surprises here I'm gonna now exit out of Python and I'm going to navigate to the root of this so not with that last portion whatever that is I'm gonna go ahead and copy this and we're gonna bring this into our actual file finder so I'm going to copy that open and open up the file explorer excuse me and I'm gonna go ahead and paste that in there so if I hit enter this is what I see now Mac users it's not a whole lot different actually Mac users you can probably just do open Mac and Linux open and then whatever that path is so all the way to cv2 and then open that path and then you'll see something like this so you can do a symbolic link or you can actually drag this folder in and copy it so that's what I'm gonna do I'm gonna drag this folder in and copy it in to where my project is so my projects in dev it's in OpenCV - SRC and i'm gonna make a new folder in here called Cascades so Cascades and inside of cascades I'm going to bring in this data folder that's in OpenCV so inside of that data folder you should see har Cascades you should see all of that stuff I'm just gonna bring that entire thing over and we're gonna copy it so copy it over whatever system you have that you need to copy it over just copy it over that's one quick way to do it we have all these XML documents in here now so we have the hark escalate Cades ready just to recap we went into the Python shell right here and then we opened or we imported CV too and then we opened up where that location was by printing out where it is with underscore underscore file and underscore underscore this is it this is a really useful way to find where any module is let alone the one that we're working on okay so now that we've got that let's go ahead and back into sublime text where our code is and we see that we have our Cascades in here great I'm actually not going to change anything about that data but instead what I'm gonna do is start writing out various paths that I want to use so I'm gonna open up my sidebar a little bit here and take a look at these things actually I'll go back into my data for cv2 in my Cascades I'm gonna just go in here so I can zoom in a little bit easier so you guys can see it easier as well so we'll view this as a list it might not let me zoom in there yes so it's not letting me zoom in with the names okay so unfortunately I can't zoom in with the names but basically what we want to do here is I want to get the frontal face something in the frontal face in here right so you're gonna want to play around with these in general you're gonna wanna test out different ones to see how you actually can grab the face or the facial features differently so the one I'm gonna go off of is this alt - one it's just one that I tested and I'm just gonna use that specifically so above cap I'm gonna go ahead and just say face underscore kes Kade equals to see v2 dot cascade classifier and this is gonna be in the string of Cascades slash data slash cascade underscore frontal face underscore alt - dot XML okay so if you have any typos or type errors at all like actually typing this stuff out this is where I just moved everything right so looking back in my data my SRC folder if we go here we are identifying Cascades for this directory and then of course data is that one and then finally the name of the file is the one that's in here so if we view it by a list it's one of these file names so I used alt face or frontal face alt - so I I can quite literally copy and paste that in there but it looks like I typed it fine so now we just need to actually use this face cascade and detect the faces in this frame but before we do that we actually have to convert the frame to being gray this is how this cascade works so you have to use it off of grey there might be another way to do it but as far as I know it's off of gray and that's from the documentation and everything I've read about it so we want to put it into grayscale and we go ahead and do CV to convert color and of course it's the frame and then C v2 dot color obviously all these key things capitalize BG are to gray and that's now our gray frame so we're gonna use that to find all the faces in this actual weather in this frame so whether it's a video or an actual picture we'll do faces face cascade so of course what we said here dot detect multi-scale of gray and then we can take in the scale factor I actually think the scale factor if you scale it up a little bit more it might be a little bit more accurate but going to this is we're gonna play with these numbers but going too high ends up being potentially a problem and then we do min neighbors min number of neighbors and I'll say five this is going off the documentation I believe the documentation is exactly like this one point five and then five again you can play around with these numbers if your results aren't as great as you would have hoped so now I can actually iterate through these faces so for well this is going to be the X Y W and H values in faces so I'll go ahead and print those things out first so print X Y W in H so we can talk about them so it's going to save it and let's go ahead and run it and what you see on the left hand side here is all of the values where my face is if I if I remove from it it stops moving and you probably couldn't hear me that well but when I'm not there so if I cover it up it stops moving so those numbers are gone and then as soon as it starts recognizing again it sees it again right cool so what's happening here I'm gonna go ahead and close out that frame is something called a region of interest so ROI yes it sounds like return on investment but it stands for region of interest so that means that my face is actually showing up in this frame now if you're concerned or thinking like why isn't it gray we converted into grayscale hopefully you remember back to when you actually display the resulting frame you like I'm still displaying the color one even though I'm detecting things in the gray one so what that means is my region of interest for gray is equal to gray and the location of this frame so the location of this frame is as this so y equals to y plus h I'll explain this in just a second 8 X colon X plus W and we can actually save this image so I'll just go ahead and say image item equals to my - image PNG and then we'll do CV to dot I am G or excuse me I am right not IMG it's a common mistake that I make is IMG right because I'm so used to writing IMG myself you might not be but so I am G item and then ROI gray okay so if I save this what's gonna happen after I run it is you'll actually save just my face so just the portion of my face that's actually showing up it'll remove everything else so let's go ahead and see what I mean by that if I look at my image there it is there's me looking to the side now let's try it again and we'll run it this time I'm gonna look at the camera and try and close it out at the same time there we go so now it should have a little bit of a different one and it does so granted with this same call I could actually write a ton of images here I could write my face multiple times which might be something that you end up do in the future I'll leave it just as that so you guys can see that but this is my region of interest for only the gray frame but since the gray frame or the the grayed out frame the grayscale frame and the actual color frame are the same I can do this same thing twice one is the region of interest for gray the next one is the region interest for colour and that just takes in instead of gray as the frame it's going to be the frame okay so what's going on in here it has to do with the actual pixel values for that item so the pixel values are taking the Y value and the y plus h so the actual square that's in there right so y plus h means where it starts at y so the X and Y axis YZ and then it adds H and that's the bottom part right so then the X starts with the left side of it and then adds W and goes to the to the right side so that gives you those two coordinates so it's basically saying coordinate in other words that's a coordinate one as far as height is concerned and then coordinate two as far as plus height right so that's the Y value that's what's happening here or it's probably better to have it in parentheses than anything else so it's starting at some X or Y coordinate right so in this case it's y coordinate without height so we'll just say y coordinate start and then this is y coordinate and and you could you could say the sort of same thing for X 2 it's the exact same it's just taking into account the height and the width that's it ok so we now have our region of interest so all we have to do now is actually draw a rectangle around it or something in the frame I want to actually have a rectangle so to draw a rectangle first thing I want to do is declare what color I'm going to make it and that color is 255 0 and 0 and this is not an RGB like you might be used to for whatever reason it's in BGR I'm actually very interested to know why it's this way but BGR of course is blue Glee green red and it's the values of 0 to 255 so you 255 means it's totally blue and then 0 and 0 means all it's gonna be is blue so you you put those three together and it comes to a new color if you're interested in those things please let me know in the comments we'll check it out and then finally we want to have our stroke so how thick do we want the line to actually be so I'm gonna give it a stroke of two you can change it to 20 you can you can experiment with both of those numbers to see the results yourself and then finally cv2 dot rectangle we are gonna grab the frame so we want to draw the frame or we want to draw on the frame so I'm drawing on that original color frame of course I could do the gray frame if I wanted to and then after that we want to do these starting coordinates which are x and y and then we want to say how big it's going to be so how wide and how tall so you know width and height something like that so with what's with well it's going to be equal to X plus W and then what's height it's going to be equal to height is equal to y plus h ok so this is giving the actual coordinate of those places so this is really the starting coordinate and the starting coordinate for x and y and these are actually the ending coordinates so I'm gonna change it back to being in chord and in chord X and then in cord why just for my own reference and as well as yours so it's not actually the height and width it's the coordinate so we have starting coordinates and then ending coordinates and with the rectangle it OpenCV knows that it's gonna draw a rectangle it might turn out to be a square it might turn out to be an actual rectangle it might be something different than that so that's how we would draw it and then we add in our color so color and then our stroke we save that and this right here draws that rectangle on that frame I don't have to do anything else so I can go ahead and save it and run it again and now what we're gonna see is actually a color of blue where the region of interest actually works notice that it does cut out and that's because our classifier won't get everything right it's not perfect so when I'm turned like this is a completely profile it's not working why is that well a big part of it is I said frontal face right so the heart cascade classifier that I used is frontal face so it's looking for the front of my face it doesn't have or I'm sure it probably does have one there's probably one out there that does profile and actually does right here the profile of my face so you betcha I could have both of these things working at the same time not something I'm gonna cover just yet but I do have now a lot of really cool stuff going on with this actual frame great so now what we need to do is recognize these frames or recognize what's going on here so we have a region of interest now how do we recognize this that's the big question right well what this means is we have to have some sort of algorithm or some way to recognize that region of interest so luckily for us OpenCV has the ability to train a recognizer but this is where we could use a deep learned model to predict things here as well so if you're familiar with chaos or tensorflow or pi torch or among other things you could use scikit-learn to basically you would be looking at this as how do we recognize this well there's a lot of different methods on how we could identify who this person is right so what OpenCV did was use this horror classifier to say hey there's a face right there now we just need to figure out how do we actually recognize who that face is how do we identify who that person is and obviously in my case it's it's Justin but we might have other faces that end up in our frame and we might want to try and attempt to recognize them now what I'm about to show you is not foolproof it doesn't work perfectly and I think there's a lot of downsides to using it but the main thing here is it actually works and it's really not that complicated where when you want to start talking about putting in a Karros trained model it might start to get a lot more complicated I mean it does it actually does once you start getting deep learning in there it does get a little bit more complicated but we will probably cover something related to that so make sure they subscribe to get everything and know about it and hit the bell for notifications when that thing actually drops so we will most likely do our Whitten not most likely we are gonna do something with chaos and OpenCV so stay tuned to that well in the meantime let's go ahead and implement our own okay so I'm going to go into my SRC folder here and I'm gonna make a new file and this is gonna be called faces - trained hi and in here I'm gonna actually have to do several things that is related to my images okay so right now would be a really good time to download those images or images of people so if I go here peter dinkledge he plays tyrion lanister on probably my favorite show Game of Thrones and here are some pictures of him so a quick google search for Peter Dinklage face gave me these things right so that's true for Jaime Lannister as well I've got several pictures of his face and Kit Harington all of these are Game of Thrones characters right and then Emilia Clarke right so basically I have all of these different characters and then I have pictures of me these are just various pictures of me that I've used and they are over the over the years as you see here I don't have a beard there and so what I want to do is take all of these images and make something out of them I want to I want to actually first see all of these images and start to turn them into trainable data so that means that I'm gonna have to go in to import OS and I'm just gonna be opening my image directory so this image directory here is right there if you're already familiar with this method you could probably skip this portion but for those of you aren't what I'm doing is I'm gonna be literally walking through the directory looking for either PNG files or JPEG files and then adding them to some sort of list that I'll end up training okay so we're gonna go ahead and say base dirt equals 2 OS dot path that durn aim and is gonna be OS dot path dot ABS path file ok so if you've never seen this call before basically what I'm doing is wherever this file is saved this current python file I'm looking for where that path is that is the path on my actual system in other words right here so if I you know bring this over this is the path right so this is part of the path and then that's the file path with faces trained in there so again going into properties even further I see that the location is here and then the path of the actual file would include that final location so I actually wanted the directory so that's what OS path doctor name does it gives me this value whatever that value is and it's true for any system that's why I have it it's true for every system so if I move this over to a Linux server or a Linux computer or Mac computer that baster will be the same wherever that file lives that's where it is ok so then I want to get the image d'oeurves so the image directory with OS path join and this is gonna take in that base directory and give me the images really simply it's just looking for this folder right here if you have a different name you're gonna want to change that so next I want to actually see those images in there so we'll just go ahead and do for root doors files in OS lock and image dirt so I can then inside of this four loop I would do four file and files this would iterate through the files in there then finally I'll say if filed ends with PNG or filed ends with jpg then I'll actually print out the path of this file so I'll just say path equals 200 s dot path join root and file so let's go ahead and print out that file or that file path itself so I'll save this and we'll go back into my powershell here and I'll just do Python faces - trained PI and I hit enter I get all of those images now the key thing with how I have this set up I have this set up basically have the folder with that person's images right so that's important that's actually very important to how we're going to do this because in a moment I'm actually gonna get the label of these images so the label meaning you know all six of these images are adjusted the actual file name is one two three four five six but the label is the actual directory this is actually common practice so you'll see this in other areas as well of machine learning so basically I'm saying hey this is this is the folder that holds all of Justin's images so eventually what I'm gonna do is every image for Justin has the label of Justin and then the image itself has whatever file name the file name doesn't matter in this case now this is for computer it's not first humans necessarily but it's for computer so I only named them one through six because to me that's a lot easier than some random thing from the internet right so these did have random names from the internet but I just changed them to being one through six another thing to note is they are not all the same size they're close but they're not all the same something that's kind of cool as well okay so now how do I actually grab from this path how do I grab the name of the folder that it's in to give it a label well it's actually really easy we say label equals to Oh s dot path dot base name and then the directory name of this path so we'll say Oh s dot path doctor name of this path now we can probably use route but I'm gonna do double-check and make sure that I'm just going off a path I can probably like in other words they can probably put roots in here so we might do both of those but another thing that I want to do is I want to make sure that I'm replacing any sort of space with a dash this is mainly for you if you accidentally name the directory incorrectly and then I also want to lower case everything on that label so now I'll go ahead and do print label and path we'll save that and we'll run it and now it's showing me the label and the path great so the label let's go ahead and just try that with roots let's see if that's the same we say that we run it and it turns out to be the same knot now why did I show you that I just wanted to show you that yes sometimes you might see code that's a little bit more complex than it needs to be so yeah and and with rate replace and lower this is just a safeguard it's probably not the best safeguard because you can you might need to clean it up even further but in my case I'm not going to I'm not I'm not really worried too much about the label you could run into issues with it but again it's not that big of a deal okay so so now I have a label and a path so I want to keep these things so let's first say X train and we'll equal to an empty list and then we'll say Y labels and we'll put that equal to an empty list so intuitively I would just go ahead and say you know X train dot append path and Y labels dot append label right that's that's my intuition that's what I would end up doing but really we don't want to do that what we want to do instead is we want to use a number value for our labels so some number for our labels and then also I want to you know verify this image and I probably want to turn it into a numpy array you might not be familiar with this portion but we do want to do that and we also probably want to convert it into gray so we probably want it to be a gray image much like our actual Cascade and all this stuff is going on right so that means that I have to do additional things with these images so I actually don't want to append these here just yet so instead what I'm gonna do is say I'm gonna give a pill image and this is gonna be image dot open of the path so I need to actually grab pill pill of course is the Python image library so I'll do from pill import image and if you don't have it installed which I think you do because we've talked about it you just want to do pip install pillow and make sure that pill is installed there and this will now give me an image of this path and again I made sure that the the file extension ends here and I'm not really run worried about exceptions necessarily there might be exceptions that come up in other words if you if it's not actually an image but it does in with JPEG that does run an exception that causes a problem but what I'm gonna do is just convert it into L so cannot convert L turns it into gray or grayscale rather not just grey with grey scale and then the image array so now I'm going to convert it into a numpy array by importing numpy so import numpy as MP that's a common designation for numpy so we'll do in pi MP array and we'll bring in this pill image and we'll use u int 8 as the type so we're turning this into a numpy array we're doing a lot of stuff here that like might be outside of what you're normally used to so let's just look at this image array just like that so we'll save it and let's go ahead and run it oops we don't need to install it we'll just run it and this is the numpy array now okay so so this is taking every pixel value and turning it into some sort of numpy array so every pixels in here and it's now in two numbers so I basically just converted our image into numbers it's pretty cool because like if I open this it's an image right it's showing me an image but now when I printed it out it actually shows me what is in that image it shows me the numbers that are in that actual image so this is what we want to train on this is actually a very common thing to do and if it's if it's like a little complicated I understand that's okay it does get a little bit complicated here as far as their race is concerned and if you're really really confused about this let us know in the comments below otherwise just kind of stick with us and just bear with us on on the understanding of these two things but essentially every image has pixel values what we did here is we turned them into first a grayscale and then we turned that grayscale into a numpy array so we converted it into grayscale and we use that as a list of numbers that are related to this image and with that then we can actually start training it so before I can actually start training it I want to actually know the region of interest for this image so that means that I need to go back into my faces alright like where I'm gonna be doing the prediction so I'm gonna go ahead and grab this face cascade and use the same thing here so I need to import cv2 as well so let's import cv2 so we have our face cascade again and then we are gonna just go ahead and say for or add that rather we need to do our faces and that's gonna be our detector which is our face cascade so face cascade dot detect multi-scale and honestly we could do it the same way as we were doing it like with the same scale factor but you really don't have to so I could I could literally copy and paste that whole thing and we could do that whole entire thing or we could just get rid of the scale factor and the minimum neighbors to whatever the default is that actually might be what the default ISM I actually don't know you look at the documentation to find that out but now what we do is just this image array see when we actually capture it in CV - Stephen - or open CV already has it in a way that the that face cascade can do it right so we when we open it up in CV - I believe it's already a numpy array which we could test out by printing out the frame but what we have here now is we are taking it to gray we're putting into a numpy array and then finally we're actually doing that same face detection inside of the actual image because we still need to detect that image so now we can just go ahead and say for again X Y W and H in faces we can grab the ROI so the region of interest I'll just leave it as an ROI of the image array and we're getting the Y value all the way to the Y plus H and then X value all the way to X plus W and that we want to append to X train got append and will append the ROI okay so now we have our training data except we don't have the labels or the actual person that's associated with this region of interest see we want to take our labels because we know them this is all about the known values and we want to associate it with this training data now so we want to do that using an ID so we actually want to grab whatever ID would be here and bring that into our Y labels so I don't have any IDs so how or what would be a good way to do this well I want to create the IDS in with this label so I know what the label is but I don't have a number value associated to it so one of the ways we can do is just say current ID equaling to zero and then for every new label that is created or never eaten every new label ID that is created we would add one so in other words we need to create a dictionary that says label IDs and we're gonna put it equal to an empty dictionary okay so now when i iterate through all this what I want to do is I want to say we're gonna say for or sorry not for but rather if label in label IDs so if it's in this dictionary then we'll do something for now I'll just say pass otherwise like the default for most of them they won't be in there we'll do label IDs of the label equals to whatever the current ID is and then I'll just go ahead and append to the current ID right away so plus equals to one and then finally I'll just say the ID is actually equal to that value right so what this value is okay so that also means that I can just go ahead and well tab this back right because if it's in there then we will set it to that if it's not in there then we'll well add it in there and then give it a current ID or whatever that current ID number would be so really I can just say instead of if it's in there I'll just say if not label in then we actually create a dictionary with the label or the person's name and an ID associated to it this is just a way to do that in another way we could actually just look at this and we'll print out label IDs so we can see it and then what I'm going to do is now that I have an ID for that label which is this right here I can actually do Y labels equal up got a pen that ID and keep that underscore there too okay so why is it that I can do this well if there's ten faces in here I have a label for it now again my faces don't have ten they have one so more than likely you won't have multiple faces you shouldn't in these images you should not have multiple you should have one because we're identifying that one single person that's really important for the training okay so let's save it and then in the final thing we'll just go ahead and print out why labels and we'll print out X train let's save that and we'll run it and of course it's giving me so much stuff like he gave me a bunch of things printed out so I'm gonna comment out these printed stuff in just a moment or for now so save it and I run it again and what I see here is look at this look at this dictionary so I've got Emilia Clarke just in Kit Harington and so on it gives me the number value that's associated so I actually created an ID for all of these labels that's awesome that is really really useful for us okay so what I need to do now is actually save these labels so I can use them inside of faces up high so let's go ahead and do that I'm gonna do it by importing something called pickle so import pickle and scroll down to the bottom after all of those things are parsed out they're identified they're put into a basically a list of numbers right so one has the numbers of the actual pixel values which is this and another one has the actual numbers that are related to the labels once all that stuff is done then we're gonna go ahead and do something like with open and I'm gonna call this labels dot PKL for pickle and we can actually name I think you might be able to name pickled items anything I think probably we should just call it pickle with pickled labels the pickle WB as in I'm writing bytes as file or F pickle dump and we're gonna dump in the label IDs to that file so the label IDs again are all of these label IDs and we need those because when we've run the prediction we want to be able to get what those actually think what those things actually are what the label actually would be so that's something that's important all right so that's pickle and now that we have the label IDs saved now all we need to do is actually train the item itself which we haven't built in yet so now we'll go ahead and make our facial recognizer there are a few different ones but I'm going to be using the LB pH face recognizer I've been from the tests I've done this one worked pretty well so I'll go ahead and say recognizer equals to C v2 dot face dot lb pH face recognizer underscore create there we go so this is our recognizer I actually don't have to put this up here but I am because this is where I'm declaring a lot of things and then down here at the very bottom we can actually use recognizer dot train and we're gonna do X train and then a MP dot array of our labels which is why labels and then finally recognizer dot Save and then I'm just gonna call this trainer Gamal yml that's how you would save it so a couple things here you want to put the training data which is the actual image arrays so the numpy rays that are coming through from here from this region of interest and then we wanted to convert our Y labels into dump I arrays as well I actually I'm not positive if you needed to do this here you could have done it somewhere else but now we would have our recognizer and then all we need to do now is actually run the training so I'm gonna get rid of these print statements altogether and we'll save everything and let's go ahead and run it so make sure you save everything and we just run Python faces - train dot pi and it looks like I have a typo recognizer there we go try it again and this is what you should see this is the actual training and it's done so now what we need to do is just implement this recognizer and there's a couple things that we need to consider here when I jump into faces well how is it that I'm gonna use this recognizer well first and foremost I probably need to bring in the actual recognizer itself right I need to initialize whatever recognize that I'm using of course this could be another trained model it doesn't actually have to be this OpenCV recognizer but that's what we've been doing so that's what we'll stick with and actually I want to reorganize how I did this okay there we go and then the next thing would be actually bringing in our trained data right so we want to add in recognizer read and of course we called it trainer Gamal or ynl so we now have a recognizer and we can run predictions on our item we can actually predict things now so let's do that I'm gonna scroll down to where our recognizer is and the predictions are ID and confidence and we'll do recognize a dot predict and what do we predict well we predict the region of interest all right so the ROI great that's all we're predicting so again we use the region of interest with the training and we have labels so basically what this is saying is giving us the label back and the confidence that it is now I've mentioned that this is not necessarily the best way of doing this so during my tests the confidence level varies quite a lot and even with the accurate image or even with the more images the confidence level doesn't seem to be accurate to 0 to 100 so I'm actually not sure what this confidence level is supposed to be because it does go over a hundred so I'm really curious to see if you guys find research on that I actually wasn't able to find anything in relation to what that confidence would be so what I did was as I said if kampf is greater than or equal to 45 oops not 54 but 45 and calm is less than or equal to 85 then we'll just go ahead and print out this ID first okay so let's save that and let's run it and of course now it's just faces so we're actually implemented with our trainer and there it is it's giving me a number so it's giving me a couple numbers so let's go ahead and get rid of those I'm gonna save that and quit out of here and then run it again so so really it should only give me a number one single number if I'm not in the frame or if I if I cover it up it's it's not gonna keep going right so let's let's just start with it covered up so I have my hand over the webcam and they just thought it's showing a nice zoomed in view of my hand if I go in front they've recognized this close it up same thing okay cool so it looks as if it's working but of course I don't have I have an ID I don't have a label I don't have the actual label right so I have the sort of training label which is an ID but I don't have the actual name the the person's name from that training label so what I want to get is the name from my pickle right so down here I want to actually load this pickle in so I'm gonna go ahead and copy this value go back into faces and we're gonna go ahead and import pic whole and in here right underneath recognizer I'm gonna bring in this pickle so just paste all that stuff in instead of WB use RB and now it's just going to be load and it's going to actually have just the file okay so what this will give us are our actual labels so the label dictionary so I'll say labels equals to a dictionary here and we'll make it empty first and then I'll do labels equals to that load file okay so I now have my labels in here based off of the label IDs which is a dictionary but if you think about it the label IDs are actually going off of the name so the lookup is a little bit different than just loading it in here so I need to actually reverse this lookup I need to turn this in from being like the person's name equaling to some value or some ID I need those to be reversed so to do that we basically are just going to say labels equals two and we want the invert of it so we're inverting the actual labels so we'll say v : k for k v in labels the items so the original labels so this is going to be let's call this og labels and this is now our new labels and of course this is value key value pairs right so that's just a quick way of doing that of inverting it so now down here with my D I'll just go ahead and print out the labels of the ID so we save that let's run it again and what we see is it's actually saying that I'm the the actor who plays Jamie Lannister of course that's not who I am so there is some issues with our training so maybe you know maybe it's possible that the confidence I shouldn't have put it less than or equal to 85 so let's just say if it's greater than 85 then we'll print that out I'm guessing this this might make a difference it might not but of course in my initial tests I actually had me in here working just fine so I'll go ahead and close that out so there's definitely some things going on here with our recognizer that's just not getting stuff great so a couple things it could be the size of these images I mean it could think that I actually look like this guy I don't necessarily think that I do but maybe maybe that's what the recognizer is seeing maybe these images aren't very good representations of what I look like right now maybe that's why so there's there's definitely some some issues with this recognizer and that's where using something like a deep learned model might make a big difference but I will do is I will save one of my images with color so ROI I'll just go ahead and save that one and we'll call this seven so I'm gonna go ahead and save this image so this is obviously an image of me but it it might actually have the frame on there so let's go ahead and take a look and we called it seven and there there's there we go so what I'm gonna do now is I'm an open containing folder and I'm actually going to put this seven image into my trained items I'm grant it's only showing me the region of interest so there's a chance that this might mess up because it's not giving me the whole frame and if it does mess up then we'll have to take a step back and grab that frame so now let's go ahead and train again face his - train running it okay so it takes it just takes a moment it's really really fast and you know if you do deep learning libraries they take a lot longer so now I run faces again and same results still giving me the wrong actual item here so again it might have to do with the size of these images because of how I'm treating the training data I actually don't resize these images at all so that's a problem that's something that we might still need to improve before I do that I actually am going to put my label on the actual image itself by doing something called put text so what we're going to use is a font so we'll say font equals to C v2 dot font underscore Hershey underscore simplex and we'll just say name equals to labels and then ID and then we want to give a color so again the same sort of setup we had before this time I'm just going to use white so 255 255 255 and then the stroke again and then we just do CV to dot put text the frame the name or the text that we're actually putting down the location I'm going to put it at X&Y and of course X and y it has to relate to where the face actually is and then the font the I believe the font size so I'm gonna leave it at one color the stroke and then cb2 dot line underscore a oops and that's that should actually handle it for us you can look over the docs for the text parameters if you have other ones so now that I do that it's it's still showing me you know the other guy okay so yes incorrect so now what I want to do is actually fix the images perhaps I need to resize the images first and then convert them so we'll resize it to a specific size and I'll just say 550 550 and I wanted to scale correctly so it's not like it's gonna distort the image so to do that we'll just do final image equals to the pill image so of course that's this right here dot resize to that size that I just wrote and then image dot anti-alias and then of course now my image array is going to be coming off of that final image so I saved that and I'm gonna go ahead and run it faces train Oh looks like I in my tests I accidentally comment that part out again so going back it's running and then it ran and now faces its it's got a better idea it's not perfect still but it's definitely a better idea that it's me now yet again while it runs you can see that's me but it's still getting me confused with other people okay so how would we actually go about making this better well number one is getting better pictures of all of these people right so like I don't think I really looked like him and I don't think I really looked like him so it's definitely confusing me with these two guys and granted I don't have that many pictures but it's never confused me with him so it's getting that part accurate and it's also never confused me with her which was great now I did have to add in some more images so I even have images of me in here although they're not perfect images they are images of me in this frame this could potentially help with the results but in the long run would you be in abusing this sort of recognizer in general yes or no I mean this is fairly straightforward and easy to set up but again you're gonna want to have more images that will help with the training and you also want to have your images roughly the same size so even my resize method here that works but perhaps it's not the best way to do the resizing perhaps we want to have good solid data from the get-go and that is one of those challenges with recognition in general or when you talk about using deep learning or machine learning the problems come up with the data the better data you have the better these results will be but overall it's pretty good I mean like I'm I'm not upset with it you know like I think it's actually pretty solid and all I would really need to do from here is snap some of these pictures and then categorize them and then throw them back into my trainer so then it knows it knows for sure that hey these things are me and I mean it's doing a pretty good job it's not it's not great it's not perfect but it is pretty good as far as like showing hey that's that's just it cool so I do want to have a couple more things to just show you generally about this recognition stuff so the first thing is actually doing eyes so like actually recognising your eyes and I'm gonna I'm gonna go and copy and paste a few things in here just to make it a little bit faster as a final portion so the eye cascade that's that's the eyes right there I can actually use this to iterate through and find my eyes or anyone's eyes for that matter this also is not perfect but it's pretty good so if I paste this in here there's there's roughly the exact same thing that you've already been seeing and then you want to do a very similar iteration to what we've seen before as well with up with eyes you would do that and then finally we would put you know some sort of frame around it or some sort of rectangle by now you should have an idea of how to do all of those things so let's go ahead and save that and close out I have my item running there still so let's go ahead and run it again and now you see that it does get my eyes maybe not maybe not perfectly but it does get them well and notice that when I close them close everything it should actually get rid of and show that my eyes are closed so that's that's something that's kind of interesting hey if your eyes are closed maybe I need to buzz you or wake you up or something okay so the next thing is actually smiles and smile is not a whole lot different so with smile I'm actually going to just replace the eye cascade because it's basically the same call so everything in here is roughly the same so I'm gonna replace that and I'll just call this instead of eyes I'll just call this sub-items that's it and that way I can replace whatever those items are at any time and so now let's go ahead and run that and now it's gonna get my smile notice notice that it smiles even worse so smile just really doesn't fully grasp it gets my eyes my nose it's it's really not that great as far as recognizing my smile but again you could break certain thing down and use your own third-party recognizer to make this a little bit different and a little bit better but overall this is pretty awesome I mean think about the things you could do with this assuming that you have a lot of images and you have a lot of data this should be able to actually give us really really good results they're not great but I mean it's me I mean it hopefully following along with this it's fairly simple to do I'm gonna use an image from the internet from my phone to see how that image does okay so it's it's looking for me and then let's see here it doesn't seem to be recognizing that face maybe that picture is too dark let's try a different one okay so so this this is sort of thinking it's me but most of the time not right so if I get out of it the frame it no it's looking like it's me still thinks it's me there's a good chance for that the reason being is that mine has more training data because I have eight images versus everyone else that has six so that's another challenge with this is is having just a more consistent amount of data for every single person so yeah that doesn't actually show me unknown and and the part of it is hey the confidence is only at 45 so let's go ahead and bring it back to being and 85 so I save that and let's run it again so really with an unknown face it should probably not even show anybody let's try that again and now here's an old guy with a beard right so nothing that's Barristan Selmy but you know it's not even listening anything so so he doesn't look like any of those people which is kind of cool so yeah it does work it's not perfect but it's something so a couple final thoughts if you were gonna implement this yourself you are going to want to get a lot of images to work off of right that way your actual facial recognition is better another thing is to consider a deep learning library to actually handle the recognition as well so if you built in something like that where you have a good amount of data and it doesn't necessarily have to be that much with the deep learning library that will probably give you better results although the training the training itself will take a lot longer than what we saw here and it also doesn't mean that they're gonna be perfect results it just means that they'll probably be better and when I say perfect I mean 100 percent accurate every single time it's just that's just not that likely to happen but with deep learning you'll have a little bit better of confidence with the actual person that's being predicted or the person's name so there's a lot of things that we can still do with this right it's it's still could be useful for a variety of applications but it's not really super secure where it's gonna be something that you use to authenticate into your computer or authenticate at all you probably wouldn't use it for authentication but when you could use it for is something as simple as walking into a room and then boom your camera starts recording because it recognizes the person and if I was out of the room it wouldn't record what as soon as I came in it would recognize me and then it would start recording that would of course assume that my computer's on and my webcams going but it would totally work it absolutely would which does have some features of being you know a security camera whether it actually recognizes my face or not those are two different things right so there's definitely a lot of really cool stuff with this and in a little bit over an hour we figured out how we could do so much so many things it might have taken you a little bit longer than that to write all the code and keep up but I mean if you think about this in terms of what we could do in an hour with OpenCV what could we do if we had 10 hours or 15 hours with more training data and also using deep learning and all these things you could probably make a really sophisticated system a you know how with very little time and the other cool thing about this is you can actually attach this to something like a Raspberry Pi and place it in different areas and the Raspberry Pi can run a lot of these things including trained models from something like Karros or tensorflow or py torch you could actually run those things on Raspberry Pi with OpenCV and not be connected to the Internet at all and get really pretty solid data as a response I mean of course it's not gonna be perfect but it will be better than random I'll buy a lot as you may have seen here so I'm really excited I'm really excited about what OpenCV can do and I'm excited about what you might do with it so if you are interested in seeing us build even more advanced stuff on OpenCV consider subscribing to our youtube channel join CFE comm slash to youtube that's where a lot of this stuff is going to live in the long run because I'm still experimenting with OpenCV myself so I want to learn more and the way I do that is with you guys on YouTube so join CV comm / YouTube make sure you subscribe to get everything and then also leave a comment when you find something new or interesting or your you're struggling with something in relation to open CV I try to read as many of these comments as possible and also we do live streams so I want to help you guys answer questions there and if you have suggestions and all that I'm all for it I realized like I I've talked a lot today and you're probably done hearing me so thanks so much for watching look forward to seeing you next time [Music]

Channel: CodingEntrepreneurs

Views: 807,649

Keywords: python, opencv, python3, face recognition, face recognition with python, deep learning, facial recognition, opencv3, tutorial, cfeopencvseries2018, coding, cv2, computer vision, ai, keras, tensorflow, raspiberry pi, video analysis, video security, raspberry pi video security

Id: PmZ29Vta7Vc

Length: 66min 24sec (3984 seconds)

Published: Wed Apr 11 2018