Facial Recognition with Python and the face_recognition library

what's going on everybody and welcome to a tutorial on facial recognition with the face recognition package so to get started you should be able to just do a pip install face recognition but depending on your operating system and depending on whether or not you want to use CUDA and so forth there's many different paths that you could take for the installation so check out the text-based version of this tutorial if pip install face recognition just doesn't do it for you so with that what I want to do is basically just do kind of an overview of the package and how to use it the package itself is actually extremely simple and there's really like three calls you need to make to the package the rest is just boilerplate kind of stuff that you need to do around the code to make use of it so to get started basically I'm going to be working in the following directory so this is the code that I'm going to be working on just as a reminder never name your code the same thing as the package you're gonna be using so don't call this face recognition pie then we're gonna have two directories one directory is known faces and inside of there is more directories in this case we just have one it's called semtex now each directory in the known faces directory is going to have basically a directory of an identity that you wish to identify so just as a reminder face detection is just finding a face right in imagery or videos or whatever facial recognition is identifying that face so face recognition goes a step further the text to face and then it recognizes that face tells you who that person is or whatever so inside of this directory send text we've got three images none of these images is the greatest in the world but you can see here we've got this one this one's kind of goofy because it has obviously I've got the EEG headset on but the face itself is actually a pretty good face and then this one is not very good the eyes are covered up and all that so not the greatest images but kind of it'll get the job done so you can do whatever you want all these images are on the text-based version of this tutorial you can use my images if you want but I really encourage you to use your own images grab some images online or whatever do do whatever you intend to do with face facial recognition so those are the known faces that I have and then there's also a directory of unknown faces in this case we don't know the identities so it's just a bunch of images both of me and of Daniel and the idea is to see can we recognize her own face this one's an interesting one because it is the deep fake of Snowden's face on my face anyway okay so this is also some deep fake so this will be very interesting to see which ones it detects or or none or what anyway so that's how everything's set up you've got the face recognition directory that you're gonna work in you've got known faces identities and then images of each of those identities they've got unknown faces which is just a bunch of images that might have many phases so in this case we only had yeah and all these images there's actually only one person although there is like in this one there's there you know there's a face back here the back of a person's head another face here and so on so I mean there can be multiple faces but most of these just have one face and I guess this has a lot of faces as well anyway it doesn't have to be all like the unknown faces don't really need a single identity but for known faces these really need to be just one face that is the identity of you know whoever this directory is okay so once you've got those things set up feel free deposit if you need a second to set everything up otherwise we can get started so let me just move some stuff around so the first thing we're gonna do is we're going to make some imports and then set some constants so we're going to import face recognition so that's the package that we intend to use or import fo s camera import cb2 so you'd also have to pick install Python - open CV I always forget which one it is install - it's either open CV - PO so unless we open see me - Python so pip install Python that's probably it oh I guess I upgraded that probably just broke the whole tutorial oh my life okay so continuing long once we have those things so yeah OS to iterate over directories CV to to do image tasks we're gonna use CV to draw rectangles and label the images and stuff like that so now we are going to have a known faces dirt and that will be known faces and then we will have an unknown faces derp so unknown faces derp unknown faces then we're gonna have a tolerance now this is a degree of try think another word besides tolerance but the the the lower the tolerance the less chance you're gonna have for like false positives so if you set are very very low tolerance but you only have like one or two pictures of the identity chances are it's never gonna make a match so the default with face recognition is 0.6 if you want to make more matches but you're willing to risk being incorrect on the match that you make you can set tolerance to be like 1.0 very high but if you want to only match if it's a like a for sure thing set the tolerance to be quite low I have found 0.6 to be pretty good but as we'll see we'll probably have issues with that both missing predictions and then as we try to lower it it's quite challenging ideally you would have like 100 images of you know somebody's faced in different lighting conditions and looking up in every all directions and stuff like that but it's actually it's still pretty darn good even if you have like what I just used here are some pretty crummy images so okay so that's tolerance again the higher you go the more likely you are to make matches but you're likely to get a lot of false positives the lower you go the more likely you are to get I guess you would call it false negatives right so you're gonna miss matches so now we're going to say frame underscore thickness this will be in pixel values basically we're gonna draw a square around the head how many pencils do you want them to be just really just good that's going to depend on how big of an image are you using in our case we'll go to I'm going to use three and then we'll do font thickness that will also I'm gonna set that to be two and then model I'm gonna use CNN so convolutional neural network you can also use hog which is I always want to say hierarchical histogram something gradients histogram oriented gradients I don't know something like that anyway it doesn't matter what it stands for it is like kind of the older way of doing object detection so you can use either one you want I suspect hog is gonna run better on the CPU so if you're only on a CPU you didn't install the CUDA D lib then I might suggest that you use hog instead that's quite literally a Choji feel free to tinker with both CNN will still run on a CPU it just might be kind of slow ok so ok so once we've got that the next thing we're ready to do is start loading in faces so to do image facial recognition we need to be able to do kind of two things right we're gonna detect the face and then for every unknown face that we happen to find we need something to compare it to so the first step that we're gonna do is load in all of our known faces so that we have something for the unknown faces to compare to so we'll come down here and we will just print just some debugging information loading known faces and basically we're gonna have two lists we're gonna say known faces these will be the actual encoded faces you'll know what I'm saying a minute and then known names these will be the names associated with those faces so as we populate this out later when we go to compare the unknown faces we'll see does the encoded unknown faces match any of the encoded known faces if so at what index value did we find one and then we will just reference the known names list at that exact same index and that's boom the identity so so no one faces no names and now we want to do is irate over all the known vases and store that information so for name in OS list der known faces der so that gives us the name now remember that was the identity directory now we need to iterate over all the images in that directory so we're gonna say for books or file name in OS dot list Durer and we will say known faces der and then slash name and this also needs to be variable cool you also gonna use OS type path join like I haven't really decided which one I like better is using like F strings for paths or OS dot path joint I mean it's like it's meant for this but Melissa I've had though join came before we could do really nice F strings like this so I find this to be quite legible so anyway if anybody knows a good reason why not to use f strings for pads let me know okay so once we load in that file name we want to oops so for file name we're going to say image equals so this is the first call that's a unique face recognition call face recognition dot face underscore in coatings and we want to do this for the actual image that we bring in and we haven't brought in the image so so actually what we want to say is so really this will be our encoding the image is going to be face recognition 'don't load image file and then here here's what we're going to do we'll do another f string known faces their faces door [Music] name and then the file name file name so then we load the image with facial recognition we encode that image and then we were gonna add these to our list so we're gonna say known faces dot append the encoding and then known names dot append the name cool ok so now that we've got that oh and also when we do face encodings this is going to encode all of the faces that it finds so what we actually want to do is we're going to say the encoding will be at the zeroeth index so we only want the first face it finds so again your identity images so these ones here really ought to just have one face and it should be your identity right so the every you should only have one face in these images and each of those faces really needs to be the identity here I mean this should be pretty obvious but in case that's not obvious there you go so so when we encode the the images it will it was it's gonna find all of the faces unless you specify otherwise which what you'll I'll show you in a little bit so we're gonna say at the index 0 that's the face that we're interested in so okay cool so now we want to do is so we've got the known faces so we can compare to them now we want to do is we want to iterate over all of the unknown images find all of the faces and then for each face compare that face to each of the known faces so ok lots of lots of looping about to occur so now we're going to say is prints processes unknown faces and cool so what we're gonna do is we need to iterate over first the unknown faces directory so for filename in OS dot list derp unknown faces derp and again we don't have identities so we don't need to iterate over those directories but instead what we're going to say is print I'm I swear I hit the caps lock print file name just so we know where we are now we want to do is load in that file name so we're gonna say image equals face recognition dot load image file and we want to load in on that path so again f string we're going to say unknown faces derp slash file whoops don't forget this file name cool so we're loading in that file so I've got the image now what we want to do is we want to actually find all of the faces inside of that image now this is not something we did before because we're kind of just assuming we already know there's only going to be one image and so on but in this case we're gonna say locations and what this is going to do is do face detection for us find all of all the locations for faces and it's just going to find the coordinates so when I say locations equals and that's going to be face recognition dot face honors face times or locations and it's going to look for those locations in the image and then the model that we're going to use is going to be model so ok cool so now that we have all of the locations that we have we're ready to make the encodings so in and coatings will be able to face recognition dot face encoding and we will do this for image and the locations so we're gonna actually specify where are these locations that we wish to encode and again we just we didn't do the locations thing up here for the known faces because it's irrelevant we don't care where the known faces are in their respective images we it's just not important but we definitely care where the unknown faces are so that later we can go in and label them so okay so encoding is for okay so once we've done that we are ready to start drawing well we're in iterate and try to make matches but we want to be able to draw on the image so first we're going to convert this image so we can use it with open CV so we're going to say image equals C v2 dot and what we're looking for is convert color yes convert color and we are going to do this on the image and then we're going to do C v2 dot color or GOC RG what are we looking for I think we're looking for RGB to BGR so color underscore RGB to BG are cool so also just as a side note this video is sponsored in part by kite which is a autocomplete engine for Python by far the best autocomplete engine both it's learning as you're typing things here to like kind of give you suggestions on your autocomplete it also has like snippets for like functions especially really well known functions like when I'm working with like tensorflow and chaos it's amazing for like like when you do like fit mints and trainings and stuff like that it will it helps you just like write those lines super fast for something like face recognition I don't really know that it knows too much about face recognition but still it's been helping a ton on some of these autocompletes and at least for me I hate most autocomplete engines I think they're just trash I'm usually like working around the autocomplete rather than getting any benefit at all from it so I actually really like kite and then also just as a side note as you're coding if you use this kite copilot thing it's basically like a rolling documentation so whatever the last thing is that you were working on you like basically wherever your cursor is it's just constantly updating with the documentation so if you kind of forget like what are these things or how do I use this thing or whatever it's just it's already there I spend so much time like having to Google like a specific function or like converting the color I don't have that memorized I don't have like I couldn't remember it was like color underscore I can't remember you know so it's it's just really convenient to have that stuff there so kites like totally free to use works on pretty much every major editor I definitely suggest you check it out I've really been enjoying using it myself anyway so okay continuing along so now we've got the image we've converted it to BGR because BGR is what OpenCV thinks is best I don't know so okay so we've got our image now we're ready to do is iterate over the encoding and locations for the unknown faces that are found so what we're going to say is we'll come down one more line for face underscore encoding and then face underscore location in zip and codings locations so for each of those what we want to do is first we want to see are there any matches that we can find so here comes what I believe to be the third and final line of the face recognition package and the rest of this stuff is just like like I said just code to do all the other things that you want to do we're gonna say results equals face underscore recognition dot compare faces compare faces and then we're going to compare known faces to the face encoding at the moment so we're basically this will be the list of known faces that you have this is the thing that you're trying to compare to see are there any matches and then at what tolerance do you want to use again the default tolerance is 0.6 that's what we're gonna go with to start but I'll show you guys some some variances as well so now what results are gonna be is is it's going to compare the current encoding to every same Coface that we know about and that's going to return to us a list of boolean's so it's just true or false and you might have multiple throughs and then it you know like in our case we only have one known face so it's either gonna be hay or any of these a syntax match you know but in the future you could have many known faces so just keep that in mind but this will be a list of boolean's going through the known faces for any boolean that is true you would want to say okay at what index was that true and then check the known names list so in the known names list you're gonna have that list of identities so at the exact same index you'll have the name for that face encoding that we know about so what we're gonna do in this case is we're just going to say currently match equals none and then we're gonna say if true in results so as long as we got a single true we're going to say the match is the known names and then we're gonna grab the index and basically were going to say results dot index for true so again in this case we're kind of cheating a little bit but but we're just gonna get the one single identity here and then what we'll do is we will save prints match found and then we'll say match we'll make this an F string cool all right so cool so now all we need to do is some open CV stuff to like draw the rectangle and say who the match is because obviously it's one thing to like find a match like we can prove that we found a match but what we're really more interested in is like seeing it on the image you know everybody knows that iconic like boxes around the head thing so yeah we got we definitely have to do that so so what we're gonna say here is we need to get the coordinates for the match so we're gonna say and the way that we wait you're gonna do any time like we're gonna use C v2 dot rectangle to draw a rectangle like around the face to draw a rectangle you just need two things you need the top left coordinate in the bottom right coordinate and then it draws the rectangle so we're going to grab top left and that is going to be the face location um at the third the index of 3 and then face phase underscore location at the index of 2 or actually this will be 0 yeah and then we'll just copy this top left then this is bottom right bottom right and then for bottom right yeah it's 1 into 2 and then we're gonna do color for now I'm just gonna say color 0 255 0 255 0 so green and if you want you can check out the text base tutorial for a much more fancy version of dynamically picking a color based on the identity maybe in the next video too when we do it with like video we can show that because in this case we're gonna run out of time but we're probably already out of time but we're just gonna do it on images but of course you can do this exact same thing on video because video is just a succession of images so you know we can probably do something on video and then kind of make this a little more advanced so I've got lots of other things to show you depending on interest here but anyways for now we'll just have a static color but that you can check the text based versus a tutorial for a a function Daniel came up with which is pretty cool so now we have our color now we want to actually draw the rectangle so to make a rectangle it's just CV to rectangle now again so CV to kite is a little better at CV - they know a lot of things there so lots of great suggestions from here so if any way image so we're gonna draw the rectangle on the image we're gonna draw it at top left I have no did I hit my insert I have no idea what I've done or is this just frozen on me I think I think I might've jit yeah that's ugly oh dear just what you want to happen during a tutorial yeah yeah I don't really know if I should open sublime and it will restore ok I think worse I think we've survived [Music] okay hmm so okay see v2 dot rectangle so top left bottom right yes and then whatever color we decided to use and then the frame thickness now that'll draw the rectangle but we also kind of want to label the rectangle so the next thing that we're gonna do is again a top left bottom right so I'm actually gonna copy this come down here and paste now this rectangle is going to be just like at the bottom so we've got a big rectangle then we're gonna have a smaller rectangle and this is just going to house the text at the bottom of that rectangle so in this case we don't want exactly the same dimension so it's actually going to be three two and then we're gonna pull it upwards so we're actually gonna say 1 and then at the phase 2 we're gonna say plus 22 pixels up okay so top left bottom right cool mm so now we're gonna draw that rectangle and actually again I'm going to copy this I'm going to come down here paste so image top left bottom right color and then rather than frame thickness and CV to see two dots that's it just filled maybe yeah filled see that's convenient so so now we've drawn that rectangle now we want to actually so the rectangle is there it's filled with green whatever color was or the custom color if you want to use a cool function now we're ready to actually put the text there so we're gonna say CV to put text yes image what's the text whatever the match was and then the location for the text is going to be this like three two but then we're gonna kind of place it in a slightly different location so at the this coordinate we're gonna say plus ten and then over here we're gonna say plus 15 just to kind of adjust it somewhere in the middle then we're going to specify our font so the font that we want to use again we'll see v2 dot her.she simplex they'll be fine then we're gonna say 0.5 will be the size now you can specify the color in this case we'll just do like kind of like a barely off-white and then whatever our font thickness was if you want to make that a multi-line function our multi-line of parameters feel free that's quite quite a long line okay so that puts our text on the image and now now that we've done that basically what we want to do is we we want it for every unknown image we want to do this whole block all at once and then when we're all done right for each image we want to show the image so I'm gonna say see the - whoops see me - dot what em show and we're going to show the filename and then we're going to the actual image did that just freeze again on me are you kidding me okay trying that again new show I need to figure out why is that lagging out okay so we Emma show this is the title of the image just in case you don't know what cb2 is doing and then this is the actual image that we wish to show and then we gonna do C v2 dot weight key 0 so just as soon as we're ready and then C V to say v2 dot destroy all window or the window it really doesn't matter but if you want to destroy the specific window you can give the filename now I'm on a boot and for whatever reason open CV doesn't like Kubuntu very much and the destroy window won't really work so my solution is - so if you're on Windows and maybe Mac I'm not sure but I think it's for any Nix type of operating system but anyway I'm gonna comment that out and I'm actually gonna say CBT got way key ten thousand so it's in milliseconds so ten seconds but if we press a key at any point we'll continue to the next image but anyway it's super messy I'm so ready work with open CV on a boon to to actually I work with it but I don't really display things live so I haven't really run into this issue very much but anyway that would be a temporary solution to that problem that's not the directory we want to be in so let me just open it here open in terminal will do Python face rec example that PI so let's see have we done so it loads the known faces boom okay no match here that's good I'll press the key hopefully we're getting great awesome I think we just closed that it'll be no good so I just keep pressing keys for now okay so then here we've got a ton of matching syntax I did not detect that as Centex okay but did clearly detect but put your jokes in the comment section clearly detected a definite Snowden faces Centex that's funny okay no detection there detection there but also detection like clearly I'm just a man with glasses to the machine because both of those okay fine also another deep fake although this one's a little less far along so no real surprise there no detection that's good that's good a pretty challenging detection and it got that one that's pretty cool another good detection okay so that was all them very good so the only other thing is okay so in that one image right like we got a false positive right that guy in the background so the next thing you could do is like okay well let's say we want to lower tolerance you know like I don't know let's go with 0.3 tolerance let's run that again nothing nothing Wow none of these actually move we're in trouble I think 0.3 is gonna be too low oh boy so so we did not get that guy in the background but as you can see you said the tolerance too low you get no matches so many false negatives here really not a single match not a single match okay let's try a 0.5 let's just go slightly different than the other tolerance okay so that worked out pretty well I mean I missed some of these clear examples but that's okay got that one did not get that one which is actually correct but missed that one I do think that's a hard one got that one though cool okay so you can see that you could kind of play with it and I will say if you had more faces more examples you're gonna be even more accurate you can keep your tolerance kind of high you're gonna get much less false negatives so yeah feel free to tinker around like I said you can do this not only on images but also video framerate is gonna vary significantly depending on what kind of machine you're running it on if there's enough interest I will follow up this video with another video doing this with video and then hopefully adding something a little more unique and interesting to that and we'll see if we if I end up doing it but if you have questions comments concerns whatever feel free to leave those below otherwise I will see you guys in another video

Channel: sentdex

Views: 207,922

Keywords:

Id: 535acCxjHCI

Length: 31min 4sec (1864 seconds)

Published: Sun Mar 08 2020