Face recognition + liveness detection: Face attendance system

now I'm going to try to login into this system but I'm not going to log in with my own face but I'm going to hold my picture to the camera let's see what happens I press login and I get a welcome Felipe so you can see I am logging in into this face attendance system holding a picture of me there are many cases where people are trying to spoof these systems and they are trying to do something like I'm just doing right now right they are trying to hold a picture of someone else and they are just pretending to be someone else this is a very very very important problem in the industry in the computer vision industry and there is a lot of research involved and there is a lot of resources assigned in order to solve this problem in order to come up with different solutions in this case it's not really a problem because in this case it's me holding a picture of me but you get the idea right in many other situations there will be other people holding pictures of other people so this is definitely something we need to solve so what we are going to do today is that we are going to use this software this face attendance system but we are going to integrate it with this spoofing detectors so we are only going to log in or log out our users if they are really users if they are people if they are like people in front of a camera and we are going to take this type of situations of someone just holding someone else picture and we are going to deny the login or the logout if the user is doing something like that so let me show you the three repositories we are going to use in today's tutorial the first repository we are going to use today and actually the most important repository is this one which is one of my previous projects an easy project regarding the face attendance system I show you how to make in one of my previous videos and this is very important because absolutely everything we are going to do today into this tutorial we are going to build it on top of this previous project so this repository is super super important and is one of their posters we are going to use today then I mentioned we were going to work with spoofing detection we are going to detect smooth things and this is exactly the repository we are going to use in order to detect or spoofers it's called silent face anti-spoofing and this is exactly the repository we are going to use you may notice this is Chinese if I'm not mistaken everything is written in Chinese and I'm not sure about you guys but I I definitely don't speak Chinese and if any of you don't speak Chinese either don't worry because we have another rhythmic file which is reading incomplete English absolutely everything is written in English so this is another repository we are going to use today and then the third repository which is also very very important is exactly the same repository we used in my previous project right we are going to use the same face recognition Library we used in my previous project so we are really familiar with this repository but I am just showing you that this is another posture we are going to use today so the in order to get started with today's tutorial the first thing I want to mention is that I have noticed many of you have had some issues have had some challenges when you were trying to execute this a project right if I'm not mistaken it was basically many of you Windows users were having some issues we're having some challenges so I have been doing a lot of research so I can see exactly how I can help you all of you in order to solve this problem and I have come up with a very good solution so let me show you the first thing all of you Windows users will need to do in order to in order to work on today's tutorial in order to execute the face attendance system I showed you in my previous video is to follow a few steps and the first one is described in this video I am going to post a link to this video in the description of the video you're currently watching and going to post a link to this video so you know exactly what you have to do in order to work on today's tutorial in order to work with the face recognition Library some of the issues you guys were having were due to the face recognition libraries and some issues or some incompatibilities which is related to Windows so this is exactly the first step you should follow you should take in order to in order to solve the issue then another thing which is going to help you a lot is to use a different set of requirements you may notice that now in the repository of this project we have two files two requirements files one of them is called requirements txt and the other one is called requirements windows.txt and this is exactly the requirements file all of you Windows users will need to install right this is a very specific set of requirements which is exactly the requirements you will need to install in order to work on today's tutorial and now let's start working on today's tutorial and the first thing we will need to do is to clone this repository let's clone the face attendance system repository so I'm going to copy this repository URL and I'm going to pycharm this is a python project I created for today's tutorial these are completely and absolutely empty project you can see that the only thing it's in this project is a virtual environment that's the only thing and it's a completely completely empty virtual environment so I am going to the terminal of this python project and I'm going to say something like it clone under repositories URL so the repository is cloned and then if I go here I should be able to see it here yes okay so the first thing I'm going to do is to install the requirements for this project remember if you are running this in a Windows computer please remember to install the windows requirements right but in our case I am running this in Linux so I'm going to PP install the requirements file which is in face attendance system requirements oh sorry that's not going to work because I forgot the txt okay so this is going to take a couple of minutes and in the meanwhile let me show you a few edits I have done to this code and I invite you to take a look at each one of these commits so you can see exactly what I have changed in this project but it's basically a few changes regarding how the face recognition Library it's executed and also the an additional functionality I have a little this repository so this is for example some changes in how the face recognition is executed and I I invite you to take a look at it and invite you to become familiar with how to read a commit basically you can see that everything that's new is green and everything that has been removed is red and this is not really that hard but it takes some time to become more familiar how to read a commit so I invite you to take a look at it in the repository of this project and also uh yeah I invite you to take a look at absolutely all the changes I have done now let's go back to pycharm let's see if it's completed everything is completed so we can start working on these projects so I'm going to Main and the only thing I'm going to do is to execute this file and let's see what happens so I'm executing this file I have already connected my webcam and you can see it's taking my webcam so let me show you how it works let me remind you how it works the idea is that each new user is going to register so for example in this case I'm going to register into the system so I'm going to look at the camera I'm going to try to smile and see what happens okay yeah that's okay that's a good picture let's say we can use this this picture my name is Felipe so I'm going to accept and then I am going to try to login into this system so I'm just going to press the login button and I have been testing this face recognition Library I have been testing this software and I am amazed with how well it performs for example in this case I'm not looking at the camera the camera is right here and I'm not looking at the camera and take a look what happens if I try to log in even if I'm not looking at the camera I am recognized with a absolutely no problem so the face recognition library is super super robust and that's very important now I'm going to continue testing the other feature which is the log out button and if I just press this button I am not going to look at the camera I'm just going to look straight to the computer and I press logout and I get with by Felipe so everything seems to be working well in order to show you like a more comprehensive test of how it works remember that in my previous video I show you a few tests uh logging in a picture of Rolo and a picture of capital Picard now I have other pictures which I'm also going to show you which I'm also going to use in order to show you how this works so in order to remember how this works so I'm going to register more views into this system you can see it is a picture of Morpheus I'm going to Press Register new user let me uh hold it well something like this register new user input a user name and this will be more fuse except a user was resistor successfully and I'm going to do the same with data I'm going to use a picture of new tenant commandant data if I'm not mistaken so I'm going to register an user username will be data and I'm going to accept user was resistor successfully and now let's see what happens if data tries to log in into the system we get welcome data so everything works properly and now let's try to do the same with more views I'm going to Press login and welcome more views everything is okay and I have also a picture of me this is a picture of me so this is me holding a picture of me now I'm going to try to login into this system but I'm not going to log in with my own face but I'm going to hold my picture to the camera let's see what happens I press login and again welcome Felipe so you can see I am logging in into this face attendance system holding a picture of me and this is exactly why we are going to use this spoofing detector because this is a very well known problem in the face recognition industry because once we are developing software in order to recognize a person's face or in order to recognize a face in a picture there are many cases where people are trying to spoof these systems and they are trying to do something like I'm just doing right now right they are trying to hold a picture of someone else and they are just pretending to be someone else this is a very very very important problem in the industry in the computer vision industry and there is a lot of research involved and there is a lot of resources assigned in order to solve this problem in order to come up with different solutions in this case it's not really a problem because in this case it's me holding a picture of me but you get the idea right in many other situations there will be other people holding pictures of other people so this is definitely something we need to solve so what we are going to do today is that we are going to use this software this face attendance system but we are going to integrate it with this uh spoofing detector so we are only you going to login or log out or users if they are really users if they are people if they are like people in front of a camera and we are going to take these type of situations of someone just holding someone else picture and we are going to deny the login or the logout if the user is doing something like that so in order to continue with this tutorial what I'm going to do first is I'm going to show you how this spoffy detector works so the next step will be to clone this repository and I'm just going to show you how it works for now right I'm just going to clone it in a random location in a random directory I'm going to close it in my desktop because for now the only thing I'm going to do is I'm going to show you how it works and then in the next step I'm going to show you how to integrate this spoofing detector into the face attendance system so the first thing I'm going to do is to create a virtual environment remember that the only thing you're doing now is to test this repositories I can show you how it works and then we are going to integrate it into the face attendance so forth so I'm going to call virtual M bam I'm going to use pylon 3.8 and then I'm going to activate this virtual environment so now what I need to do is to install these requirements but I have been doing some tests already and it's going to be a much better idea to change some of the versions of these packages so I'm going to install the same exactly the same packages but with different versions and I'm going to post exactly what are the packages I am using obviously but this is only to let you know that I'm doing a few changes regarding the um the requirements which are in this repository so I'm going to PP install this file and then I'm just going to wait let's see what happens PV install minus r requirements I have to wait a few seconds but this will be enough in order to install all these requirements and something I have to say is that I have been doing many tests of this repository of this spoofing detector and I would say it works very very very very well and we can definitely use it in our project but I can also say that there are some situations where I tested this algorithm this detector and it's um it's not perfect so we are going to use a spoofing detector which works very well but it's not 100 perfect that's something that you should be aware of and I'm going to say it's not perfect it means that in some of the images I have used this repository this project is not really detecting the spool things and some pictures which are actually real pictures it's misclassifying them as spoofs so the but nevertheless I think it's a very very good repository and we can perfectly use it for now and this is going to be a very very good example because when you are creating a project when you are building a solution you definitely want to take a look at all the available Solutions and everything you can just use out of the box in order to build your product in order to build your solution but some of these Solutions may not be 100 fit to your specific use case or to your specific needs so this is a very good example that when you are building a solution you definitely want to use all the available resources you can find but you can just use it like a way to start working on that project or to start working on your solution right if we will want to make something that's like a 100 accurate spoofing detector for the specific images or the specific use case or the specific environment we are going to use this system in what we will have to do what we will need to do ideally is to collect datam to annotate data to train a detector to do something right if we are using something that's publicly available it means it's a very good starting point it means that's going to be a very good solution in order to start working on this project right so that's something we should be aware of the delivery we are going to use the project we are going to use it's very it's functional and it works very well but there are some cases in which we are going to have some Mis detections so let's continue other repositories where all the packages were successfully installed and what I'm going to do now is to test how it performs and I'm going to use one of the images in their examples right they provide a few examples you can see for example in this case it's someone called in a picture in this or case it's someone holding a cell phone with a with a purse on it and in this case it's a real person so these are a few examples which they provide and what I'm going to do is just to use these pictures in order to show you how to execute it the idea is that we are going to call the test script we are going to call this script and we are going to just run it and what this script does is to call anore function which is a function which is also called test and the only thing we need to provide for this script is the image we want to inference the image we are going to we want to attack if it's a spoofing or not and also we want to detect the we need to input where the models are located right so let me show you how it works I'm going to input one of the images I show you in these samples so I'm going to test dot pi uh sorry python test.vi and I'm going to input image F1 there's something I'm just going to there are a few changes we need to do obviously but let's see how this is used okay so I'm just going to leave this empty as I mentioned this is a perfect and amazing repository we can use as a starting point but obviously we have to make a few changes if we if we want to fit it into our purpose right so the image name will be the full parts and then image f1.gpg okay let's see how it works with this image I'm just going to run it and it says this is a fake and it also provides a score and this is exactly the output which is actually the same image but with a bounding box on top and it says a fake underscore right so it's basically that's what we we are getting so I'm going to execute it for this one as well because these are real image and we get this enemy C mesh T1 so I am going to replace this by image T1 uh okay and you can see in this case we get real and the score is 0.99 right and then we have an output which is uh exactly the same image but with a bonding box on top so this is exactly how it works and you can see that this is very very easy to use the this is very very straightforward the only thing I did was cloning the repository I install the requirements I had to change the versions of the libraries but that's only like a detail and then in order to execute the test in order to execute the inference of this spoofing detector the only thing I had to do is to execute the test file and now it's the best part of this tutorial because it's when we are going to integrate this buffing detector into our face attendance app so this is how we are going to do we are going to follow a very similar process as we just did when we were testing the spoofing detector but we are going to do it from here and so I am in the in my python project I go into the terminal and I am going to 2 CD into face attendance system okay and now I am going to clone this spoofing detector here so something like git clone okay now we are going to clone the entire repository and I am going to take the requirements file and I'm going to make exactly the same edit as I did here so I'm going to copy this versions and I'm just going to paste it here and now obviously we need to do something like pip install minus r and this requirements file which is here okay so requirements are now installed and now let's go to test because I am going to use exactly the same script as before but now I am going to use it as a function I'm going to use it I'm going to import this function from my main script so we are going to use this test file and I'm going to do a few changes into this file but the first thing I'm going to do is go into settings and then project structure because we need to add this directory into my sources because remember we want to import a directory we want to import a file which is in this directory and otherwise it will be a little hard to do it so that's going to be the easiest way to do it so now I'm going to Main uh let me do something like this in Maine I'm going to say something like from test import test right we are going to import the test function in this files so we are going to import this function and now it's the time where we are going to make a few edits into this function because this is a very good starting point but we need to do a few edits the first thing we are going to edit is that we no longer need to write the output as we were writing it before remember I show you we every time I execute the test file we generated an image with a bonding box on top and the result and so on and we don't need to do that anymore so I'm going to delete all of these also the rectangles and also the print ok and something else I'm going to do is that we don't really need the we are not going to input an image path but we're going to input an image so what I'm going to do is this will be image and this will be something like this right we are not going to read an image from our hard drive but we are going to input an image that's another change and then another thing we need to do we will need to resize all of our images so they comply with this right and we need to do something like image equal to CV2 resize so this size we need for our image will be something like image.shape I'm just going to write it first and I'm going to explain it in a couple of minutes times 3 divided by 4 and then image dot shape zero right and I need to cast this as an integer or what is not going to work and this is a requirement in order to execute this script absolutely all the images we are going to inference absolutely all the images we want to know if they are spoof or not we need them to comply with this image size right when in this relationship regarding the width and the height of our images this is related to how this model was trained that's what that's what I think and this is exactly what we need to do and this one already changes we need to do and then we need to do over changes if I look at this function that's pretty much all we have already removed the last Parts something we need to do we don't really need this anymore just going to read it and then what I'm going to do is I'm going to return the label right the label information is the one containing if the image is a spoof or not and if I look at this uh if label is 1 means it's a real face and if it's zero means it's a fake okay let's remember those values for now the only thing I'm going to do is to return label and I need to do an additional edit which is in another function let me take a look this is a function we need to edit so this is under source and this is called anti-spoofpredict.pi and basically you you can tell that we have all of these hard-coded values we have all these hard-coded paths and we are going to make an edit which is exactly this one I already prepared the um the sentences we need to in order to replace these two lines and these are the sentences we are going to use and another edit I need to import traceback and that's pretty much all right long story short this is it's never a good practice to hard code anything and especially something like a path to a pad to a file so this is basically what we have done we have made the what used to be something hard-coded into something that's relative to the user's pad location so it will users current directory and this is a very good edit that's pretty much all if I remember if I remember correctly and now let's go back to main because we need to add some additional logic because remember that we only want to log in or log out our users if they are real users so this is where we are going to do so I'm going to take my login function and I'm going to do exactly the same with my logout function and I'm going to do something like if test I need to input three parameters image and let's see what are the other two parameters moldier and device ID so this will be something like this and something like this and the image remember this is the actual image this is the numpy array so I am going to input most recent capture then model directory this is the model directory of my let's see I have so many directories code this is exactly this directory right this directory into inputs let's do like the absolute part okay and then device ID I'm just going to leave it in zero um and that's pretty much all so if this remember is going to Output a number which is zero or one one means that it's a real image and zero means is a fake so let's do it a little better and let's do something like this remember this will be labeled and then I am going to ask if label is one and if it is I'm just going to proceed as usual and if it's not I'm going to print something so I can say something to the user because if it's not one if it's zero means the user is actually ah it's trying to spoof this system it's trying to do something that is he's not supposed to be doing so I am going to Output a message box saying something like hey you are a spoofer and then um I don't know something like you are fake absolutely it doesn't really matter the idea is just that we are going to see this message box every time the label is zero so we are only going to use the data test but you get the idea we're just going to tell the user that he's an imposter that he's a fake so now let's continue now this is pretty much all we have to do I'm going to do exactly the same in the logout function you can notice that we are repeating code and that's never never never a good idea but let's just let's just do it like this for now and we can do it better in the future let's see how this works or if we have to make an additional edit I'm going to run this process again so now I'm going to try to log in I'm going to try to do it with my own face I mean I'm going to do it with my real face and then I'm also going to test with all these different pictures I have over here so I'm going to start uh I'm going to do it first I'm going to be super super still because I noticed that if I move remember that we are taking frames continuously and we are plotting these frames and yeah so it looks like it's a video it looks like it cons it's continuous but remember we are just plotting one frame after the other so if I move it's going to be blurred and the the spoofing detector may think it's a spoof so I'm just going to be super super still as a real person which is like a very good thing to do because I am a real person and now let's see what happens if I try to log in with all these different pictures so this will be a picture of Morpheus and it says you are fake which is uh exactly the case because this is a picture so let's see what happens with my picture of data something like this in the uses Enterprise will have been amazing right so something like this and it says you are fake which is also uh true because this is only a picture and then I'm going to try to log in but with my face uh in this case it's it's completely unimportant because it's just me holding my picture but remember it could be another person which is holding my picture right now so this needs to work super super properly and Express login and it says you're fake so you can see everything works super super purpley and uh yeah so this is going to be pretty much all for today's tutorial this is exactly how you can how you can add a spoofing detector feature into a face attendance system and this is going to be all for today if you enjoyed this video I invite you to click the like button and I also invite you to tell me what you think about this video in the comments below my name is Philippe I'm a computer vision engineer and in this channel I make tutorials coding tutorials which are exactly like this one and I also share my resources and my experience as a computer vision developer so if these are the type of videos you're into I invite you to subscribe to my channel this is going to be all for today and see you on the next video

Channel: Computer vision engineer

Views: 110,892

Keywords:

Id: _KvtVk8Gk1A

Length: 29min 33sec (1773 seconds)

Published: Wed Dec 21 2022