Build a Deep Facial Recognition App from Paper to Code // Part 1 // Deep Learning Project Tutorial

what's happening guys my name is nicholas renate and welcome to the very first episode of papers to code in this series what we're going to be doing is building our very own facial recognition app using deep learning from scratch so this series is all going to be to do with grabbing the data building up the model and implementing it inside of an application ready to do it let's take a deeper look as what we'll be going through so in the very first episode of this bigger series what we're going to be going through is just our general setup so what we'll do is we'll install our dependencies we're going to import all the different tensorflow specifically deep learning components that we're going to need this includes our layers our models and our inputs we're going to also configure our gpu so if you're using collab or if you're using your local machine this is going to ensure that your don't blow out your gpu completely and then what we're going to do is we're going to set up our data folder so there's going to be a bunch of episodes following this but this is really just the setup ready to do it let's get to it alrighty guys so we're going to keep this pretty simple and easy at least for the first episode in the facial recognition or face id type series so what we're going to be doing in this episode is we are going to be installing our dependencies importing our dependencies setting our gpu growth and then creating our folder structures now what i'm hoping is that this sort of episode of this style of video is going to become a little bit more common on the channel so we'll be able to take papers and convert them to code and actually implement them with practical examples so if you like this style of series do let me know in the comments below i'd love to hear your feedback so what we are going to do in this particular case is first up we're going to install our dependencies now the first part of this series is going to be all inside of jupiter so this is all going to be the data science and deep learning side of things where we actually create our facial verification model and we'll talk a little bit more about the paper that we're actually going to be interacting with a little bit later but we'll see that and then once we've gone and done that we'll actually transition and start coding up in vs code when we go and build our facial verification app so first things first let's go on ahead and install our dependencies all righty so those are our dependencies now installed now let's actually take a look at the line of code that we wrote so first up we wrote exclamation mark pip install so this is your standard python install and as per usual we're going to be taking this entire tutorial step by step so again we're going to write the code and explain it so exclamation mark pip install tensorflow equals equals 2.4.1 so this is going to install the tensorflow 2.4.1 version into our environment then we're using or we've imported or installed tensorflow gpu equals equals 2.4.1 and if you don't have a gpu on the machine that you're running on there's a few things that you can do about it you can try running this tutorial again without a gpu but it's going to be a little bit slower to train so which is perfectly fine it'll still work if you want to use a gpu there are a whole bunch of cloud platforms out there that leverage jupiter notebook so colab is a free one that you might take a look at but in this case if you don't have a gpu it's perfectly fine then we've installed opencv and so in order to do that written opencv dash python and this is going to give us some of our image processing capabilities and we're really going to be using this when we actually go to do real-time testing right at the bottom of this tutorial and then we've gone and installed a matplotlib as well cool so all up we've installed four different dependencies so exclamation mark pip install tensorflow 2.4.1 tensorflow gpu 2.4.1 opencv python and matplotlib so those are our four dependencies now installed cool so that is part 1.1 and now done now the next thing that we need to do is import our dependencies so we've sort of got two different classes of dependencies that we want to import here so we've got our deep learning stuff so all of our tensorflow components and then we've got some sort of standard dependencies that are more going to help us actually work with our data so let's go ahead and import our standard dependencies first and then what we'll do is we'll import our tensorflow dependencies after that okay those are our standard dependencies now imported so i've written one two three four five lines of code there and one comment so the comment just says we're importing our standard dependencies again it's good practice to comment your code so what i've written is import cv2 so this is going to import opencv into our notebook then we've imported os so this is just an operating system library and we are going to use this when we actually go and create our folder structures so this just abstracts the operating system from our python code so it just makes it a whole heap easier to work with different directories different file paths then we've imported random this one is a little bit optional but i've brought it in just in case so if we are testing uh generating new data this comes in handy then i've imported numpy so import numpy as np and numpy is a great library when it comes to working with different types of arrays so tensorflow which is the deep learning library we're going to be working with is all to do with different tenses or working with different tenses numpy helps us work with those so what we're then going to do is import matplotlib and numpy is really just a really great array transformation library so again if you need to resize or reshape erase comes in super handy there so we'll probably use it later on then we've imported matplotlib so from matplotlib import pipeline as plt the main reason that i've brought in matplotlib is because there is a method called plot.iamshow so if i'd have employed.i am show this effectively allows us and question my question mark this effectively allows us to visualize an image so you can see that it says display data as an image so it just makes it super easy to visualize it when we're working with our different data components which we're going to need when it comes to facial verification cool so that's done the all right so we've gone and written those five lines code so import cv2 import os import random import numpy's mp and then from matplotlib import pi plot as plt cool so those are our standard dependencies are now imported now the next thing that we're going to go ahead and do is import our tensorflow dependencies so i'm just going to change this so we're going to import tensorflow dependencies and really we're going to be using uh well not really we're going to be using two key oh we need to import two key things here we need to import our model components and the different deep learning layers that we're going to need now what we're actually going to be doing is we're actually going to be using the tensorflow functional api so a lot of tutorials tend to use this sequential api but what i've actually found is the functional api is a lot more flexible when it comes to building really hardcore deep learning models which in this case this one actually is now the model that we're going to be building is called a siamese neural network so this actually allows us to do something called one shot classification we can actually pass through two images and determine whether or not that image is the same so if i actually bring up that paper let's see if i can bring it up and i'll link to this in the description below as well so basically what we're actually going to be doing is we're actually going to be building up this network described in this paper so siamese neural networks will one shot image recognition and if you actually take a look at the architecture here so this is really important so we are going to have two inputs so you can see that there is a break between these hidden layers here so we've got this input and we've got this input what we're effectively doing is we are passing through two images at the top at the same time and then we're going to have this distance layer so this actually measures the similarity between the two images and we're going to train our neural network to determine what that similarity is like so if it is very similar then we're going to output a one which basically means it is verified if they are completely different then we're going to output a zero which means we are unverified now in terms of the neural network that we're going to be building we're effectively going to be implementing this or something very very similar to this i think the input shape that we're going to be passing through is a little bit different we'll have 100 by 100 but you will see that towards the end of this series so we're actually going to take this model and actually implement it in real time which is super cool all right we've digressed so this is the model that we're going to be building we're going to be using the tensorflow functional api and i will link to this in the description below but for what we need to do now is we we actually need to import those functional api components a functional api so let's go ahead and do that okay those are our functional api components now imported and we've also imported tensorflow as well so let's take a look and take a step back as to what we've actually gone and imported so what i've written is from tensorflow.kstop models import model this is probably one of the most important layers that you're going to need so when you're actually defining your model using the functional api what you effectively do is you type your model and you pass through two things so you pass through inputs and then you set that and then you also pass through outputs and you set that so what we're effectively going to be doing is we're going to be building up our neural network and then passing through what we want our inputs to be and our outputs to be so this is a slightly different to the sequential api but it allows you to define a significantly more sophisticated neural network so ideally what we'll actually have is we'll have our input image put this in brackets our input image and then our verification image and what we're effectively going to get out is a layer which effectively outputs one or zero so that is effectively what our model is going to look like by the time we actually get to it so that is what the model class is used for so in order to do that i've written from tensorflow.keras.models import model and this is effectively giving us this and then a big part of our deep learning models are a whole bunch of different layers so in order to do that we've imported a bunch of different layer types so we've imported or we've written from tensorflow.keras.layers import layer so this layer class is a high level class so this actually allows us to define a custom layer and when it actually comes to doing that we can effectively create a whole new class to actually go and generate that layer so it's effectively class um l1 or the class that we're actually going to be implementing is going to be called l1 distance and we can actually pass that layer through so we can perform inheritance for that layer so this effectively allows us to create a custom neural network layer so we'll actually be going through some really cool concepts as part of this bigger series okay so we've imported from tensorflow.layers import layer and then we've actually gone and imported some standard layers so conf 2d this allows us to perform a convolution so if you've heard of convolutional neural networks that is exactly what that allows us to do we've imported dense so this gives us a fully connected layer which is effectively what you typically see in most neural networks that sort of looks like these layers that you're seeing here we've then imported max pooling so this allows us to pull our layers together and effectively shrinks the information that we've actually got so this is almost like averaging or in this case it's not averaging it's actually taking the max values over a certain region to effectively reduce how much data we're actually passing through to the next layer and then we've got our input so this is again another base class so if you think about the three most important clusters we've got here it's model layer and input so our input allows us to define what we're passing through to our model and our layout effectively or our model actually allows us to compile it all together and our layer gives us our base class and when we're defining our input we're effectively typing input and then shape and you're specifying the shape there and then the last layer that we've imported is flattened so this effectively takes all the information that you've got from a previous layer and flattens it down into a single dimension so that allows us to pass convolutional neural network data to a dense layer so that's what we're going to be using that for and then we've imported tensorflow as tf so import tensorflow as tf so let's quickly recap on what we've imported to from tensorflow.keras.models import model from tensorflow.ks.layers import layer comma conf 2d comma dense comma max pooling 2d comma import comma and then flatten and then we've imported the tensorflow base or the tensorflow class overall or tensorflow overall so import tensorflow as tf so we're going to use this for our image processing and a whole bunch of other helpers but that is section 1.2 now done so what we can actually do is uh delete this because we don't need that and we can delete that so that is section 1.2 now done so what we can now do is go ahead and set our gpu growth so this you only need to do if you're doing it on a gpu based machine so if you're not doing this on a gpu on a machine that has a gpu you can skip this step what we're effectively doing here is we're limiting how much vram tensorflow can use on the gpu because by default tensorflow is going to expand to take it all over and if you don't go and set this then you're effectively going to run into out of memory error so this helps avoid this so let's write a comment so avoid out of memory errors by setting gpu memory consumption growth right so think of this about like or think of this as like putting some chains around your gpu to let it go to stop tensile from from going crazy or it's more chains around tensorflow than the gpu but effectively we're going to be locking intense flow down so it doesn't go crazy right so let's go ahead and do this oh okay hold on we've uh list fizz okay so that is our memory consumption growth now set so we've gone and written three different lines of code there so first up what we're doing is we're actually accessing all of the different gpus on the actual machine so if i actually type out gpus you can see that we've actually only got one there so if i take a look at the length of that so you can see we've got one gpu so in order to get that we've written gpus equals tf.config.experimental dot list underscore physical underscore devices and then we've passed through a string which says gpu so what this line is doing is it's actually getting all of the different gpus that are available on our machine then we're actually storing that inside of a variable called gpus which is effectively what i'm showing here right so you can see it says physical device name equals forward slash physical underscore device gpu zero so this basically means that we have a gpu here in this case the gpu that i'm using is a rtx 2070 super so i think it's got around about eight gigs of vram so just something to keep in mind sharing some information you know all right then what we're doing is we're actually going through and we're setting our memory growth so we are looping through all of the gpus so again if you have multiple gpus this is going to work as well which is why we do the loop so for gpu in gpus that's effectively just looping through this a4 gpu in gpus we could effectively print gpu right so we're effectively looping through them but then what we're doing is we're actually setting our memory growth which is this function which you can see right there so i've written tf.config.experimental.set underscore memory underscore growth and then we're passing through the gpu that we're processing at that time so if you remember we're looping through each one of them up here this is actually going to go and set that gpu and set memory growth equal to true got a bit tongue-tied there all right cool so you can see that that is now done so we're grabbing all of our gpus we're looping through each one of them if we've got multiple this works and then we're using the set underscore memory underscore growth method to set our gpu growth equal to true and remember we're passing through that gpu and passing through true to actually set that cool so uh we can delete this cell down here so let's delete that and that is section 1.3 now done now the next thing that we need to do is a little bit of setup so we actually want to set up our folder structure so that we are in a good place when it actually comes to getting our data so if we actually do let me actually show you where i'm working what i'm working with so the folder that we're current i'm currently working in is looks a little bit like this so right now i've just got a virtual environment i've got some jupiter notebook checkpoints and i've got the actual notebook that we're working in at the moment what we need to do is create a couple of folders so we are going to create three specific folders for our data we're going to create one called anchor one called positive and one called negative now let me explain what each one of these is going to do so if we actually take a look at our neural network let's see if there's an example of it okay this is a good example so when we actually go to perform our facial verification we're actually going to pass through two images we're going to pass through an image called an anchor which is think of this as your input image right so this is if you're going and doing it in real time it would be the image that's captured from your iphone or the image that's captured from your webcam or whatever image that is verifying you in real time so think of this as your anchor we then pass through a verification image or a um positive or negative image so what we're effectively going to be doing is saying hey does our anchor match the positive image so in this particular case you can see the soccer ball matches the soccer ball so it outputs same when our neural network does it it will output one we then also pass through a our input image again or our anchor image and then we verify it against the negative and we want to make sure that that is different because we effectively want our neural network to be able to distinguish between ourselves and another person so we're going to call the input image and anchor image we are going to call the verification image which is positive the positive image and we're going to call the verification image which is a negative example the negative image now all the negative data we're actually going to get from a repository called labeled faces in the wild which is a sort of like a standard repository but we're going to be doing that in another tutorial i'll show you all in another episode of this series but i'm going to show you how to set all that up so when we actually go to collect our positive and our anchor data we're going to be using our webcam for that so again we're going to be going through that i believe in episode 2 of this cool that is that uh explained so what we now need to do is actually create some folder structures so what we'll first up do is we're going to set our paths and then we'll actually use the os library to actually go and create those directories so let's set up our paths first up okay so those are our paths set up and we haven't actually created the folders there we're just defining what our paths are going to look like so this is effectively what our directories are going to look like i'm calling them paths but it's just our directories right so i've written pos underscore path and i've just defined all of our paths in negatives because it just makes it a little bit easier to understand what they are so pos underscore path equals os dot path dot join so what os.path.join does is it effectively joins different directories together to be able to form a full file path so what you'll get out of this is data and then it's either going to be a forward slash or a double backslash depending on what type of os you're using so the file path is going to be data and then forward slash or backwards positive so if i actually take a look at that and type out positive path you can see this is just returning the file path that we're about to create and then we've gone and done it for our negative images and our anchor images so remember our positive images are going to be our positive verification images and negative images are going to be our negative verification images and our anchor images are going to do the same now when it actually comes to implementing this on a or on a real life implementation i've got a slightly novel or special implementation that we're actually going to use but again we're going to do that more later on in this series but um we've got something special to make this actually work really really well all right so we've got positive path negative path and anchor path and then what we are going to do now is create those directories so let's go ahead and do that and let's actually just take a look at the folder so we're going to basically create one big folder called data and then inside of there we're going to have three different folders so one chord positive one called negative and one called anchor now what we need to do is actually go ahead and create those so let's go on ahead and do that so i'm just going to delete this cell here and then create a new one let's do it okay so those are our directories now made let's add a comment there make the directories so in order to do that i've written os dot make dears now keep in mind it is m a k e d i r s not mkds uh make dears actually goes and creates all the subsequent folders so if you don't have a top level directory created it's actually going to create the full file path so i've written os dot make dears and then i've passed through our positive path os dot make dears and then pass through our negative path and then os dot make dears and pass through our anchor pass so effectively doing this part this path and then this path now if you wanted to you could put this inside of a list and then loop through them i figured it's just as easy to do this but basically we're going to be creating all of our different directories so if we now go into our folder and mine is inside of youtube and face id you can see let me zoom in on that so you can see it a bit better so you can see i've now got this folder called data and inside of that i've got three folders so one called anchor one called negative and one called positive there's nothing in there at the moment but our three folders are created and you can see that we've also gone and created this top level data folder so everything that we we're going to use when it comes to getting our data is going to be in here and then in one of these three repositories cool all righty and i think on that note that is us done so we've gone and now created our folder structures so we defined our paths and we went and set up our directory so we've gone and done a whole bunch of stuff in this tutorial so we went and installed our dependencies we went and imported them we then went and set our gpu growth and we also went and created our folder structures to get ready to actually start collecting our images and building our face id or facial verification models so on that note that is this tutorial now done see you in the next one thanks so much for tuning in guys hopefully you enjoyed this video if you did be sure to give it a thumbs up hit subscribe and tick that bell and let me know if you've got any questions or comments in the comments below thanks again for tuning in peace

Channel: Nicholas Renotte

Views: 7,746

Keywords: face recognition, facial recognition, face recognition with python

Id: bK_k7eebGgc

Length: 23min 53sec (1433 seconds)

Published: Wed Sep 08 2021