Build a Deep Facial Recognition App // Part 6 - Making Facial Recognition Predictions // #Python

what's happening guys welcome to part six in the series on facial recognition in this series what we're doing is we're going from a state-of-the-art research paper and building up a full-blown implementation with the final end product being an app which integrates that model and allows you to actually use it in real time in this part of the series we're going to be focused on evaluating and testing out our model which we just finished training let's take a deep look as to what we'll be going through so in this part of the series what we're going to be focused on is testing out the model so in the previous video or part of this series what we did is we finished training our model using our tensorflow implementation now we're going to test it out so we'll actually be able to see how well it actually performs so we'll test it out what we'll then do is we'll evaluate our performance so we're going to be using precision and recall to evaluate how well our model is actually performing and then last but not least we're going to save our model down so we can begin the processes of actually building this up for deployment ready to do it get to it alrighty guys so welcome to part six in the series on facial recognition using a siamese neural network so in this part what we're gonna focus on is step six and step seven so i'm just gonna add in a couple of cells so we can see what the hell we're working on here all right so step six we're going to focus on evaluating the model and in order to do that we're going to be using two key metrics these are precision and recoil so ideally these aim to measure so ideally these aim to measure how accurate our model is and how well it's actually performing classification now if you wanted a deeper explanation of these specific metrics i'll link to some stuff below so you can see exactly that so what we're going to do here is we're actually going to run those metrics and calculate them for our siamese neural network on specific batches and then we're actually going to make some predictions because up until now we've really gone and done a whole bunch of training and a whole bunch of pre-processing to actually get to this step but we haven't actually gone and done our evaluation so that's exactly what we're going to do here and we're also going to save our model so we can actually go and use it when it comes to actually deploying some stuff out so first things first what we need to do is import some metrics in order to perform our evaluation now remember i said we're going to be using precision and recall so let's go ahead and import these okay so i've gone and written one line of code there so i've written from tensorflow.keras.metrics import precision comma recall so these are two metrics that you're able to get from the keras library so if i type in uh keras metrics there's a metric whatever so there's a whole let's actually bring up the tensorflow version and sort of flow of my typing is a shocker today all right so there's a whole bunch of different metrics that you can use here so what we've chosen to use is precision which you can see that there so right over here and recall so in order to use these we are effectively going to do what you can see down here so we're going to set up the metric we're then going to update its state and then we're going to calculate the result so we can do that pretty easily using our model now we can do something similar for recall it's pretty much the same really we're just calculating a different metric but it is a good practice to look at a number of different metrics whenever you're going and building out models because this gives you a better estimation of what it's performing well versus what it's performing not so well at okay so we've got our two different metrics that we're going to calculate and whenever you're using precision and recall a higher number is a better number so ideally that's what you want to look for whenever you're evaluating any metrics or kpis whether or not a higher number is better or whether or not a lower number is better whether or not a negative number is better whether or not a positive number is better okay so we've got our metrics what we now need to do is actually go on ahead and make some predictions now before we do that we want to grab the battery data so let's grab a batch of data and we're going to do it from ow where is it keep scrolling nope maybe it's a little bit further down here we're going to do it from our test data which i believe we created yeah so we created our testing partition so we're going to be using this data to actually test out our model so let's go ahead and grab a part of our partition and then we'll actually be able to compute these metrics okay so that is our test data now collected so what i've gone and done is again written one line of code and what i've written is first up let's focus on this so what i've written there is test underscore data dot as numpy iterator and then a pass through a set of parentheses dot next so this is going to convert our tensorflow data set into a numpy equivalent so if i run test underscore data dot as numpy iterator what you're actually going to get back is this numpy iterator so basically it's looping through and dynamically producing one batch of data back now if i type in next it's going to get the next batch of data so you can see how tensorflow actually uses this and continuously goes to the next batch next batch next batch next batch so on now what we can do is we can use this to grab a batch of data so what we're actually going to get back is three things so if i store this in a variable so i'm just going to call it a test var super descriptive variable name so if i run testvar and take a look at its shape let's put in a numpy array first uh what have we done that np array could not broadcast shape into what have we done oh because it's going to be three different uh let's just have a look at the length so what we've actually got is we've got three different variables inside of this test var variable here now the first variable is going to be our test input so think of this as eventually what's going to be coming directly through our webcam so if i grab the first value this should be 16 different components or 16 different samples so you can see we've got 16 test inputs now if we take a look at the second value that's going to be our validation data so remember we're going to either be passing through a positive or a negative uh sample to through our siamese neural network in real time so ideally the second one is going to be whether or not it's a negative example or of a positive example so again we've got 16 of those and for our third value and let's take a look at the shape of that or what it actually looks like so again we've got images here and then the last thing that we're going to have is the labels so this is whether or not the values match or not so basically it's the classification label now we can delete this so what we've actually gone and done as part of writing this line is we've actually gone and unpacked it in real time so we're grabbing the test input the test vowel and y true which is effectively our labels so the full learning code is test underscore input comma test underscore val comma y underscore true equals test underscore data dot as part numpy iterator and then parentheses dot next and then parentheses again and that's what we're getting back so we've actually got these values pre-unpacked so if i take a look at test input those are going to be our input images test val is going to be our validation data and why true is going to be our labels boom cool so now what we need to do in order to calculate our precision and recall is we're going to pass through our test input and let's unscrew that we're going to pass through our test input and our validation data we're going to make predictions and ideally what we'll get back from our model is whether or not it is a verified person or a non-verified person so let's go ahead and make some predictions okay and there we go so we've now got our predictions back so we went and wrote what is that two lines of code there and one of them is just rendering the results but what we've effectively gone and done is we've gone and used this siamese model which we went and ran our training loop on already right up here in steps really in step five so we trained it all up and now because we've gone and trained our model up we can go and use the predict function to actually make a prediction so what i've written is predictions this could just as easily actually let's be consistent so pretty which should be equal y hat white underscore hat all right so we've written y underscore hat equals siamese underscore model dot predict and then inside of parentheses we've passed through a set of square brackets and remember when we use our sima's neural network we're going to pass through two streams we're gonna pass through our input data and our validation data so either positive or negative samples in this case what we're doing is we're passing through the values that we got back from our numpy iterator so i've written y hat equals siamese underscore model dot predict and then inside of square brackets to that function we're going to be passing through test underscore input and test underscore valve now remember order is important here as well so the input first and then the validation data and then the next line is just rendering that result so i've written y hat and these are our predictions back so you can see that we've got some positive examples so this is going to be a very small number so 1.0779765 to the power of a negative 5 which means it's going to be a value closer to 0 which means it is a negative sample this is one so you can see 1.000 to the power of zero which means it's going to be a positive example or effectively one let's see if we've got any other positive so there's a positive and there's a positive all right so what we can actually do now these look like they're pretty large numbers as well so they're 0.99 effectively what we can do is we can convert these into a classification output so either a zero or one so a binary outcome so in order to do that we basically just need to set some conditions so over a certain threshold we're going to say that that is a one or a positive example so let's go ahead and do that so it's a little bit more interpretable um so what we're doing uh post processing results okay let's do it okay so what i'm doing there is a list comprehension so what we're effectively doing is we're going to be looping through each one of the values inside of our y hat example and we're going to be converting it so the threshold that i've set is 0.5 so let me break this down so i've written 4 prediction in y hat if y hat is greater than 0.5 then return 1 else returns zero so this is just a list comprehension it's a little bit of a it's very pythonic in the way the interpret data but basically it's just doing a loop and we've got an if statement here so the full line is inside of square brackets i've written one if y hat is greater than 0.5 else 0 for prediction in y hat now the other way that you could write this is for prediction in y hat so you'd have a result array so res equals that array for prediction y hat if y underscore hat is a wait no if for pretty if prediction is greater than 0.5 then uh what are we doing so uh we are going to go res dot append one else res dot append zero so we're effectively converting this series of confidence metrics into a binary outcome so zero one now we don't wanna run it like this because it's not super pythonic so we're gonna do it this way and i've gone and done something wrong so for all prediction in y hat if y hat is great uh this should be prediction there we go okay so what we've actually got there is we've got the results of our y hat output over here so you can see that rather than having these values which are a range or a continuous value we've actually got a binary outcome now so 0 1 0 0 1. now what we can do is we can actually compare that to our labels that we extracted from up here if we go and take a look at that so what was it called y true let's take a look so we've got zero zero one one zero zero zero zero this is actually looking very good so one one one one so that's what six ones one one one one one i think we might have a pretty good model there yeah that's looking pretty good you can see that that actually matches that exactly now rather than doing this manually and visually inspecting it we can actually go and use our metrics from up here to actually test this out so let's actually do that okay so that is our metric now calculated so a couple of errors there i was just typing stuff in or passing through the wrong passing through the inputs in the wrong format so i had this inside of square brackets but that's perfectly fine it's resolved now so let's actually break this down so first up what we're doing is we're creating so we've actually written three lines of code there so we're creating a metric object and this is a metric object for recall so i've written m equals recall and then inside a parentheses or a set of parentheses at the end then what we're doing is we're updating the state so this is effectively like calculating the recall value so i've written m dot update state and we're actually passing through two positional arguments to this function so we're passing through y true comma y hat and this is what actually passes that through to our recall object so then what we can do is access the result using m.result and then we're converting it to a numpy array so or numpy value so return result or recall result cool so three lines of code and you can see there that our recall is 100 so in this particular case we've got a really good output or a really good model that's actually able to identify us versus other people so again pretty pretty good in this case now if we wanted to we could actually calculate precision as well just to keep you know on that so recall a higher value is going to be better and precision a higher value is also going to be better so again this particular case we've got a hundred percent for recall and if we do precision as well again 100 so in this particular case this is pretty good but just keep in mind we've only done this on a single batch so best practice would be to calculate it over the entire test sample rather than just doing it on one okay now if we wanted to we could actually grab another batch so all we need to do is run this line here again so if i run that run that and then again so just to prove you'll actually see that these values actually change right so we're going to get different predictions boom and if we take a look at our y true values again different values if we go and now calculate precision again 100 we can switch it back to recall so again we've got a pretty good performing model given the fact that we only trained for what like 50 epochs and we passed through what was like 300 lines of data when or 300 examples or samples uh what how much data did we actually pass through it was somewhere up here uh it should have been take take take take take take this is that anchor and i'm positive i'm trying to work out how much data we brought yes all right so we took 300 samples right so with 300 so on just 300 images of data we've been accurately able to identify myself versus other people so again pretty good in terms of facial verification now so that is done now what we can also do is we can also visualize these results i'm telling you that they match but you can't actually see anything at the moment so let's actually go on ahead and plot some stuff out let's also break this out into some sections so this first section here we're going to call it uh 6.1 so import metrics and then we are going to get a batch of data we'll actually call it make predictions and then down here we're calculating metrics and we can also just copy this down so we've got recall separately and we can have precision separately cool and then the last thing that we want to do is actually visualize our results so let's actually create another section for that what are we up to 6.4 these results okay so what we're going to do now is we are actually going to compare the results for different samples in our batch so let's go ahead and do this okay and that is our data now visualize so can we zoom out a little bit this is a bit big on the screen all right so you can see there that what we've actually got is the positive example so this is going to be our input this is going to be our validation example so by passing through our first instance let's actually take a look at our outcome so you can see that the result for that particular prediction is one which means that this is effectively saying that the people in both of these images actually match which is valid right that is me that is also me hence we've got a positive example now i'll explain this code in a second but let's actually update this and see another example so let's see a negative example so the second prediction is going to be a zero so let ideally what you should see is this first photo is still me but the second photo is going to be someone else or a different person so one and there you go so we've got different people now so that's me but that's definitely not me and the output that we've actually got from our model is the fact that these two people do not match so the third sample should be a matching example so let's go to third which is going to be index two so again you can see me here with my eyes closed we go to our fourth example that is going to be a non-matching example so you can see it's got a zero value over here inside of y true so if we run this non-matching so that is me but that is definitely not me so again you can see that it is pretty accurately working now let's take a look at the code that we write there so we wrote six different lines of code so the first line of code is setting the size of our image or the size of our plot set plot size and for this we're using matplotlib so remember we import a map plot lib right at the top here so we wrote from matplotlib import pipeline as plt we scroll on back down uh where are we yeah so read and plot dot figure and then to that i'll pass through one keyword argument which is fig size equals and then a tuple which has the values 18.8 this controls how big the plot is so if i wanted a smaller plot i could change this so say i said it's 10 comma 8 it's going to make our plot a little bit smaller so we can see it a little bit more accurately so if i set it to 18 it's going to be huge actually 10 is probably a better example because you can see that a bit more a little bit clearer okay so that is the first line which is setting our plot size then because we're actually plotting two images side by side what we're actually doing is we're using a subplot so set first subplot so what i've read in there is plot dot subplot equals one comma two comma one so what we're saying is we're going to have i can't actually remember how this works so plot dot subplot uh all right so we're specifying the number of rows the number of columns and the index so the number of rows is going to be one row so you can see that we've only got one row there and we've got two columns so one column and then two columns and then inside of those we're specifying the index so we've got one image and then the first image and then the second image so i think we could actually tweak this a bit and actually just have two columns let's try that so if i set that equal to one did that nope got errors never mind all right so we've actually got uh we've gone and screwed that up so one this should be there we go cool so we're going and setting our first subplot so we're in plot.subplot and then we're passing through the number of rows the number of columns and the index so that's going to be referencing this image here so if i change this value in our next line which actually determines what to actually plot it would change this value so if i go and change this to two you can see that i've gone it's me just smiling out of uh out of sheer happiness for doing deep learning so we've gone and changed that second index which is changing this image so that is controlling all of this is controlling this image here this second section is controlling our second subplot right now the second line that we've actually gone and written to render this is plot dot i am show and then we're passing through what we actually want to render so i've passed through test underscore input and then we've passed through that we want index number two which is what we originally got from over here so this is the image that we got from our test data batch so if we wanted to change our image we could do that and remember these two indexes should be the same to be to ideally show that you're looking at the same sample because they're going to be in ordered flow right so the second line is plot dot i am show test underscore input and then we're grabbing the third or fourth index which is going to be index key three and then we're effectively doing the same down here for our second plot so i've written plot dot subplot in this time i'm specifying i'm passing through the first row in the second column or in within the two columns and then we're specifying what index number two is which is going to be this image over here so one row two columns and then this is going to be the value in the second column cool then the next line is identical to over this over here but rather than passing through test underscore input or passing through test underscore val which is because we're grabbing our validation data which is coming from over here and then i've written plot.show to actually show it nice and clearly so if i delete this so you can see that we get this little line over here which is just a function from matplotlib if i add plot.show it renders cleanly and there you go those are our images rendered so again if we wanted to go and run this on a different batch what we can go ahead and do is run this line over here because this is going to go and grab the next batch remember so if i go and do this again or run through the pipeline again bang bang bang bang bang calculate our precision metrics which again still looks good but now our images are changing so what are we looking at we're looking at the fourth index which is going to be one two three four which is a non-matching sample which you can see there so these two people don't match if we go and grab the first value these people match they're both me pretty good right so that ideally shows that our siamese neural network is working in this particular case now the last thing that we're going to do and this is really step what is it step six evaluate our model now done so we've gone and done a ton of stuff there so we imported our metrics we made our predictions and remember to make our predictions all you need to do is use siamese model dot predict that's going to allow you to predict or make predictions and then we also calculated some metrics and did some visualization now the last thing that we need to do as is always good practice when building deep learning models is not to add just more roses to actually go ahead and save our model so this is what we're going to do now now there's a couple of nuances here because we have gone and created a custom layer which is our l1 distance layer from i believe this was step four step four section four whatever yeah section four so we went and created this uh l1 distance layer which we need to do a little bit of tweaking to our export to make sure that we save so let's go ahead and do this alrighty so let's go ahead and save it okay so the first thing that we're going on ahead and doing is saving our model weight sovereign siamese underscore model dot save and then to that we're specifying what we want our model weights to be called so in this case i've written siamese model dot h5 which if we go to where we're currently working from so youtube face id you can see it's now saved and so the date is the 14th of the 10th 2021 so you can see that that is our model now save that let me zoom in on that so you can see that a little bit better so you can see that's our model there siamese model dot h5 cool that is the first bit now done now what we actually need to do is do that slight customization to be able to reload this so let's go ahead and do that okay that is our model now reloaded so there was a little bit of customization that i had to do in the second part to be able to load up our custom models but let's actually take a look at the full line of code that we've written and again it is just one line of code i've just split it into separate lines so i believe we can just uh do that to go into separate lines nope never mind cool that's our model now loaded i think we can delete that can't we yeah okay so what we've gone and written is model equals tf.keras.models.load underscore model so this line over here actually allows us to load a model up from our h5 weight so if i go and take a look at that so the documentation says let me zoom in on this so you can see it so this function actually allows us to load a saved a model saved via model.save so that's effectively what we're going and loading up now now what we can also do is i believe we can also load up our weights save model dot load load weight so you've also got the ability to do that as well but in this case we're going to do it this way because that's the way i've developed the rest of the tutorial so just follow this weight guys please okay so i've written model equals tf.models.loadunscoremodel then the first positional argument that we're passing through is the name of the file that we've gone and saved as which in this case is going to be siamese model dot h5 which is what you can see there so that's pretty sort of straightforward there's no real issues there the second bit which is very important is actually going and loading our custom objects now because we've got these inside of our notebook already we don't need to do anything else aside from that but when we go and actually build our kiwi app this is going to be really really important because we're going to need somewhere to reference to our custom layer but i'm going to show you how to handle that later perfectly fine so the second keyword argument is custom underscore objects equals and then we're passing through this dictionary here so it's squiggly brackets and then we're specifying l1 disk which is our l1 distance layer colon l1 disk which is effectively our l1 distance layer which is what we created from again it was inside of step four and we've done a title of stuff here which is this over here so this is a custom siamese distance layer all we're doing is we're passing that through as a custom object down here and we are also passing through binary cross entropy which is going to be this key over here is a binary cross entropy then colon tf.losses over there and i believe this is because we probably created a separate key for it now if i remove this this is actually going to cause a whole bunch of errors so what you're going to see if you don't pass through the custom object is you're going to get this error here so it's going to say value error unknown layer l1 disk and this is because that is a custom layer so if we go went and added that back in let's just go and add our one distance layer for now okay so it looks like we didn't need the binary cross entropy bit but in this particular case looks like it's worked so again by passing through that custom object we're able to effectively load up our custom objects now i'm going to leave in that second line because i don't know what that's going to do if i'd remove it but in this particular case that's worked successfully it looks like you might be able to drop this off as well but who knows i need to do some more digging into it if anyone in the comments knows let me know all right cool that is our model now reloaded so if we wanted to go and use this model now we could type in model dot predict and then pass through our what was it test input test val you can see we are now getting predictions so that is effectively showing how we can go and evaluate our model make predictions save it down and then reload it so we can now get to actually deploying and building this up into an app let me just double check that we didn't have anything else there nope and again we can go and take a look at our model summary so again we've got the exact same architecture that we had for our model up there so let me just add some comments so make predictions with reloaded model and then view our model summary cool so that is about it for this tutorial so we have gone through a ton of stuff so we've gone through insider step six we imported our metrics we made predictions remember we can make predictions by grabbing our batch i'm going to show you in the next couple of episodes how you can make predictions with a non-tensorflow dataset so you can see how we'll actually productionize this as well we made a bunch of predictions calculated some metrics have visualized our results we then went and saved down our model so we can begin using it elsewhere but on that note that about wraps it up much for tuning in guys hopefully you enjoyed this video if you did be sure to give it a big thumbs up hit subscribe and tick that bell and let me know how you went with this and if there's any changes or any help that you need thanks again for tuning in peace you

Channel: Nicholas Renotte

Views: 2,172

Keywords: python, deep learning tutorial

Id: W1pw_ojO7XE

Length: 30min 41sec (1841 seconds)

Published: Wed Oct 13 2021