Build a Deep Facial Recognition App // Part 3 - Preparing Data for Deep Learning // TF Dataloader

what's happening guys welcome to part three in the facial verification or facial recognition series where we go from a complete research paper all the way through to building a front end implementation so we're gonna be building up the model and then implementing it inside of an application alrighty let's take a look as to what we'll be going through in this video so let's make sure we go to present mode so first things first what we're going to be doing is we are going to be pre-processing our images so we've got a bunch of images that we collected in the previous tutorial what we're going to be doing is starting to pre-process those then we are going to create our positive and our negative sample so remember we took a look at the different architectures so we're going to be passing through our anchor image plus a positive image that's going to go through the pipeline we'll then pass through a anchor image and a negative image that's going to go through the pipeline when we train as well we're going to be able to create those samples and actually load them into step three which is our tensorflow data loader so we're gonna be able to do all of this and by the end of this tutorial we should effectively have our data ready modeling so we'll actually be able to get to the deep learning part of this tutorial right after this okay ready to do it let's get to it okay so let's get out of slides that's plenty so what we're going to do is we are effectively up to step three in this process so we're going to get our image or get our image directories we're going to pre-process them so scale and resize and then we're going to create a label data set which is step 3.3 let me have some tea guys and then what we're going to do is we are going to build our train and test partition let me make this a little bit bigger okay so first things first we need to go ahead and get our image directory so remember we went and created our image directories right up here so we created a folder called data and then inside of that there was a positive image and then we created another one called negative and another one called anchor so if we actually go and take a look at our data so if we go into youtube and then face id we've got this folder called data if i zoom in on that so you can see it we've got a folder called anchor which you can see a whole bunch of mug shots of me there and then if we go into our negative folder got a bunch of images from the labeled faces in the wild directory and then if we go into positive we've got a bunch of additional mug shots in myself that's priceless oh god so there's some some great photos of me in here but anyway we've got our data right now what we need to do is we're going to use the tensorflow data loader so if i bring this tensor flow data loader so this tf data pipeline allows you to build these data pipelines and i've found that this makes building deep learning models a whole heap easier so what we're actually going to do is we're actually going to first up load all the folders from their respective directories we're going to create a pre-processing pipeline which is going to be around about here so we're going to create that pre-processing pipeline and then we're going to associate our positive and our negative classes so we'll actually build up all of this okay so first up things first let's go on ahead and get our image directories okay so those are our image directories now obtained or specifically the files within our image directory is now obtained so what i've actually read in there is three different lines of code and again i'm going to explain this in great detail so we've gone and created three new variables so anchor positive and negative and then what we've actually done is we've used the tf.data.dataset.listsfiles method to actually go and grab all of the different images within a specific directory so what this is actually doing is creating a data gen or creating a generator or using a generator to be able to loop through and grab all of the files within that specific dictionary or specific directory not dictionary so if we went and grabbed anchor now and i'll explain that in a sec in a little bit more detail so if i said anchor dot as numpy iterator so i'm just going to show you something briefly so dear test and if i type in dear test dot next so what you can actually see there is we've actually got the full path to a specific image within our directory so this tf.data.dataset.com files actually creates a set or a pipeline for all of the different files within a specific directory okay so what we're effectively doing is we're going two-way sp or passing through a specific path so anchor path and then we're doing a wild card search so we're adding backward slash and then start.jpg so this is effectively going right grab everything inside of our anchor path which has a jpg extension so the full line is anchor equals tf dot data dot data set and data set is in caps dot list underscore files and then we're passing through the anchor path and then plus plus backwards star dot jpg so if i grab this little bit here you can see that it's effectively doing a wild card search inside of that directory so it's grabbing data backwards backwards slash and car backwards backwards star dot jpg and if you take a look inside of our data folder so youtube face id data and then income they should all be dot jpg files so you can see that so say for example you were working in png files you could change the dot jpg or dot png for example and then what we're doing is we're specifying that we only want to take 300 images because remember when we collected our data in the previous tutorial we went and grabbed i think it was roughly 300 anchor images roughly 300 positive images and then negative we've got plenty from the labeled faces in the wild data set but we ideally want to make sure that we've got a matching number of samples for each of these different classes so anchor positive and negative so we're going to take 300. so in order to do that we're just appending dot take and then passing through the number 300. if you wanted to take more images you definitely could if you wanted to take less images you could do that as well but try it out so see what your performance looks like in the end which we'll obviously get to but in this particular case we're going to work with tray 100 images okay so that's our anchor line done so what we've gone and done is we've gone and done a similar thing for our positive and our negative line so positive equals tf dot data dot data set dot list underscore files and then we're passing through the positive path plus and then backwards dot jpg so exactly the same as what we had in our anchor line but this time we're using a pos or passing through our positive path which was defined right up here so remember positive negative anchor path these are our different paths and we've gone and done the exact same there so we've also gone an appended dot take 300 to grab 300 images we'll grab 300 image paths right because remember this has only got the paths it hasn't actually loaded our images into a data pipeline yet perfectly fine we're going to handle that in a second and then we've gone and written negative equals tf.data.dataset.listfiles and then this time rather than passing through the anchor or the positive path we're passing through our negative path and again we're taking 300 images and what else so down here i was basically showing you how to actually access this iterator so you can actually type in the name of the data set or the data pipeline and type in dot as numpy iterator and then by using the dot next method you can continuously grab the next component inside of that pipeline so if i run dot next again you can see like watch here so you can see that number is changing so right now it's 8294 because that's the image id changed grab the next one grab the next one grab the next one grab the next one grab the next one and the nice thing about this is that when we actually go and batch our data together the way that our model is going to train is we're going to train batch at a time so we will grab one batch out of this data set which we'll eventually create we'll then go through train it so we'll do a forward pass we will calculate the loss we'll then go and what do we typically do then uh we then go and calculate the gradients or we actually go and yeah go and calculate the updated gradients and go and apply them to all of the different weights in our neural network so basically this iterator this data pipeline allows you to do that with a little bit more flexibility okay that is step 3.1 now done so we've gone and grabbed our anchor our positive and our negative data sets or specifically our image files and you can see that's an example of what we're actually getting out of here so this particular case we've got we can print it out to make it look a little bit nicer so we've got didn't actually do anything so uh so in this particular case i've got data backward slash backward slash anchor backward slash backward slash and then the image within that directory cool alrighty so the next thing that we need to do is go on to step 3.2 and start doing our pre-processing so what we're actually going to do is we're going to write a pre-processing function that loads our image uh from our directory we're actually going to resize it and we're also going to uh perform some scaling so we're going to convert all of that image values from 0 to 255 to 0 to 1. so this helps our neural network optimize a lot easier so rather than having a huge range of numbers having it from zero to one effectively makes the uh gradient descent a little bit easier and helps us achieve a better performing model a whole heap faster so what we're going to do is we are going to write that function now so let's do it okay so the function that we are going to create is going to be called pre-process i know super unique and creative and then to that what we're going to do is we're going to be passing through this specific file path so we'll actually be passing through one of these file paths to this function so we'll be able to use the map method and again don't freak out if i haven't explained this yet and we're going to go into it in more detail but what we'll effectively do is once we've got our data set so let me explain it in a little bit more detail so we'll have a object which is a tenseflow.dataset iterator or generator and we'll be able to run dot map and to that we can use the pre-process function and this is effectively going to allow us to apply the pre-process method which we're defining over here over every single example that we've got within our anchor positive and negative classes but again we're going to go into that in more detail i just wanted to explain why the hell we're writing this in the first place okay so what are we doing so we're going to finish this off so let's do it okay that is our preprocessing function now done so we've gone and written an additional one two three four five different lines of code and these effectively load up our image from a file path they also decode the jpeg they then it then it resizes the image it divides it by 255 so it performs its scaling and it returns the image i'm going to explain this in more detail so uh let's actually see how it works so if i go and run preprocess and if i take this file path here you can see what it's doing is it's returning a numpy equivalent of our image so if we wanted to we could um so let's call it image if i ran plot that i am show so you can see it has effectively loaded up our image from this file path it's gone and done a bunch of other stuff so it's actually gone and resized it and it's gone and divided it by 255 so it's effectively scaled that image to be between one and zero so if we run um what is this so dot numpy is that gonna work yep so dot min so you can see our minimum value is not less well it shouldn't be less than zero anyway either way but it is the 0.1 or 0.1 and then if we take a look at our mac so this is the most important thing you can see the maximum value that we've got is one so it's not going past 255 so we're good there okay let's take a look at each of the lines of code that we just wrote so under our pre-process function so in order to define our preprocess function we've written d f d break pre-process and then we've passed through our file path and then colon then we've gone and written one two three four five different lines of code so first up what we are doing is we're reading in our image so read in image from file path and in order to do that written byte underscore image so we've created a new variable and we've set it equal to tf dot io dot read underscore file and then we're actually passing through the file path that we originally passed through to our function so think of this like just passing through this path that we've got over here to our function pretty straightforward so that is then going to read our file as a bytes-like object and then what we're doing is we're using the tensorflow decode jpeg image to actually load in the image to load in the image and so the line there is image equals tf dot io dot decode underscore jpeg and then to that we're going to be passing through this byte image that we've got here so you can see we're passing that there so the full line is image equals tf dot io dot decode underscore jpeg and then we're passing through byte image then the next line is actually doing a little bit of pre-processing so think of this as our pre-processing steps and the first one is we're actually resizing the image to be 100 pixels by 100 pixels by three channels so in order to do that with an image equals tf.image.resize and then we're passing through our image which we've originally loaded from up here and then we're specifying the dimensions so we want it to be 100 pixels by 100 pixels the reason that we're doing this is if you actually take a look at the original paper i wish i saw means uh why didn't i have it up let me open it up now so i can show you it's in here all right so if we take a look at the neural network network um so the input image that we actually have in the paper is one image by 105 by 105. we're going to do it 100 by 100 just to keep things a little bit tweaked but if we wanted to we could actually make this 105 pixels by 105 pixels wouldn't cause an issue perfectly fine we're going to make it 100 by 100 um this isn't going to implicitly impact our neural network in any negative way at least from what i've seen but if you wanted to make it 105 by 105 perfectly okay all righty then what we're doing is we are scaling our image so scale image to be between zero and uh so i've read an image equals image divided by 255.0 so this effectively takes every single value or pixel value which is traditionally from 0 to 255 and divides it from or divides it to be between 0 and 1 now so this effectively scales our image and then last but not least we're returning our image so that we're able to do something like this so this is an example of actually going on ahead and running it and you can see it's actually loaded up our image so we can begin to work with it if we need to all right so the next thing that we need to do well we've actually already gone and shown it so what we can then do is actually begin creating our label data set so what we actually need to do here is we are going to create a positive and our negative example so remember when we actually go and validate we are going to pass through an anchor image and a positive image and this ideally should output once ideally verified and we can also pass through an anchor image and a negative image and that should ideally output zero so what we can actually do is we can actually use a function called tf dot um what is it one's like yep so we can actually output tf.one's like and say for example i pass through one this is effectively going to output an array which represents the number or has the value one but if i change it to be one to a whole bunch of numbers oh god that's not a great example what is that so what we're actually going to get as the output of that is an array which has the same dimensions as whatever i've passed through the tf.ones like function and x rather than having a whole bunch of different values we're actually going to have a number a set of ones so what we're going to do here is we're actually going to pass through our anchor images our positive images and we are going to specify so that's going to be our think of this as our input and the output that we're going to have as a result of this is going to be a array which is made up of ones so think of this as representing our positive class so we're going to create an example like this which effectively represents our positive examples and we're also going to create a similar version which is going to be anchor and negative images except here rather than having ones we're going to pass through zeros because these are unverified right so this is effectively creating a labeled data set so let me just comment that because we don't actually need it so let's go on ahead and do this so we're actually going to create our label data set now okay it looks like we've got a bit of an error there uh what have i done okay this should be ones hold on zeros and we've also forgotten dot zip yeah okay cool all righty so that is our directories now concatenated together so what we've actually got is we've actually got uh one big data set which it has the shape of uh so it'll be our anchor images and then either a positive or negative file paths and then our either it's going to be ones or zeros so let me explain what i've gone and written there because it is quite a fair bit so i've written positives equals tf.data.dataset.zip so this zips stuff together and allows us to iterate through all three at the same time and then i've got inside of here i've got a set of brackets which effectively creates a tuple and then we've passed through our anchor and our positive so this is effectively going to be our different directories from what we had up here or specifically our file paths and then i've passed through tf.data.dataset.from tensorslices and then to that we've gone and passed through tf.ones for our positive so this is effectively doing this so if i show you that so tf pf dot ones and then len anchor so uh should be one s so that is creating a big set of array which represents ones so this is effectively creating our labeled positives right because we're effectively saying our anchors plus our positives or if we pass through our anchor and our positive at the same time it should effectively represent a one which is a positive verification or a positive recognition so say for example you also wanted to do this on pictures of other people you would also pass through them as anchors and positive examples of them as positives and you'd be passing through the ones as well we're obviously doing it on one person we could definitely extend this out okay so that is our positive data set or beginnings of our data set now sort of staged so you can also see the shape here right so you can see that it's tf.stringtf.string and then tf.float32 so the string is going to be the file path to our specific image the second string is going to be the file path to either a positive or a negative image and the last value is going to represent uh whether or not it's a positive or a negative verification or recognition which in this particular case our positives are going to have ones and our negatives are going to have zeros so the second line that we wrote is negatives equals tf.data.dataset.zip so again it's zipping all of these together so we can iterate at the same time and then we've gone and passed through our anchor and our negative right and so when we have an anchor plus a negative we want to be or we want our model to be outputting zero because it's a negative recognition or verification and so what we've gone and done is we've gone and created a separate data set for our zeros so tf.data.dataset.from underscore tensor underscore slices and then to that we've passed through tf.zeros which have the shape of our anchor so if i go and type in tf.zeros and then lan anchor that is effectively representing or creating an array of zeros which match the exact shape of our negative classes or our negative samples so you can see that it has its shape 300 and we're effectively wrapping this inside of this uh tensorflow.dataset class so if i go and do that so again this is just creating a or putting that data inside of a data loader so tf.data.dataset.from underscore tensor underscore slices so we can begin to treat it exactly as what we did for add directories up here so let's create an example so this is let's say this is a class reps or class labels so then what we can do is convert it into an iterator labels so this is purely for example you don't need to write this um and then what we can do is go iterator labels.next and this is going to get the next value right so we're effectively looping through each one of the labels in our class so this converts it into the same format that our anchor and our negatives are in effectively atf.data.dataclass okay we can delete that so we don't need that and then the last line that we wrote is data equals positives dot concatenate negative so this is just joining our positive and then negative samples together so we've effectively got it in one big data set which you can see there so we could do uh convert this to a numpy iterator let's call it samples and then samples dot next so you can see that this is one representation of our data or one sample right so in this particular case we have a so remember that label is going to represent whether or not it's a positive or a negative class so in this particular case we've got a one which means it is a positive example so this is a the file path to our anchor this is a file path to our positive image and then we've got a one which represents the label so again we could keep iterating through this and right now it's not shuffled so we're gonna have to go through all of the positives before we get to the negatives but once we shuffle it up or once we do a couple of next pre-processing steps you'll actually be able to see them in a better state okay so that is that now done now what we can effectively do is go on to our last step which is pre-processing these images so right now we haven't actually oh what did i write that so build train and test partition so what we're actually going to do is we're now going to use our pre-process function to pre-process these directories so that rather than just having the directory we've actually got the anchor image we've actually got the positive image and then we've got our label so either one or zero so what we need to do is we need to write a second function which actually processes a twin so right now if you think of our negative or anchor and a positive as a twin we need to pre-process these so load in both of the images and then concatenate it back into or return it back to this data set so let's go ahead and do this okay that is our twin pre-processing function now done so again this one's a little bit more straightforward so what we're effectively doing here is we've written d f pre-process underscore twin we've then passed through our input image or we're going to pass through our input image our validation image and our label which is effectively what we've got over here and out of that what we're going to do is we're going to return the loaded and pre-processed image so remember we're going to use this preprocess function which we defined over here so we're going to run preprocess on the input image preprocess on the validation image so this is effectively our anchor and this is could be either a positive or a negative we're also going to return our labels so let's actually test that out so if i run pre-process non-caps pre-process t break all right so pre-process twin and then if we uh this it could be an example right so let's take a look at example so what we could do is we could pass this to a prepress test twin function and that effectively is what we're getting back so what i've just gone and done there is i went and spelt example wrong but we did more than that so we're grabbing the one example out of our data iterator which we had from up here and we're passing this through to our pre-process twin function down here and this star is effectively unpacking the values that we've got inside of this tuple so rather than writing out or extracting each one of these or unpacking each one of these it effectively unpacks them and passes it through to our function so could we do it like this no it's going to throw errors we pass through a star that effectively unpacks it now you can see that the result added here so if i type in res equals what i'm actually getting is three things so uh if i grab the first thing so this is ow well let's actually take a look at the length so you can see it right so we've got three different values or objects inside of this result the first thing is going to be our pre-processed image so you can see that this is actually running our preprocessing function and loading our image as 100 pixels by 100 pixels by three channels so if i went and plotted that plot.i am show you can see that's an image of yours truly and if we take a look at our second value inside of that results variable you can see that that is a positive sample so this should effectively have a label of one so if i go and type uh what is it res two you can see we've got a label of one there so all we've done up until here is we've effectively this is the state that we want to get to but now we need to do it on every single one of our examples inside of this data variable here so let's quickly do a recap because we've done quite a fair bit so first up what we did is we used the tf.data.dataset.list files method to load up every single file inside of our anchor a positive and our negative directory then we went got voice just broke there then we went and wrote this pre-process function to be able to load our image from that directory resize it to be 100 pixels by 100 pixels and we also scaled it to be a value between 0 and 1 because that is going to help our gradient descent and effectively help us to optimize our model so we haven't used that yet effectively i've demonstrated how to use it then what we went and did is we went and created our labeled data set so we're effectively creating sets of twins not triplets so where you might have seen this done in siamese neural networks before you typically create a triplet we're creating a twin or a tuple rather than a triplet because we're going to be passing through two values at the same time and so we've created our positive samples and our negative samples so really our positives are just going to be our anchor image plus our positive image and the label that we're going to get out of this is a one we then created our negative samples and our negative sample is going to be our anchor image plus our negative image with a label of zero and we went and concatenated those together so we can begin to work with them then we went and wrote our pre-process twin function and the last thing that we actually need to do is actually go and apply this to this data object over here so we're actually going to use the data.map method to be able to go and apply our preprocess twin function so we've gone and done a lot of hard work already all we now need to do is go and build up our data pipeline so i'm gonna do this and then i'm gonna show you the results okay that is the beginnings of our data loader pipeline and now done so what i've gone and written is data equals data dot map and this is effectively doing app or running a pre-processed twin function from over here so it's going to take our input image path our validation image path and our label and it's going to return the actual image loaded and pre-processed for our anchor the image loaded and pre-processed for either a positive or a negative image and it's going to return our label so now when we go and take a look at this data set or this data object we should effectively have our images i'll show you that in a second the next two steps that we applied to our data loader pipeline is we wrote data equals data dot cash so we're cashing our images and then probably one of the most important pieces is we've gone and shuffled it up so this effectively uh muddles up all of our images so that we're not just getting positive images and then negative images they're all mixed up so when we go and split our data into our training and testing partition we're going to have ideally we should have a mixed set of samples so in order to do that i've written data equals data dot shuffle and then we've specified a buffer and then we've specified a buffer size of 1024 so now if we go and take a look at our data object you can see that we've got a shuffle data set which has the shape of 100 pixels by 100 pixels by 900 pixels 100 pixels by none so this is our anchor image this is going to be either our negative or a positive image and then this will be a label so if we go and create a numpy iterator again samples and then if i grab samples dot next you can see that we've got our data all ready for us and ready for training so we've got let's take a look at our length again we've got three values and remember these are going to be either our anchor image or it will be an anchor image as our first value it'll be either our positive right negative image for our second value and it's going to be our label so if i run dot next and then grab the first value that is going to be our anchor image so we can go and plot that out using plot.i am show cool and then if we go and grab our we should actually save this in a variable otherwise it's going to keep changing um so let's go samp samples.next and if i pass through samp zero that's going to be our anchor image pass through value one or index one that's going to be so this really this is really really good so this should ideally mean that this is a negative class so what does that what label should we be expecting here i'll give you some thinking music all right that's enough thank you music that was terrible okay so ideally our label should be label zero if it's not label zero it's gonna be okay cool all right so we've got our we had our what was it our anchor image we've got a negative image and we've got our label which is 0.0 so that effectively means that we have the correct label in this particular case okay we could do this again so if we go and run our samples.next and if i type in plot.i am show samp zero so that is an accurate image of me we take a look at our sample again so that looks like it's a negative sample and again we've got class zero so if we go do it again let's see if we can get a positive one so myself i would need to run this again another image of me that looks like it's another negative sample let's wait for a positive one nope come on okay this should be a positive sample so you can see we've got an image of me we've got another image of me as our positive and if we go and take a look at our label you can see it's one so this is effectively how a neural network is going to work we're going to pass through and this is effectively one shot classification so we can pass through a anchor image or an input image and then you can pass through a separate image which you want to validate against and it's going to tell you whether or not that's the same person so say for example you wanted to do verification on multiple images or recognition or multiple people you would effectively just include more positive examples right so you'd include say for example uh you wanted to do it on your best friend you'd include images of yourself as a positive example so you'd have images of yourself self as an anchor yourself as a positive and the label for that would be one you then include also images of your best friend as an anchor and your best friend as a positive and that label would be one when it comes to actually verifying what you would be doing is you'd be passing through your anchor and if you want to verify against yourself you'd pass through an images yourself as the second input to your siamese neural network if you wanted to do it on your best friend it would be an anchor image and then a verification image of your friend to determine whether or not that's the same person okay but enough on that we've actually gone and created our data set now so the last thing that i want to do is just create a training and testing partition so i'm just going to delete all of this so let's create our training partition first okay that is our training partition now done so i went and wrote three additional lines of code there so train underscore data equals data dot take and this effectively is grabbing a certain percentage of our data sets i've written data.take and then i've passed through round and then specifically so this is effectively just coming up with how many images right so i want to effectively take 70 percent as my training partition so this does that calc so we're grabbing the length of all of our data multiplying that by 0.7 and then we're rounding the image which just so happens to be 420 and then what we're doing is we're using the data.take method with that value to take the first 420 images so i could just as easily replace this with 420 yep and then we've gone and batched our data so we're now going to be passing through our data as batches of 16 images so i've written train underscore data equals train underscore data dot batch and then to that will pass through the value 16 and then we're using the prefetch method and this effectively um starts pre-processing the next set of images so that we don't bottleneck our neural network when we go and train i've written train underscore data equals train underscore data.prefetch and then eight so that is our training partition now done so we went and wrote went and wrote three additional lines of code there's data.take data or traindata.batch traindata.prefetch cool so that is now already uh and again if we go and take a look at our data set we should if i type in train data we will now so the shape is slightly different now so we've now got a so if we took a look at our data values of data that was what our data value looked like our data objects we had a hundred so you've got this is the shape here this is the important bit you would have had a hundred by a hundred by none hundred by a hundred by none and this is 100 pixels by 100 pixels by the number of channels in our image or our anchor same thing for our positive and then we had this value here which represents a single unique value in our train data we've got a slightly different shape and this is because we're now batching it so you can see that this none value is actually representing the number of batch or number of images that we've got in our batch so it should effectively be 16 right because we have gone and created batches of 16. so if i grow and grab let's do the iterator thing again so train data dot as numpy iterator i'm going to call this train samples and then i can type in train dot sample or sample equals train samples dot next if we take a look at one sample and take a look at the length it's three uh and if we go and grab the first value this should effectively be 16. cool so we've now actually got 16 images inside of each sample that we actually have so rather than having just one image per value that we get back from our generator we've actually got 16 images now cool so that is our training partition now done let's just create a validation partition i don't know if we're actually going to use this but we'll do it anyway so uh testing partition sorry okay so let's go ahead and create our testing data okay that is our testing partition now done so what we went and did there is we skipped the first 420 observations so that we don't pick up our train sample and then we went and took the last thirty percent so again i just wrote two lines of code there so test underscore data and i haven't batched this we might batch it later if we need to perfectly fine for now to test underscore data equals data dot skip and then we've passed through again so exact same value that we had up here so this should effectively be 420 420 and then we've gone and written test data equals testdata.take and then we're passing through the last 30 so if i grab that that is gonna grab the last 180 images cool all right so that is pretty much our data now ready so we've gone and created our training partition and our testing partition we should probably batch this as well let's do it while we're here test data dot batch and i don't know we'll do batches of 16. and prefetch eight cool all right that is our that should be everything now done okay so we went and did a ton of stuff there so we went and grabbed all of our image directory so in this video we went and did effectively four different parts so we went and grabbed all of our image directories using tf.data.dataset.list files so remember we went and grabbed all of the specific strings for the images we then went and pre-processed them so we went and loaded them up we then went and resized them to be 100 pixels by 100 pixels and we scaled them as well we then went and created our label data sets we created our positive samples and then negative samples and i sort of explained how you might go about doing this if you wanted to verify multiple people and we concatenated those together we then went and pre-processed our twins so rather than just working with the strings we actually loaded up our images into our data set or set up our data pipeline to do that and then we went and created our data pipeline so we went and fleshed those out so let me just clean this up so you can see it a little bit better so we built up our data pipelines we mapped using the pre-process twin function which effectively leverages our pre-process method from up here we then when encased our images shuffled them up we then created a training partition and a testing partition so this puts us in good stead to actually get to some deep learning in the next part of this tutorial but on that note that about does wrap this up so thanks again for tuning in guys hopefully you enjoyed this tutorial if you did be sure to give it a big thumbs up hit subscribe and tick that bell and i will see you in the next one

Channel: Nicholas Renotte

Views: 2,860

Keywords: face recognition python, face recognition, face recognition app, face recognition project, face recognition using python, tensorflow data pipeline, tensorflow dataset, tensorflow data augmentation, tensorflow data validation, tensorflow data input pipeline

Id: QBxQxYvdNek

Length: 43min 20sec (2600 seconds)

Published: Sat Sep 18 2021