Creating a Deep Neural Network in ML.NET - Microsoft.ML.Vision Update

hey everyone so i've gotten quite a few comments on this neural network model on mmo.net video which was actually done just a bit over a year ago in last september and the comments mentioned that some things in this video no longer apply and so in this video i'm going to do an update project for this and instead of the sushi and sandwich data that i used in this video i'm going to use this flowers recognition so if you look at the data it's got a set of five flowers daisy dandelion rose sunflower and tulip and we're going to build a deep neural network model in ml.net to classify a flower as one of these types and so let's get started here here in a console application here i already have my data inside my solution so under an images tab i have each of the flower labels as a directory and in each of these directories is a set of 20 images for each of the labels and then i have some test images here one for each label that we'll be using to test our model right so let's look at some nuget packages here so first off we need microsoft dot ml for ml.net version 1.5.2 here and since we're doing with image data we need the image analytics nuget package as well and because we're creating a deep neural network for image classification we need the microsoft email.vision new gear package all right so with that updated let's start on our console project and the first thing is and by the way most of the stuff we're going to do here is kind of similar to this video here uh just with some updates from the packages so how we get our images and all that are going to be similar to the previous video and so first thing we do is get our images folder that we use path that combine and i'll use environment dot current directory and i'll go back one two three item three folders back and i'll get the images folder and i get all the files in that folder using directory dot get files now start with that images folder i'll get everything using the wild card and i use a search option all directories now i'm going to get my images using files that i already listed files so using that that's select and for each file i'm going to create a new image data class and let's go ahead and create that before we continue so this is going to have two properties first is a string image path then a string label that it belongs to let's go back here image path is going to be the file and our label is going to be directory that get parent so get the parent of the file and then the name of that parent so each image data is going to have the image path of each of these and then the label is going to be the directory name next let's create our context new ml context and then we get our image data by using context data load from innumerable pass in our images and then we shuffle that data because right now all is coming in is in this order here so we can shuffle it so it can be a bit more random so the context data shuffle rows and passing that image data and now let's split our data using context data train test split use our image shuffled data set and i like to use a 20 test function so we have our split now let's create some validation data and for this this kind of a pipeline that we're going to do to our validation data so it's in the correct format when we pass it in so context that transforms conversion map value to key so we're going to map the labels to a label key and pass in the label column name there and we're going to give it the key ordinality of by value so it's going to transform these names into keys to numerical keys and then it'll append contexts transforms i'm going to load raw image bytes it's going to load in the raw images from the path and it's going to create a new column called image given the images folder and then the input name is going to be image path that's from up here in our image data class and then we'll go ahead and fit on our test train data test set and then we'll transform on our test set as well so that prepares our validation data next let's build a some separate pipelines here so for images we'll do something similar to what we did for our validation data sets uh we will run the conversion map value to key transform so it's going to be label key and pass in the label and then that by value look then i'll append the context transforms load raw image bytes gonna be image images folder and then image path as input let's set specify what parameter we're filling in here so here i have that pipeline now we get the images data model but calling the images pipeline fit on our training data set and then we get our image data view by calling image data model at transform also on our training data set this prepares our test set to be in the correct format that we need next we're going to build some options and these options are required for our image classification trainer so the new image classification trainer dot options we have several options we can do first we'll tell what architecture we want so architecture that i'll do resnet 250 now these options is one of the different changes from the previous video instead of a options class we specified them directly as parameters we set the epoch to 100 batch size to 20 learning rates 0.01 f the label column name is going to be label key then the feature column name is going to be image next we have the option of sending in a validation set which is why we did our validation data up here so we can set our validation data and there are plenty of other things early stopping criteria final model prefix if you want to change the default checkpoint file and folder prefix metrics callback if you wanted to kind of print to the console or something score call name if you want to change that name and a few other things but we'll leave it like this for now and now for our main pipeline context that multi-class classification and we do trainers that image classification which is what we get from that image analytics video package we pass in our options and then we append on that context transforms conversion map key to value so up here we map value to key to get our label key and down here we can map that back from the predicted label back to a label that was one of these it does conversion for us so we don't have to do any other calculations to get what label that we are getting and so for our model we just do pipeline fit on our image data view and now we can create a prediction engine using context a model create prediction engine i'm going to do an image model input instead of our image data up here and i'll do image prediction as our output and pass in the model now let's create these real quick for our image model input now this is going to be different because we have in here because we did load raw image bytes so we have a different input than what we have here so what we have is a byte array and that's the image data then we have label as key then we have the regular image path and then the string label so these two items are new and that's going to be what's needed for our prediction speaking of our prediction let's do that i'm going to also add the image path and then the label and here as well but i'm also going to add the predicted label and that is what we get back from here in this conversion all right so now to do some testing on our test set there now similar to above we're going to create a test images folder using path that combine so environments current directory and google back three and get our test folder and we're gonna get our test files using directory.get files using the test images folder just like before we get everything with the wildcard and search option is all directories then we create our test images using test files dot select and for each file we're going to create a new image model input instead of the image data and then here we give it the image path of the file and i'll do a console brightline as an environment that new line so it separates our test output from our predict our training output up here so test images data so we context our data load from enumerable on our test images then we do test image data view by using our images pipeline and fit on our test images data view and then we transform on that as well so we're using the same images pipeline on our test images data just like we did up here in our training set because it needs to be in that format before we can pass it into our our model and speaking of which we can create our predictions by using model that transform and then using that test image data view so our test image data view is going to be in the correct format with the load image load raw image bytes and all that now we can get our test predictions by using context data and create a numerable it's going to be image prediction we pass in our predictions and set reuse object to false now we have an innumerable of our predictions we can loop over those so for each prediction and test predictions in here we can just console. and so for our image in path.getfilename on our prediction.imagepath so i guess the file name on the image path which by the way file name contains what the correct label is and then our predicted label from prediction that predicted label so let's run this and see how it works all right so running this we get this error unable to load dll tensorflow or one of its dependencies so if you get that should be able to solve that by getting another nuget package from the psi sharp folks tensorflow and redistributable so it's version 2.3 but i know 1.5 works so i'm just going to do that instead and now we got that and now let's run this again all right so that ran and we have our predictions here so our daisy was correctly we did that as a deities daisy dandelion was a dandelion rose as a rose see sunflowers incorrectly predicted as a dandelion so we might need to add some more training data for sunflower and dandelion just to add more data to it and our tulip was successfully predicted as a tulip so hopefully that that helps you on the new microsoft ml dot vision package instead of the microsoft.ml.dnn package that we used before hope you enjoyed the video and i'll see you all next time thanks

Channel: Jon Wood

Views: 4,583

Keywords: ml.net, ml.net dnn, ml.net vision, ml.net deep neural network, deep learning, c# deep learning, .net deep learning, c# deep neural network, .net deep neural network, ml.net image classification, c# image classification, .net image classification

Id: ppRauvf6uCs

Length: 14min 10sec (850 seconds)

Published: Wed Oct 07 2020