How To Train an Object Detection Neural Network Using TensorFlow (GPU) on Windows 10

[Music] hello this video is a tutorial explaining how to use machine learning to train an object detection neural network through Google's tensor flow object detection API on Windows 10 by following the steps described in this tutorial you'll be able to create your own object detector that can locate and identify specific objects and images videos or live webcam feeds for the tutorial I'll explain how to train a pinochle deck playing card detector that can identify card rinks 9 through Ace even when there is a busy background or when the cards are overlapping you can use this tutorial to practice training your own pinochle deck detector or you can use your own images to train your own object detector all the images data and scripts I use to train the card detector are provided in the github page linked in the video description below the github page also has a written version of the tutorial which this video will fall out this tutorial is made for Windows 10 but it should work on Windows 7 or 8 it can also be used for Linux based operating systems but the file paths and terminal commands will have to be changed to match the Linux style this tutorial starts from scratch and explains how to set up the op tensorflow object detection API prepare your training images and data and train the object detector there are eight steps to this process 1 install tensorflow GPU to set up object detection directory structure in anaconda virtual environment 3 gather and label pictures 4 generate training data 5 create label map and configure training 6 train the object the texture 7 export the inference graph and 8 test and use your newly trained object detector alright let's get started the first step is to install tensorflow GPU on your computer I'm actually going to punt on this step because there's a number of good YouTube videos that describe how to do this I recommend watching mark JS video how to install tensorflow GPU version 1.4 on windows 10 which is linked in the description below this video is made for tensorflow GPU version 1.4 but the most recent version is 1.5 this tutorial will use version 1.5 because it supports newer tool kits and because it's generally a good idea to use the most up-to-date version of software when you issue the pip install command as instructed in marks video it will automatically install version 1.5 or whatever the most recent version is instead of installing CUDA version 8.0 and CDN version 6.0 has instructed the marks video install version cuda version 9.0 and c UDN version 7.0 for CUDA 9.0 because they are the these are the newer versions and they're supported by tensorflow GPU version 1.5 as future versions of tensorflow are released you will likely need to continue updating the cuda and CU DNN versions to the latest supported version be sure to install anaconda with python 3.6 as instructed in the video as the anaconda virtual environment will be used for the rest of this tutorial once you've got tensorflow GPU cuda CU DNN and anaconda installed the next step is to get the sensor flow object detection directory and anaconda virtual environment set up the tensor flow object detection API requires using the specific directory structure provided in its github repository it also requires several additional Python packages some additions to the path and Python path variables and some extra commands to get everything ready to run or train an object detection model setting everything up is kind of painful and meticulous but once everything ready is ready the rest of the tutorial is easy first create a folder directly in C and name it tensorflow one download the fold tensor flow models repository from the link given in the video description below once it's done downloading open the zip file and extract the models master folder directly into the tensorflow one folder [Applause] okay once it's done extracting go in and rename the models master folder to just models second go to the tensor flow model zoo page by using the link in the video description below tensorflow provides several different object detection models which are pre-trained classifiers that have a specific neural network architecture some models such as the SSD mobile net model have a high speed but lower accuracy while other models such as the faster our CNN inception model have a low speed but high accuracy I initially started with the SSD mobile net via one model but I didn't do a very good job identifying the cards in my images I retrained my detector on the faster our CNN inception v2 model and the detection weren't considerably better however it had a noticeably slower speed you can choose which model to train your object detection classifier on if you're planning on using an object detector on a device with low computational power such as a smartphone or a Raspberry Pi use the SSD mobile net model if you'll be running your detector on a decently powered desktop or laptop go ahead and use one of the faster our CNN models this tutorial will use the faster our CNN inception v2 model so click the link to download it go back to the tensorflow one folder and navigate to the models research object detection folder once the faster our CNN tar.gz file has finished downloading extract its contents to the object detection folder finally go to my github repository for this tutorial which is linked in the video description below the repository contains the images and annotation data needed to train the pinochle deck playing card detector it also contains Python scripts that will be used to generate the training data finally it has some scripts that we can use to test the detector on images video and webcam feeds download the repository once it's finished downloading go ahead and open it up and extract all the files directly into the object detection folder you can go ahead and replace the existing readme file with the new one in this video I'll start from scratch with just the card images and show how they're labeled how the TFR records are generated how the label map is created and how the training file is configured since we're starting from scratch go ahead and delete the following files from the repository you've just downloaded and we'll regenerate them in the next part of this video first go into the training folder and delete the label map and config files there then go to the inference graph folder and delete the placeholder file there finally go into the images folder and delete the test labels and train labels CSV files now if you want to practice training my pinochle deck detector leave all the pictures and xml files and the test and train folders intact if you want to create your own object detector using your own images we'll go ahead and delete all of the files in the test and train folders and we'll replace them with new ones since we're starting from scratch in this tutorial I'm going to go ahead and delete all of the files in the tape train folder and in the test folder at this point here's what your object detection folder should look like the following folders and files should have been added to the original object detection directory first the faster our CNN model file that you downloaded from the model Zoo oops then the dock images inference graph and training folders that you got from my github repository as well as the generate TF record object detection image video webcam and then XML to CSV Python scripts and the test and want files the training folder should be empty and so should the inference graph folder the images folder should have the images and XML files from my pinochle deck detector if you're training your own detector from scratch both of these folders should be empty now that the directories are all set up it's time to set up an anaconda virtual environment for tensorflow GPU this portion of the setup involves typing a lot of commands in the command prompt I have all the commands written out in my github root tutorial so you can either type the commands in yourself or just copy/paste them from my tutorial first find the anaconda command prompt in the Start menu run it as administrator create a new virtual environment called tensorflow one by entering Conda create - n tensorflow one tip it might take a while to set up the new environment once it's finished setting up activate the virtual environment by entering activate tensorflow one this activates the environment anytime the environment is active you will see the tensorflow one in parentheses before the command prompt install tensorflow GPU in this environment by entering pip install - - ignored install - - upgrade Pinsir blow GPU it'll take a little while for tensorflow GPU to install [Applause] after it is installed install the other necessary packages by interests issuing the following commands under install - si anaconda protobuf this will install our protobuf compiler and again it'll take a while next install pillow by typing pip install pillow easy enough then install a lemak LX ml by typing hit install Alex now and then in Jupiter hit install Jupiter Hawaii [Applause] okay and the last one to install is Matt plot line type pip install matt lock pipe okay there's two other packages pandas and python OpenCV which we aren't used by tensorflow but they will be used by the XML to CSV and generate TF record scripts from my repository so we need to install them too to install them enter pip install pandas and then type pip install OpenCV pip okay that's all the packages now that we've got all the necessary packages installed in this virtual environment we need to add some folders to the environment variables navigate to the models folder from the object detection folder and copy the path to the models folder then in the command prompt enter set python path equals and then you can just paste the path but the models and then semicolon models research semicolon one more time models research slim and enter that now we've given this python path variable direction to the models folder the modeled research folder in the models research slim folder then we'll add that python path variable to the path variable by typing said path equals parentheses I mean percentage marked path percentage mark semicolon Python bath there we go now you can see what this does to the environment variables by typing echo path this will give a big long list of directories that the path is pointing to and then you can also echo Python path which you can see is added at the very end of our path and that's got the three directories that we pointed it to now if you ever close out of the aniconic command prompt window these environment variables will need to be reset so you just need to redo this step if you ever wind up closing out of this window okay we're almost done setting everything up now we have to compile the protobuf files which are used by tensorflow to configure model and training parameter parameters mm-hmm so in the anaconda command prompt go ahead and change directories to the models research directory whoops not like that see 1001 and models research okay and then from there go to my github repository where the tutorial is and go to step to F and copy this big pro top command it's a long command so it's better to just copy and paste it rather than try and type it all out and then go ahead and just paste it right into the command prompt and press ENTER and you'll see that goes and creates an underscore pb2 file for every proto file that was in that produce folder okay finally from this directory from the research directory there's two more commands to run Python setup I build which takes a little while and then Python setup I install okay that's it the tensorflow object detection API is now all set up to use pre trained models for object detection or to train a new one you can test it out at this point by opening the object detection tutorial script and Jupiter notebook change to the object detection directory oops and then issue this command Jupiter notebook object underscore detection underscore tutorial type I notebook misspelled type I okay this opens the script in your default web browser and allows you to step through the code one section at a time you can step through each section by clicking the Run button at the top part of the script downloads the full SSD mobile net via one model so it takes a while to complete it's about 74 megabyte download [Applause] okay looks like it's downloaded okay so once you've stepped all the way through the code takes a little while to run through the slacks last section too but you should see two images at the bottom one with some dogs labeled and want them to some kites and people labeled it if you see this that means everything's working properly if not the notebook really poor any errors that were encountered you can look at the appendix of my github tutorial to see the common errors and their resolutions now that the tensorflow object detection api is all set up and ready to go we need to provide the images it will use to train a new detection classifier tensorflow needs hundreds of images of an object to train a good detection classifier to train a robust classifier the training images should have random objects in the image along with the desired objects and should have a variety of background and lighting conditions there should be on some images where the desired object is partially obscured overlapped with something else or only halfway in the picture for my pinochle deck detector I wanted to see how well an object kinetic detector could distinguish between different playing cards and if it could work when the cards were overlapping I took all of the nines tens Jack's queens kings and aces out of a deck of bicycle playing cards and then place them in various areas around my house I took pictures of all 24 cards in one area then moved them to a different area and repeated the process each time I repeated the process I put the cards in harder and harder visual environments with more random objects and busier backgrounds I tried to include card like objects in some pictures to help the detector avoid falsely identifying things that look like playing cards then I took around 100 pictures with multiple cards in the picture and with the cards overlapping overall I use 311 pictures to train my card detector alternatively you can get pictures of your objects through Google Image searches or other places online I recommend having at least 200 pictures overall make sure the images aren't too large they should be less than 200 kilobytes each and they shouldn't have a resolution higher than 720 by 1280 pixels the larger the images are if the longer it will take to train the classifier after you have all the pictures you need with 20% of them to the object detection test directory and then 80% of them to the images train directory make sure there are a variety of pictures in both the test and train directories now that we've gathered lots of pictures of our objects it's time for the fun part labeling the objects and the pictures for tensorflow to train on what the objects are you have to label each picture showing where the objects are located label image is a great open source image labeling tool for doing this you can download label image and view its uses instructions on github from the links in the video description below once you have it installed run it then open the train directory the first image in the directory will Peter clear it click the Create Rex box button and draw a rectangle around the object in the image type in a label for the object repeat this for the other objects in the image then click the Save button to save an XML file with a label data for the image click the next image button to move to the next picture in the directory and repeat the process do this for every image in the train directory and then any every image in the test directory this is a meticulous process that takes a long time but it's a necessary part of training an object detection classifier overall it took me about an hour to label all of my images once the image is labeled it's time to generate the TF records that serve as input data to the tensorflow training model your test and train folders should now be full of XML files these XML files to contain the coordinates of the bounding rectangles for each object in the image to generate TF records from these XML files we will use the XML to CSV and generate TF record Python scripts in the command prompt make sure you're in the object detection folder and then enter Python XML to CSV pi this creates a train labels and test labels csv file in the images directory these csv files just have the coordinates of the bounding rectangles for every object in every image next open the generate TF record file with the text editor replace the label map starting at line 31 with your own label map or each object is assigned an ID number then generate the TF records by entering the following command Python generate TF records by - - CSP invalid equals images slash train labels dot CSV - - image deer equals images train - - output path equals train record there we go then repeat the same command except to replace every instance of train with test okay this command creates the train and test record files that will be used to train the needle classifier there's just a couple more things to do before running training we need to create a label map for a classifier and then edit the training configuration file we'll start with the label map the label map tells the trainer what each object is by defining a mapping of class names to class ID numbers you can see an example label map in the data directory in the pet label minute file use a text editor to create a new file and save it as label map dot PB text make sure to save it as a dot PB text file not as just a text file save it in the training directory then type your label map in it is the same format as the pet label map file my label map has already typed up in my github repository so I'm just gonna copy it over from there save the file and close the text editor now we need to configure the object detection training pipeline it defines which models and what parameters will be used for training and points through the training images and data and the object detection folder navigate to the samples and configs folder and copy the faster are CNN in Section V - pets config file copy it into the training folder and then open it with a text editor there are several changes to make to the config file mainly changing the number of classes and examples and adding the file paths to the training data first change the num variable to the number of classes you want to detect I'm training my card detector to detect six different cards so I set num classes to six then change the fine-tune checkpoint variable to point at the faster our CNN inception model file make sure to use forward slashes in the file path or else it won't work I'm just going to copy the path from my github tutorial okay there we go similarly change the input path to point add the Train dot record file and the label map path to point at the label map file again I'll just copy these you might get hub tutorial okay we're all set there now they have the null examples variable this is how many test pictures you're using to train your object detector so to figure out how many test pictures are you're using in case you forgot you can just go to your test images directory select all of them and count the number of items here you can see I have 67 test images so I'll change the number num example is variable to 67 okay and then here in the eval input reader section we have a couple more input paths to configure this input path will point at the test record file and then this one again will point at the label map file so I'll just copy these from my github tutorial if you copy it from the label from the windows folder it copies it over with back slashes and we need to use forward slashes I'll show you what I mean see it uses back slashes if you use back slashes in your file path and it gives you an error for some reason so I put all of it in forward slashes mm-hmm okay and then that's all the changes we have to make so we'll go ahead and save the file and close out of it all right we're finally ready to start training the object detection classifier in the command prompt make sure you are in the object detection directory then type python train pi - - log - STD err - - train fear equal training - - pipeline con fig path equal training slash faster our CNN inception v2 pets.com Big O and press Enter if everything has been set up correctly tensorflow will initialize the training the initialization can take up to 30 seconds before the actual training begins [Applause] the training begins it will start stepping through training batches and reporting the loss at each step the law sloth starts off high and then gets lower and lower as the object detection classifier trains training the classifier uses 100% of your CPU and GPU while training is running I recommend not doing anything on your computer that requires lots of processing power should constant you should keep training the classifier until the loss is consistently below 0.05 or so or until the law starts to plateau out you can view the progress of the training job by using tensor board to do this open a new instance of anaconda prompt running it as administrator activate the tensor flow on environment by typing activate 10 1001 then change directories to the object detection directory and type tensor board - - log dear equals equals training no spaces this will create a webpage on your local machine at localhost six zero zero six which can be viewed through a web browser you can go ahead and copy the address paste it into your web browser and shows you tensor board the tensor board paid provides information and grid graphs that show how the training is processing one important graph is the loss graph which shows the overall loss of the classifier over time the loss graph will only start to populate after 10 surf laundry codes the first training summary which occurs three to five minutes after training starts so if you're not seeing anything in the loss graph just give it a couple minutes anyway will allow training to keep running until the loss drops below about 0.05 [Applause] okay so the model has been training for about three hours now and the loss is consistently below 0.05 the total loss graph and tense record shows that the losses plateaued out the models been training for long enough terminate training by pressing ctrl C while in the command prompt window tensorflow saves training checkpoints every five minutes or so and stores them in the training folder the checkpoint of the highest step count will be used to generate or classifier now that training is complete the last step is to generate the frozen inference graph make a note of what the step count of the highest model checkpoint is in the training folder in this case it's fifty three thousand six hundred and ninety eight go to the command prompt and make sure you are in the object detection directory paste the export inference graph command from step 7 of my github tutorial into the command prompt replace the X xxx with the actual model number of the highest checkpoint which in this case is fifty three thousand six hundred ninety eight then press Enter [Applause] okay this creates a frozen inference graph PP file in the inference graph folder in the object detection directory the dot PP file contains the object detection classifier now the object detector is all ready to go you can use the Python scripts provided with my repository to test it out on an image video or webcam feed to test your object detector out on a USB webcam feed type idle in the command prompt and press enter to open the Python shell from there open the object detection webcam by python script before running the script you need to modify the num class's variable to equal the number of objects you want to detect for my pinochle deck card detector there are six cards I want to detect so I set none class is equal to six now run the script if everything is working properly the object detector will initialize for about ten seconds and then display a window showing the webcam feed with the desired objects identified as you can see the object detector works decently well identifying the playing cards laid out on my desk it also works when the cards are overlapping which is good because that's what I was going for you can quit the webcam window by pressing Q you can also test images and videos in a similar fashion by using the object detection image and object detection video scripts from my repository just remember to change the number belen each script also make sure you're running the scripts from inside the tensorflow one environment rather than just from the desktop if you run it from the desktop they won't work if you run into any errors while setting everything up you can look at the appendix in my written github tutorial to see more errors and their solutions you can also try google searching the area as there are usually solutions on tensor flows github issue page or on Stack Exchange or you can comment on this video with what the error was and I can try and figure it out if you use this tutorial to train your own object detector I'd love to see what it is and how it works feel free to upload a youtube video showing off your detector and then post a comment on this video with a link to yours thank you for watching this tutorial and I hope it helps you with your machine learning endeavors have fun training [Music]

Channel: Edje Electronics

Views: 919,382

Keywords: tensorflow, machine learning, neural networks, object detection, classifier, image processing, convolutional neural network, GitHub, training, Windows 10, tensorflow-gpu, playing card detection, playing cards, pinochle, tensorflow model zoo, label map, labelimg, training pipeline configuration, inference graph, tensorboard

Id: Rgpfk6eYxJA

Length: 30min 7sec (1807 seconds)

Published: Mon Feb 19 2018

Reddit Comments

[u/alcalde]

Ok, recognizing the cards is nice... but how about teaching it to count them and help you win at blackjack? :-)

[u/Taxi-guy]

Here's a link to the written version of the same tutorial on GitHub!

[u/RetardedChimpanzee]

Looks like I have new plans for the weekend!

Edit: 3 hours later I have my own custom dataset training. One of the most strait forward tutorials I’ve seen for TF training. Warning, there seems to be some issues that need changed due to version issues.

[u/commanderCool09]

Dude this tutorial is amazing!!! Everything is laid out extremely well and the content is thorough. Thanks!!!

[u/MessiGOAT24]

I started learning Python (udemy course) yesterday. I only know the basics of HTML & CSS (just barely to put up a website of some kind..) but this thing is blowing my mind. In a good way. I had ZERO idea of what Python even does and what's possible with it, so these kind of videos really hypes me up. Obviously I can't understand anything related to it (YET) but I couldn't be more excited to learn this. I really hope I do.

[u/yoonch9009]

So wonderful!!!

[u/GotRedditFever]

Will have to check this out

[u/flat_rhino]

thanks so much!!! been meaning to attempt implementation in my research

[u/setyte]

This is interesting. Probably outside my skill set for now but on a few months I may revisit or. I want to make a better video doorbell or face tracking security camera.