How to Train a Custom Faster R-CNN Model with Facebook AI's Detectron2 | Use Your Own Dataset

hey there joseph from roboflow and i just finished training my very own faster rcnn model using detecttron 2. check it out i can see my tensorboard results for each performance by chess piece here i can also see that my model did a pretty good job of finding individual pieces in each of my chess boards you see here it found the white bishop it found white pond black king even in this board full of pieces it does a great job and in today's video i'm going to show you how you can train your very own detectron 2 faster rcnn model using completely free tools okay so for context today we're going to be using the chess pieces data set to train a model to identify individual chess pieces on a chess board but the steps that we walk through you can use for any custom data set okay so first things first what does the data look like that we're using today so i have the chess pieces data set here on my computer and i'm going to pull up one of these examples so it's just a chess board and the annotation is the individual piece the location of a bounding box of where on the chessboard a given piece is now if you want to start from using the same exact data set that i'm using you go to public.roboflow.com and grab this very same data set i'll put the link in the description so if i go to public.roboflow.com i have a list of a high number of public data sets that i could use to train a given model so here i see all kinds of different data sets that i might want to try as i said today we're going to be using the chess pieces data set okay so first things first you need this data set to follow along with today's tutorial now once you have this given data set you'll note that it comes already labeled and it has i mean we can go here and check out the dataset health check to see it has 289 images in 2870 annotations approximately 10 annotations per image meaning we get approximately 10 pieces per image when we capture this data set now when you're on the data set summary page i want you to go up here and click fork do you see this button here click fork if you click fork this data set will come to your own roboflow account it's a free account and you'll be prompted to create an account if you don't already have one you can log in with github or use whatever tool you want to use to create your given data set so this data set to follow exactly today's tutorial we could fork it as i mentioned and then it's just going to show up on our roboflow account the same way as any other data set would so if i go to my roboflow dashboard here i actually have a few data sets in my account already i already had chess pieces and then i just fork chess pieces again so i actually have two chess pieces data set i have the hard hat sample data set and i have thermal dogs and people now if you're starting from the very beginning on your own data set here's what you would do okay so first things first you need to have your images and your individual annotations i've got mine i have them downloaded here the next thing that you'll do is that you'll create a data set in roboflow so you'll create a free account by going to app.roboflow.com there you can sign in and create a data set so here i'll create a data set and again i'll be doing my chess pieces problems i'll call this chess pieces youtube for all of you and the annotation name so annotation group this is like i've labeled all of the blank in my data set in my case i've labeled all of the pieces in my data set so i'll go ahead and create that data set now i have the blank slate here for things that i might want to upload so in my images here in my data set here i have the chess pieces data set so all i have to do is drop my images and their annotations directly into roboflow so i actually have my folders just like this that are nice and neat it's okay if your folders aren't already split into train valid tests and you'll see why here in a second so if i drop those images in what roboflow does is it looks for each of my images and my annotations and it matches them up so i can easily visually have a look and see hmm yes it looks like things are being um added correctly and i can see the bounding boxes around each of my pieces uh now this is a public data set so you remember i had 289 images so i should see all of those 289 images processed and reflected here for me to continue in my upload process now this is actually not uploading the images yet this is just showing me the processed images and by that i mean it's showing me when my images match each of my annotations and so you'll see up here that i have annotated this counting up and not annotated is still at zero that's good that means all of my images are matching their annotations if you have a problem where your images are not matching your annotations you can either write us help rebelflow.ai or have double check make sure that we support your annotation format by going to roboflow.com formats okay so i can look at these individual images and i can see that you know all of these um they look pretty good it looks like everything was was processed correctly so go ahead and finish my upload and i can create a train valid test split so roboflow will split my images into training validation and test so that i'll have an image data set ready to go now i don't actually have to wait for this because i've already uploaded my images so i'm going to go back to one of my data sets that i've actually already uploaded okay so in this one that i've already uploaded we'll go to this one here that i forked and it was last updated three minutes ago after all of my images get added to roboflow i have this sort of screen and this is your data set command center so this is where you can unders explore things like how many images you have you go to the data set health check just as we did on the robofolk public page and gain an understanding of how many images we have our class balance so in this data set we see that we might not have enough of the queen pieces in particular or maybe even the bishop or king pieces our pawns are pretty well represented so we hope that our model would do a good job with these pawn pieces all of my images in this data set happen to be the same size but if my images were of different sizes i might want to resize with a different type of decision now wolverine says everything to a square in our example here and then i can also see my annotation heat map so this shows me where my bounding boxes are across my given data set in this example we'll see that my pieces are pretty well distributed across the board i mean it looks like the black queen piece could have been distributed a little bit better but you know a free data set is a free data set um okay so now with all my images here uh i'm gonna go ahead and delete my uh my duplicate data set uh this duplicate or the the data set here that's getting uploaded i uh this is the one that we just uploaded that we just started with uh and so this one will also get to 289 images when everything finishes uploading now because i have a bunch of duplicates i'm actually going to go ahead and delete one of my examples here just so i have fewer images on my account so that i can proceed in a nice clean easy way but obviously you can just zoom right into each of your individual data sets so okay if i'm looking at one of this individual data set here the question i should be asking myself is how can i pre-process and augment my images to in uh to prepare my images for my model through pre-processing things like resize or changing contrast or making everything need black and white if i want to do that and then augmentation and augmentation remember is increasing the number of images in my training data set and adding variation to my images so that my images will generalize better to different conditions so for example an example of an augmentation would be something like flipping my images in a horizontal flip across the y-axis so that my model doesn't memorize the orientation of any of these pieces so you'll notice here that flip is an augmentation that makes a lot of sense for our chess problem so i'm going to leave that applied now there's a bunch of oops i just unapplied flip let me reapply flip i do want that apply now let me also apply brightness because my chest pieces could be in an image that's bright or dark and you know i want my model to not memorize what brightness or darkness my given room in so i'm going to add that as a step too i'm also going to add a little bit of random crop so this will randomly zoom in on a part of my image so to make it look like the camera is randomly getting closer or further from some of my images and then maybe another augmentation step that i can do is a little bit of blur a lot of blur would be too much but a little bit of blur is a good idea so each of these augmentations will randomly be initiated and apply to my given data set now let's see any other augmentations i might want to add at this point um maybe a little bit of shearing so shearing kind of tugs at the corners of my image and that allows my my model to be resilient to perspective changes meaning the camera might not always be head-on to my data set so this allows us to control for that change okay now i've i've got enough the other thing i want to note is that i applied a resize decision here so as a preprocessing step i'm going to resize all of my images to a square because many object detection models assume square images that's because they have fully connected layers in the neural network and the way the math works out requires that there's a one-to-one match of the squares so i'm going to do a 416 by 416 a pretty common resize okay the next thing to note here is i'm going to add three versions each of my images so in other words my 202 images that are in my training set is going to increase big and in times three in fact because i'm going to take my individual one image is going to come in it's going to be randomly flipped horizontally randomly cropped randomly sheared randomly made brighter darker and randomly blurred so i'm going to generate this and call this v1 youtube and generate that given data set while that is that's generating let me go back and show you that on my account my uh my youtube dataset finished uploading but i don't need that anymore so i'm just going to go ahead and delete chess pieces youtube just because i don't need that one anymore okay so with my images generating here i'm creating a wide variety of different images that allows my model to be ready for training now this is the key part now that i have a data set that i'm happy with i need a model that i want to use so go to models.roboflow.com to grab any given model from the roboflow model library so today i said we're going to be using detectron2 and you'll see that model here and i'll also put a link in the description now what i want to do is actually grab the detectron true collab notebook okay so you see what i did there is i clicked the tektron 2 colab notebook and this collab notebook comes batteries included with everything that you need to do to get started to train your very own object detection model using the detectron 2 framework now you might be asking yourself what is the tektron 2 well detectron 2 is the facebook ai teams wrapper around pytorch for creating computer vision models they have key point detection semantic segmentation and object detection today we're going to be doing object detection and specifically we're going to be using a faster rcnn architecture okay so let's start by going through our notebook here first things first i need to install oh i forgot a key step first things first is i need to make a copy of this notebook i'm not the owner of it roboflow is so i need to go to file save a copy and drive file save a copy in drive and so this copies the entire notebook and then creates my own example that i can then use and run okay so that copy is is being created and while it is let me talk a little bit about the first cell so the first cell that we're going to run once our notebook gets copied is installing all the dependencies that the detectron2 library is going to assume so that includes a specific version of pi torch a specific library for handling yaml and then installing also the cocodataset reading in tool and that's a hint for the dataset format that we're going to use here in a minute we'll then import torch in torch vision and print out the version of each of those respective libraries to verify that everything worked okay and that just makes it basically make sure that our environment is set up correctly now once that finishes we're also going to pip install detectron 2 from the detectron true repo and that's directly from the facebook api public files once that installs we'll do all of our imports okay so we're going to import detectron2 import our logging function we need numpy opencv random our ability to show images and then some specific common detectron 2 uh model training tools getting our predictor getting our visualizer all these sorts of things now the next thing you'll notice is that we'll get to a spot in the notebook where we have to import our own data set so this is where we need our data set from roboflow okay so in roboflow you'll notice that we were prompted to create an export with this export we are actually going to do coco json that's the format that the uh detectron true framework is going to assume so i'm going to click continue here and that's going to zip everything up and then create a link for me to use this given data set now this link here is unique to your account and you should not share it i'm going to delete this data set and also change my account key so you're not able to use it but you should not share that with anyone else okay so i'm gonna go to my copy of the the notebook uh and i'm gonna also call this uh i don't know jn youtube okay so now i'm gonna run that first cell just like i was talking about while everything was getting copied i'm going to run the next cell i'm going to run my import cell okay and this is where i'm going to paste in the link directly from roboflow and so that curl link grabs my data set and brings it into my colab notebook in the exact format that i expect now the next thing i need to do is just set up my train valid test split remember roboflow created those for me and so when i created my export all that stuff was already handled for me and roboflow intelligently does all of the uh splitting and such that the names are generic and the annotations refer to their specific spot so my train valid and test are already all set up for me now this next cell here just prints out a couple of random images in my data set to show that everything looks okay now this is a technically a not required cell but it's useful right because it shows that everything is generally loading the way that we expect it to load and here we're just waiting a moment for the imports to get fully set up in our environment which it can take a moment based on your internet speed and all the packages that have to get downloaded okay so after um after all those have run as i mentioned we'll run this cell which prints out examples of our given data set that might be of interest to us and then after that we'll actually get into training okay so before i jump to training let me show you that roboflow stores my data set at this link on zips and then removes that zip file from the co-lab environment because all the files will be imported directly okay so in colab i can go over here and i can see which files are being added to my given directory so as things get set up this will update with what's happening in my given environment now a quick note about collab the environment isn't permanent and so i mean you on the free tier especially you may only get 12 hours and if you leave a notebook on run then you don't have the ability to uh rerun it later so it's useful for trying things out but when you get to production i recommend using something a bit more official if that makes sense okay so it looks like everything here has been installed correctly yep that was installed correctly and now detectron 2 is getting installed correctly and now we're importing everything and now we're downloading everything from roboflow just as we said and we can print out some of our examples from our training set awesome it looks great okay so the next step here is we're just going to set up our directories and then this cell here is key this is where we set up our configuration for our given data set now you'll note here that we are specifying in our config file which model we're going to use right here now i'll put a link in the description for how to change this model but we're basically grabbing it from the facebook object detection baseline okay so any of these yaml files i grab this one and you can copy link address and then paste it right here and it's that yaml file so i'm going to use faster rcnn today but as i said you could use any of these other ones as you may want to do so the other key thing here is the training iterations right here i'm only going to do 1500 and even that i'm not going to get through in my youtube video i'm going to fast forward to a different notebook but as you see your training improve increase this number because that's basically improving your model's fit over time okay so that actually kicks off model training and here i'm gonna jump over to a model that already finished training in my other notebook environment okay so if i go to my other notebook here training has finished in this notebook for me and so when training finished or actually as training completed it prints out summaries of how my model is doing across each of my individual classes which is really really nice right so i can see that which classes it's doing a good job on and you know not surprisingly it's doing the best job it looks like a really good job on the pawn pieces because it had a lot of those examples the king pieces also did a really good job on that king piece must be really distinctive now i can run my tensorboard and i can see the bi-piece performance for the mean average precision which is how well my model is finding each of these individual pieces now in this example i actually trained for 3 000 iterations and my graphs are still getting better which means i could probably train for more time last but not least i can print out the results and show the test performance predictions and that's what i've done here is i've run these cells and printed out my results and just like that between each of those steps you will have trained your very own faster rcnn model using the detectron2 framework please like and subscribe drop any comments that you have in the comments section and happy training

Channel: Roboflow

Views: 26,057

Keywords:

Id: 4OXntFVfFio

Length: 21min 10sec (1270 seconds)

Published: Mon Nov 02 2020