How to Train YOLO v5 on a Custom Dataset

hey there it's Joseph from Roma flow today I'm gonna show you how you can train your own Yolo v5 object detection model to any image data set of your choice that is we're gonna train a model to identify bounding boxes in state-of-the-art performance in any custom set of images that can be finding logos or finding machines aerial imagery you name it before I get too far let me show you how powerful the Yolo family of models are okay so here I actually have a video that I ran through Yolo and it's a video of me just riding my bicycle in my driveway by default I use the Koko weights so it's looking for things like bicycle person car etc and you'll notice that these predicted probabilities are updating extremely quickly these changing numbers indicate we're seeing over 50 frames per second now freeze frame you'll notice that this model even does a pretty good job at finding small objects in the background look at this it found this umbrella it's not very confident that 0.47 then it's an umbrella but still that speaks to the power of even seeing and identifying that object okay so now the question naturally becomes how do you get to use a model that powerful on your own data set and that's what we're gonna walk through today now let me show you a little bit about the data set I'm going to be using but to be clear you can use any set of images video as long as you break it in images and label them and this walkthrough will work from there so the set of images I'm using is called the blood cell count and detection data set and by that I mean I have a series of images that are blood cells let me get myself out of the way here a little bit so here I have a white blood cell and red blood cells and these images were captured from a microscope this is an open source data set that is free to use and I'll show you where to grab it here in a second but I have a few hundred images and notably each of my images I already have the annotations so Miami just happened to be annotated that is defining where the bounding boxes are hello in vaak XML okay vaak XML you can use any annotation format for today's tutorial Cocco json tensorflow object detection csv Vaught csv label image see that scale label box however your images were labeled you'll be able to use them for today and if you want help labeling your images be sure to reach out we're able to label images for you or point you to tools that you can use that are free I'll also link that in the description here today okay so where did you get this data set if you don't already have one ready and available I'm glad you asked you can find this data set on public dot Roma flow dot a I so if I just go back to public dot Rulfo AI I see a series of actually datasets the row flow team has made free for anyone to use now as I mentioned the one we're using as BC CD blood cell count and detection it was a virtually open source spy on github by these two authors a couple years ago and they were kind enough to use an MIT license so you can use it for commercial purposes now if you have if you don't have a data set you want to follow along with this tutorial exactly go ahead up here and click fork data set and that data set will be get added to your Robo flow account um all turn ative lis what I recommend that you could do is you could go into this specific version of the data set and download this data set now that's actually what I'm going to do is I'm gonna download this data set because I want to show you the upload flow of getting images into Robo flow before I go much farther let me show you what we're striving towards what is the model going to do when all is said and done glad you asked when we're all done with this model we'll be able to see the outputs like this you'll notice that each of these images there's these bounty box that are drawn with predicted probabilities of how sure are you low V 5 model is and here it's doing a pretty good job it's finding the white blood cells the red blood cells it even finds these little platelets you see in the orange bounding boxes so Wow how do you get to do that on your own data set well okay first things first I want you to be sure that you have your data set ready and then I want to go to Robo flow dot ai once you're here you can go ahead and sign in or try it now if it's first time roll flow is free for small datasets like the one that we're going to be using and only a cent per image for datasets over a thousand images so it's really easy to get started what we're doing today is completely free we'll also be using free training resources via Google collab so you won't have to pay a dime ok so I'm gonna actually already have an account so i'm gonna click sign in here and boom i have this empty account state I don't have any datasets that are created if you're totally new to row afloat you might want to do our sample data set here and walk through this chess example I've done the sample so I'm actually just gonna jump right in to be able to create my own dataset ok so I'm gonna go up here and I'm gonna click create dataset and I'm getting my data set a name I'll call it B CCD Yolo b5 because why not and the dataset type is in fact object detection that is we have a bunch of bounding boxes that we want to find in our images now the annotation group this refers to what it is that you have labeled in your what objects you've labeled so in mine I have a bunch of cells that are labeled and specifically I have red blood cells white blood cells and platelets so I'll go ahead and create that data set ok now I have the empty data set the next thing I need to do is just get my data in to Robo flow and the way that I can do that is I can just open up my my images here or hoops keeps flipping over open up my images here and you'll see that I just drop all of this data directly into Robo flow and it will process it for me so I'm just gonna click and drag and just like that look Robo flow matches up all of my images and my annotations and it actually performs a couple of checks for me you'll notice that this dataset had a couple of problematic annotations there were 3 0 area annotations which means by accident the person that labeled this might have accidentally clicked and made like a zero pixel annotation those are going to confuse your model so rebel flow by default drops bad annotations like those rubber phone also performs a variety of other checks like if your annotation extends outside of frame you can here if your annotation is slightly outside a frame but partially in frame rofo will crop it's that it's only the part that's in frame or if you have an annotation that's completely out of frame like my hands right now Robo flow would not include those those are all the checks that we perform on a data set by default okay great so I can see that you know my data looks it looks pretty good you'll notice here that maybe I could have done a better job labeling this guy but overall I'm I'm pretty impressed with how things look so I'm gonna go ahead and start upload when I start my upload Robo flow prompts me to do a trained valid test split by default this is best practice in machine learning you want to have a training set where you teach your model you would have a validation set where you test what you just trained it and then you wanna have a test set that you're used when you're finally done after your training and valid back and forth back and forth so you never use the test set for until you're all the way done with your problem training and valid now by default a 70-20-10 split is pretty common you can adjust these if you like I'm gonna you just stick to the defaults and do 70-20-10 okay now this is gonna upload all my data as quickly as possible while this is going I've prepared a little bit of a talk here about why you want to use Yolo v5 in the first place I'm glad you asked so not only is it extremely fast and in a really relatively performant model but there's a couple other reasons first and foremost Yolo v5 came out really recently only 50 days after Yolo v4 it is the first model in the Yolo family to be implemented entirely in pi torch Glen Yoker is the person that introduced Yolo v5 it's still under active development that means actually even yesterday it got a little bit better and so it's gonna continue to improve the name may change as well there was some discussion about whether Yolo v5 is the right name for it because it's new author and a new framework for now the community seems to be calling at you love you 5 so that's how we'll continue to call it ok so yo la vie for vs. Yolo v5 performance benchmark now on the official Coco data set Yolo v5 is actually slightly less performant than y'all a v4 which is why are you introducing a model that might be slightly less performant well if you think about it the goal of models is to be easily accessible actually in fact in the Yolo v4 paper there was this quote from the author that says we hope that the designed object can be easily trained and used for example anyone who uses a conventional GPU to train and test can achieve real-time high quality convincing object results Yolo e5 builds on this philosophy Glenn even wrote in a comment just yesterday that his goals are not just to have high ma P are not the best I'm ap even but ease of use export ability to things like mobile memory requirements for fast training which we'll see here in a second speed of inference good ma P and market size there's so many problems that you can work on in object detection so okay so if that's like the why for Yolo v5 let's talk a little bit more about what we got going on here so our geta finished uploading now the first thing that I recommend you do is you do a data set health check of just inspecting and getting a sense of what's in my data set and so we'll see you that our red blood cells is dramatically over-represented compared to our other classes well across three hundred sixty-four images we have over five thousand annotations the majority of which are in our red blood cells some white blood cells and some platelets now you can upgrade your Robo flow account to see dimension insights in annotation heat maps I actually have to do to do let me let me show you actually because this is a public data set we can look at the public health check and we can see a preview of these things so actually look at the annotation heat map here the red blood cells are generally all over the image the platelets are very small so there's small objects and that's good to know because small objects are sometimes trickier to find and the white blood cells are generally concentrated in the center of the image hmm if we do a crop we might want to keep center for that reason I don't know okay so that's the general gist of what's going on in my dataset from here inside Robo flow I can perform a series of pre-processing and augmentation steps on my data pre-processing are the steps that are required to get my data into the right format that my model is going to expect augmentation our ways that I can increase my dataset size so that I have more images to learn and train from it's gonna increase my models performance because it will reduce memorizing this small set of examples that we give it now by default Yolo actually expects images to be in multiples of 32 so a pretty common size is resizing to 416 by 416 you can actually resize as a stretch as you see here I could do black edges in so I keep the aspect ratio the same or I could do white edges it just kind of depends now regardless of what I do I want to make sure that I do the same for my train and test sets in my resize decisions and for this example I'll go ahead and I'll do stretch so I'll stretch I'm also gonna remove Auto Orient which we have a blog post about I'll link in the description generally you want to do that if you don't know what it is okay augmentation steps you know some things that might make sense for my dataset are 90 degree rotations because these cells could come at any orientation so I'm gonna add that augmentation step and maybe I'll add the ability to do flips as well which reverses the perspective either horizontally or vertically so you notice that the original image the vertical and the horizontal flips and that's gonna help my model not memorize the training data and see more examples now the last thing here is that I'm gonna create three versions or three augmentations of my source training data so with those examples in mind I can go go ahead and generate a version of this dataset so I'm going to generate I give it a name I can either give it the timestamp name or I'm just going to call this BCC dv1 I don't know I'm not very creative okay so I go ahead and fire off this training jar or this schema generate job and what this does is it creates three versions of each of the images that are resized correctly flipped randomly or randomly rotated 90 degrees at various different angles now while this generates I want to introduce you to the Robo flow model library data models dot Robo flow die we have a bunch of models here that you can use by default now today we're gonna use yellow d5 but if you want to try Yolo v4 because you needed that extra bump in performance or maybe you want to be efficient debt for faster training maybe you wanted faster our CNN those are all here and they're easy to use now okay first things first for the Yolo v5 model oh look it's me riding my bike again I we also released a a performance breakdown so if you want to dive deeper into the details of exactly how yo of you five does feel free to do that okay but what I want from here is the collab notebook do you see this link click to open that collab notebook and so by default I get this how to train Yolo v5 on custom objects we're gonna leave this here for a moment and go back to my Robo flow data set I'm gonna X out of the version here so what happened so we created a version you'll see here on the Left I have a version of this data set and all my images have been either rotated and flipped they're resized correctly so yeah I mean I now have a lot more images to learn from and my data set looks pretty good now I need to export this data to put it in collab for the purposes of training right so okay so if I want to export this data set that begs the question of what format do I want my data in and as you may have guessed we're gonna you on yo v5 pi torch yo o v5 pi torch now how do I know that well inside my notebook we actually put in here hey be sure to use Yolo v5 hi George so it's called out there right they right there for you fits you right in the face so okay so I'm gonna create that export now warning warning warning this export creates a key an API key that's unique to your account do not share this link publicly so I'm gonna actually generate this and while this is zipping I'm gonna actually move this out of frame because I don't want you to see the API key that generates when this is all said and done so actually look it's generating generating generating and finishing okay now I get a link at the end of this process that I'm gonna drop into my notebook now I'm not gonna show you my link because as I said it's okay so let's jump into our notebook here we're gonna be using PI torch to train our given model now first things first I just want to clone the oh I forgot I don't own this notebook this is the Robo flow one from the ruble flow model library this colab notebook so I need to go up here this is important and click file save a copy in Drive this creates your own copy of the notebook that you can edit and do whatever you want with it okay so files save a copy to your own Drive and now I'm gonna X out of the first one and now all I have is my copy now I'm gonna rename this from copy from to something a bit more descriptive I'm gonna call it JN because it's mine Joseph Nelson rebel flow custom you owe me five ok great so now I have this this this notebook I'm gonna run the first cell here and basically get my dependencies all set up and that's gonna run and get my dependencies meaning things that are required to run this bit of code so the Yolo v5 notebook requires things like by torch and a specific version of PI torch and basically helps with reproducibility so a specific version of numpy OpenCV etc now in the next cell while that's important all those things let me show you over here you can click to see all of the files that are in your collab at this given time and so all so far I've installed this yellow v5 or I've cloned this yellow v5 into my environment I'm gonna CD on into that directory and I'm gonna import PI torch ok great now export codes dip it and paste here what's that that's the code snippet that you made from Robo flow and so I want you to copy and paste the code snippet that Robo flow gives you and drop it directly into this source so again I'm gonna sneakily remove that sneakily remove that paste mine in hit enter and then I'm gonna remove my key and bring it back my key so I removed my key and then I just wrote my keys you can't actually see what it was and you'll see that this extracts and runs everything and it asked do I want to replace the data yeah Moe the answer is yes so I'm gonna hit a because I'm gonna replace all instances of it so hey okay great so now if I open up my yellow b5 wedge oh look here I have trained valid and test an inside train I have my images and my labels and everything is formatted correctly for how this model expects to run to use yellow v5 you need to have a specific mo file that defines where your directories are structured rubba flow mix that for you and so we can actually print out we can cat that data yeah mo and you'll notice here that it points to the trained and valid spots the number of classes is 3 the names of those classes are platelets red blood cell and white blood cell now the next thing that we're gonna do here is we're gonna define the model of configuration and architecture now I actually um I'm gonna go ahead and get the number of classes and then I'm gonna define all the layers here now you don't have to edit this but if you feel comfortable you can you can actually write additional you can write your own anchors if you want you can change the backbone or the head of the architecture I'm just gonna use the defaults and just go ahead and make that and when it's done it says print custom model config is written whoa now we're ready to start training our data is in the notebook we've already created a specific version the augmentations worked in all of my bounty boxes are updated thanks to rubble flow and so now I can call this train command this train command includes a series of arguments so in image is the image size so it's 4:16 batch is the batch size number of images that are each training batch epoch is the number of epochs we're gonna train for now often you want to do like 3,000 or more for pretty serious data sets for this little itty-bitty data set just gonna do 100 and we'll probably get away with it just fine and the data is the data ya know file defining where we're on the config is the model configuration file we just created the weights if you want to start training from a specific set of weights you can specify this here or you can just start training from no name are the the result name of what you want to be called for this specific run so we just gonna call it Yolo v5s results and then no save which means only save the final checkpoint it's a little bit faster training if you do that in cash we're gonna cash the images for for faster faster training okay so a couple of notes here so I'm gonna kick this training job off and we're training Yolo d5s the smallest one there's actually a little bit bigger ones if you remember on my little slide here this comparison know that that uh Yolo v5 large is also available this is Yolo be 5s which trains super fast and does inference really fast as well so I'm gonna train this for actually a hundred epics here and I'm gonna actually start this training job but I might do some movie magic to cut out but I want to show you how fast this model trains on a hundred epics for free in Google collab okay so I've just kicked off and started training it's setting up all of the layers of my network it's setting up the directories for where it's going to save the weights in the model configuration and it's printed out the model summary okay and here we go it's gonna cache each of those labels and now it's gonna kick off training for a hundred epics starting at zero index okay so if you look at this it trains really quickly we're passing what it was it a thousand how many inches were we will be doing oh I moved that to my other window we're passing through a thousand and ninety two images across a hundred epics and this thing is zippy quick we've already completed two epics in the time of me me blabbing here now I'm gonna go ahead and do a little bit of movie magic and I'm gonna fast forward to when this finishes training if you want you can you know pass the time by maybe labeling some more data or doing something that you need to do okay our training is finishing up here you'll notice it's on epoch 98 of 99 and our ma P is kind of topping out around 0.9 our recall is pretty consistently 0.98 0.99 so I think we've got the most we're gonna get out of this given dataset you'll notice that these a hundred epics across 1092 images took only 18 minutes and 14 seconds which is amazingly impressive for how accessible this machine learning model is okay so I can actually start tensor board and be able to evaluate and view my given results so tensor board is an easy way for visualizing the performance of any given model so give it a second here to queue up you'll notice that from tensor board I'm able to see that actually you know what it looks like my model topped out at frankly around mmm step 40 or so and I didn't need the rest of those epochs if anything I might want to be worried about overfitting the same is true for recall and precision and my f1 measure so this all seems to say that I've gotten the most I can get out of this given data set either need to collect more data or improve my labeling quality to see if things could be better from here so okay that's that's great and then the nice thing is that this volt implementation from Ultra Linux also contains the ability to print out the results and at a non tensor board format in an image file that you can save and do whatever you want with so it's inside results dot PNG so I can actually see that over in my directory okay now hmm alright so the next thing that I may want to do is first I'm gonna print out my ground truth data what are the data that I used to Train look like what was my ground root training in it so this was the ground truth training data this came from Robo flow and you'll notice that I had my white blood cells my red blood cells my platelets and you know everything looked pretty good what did it like with a little bit of augmentation so print out a training image but with that augmentation that's been acquired and you know we have these shifts and changes and so forth and overlaps and you'll see that it's you know intentionally distorted a little bit okay so how did that perform on my test set first things first my weights file is inside the weights directory so last Yolo v5s results dot PT dot pi George these are the weight that I can use and save download use elsewhere convert to own and X for the purposes of doing on mobile it's there that's the weights file that's the magic now okay I'm gonna see beyond into the yellow Fe Phi directory and I'm gonna run detect by using the last weights image size 416 confidence interval of 0.4 and the source is going to be these test images now remember my model has never seen these test images before and so let's see how the model does okay so it rolled loads in all these images and then one by one once the model is compiled it conducts inference and you'll notice look at that inference time point zero zero eight seconds really really impressive per image now let's print out and display each of my results so print this out and I'm gonna print out everything that contains a dot JPEG format in the inference directory and here we go so look at even see it you see that I almost missed that that little platelet there this right blood cell a red blood cell each of these red blood cells this white blood cell man this model is killer it does a great job at finding everything that I needed to find for counting and detecting these blood cells see the platelets there in a notice that I set the confidence to 0.4 which means nothing below 0.4 is gonna display across all of these images okay so now what I want to do I might want to export these weights for future use so I run this cell to connect to my Google Drive so this little thing here and then I use that and I drop in my authorization code I'm gonna do that off-screen I'm gonna do that off-screen copy in my authorization copy and paste and bring that on back ok so cool now you see it's mounted here so now what I'm gonna do is I'm gonna copy the last weights to my Google Drive and boom the weights are now in my Google Drive if I go ahead and open up my Google Drive folder they would be there Congrats you've done it you've trained your own custom Yolo v5 object detection model now with your results honestly cannot wait to see what you build next with Robo flow reach out if you have any questions help a trouble flow da TI if you can't get your images loaded in if you want to collect more data if you want some labeling tips you name it we're here to support you now can't wait to see what you build and good luck

Channel: Roboflow

Views: 81,099

Keywords:

Id: MdF6x6ZmLAY

Length: 28min 38sec (1718 seconds)

Published: Sun Jun 14 2020