YOLOv9 Training on Custom Dataset in Google Colab with Roboflow

so Yol v9 has just been released in this video here we're going to take a look at how you can train your own custom YOLO v9 model so we're going to cover the whole pipeline how you can take a data set we're going to use Robo flow have them annotated in there then we're going to explore our data set then we're going to jump into a Google call notebook we're going to run through a couple of blocks of code we're also going to take a look at the G repository for Yol v9 and now we can go in and train our own custom optic models with Yol v9 so this is the new version of the YOLO family we're also going to compare it and see the benchmarks with yolo V8 and also all the other different viations of YOLO in the YOLO family but now we have a new version YOLO v9 is out first of all here let's just jump straight into the GitHub depository we have YOLO v9 we can just go down take a look at it this is based on a lot of YOLO V5 code so you guys are probably familiar with all of that and the whole code structure how it works with the weight files how you can set the different arguments and so on but we have all the code here for both for detection we have panoptic segmentation we have classification detection optic detection segmentation and all of those different on things with these new models if we just scroll a bit further down we get to the interesting graph here where we basically just doing comparisons between all the different YOLO variations out there the different versions of YOLO there's also some other variations and so on the real time uh detection Transformers and now we can see that we have this YOLO v9 and also this Galan here which is the new released models so over here to the left we can see that there's actually like a lot of stuff like a lot of grabs here being compared to each other but we can take a look at it we have a new stateof the-art OP detection model on the MS Coco data set so here we can basically just see that these are the model variations we also have different model variations for this new Yol 9 architecture if we just scroll a bit further down we can see that we have the small medium C and also the E model we see the test side the aror position on the validation set here and also the error position um 50 and 75 the number of parameters and the floating Point operations which is needed to run these models so just by looking at the parameter size here we can actually see that these are some smaller models so I'm really excited to test this out so we're going to do a bunch of comparisons both with y v 8 Yol V 9 going to have them side by side test it out on custom data set try pre-train models basically just compare them and see if we actually have an improvement I know that we have an improvement here on the graphs but sometimes it is not better when you're actually training on your own custom data set and this is only like slight Improvement as well so if you just take a look at what stands out the most I will probably say that this is the number of parameters for each model where here we can see that we have YOLO V8 in this color so we can see that we have Y8 here in the light purple so these are values here we can see that it is slightly larger compared to if we just take a look at Yol v9 I'm also going to test out the Gan model but let's just take the Y 9 where we can see that it actually has comparable average position except like the larger models up here at the top but the model size has been decreased significantly and act like um it looks like it is half of the size of the U V8 models but let's see if it act like speeds that up when we are running inference with it later on so yeah now we have the Yol v9 model again we can see here that on the benchmarks it has better performance compared to Yol V 8 but it all comes down to the specific task and also the specific data set that you're trying to solve and train your models on so now get into the fun part how can I like go in and train these models on a custom data set so we now jump straight into the Google cab notebook first of all here we just have the graph that we just saw in the GitHub repository as well so let just scroll over that now we need to get started first of all when we're going to train our own custom models we need to go in and choose the runtime so you can actually like use this free GPU on Google collab so we need to choose the hardware accelerator S4 GPU and now we can just connect to the runtime and we can call this command Nvidia uh- smmi where we'll get all the information about our GPU which is connected to our runtime after that we're just going to run a couple of box of code here where we're going to import like the OS we're going to get the current working directory now we can see that we get our information we're using a Tesla T4 dpu with 15 GB of RAM now we just going to run this blog code now we have just set our current uh working directory to our home so we just have a variable that we can work with later on now we're going to clone and install the GitHub repository for yov 9 so that's pretty much everything that we're going to do here we have our git clone and then we just CD into that directory and install the requirements from the requirements file so that is going to set up our whole environment and we can use it directly after that then we're going to P install Roo flow as well because we're going to use Robo flow to take our data set annotate our data set in there export it and then we can just directly copy paste the code snippit into our Google C notebook and train our models directly on that so let's now go down and P install rlow as well and then we can import a rlow and also this image from I python. display after we've done that we can go down and download our weights while it's just doing the installations so first of all we're going to create a directory for our weight files and then we can go inside the GitHub repository and basically just download each of these variations we have our Yol v9 c e d Lan c and e so we can do some comp comparisons as well and we're just going to throw those files into our weight files if you want to download locally you can also just copy paste this throw it into um the search bar and it's going to download it automatically so it's basically go just going in here extracting these ones you can also just get them directly from here if you press on this uh hyperlink it will just download automatically to your computer so now we can go back again let's download the models only need to run this block code and it will do everything for us and then after that we can just show what we act like have in our WS directory there we go now we can see that we have our Galan Galan e Yol 9 c and e version and these are the P sour versions so now we can download an example data so first of all we just need to make a directory for our data then we can either go into our Google Drive uh download an image from uh from Google or something like that or we can just upload one to our runtime here in Google collab notebook from our own computer and just specified image source path so now I've just uploaded an image from my own computer and we can BAS basally just specify that as the path so let's just go and copy the path paste it in here and now we can just specify the source image path and call the models directly so as I sh in inside the GitHub repository we have this detect. piy then we need to specify our weights and here we can just choose the weights that we have act like downloaded and put inside our weight directory we can set the confidence score SCE and also the device so here for the source we act like need to specify the image path so we can just set it like this there we go and now we should be able to run this block of code and it will do detection of this image that I have uploaded so just now try to run it we're going to use the Gan C model and we should get some results in just a second so it's going to set up the model run to detect and then it's just going to do inference on a single image so when you're running this command here you can see that this looks pretty much the same as when you're using Al lytics for UL V 5 and UL V 8 so we have the detections we get the model summary of the layers number parameters and we also get here the output results from our detection when when we're doing inference so here we can also see the inference time the pre-running time and also the shape of our image so we can also see that the results is saved to this directory so let's go inside our Yol 9 runs detect and then we can see that we have X4 and now we can see we have our screenshot 70 PNG and it should contain all the annotations as well so just make this a bit larger it will open up the image and we should be able to see all of the detections from this Galan model so this is just how we can do inference on a single image you can also specify like uh video file path webcam pretty much like everything inside the detect Pi file so that's what we're going to use when we're using these models so here we can see the results from our first run again we're just running this with a confidence score of 0.1 so it's just going to show us all the detection so we should get some false predictions here and there but generally we can see over the whole image we're able to detect these cars in the image and let's just go back I think we're running this on 60/40 images yeah that is correct so if we were running this on 1280 images we would probably be able to detect the cars up here at the top and also in the line here going down on the highway but generally we can see that we're pretty much detecting all the cars here this looks very good just initially I will probably expect these results with Yol V8 as well so it's hard to say much about that um already but this is how you can run inference let's now go down and see how we can actually like do this uh on your own custom data set so first of all we can just go inside and display it directly so if you want to display it here but I just open the file up here as well so just specify the home directory and the path to that image we can also use y v9 so right now we have just used this Galan let's just go and take this image instead specify the source down here at the bottom there we go and now we can just directly run it again and now we're going to use the Yol 9 e model then we can do a comparison between Galan and also the y9 model so I think this image here is pretty good example just to demonstrate how the object models work because we have some very small objects in there and I think that like most object detection models that have some problems here and there when it is act like smaller objects in the image especially like on Lower resolution images which also makes sense so this is the results that we get from the Yol 9 model compared to the Galan model I don't think that this is good compared to the Galan model like we're missing a lot of detections like a lot of these cars here were detected also up here in the line all of those were detected with the Galan model so we definitely need to take a more in-depth look at that why it's causing those problems because again I feel like these cars here should be detected and would definitely be detected with Yol 9 as well so we definitely need to test this model here out and more and I'm going to create a video where we're going to comp compare it with on a bunch of videos like we're going to run over a lot of videos with different image sizes different model variations and so on and then we're going to compare ul v9 and Y 8 directly side by side so let's now go down and see how we can train a y 9 model on a custom data set if you just start inside of Robo flow I have a bunch of projects inside my Robo flow account you can just go and take one of those I'm using this cup data set here for showing you guys just some basic custom Training of different models this is The Benchmark or like my data set that I've been using throughout all my videos so you guys are probably familiar with that but you can go inside Ro flow create a new project you can upload your data and you can go in annotate them very easily so just have a bunch of CT that I've captured with my webcam I have videos about all that how we can set that up how you can create scripts to generate your own data set with your webcam and so on so definitely go in and check those out so just going to delete this Halloween cup here with roof flow you can just go over and choose the bounding box tool then you're just going to draw a bounding box around your object and then you can specify the class over here once you've done that on the whole data said we can just go through a bunch of image examples so here we can see that I've annotated all my images already once we have done that we can go in and generate a new version of our data set we can choose different augmentation steps and so on we can even see that we can go in and train a model just to see okay is our data set good enough what accuracies what different metrics can we act like expect from our models again this is pretty good so we we can definitely use this data set on our own custom models try to do some fine tuning here and there so this is good for just getting like a quick overview but if we just scroll a bit further down we can see the data set details we have augmentations so horizontal vertical clockwise counterclockwise upside down some cropping some zooming effect in our data augmentations as well and also rotations so if you want to generate a new data set you can basically just go down to generate create a new version we can do some pre-processing depending on the image size that you want to use as well you can also play around with that and choose like if you want to use 1280 640 and so on if you want to run real time we need to run it on 640 images after that we can use augmentation flip rotate crop rotation and we can see we have all of these different Image level augmentations but we also have some augmentations for bounding boxes once we have that we have our whole data set we don't need to do anything more than just exporting a data set copy pasting a coast nit into our Google collab notebook train our custom Yol v9 model export the model using an own applications and projects as we're used to so let's go inside the versions here once we have our data set we can export the data set directly now we need to choose the YOLO V8 or we can also choose the YOLO v9 here they have already updated it to YOLO v9 there we go we can hit continue you can also download a zip file to your computer let's show the downloadable code and then it's going to zip the files here we can copy paste the cnpp it here it's just going to like blur out my API key you need to set your own API key up and then you can use it directly out the box so there we go now we have copied this code snit we can paste it into a Google collab notebook run that blog of code I'm just going to do that so I'm not Shing you my API key but you just need to need to do the exact same steps then we have our data set uned in our Google hold app notebook and we can train our models directly so after running this bloger code we can see that it is loading the roboff flow workspace loading the roboff flow project downloading the data set and also extracting the siip version into our directory so here we can see all the images that we have if we go inside cop detection version two we can see that we have a test train and validation split and we also have our data yl file which we need to specify for act like running and setting up our data set so we have the number of classes we have the names for our classes so these are labels we have the path to our train test and validation split and then we just have our roboff flow workspace and information here as well if you just take a look at it we can see that we have our images and also our labels so this is the folder structure that we need to have so now we have everything we can just CD into y9 again and now we also have a train script so train. pip where before we used to detect that pipe to do inference we need to specify the bat size EPO image size the device right now we're just using a dpu so we specify zero minimum items the data set location to the data yl file and also the weight file that we want to fine-tune our custom model on so now we're good to go we can just run this blog code and let's see the results so right now we see the into that directory we're going to do all the setup okay so we ran into an eror here where our data set is not found I've experienced this with rlow a couple of times so it's the data y file so first of all here up at the top we just need to make sure that this is actually like pointing to the correct directory and I like to just go in here and just copy the path for the images instead so there we go let's just p in the full path and let's also do a for validation and the test so here we just need to change this to valid and also train here so that's test and then we need train up here at the top there we go now we should have all of it let's just make sure it's called valid we can close the file again let's try to run it and see if it's able to load any data set now so usually just going and check this data Gil file just to make sure that it's pointing to the exact directory so we can now see that our model has started training we can see here that is just running Epoch Z out of n it's just running through all of the badges so right now we have a badge size of eight you can see that all of this is familiar with uh allytics and also when you're using Y 5 and Yul 8 now we can see that we're at Epoch 1 and we can just track the metrics here over time so now we can see that our model is done training at the end here it pretty much converts to 08 in the mean AR position 50 and here we see that is still increasing in the mean position 50 to .99 in intervals of 0.5 so it has not fully trained yet but again let's just stop it here and let's take a look at results you can just let it run for more Epoch we can also take a look at the class lot class loss it's still um decreasing here which is should the Box loss and so on so this can definitely be fine tuned for um a longer time and more EPO here we can see the validation runs at the end so this is basically just on our validation images we can see each individual class as well then number of images instances the Precision recall and the mean aage position so again we can see that for the cocka cop we have pretty good Precision recall so that is also very good and also for the mean average position where we're lacking a bit is the hand painted cup here as you can see so this is has a bit lower Precision um The Recoil is pretty good but the Precision is significantly lower for a hand pained cup so this is really good for going in validating your model on specific classes and so on that you have in your custom data sets so we can now see that the results are saved into a runs train x3e and if we just go over to our y 9 we can see we have our runs train x3e and then we can see all the results in here you can take the weight files you have the best train model here and also the last train model so you can just go and download it then you can use it in your own applications and projects exactly as we're doing in the tech. P script you can also just copy the path here and throw that into the detect. P script that we had at the top and then you're going to use your own custom models so this is one way we can do it we can also just go inside our directory and just take a look at them and directly display them in our Google cab notebook so here I'm just going to change this to free so just make sure that you change this depending on the number experiments that you have run so here we can see that we have our results so we have our box loss over time so we have our number of epoch and then we have our values so we can see all the losses they're pretty much decreasing we can see our Precision recall and also the mean average Precision the mean aage Precision has pretty much converged but it can be trained for way longer we can also see that the losses are still going down % recall is going up so this can be trained for longer but this is really nice to get the results directly after we have trained we can also get the confusion Matrix and we can also get a validation batch zero prediction so we're just going to take a batch do predictions on that and then we're going to see that from our validation set so these are predictions on images that the model has not trained on before so these are the results that can be expected when you export your model and use it in your own applications and projects so these are pretty much the three most important graphs and and like images to just go over after you trained your own custom models where whether it's jol V 9 8 5 or like whatever models that you're using here we can see we get some really good detections pretty much spot on we also go do we can also go down and do validation of our costom model we just have our valid. Pine instead of train or detect but this is just going to do the validation that we have up at the top so here we can see the exact same results as we had in the end of our training script now we can do inference with our custom model again is just specifying the path to model that we have for now the one that we have just downloaded over from the directory and now we can use the detect that PES script I'm going to create other videos where we're going to do a custom class or we can just go in and swap it out with the ultra ltic ul8 model that I have as well I have all of that on my GitHub but we can do a class set it up and then we're going to show you how we can extract the results so it can be used in our own applications and projects but yeah this is everything that we need to do you can also just go and clean up this detect. P file and use it in your own applications and projects so here we should be good to go I just need to change this one and we should be able to run it here on our validation images and then after that we can go down and show the results on each individual image that will be kind of similar to this one here but these are results that we will get when we are running it in our own custom Python scripts later on so here we can see that the results are save to runs detect experience we specify down here at the bottom let's just show the first three examples and here we can just scroll through them we have our cup cup white cup hand painted cup Halloween cup we get some false predictions over here to the right but it can probably be filtered out here we can see that we're detecting cop with a confidence score of 11 so that can definitely be filtered out with a higher confidence score but the rest here looks fine cop Halloween cop hand painted cop y cop and our cocko cop so now we have a custom trained Yol v9 model everything that you need to do is take your data set use rlow to annotate it export it throw it into your Google collab notebook here everything will be down description and on my GitHub run through a couple of blocks of code and then you have your own Yol v9 model fine tune on a custom data set then we can export a model use it in only application and projects and create projects around that I'm going to show you that in upcoming videos but also do comparisons if you want to know more about like how we can use these models here how we can use them for tracking detection also how we can do research papers how we can Implement these architectures from scratch I have a bunch of courses on my website as well so definitely check those out because again we're doing op detection but we can also apply trackers on top of that and I go over all the Fig Behind The Trackers and so on different variations and how we can on them code everything is avilable in there so if you're interested definitely check it out and then I'll just see you guys in the next video Until Then happy training

Channel: Nicolai Nielsen

Views: 22,027

Keywords: object detection yolo, object detection python, opencv, opencv dnn, Yolov9, How to Train Yolov9, Yolov9 Custom Obejct Detection Model, Yolov9 Object Detection, Yolov9 Tutorial, Opencv Yolov9, Object Detection with Yolov9, Deploy Yolov9, How to run inference with Yolov9, How to train custom object detection model, Yolov9 the new state of the art model, How to train custom Yolov9 model, Yolov9 vs Yolov8, Yolov9 Setup and Train

Id: dccf_sJF0Gg

Length: 20min 7sec (1207 seconds)

Published: Mon Feb 26 2024