How to Use MMDetection | Train RTMDet on a Custom Dataset

let's talk about mm detection one of the most popular computer vision libraries with over 25 000 stars on GitHub yet I bet many of you didn't use it or even heard about it today I'll show you how to install mm detection in your python environment which is not so straightforward as you will see how to run the inference with pre-trained Coca model and last but not least how to train the model on custom data set and Benchmark it if you don't know mm detection is part of openmm lab a set of over 30 Vision libraries covering topics from detection and segmentation up to stuff like action recognition and human pose estimation mm detection is by far the largest library in that ecosystem and it allows you to train and deploy dozens of popular models like DTR yellow X and RTM debt that which by the way is the model that we are going to train today RTM debts speed and accuracy is comparable to other state-of-the-art real-time object detectors but as part of mm detection it is distributed under Apache to license this makes it a very interesting choice for anybody looking for model that can be used in Enterprise or closed-sourced product to make our life easier I prepared a Google collab that we are going to use to train the model the link to the notebook is in the description below but you can also find it in roboflow notebooks repository along with other demos and examples now let's scroll just a little bit lower into the model tutorial section and for now the first link from the top is RTM dead object detection let's click the open in column button that should take us to Google collab website and we are good to go in the top section you can read more about RTM debt model take a look at the Benchmark result comparing its speed and accuracy with other top real-time detectors like yellow V6 V7 and V8 we also linked the RTM that paper and our blog post covering the model so you can read and learn more about models architecture if you want okay enough of the talking let's Dive In the first thing that we are going to do is to run Nvidia SMI command just to confirm that we have access to GPU after a few seconds you should see the output similar to the one visible on my screen it shows you the exact GPU model and the Cuda version you have if you don't see the same result that probably means that your environment is not GPU accelerated in that case try to follow instructions in the notebook change the runtime type and you should be good to go now that we confirm that we have access to GPU we can install all required packages we will need pytharch along with four libraries from openm Lab ecosystem mmcv mm engine mm detection and mm YOLO because I'm using Google collab I already have pytorch installed in my runtime if you try to set up project locally make sure to visit pytor.org and pick the right installation command all you need to do is provide information about your operating system the hardware you use and the python version you want to install and the command will be generated for you as for mm detection and the rest of openm labs libraries the instructions you see on the screen are coming from mm yellow installation guide interestingly they recommended way of installing open mm lab packages is not through peep but through their own packet manager called mem and if you take a look into commands running in the notebook we can see that the first thing we do is install openmem from pip and then we use mem to install the rest of required packages I know it's weird this is probably the first time I'm using proprietary package manager to set up my python coding environment before we start to play with RTM that we have one more step to go we need to install a roboflow and supervision packages roboflow will allow us to download the Open Source data set from roboflow Universe we will use that data set to train the model later in the video supervision will give us a bunch of useful computer vision utilities that will help us to filter and annotate detections process data sets and Benchmark the model one of the best ways to confirm that the installation process was successful is to run pre-trained model on example image to load the model with mm detection we need two things config file and the weights file config file is model recipe but it goes Way Beyond Network architecture we use it to Define training and validation data sets data processing pipelines hyper parameters values or even loggers all of those properties are defined in reusable way so you can simply copy and paste them between configs I will tell you more about configs and how to customize them in a second but for now let's just focus on working with pre-made configs config and weights files for all pre-trained models supported by mm detection are listed on GitHub in tables like this separate table for every model architecture RTM debt comes in different sizes ranging from Tiny to extra large in this tutorial I'll be using rtm.l I looked up the weights address for this specific model and I'm using wget to download them to my hard drive the config file is already there as it was part of mm YOLO repository that we cloned a few minutes ago now you can use an example image and run inference in my case I'll be using photo of me and my dog the model is pre-trained on Coco data set so it should be able to pick up person dock car and a backpack in the image so let's try it out we load the image and run the inference the first time we do it the actual model is being loaded into the memory that might take a little bit of time but once the loading process is completed we can run the inference and examine the results we will visualize them in a second but first let's print out the results object we see that it contains prediction instances with labels boxes and scores fields it's not the most complicated structure to navigate but to simplify our post processing even more we will use supervision we can simply use the from mm detection method to ingest the inference result and with three more lines of code draw the detections on the source image and display it interestingly instead of just few bounding boxes we have hundreds of them looks like mm detection is returning all results without any post processing let's use supervision to solve that problem first we add confidence filtering using 0.3 as the confidence threshold and additional non-max suppression the rest of processing stays the same and when we annotate the image with our new results we are only left with valuable detections awesome of course to train custom model you need data in principle mm detection expects your data set to be in Coco data set however it requires your category and image IDs to be numbered starting from 1 and not zero most Frameworks don't care but for mm detection is kind of big problem trust me I spent hours trying to debug it to save your time we decided to add additional export format to roboflow very similar to Coco but fully compatible with mm detection requirements in this tutorial we will use Domino's data set I found on roboflow Universe I included the link to the data set in the description below now let's open the data set card and click download data set select Coco mm detection format click continue and after few seconds we get a code snippet that we can copy and paste into collab when we do that and run the cell we will see a prompt with link that we can use to authenticate we click the link to open roboflow website now we can generate the token copy go back to collab and paste it once again run the cell and the data set is being downloaded depending on the size of your data set it might take few seconds or even minutes to complete once it's done we can open Google collab file explorer navigate to mm YOLO data directory and inside we can find the data set we just downloaded split into train test and validation subsets I'll be honest crafting a custom config file was probably the hardest part of this project in theory it should be very easy pick the model you want to use modify its config and you should be good to go in practice it took me four hours to figure it out luckily for you you can just use my config and have it done much faster as I said before in this tutorial we are using rtm.l as our based model I found its original configuration file in the mm YOLO repository and copied it into Google collab then used python string form meeting to inject our variables we will add information about the data set location the classes we want to detect and some training hyper parameters that we want to customize I also configured tensorboard logger as I want to examine metrics after the training is completed now you can save the updated config file on the hard drive it will be stored in mmiolog configs RTM debt directory as custom Pi once again we can use Google call up file explorer to navigate to that catalog and we can even open the file to make sure that it is filled correctly looks like all our variable names were replaced with corresponding values once you have the data set in correct format and the configuration file properly crafted the actual model training is very simple all you have to do is run the train Pi script and wait for it to finish my data set is quite small so it shouldn't take more than an hour to train the whole model still let's use the magic of Cinema to speed up the process now that the training is completed we can examine training metrics as I mentioned before we configured mm detection to use tensorboard logging format let's visualize the data and take a look first we need to pass the path to the training log directory we should be able to find it in mm YOLO work dears there is only one subdirectory custom the name corresponds to the name of our configuration file now we can pass it as log dear argument value and run tensorboard after few seconds tensorboard will start let's close less interesting charts and focus on validation set loss loss box and loss class we can see that all three were converging there is no sign of overfitting here as a matter of fact I'd say we could keep the model training for a bit longer and squeeze a bit more accuracy from it okay let's Benchmark the model to understand exactly how accurate it is when we run computer metrics Benchmark we see that the model is performing well most of the detections are on Matrix diagonal that means that they got classified as true positives correct detections only a few of them are in the bottom part of the image classified as false positives and few more have the correct bounding box but with incorrect class before we wrap it up let's quickly calculate the overall map and per class map in both cases we got around 0.7 most classes score similar except peep3 where the score is around 0.4 still not bad but significantly lower than with other classes we'd probably need to dig deeper into the data set to understand the reasons for it maybe the class is simply underrepresented overall there is a high chance that the model will perform well on production as long as it will receive images coming from similar distribution so a similar setting lightning and camera angle that's all for today I hope this video will inspire you to use mm detection and hopefully you will learn from my mistakes watch out for dataset format if you use roboflow make sure to export your data set in Coco mm detection and not the regular Coco format if you use your own data set make sure your category and image IDs are numbered from one and not from zero if you plan to train RTM debt you should be good to go with my configuration file if you want to train other supported models start with pre-made config and make sure to include information about training set and Target classes that was the biggest problem that took me the most time to figure out if you enjoyed the video make sure to like And subscribe and stay tuned for more computer vision content coming to this channel soon my name is Peter and I see you next time bye

Channel: Roboflow

Views: 7,230

Keywords: MMDetection, RTMDet, Python, Computer Vision, Object Detection, OpenMMLab, PyTorch, MMYOLO, Supervision, COCO, Roboflow, Model Benchmarking, TensorBoard, Custom Dataset

Id: 5kgWyo6Sg4E

Length: 15min 27sec (927 seconds)

Published: Thu Aug 24 2023