YOLO-NAS and StrongSORT | Object detection and tracking

hello everyone this is arohi and welcome to my channel so guys in my today's video I'll show you how to perform object tracking using YOLO Nas and strong sort algorithm so whenever we want to perform object tracking the first thing is to detect the object in the initial frame and then we have to track that object in all the next frames of the video so we need two algorithm first we will use the object detection algorithm to detect the object in the first frame so we are using YOLO Nas for object detection and then once we detect the object in the first frame then we want to track that object in the all the next frames of that video so for that tracking purpose we will use strong sort algorithm okay so um by the end of this video you will be able to you know get these kind of results okay so we are detecting a person class and then we are tracking the person class okay so let's start the coding part so for that here what you need to do is just clone this GitHub repo so guys this GitHub repo uh as for YOLO V5 so I have done changes in that repo okay so let me show you this repo so we will open this repo and inside this trackers folder we have various trackers we need a strong sort folder from here this folder will help us to perform the tracking using strong sort algorithm okay so we only need this particular folder from this GitHub repo okay so that's what I did clone the repo then just pick the strong sort folder from that repo okay and then okay now let me show you my code here in this folder I'm doing my work okay so this is the repo you will get once you clone it okay and inside it when you will open it inside the trackers there is a strong sort folder just copy this strong sort folder from here and then I have created this another folder in this folder I will perform all the code all the functionality which is related to you know Nas and strong sort okay so open this folder and you can see that we have that strong sort folder over here okay so inside that we have different folders and files so now you have the tracking code with you and this images folder is having a image on which I want to perform the testing and Test videos are the different videos on which I have performed the testing and this output folder at the outputs okay detected object and with the tracking numbers all these videos you'll get here and then the code which I want to show you is and this this is the pre-trained weight for tracking and this is the pre-trained weight for yolonas and YOLO Nas model which I'm using is that model was um trained on Coco data set okay so we are tracking a person class so Coco data set have portion class so that's why I'm using this uh pre-trained model to detect uh the object and we'll use this uh in our strong sort algorithm code to track the object okay track the person so here is my code this is the Jupiter notebook which I will open now and our code is in this okay let's open this to pattern notebook so these are the required modules so we entered in that folder the folder we have created for YOLO nascent strongs or task and then these are the required modules and these are the modules from Super gradient and these modules are required for our object detection task okay and these are the modules which we imported from the songs the strong cert folder these um these modules will help you to perform the uh tracking task okay and then guys this function okay this function is a function which will perform the prediction this this is basically handling the detection predictions okay so detection we are making with the help of eulonas so we'll provide the input to this function this function will then pre-process that input it will convert the input image into a torch tensor that torch tensor will become the input to the model and then your model will decode the output in the format okay so this function is performing all that task and there is one more function this function this function is just plotting a bounding boxes all right so now inside the device variable I have mentioned the device here I am calling the YOLO Nas large model you can call any model medium or small okay and that model was trained on Coco data set and Dot Cuda okay and then here is our pipeline we are creating a pipeline so we are calling a detection pipeline over here and what this detection pipeline is we have imported it above over here you can see okay so from here from here we have imported the detection pipeline okay this pipeline will perform the detection it will provide you the bounding boxes confidence and the labels of the objects which are detected from a video or on an image okay and now let me show you how to use this to see the output okay so that's what we are doing so this is the image on which I want to perform the detection okay and then here I'm reading that image through opencv and then we are passing that image and we are passing the pipeline this pipeline over here and then we are printing the prediction now you can see that we got bounding boxes these are the bounding boxes and then these are the different confidence these are the different confidence values and then these are the labels okay so this is this is how we use detection model you learn as correct now I will show you how to use YOLO Nas and strong sort on a video okay we'll perform the tracking on our video so in this variable I have mentioned the name of the pre-trained await file of strong sort and inside this tracker variable I'm initializing a strong stock strong sort tracking class and by providing the model weights device and the max age max age 70 means when um the when the number of frames are 70 after means after the 70 frames the frames will be removed okay so this is the max age and then now we see by these two lines we have imported the uh tracker strong sort a Tracker now on this video I want to perform the tracking so we are capturing the video by using a video capture and then if video is not there then you will get this error in these variables we are getting the height width of that in fps variable we are capturing the FPS and then these three lines will save the output video with the detected object and the tracking ID okay and from here we are starting the work so we are reading all the frames and on every frame we are calling the get prediction get prediction function remember this get prediction function is responsible for predicting the detection results it will perform the detection on the frame okay so on every frame using that detection pipeline we are uh performing the object detection task and then we are storing the labels in this labels variable like this and then we see you'll see over here that the labels are are we have the labels with the IDS but we need the corresponding names so for that I've done this step okay so where is it so I'm just mapping the IDS with the class names okay so with these three lines we can map it now guys uh once you've done the detection then we need to send that uh frame with the detections to the strong sort algorithm and strong sort algorithm accepts the data in a different format so uh over here you'll see that we get bounding boxes in this format X Y X Y that means X1 y1 X2 and Y2 but strong sort accepts the bounding boxes with x y w and H W and H means width in height so you have to convert your bounding boxes from X Y X Y to x y w h then only strong sort algorithm will accept your bounding boxes okay so that's what I'm doing here now so I have a blank list and then this is what we have from our detection model so with these few lines I am getting B box x y w and H and then I'm converting them into a array now we have the bounding boxes which strong sort will accept Okay so here you'll see that we are using the update function of a Tracker okay so this update function is present in let me show you where is this update function okay so this update function is in so this is my code inside this I have the strong sort folder where we have all the related files open it here you will see one file with the name of strong underscore sort open it inside this here inside this we have a strong sort class which we have called for tracking and inside this class we have a update function we are calling this update function okay so this update function accepts the data in the format of x y w and H so that's why we have converted our data into you know bounding boxes into this format okay so okay let's go back to our code Yahoo okay yeah tracker dot update we have provided the frame classes which we got and the confidence score and the bounding boxes which we have detected to the tracker then we are using a loop to get the tracking ID on all the next frames okay so that's what we have done and then yes guys after that this is important we are putting a rectangle on every frame and then we are writing the FPS also like you can write the FPS details we are appending the frame list and then finally we are storing the output video okay the output video will have the uh bounding boxes and uh class name and then the unique ID for tracking ID for every person okay now let's run this code and see what we get okay so you can see here so this is one of the video so every person have different ID in the class name and the object name okay so this is how it work now let's see the output video let's come here here and this is the output video let's open it so you can see that we have different person ID for that right for every person so this is how you can perform um object tracking using yolonas and strong sort so guys this code is available for level 2 members so if you want to join the channel there is a join button on my channel click on that and then click on contribution level 2 to join the level two members and then you can have access to this code okay I hope this video is helpful thank you for watching

Channel: Code With Aarohi

Views: 7,442

Keywords: yolo-nas, object tracking, object detection, strongsort, computervision, AI, objectdetection, objecttracking

Id: PjJu1HV8E4E

Length: 13min 31sec (811 seconds)

Published: Sat May 20 2023