Real-Time Object Tracking with YOLOv9 and DeepSORT Algorithm

in this video tutorial we will learn how we can track each of the detected object using deep sword object tracking algorithm we will be integrating YOLO v9 with deep sword object tracking algorithm so we will use YOLO v9 to detect objects in image video or in a live webcam F and then we will be using deep s object tracking algorithm to track each of the reeded object until that object remains in the frame of a video or in a live webcam feed so what we do in object tracking in object tracking we basically assign a unique ID to each of the reded object and then using that unique ID we track that object until that object remains in a uh in remains in the in the frame of a video or in a live webcam feied so using that unique ID that we assign to each of the detected object we then track the object we can also draw Trails follow the path check the path the object is following so that's all we are going to learn in this tutorial we will be integrating YOLO 9 with the S object tracking algorithm you all know YOLO 9 was released last month on 21st Fe 2024 uh which is the state of the odd object detection algorithm and y9 outperforms on the previous object detection algorithms as well while deep sort algorithm is the state of the art object tracking algorithm so by combining the state of the art object track detection algorithm with object tracking algorithm we will be doing object detection plus object tracking so that's what we are going to learn in this tutorial plus we will also see if the object gets blur or hidden we will also be tracking that object and that object we make ID will remain the same okay so let's get started so over here you can see in P Cham I have opened P Cham over here and I have python 3.1 Z install you can see over here plus I have just created a project over here by the name night deep sort OT so I created this project in my app directory so in the same way you can create this any project in any of your directory plus you can also use Visual Studio I'm using py Cham you can also use Visual Studio as well so in the first step uh we will go step by step so if you just see this node file I have written all the steps so which I will be B so first we need to clone the yellow n grab so I will just open the browser over here and I will just go over here and I will just write Yow nine GI up over here9 get up so it will just search for this will v9 get up over here and I will just open this post link from here okay so this is the yo9 official GitHub repo it has around 7,000 Stars so I will just um CL this complete uh repo into my uh py jum directly so I will just click copy this link from here then I will just go to the by charm over here and then I will just here and then I will just WR get CL and here I will just add the repo link so now I can clone this complete repo into my local directory over here and find this okay so it's done now if I just refresh this so we can find this complete y 9 repo over here so now in the step number two I will just start this uh yellow v9 repo which I have flown as my current directory so right CD over here so now this is being so now if I just open the requirements.txt file the requirements.txt file you can find I have they have listed all the required packages that are required uh to run the yo9 report successfully okay so okay so I will just uh as we will be integrating object tracking using deep sort algorithm so we need to add few more uh packages as well so which is you can add it and plus I will see if the numai is packages over here so okay so are we require n package less than 1.24 andse we will be getting an error so it's all this F do this and we also required G down package because we will be uh installing demo videos or deep sort files from the drive into directly into this P Chum uh ID okay so now in the step number two what is install all the required packages okay so we will be installing all the required packages over here well now to install all these required packages I will simply write over here P store minus our requirements. dxt so we don't need to install all these packages one by one if we just write requirements.txt pip install it will install all the packages that are listed in this requirements. DxD file and I will now click for enter so now all this packages will be get installed over here so this will take some time for all these packages to get installed so let's wait for this packages to get installed and then we will go ahead so now you can see that the packages are installed so now in the step number three what we need to do is we need to download a deep sort files so I will just copy this link from here just copy this from okay and I will just H paste this link over here and I just click on enter so this will download the Deep sort files from the drive into this Google C app notebook or into this ID so I've already placed this deep sort zip file into my drive and I'm directly downloading the Deep sort py zip file uh from my drive into this pyam ID uh so now we just need to unzip this Dort files so what I will do for this I will just uh click over here and create a new script. file uh in the script. file I will just write some code uh to zip uh unzip these de sort files so over here import I will just write over here import Z file with zip file do zip file here I will just write the name which is D sort by. zip we need to read this as Z Dash ref and we need to extract for this complete file so just do this and over here I will just run this file and let's see how does it works okay so do we have the okay so now now you can see I just U I completely unziped this file over here okay so there is a function in Python unip but that function does not work in pyam I have tried that function in Google cab it works fine there but that doesn't work in pyam so in the Deep sort pyot if you I have now unzipped it so you can see here we have the Deep sort folder where we have the uh complete de. file cod as well and we have the configuration and read me file over here as well so this is what we all require okay next what we need to do is we need to download a demo video from here so I already edit the link over here so okay so then we are just good as so now in the terminal I will just write add the link over here so this will download a demo video from a drive into this py Cham ID over here okay so now you can see we have the demo video over here as well so if I just okay so no need to so you can see here test. MP4 we have this demo video over here as well on which we will be doing detection and tracking okay so in next part I will do is um so in the detect d.y we have all the object detection code so I will just add another file detect DD tracking. in which we will be doing object detection plus tracking over here over here you can see that I have added detect dd- tracking. P file so here I will be uh adding the Deep sort code I have added some deep sort code over here which include initialization initialization of deep sort algorithm so over here what I have done for now is I have initialized Ive imported all the Deep sort required packages over here which you can see over here and then I have initialized deep sort over here I have initialized a dream sort tracker over here and all the parameters which you can find over here so the important parameter we should which you should need to know I have written all the comments about each of the parameter above you can read that as well so here is the maximum age parameter so you should need to know about that parameter so in the maximum age what we Define is that for example uh due to an illusion or if um for any in any case if an object tracking ID is lost for an object um so in the maximum age we Define how many frames it should be to get that uh ID back so uh for example if there is a car and we have lost that uh car object tracking ID due to an illusion so how many frames that it should wait for it to get back or or if it doesn't get back in the in much frames then it will assign a a new ID so for example if I set the maximum age value to 20 so it will be 20 frames uh to get the object tracking ID back and if in 20 frames it does not get the object tracking ID back for that specific object it will assign that object a new ID so if you can see over here if an object tracking ID is lost this parameter determines how many frames the tracker should wa wait before assigning a new ID so you can see all the details over I have written over here then over here I have defined the minimum cont Ence maximum distance nms maximum overlap maximum IU distance so you can read all those commands I Define every parameter over here this will take quite some time so I will skip this and here I just cre the fun initi light deep sort so I have just uh initialize that over here okay and here I uh because our YOLO v9 model has been trained on the C data set so the Coco data set consist of 80 different classes so I written all those 80 different classes names over here and then uh for example here I defined the color label so if we have the CL the if in a video frame or in a live webcam feed uh person is detected then we have assigned uh if the person is detected in a live webcam feed or in a video then the bounding boxes will have the following color so if the car is directed then the bounding boxes will have the following color if the motor bike is directed in a video frame image or in a live webcam field then the bonding boxes will have the following color and if the B bus is detected in a video frame or in a live webcam feed then the bonding boxes will have this color okay and over here U there we have loaded the model we have applied non Max expression over here so this is all what is going over here so from now where I will just add the object tracking code let me show you so from here I will start adding the object tracking code over here but before I add this so you can see over here so so in the YOLO v9 we get the output in this form like you can see that the YOLO v9 object detection algorithm Returns the output in this form we have the bounding box coordinates we have the uh top right uh corn uh top left corner bounding box coordinate you can see this is the top left corner and this is the bottom right corner so the y in the Yol the YOLO v9 model outputs return the top left corner bounding box coordinate and the bottom right corner bounding box coordinates so this is what 9 returns top left corner bonding box coordinates and the bot bottom right corner bonding box coordinates so we need to pass the yo9 object detection algorithm output to the Deep sort algorithm so the Deep sort object tracking algorithm accept uh the input in the form of width height of the bounding box so this is the width okay here I'm wrong this is the height so this is wrong if I just correct this support here okay so this is so let me just correct this so this is basically the height and this is the width of the bonding box so the Deep s object tracking algorithm accepts input in the form of height width of the bounding box and the center coordinates of the bounding box so these are the four parameters that are required that we should pass in the input of the deep s object draging algorithm is the center coordinates of the bonding boxes and the width and the height width and the height of the bounding boxes so from the yolow v9 object detection algorithm we get the uh top left corner top left corner bonding box coordinates and the bottom uh right corner bonding box coordinates so this is what we get from the Yul n output so we need to transform that output into the height and width of the bounding boxes and the center coordinates of the bounding boxes so first we need to do this we need to transform the y9 output into this height width and the center coordinates of the B so over here I will start writing the code so from um yellowing N9 we get that uh top left corner bonding box for and the bottom right corner bounding box coordinates which we are here plus we also get the confidence score and the class names so these are the output we get from the yolow N so here if you just write so X1 y1 represents the top left corner and X2 Y2 represent the bottom right corner b box coordinates so the output in the form of tensors we need to convert the output into integers first so I will just write over here in X1 in y1 in X2 and in Y2 so we need have transformed the output from tensors into integers so now we will find the center coordinates for each of the detected object so as I told you that in deep s algorithm uh we need to pass the input in the form of Center coordinates for each of the bounding box of the detected object and the width and height of the bounding boxes okay so we will find the center coordinates first and then we'll find the width and the height of the bounding boxes as well so I will just write over here so here I will just find the width height of the B so to find the width uh so you can say that basically we here we have X1 okay so I just try have X1 and here we have uh X2 so if we subtract X1 from X2 we get the width and if we subtract y1 from Y2 we get the height so I will just I will just write over here X1 - x this will give me the width and for the height I will just write over here so the Deep sort algorithm accept in input in the form of Center coordinates and width and the height of the bounding box so we write over here s okay so now we need to append all the is WID and the how because in a frame there will be multiple objects that will be detected and so we have Center coordinates for their multiple objects and width and height or values for multiple objects that I detected in a video frame in image oring live webcam feed so I will just here append do append okay then then I will also pend all the confidence values over here as well plus we have also have these class uh IDs as well for example the class ID for the person class is zero so but these values in the form of tensors so we need to transform those values from tensors into integers as well and I will append these values over here and uh okay so that's done then we have so now we need to convert these values into tensors again and as I told you that a deep sort algorithm accept uh input in the form of uh deep sort algorithm object tracking algorithm accepts into input in the form of uh Center coordinates and width and height of the bonding boxes and along with the confidence score and the class names uh confence store and the class IDs we don't have the class names yet so so as we have transformed our output in the form of tensors and have confidence for over here as well so here we have the class IDs and is IMS represents the uh current frame in which we will be doing this so if we have detections like if we have some I detections or then we will be doing the tracking as well so so now you can see over here uh in the Deep sort object tracking algorithm out input we have passed the center coordinates width and height of the bounding boxes and the confidence score and the class IDs and the current frame so the what output we get from the deep sort algorithm so here in the Deep sort algorithm so if I just write here deep sort algorithm Al so so this is what we get if I just again Zoom it up further Okay so here we go so in the Deep sort algorithm output we get uh uh the bounding top right uh uh now this is a top left corner bounding box coordinate and uh the bottom right uh bounding box coordinat so de s algorithm output Returns the top left corner bounding box coordinate and the bottom right corner bounding box coordinates which you can find over here so we get the bounding box coordinates in the Deep sort algorithm output a unique ID for each of the detected object plus the Deep sort algorithm also return the class ID for example for the person cloud is zero so there are three things that we get from the deep sort algorithm output one is the bounding box coordinates the second thing is the unique ID for each of the directory object this is what we do in object tracking in object tracking we assign a unique ID to each of the detected object and we also get the class ID for each of the class for example for person class the class ID will be zero so in the bounding box coordinate we get the top left corner bounding box coordinates and the bottom right corner boting box coordinates okay so first thing we I will write over here first we get the bonding box coordinate we get the top left corner bounding box coordinates and the Bing bottom right corner bounding box coordinates let we get the IDS for Unique identities for each of the detected object plus we also get the object ID for example for the person class the object ID is zero and for the car class it's one or two okay so now uh we need to draw those bonding boxes around each of the DED object and we need to add the ID uh for this as well so I will be creating a function by the name draw Dash boxes and in the draw Das boxes I will be writing the code so that I can draw bounding boxes around each of the detected object assign a unique ID for each of the object and write the class name above each of theed object so so in the input I will passing the bounding box coordinates the then I will be uh drawing the trails so to see what path the object is following so to track the path then I will passing a unique ID for each of object in the input and uh object ID which is the class ID as well so object ID is basically the class ID like for the person glar zero so these will be all the inputs so let's write the draw Dash boxes function okay so we can just narrow around this so here I will be creating a writing down the code for the function draw Dash boxes the input will be the frame the bounding box coordinates do real so that we can track the path the identities so we can easily set this value To None over here then we we have the categories as well so we can set this as none as well so categories are basically the class IDs then we have the offset we can set this to zero so let's write down the code complete code over here so let's find out the height and width of the current frame okay then I will write okay for I comma box so now we will look through each of the bounding box coordinates okay now I'm looping through each of the bounding box coordinates over here so um in the bonding B coordinates let's add some offset value over here okay so here we have to find the offset value over here so so now we are looping through each of all the uh bounding boxes and here we are just getting the coordinates values for each of the bounding boxes so we are looking through each of the bounding box coordinate one by one so now we can find the center point of the bounding box so find the center point so here we are just in the center point of the Bing BLX then over here we are calculating the class IDs for each of the directed object then uh using the class IDs for example if the there is a person then we will have the bonding boxes will have this color so we will simply write over here and here we will pass the class ID then for each of the detected object we will have a unique ID for each of DED object so that we can track that object using that ID so uh in the identities we have the unque ID for each of the reected object so so now we will just uh draw a rectangle around each of the directed object or you can simply say we are drawing bounding boxes around each of the detected objects and the thickness of the bonding boxes will be two so now what we will doing is that we will be using that ID so the if the ID is zero then the there will be person class if the ID is one then bicycle if the ID 2 is then it belongs to the card class so so in the category we have the ID for each of the class and using that ID we will find the class name so in the label what we will display is that uh first I will display the ID then we have the class name over there as well so two things will be have in our uh uh above the bounding boxes where we will display the ID unique ID and then we have the class name we will not be displaying the confidence score over there so now we'll finding the text size so we want to keep the font as 0.5 so now we will be creating a rectangle above the bounding boxes as well and we want the rectangle to be filled so we'll just write minus one and then we will be adding the text in the rectangle which we have created above the bonding box in the text we will be displaying the label which contains the class name and the uh with ID unique ID of that object just setting the location of that label and the color will be white for the text and the thickness of the text will be one so now we will be creating a circle at the center of each of the object okay so now we have done this Cod plus we also want to draw the trails over here as well so we can track the path the object is following so I will just write down the code for this over here as well okay if iing not in data DQ so data DQ is a type of dictionary that we have initialized above and it has a maximum length of 64 okay so now we'll just write over here if draw Trails so the user wants to draw the trails for I in range is none oh now we will just calculate the thickness over here so we need to initialize the numpy package at the start over here as well so I will just write import numy as NP okay okay so we have already imported this so no need to I spot that not imported so we need to multiply this by 05 over here or we can make it 1.5 next we need to draw the trails so to draw the trails we will using C2 do line so that we can RW rails the input will be frame one and here we have data DQ ID one then we have the color and the thickness as our input as well and so that's quite fine for now I think I have written all this so let me execute this script file and show you how our output looks like before I run this I forgot that I have not downloaded uh the model vids so I will just go over here and just click over here and the model VS will start downloading so if I just go over here let me just close this [Music] files so I should have over here the model with save so if I just check over here okay uh so let's wait for this model W to get download so it will take few more seconds for this model WS to get download okay so they are about to complete okay so that's done okay so let me just add over [Music] here so I will just add the over here now you can see it so let me just run this uh now in the py charm so let me just run the script and show you how our output looks like so now to run this up over here uh we just need to write this python detect D du d tracking. by here I will just Define the vs file uh which we have downloaded and that's my input video and I'm just running it on CPU because I don't have GPU in my local computer so CPU and dash view image so that uh it can show me the output image or output in the it will open a window where I can see the output video over here as well and my output will be saved into this uh folder as well so that's on this up okay no such file or okay so I think mistake is on my side let so we need to set the Y for as current directory so now let's run this one okay so this will take few seconds before it start executing over here so here we have the model summary gradient zero parameters the parameters are in millions okay so you can see in the output currently there are no deductions but uh let's see as we go ahead in the frames do we have any direction or not okay so now you can see here we have the detections and you can see over here a unique ID is being assigned to each of the person look so here we have the person like you can see over here the 19 is for Unique ID is for this person two unique ID for this person 28 unique IDs for this person which disappears five unique IDs for this person 17 unique IDs for this person 15 unique ID is for this person and here we have deducted the handbag so the 30 unique ID is for this person the 14 unique ID is for this person 12 unique ID is for this person 11 unique ID is for this person nine unique ID is for this person uh the eight unique ID is for this person 16 unique ID is for this person 21 unique IDs for this person so each of the person is being assigned a unique ID over here okay so if you want to draw the trails and plus you can see we have detected a handbag and we have assigned id30 so as well so if you want to draw the trails uh you can just write over here Dash Das draw Dash trails and if you want to select any specific class like if you don't want to detect handbags in the video you can just write classes zero so zero is for the person class so if you want to know more about the class so you can see Zero is for the person class one is for the bicycle class two is for the car class three is for the motorbike class so here we I have listed 80 different classes so it start from zero and end at 79 okay so I uh here in the classes you will assign the ID for which specific class you want to detect so if you want to detect all the classes you can skip the classes over here but if you want to detect specific class you can just pass out the ID of that specific class or multiple classes you can also create a list and pass the ID of the multiple classes that you want to so now it will not be detecting handbags or any other things it will only be detecting the person class because we have passed the ID classes at zero and we have draw that fast draw Trails so it will be drawing the trails over here here as well so let's see uh currently there are new no detections after few frames initial frames we start to see detections usually okay so now you can see over here we have the reductions uh okay okay so now you can see the red Center Center as well so these are the Trails which will not be drawn so now you can see as the persons go ahead the trails increase as well so you can see over here it's a bit slow but now you can see that Trails is also increasing so these are we just tracking the path that the person is following you can also do is for other classes like car you can track the path the car is uh going on and so on so now you can see I am drawing the trails over here as well we have assigned a unique ID using object tracking algorithm we are also tracking the path the person is following over here as well okay so this is all we have done in this code we have using object tracking algorithm we are tracking each of the DED object and we are drawing the trails over here so using Trails we are tracking the path like you can see so this Trails length will increase as the person goes ahead you can see over here these are the trails that we have drawn and this Trails length will increase as the person goes ahead further as well okay so now you can see over here I have run the script in on the Google callab free GPU as well so so that I can use the GPU and show you how fast it works so I have the demo video up over here with me as well so let me just play that demo video so I have run this script on GPU so you can see the processing comes very fast and we have it offers good uh frame rate at g on GPU so you can see we have a good number of detections as well and it's the speed is very good as well so let me show you the output video over here so now you can see over here we are able to detect the person we have assigned a unique ID to each of the person as well plus I'm not drawing the trails you can draw the trails if you want as well so that you can track the path the person is following or any other object so in this tutorial we have learned how we can track uh each ofed object uh by assigning unique ID to each of the object detected object so in this tutorial we have integrated yon v9 with deep sort object tracking algorithm and we have um direct and deduct each of the object as well and we have assigned a unique ID to each of the object plus we have also uh drawn trails to track the path the object is fing so that's all from this tutorial thank you for watching

Channel: Muhammad Moin

Views: 4,040

Keywords: yolov9, object detection, object tracking, deepsort, object tracking with deepsort, yolov9 with deepsort, yolov9 with deepsort object tracking, yolov9 plus deepsort, yolov9 plus object tracking, yolov9 and object tracking, object tracking with opencv, object tracking using deepsort, real time object tracking with yolo, real time object tracking with yolov9 and deepsort, real time object tracking, real time object detection and tracking

Id: Jx6oLBfDxRo

Length: 47min 1sec (2821 seconds)

Published: Fri Mar 22 2024