Real Time Object Tracking using YOLOv8 on Custom Dataset: Vehicles Counting (Entering and Leaving)

hello everyone in this video tutorial we will see how we can integrate deep sort object tracking with yolo V8 we will train our yellow V8 model on a custom data set of vehicles then we will track each of the vehicle using deep sort object tracking and also we will do the vehicle counting as well like the total count which include total counts and the sub count of each vehicle like car stroke and buses and Pickups so let's get started so here I have already created a notebook script A olap notebook script and I have implemented this whole project already as well so that we just to make sure that this video is short and you should not be very prolonged if I start incrementing every step over here so this video can get very long so that's why I've already implemented this whole project so first of all you need to make sure that you have selected the runtime as GPU so as you can see that Hardware accelerator is being set as GPU so that's fine if it is none it means that it is using CPU and on CPU when we when you train your model or do the prediction on any video the processing is very slow so please make sure that you have selected the hardware accelerator as GPU then click on save so then we will import the required Library so we only need this library in this project from IPython dot display import image we need this Library if because if we want to show the output in the Google Input image or output image in the Google app notebook we need to have the ipython.d display import image library to display the output image or input image in the Google colab notebook so just run this cell so next I will for this project I will be using my deep sort object tracking GitHub wrapper so let me show you this repo first then we can further move ahead so this is the object tracking wrapper yoloba deep sort object tracking repo I will be using this in this project and in order to assess this notebook file you can get this notebook file from here if you just click over here open in new tab this notebook file will be open in front of you so okay so here we have the yellow via deep sort object tracking repo so this will be used in this project so first of all we will clone this repo into our Google home app notebook so it might take few seconds next we shall check the present working directory so present working directory is set as this folder so we need to set the present working directory as this the Clone folder which we have cloned object tracking wrapper so we want to set our current directory as this folder so just copy path over here and paste it over here so now your clan directory is being set as this folder so just run this cell next we need to install all the dependencies so dependency means all the requirements or the required libraries that are required to run this project so as you can see that I have already run this cell so we need mat for matplotally primary numpy Library opencv python Library so when we install the dependencies it will install all the required libraries required to run this project so if you skip this step then when you run the training or the prediction or the validation script you will face the error that following library is not installed then instead of installing each Library separate leads better to install the dependencies which will automatically install all the required libraries required to run this project so just run this cell now so uh it might take few seconds so let's wait and please bear with me so in few seconds this will be completed okay okay so it's about to complete now okay so it might take few more seconds okay so all the required libraries packages have been installed so as we are performing the detection so we will go to ultralytics YOLO V8 detect so we will set our current directory as this folder because this contractor you can see that contains the prediction trailing and validation script so we will be using all these three script because we will be training our model on the custom data set we will be validating our model and we will be doing the prediction on some demo test videos and some test images so let's set this folder as our current directory and just run this cell so now we are downloading the data set from roboflow Universe the basically we see the drone images of the vehicles the data set consists of drone images of beakers which include cars pickups Lorry and trucks and buses okay so just downloading the status foreign so the data side is size is quite big so it might take some time so it's completed now let me show you where we have the data set so here we have the data set you can see over here so let me show you some demo in images as well so it might take few seconds to load instead of this if I open the validation it contains few images so it will load quickly so just clicking over here and just click on this image so here we have like you can see that we have the drone images of cars and in the label folder we have the labels for each of the image so data set is quite big so okay so here we have a labels folder as well for in for each image we have the labels as well like you can see this demo image as well we have the Drone image of these two cars as well in the drone images we can see these two cars and then we have the labels folder which contains the label for each of the image like this represent the class to represent the class which is of car and these are the X1 y1 coordinates like the top left corner of the bonding box and these are the X2 Y2 coordinates like the bottom right corner of the bonding box okay so next just close this and let me explain you so now we have downloaded a data set now we need as we are implementing object tracking using deep sort so we will download the Deep sort files from the Google Drive so just download the Deep sort files from because I have placed these deep sort files on the Google Drive I am directly downloading these files from the Google drive into the notebook and now I'm unzipping these deep sort files okay so in the next app I will train the train I will do the training of the custom model okay so here I've because the training takes so much time here I have already trained my model so I will not train the model again I have already trained the model for 50 box you can see here so the model is already trained for 50 bucks and we are getting a mean average Precision with IOU 50 as 0.766 and when the IOU varies from 50 to 95 we are getting a mean average Precision of 0.417 and here we are the mean average Precision with respect to each class we have a board class campaign car car motorcycle pickup plane tractor truck van so we have around 12 to 13 different classes and here we have the mean average Precision 50 and mean average Precision when IOU varies from 50 to 95 so we can see that the model gives on the results are very fine okay so these are all the files which we have in the training folder and here is the confusion Matrix so confusion Matrix basically tells us how our model handles different classes so from here we can see that in case of campaign car for example here we have the camping card and let me just uh zooming out a bit okay so you can see that here we have the camping card so 67 percent of the time model was able to successfully detect that this is the camping car while ten percent of the time when there is a camping car model was detecting the camping car model was not detecting the camping car instead it was detecting it as a car okay so 67 percent of time model was able to detect successfully that this is a camping car while 10 percent of the time model mistakenly detect camping car as a simple car okay so model did not detect the camping car it detected as as a car although it was a camping car okay while 24 percent of the times model failed to detect like when there was a camping car model failed to detect anything model it was a model did not detect anything okay while so 2400 percent of times model was not able to detect anything while 10 percent of the time when there was a camping car model detected as a simple car while 67 percent of the time model detected successfully that this is a camping car so in this way you can look for the other classes as well so while these are the training and validation law so you can see that just so you can see that our loss are decreasing continuously while the mean average Precision is continuously increasing you can see that here it's also increasing so here are the model predictions on a validation batch so validation data set is not using the training so it is always better to take a look and see what results do we get so model was able to detect successfully the pickup cars we can see that results are very good so I have saved the model weights onto my Google Drive so I will directly download those grades from the Google drive into this Google app notebook okay so now we are validating our custom model on the test data set and here are the result of validating the model on the custom that on the test data set validating the custom models on the test data set and the results look fine with respect to mean average Precision is also good and the mean average Precision when IOU varies from 50 to 95 is also good now we will test our model on a demo video so just downloading a demo video from the Google drive over here and let me test our model so we have integrated object tracking uh here so in this case when we were able the model detects the car or a bus or a truck it will draw the trails as well in the next step we will do the vehicle counting okay like the number of vehicles entering and number of vehicles leaving so the next part I will be doing the vehicle counting so let's first uh check this output demo then I will be integrating the vehicle supporting into this script so it might take few seconds so let's wait and see what results do we get so we have done the predictions on the demo video Let's display the demo video was here and see what results do actually we get so just running the script as well and now we will be able to see the demo output video over here in few seconds so let's see what results are do we actually get so and let's wait for few seconds and we will have the demo output video over here so we can see so basically the we do the predictions frame by frame so here we have total 8038 frames so we are doing the predictions frame by frame and then we will save all the predictions in a output demo output video in the folder and now we are just displaying this output video over here okay so basically you can here see as well the model was able to detect 15 Cars one pickup in each frame the model does the detections and then all this Frame are saved as a video so you can see that let me just download it and show you so let me just lay it out okay just let me just open it in front of you so you can see here this is our output demo video you can see the model was able to detect the cars the truck as well and here the model is detecting the pickup but it's a truck here the model is detecting correctly the truck and it is also correctly detecting correctly that there is a truck as well okay so the model detections are quite fine this is a pickup as well we can see here and so the model works fine like we can see that model is giving some good results and here we have the object ID as well and here you can see that each object is assigned a unique ID so the here we are also implementing tracking basically deep sort applied assign each object a unique ID so this is the function of the Jeep chart to assign each detected object a unique ID so we can track it out and here we have the prayers we show the path of the objective solving so here you can see these are the trails these are trails in lines you can see that these are trailers which show the path the object is following so now I will increment the weaker counting in over here so I will open the predict.pi file okay so I will explain this once they provide here so just watch and see so first of all what I will do is I will Define two dictionaries over here so I have already written the script so I will not just add copy and paste over here so if I start doing the coding over here so this video will get prolonged so I will explain the step-by-step implementation over here okay so here I I have basically added two dictionaries object counter one and object counter so okay so basically in object counter we will save the number of vehicles entering and in object counter one we will save the number of vehicles leaving so what I will do is let me open the print section just give me a minute let me open this okay okay just a minute I'm just opening it I have opened the paint for it does output sample image from the output video into the paint folder so let me explain you the whole process what we will do first of all we will create a line over here okay that's not the straight line just to show you okay so first of all we will create a line over here so when the this Vehicles passes this line okay so we can say that these are the number of vehicles entering and when the vehicle from here just give me a minute okay so when the vehicle from here leaves this line like when the this speaker passes this line we can say that this speaker is leaving and the vehicle when this truck for example passes this line we can say this these vehicles are entering okay so the weaker going over here are the weaker so on the beakers going over here are the let me take some bright color so the vehicles going in this direction are the vehicles which are entering okay and the vehicles which are going towards this direction which are the vehicles which are leaving okay so when this the trails so you can see that these are the trails which we have drawn these are the trails and these are the ultimate Trails the uh Pink lines which you can see when these lines intersect with this this line which we have created we will have the counter that the vehicles or number of car is leaving this car is leaving and we will the counter will incrementing like fourth also leave five class at least six car have left so whenever these Trails intersect with this line the counters increases like the number of cars have leave the number of bicycles have leave the number of trucks have leaked and when these from the beakers from this side intersect this line like these Trails intersect with these lines we will have an accounter increment that number five vehicle cars have entered six bicycles have entered six stocks have the enters okay so the when these vehicle passes through this line we have an incremented number of because entering and when these vehicles passes through this line we have the increment that number of vehicles leaving okay so let me just get get back to the code okay so here you can see the code as well okay so you can see that we have created two dictionary object counter one and object counter so object counter will store the number of vehicles entering and object counter with one will store the number of vehicles leaving okay and in up count up and countdown in count up we will store the number of total number of vehicles entering and in countdown we will store the total number of vehicles leaving okay and in object counter we will store the like number of cars leaving the number of number of cars entering the number of trucks entering and number of uh number of pickup centering so we will store the total weaker sub counts like the total Car counts the total uh pickups count and the total basically Lori's counts okay so in object counter in this dictionary we will store the vehicle sub count in the count one and count do we will store the total Vehicles entering and leaving count okay so here we have initial tracker and I will not explain this over here so in now we will what we will do is we will basically I have created two dictionaries and count up and countdown which I have defined now in the next step I will write up Define the function for the intersection okay so now I will be defining the functions for the intersection over here so these are the two functions which we use for the intersection let me explain you this as well so I will just go over here and just open this file so you can see over here when these basically Trails when these Trails intersect with these lines when these Trails intersect with this line we will have an increment okay we will have an increment like the number of cars entering similarly when these trains intersect with this line we will see them incremented number of vehicles with cars leaving okay so how we can write the formula in which we can do all this so if we I have just written let me just secure this so now you can see over here so when these two lines intersect how we can show this we can show this in Python using these two functions if you show this in Python we can use these two functions like these are the coordinates for the first line the X1 y1 the X2 Y2 and these are the chord segments or the coordinates for this line and these are the coordinates for the first line to show this in Python we use this function okay so in the Google app notebook so you can see that I have added these two functions over here to show the intersection of the lines okay so now what the next step I will do is I will add the direction function Okay so now I will add the direction function over here so you can see that this is a direction function so in this project we are only interesting the North and South Direction so how we are getting this Direction let me show this as to you as well over here and just remove this from here currently okay so let uh how we are getting direction let me explain this to you as well I know that's becoming a quite tricky but okay so the vehicles passing through this direction uh this is a not traction and the vehicles passing in this direction this is a subtraction okay so we have seen written that from the trails from the weaker Trails we have seen that if 0.1 if the trail number one is greater than that trail true then it is the South Direction and the trail number one is less than the 12 2 then this is the not Direction so here we are calculating the direction so after calculating the direction in The Next Step what we will do is we will uh now add the now we will create a line and just finalize our script of the count down and count up so just let's do it so I will just go over here and just add this so if we have a okay so we have not created a line so first of all what we need to do is we need to Define this line so we need to Define this line so I have not defined this line first we need to create this line so I will go back to my collab notebook and add this line so let me add this line so I will go at the start and just add this line over here okay so now I have added the line select so when the trails and this line intersect when these trays and the line intersect then we will have an increment so if the object name for example if car name is not already the object detected then it is will be one and in the count up we will have account number increase that there we have a vehicle for example car has been detected so count will increase plus if the object is already there in the list so we will do the plus one we will further increment but if there is no object in the list then there will be that Define that this object has appeared over here similarly if it will do the for the South direction as well you can see that uh not direction is beyond the because our entering at the South direction is when the vehicles are leaving okay so here in when the vehicles leave we will do the down count that the number of vehicles leaving so in this way we have just implemented this project Plus now let's now just I want to change the center coordinates I think it would be better if we change the center coordinate so what I should choose in the center coordinates is I think the value which I need to choose is X1 plus X1 I think X1 plus X1 and Y 2 plus y1 will work so basically I'm just choosing the center coordinates of the bounding box you can remember okay so plus I have defined here count up countdown I need to declare them as a global variable so just right over here Global counter global countdown and just run this now we have done this so we just need to run this the predict.by file and see what results do we get so just close this out and just go over here and now just run this script because it might take few seconds but it will be executing Okay so now we are just running the script so it might take some time for the video to process so as the video is completely processed I will be back and then I will show you the output results as well while running the script I just realized that I have missed one thing I have not defined the place where I want to the count to appear in the UI that in the top right corner where should the in the top left corner where should the count appear in the UI that so for this I will Define the count over here so just adding the count okay so I miss this so I'm just defining now so this will appear in the top right corner the vehicles entering and basically here we are just setting the UI and this will appear the vehicle leaving will appear in the top left corner lock left corner found in the top left corner so this count will appear in the top left corner while this count will appear in the top top right corner so just again let's again run the script I just saved it now just run again the script so it will work fine now okay so just give me a minute so just run this and now hopefully it will work very fine now so it might take some time to execute okay so I will pause the video and as it completes the execution I will be back so our script has run and I have just displayed the output demo video as here as well so just download it and let me show you the results so you can see that we have that Vehicles let me just switch the screen okay so you can see over here we have the number of vehicles entering the number of vehicles leaving we have cars truck and the number of vehicles passing entering over here so the model works perfectly fine we are able to count the number of vehicles entering the number of vehicles living okay so here in this video you can see that we have implemented uh we have trained yellow V8 model on the custom data set then we implemented the object ranking you can see that we have the unique ID with each of the object and we have also have the trails over here these are the trails okay in the next step then we count the number of vehicles entering so the beakers moving in the north direction or this direction are the vehicles entering and the vehicles moving in the South direction or distraction are the vehicles review so after calculating the number of vehicles entering and the number of vehicles in leaving we displayed the count over here you can see over here number of vehicles entering a number of vehicles leaving so in this video tutorial we have trained our yellow V8 model on the drawn images of the vehicles data set the data should consists of drone images of vehicles which include cars truck trolley cam card bicycles motorbikes okay so I hope you have learned a lot from this video see you all in the next video tutorial till then bye bye

Channel: Muhammad Moin

Views: 5,853

Keywords: yolov8, yolo, objectdetection, computervision, trafficanalysis, vehiclecounting, machinelearning, objecttracking, opencv, opencv-python, python, deeplearning, ai

Id: 2QaMsAV8_YY

Length: 28min 46sec (1726 seconds)

Published: Thu Feb 16 2023