YOLOv8 Object Tracking in Real-time with Webcam and OpenCV

hey guys and welcome to new video in this video here we're going to use the new ulo V8 model it was just released a couple of days ago and then we're going to actually apply a tracker on top of the YOLO V8 model so we're just going to use a pre-trained model we're going to implement the sort algorithm together with yolo V8 you can also use deep sword and all those different kind of like algorithms I'm going to show you how we can modify the code from autolytics and also from YOLO V8 that we can basically just go in take all the information we can apply additional steps we actually just extract information put it through a Tracker and then we can track our Optics over time so this acts like really nice it can be used for a lot of different kind of things in your own applications and projects all my code will be available on my GitHub so this is really exciting we're actually able to run these yellow V8 models with tracking in real time I'm running like around 100 100 plus frames per seconds while we're actually like tracking around Optics in the frame then we're going to do live inference on a webcam with the sword algorithm tracker so first of all here we just jumped inside the YOLO V8 GitHub repository so we basically have this repository from ultralytics here we see some results for the new yellow V8 model we can see a comparison with the other versions of YOLO we can see that this actually has really good performance the same parameters and it can act like Run in real time we can run update taking instant segmentation and all those differing around things in real time we can see it's a documentation of how to install it I also have a video about that on here in the channel will show how to install it we go over to GitHub repository and how to set up these different kind of like parameters and then I'll show you how to do live inference so we're basically going to use that in this video we're also going to use the predicted class from the YOLO V8 model and I'll show you how we can modify the predicted class from YOLO V8 we can basically go in extract information from our detections feed them through a Tracker then we can track the Optics over time we can do additional post-processing steps we can extract information we can go in store information visualize the information if you want to do some visualization in another way and all those different kind of things so I'm definitely really excited for it we're going to see some really nice results at the end of this video so we're not in this Visual Studio code I've basically just cloned the GitHub repository from autolytics that I just showed you then we have all these different kind of folds folders where we have the yolos architecture we have the different kind of models we have this the scripts for doing detection predictions validation and also training so in this video we're going to use this predict file I basically just modified it where we have this detection detection predicted class which is inheriting from this base predictor and this is what is actually like handling all the predictions from our model it will basically just take the images feed them through the model and then we get the outputs out we can do some get annotator so we can get the annotator we can do some pre-processing steps so this is basically just normalizing our values we can do some post processing where we do non-maximal suppression on our predictions so this basically just a detector predicted class that you can set up and and this is actually just a default one from YOLO V8 you can go in and modify it for your own applications and projects so this is basically what I'm doing in this video here where we go in and do the tracking so we basically just add additional steps so from our predictions we go in and add a trigger on top of our predictions we also have some bright results here so if we want to write the results out to a file or if you just want to display them on our image so we can have this live interference running where we can see all the all the predictions in real time as well we can go in and set that up so this is basically just the default code then we have this predict function that we basically call when we want to run our application we want to predict with our configuration so we set the model equal to a model that we load in so a custom model I'm also going to create some videos where we're going to custom train our YOLO V8 models both for optic detection and instant segmentation and then we can basically just export our models from Google codelab after we've trained them and then we can import into here into our cfdt.model and then we can do predictions in real time with our own custom update detector here we basically just set up all these different kind of like input arguments so all input arguments should be inside of this config and then we can basically just pass in all the input arguments into our predict function when we call it or we can just call it with command line argument as well so this is basically the file here I just copy pasted the file and then we went in created this track.pi and then we have our tracker I just initialized this tracker so I have to sort that Pi so this sword.pi is act like the exact same code as I've used in one of the other videos where we did optic detection and optic tracking with the YOLO V5 model so this is basically just a sword algorithm which uses a kelman filter to track the Optics over time so we both have a predict State and we also have an update State and then based on new predictions we can actually like update update our estimate and our track of our Optics and then we can basically just trigger objects over a number of frames and then we can actually even though we lose track of our or like lose detection of our Optics we can still we can still track it because we have like we have knowledge about the past so let's say that we have an object here that is moving then we're just tracking that over time and then we lose information or like we lose detection of it then we should actually still be able to track it over time because we know how it has been like how it has been moving before that and but then we basically just have our tracker with some good parameters which we're updating based on our measurements and our measurements are based on our detections and then we basically just have this update and prediction step and then we can basically just have this Kelvin filter tracking our objects around in the image frame and we can store information use that for like tracking cars tracking different kind of like objects and also just for when we're doing object detection just add a track on top of it just to make sure that if we're losing detection we don't really like the dissertation of our of our act like detection in our image so if we lose the texting for one or two frames because again if we're running like 100 frames per seconds if we just missed like one detection we would like to like lose track of our uh lose track of our object if we're not using an optic trigger on top of our update detection model that we running so I won't really go into details with this code you can check the other video out where I go over how it's implemented this sorting algorithm you can also apply the Deep sort algorithm I also have another video where we use the Deep sword algorithm you can just copy paste the Deep sort algorithm and the code for that directly into it um as I've done here with the sword algorithm and then inside the track file we basically just initialize the tracker and do the exact same thing as we did in the previous videos so if you want to know like how we can set up these trackers and how we can use them on top of our optic detector definitely make sure to check those videos out here we're just going to use it so I just copy paste this sort algorithm into our PI file and then we just go up here so from sort we just import everything that we have inside of that file and then we import all the other things here so all the things here are basically just copy pasted from from the predict file provided by YOLO V8 first of all we need to initialize our tracker we just set our tracker equal to or we just set it as a global variable we set the Matrix 8 here equal to five so if we lose detection for five frames then we like like discard our detection but if we lose let's say we lose detection for three frames when we're actually doing and running our live inference on a webcam then it will still keep track of the update in our image frame and we want to track of it we can also set minimum hits here so before we actually initialize our track of an object we need to detect it two times you can play around with these thresholds here we also have an intersection over Union threshold so these are the three thresholds you will actually need to tune for your specific application and project to set up this tracker then basically just initialize our salt tracker with the max age the minimum hits and also the intersection over Union threshold and then we basically just have our tracker here we can then pass in the detections from our yellow V8 model through this tracker we will keep track of the update we can store the information and then visualize the tracks over time here we're just going to create a function for like creating random color list so we can just have random colors for our tracks and also the Texans in our image frame we have these drawer boxes so basically we just extract the information of our boundary boxes and then we use opencv to draw like rectangles um around the objects or the boundary boxes and then we also have this put text we're going to draw a circle at the location of our act like tracks so after we retract our Optics we will basically just store all the information and then we'll draw dots of all the previous positions our update has been in and that will basically just be the sender of our boundary boxes so over here we just take box of 0 box of two divided by two so we just get a center of it so we take the x value and the Y value so this will be the center of our bounding boxes and this is basically what we're going to draw so we can track and we can see how we have tracked the Optics over time in our image frame then we return the image and then the rest here for our detection predictor is basically like the class from the predict file but we can also go down here and actually like set up our track function so detections to sort first of all we just initialize them as empty so if we don't have any tracks at all or like any detection that's all we're just going to throw in an empty array because we actually need to pass something into the tracker but if they're just empty they will not do anything then we basically just go through all the detections so we have the detections here they are on the GPU if you're using the GPU so first of all we convert it to the CPU we get the x value y value X2 value and the Y2 so that will be the top left corner and the bottom right corner of our bounding box we get the confidence score and also uh the label of our detected class then we basically just set it up in the correct format for our uh sorting algorithm so this is the detections that we want to throw inside our sorting algorithm then we can call this update state so again we have an update State and then we have a predict state where our predict state is basically just predicting where are we in the frame with our object and then we have our opted state where we get the new detections in and then we update our previous predictions and our our new state based on the new detections then we have the tracks we will just want to get the track we call our tracker dot get tracks so then we basically just get the results from our tracked object from this get trackers we have the trackers we can just have a for Loop running through all the tracks we have the image and then we basically just display our tracked image so again we have the boundary boxes and then we'll have our our track positions we're going to display that on our image so we can see how our Optics in the image frame is act like tracked over time down here above them we can just specify like if the length of the track detections is greater than zero then we're just going to draw the bounding boxes so we're not just throwing throwing boundary boxes we only draw the boundary boxes if we act like tracking objects around in the image based on the parameters that was set to our tracker at the start then we basically just return the lock string here and now the rest of the code is basically just the exact same thing as in the predict.pi file so again you can use any other algorithm you'll just need to set it up in the same way as I have here or if you want to store the information write it out to a file displayed on the image in some other way you can just to go in and do that I basically showed here what code you actually need to modify to be able to do that so let's now run the file here and see how it actually works let's see how we can track Optics around in the image frame so I'll just open up a new terminal and we are now ready to run our program first of all I'm just going to call the deer command so we can see what directory we are in so first of all we're just going to see the inter alter Linux then I basically just copy paste this command in so we have our python so we're going to run this python file where go inside YOLO V8 detect and then we have this track.piot that we just created ourself we set the model equal to yolavi yolv8 and the media model so we can actually go in and modify that as well so here we can choose the large model uh so you can basically just this is a trade-off between like inference speed and accuracy uh and then we'll basically just have the model twice here we shouldn't have that and then we have the source equal to zero because we're going to use the webcam and then we set show equal to true then it will act like visualize the results as well when we run the program so now we're ready to run the program we just hit enter and it will run the command and then it will run this track.python file so first of all it will download the the jewel V8 model if you haven't run the code before then it will open up your webcam we can then see the results here and then it should be able to track persons over time so here we see if I'm moving uh it actually like tracks me here with a certain color I'm just going to take my webcam down here and then let's see if we can track some other some other Optics in the scene so if I just turn around here uh we can actually see this chair if I move it around we can see how it moves around in the image frame if we lose detections it will act like change the color but now we can actually see that we keep track of this chair pretty nicely and again even though we're included by other updates we still keep track of the object if I'm moving it around we can see that now it is in the top right corner of the image so this is a really nice tracking algorithm as you can see we're still able to run in real time we're running around like 10 15 milliseconds uh for the inference time and also running this optic detector on Oleg tracker on top of it we're running at around like 75 frames per second so this acts like a really uh really awesome model here we can just see me as a person it detects me as the sender we can also have the mouse so if I'm moving the mouse around in the image basically I'm just moving the camera around but again we just keeps track of all these points here so this is basically just the center of the of the tracked object or like to take that optic and then we just throw that information into our tracker so this is just a really crazy tracker and up detector that we can combine and we can get some really nice results so you should definitely use this in your own applications and projects uh when you're going to create your Optical Textures in the future this is really cool you can see the results that we get I'm going to create other videos where we're going to create our own custom object detector both for detection and also for instance segmentation so thank you for watching watching this video here and again remember the Subscribe button and Bell notification under the video also like this video if you like the content and you want more in the future it just will help me and YouTube channel out in a massive way I'm currently doing a computer vision tutorial and deep learning tutorial where we talk about like the basic things about computer vision deep learning how to get started with it I learn the basics the theories how we can train your network from scratch how does the high parameters affect the neural networks while training so all these different kind of things is really useful when creating these computer vision and deep learning applications and projects so if you're interested in one of those tutorials I'll link to one of them up here or else I'll see you next week guys bye for now

Channel: Nicolai Nielsen

Views: 17,927

Keywords: yolov8, yolov8 neural network, yolov8 custom object detection, yolov8 object detection, yolov8 tutorial, object detection yolo, object detection pytorch, object detection python, opencv object detection, opencv yolov8, opencv python yolov8, object detector, object detection yolov8, opencv, detect objects with yolov8, yolov8 opencv, opencv neural networks, deploy yolov8 model, how to deploy yolov8, yolov8 object tracking, yolov8 sort, yolov8 deepsort, yolov8 tracking

Id: UYLp0-iOvFc

Length: 14min 47sec (887 seconds)

Published: Sat Jan 14 2023