Object Tracking with Opencv and Python

and now we can see that each object each box has a number associated and the same motorbike or the same vehicle has the same number while moving hi there my name is sergio and i help students companies freelancer how to easily and efficiently apply visual recognition to their projects in this specific video today we're talking about object tracking you should know that there is a crucial difference between audio detection and object tracking the object detection is the detection of the objects done on each single frame even if we have a video it will be detecting the object frame after frame while object tracking will be keeping track of the object of the object detected during all the frames to have a history of where that object was over time and if you want to make applications like counting people entering on a certain area counting object passing on a conveyor belt or counting the cars that are passing on the highway or tracking enjoying the trajectory of an object then object tracking is something that you absolutely need and in this specific video we're going to focus at the beginning on object detection so the first part will be on object detection you will be learning how to detect objects and then in the second part of this video you will be learning how to apply object tracking to the detection so let's start to work with this project we need three files first of course we need a video where we want to track the objects in this case i have a video which is highway.mp4 where there are there is a camera from the top of the highway where we can see motorbikes and cars passing by and with this video our goal will be tracking cars and motorbikes that are moving and so see the counting of them and this is the first file the second file is an object tracker this is a file that i've written which will do the tracking and so you need to download this one from the link below because we will be using this one video plus tracker and then an empty file which we will be coding right now live which i call main.pi you can call that as you wish and this is the file that we will run and we'll import the tracker and so without adding anything else let's start with coding the main.pi file first of all i'm going to import the cv2 library which is opencv which we will be using for the detection of the objects in this case the motorbikes and cars then once we have saved it so we also want to create a capture object to read the frames from the video so cv2.video capture and here we need to put the path of the video file in this case it's the highway.mp4 that i have on the same folder so just to try with me i suggest that you download the file so that you can have the same video to try this out then once with the capture object we need to start a loop while true because considering it's a video we need to extract uh the frames one after another so on each loop we we take one frame so it would be red frame equals cap dot read now we're going to show the frame real time cv2 dot in show frame then frame now a weight key event key equals cv2.weight key uh 30 if the key is 27 i don't know it's a lot of things so once we show the frame in real time we need also a key event so that if we want to close the video we can just press a key in this case we say if the key is 27 27 is the s on the keyboard we break the loop and then cap dot release and then cv two dot destroy all windows and now to make sure that everything is correct i'm going to run this code so this is exactly the same video that we were seeing before but i am using this from python and opencv now our goal is to detect the motorbikes and how do we detect the motorbikes on this specific video well there are many different object detection methods if you want to learn more about object detection i have a full video course called object action with opencv and deep learning where you can learn how to efficiently detect anything on any kind of environment in this case we will be using a audio detection method for a stable camera what does it mean we have a stable camera which means that the background is always the same the background is the highway the road doesn't change over time while there is only the object that are objects that are changing over time so using the object detection method with a stable camera we are able to extract only the moving objects and so let me use here the object detection from stable camera we need to create the object detector detectors will be object detector equals cv2.create background subtractor mod 2 and we will get we'll see some more information about this later for the moment you need to know that this object detector is going to extract the moving objects uh from a stable camera uh how do we know and how do we extract them well we need to do this in real time here by creating a mask mask equals object detector dot apply and then here we uh we want to apply the detection on the frame right here and now let's show the mask so that we have a clue of what we are doing in real time save it to the imshow mask and then mask and now we get a mask the goal of a mask is that makes everything that we don't need black while the objects that we want to detect white so in this case that we have a highway and then we want to attack vehicles like motorbikes cars and trucks all the vehicles must be white white all the rest needs to be black in this case the detection of this method is probably not one of the best because we have so many white elements but still considering that as you can see the motorbikes and the cars are more wide so they are bigger than all the other elements we can discard everything that we don't need so we can see this step by step uh to do this the best way now is to get the coordinates of all the white elements and then remove all the really small elements that we don't need so for example uh here we have the trees we have a lot of small dots while for the cars we have like big big white objects moving down so this can be one way to solve this problem uh if that's not enough it if it won't be enough we will find for sure another solution so from the mask here we'll start with object detection object detection from the mask we need to extract the coordinates so for this we use an opencv function which is cv2.find contours we want to find the contours of the boundaries of these white objects on the mask then the mod cv2.red tree and the method c with su dot chain approx simple these are some technical stuff retro3 and chain approx simple that we don't really get need to get into pretty much is how we extract the information but nothing more uh as long as we get the contours that's fine so we need to get contours and hierarchy which we don't need so we only need this one then we can say for cnt in contours and we can now see what are these contours cv2 dot draw ctrl we want to draw them now on the frame then c and t we want to draw 1 by 1 minus 1 to say that we want to draw all of them then let's make it green zero 255 zero and thickness let's make this two now as you see everything that it's white on the mask we are surrounding here with green contours but here we have a lot of things that we don't need so as i said before let's remove all the small elements to remove the small elements we can first calculate the area cal calc calculate area and remove a small element and then area equals cv2 dot contour area of cnt then uh we can put an if condition so we can say if the area is less than for the moment let's try less than 100 pixels then in this case um we don't want to do anything so let's say if it's greater than 100 pixel we draw the contour otherwise we don't draw them and have a look how nice we got this result in this moment so already putting contour greater than 100 pixels of course we have a lot of things that we don't need but we achieve some interesting result uh i don't want to make this uh tutorial quite complicated so if you want more advanced stuff i will suggest you to go on the link below and see the object detection course while in this case we are going to select a region of interest roi and we will focus only on this specific part of the road and we'll focus only on the vehicles on this road so that we can track them correctly so let's now define an roi which will be region of interest so we need to extract some portion of the image so extract a region of interest this will be roi to extract an array we need to define from which position from the image we want to extract that well one way first to know um from which position will be to show how big is the entire frame we have height width and the channels which we don't need so only height and we will be frame dot shape so if i print height and width we have 780 of height and 120 of width this means that if we want this region which is more or less in the middle of the screen will be the x position around 500 i can guess from 500 to 500 um to let's say 700 i'm just guessing now not to take time just to find the best coordinates otherwise i will make the lesson really long so frame uh from 500 to i will say 700 and now we also need to put the height not only the width so the height it was also from the height on more or less on the middle let's say from 340 to to 600 and let's see what we get i'm going to show now the roi cv2.in show arrive and then row and let's see what we get uh it's more or less it's what i wanted to get but it's really small so let's increase this let's make this arrive from this side to this side and to exactly the bottom of the window so i'm going to increase still around 80 100 pixels to the right side and i can do that right here so uh from 500 to 800 and then from 2040 to the maximum of the height so here we're defining where it starts the height that we want to cut to the end then the width starts from 500 to 800 and and this is already an interesting section that we can work with so we'll be tracking the vehicles that are only coming inside this area and this will make our detection much easier so what do we do now we want to make detection only on this area so the mask that we were applying on the entire frame before we are going to apply it right now only to roi and so also the contour we're going to show them only on roi and let's see what we get now something more interesting is now it's working better than before still not exactly as it should but it's better than before uh i want to quickly improve the detection of this one and then we can move on the tracking so as the detection is not working fine we need to make some change on the object detection algorithm which is this one by applying first the history so we're going to change the size of the history that the agreement processes the images the the longer the history the more precise will be but of course it will uh hardly adapt to the changes if the camera moves in this case the camera is quite stable so we can try with history of 100 and then var threshold here we the lower the value the more results we get but also this is the chance that we get many false positive so for example if i give 5 you see we have more white more detection but more false positive and here we see a lot of wrong greens while if i put a higher value we can turn around 40 or higher there will be less detection but less false positive so we need to find the best range that fits for our project and this one is quite fine from this we can now extract the rectangle from the from this surrounding and we can see how the detection of the cars and vehicles is working so let's do that right here i can go on the contour instead of drawing all the contours we're going to extract the rectangle so the box that surrounds each object detected and we have um x y width and height which is saved so that a bounding rect of c and t and from this we can draw the rectangle c two dot rectangle uh we want to draw the the rectangle on r or i then we have point one x and y and point so we need the top left point and the right bottom point uh right bottom will be x plus width and then y plus height uh call let's make this green zero blue 2050 fine of green zero of red and thickness let's make this three and let's see what we get it's not the best detection but it it works one last thing that i will add and then we will remove right away on object tracking is this algorithm is giving us not only the object but also is detecting the um the shadow so we see white objects and gray shadow so if you have a quick look at this we have the motorbike white and the shadow gray we don't want to detect the shadows so let's only keep all the elements that are completely white for this case we're going to clean the mask uh mask will be uh equal to sieve to the threshold uh with this threshold operator we're going to remove all the pixels that are gray and keeping only the white pixels a threshold of of the mask all the values that are above 254 so keep in mind the pixels have value from 0 to 2055. 0 is completely back 255 is completely wide and on the middle we have different gradients of gray the closer to zero the more it's dark the closer to 2055 the more it's white so as we want only the white we want to remove everything that it's 2054 and below and all the rest will make white so from 2054 below we remove it all the rest will keep it white and this is operation cv2.thresh binary so if i run the mask now you can see how well now the motorbikes are detected because all the white from the shadow was removed um i'm quite happy with this result that we got uh quite already uh live now this is object detection so we know we see now this motorbike here is surrounded by a green box but we have no clue at least on the code of where this motorbike was before on the screen so we're now going to apply object tracking and let's do that for object tracking we need to import the file that i've created so if you don't have that download this file and put it on the same folder and then import it from tracker so you need to put the file tracker.pi on the same folder we're going to import everything and just at the beginning we're going to create tracker object uh tracker will be equal to uh euclidean dist tracker and how does this work this tracker takes all the bounding boxes of the objects so we will we need to save all the bounding boxes of the object into one array so it will be object detection or here we we discard all the elements that we don't need and we find only here if there is greater than 100 we find the the bounding boxes of the object so all these x and y uh width and height we need to put them into one array so here we print x y width and height um i will quickly show this one let's now get one object we see 141 222 1369 is this bounding box let's see now we have two and here we have two of them we want all these numbers together inside one array and we can easily do that by say detections it's an empty array each time we find the boxes we're going to put them inside detections uh detections dot append and we want to append x x y width and height so if we print print detections now of course it's empty uh now i'm looping frame after frame on the video now it's empty because we don't have anything detecting now second frame empty third frame there is nothing let's go now we have one object detected one motorbike here we have x y position of this motorbike the width and the height of this rectangle containing the motorbike when we have more bother bikes now we have two of them detected so the array will be containing two bounding boxes the first one first motorbike second one second motorbikes uh and this works of course the more motorbikes we have the more the bigger the array so now we have three motorbikes let's see and now we have three boxes one two three what do we do we give these detections arrayed to the tracker and the tracker in return is going to give us also the history so on univoc id that is associated to each motorbike and we will see how to work with that tracker so here we have object tracking so remember first first step is object detection is step one only if we have object detection of course we want we can make object tracking so the second step would be object tracking and now we we need to have detections um let's say boxes ids equals to the tracker dot update and we want to give the detections so let's now see what's happening with the tracker i'm going to print boxes ids and i'm going to wait until we get some object let's let's wait till we have two objects detected together okay now we have object one and object two and here we see we have the array so we have two objects so we have two arrays and that's exactly as we had before but there is something more we have x position y position um with height plus id so here we have three then we have x y with height plus id four so what does now happen that on each frame we do not only have the bounding boxes but also we have a unique id and we know that one of this object is id 3 while the other is id4 so if we go on the next frame so we move a bit still there will be different position but still id4 and id3 so this means that we we know that this object has its id and we we don't exchange that by mistake with other objects and this is important if we want to count the objects passing by this is essential for that operation um so let's draw everything that we have right here we have cv2 rectangle now we for box id in boxes ids here we show we extract x y with height id is from box id and the rectangle we're going to show the rectangle now from the truck not anymore from the detection because now we also display the id so we do cv2.put text we want to put the text where on the roi now the text the text we want to show just the id it's the string of id the uh now the position let's put this on x and y position minus 15. so i'm going to put the d just following the object so x is the position of the object y minus 15 so it's not exactly over the object so we have the the number a bit higher than the object so that's i'm using minus 15 just to give a small distance from the object then the font we can use cv2 dot font hair shape plain it doesn't really matter the cell of the font we don't care about that right now color let's make this blue 255 blue zero of green zero for red and two of thickness and let's see what we get so you see we have id3 id4 let's uh let's make this in real time so by the way keep in mind if you put weight key 0 it keeps the image on hold until you press some key if you put a number higher than 0 it will wait dot milliseconds so this will wait 30 milliseconds between each frame uh let me increase a bit the size of the id let's say two and let's see what we get and now we can see that each object each box has a number associated and the same motorbike or the same vehicle has the same number while moving and thanks to the ids we are able to detect uh to count correctly the objects that are passing by if we want to for example we could draw right here a line and we can check the has some id cross that specific line if so then we can increase the count of course this was just a really simple explanation of object tracking with a simple method and it's not perfect as you see the camera now is moving a bit and there is some error on the detection i hope this was useful for you i put a lot of things i know that it can be overwhelming if you are a beginner so i suggest you play around with this but if you want to understand all the secrets of object detection and object tracking from the basic to understand uh first how to detect objects in a certain space and also how which method to choose to detect the objects and then a best more sophisticated and reliable uh object tracking method i will highly suggest you to have a look on my video course audio detection opencv and deep learning i hope you enjoyed this lesson and i'm going to release soon new videos and see you on the next lesson bye

Channel: Pysource

Views: 24,653

Rating: 4.9709091 out of 5

Keywords:

Id: O3b8lVF93jU

Length: 30min 2sec (1802 seconds)

Published: Thu Jan 28 2021