Multiple Object Tracking using OpenCV in Python - Part 1

[Music] [Music] [Music] hello and welcome to multiple object tracking and videos using opencv in python video series this is part one of this video series in this particular part we will be discussing uh how can we how can we track a single object in a video in the next part however we will be discussing how we can extend this single object tracking to track multiple objects and videos object tracking is really important in several areas in data science particularly in video analysis sometimes it is really time consuming to detect object in every frame independently so and further detecting object in every frame independently is not only in most of the cases not only inefficient but also it loses the uh loses the confidence that is provided by previous frames the detections and trackings of the previous frames so this object tracking is a workaround it is really fast and in several cases when we really want to annotate a video we want to detect a particular video a particular object or set of objects in video it is okay to just initialize the object in the very first frame and then the tracker will automatically basic automatically track the object throughout the video obviously there are certain caveats there but in this particular video we are going to in in this particular part we are going to discuss um discuss single object tracking uh the next part we'll discuss will extend this idea to for for multiple uh objects in the same video um just to just to motivate you about the problem uh this is uh this is a video sequence um i i've just taken this video sequence from youtube um uploaded by i guess we talk football uh in that video sequence if you see there are a lot of objects that are that are going on i mean uh this particular object that object football is an object there are a lot of objects that are being uh interacting with each other so what if we really want to track these objects for example by tracking i mean i want to know this particular object is where in which frame so the the video is just a sequence of frames sequence of images and in each image i really want my object of interest for example this referee in this green shirt this may be my this particular object may be my object of interest and i want to know its position its bounding box in each frame so let's see how opencv can help us in in python so let's go for coding directly so first of all we need some packages to import for example import cv2 if you have not installed this you have to install this um opencv in python maybe in in on anaconda prompt and there are other ways as well i have pre-installed this so i have loaded it there are so many trackers in opencv that are available so for example uh correlation based tracker kcf boosting tracker mil tracker training learning detector tld tracker median flow moose tracker csrt tracker there are so many of those let's list all of those in in a dictionary so trackers tracker tracker list or dictionary tracker dictionary ticked is let's say there are so many trackers so let's list them csrd that tracker and the opencv opencv function name is cv2 dot tracker c csrt underscore create so we will see later on how to use this but if you really want to use this csrt tracker we need this this this name and then a brackets to call this function to initialize the tracker there are so many other trackers as well for example um kcf that tracker is initialized by cv2 dot tracker kcf underscore create so that then we have for example boosting boosting tracker and that is again cv2 dot tracker boosting create um and then we have um we we have a list of several trackers so let me let me list all of those whatever we want to use we can use there are there are situations when one tracker performs better over the other and we will see what kind of situation in what kind of situations which record should be used uh what what are the recommendations so cv2 dot tracker uh mil underscore create and then we have this um tld tracker which is again cv2 dot tracker tld underscore create and we have a couple of more trackers like median flow medium flow which is cv2 dot cracker median flow underscore create and then one last one is moose which is really fast tracker um this is cv cv2 dot tracker cracker boost create so that's a list of trackers that are available just it is good to have this list just for example whenever we need one of these trackers to to to to to be used we can we can use this list so for example let's use the crt tracker so our tracker that we are going to use let's initialize that tracker uh using this tr dictionary dick and from this dictionary we are going to use let's say c s r t that will bring that will bring this outcome and then we need to call that this way so if we for example if you for example want to call directly we have to use like this cb2 dot tracker csrt underscore create and that initializes our tracker if we really want uh directly but what happens here is this particular command will bring this string here and then we have this parenthesis for this function call so either way is fine yeah so let's see the tracker is initialized perfectly fine let's now read the video uh that that i just showed you in the slide let's read that video so that's a video stream or video maybe video is um cv2 dot video capture so that and then we have to give the path for for the video so the video i have is located in d and the name is multiple object tracker mp4 so that's the video um let's see if the video is read properly yes it is read properly now we we read one frame of it just to get um the bounding box or initialized the bounding box which we really want to track further so let's say we have return value or um yeah return value frame um and that is v dot read that will read uh the very first frame um and then what we do is we basically display that frame so cv2 dot m show we first initialize some window let's say that's my window name and in that window i really want this frame to be initialized uh after it uh after showing this frame i really want to really want to draw a bounding box on the object of interest so bb bouncing box is uh cb2 dot select region of interest that's a region of interest on which window we are operating this window remember this window this is the name this name they must be same uh this is for reference and then uh this is the frame for which we are going to crop that um and then what we will do we will initialize our tracker our tracker we will initialize our tracker tracker dot init with this frame and with this bounding box so so far the init the the tracker will get initialized so let's see let's run that and see where yeah so here let's say i let's say i want to initialize this object so i just draw a bounding box on that and then i press enter and um and the and the uh and and the object is being initialized the tracker is initialized with this box now the goal of the tracker is to really track this over the video which means we need this bounding box uh on this referee even wherever the referee goes in the video so let's write code for that while true because we are reading a lot of frames uh return flag and frame that is v dot read it will read uh the next frame if not return value if it ends if the stream ends then simply break otherwise otherwise just get the success and box by asking the tracker to update so the tracker will update on this frame the tracker will give now the box previously we defined a box but now in every frame the tracker is supposed to give us uh the box on on this particular frame now what we do is if the box if we really find the box then the success will return as true then what we do is we get that box and draw a rectangle on on draw the same kind of rectangle or the box on the image let's see so for example x y w in height width and height that's the box int int a for a in box so this box will contain four numbers so we will iterate over these four numbers one by one uh these numbers might be in the floating point so we can we round them up we basically get just the integer part and this x is the top uh x coordinate of the top corner this is the y coordinate of the top corner that's the width and height of the box so once this is returned we can call the function cv2 dot rectangle and we can say okay on this particular frame with this x and y and uh the next coordinate so we have to give the x and y of top corner and x and y of the bottom bottom left corner so we can give x plus w that's the um corner of the the other the other corner coordinate and then y plus h that is so basically this x y is the coordinates of the top left corner this these are the coordinates of uh the bottom left corner in in image coordinates then we can give the color in which color we really want to draw that maybe we can maybe we want to draw that in red so so let's say this is the code for red or maybe for green or maybe there are several colors and that's that's one parameter that we have to give now after that um we have to show this cb2 dot in show frame so that the image so this will display the image and this will draw the rectangle on on the image um we can we can wait for a particular key if the video is longer and we want to quit this process of tracking we may want we may expect a user to press a key and then we just just stop our process if the video is really lengthy and we cannot wait till the end so cv 2 dot wait key weight key this is how many units of time you want to wait so let's wait for this and 0 x f f uh what this means is i'm i'm i'm showing this uh right now what this o x x o zero x f f means what happens is this c v two dot weight key it returns an integer of 32 bits and this particular um is the this particular number is the hexadecimal representation of one one one all the eight ones uh eight bit number with all ones so when we take end of this 8-bit indeed this 32-bit integer with this what happens is the only eight bits of this uh only eight bits of this 32-bits they will remain and those eight bits they are copied here so that's basically a shortcut of getting the um getting the least significant eight bits of this uh of this 32-bit integer um and because that's the that's the value of the key that we really want the unicode the integer in the unicode then we check if this key which is now an integer is odd q this odd queue will return in a unicode integer uh the representation uh in integer form of this character queue if this really is q if you want to quit then we will break um otherwise the loop will continue so that's the basic setup we are just close to our basic setup so let's see what happened here um yeah yeah so that's the basic setup we are really close to now to to finish this now um after after this loop after this loop loop breaks what we really want is we want to release the object we want to release the video stream object and we want to destroy all the windows destroy all windows so yeah so when we will run this code the expectation is you will see a red um you will see a red bounding box on the referee so let's see let's run this object maybe we have this frame let's run this here oh it has errors what kind of errors are these cb2.cb2 has no attribute weight key um why is that wait oh this k is larger this this k is yeah this k is in uppercase yeah so i guess it will work so let's see oh it should draw it in red color it is drawing in oh it's bgr yeah so now you can see this is basically tracking this object the same object is being tracked we can run this again and again with with certain maybe changings for example we can change the color to this that may be a different color let's see what that color is or maybe a hundred this this might be a different color we can run this again for example this let's track let's track this particular object now let's track this object and then what we really go do is um we press that and we can see that this object is being tracked in the video yeah so yeah so that's about uh object tracking um in in single video the steps are really really easy by the way i have used here csrt tracker there are several other trackers which tracker to use is really i mean it's problem dependent let me let me compare some of the trackers the csrt tracker is really really accurate it's much accurate but slow this kcf is also good in accuracy um but it its accuracy is relatively smaller than csrt when the object moves quickly um this tld has problem of false positives although it can be used moose tracker this is really really fast the tracker is really fast but the problem is if the object is moving in a really quick pace moose will most will lose the track so for example let me show you an object that may that the moose object may not capture very well let's see so let me let me highlight this object for example this object and yeah so let me show you this here the expectation is the if if this object is will move fast the moose tracker may lose uh in this particular case it it works basically um but uh let's in in several cases the moose tracker it loses uh maybe we can maybe we can focus on some other um object that the all the point is if if the object moves fast uh maybe the the tracker just lose uh its track so let's say for example this and then yeah so yeah so it just changes the it just changes the the track to to referee although if you if you see csrd which is a little bit slow but really great uh in terms of accuracy so let's see for example the same object we draw the same object here and yeah so it is not losing its track it's it's moving on yeah so it really depends upon uh what kind of uh what what what is your uh what is your requirement in the in the problem uh that you're tracking if if efficiency is really a big deal for you then you may go for moose with a little bit drawback in the accuracy if the accuracy is a real goal for you and you can compromise on a little bit efficiency then csrt is is is a better choice better recommendation and the other classifiers they have their own benefits and drawbacks whatsoever so in this particular video we learned how to track a single object in a video in the next video we will we will extend this knowledge to to multiple objects so if what if we want to track let's say three or four or five objects in an in a video how can we move so in the next video in the next part of this video series we will see uh how can we extend this single object tracker to multiple objects hope to see you in the next video [Music] [Music] you

Channel: AI Sciences

Views: 14,473

Rating: 4.9370079 out of 5

Keywords: programming, python, analysis, data science, numpy, charts, coding, opencv, multiple object tracking, computer programming, learn programming, how to learn programming, computer vision, image processing, object tracking, opencv tutorial, deep learning, python tutorial, ai, machine learning, artificial intelligence, object detection, opencv python, opencv python tutorial, computer vision tutorial, opencv tutorial for beginners, opencv tutorials for beginners, opencv pycharm

Id: O1ABXetrMGs

Length: 22min 55sec (1375 seconds)

Published: Sun Jul 12 2020