Object tracking with Mean-shift – OpenCV 3.4 with python 3 Tutorial 29

welcome everyone to this new video tutorial we are going to see right now how to track object using the mean shift algorithm so let's see an example let's say that we have this video and I want to track this sticker listing zero so how can can we do that we can first take a picture just of blistering zero then we can take the histogram and using the back projection we can we can find in each frame so in the video where the object is after that we can find the exact area and draw it like a rectangle but let's see this step-by-step how it works so it will be decent for everyone to understand first let's import CV - let's import numpy us and T let's load the video is equal to C 2 that video capture and I will put the URL mouth wash dot avi and you can download this video from the link in the description then let's run the video while through underscore frame equals to video dot read next unfortunately a lot of noise outside hopefully it won't bother for this video see to that game show frame frame key is equals to see if it's without white key one so if the key is equal to 27 which means the asking the keyboard were going to break the loop and finally video release and see it so that destroy all windows let's run the code see if everything is correct and it is actually too fast I did a mistake so we're using the video so we need to pause from each each frame so let's say 60 so we're going to short frame each 60 millisecond which probably it's quite slow but we don't care about this right now then as I told you before we need to cut a screen the image of the listing stickers and I'm going to take that from the first video frame so we're going first to capture again a frame frame is equals to video dot read and let's call it first frame first frame so you can see C 2 dot in show first frame first frame you will see D it's always this one so from this we are going to get the listing stickers and I have already the position I found it before so it's x equals to 300 y equals to 305 the width is equals to 100 and height is equal to 115 so from this we can get the ROI so here is just a name region of interest so the right in this case is the sticker frame is first as we know to get a region from from an image we need to say first which rows so the rows are divided exists or from Y so from 25 to y plus height and then from X 2 x+ width and actually not from fraying but we called it first frame so we have the ROI let's see if it's correct ROI is equals to our eye let's run the code and here we have the ROI which is what we are interested in to it now as we have done in the previous video about that projection so I suggest you if you haven't watched it to watch the video because we are going to use back projection right now so from the oil we are going to convert the image to HSV so we have HSV ROI is equal to C 2 dot CV T color ROI see two dot color which are two HSV so from this we have we take a crate the the history so HSV format has three values three channels hue saturation and value we're going to use only the first channel to find our object so we are going to take the Instagram of only one channel so RI hist is equal to c v2 dot kal hist we're going to cut this one of the HSV ROI HSV ROI we're going to take only the first channels channel so index 0 we're not going to use any mask the size it's 180 we know that the first channel hue the values go from zero to one hundred seventy nine series included so all the sides it's 180 values so the range is from 0 to 180 we say 185 180 is not included so from 0 to 179 and I think that's all for the histogram and let's print the histogram all right Eastern so this is what we get on the history all the values to you to calculate the back projection we need to have an instrument which goes from 0 to 255 you can see that here goes from 0 they know West to I guess this is the highest 1128 so we have but if we want the maximum to be 255 so we have this function called normalized to normalize the Instagram which is going to put this number in proportion from 0 to 20 55 so this one thousand one and twenty eight will be 255 and for all the other the rest is going to calculate the proportion so this this one it's let's say the half of this one thousand one in twenty eight so if this is 255 this one is going to be the half so one hundred twenty seven point five or close to that so let's define ROI histogram is equals to C so doc normalize are I east RI East from 0 to 20 55 and see with saudade nor minimum and maximum let's print again our histogram print our I missed and you can see this is the maximum 21:55 and then we have all the numbers in between from 0 to 255 now once we have the histogram can find the mask using the back projection so what is going to give us the back projection back projection basically is going to tell us from this histogram so from this a history off of the region of interest tell me where this region of interest is in the image so to that we need to convert the frame to HSV so frame HSV Z is equal to C 2 dot CVT color frame see the hood dot color underscore which are two HSV and now we can create a mask so mask is equals to C V so dot car back projection on HSV we are going to use only the first channel so channel zero the histogram is the ROI histogram so our hist the range is from 0 to 180 and the scale is 1 so let's see the mask seaweed so that in show mask mask so you can see the white part is where the histogram is similar in the image in this case the distance ticker has the same Israel of this part which is moving which is the Eastern sticker okay now we we we are in the most important part well we're going to apply them in shift algorithm so let's see what is the moonship algorithm let me explain it in most simple words so me shift algorithm is going to find the area where their wisdom highest concentration of white points in this case so in this case using with the mask the highest concentration of white points is this one so where is the sticker moving once so forth fine found the concentration is going to give our us the point where we can draw the rectangle so let's do this the me sheet gives us two values first one is red which basically tells us if he found something or not right true when you're at and and actually not even sure if it's right let's call it under we don't need it anyway so under score we need just the position so truck window window is equals to C we through dot mean shift the image is the mask that we created using the back projection now we need to define a window where is going to start so the mean shift start tracking from a certain window that we need to define at the beginning and you will understand later on why and how it works let's first put the window that we actually calculated on the first frame so X Y width and height so X Y width and height and then we need to define term right yeah so term criteria is actually the configuration of min shift algorithm we're not going to get into details about about it so let's just copy what I'm writing term criteria as equals to see me so dot term criteria e s term then c2 dot term criteria count and one now truck window is is giving us the point where we can draw our our rectangle so let's first see this points print track window you see the position x and y changes all the time and here we have the width and height which is always the same so let's get this details into a variable and then from them let's draw our rectangles X Y width and height is equals to chart window and now we can't order tangle seaweed so that rectangle we're going to tour the rectangle the frame the first point is x and y the second point is X plus width actual X plus W and y plus h the color we're going to make it green so zero blue 255 of green and 0 of red thickness like let's make it so now we have everything I guess so let's run the code let's see how it works I will say it works really well so basically this put this if I'm not wrong let me check one moment through the to the dogs okay this how many points is going to move on the mean shift and is the iteration of the algorithm okay anyway let's keep this now what you are probably most interested in to it is to see how this is going to work in real time getting the frame from the camera so I'm going to just copy this video and put it on the website for you and now let's I'm going to edit this one so from the camera so we need to get the camera cap is equal seem to that video capture 0 and instead of video we have cap dot rate and that should be actually everything and let's run the code okay this is real time I'm talking right now and it's working real time not not perfect one percent you need to consider that I took the first frame before so actually we are working with the frame that I took from the other video I mean the Aurora I see so that in show ROI ROI so I'm going to use always this array which I took like a couple of hours ago when there was even a different lightning so it is a bit different right now but anyway you can see that it works now we are going to find two problems which the mean ship algorithm has first one the size of the rectangle is always the same so it doesn't matter if the object is big or if it's really small it still follows the movement but the size of the rectangle is always the same so me ship algorithm doesn't detect the size another problem me ship start tracking the object from a specific area so now where you see the rectangle is going to find the biggest concentration of white points closer to this area so if I can move the object from another unexpected side the earth do mean ship cannot detect the object it's looking only for the concentration of white points in this area but it does not get to this area so to get again the object tracking we need to move where the object is now he can detect bigger concentration and it will follow them this problem we're going to see how to solve these problems in another video specially we're going to cheat to see the cam shift algorithm as next to solve the problem about the size of the rectangle and this is all for this video and thanks for watching

Channel: Pysource

Views: 45,479

Keywords: mean-shift, obect tracking, python, opencv, tutorial, back projection, histogram

Id: EDT0vHsMy34

Length: 20min 13sec (1213 seconds)

Published: Thu Apr 05 2018