Contour Detection In OpenCV 101 (3/3): Contour Analysis

hi in part one of contra detection 101 we learned the basics how to detect and draw the contours and do some other things in part two we learned to do some manipulations with contours in this part we're going to learn one of the most important things regarding contours in opencv and that is contour analysis this is really important because by doing contour analysis you will be able to detect and recognize objects you can also create some interesting applications like real-time shape detection using the techniques that you're going to learn in this tutorial so let's get started so for this tutorial we'll be using this jupyter notebook so make sure to download it from the link in the description uh below all right let's start so first i'm gonna import all the required libraries as you can see i'm importing opencv numpy pandas transformation module which i just created i'll later on get into what it does exactly and matplotlib and i think the email too yeah so then i'm gonna read in the image using uh cbto dot emulate function and then i'm gonna display it using matplotlib all right so let's run this cell so this is the same example that we have been using the past two tutorials so after displaying this image i'm gonna go ahead and you know like convert it into grayscale since this is already a pre-processed version it's gonna be converted into binary and then i'm gonna pass it directly in the find contours function and then get the contours back then i'm gonna pass this these contours in the draw contours function to draw each individual contour and then i'm gonna display this image uh with network all right so all these contours are drawn out so let's get to the real business which is content analytics and the first thing that we are going to see is image movements now image movements are weighted averages of pixel intensities in the image and these are really useful because by using image movements you can find out useful things like the center of mass of a control or the centroid of the contour or the area of the control things like that and these properties are really useful to understand what the contour looks like what the shape of the contoured and properties like that and this way you can differentiate between one corner and another one and by doing that you can recognize certain objects so some of these properties are invariant to geometrical transformation like translation rotation scaling and things like that so we'll look into which property is invariant to what in a bit so first of all let's take a look at how we can use like extract or movements image moments of any contour so for that what you need to do is first select any individual contour so from the list of contours i'm going to extract a single contour and i'm right now i'm extracting this contour this the biggest one and then i'm gonna pass this contour into the cv two dot moments function and then i'm gonna get the moments back and store it in the variable called m and i'm gonna then print out this moments variable so what this is gonna give me if i run this code is a dictionary of all the image moments that are extracted from this contour and to learn more about these movements what you can do is take a look at this wikipedia article that is linked here which discusses how each image movement is calculated and it goes into a bit of a lot of maths regarding you know like how the contrast works the math is not that complicated in in case you're like interested in how contours are calculated as you can see all these the formula required for these contours and so some details regarding uh you know like other things so this is the most interesting you know like section here translation variant moments skin variant movements and rotation variant movements and we're going to look at the these particular moments are called the hue moments and we're going to discuss these moments in a bit so this is a really interesting wikipedia article that goes into a bit of maths regarding how contours um image movements and hue moments work image movements are in turn divided into three parts uh raw movements center moments and then normalize center moments and then there are eu moments so it's really interesting i've also you know like printed out in fact included an interesting learn opencv article here which also discusses upon some of the things i mean image movements it's an interesting read all right so now that we have briefly covered what image movements are why they are useful let's use them to find out some interesting properties like the centroid of the contour so we can do that by like this formula what i am doing simply here is dividing one image moment with another one and this is this particular moment is the area of the contour so by doing this for m one zero i'm i can get the x coordinate of the centroid and by doing the same thing with m01 contour which is the other one i'm i can get the y coordinate of the center centroid and then i'm printing out these values so let's run this code so this xy point is the centroid of this contour and i extracted this uh centroid by using image movements these two particular values and now what i'm going to do is you know like extract the centroid of all the contours in this image so for that i'm gonna first make a copy and then iterate through all the detected contours so for each contour i'm gonna extract that it's image movement and then i'm gonna use these extract its x and y coordinate for the contour and then using the these coordinates i am going to draw a circle around the center of each contour and then display the results as you can see a circle is has been drawn on the center of each detected contour this is really useful in a lot of cases a lot of times maybe you need us the centroid of the detector object in order to like do something or do some manipulation around that so really useful technique now the next thing we can do is find the area of contour we already have a function which we learned in the previous tutorial called cv2 dot contour area right this gives us the area of any contour you just pass in every individual counter and it returns to you the area so let's print it out so this is the area similarly we have an image movement called m00 which also gives you the area of the contour so let's run this code so as you can see both methods are returning the same area and these two other methods if you need the area of the contour all right let's take a look at some other properties of contours and these properties are really interesting because some of these properties would be invariant to scaling translation and rotation things like that so for that what i'm going to do is first read an example image and then display it and then we're going to work on that example image for the rest of the tutorial for the rest of the few examples so first i'm going to read an image called sort jbg and i'm going to convert it into grayscale and then i'm going to threshold it and then i'm going to get this binary image and pass it into the contours function and extract its contours and then i'm going to select the first contour that is detected and this image only contains a single corner to the so the first contour would be the only contour that is being detected on this image then i'm going to draw the contour out and then display the results so this is the sword uh you might have seen this example in the previous notebooks so this is the sword and i have drawn out an out contour outline on this example all right now what i'm gonna do is create some uh you use some contour properties and show you how they are invariant to things like translation rotation and scaling so for doing that i have created a python module inside you know like this directory then you'll get the python file so the file is called the transformation.p by which contains a function called uh transform and what it does is it actually this function you know like either translates or scales or rotates the image and what you lose is just passing the image file and select display goes to true which displays the result and it then randomly translates or scales and rotates depending upon if you've set these variables to true or false so the syntax for that is this and this is how it looks like so let me just run this cell and as you can see here by just running the single line of code it has taken this in like a sword image which is the original image and then modified it to something like this so what it did is it replied a translation of minus 43 in x direction and plus 30 in y direction and then it applied a rotation of angle 154 and then it resized or scaled the image by 89 meaning it sort of reduced the original size so right now what i've said it is that i'm modifying each image randomly so if i run this again there's gonna be a different modification so if i run it again there's gonna be a different modification so it's randomly you know applying all these three transformations uh to this image so if you don't know already the scaling is something like resizing the image then rotation is like rotating the image and then translation is like shifting the image here or there things like that you just shift the object in the image to you know like in x or y direction so this this is how these three basic geometric transformations you know like work so now we're gonna take a look at some image properties that are invariant to these things which means that even if you translate or rotate an object these properties will be remain same throughout so this is really useful so for example consider this so consider an example i have you know like i have this remote right and if this remote was a single occur like black or maybe red or any other color and then what i did is create a color segmentation and then after color segmentation i would probably have gotten a mask a binary mask giving me black in the background and white in my detected object so this area would be become a white and all the rest of the background would become black which is a binary image and this is the image that i want to pass in the contour but what if i calculate some properties when like oriented the this object is like this and then i rotate it like this then the contour properties would probably change for example if i take this image and you like move it to the camera the area of this contour will increase and if i move it back the area will decrease similarly there are other properties that maybe if i you know like pick this up object here and move it sideways the some properties would change so a lot of and then if i rotate this like this some other properties might change for example if i'm drawing a bounding box right so if i draw a bonding box a box would be like this you know like it would be drawn like this but if i oriented it like this then the bonding box would take a lot of space so let me just draw it this out so if i have an image which contains a segmented you know like a rectangular object like this and then if i wanted to draw a bonding box you know like this is the object and if i wanted to draw a bonding box around it then it would be something like this right but if the same object is sort of rotated like this right the same you know like the object is rotated like this then the bonding box around it would be something like this the bonding box would be much bigger so as you can see as the contour as the shape of the contour as the orientation of the contour changes some certain properties around that contour also change and the best properties are those that are in variant those changes so and these are the properties that we'll take a look at because by analyzing these properties we can like differentiate and recognize the same contour in different you know orientations or translation or different scales so this is really useful uh technique so the first thing that we're going to see is the aspect ratio and it's really simple it's the ratio of with divided by height and the thing about this ratio is that this property is invariant to translation so even if you if you i take this object that is detected as a contour i move it around like this this is gonna give me the same aspect ratio so the width and height would remain the same but aspect pressure won't be the same if i move this object closer or further from the camera because then the weather height would not be remain the same and this would you know like cause uh problems but this is this particular value is invariant to a translation so what you need to do is first you know like draw a bounding box around the contour like i just i just showed you and then what you need to do is just divide the width by the height you get the aspect ratio and what i'm going to do is just print out the aspect ratio value and then i'm going to transform this image and set translation equals to true so it's going to randomly apply some translation to this sword image and then i'm gonna get the x x y with height back with this modified contour and then i'm gonna display the second aspect ratio and we're gonna see that both will aspect ratio would be the same so what's happening here i think i just like uh yeah so let me just run this so as you can see the aspect ratio initially was 0.994 and after the translation has been applied the aspect ratio remained the same so the expectation property of a contour remained the same even after translation was done so this property is translation variant let's take a look at another property called extend this is the ratio of contour area to the bonding rectangular area so consider this so if i have an if i have a you know like uh an image which contains a circular object which is detected as a contour so the area of this circle is the contour area and the bonding blocked area that is encloses this object object is the rectangle and what extent is doing is it's dividing the the contour area meaning the circle's area circles area with the rectangular area that's enclosing uh this contour so it's dividing the circle area with this which is just with multiply by height which is the radius of the rectangle and then it's dividing it dividing quantum area with this area so it's really simple and this property is invariant to i think it's invariant to translation and it's also invariant to scaling so if i move this object which is detected as a contour closer and further from the camera the extent would not change the extent would remain the same if i move it up and close or if i move it move it sideways the extent would remain the same so this is a interesting property to use in order to analyze control so in order to calculate the extent you can get the contours area you can extract uh the bonding effect and then you can get the rectangular area by just multiplying with with the height and then you can get the final extent by you know like using this formula by dividing the original area of the contour which we got with contour area function and with the rectangular area so this is how you get the extent i'm going to print out the extent before the modification and then i'm going to print out the extent after the modification and then we're going to analyze both of these values as we can see both these values remain the same even after a translation of 3844 was done and the image was scaled to 170 percent office uh its original size so now let's take a look at another property called equivalent diameter now the definition for that is that it is the diameter of a circle whose area is the same as the contours area so if i have a contour something like this then it has some area right and i can get that area by using contour area function like here so what equivalent diameter mean is that draw a circle which should have an area equal to that contours area and then take the diameter of that circle this is called the equivalent diameter so this is the definition of equilibrium diameter and the formula for that is this all you do is just you know like take the area of the circle and then rearrange this equation so that it becomes this and then you put this equation here so what i'm doing is just putting this equation here taking the square root of 4 multiplied by the modified area divided by the pi and yeah so it's really simple and then i'm printing out the equivalent diameter before the modification and then i'm printing out the equivalent diameter after the modification and this particular property it's invariant to translation and it's also invariant to rotation so if you translate or you rotate the equivalent diameter would remain the same but aspect ratio and extent would not remain the same if you rotate the object so this particular property is invariant to the rotation and the like the translation so if i run this as you can see the equivalent diameter is approximately the same here before and after the modification even if the rotation is done by an angle of 30 year and translation is done uh like here so these were the three interesting properties of contours that you can use to analyze contours now we're gonna look at some more interesting and more robust properties called humans but before that let me just show you that how you can calculate the orientation of a contour so for example if i have a contour and which is related something like this and you want to know what angle this contour is rotated at so there are several techniques to do that let me first randomly transform a contour with some angle of rotation and maybe just let me make that true so so yeah i've rotated this angle by an angle of 115 degrees all right now i'm gonna get the angle and gonna calculate what angle this modified contour is rotated at by fitting an ellipse on it and then i'm gonna do the same thing again but this time i'm gonna fit a rotated rectangle on it and then get the angle so if i run this i'm gonna get an angle of one zero seven point nine something like that and then i'm gonna get an angle of 18 point something like that if i run this again i'm gonna see some changes so this is 46.88 this is 45.0 so i've seen maybe some angular limit these two are approximately same but past a certain angle these two vary a lot so one thing that probably confuse you is why all the three angles are different i have applied a rotation of 176 here and the fitting ellipse is giving me 46 something and the circle minimum area is giving me 45 something so why are these angles a lot different by the way right now these two angles are approximately the same but as we see you saw earlier these two can be drastically different depending upon what the original angle is so the thing is what's happening here is that when i rotated this sword what the geometrical transformation did is that it picked up the sort from the center and then rotated it from the center but when i do benefit an ellipse around it so what it's doing is that it's fitting a sort of an ellipse and then the ellipse is making an angle to some reference point and then it's calculating and the angle for that ellipse from that reference point the same thing is happening with this rotated track so that rotated direct is making some angle at some reference point and from that reference point the angle is you know like returned to you so if you're making a computer vision application that depends upon how the target object is rotated and you need to trigger some things regarding how the object is rotated you need to figure out what are you going to fit on that contour so you can fit a rotated track you can fit an ellipse you can also fit a line i've not covered a line because that the cord for that there's a little bit cryptic but it's the same concept you you fit you fit a line so you do something like this you draw a line and and then you calculate some reference but the reference point can be a horizontal line and then you calculate the angle that the line is making at that reference point and then you get the angle so the thing is that you first need to calibrate what angle that your target object makes when you fit a particular object on it and then you like note these angles down so the next time the contour makes that angle you you trigger the events that you want this doesn't have to do with analytics exactly but this is a really useful technique which i've used in a couple of applications so that's why i've covered it here so now let's take a look at the final analytical technique that's called hue movements and this is a really really powerful uh technique you can use hue movements to match different contours or differentiate between you know like a different contour or you know like recognize same contours in different orientations translation or skills so all you need to do is first use the function called c movements to get the movements and then pass the moments in another another function called cv2 dot hue movements that's it and once you pass this moments in the human function you get the implements back and then i'm just you know like printing out the humans so these are the seven hue moments and all these seven are invariant to rotation translation and scaling so all these seven properties are invaluable to all those three basic transformations now let's take a look at the humans but before that what i'm gonna do is transform these values to log scale this is in linear scale and i'm going to transform it to log so the reason for doing that is that take a look at this so these uh values are vary in different ranges so for example uh the you for the first two moment you know like it's just exponent minus one and then the uh the seventh you movement is exponent minus eleven so the range for each of these values is drastically different so in order to get all these values in the same range i'm gonna transform this into log scale so that it's comparable so once i do that and then display it out i'm gonna see these results as you can see now the range is pretty similar all these are in a similar range all these seven hue properties let me just run this code again all right so uh what i'm just doing is applying uh this line of math to convert this linear scale into a log scale and or then i'm using pandas to just display and like display these movements you can just print it out if you want but i wanted the index to appear besides these actual values so that's why i'm using pandas to display this is just optional so after i've displayed it i'm gonna modify the contour so i'm gonna modify the short contour so let me just turn this again so it's somewhat different so yeah this was the original image now the modified version looks like this and in the modified version i have applied a translation of minus 3148 an angle of 122 and image has been resized to 87 of its original size so all these three transformations are done now let's take a look at the modified hue moment so i'm going to calculate the hume moments again in this modified version and then we're going to take a look at the difference between the original view moments and the modified moments values so as you can see all these values are approximately the same all these seven properties are same regardless of the fact that i translated the image i scaled the image and i rotated the image so this is something really powerful so you can use these hue moments in order to match shapes and in fact i'm gonna teach you how you can build some exciting shape detection based applications on top of these things that you've learned today and we're gonna learn to do some other exciting things with contours like how to determine the distance between an object and the camera how to sort objects and how to use the hierarchies properly and a lot of other nitty-gritty details regarding contours and then we're also going to learn a lot of other applications that can be built around contours in opencv but all that would be covered in a separate course the course will be called building control application with opencv and python it's a really interesting course and it's just a one day course so it's a mini course and it would be released in a couple of weeks on bleed air so stay tuned for that and if you enjoyed this video make sure to subscribe to this channel and i'll see you the next video bye [Music] [Music] you

Channel: Bleed AI Academy

Views: 11,283

Keywords:

Id: g7JegWRtMi8

Length: 25min 15sec (1515 seconds)

Published: Mon May 31 2021