Contours and Edge Detection with OpenCV in C#

contours and edge detection are two of the key fundamental techniques in computer vision and in this video we're going to cover just that let's dig in [Music] hi my name is chris and welcome back to my opencv in c-sharp series this series is meant to introduce and give you all the tools that you need to use opencv to its fullest extent as well as some foundational building blocks to your computer vision journey in this video as mentioned we're going to talk about edge detection and contours which are both very large topics but i'll try to break it down for you we're going to cover three important functions and talk a little bit about the computer vision ideas behind them and the information you need to use them effectively i.e being able to tune them to get the best effect if that sounds good to you well let's get started so what exactly do we mean by an edge on an image intuitively you pretty much know what we mean but if you think about it a little bit what we're talking about is a significant difference between two areas in an image some kind of distance between the values of two sets of pixels right well turns out it's much harder in practice but luckily some mathematicians figured out a few ways we can do it the first we're going to talk about is applying what's known as a soluble operator through convolution to our image the important thing to note about solvable operation on an image is that you're using a kernel just like we did in the last video which is convolved on the image and that's giving you an approximation of the derivative of the image usually this is done once for vertical changes and then once for horizontal changes and then these two outputs can be combined together and give you a pretty good edge detected image let's see what that looks like all right so let's talk about sobel the first thing i'm going to need you to do is grab the next image we're going to use you can use any image you want really but i feel like we need one that gives us a more defined object to look at starry night has a lot of edges to it the the resulting matrices that would come out of these operations would look pretty noisy but this one i got this cute little doggo here and it's a pretty well defined you know especially on this side you get a pretty well defined edge here you're going from uh pixels that are almost turned off two pixels are fully turned on the backside is a little less defined but um with a good edge detection we'll be able to get that too so um as always these pictures are in the github repository if you'd like to download them use the exact same ones or you can use your own and just experiment with it you may need to use different threshold values things like that but we'll look at that in a second so first let's as always as is tradition let's create a map um actually we'll just do this no need to instantiate so we're gonna do i'm read and this one's going to be img dog okay so now we've got our pick the next thing to know about a lot of these edge detection functions is that they usually at the very least you want to do some noise reduction to for the algorithm that'll function better so in our case we're going to do that with a gaussian blur pretty standard so i'm just going to create a new mat called that um and we're going to create a few other mattes um just to kind of give us something to work with and some for some examples so the first one i'm going to make is called syllable x the next one is going to be syllable y and if you're paying attention to the intro you might be able to guess what the next one is it's sobel x y um and this is gonna give us uh we're gonna have three examples of this solvable function and the kernels that are used the x kernel looks different than the y kernel and they both give you sort of a different version of the image and when combined give you a really good edge detection so we'll print all three out so we can use this example and the next thing i'm going to do i think i need to copy copy this to our guys okay all right and uh so now we're gonna do our gaussian blur just like we've done before and that input is pick gaussian blur uh a new system of drawing of size we're gonna do a three by three kernel and then um a sigma which let's see um we'll just do for now we'll do a five um okay sorry what is it oh comma all right so now we've got our gaussian blurred image so now we just need to do the soluble function and uh as you might have guessed that is um another cb invoke and it's gonna take uh these these parameters so there's input ray which is we're gonna use our gaussian blurred image there's the output destination which for this first one we're going to do syllable x and then there's a depth type we're just going to use the default type [Music] that is an uh cve num structure called depth type where you use default and then these uh next parameters are kind of what decides what type of uh solo you're doing so this first one is x order so we want to turn that on so we're going to give it a one y order since we're doing syllable x here we just want to do a zero there because we don't want to see the y order on this and then we're going to do a k size uh that is again our kernel size the default is three you can do pretty much what you want i'm gonna throw in five just to see what kind of effect we get um and then we're gonna do the same function for y and x y so let me change this and so the difference between x and y obviously we're going to use a different map and then we're going to turn this bit off this bit on and for syllable xy we're going to do the same thing except we're gonna have them both on okay so we've done our syllable functions on our three images we want to show now we need to just show them all right so y there and y there and then of course as always we need our weight key all right let's run this okay so as you can see uh there's some interesting stuff going on here so sobel x important to know here when you look at this image it looks like you see a bunch of verticalness to it right and for instance if you look at syllable why you see a lot of horizontal to it if you think about it a little bit it makes some sense because your solo x function is giving you differences across the x-axis so those differences are going to appear as lines that go up and down right so here actually if we look at the original picture real quick you can see there's this vertical difference here right and so if you're going across the x-axis you hit an edge there this is why that the syllable x defines an edge there but the syllable y do not because going uh you know vertically there's not much of a difference in those pixels so hopefully that makes some amount of sense you see a pretty well defined edge around our dog on both of them and then this is the syllable x y so when you combine both um you know outputs you you get places where there is a distinct change going kind of both directions essentially so you notice there's a pretty strong edge here a pretty strong edge here and so that edge shows up here our edge here does not show up because it's not that strong on the y syllable why so as you can see syllable xy pretty good at edge detection among an object so you could you could use this image to kind of um you know if you wanted to extract this object out you could do some math on the the values of this uh image here and kind of extract that out so that's pretty much how sobel is used let's move on to another edge detection function called canny edge detection annie edge detection was created in the late 80s by a man named john canny and really what he did was come up with a five-step process that leads to a well-defined edge on an image once you get the parameters right if you'd like to know more about exactly what that algorithm looks like i'd actually recommend reading the canny edge detector wikipedia page because it lays out all the information in a very nice way and kind of takes you through the math it's it's a good explanation but for our purposes we actually just need to know how opencv implements it so let's look at that and luckily it's actually pretty easy it's basically just one function call all right so we've already got some of the fundamentals we need which is basically gaussian blur that gaussian blur image is going to be our input let's go ahead i'll just comment this out so we don't show that and we can focus on candy so first thing we're going to do we're going to create a mat we'll just call it candy pick okay and then um let's see so probably the best way to start is with invoke dot canny so you can see the parameters here there's an input array there's an output array there's a threshold one a threshold two an aperture size and a uh l2 gradient we're going to ignore the l2 gradient pretty much for now the rest of them let's talk through them so the input is going to be our gaussian blur the output we're going to do candy pick and then so these thresholds thresholds are basically what's going to determine how it defines the edges it's going to take the lower of the two amounts and it's going to use that for one side of the function and it's going to take the upper of the two amounts and and use that for another part of the algorithm so there's you can set these manually and we can show what that does there's also a way to calculate these um automatically basically and let me show you exactly how we might do that so and this is kind of just for our purposes here today you don't necessarily need to do this but i want to show this because i think it's actually a pretty handy tool okay and so the way we're going to calculate automatically our threshold values is we're going to actually calculate the average value of our image as a grayscale image first so we're going to do that is with dot to image and then um we need a gray structure which i believe sorry is from using imgu dot cv dot structure okay ah i keep not being in bim mode um so i believe we just needed that yep all right and so we're gonna do a byte and then so that gives us our image as a gray biotic tuple and then we're going to do a function on that called get average and what that's going to do is going to iterate through the image and it's going to produce the average value of this grayscale image and the reason we're doing that is because we're going to use that to calculate the lower threshold and the upper threshold so the way we're going to do that is we're going to do the max of for this is for a little lower of zero or one minus three three i'll explain that three three in a minute um times average dot intensity and then for upper threshold it's it's sort of similar um but except instead of zero we're going to do 255 um and instead of subtracting we're going to add one through three okay so let me explain this a little bit so um actually i found this on a stack over full post uh it it's a kind of a nifty way you can calculate a decent probably the best results you're gonna get for your candy edge detection and uh what we're doing is we're taking the average of the grayscale image we are taking the max of either 0 for the lower or 1 minus 0.33 which is a value it's a sigma and it's it was basically determined at some point with some math but that's the value you want to use that's kind of the that's what for instance say a learning algorithm would converge to and then we're going to multiply that by the average intensity so whichever is um higher of these two and then the reverse for uh this so um all right we're going to do lower threshold upper threshold and then we have this aperture size value this it has to be an odd value usually i think it's between 3 and 9 if i'm not mistaken but that aperture size is going to basically determine how much stuff gets picked up in our edge uh detection if you turn it up if you experiment with it a little bit and you turn it up you'll start to see edges on places where the human eye definitely is not seeing edges so um it's almost like an intensity kind of thing uh we're just gonna stick with the default of three for now um and we're gonna call it there so okay so now in candypick we've got our uh candy edge so let's show it we'll just call it kenny okay and let's run that okay so this is what caney gives you it gives you extremely well-defined edges um though you will notice part of the animal is missing um the threshold values we chose uh pretty much ended up there uh there wasn't enough difference to to you know get through some part of the algorithm of candy uh on that back side so front side very obvious um as you can remember this this color is white on the original picture this is darker so very obvious edges there not enough on the back end um this is where you would play with your thresholds a bit try to try to get to where you can kind of uh do that you may also need to do other processing on the image as well for one thing one thing you could do uh would be to play with your gaussian blur you know increase the intensity of it decrease the intensity of it because you know that's going to affect how edges are determined as well um you could also brighten it the image you could you could do a lot of things to kind of hone this in for right now i think we'll just leave it there because that's basically the the basics of canny uh as you can see it's it's a it's a pretty cool effect you really do get very distinct edges and you can see how that would be useful in some cases so but for right now we're going to leave candy right there and we're going to move on to contours first thing to do would be to get the new image we're going to use for contours i found this image it is of an iphone and the uh nice thing about this it has very distinct contours it's also got inner contours here as we'll explain in a moment there's a hierarchy to contours and it's got obviously the inner apple things like that so very nice image for displaying um contours and doing effects with it so that's the image we're going to use for this part so the first thing we're going to need is well our iphone um and we're just gonna do i'm read okay and then um the next thing we're going to need is a data structure to store our contour information um there's one specifically for that and basically what this is gonna be it's gonna be a sequence of points that then uh opencv can take in and um for instance draw the contours but it's it's how the find contour function will store the data given back to us and that will be from a data structure called vector of vector of point um and yeah that's it's basically kind of how it sounds uh it's it's a wrapper in in goo for the vector of point data structure from opencv and it will store our contour information so and then from there there's a few more things we're going to need we're going to need a threshold uh pick i'll explain what that is in a moment and then we're also going to need a hierarchy and again we'll explain what that is in a moment um so first thing we need uh we actually need a grayscale image of the um iphone and so i'm gonna do image gray light and we'll call this uh gray phone okay and then from there we have pretty much all we need to uh determine our threshold image so threshold is going to be a binary image meaning every pixel is either turned on or off and it's going to kind of use some logic that's in this function to turn on certain bits turn off other bits duh that makes sense um so let's invoke that and you'll see what i mean so it's the threshold function so our we're gonna do gray phone um and then oh sorry threshold pick and then we've got some threshold values um threshold and max value um i i tweaked these numbers a little bit when i was playing with this stuff i'm you're just going to trust me on these numbers for the effect we want but obviously these are parameters you could tweak as you as you play with the stuff and then this last one is a threshold type threshold type uh let me show you what those are it's a cv and num and there's a few different ones um there's uh two zero which is one i've used before binary which is the one we're going to use uh it's basically just saying which type of um threshold you want and so in this case we just want binary and then from there actually let's just let's just show what this gives us okay so this will give us our threshold image all right so yeah pretty much as you can tell it's you know basically if a specific pixel is over a certain value it will turn it on if it is under a certain value it will turn off in our case since our iphone is white that's going to be usually the ones that are turned on you can see some light reflection pixels in here which is what those are that's why they're turned on because that light reflection met some threshold value r210 and you can also see the shadow here there's a slight curve or a shadow to the picture down here and so the threshold you know it didn't meet that value so this is this is a pretty clean image it's about as clean as we're gonna get just for this image without doing more processing on the front end um so yeah that's our threshold image so what can we do with that well there's a cv invoke function called find contours and its parameters are threshold pick so we we give it our our threshold pick our contours um and then we also give it a hierarchy mat which is where it's going to store the hierarchy information um to explain that a little bit if you think about contours and how they work so an object can have depending on what type of structure you want it can have a tree of contours right so obviously in this iphone this is the outermost contour i think that one's pretty obvious um then there's inner ones too so a child of that contour would be this one right and this one and then the children of those contours would be this one maybe this one maybe this one depending on your values there's none in here so this is not this has no children but you can see how it would build a hierarchical hierarchical structure um with contours and so that's exactly what we wanted to do and it's going to store that hierarchy information in this matrix and then from there there's also a uh retrieval type and in our case we want it to give us everything basically and so we're going to do [Music] a retriple type enum and it's going to be of type tree you can also get the information as a list and it explains what these do on this tool tip here but in our case we want a tree and then the last value is a chain approximation method this is actually basically just how does how do you want the function to work and in our case that is again a cv enum and uh of chain approximation method and there's a few different ones we can use in our case we're going to use chain approximation none and as it says translate all points from the chain code into points easy enough there's other ones too uh this one is actually just straight up a different algorithm but it's it's listed in there and you can do that as well but we're going to use chain approximation none okay so this uh this will give us our contours um and if we ran this and then inspected what was in this uh this structure here you're to see basically a bunch of points and those points in a sequence and that sequence would be used to link you know if those sequences of points were linked together you'd get your contours so i guess let's start with drawing the contours just to kind of show that so there's a function called draw contours as you might imagine and we're going to draw them on our iphone and we're going to pass contours this contour idx is a value you can pass in our case we're going to pass it negative 1 and that just means give me everything there's the other values you can pass i believe you can basically change the structure that you're sent back mcv scalar color so this is this is how the contours are going to be drawn what color they're going to be drawn with so we're just going to do a new mcd scaler and let's do green remember this is bgr so green the second byte and then this last value is a thickness um we're gonna do two okay so now if we i'm show our iphone again with contours this time alright so now we will see our threshold pick and our iphone pick okay so you can see um it grew it drew a green line where our contours are so um yeah pretty cool algorithm um it gave us our hierarchical structure like i said this is a child of this which is a child of this etc um so now that we have that information let's actually use that information to then uh color inside of our contours so the way we would do that would be cv invoke dot bill poly as in fill polygon and uh the values we're gonna pass are our input image um the input output image so whatever you pass there is going to be the input and also where the output is going to be stored we're going to pass in our contours which is our input array of points we're going to pass it a new ncb scalar and in our case let's do we'll just do a nice blue and then that will fill in our contours with uh our color so we run it here and we see a blue iphone um the the contour lines are still drawn on it without it you know it may look a little more natural i will say it did miss this area here uh and that's because under threshold pick that area is missed you could again you could tweak that a little bit and you could get rid of these light values out here and get rid of that and you could you know have a little more defined color but hey we just painted an iphone you know so all right in this video i gave you some more of the fundamentals that you need on your computer vision journey but what comes next well next we're going to talk about a tool that can help you solve some interesting problems and has some very cool applications it's called template matching so lots of exciting stuff to come with opencv if you'd like to see more of this type of content please leave a like and subscribe and let's get right into the next topic template matching

Channel: Programming With Chris

Views: 3,672

Keywords: Computer Vision Programming, Computer Vision tutorial, Emgu, Image Processing in .Net, Image Processing in C#, Image Processing in C# Tutorial, Object-Oriented Programming, OpenCV, OpenCV in C#, computer vision, image processing, opencv, opencv tutorial, C#, csharp, Introduction to Computer Vision, Computer Vision Tutorial, OpenCV Tutorial, OpenCV for Beginners, Introduction to Computer Vision and Image Processing, Image Processing, OpenCV Image Processing, intro to OpenCV

Id: wuQsW-LZ3kw

Length: 28min 14sec (1694 seconds)

Published: Sat Jul 09 2022