Emgu CV Hand Gestures Recognition Advanced Tutorial

miss Liu Goodell Tongo and I'm an Italian their science student in this computer vision tutorial I will show you how to detect and recognize and adjusters using an open CV wrapper called mu CV I want to share with you my code because I've seen here on YouTube a lot of videos facing a similar problem but at the end none of them show how to solve it in a practical and useful way before starting I also want to thank you for the great feedback I have received since the publication of my first mu CV tutorial so before diving into the C sharp code I think that it's better to have an overall idea of the possible methods to do ant action and just a recognition the one I have used and that I will show you in this tutorial is based on a first step that involve skin detection and a second step that involves convex hull and convexity defects computing OpenCV efforts out-of-the-box and adaptive skin detector since the latest OpenCV 2.0 version but to be honest its performance are not so optimal so i have decided to implement classical colorspace filtering to detect skin pixel this strategy consists on localize a proper sub region of for example h sv or y CR C B color space by applying a lower and upper bound on the possible values of each channel once we have obtained proper skin masks we use some morphological operators like erosion and deletion to remove some possible noise and to obtain a smoother and cleaner mask of our hand our binary mask image then is used to extract the biggest control and compute convex Al and convexity defects this information are then used to define some logical condition that for example represent open or closed and all that can allow us to count the number of our fingers this approach is not suitable for a real scene seanny scenario it needs more testing but it's a good starting point for your own custom project and by the way show some mu CV best practice that can boost your performance so now it's time to fire up with the studio and take a look at the running damn okay so let's start our demos on the left side you can see my hand open and on the right one you can see the skin pixel binary mask starting from right from the right side binary mask contours are extracted and then the biggest one that should represent my hand is drawn using the lighter green color the light blue line that you see around my hand is the convex hull red dots on my fingertips are computing using convexity defects so now I will stop my demo and show and using an online resources convexity defects and convicts are as you can see from this picture OpenCV documentation refers to convex hull and convexity defects information and it is pictured a Dirac the red line is the convex hull and these spaces between the convex hull and then contour are the contacts of defects convexity defect and class has a three useful information one is the start point one is the end point and one is the depth point as you can see from this simple description so let's take a look at the solution the solution and just a recognition is composed by a true project the first one and just a recognition is a classic windows form project that contains the guy while the second project it's a classical library project that includes some classes to detect skin pixels using various color spaces for example you can see that I have defined a skin texture skin detector 4yc RGB color space all one for HSV color space I have not defined in one skin detector for RGB color space because this is not a proper color space because it's variant to illumination so let's take a look at the contraflow of this solution first of all we need a skin detector we need a variable to track also the current frame we need a variable for day for acquiring images from either our camera or our video file there are also some other variables that are used to define the way some of the of my videos property here we can see some threshold the lower and upper bound on HSV or YC RCB colorspace convex al and convex the FX variable so in our constructor we acquire from this video our frame using our Graber variable then we define another and skin detector in this case this is the open sea biscuit Apter but I will not use these here we define some threshold for HSV lyz RGB color space these needs some adjustment in some other kind of videos they are not Universal but they are quite robust in the application idle event we attach our frame grabber event handler that compute and process each frame let's take a look at the frame grabber middle event handler so first of all we acquire our frame then we using our skin detector detect our skin let's take a look for example at the YC RGB skin detector class it's quite simple code as you can see we receive any current a frame from the as an input also I mean lower bound and an upper bound and then using enrage method of MU CV we obtain a skin mask that after we improve using erosion and dilation as I told you previously so let's get you back to the code after adding detect our skin we extract our contour and compute convex hull here the curve the code is quite quite simple we simply use fine contour on our skin mask and then we look to find the biggest contour after adding finding DB East contour that should be our hand we approximate it to make it simpler using a proxy poly function and then we compute convex hull using get convex hull method of our biggest control we then draw some useful information especially for debugging purpose as you can see I have a lot of commented code this is used for debug purpose then we filter our convex I'll point and the disease because I have some kind of outliers when because I have some convex hull point that are very very close one to each other so we filter it to have less convex hull points and then we compute convexity defects using get complex is FX function the next step is to draw and compute our fingers number as UK as you have seen in the demos here we compute the start that and and pointed at all the previously and then using some AO cast on heuristics I have defined we can count the number of our fingers let's run again the demos to explain better this part of the code the darker green line that you see between our red and the yellow point are the number of fingers this is the eristic that I have used so 1 2 3 4 5 5 fingers so at the end I'm computing the distance between my start point and between my depth point and this depth should be big enough to have and define a finger one as you can see it's quite rubbish - the movement and it's quite easy so I hope you liked this tutorial feel free to contact me or leave me some feedback here on YouTube thanks a lot

Channel: luca del tongo

Views: 116,783

Keywords: emgu cv, opencv, emgucv, hand gesture recognition, emgucv tutorial

Id: Fjj9gqTCTfc

Length: 11min 58sec (718 seconds)

Published: Thu Sep 02 2010