OpenCV tutorial 5: Emgu CV with C#

welcome back everybody in this video we're going to take a look at em grow CV for those not familiar with em goosey meet em go CV is essentially a youth set of wrapper functions that allows calling open CV functions from a Visual Studio or Windows forms forward this is necessary because visual studio slash net is an interpreted environment that cannot directly call functions written in native C and C++ chemical is free and open source and we can use c-sharp to make a nice graphical front-end let's get started alright so to get started today let's download c-sharp 2010 Express and we can find that easily on the Microsoft site go ahead and download from this link here and when you do the install you can choose all the defaults that will get just great for you alright so now we've done a little bit and installed c-sharp 2010 Express now let's download the latest version of M glue CD and we can go on the main n glue page here download an installation and then follow this link to the file server I'm the source for each page and let's go ahead and get the latest version two point four point zero and even though we'll be using a 64-bit computer today the 32-bit version still seems to work great and it's a little bit more stable tested than the 64-bit version so let's stick with this download here live CC be windows x86 to point 4.0 point 1717 Exe and go ahead and download that and save it and once you saved it go ahead and run it and choose all the default options to complete the install so before we go any further now that we've installed c-sharp 2010 Express and MVC be to point 4.0 I should probably bring something to your attention here if we go back to the younggu page and then we go back to download and installation there's a section here on troubleshooting the installation specifically if you get a certain error the type initializer for M godet CDN bulk through an exception if you get this error at runtime you come back and follow the instructions on this page especially probably the first thing you should do is download this ms v CRT 9.01 service back one redistributable and then retest and what this is is it's a redistributable portion of microsoft visual c++ 2008 it's not the entire microsoft visual c++ 2008 download at all it's very small download this is directly from the Microsoft site so it's perfectly safe and the reason for this is that some parts of MgO cv were written and compiled and microsoft visual c++ 2008 and therefore require this even if you're using 2010 now already so now our downloads and installs are done and now we're going to add to our path variable so if you stuck to the default installation directory your install is going to be on the C Drive and then M go and go windows x86 to point 4.0 point 17 17 and then what we're going to add to the path variable is going to be bin and then x86 so it can simply click on peer control C to copy that and then if you're using Windows 7 Start control panel system advanced system settings environment variables and in the system variables section so it applies to all users go to path and then edit scroll to the end a semicolon and then ctrl V to paste the directory that we just copied a moment ago and ok ok and ok and it'd be good idea after all those installs and modifying the path variable to go ahead and reboot all right so now we modified our path variable and rebooted and we're ready to start Visual C sharp so we're going to go to file but before we get started here I should probably mention under tools and settings make sure you have expert settings chosen otherwise it's going to hide a lot of the menu features from you that you're going to need to access so we're going to go to file new project and windows forms application and let's call it cs4 c-sharp tracker six sounds good and then okay one other thing I should mention before we get too much further is it's a good idea to go to tools and then options and by default show all settings this is unchecked and it hides a lot of the settings from either way it's a good idea to check this so they all appear for example the windows forms designer choice does not appear with this not checked and that has a lot of Handy features to it for example the layout settings you can choose to enable the grid and snap to grid feature and having the grid on your form makes it easier to place components all right now there's three steps that we're going to do to configure both our c-sharp program and the c-sharp environment for our MgO install and the first one is we're going to go to project and then add reference I should mention that this project menu is context-sensitive in other words the options you have chosen on the solution Explorer on the left for example if you have the solution versus one of the parts of solution chosen the full project menu might not appear so you definitely want to choose your project and then go to project and add reference and then when the window comes up here we're going to go to browse and then we're going to navigate back to our I'm going to install again so that's going to be C on blue x86 Embu and then we're going to go to the bin folder and you'll notice that there's nine dll's here that start Embu and end in DOL depending on if using Visual Studio 2008 or 2010 you're going to choose one of these and then the other set up so you're going to choose eight total of the night so since we're using 2010 in this example we'll choose these 8 and then okay and it's going to take a moment to add those references now that we've added the references you can see they appear under references here on the left and the next thing we're going to do is add some existing types so now we're going to go back to project and then add an existing item and now we're going to navigate to the UM go install again but this time into the bin directory so I'm GU and we windows x86 then and x86 there we go and so now we're going to you see at the moment nothing appears here that's okay go down the bottom right here and choose to show executable files now we get all the files are during this directory and what we're going to do is we're going to choose all the files that start OpenCV underscore and end in DOL so from here down to here all those and again it's going to take a moment to add those and the next thing that we're going to do is we're and highlight those 17 existing items that we just added and then we're going to go to properties and we're going to change under copy to output directory we're going to change that to copy holdings and let's go ahead and save before we go any further tracker six tracker six create directory for solution that's good go ahead and save and the final step that we're going to do is we're going to customize our c-sharp environment by adding the special angle controls to the toolbox that we're going to add to our form eventually so let's bring up the toolbox here and what we're going to do is add these four special controls here especially the image box control you're going to use this on probably every younggu a c-sharp program you ever do so at first when you start C sharp these controls will not appear here so I'm going to right click on my toolbox and go to reset toolbox yes for sure one reset and give it a moment here and so this is probably what your tool box will look like when you start C sharp so in the all windows forms there's this control down here picture box you don't want to use this you have to do a whole bunch of conversions of the images from OpenCV to be able to draw them in this Microsoft control picture box rather what we're going to do is we're going to add the impose specific controls and here's how we're going to do that so expand the general area if they haven't already and then right click there and go to choose items and when it comes up we're going to go to we're going to go to browse in a moment but first let's take a look at our list of controls here and we're going to see if we look for image box image image button image list no image box though so we're going to go to browse and then we're going to browse back to our MgO and so again and then the bigger entry and what we're going to do is we're going to double click on this file here m gu c v dot UI dot dl l and now if we look for image box sure enough it's there but it usually checks it by default if it doesn't make sure you check by image box and then press ok and there's our four fancy controls and we're going to move image box to the top here and then the general section to the very top since those are very commonly going to be used and let's read expand all the windows forms and now our tool box is all set all right it's now ready to start placing components on our form so i'm going to use two lumens boxes and we're going to try to be as consistent possible the previous tutorials and we'll resize those in a moment or else it in put a button down there and that'll be a positive resume button just like the previous video and also a text box all right so let's bring the properties window back up and let's go ahead and name our control source so to be well rear gate image box IB and to be consistent with the previous tutorials we'll call the first one I'd be original and we'll call it the second image box IV processed and call the button BTN posi resume and we'll call the text box txt XY radius let's go ahead and set some properties here for the to leverage boxes if set sizes individually set those to some sorting by 480 and now we'll set some properties for the one and the text box so let's make the button text by default the video is going to be playing to start so we'll change it to the option of pausing and let's make fun a little bigger so we don't have to squint let's go to 12 on there and then for the text box we have a few more properties to set let's set the font to 10 so we don't have to squint and it's best to pick a fix with not because we'd like our numbers to display with the columns lining up when we're looking at the X&Y coordinates and radius so let's choose a fixed-width font courier news of the choice and then we're going to need the text box of course to expand the multiple lines as multiple lines of data fill in so we're going to choose multi-line to be true and now we can precise it and read only we're also going to set to true since this will be ideal it'll allow the it's going to grayed out and it's going to let the user highlight text that's in here if user wanted to copy and paste it into something else to save for later but at the same time user won't be able to edit the contents and then for scroll bars we're going to change that to vertical and say go ahead and do a final resizing here and the last thing we're going to do on our forum here is work on to double-click the forum to have the environment right to form onload forests or at least part of it and then we're going to double click the pause button to have the environment start to write the plasma and click force all right now we're ready to start code so we're going to start with the are using statements so we're going to be using I'm going to CV m gu cv cv a new MVC B structure and then we'll see a user interface now we're going to add some member variables again these are going to be very very similar to the previous programs capture object and we'll initialize that to note and we're going to have a boolean very well this is going to be a little different than the previous programs we're going to call this capturing and process well initialize that to false and that might not be especially clear at the moment but we'll get to that shortly now we're going to have our two image variable so if you're new to c-sharp the syntax to these might look a little strange at first but you'll get used to loose in practice so we're going to say image and then bracket BG or so but then M go C V we still stick to the BG our color order rather than RGB just as with OpenCV and white of course it specifies an 8-bit color depth for each of the blue green and red portions of each pixel and we'll call that image original and we'll be able to display these in blue CV image objects directly on the image box controls that we've already added to our form no conversion will be necessary as in the previous program and now we're going to declare the processed image and that's of course a grayscale image also a bit depth and now we're ready to move on to form lope we can skip right on by the constructor and start writing the form load and here's how we're going to get the webcam we're going to use the try catch block again for you notice a sharp this might seem a little strange but do get you saw Wilson practice so try cap webcam this design new capture it's coming as we go here so see you capture object to the default webcam and then we're going to have cache block and for that we have to make the parameter a null reference exception and simply call that except - err it's not successful and will simply show as what the previous program in QT will show that on the text box so text next one radius text is assigned and we can use a property of the exception object here except to navigate that message shall we object message textbox and of course we have to return otherwise the program in crash glad to continue there so let me get this far and we only have a good capture object then we're going to add a line here and again this is going to look a little strange if you're not used to see sharp we're going to call application label and then we're going to increment that apply and this is a function we're going to write later so it's going to be underlined red for now but that's okay process green and update GUI to be consistent with the previous program and what this line is going to do is it's going to add the process image function to the applications list of tasks so let me explain that a little bit so the application dot idle property or what happens is the application has a list of sort of errands it has to run now resize the form you know if the user chose to resize the form handle one press events all that sort of thing and then it has to bounce control back and forth to the operating system so when the application has control of the processor it shoots down its list of errands and then gives control back to the operating system of X so what we're doing here with this application title and then increment is we're adding our function here that we're going to write a little bit later press SRAM and update GUI we're adding that to the applications list of errands and then when we want to remove that from the list of errands we're simply going to use the minus equals operator instead of plus equals operator to remove it we'll see that later on when we get to the pause button so the previous project we use the timer control that will be one option here and another option would be we could use multi-threading but this one statement essentially achieves really what we need to the application is going to give as much time as possible to processing the frame and updating the front-end and then it will also be able to work in its other tasks and then bounce control back and forth the operating system is need B so this really is an ideal way to handle it both in terms of the time sharing and also in terms of how easy it makes it for us to write the program and one more thing we have to do in conjunction with that is we're going to update our member flag variable or state variable call when you prefer since capturing is now in process we're going to change that to true and then comment that update memory buddy very well and that's going to get perform one mode so now we're going to write for one closed this is going to be a very short function force private bully form one underscore form close and we have to pick certain parameters here for this function because it's in conjunction with the c-sharp environment so we're going to even know what I actually can use these parameters in the function that's okay we're going to have object sender and then form closed event args and we'll simply call that e again when I can use this a really doesn't matter and we're going to say if cap webcam is not normal then we're going to call the dispose function for Kaplan Kim and that's it honestly this function might not even be necessary I think the operating system would probably handle that for us anyway but it certainly doesn't hurt to put it in and now we can move on to the longest function in the program process frame an update group and to make this a function that we can evaluate and remove from the application that idle property we have to choose these as the parameters otherwise it won't work the compiler would fly it but that's okay we can simply put those objects in there and not use them all right so now we're ready to start our function this is going to look very similar to the previous programs there will be some syntax different systems we're now using MVC B but the logic at least will flow the same so we're going to have image original is going to be assigned cap webcam to get things started here and query frame is the function that's going to get the next range and then we're gonna have an error check line here so the program doesn't crash in case say somebody accidentally kicks the cord to the webcam and then it gets disconnected while the program is running so we're going to do this page original is no turn and our error handling something we're ready to start processing the images image original so with the simplest syntax this is really nice we don't even have to declare or initiate image process we can simply return the result of image original dot in range and that will go into the process and in range since we don't have to pass in you recall in the original program in range took four parameters we had to pass in the original image and the processed images the first and last parameters and the men and max filter values for the two of the middle here since we're calling the function from the original image and we're returning to process image we're only going to need two parameters to pass into the function and those are going to be new and we can declare new BGR object and we'll search for the same red ball is in the previous video so this can be 0 0 and 75 and that's going to be the minimum filter blade and second parameter the next caller ID and now we're going to call these smooth Gaussian function and here's how we're going to call smooth Gaussian once again smoothing a scene is going to return something just like in ranged it so we can simply say image process is assigned in this process that's cool gasps whoops go underscore at the front of that and if we pass in only one number here it's the swill gassing function it's going to take that as well the X&Y size of the filter window now the next function is going to be a little unique to M gu in terms of the syntax we're going to use the embassy B circle F class to store our circles in this program and here's how we're going to declare that we'll simply call the array of circles will call it circles and that's going to be assigned the result of the image process got two circles and these parameters are going to look relatively similar to the previous programs the order is a little different but essentially the data that were passing in is the same so the first round will be new great hundred and that's candy threshold and second parameter is going to be new grade 50 and that's going to be accumulation and the third parameter is going to be the accumulator resolution actually the size of the image divided by this parameter is going to be the accumulator resolution so we're going to use two for this parameter and please consult the MgO CD and open CD documentation as applicable for information those three parameters that'll give us a nice balance of at least detecting the ball but not over detecting false objects in our image and the fourth fifth and sixth parameters will be a little bit more straightforward here so the fourth parameter is going to be image process height divided by four and that's simply going to be the mailing distance pixels between the Centers of the detective circles and almost there for this function we're going to pass in ten-year that's going to be the minimum radius of the detective circles and then 400 will be our last franker and then we're going to pass in the index 0 so the 400 is the max radius the detective circles and the index 0 specifies to get the circles from the first channel and now we're ready to begin for each loop and the syntax of the for each statement is really essentially the same thing is it's doing the same thing as a for loop that's just a slightly more convenient way to do it taking advantage of the fact that we can iterate through objects in c-sharp so we're going to say for each for each all being one one word there that is a key word in c-sharp good circle up circle in circles so as we step through the circles object which is on type circle circle F declared up here within the for loop we can refer to each circle as we're iterating through it with this variable here circle so the first thing we're going to do is the output to the textbox and we're going to take advantage of some formatting functionality in c-sharp to assistance with that so first we're going to say if text XY radius that text is not equal to a blank line then we're going to add a new one so to see X already is that a clean text environment that you want let's explain the logic a little bit the idea is if we're not on the first line in text box insert a new line character so the idea here is the very first time a circle is detected and then we're going to display the result when we come in the text box is going to be blank so that first time through this if statement is not going to execute and we're not going to insert a new line character every time after that when we get to this point there'll be something in the text box already so this statement here will about will evaluate to true because in other words it won't be empty so then we're going to add a new line and that's going to kick us down to the next line in the text box so that way we're not writing on same line over and over that's really just a minor formatting detail now we're actually going to in our next statement here we're going to add the X Y and radius positions to the text box here's how we're going to do that txt XY RS and you can be assigned a pen text ball position x equals and again this formatting is going to look a little similar to the previous video the functions are slightly different and QD versus NC sharp to G formatting in a text box but it's really the same idea so we're going to circle that Center that X that tooth string dot pad left and we're going to padded it for characters so that's going to do is for example let's say the pixel location that expects the location of the center of the circles it picks up five we'd like it to pass into the box at the text box three blank spaces and then the number five if it's five hundred we'd like one blank space leading and then five zero zero and this line is going to achieve that force that's going to be the x position of the center point and next line is going to be pretty much the same thing but for the white circle dead center that why that two string ad left for Y position with center one circle and now the last line is going to be a little bit more involved because the radius can be a floating-point number it's going to start at the same circle that radius to string and here's what we're going to do the pound character specifies if there's data put the data there if not what in conjunction with the pad character specify a blank space so for ample three pound characters and then the decimal point which is always going to put and then the 0 character specifies if there's data there put the data if not put a 0 character and then can left and we've had that itself the idea is we'd always like three spots before the decimal and three spots after so for example say the radius is 50 we would prefer that to display as blank space five zero decimal point zero zero zero if it's fifty five point five five we would prefer that to display as blank space five five point five five zero and this line is going to achieve that for us now let's knead up our spacing a little bit here and now we're going to have our last one one in which we're simply going to drop the carrot down to the very bottom of the we're going to drop the scrollbar rather down to where the carrot is currently located which is going to be the line we just wrote which is going to be the bottom of the text box so that way we're always going to see the bottom of the text box otherwise the scrollbar would stay at the top and the data is being written in wouldn't be scrolled down too so that will simply be text XY radius scroll to carrot and now we can call our two functions to draw circles now the first of these is going to be a little bit tricky but in a sense it's good that there isn't a function that directly does it in embassy be using the opencv two functions because that's going to give us a chance here to look at how to call one of the opencv one functions from within and even though it's not very often done it's nice to know how to do it if you need to so same as in the previous program we're first going to draw a small green circle at the center because we detect it outfit okay so to do this we're going to call the open CV one point X function okay this is necessary because we're drawing a circle of radius 3 even though the Desai's the size of the detected circle is going to be much better and the reason this kind of helpful to illustrate this is because the CV book object can be used to make open CV one point X function calls so here's how this thing is going to look we're going to call CB dot and invoke I'm sorry CV invoke dot now if you look at the functions that are here these are going to be the open general speaking the open CV want functions these will look familiar for example CV and range s is here you'll probably notice that from the very first tutorial in this case what we're going to do is we're going to call CV circle so first we're going to pass in in the journal and the reason for that is because we're drawing on the original page and then the second parameter is going to be the center point of the circle and we can you clear a new point object and we have to cast these folks to ants and now the next parameter is the whole reason we have to use this open CV one function rather than the open CV to function is because we need to pass in a three here and that's not available the open CV to function is you'll see momentarily when we draw out one circle so three will be the radius of circle in pixels and then we're going to test any new MCV scale now what this MCB is all about is you probably recognize from the first tutorial objects that start CV for example CV scaler m cv is managed meaning within the dotnet environment manage CV scalar so there's all these other managed equivalents to be open CV one objects here that you can also use as need be for future projects so manage CV scalar 0 255 0 because we're going to be drawing in green yeah next parameter is going to be negative one and indicates the thickness of the circle and pixels and negative one indicates to fill the circle and now we're going to have the unglue cv1 type dot CPA a might as well use anti-aliasing to make the circle a little bit nicer and then the last parameter is the shift but when I can have a chef so that essentially be zero and now we're ready to draw the circle around the outside and that's going to be much easier so here's how we're going to do that we're simply going to call the draw function of the image object we have so we can say image orginal that draw and then we need to pass in the circle variable in the array of circles that we're iterating through and then new BGR objects and will make that appear red and then index on the circle and pixels you can specify X 3 and that's going to in the 4-h and now to finish up function since we're going to be assigning the and OCD image types to the image box control which is also p.m. go see being understood we don't need any conversion here at all so this is going to be as easy as to be simply can have I the original image is assigned IMG original and then let me have ID process dot image is this an ING process and that's going to complete our process frame and update GUI function and now we only have one question left in the program right button pause or resume click event and this is going to be a relatively short function so the first thing we're going to do is we're going to check if that number level variable or flag variable capturing your process if that's true so in other words is currently processing an image that would mean that the user just shows pause so here's where we're going to remove the process frame and update GUI from the application idle list of tasks so we're simply going to put application that idle - equals press this fragment update so remove the process image function applications list of tasks now we have to update our flag variable or no longer capturing in process you know we also have to change the button text to indicate resume so the user knows when they next press the button it's going to resume and now let's say else so you're not currently classes in image that means user must have just chosen resume should be chosen chips these are just chose yours in so now we're going to add press this Raymond update going back to application that I've a page title plus equals that's framing up the goon again that's this line here is of course identical to back in the formal modem and up here where we added us to begin with and now we're going to update the flag variable will now perform the capture process again and update the button text as well so because you knows next time press the button and pause it and then that's the another function and that ends our class and our name space and I just realized I forgot a bracket here and there we go now let's save our program in tracking 5 so we're going to go to rebuild solution and hey look at that one succeeded on the first go and now we're going to choose start without debugging and here's our nice form and there we go it's working right for us so I'll press pause I'm still moving the ball at this point but we can see it's pause it nicely and there we go and we'll try the pause one more time and that's what we write for us congratulations you now know how to configure c-sharp for use with open CV and also how to write a basic object recognition and tracking program with a graphical dotnet front end and making em code CP function calls this video concludes the open CV install configure and basic object recognition and tracking program video tutorial series for upcoming videos I'm not really hundred percent sure yet to be honest probably I'll do the USB motor controller example next I know I've gotten a few requests for that after that I'm not really sure post in the comments if you have any particular area of interest let's see everybody next time

Channel: Chris Dahms

Views: 203,523

Keywords: OpenCV, tutorial, 5:, Emgu, CV, with, C#

Id: vdjoutNR2DQ

Length: 54min 33sec (3273 seconds)

Published: Fri Jun 15 2012