OpenCV 3 KNN Character Recognition Emgu CV 3 Visual Basic

hello again everybody and welcome back this is the third video in our six-part video series covering in OpenCV three character recognition and license-plate recognition and in this video we're going to perform character recognition in Visual Basic so let's go ahead and dive right into it so we're going to go to G ithu BMI C microcontrollers and more and take out the spaces and then we're going to go here and then we're going to go to repositories and before I go any further I should mention a prerequisite for today's video is definitely you want to check out this repository here open CV 3 Windows 10 installation tutorial and then I've linked to a playlist here and you'll definitely want to check out this video here as well open CV 3 Windows 10 installation tutorial part 3 visual basicnet and c-sharp with M go C V so continuing on with today's project we're going to go to open C V 3 KN n character recognition MgO C V 3 Visual Basic and let's first take a quick look at the directory contents here so let's first look at dachshund presentation and can an overview dot PNG and this is a slide I made trying to give a quick overview explanation of the knn algorithm and for those interested in further detail on this you could internet search on some of the various terms found on this slide and the slide itself of course just provides a very brief overview but I like to stick to an implementation focus in my videos rather than to get hung up on theory so I'll leave it at that for now and then we can also take a look at classifications and training images data structures and this is some of the data structures in the program today we'll get to the code later and then form design here these are the two forms that we're going to make today so to perform character recognition first we're going to make a project generate data with this form that's going to generate the data necessary to perform the character recognition and then we're going to make a second project in Visual Studio that is training tests and then this will be the form that will make for that so I'll go ahead and open a separate window so we can have the form design document separately and then we're going to return to the same repository here so see v3 cannon character recognition and three visual basic and the first project we're going to make today Jen data we can put all the code in the main form and then the second project we're going to train a test we're going to have some code in the main form and then we're also going to have a separate class as well contour with dated Phoebe and if we take a look at these images down here training characters dot PNG this is the image we're going to train on and as so as you can see in five different fonts here we have the number zero through nine and then the letters A through capital A through capital Z written out and then once we've trained on that we're going to test on test one two and three so test one is abc123 test two is the EF four five six and test three is going to be XYZ seven eight nine so let's go ahead and fire up Visual Studio and then we're going to go to Gen Data frm main and then we're going to build the project and let's see so to do that we're going to close out here and then go to file new project and we're going to go ahead and put that in documents Visual Studio 2015 and projects nice and empty they're ready for us so we're gonna go ahead and copy in the project name so at this point I'll pretty much just be copying and pasting out of this code here that I had in the screen just a moment ago so we're gonna choose Visual Basic and windows forms application and then Gen data and choose your preferred directory uncheck those and choose OK and once the project comes up the first thing I suggest to do is to rename the form so we'll go ahead and resize that and save and then we're gonna right click on the form name and choose rename frm main and enter and then it's going to ask us do you want update all references yes of course we do and now if we choose save and then run it there's actually one update one reference rather that Visual Studio does not automatically update for us so we get this error here so we're going to take form one here and we're going to replace that with frm main and then save application designer and close out of it and now we'll find that the program will start for us there's our form so then we're gonna do close out of that and close out of that and then the next thing is to set up our M goo references and again this is covered in detail in that tutorial video I mentioned earlier so please see that if you're unclear on any of these steps here so we're going to go to project and then add reference and then browse and then we're gonna add those dll's and then add and then we're gonna go to ok choose ok of course and then we're gonna go to project add existing item and then we're gonna go to this PC and then see M goo and windows Universal 3 and bin and then x86 and then we're gonna show all files and then we're gonna highlight all those and choose add and then we're gonna choose the K we can minimize references here then we're gonna choose these for dll's and then we're gonna open up properties and change copied output directory to copy always and go ahead and save it and now we're ready to add the components to the form so as we do that we're simply gonna work from the code here we can copy and paste the name straight out of here and then we're also gonna work from this form design document and this is the form that we're gonna be making so I'm going to slide these over to the other two windows and that's what I'll be working from as I assemble the form here so first thing is to place the components on there so a table layout panel so if we're going up the toolbox might as well bring up the properties window as well so if we go to let's see containers and table layout panel and then we can go ahead and make that a little bit bigger and we're actually going to size it to fit the form later but just to give us something to work with here at design time we'll go ahead and make that a little bit and we might as well paste in the names as we go so table al panel and then the next one's going to be button open training image and that I believe goes in the top left square yes it does so common controls button in the top left and that's button open training image and then label chosen file is next here copying and pasting out of the other screen so label goes there and then that's label chosen file and then our text box is going to go here and we're going to name that text info and then just one more component Open File dialog so we're gonna go to that's not in common controls rather we're gonna go to dialogues and open file dialog and then we're gonna name that oft open file and then we can go ahead and save and that's all the components we need to add I believe so we can close out of the tool box and now we can set our properties so if we choose the label in the button let's go ahead and make the font much bigger let's say 12 and then you can resize the button here just a little bit so the text isn't hidden but we're going to auto size that momentarily so for the text box we have some properties to set here so a word wrap let's set that to false and multi-line let's set that to true and scrollbars let's set that both and then we're gonna set column span to two and then doc to fill and that should jump across the two columns there we go and let's see we set the font we did not set the font for that yet so let's go ahead and set the font for the text box to 12 and then courier new so it's a fixed width font and then she was okay and now for the button and the label we can set the anchor property to left and right rather than top and left and let's go ahead and set the text for these here so for the button we're going to make this we're gonna make the text for the button ope n TR AI ni ng I ma GE and for the label we're gonna make the text blank and as well as blanking the text we're going to change the text aligned to just left rather than top and left and let's see if we choose the button here if we go to auto size true and then size mode grow and shrink and now if we choose the table layout panel and then we go to this arrow at the top here and then edit rows and columns and I believe we change the first column and the first row both the auto size there we go and then we're going to choose the table layout panel again and then we're going to choose a dark and make that fill and I think we're all set as far as our form here let's just run it and go ahead and check so here's our form we can move it around it will resize should maximize and minimize there we go so we're all set button doesn't do anything yet but will remedy that momentarily so the next step to do mm-hmm is we're going to look at the code here and we're going to check for any events and I believe the only one is the button click and there we go yes so we're going to make the button click event and then we're going to shoot down here and copy and paste in all the code that Visual Studio doesn't make for us automatically so if we choose the button here let's make sure we've named it BTN open training image yep so now we can just double click on the button and that'll start to write the button click event for us and we'll just adjust the spacing here a little bit so I'll go ahead and save that and then we're gonna go back to the code here and copy and paste in the rest of the code so we're gonna paste in at the top and then we have some module level variables it's actually just three constants to paste in and there we go and then we're going to grab this comment line for the beginning of the function and then the body of the click event function which goes right there and that should do it for that before we run the program though here's the project directory so we're in gen data currently so there's our project so what we need to do is we need to go to let's open up another github here and then we can go to whoops not that we can go to github and then repositories and that same repository we've been in so far and if we choose download zip and open that's going to give us access to downloading the images for today so we're going to go to our project directory then here and we're going to copy training characters into our project directory and I believe we can go ahead and fire up the program if everything's all set so open training image and then we're going to go to the current project directory so let's see this PC documents Visual Studio 2015 projects gen data and training characters dot PNG and let's take a look at these images here so this is a threshold image of just a moment here back to the github repository so if we go back one more screen here so training characters dot PNG so here's training characters that PNG as it is in the repository so this is simply a thresholded version of that and this is that same training characters that PNG image only as we're typing in the characters here which we're going to do to train them we're going a red box around the current character and then the other two boxes here this character here on the left is a crop of the current character and then this on the right is the crop of the current character resize to a width of 20 and a height of 30 so I'll go ahead and call it the characters as I'm typing them in here and you do want to bear in mind that uppercase characters and lowercase characters are of course different ASCII numbers so for sure you want to hold down shift as you're typing in the letters to type uppercase characters and then let off a shift as you're typing in the numbers so a holding shift now zebra Tango Robert Palmieri Frank Edward Delta Bravo alpha Yankee x-ray whiskey Victor uniform Sierra Quebec Omega Nancy Lima kilo Juliet igloo Hector George Charlie and then letting off of shift nine eight seven six five three two and zero and four and one and now at this point I'm going to fast forward because you don't have to take the time to watch me type in the rest of these nine eight six three two one and zero and now we get the message training complete file writing done so we can go ahead and choose the main form and close out of that and that will close all the windows for us and now we're going to go back to the project directory and if we go to bin and then debug we're going to find there's two files in here that we just wrote in the course of performing that program and those are classifications and images way down here so let's just take a quick look at that classifications is the ASCII character numbers that we typed in as we're typing them and you you might not know it from looking at how dotnet encodes them but these letters actually are the asking numbers that we typed in and then images is the image data and of course this is you know the pixel by pixel data for all those images each crapped-out character so this file is quite voluminous but that will complete what we need to achieve for the first program gen data so now we're going to do training tests so if we go back to projects here and then we can close out a visual studio and now we can go to the spec to the repository here so we have two files here training test frm main and let's start with that one so first we'll make the project so go ahead and fire Visual Studio and give that just a second to start up so we have the main form and then the class that we're going to add also so we're going to go to file new project and then Visual Basic and windows forms application and we're gonna call this train and test and save it to the usual location uncheck those choose okay and then we'll do the usual form renaming first and then setting up our references and give Visual Studio just a moment there we go so resize the form save and it's just resize a little bit more and then right click on form 1 rename frm main and yes we want to update all references when it asks us and then we're gonna go ahead and run it and we'll get that one usual error they may resolve this in a future version of Visual Studio and make it update all the references in one shot but that's ok it's not a big deal until they do that so we're simply going to name that if I remain and then we can save application designer dot VB and close out of that and now the program should start for successfully and there we go and the form can be resized and moved around and everything so let's go ahead and set up our references so let's see if we go to project add reference and then browse and browse and then we're going to choose all those and then unselect Visual Studio ten twelve and thirteen because we're using 15 today and then add and then okay and then we're going to go to project an existing item and then we're going to go to this PC see M GU Windows Universal three then and x86 and then we're going to show all files and then we're going to highlight those and choose add and then we're going to slide down here in solution explorer let's minimize references so then we're going to choose the four DLLs we just added and then we're going to go to properties and copy to output directory copy always and save and then we can open up the toolbox and we can start adding our components but before we do that let's make the class that we're going to make so if we go back one here we're going to go to contour with data dot V B so we can go ahead and copy the name here and then we can go you can either right click in here and go to whoops got a left click on the project name then right-click and then go to add a new item or you can go up here project add new item it doesn't matter which either way choose VB class and then we're going to name that contour with a TVB so there we go we're all set and then we're going to add that and then we can go to contour with a TVB here we're on the github repository go to raw and then simply ctrl a ctrl C highlight everything and ctrl a ctrl V paste it in there and then we're going to go back to the form design and let's right click on the form here into a view code to bring that up so we'll just sort of shuffle this order so here we've got the designer then four main stat a few spaces we'll get to back to that shortly and then there's contour with data which we've already pasted in so let's get back to the form design and go ahead and assemble a form so this is going to be pretty similar to what we did previously working from the same form design document first we did generate data in the previous program and now we're going to do train and test in the current program so now we're going to build this form again very similar to the previous form so we're going to then go back here and have the names to copy and paste for each of the components we're going to go to train and test frm main VB and then we'll just be copying and pasting the names out of here so if I remain obviously is already added so next is table layout panel so we're going to go to containers table layout panel and then we can go ahead and just reposition this a little bit of course we'll Center it to the form momentarily and then we're going to go to the name and table layout panel and then we're going to go to adding the button next so let's see close minimize containers here common controls button goes there and but an open test image and then we're gonna do label chosen file and that's going to go right there label chosen file and then we're going to add the text box on the bottom so that's gonna be down here text box and then we're gonna add that text info and then we're gonna add the Open File dialog box so let's see that's gonna be in dialogues and Open File dialog go ahead and name that as well and the only event in this program just as in the previous one is the button so let's go ahead and make sure we've named that button open test image and we can double click on that and that will start to write that event for us and we'll come back to the code later of course first we're going to finish our form so let's go ahead and close out of the toolbox you don't need to add any more so this is going to be very similar to the previous video as far as setting the properties so let's see here font for these we're going to set to 12 and then we're going to resize the button actually let's just directly choose here auto size through and and grow and shrink and we can go ahead and put in the names here so that of course is going to be open OPM tes tima je and then we'll set the anchor properties for these both here the button and the label to be left and right and there we go and then for the label we're going to set the text property to blank and then text the line to left only not top and left and now we can set some properties for the text box we're gonna set the font to 12 size of 12 and the front itself to courier new and then we're going to set let's see column span to 2 and then dr. Phil and then we're gonna set some properties at the bottom here word wrap to false and let's see multi-line where's multi-line set that to true and then scroll bars we can go ahead and set that to both and we should be all set there so now we're going to choose the table out panel and then we're going to do edit rows and columns and we can set the first column and the first row to auto size and we still have the table layout panel chosen so we can go to doc and then Phil and I think that will do it for the form let's go ahead and give it a quick test so the button doesn't do anything yet but that's okay the forms working great for us and of course it can resize and everything minimize maximize etc so it looks like the forms all set so the next step that we're going to do is we're going to go to frm main VB and then this is back to the github repository here we're going to go to raw and then copy in all the code that isn't there already so let's see here we can copy in at the top here and then get rid of some of these empty spaces and then we're going to add the module level variable section it's actually just two constants and then we're going to add this comment line just above the button click event and take out the space and then we can copy and paste in the body of the button click event and I believe that's it there we go and that should be all set okay no red underlined so that's looking good for us so then the final step is we're going to go back to the project directory and so that's going to be training test this is a project we just made now we're going to go to that download zip directory and then we're going to copy the three test images into our project directory and we no longer need these zip so we're going to close out of that and the next step or actually the final step before we run the program I should say is if we go back to Gen data and then then you'll recall and then debug this is where classifications and images are so then we're gonna copy both of those and then we're going to go to training test then in debug and then paste them into training tests here okay so and again that was in projects training test bin and then debug you have to put those two in the same directory as the executable or the program won't see it at runtime so I believe that's everything so we can go ahead and test the program now so if we go to open test image and we're in the current project so projects training test so we can open test one and there we go so here's our image abc123 and here's our text as you can see these are actual text characters that we can copy and paste for example into notepad so that's worked out successfully for us let's go ahead and try the second image here de F 4 5 6 d e f 4 5 6 and then X Y Z 7 8 9 and X Y Z 79 so we successfully completed character recognition in C++ Python and Visual Basic so the next three videos are going to be license-plate recognition in C++ Python and then Visual Basic so I'll see everybody in the next one

Channel: Chris Dahms

Views: 15,903

Keywords: OpenCV, Visual Basic, Software, Tutorial

Id: asKDXj18MEs

Length: 22min 33sec (1353 seconds)

Published: Sat Jan 09 2016