OpenCV Tutorial 10: Optical Character Recognition (OCR) in Emgu CV

welcome back everybody in this video we're going to perform optical character recognition on still images using the tesseract functions in MOOC V in this video we're going to use MQ CV and visual basicnet in Visual Studio 2010 Express Edition this video is going to presume the knowledge of the previous videos please view the previous open CV and MgO CV tutorials if you have not already let's get started okay so we're going to find if we type tesseract into a general internet search probably the first page you'll come up will be the Wikipedia page for tesseract which a tesseract in the general sense is this fancy three-dimensional shape here that can sort of rotate within itself this is a nice illustration here but specific to computer vision tesseract means something completely different tesseract is we type OCR a software project if we go to the Wikipedia page for it this first paragraph here kind of summarizes things nicely for us the tesseract software project was originally developed at Hewlett Packard software from 1985 through 1995 it kind of had a period of dormancy after that until was open source in 2005 and then subsequently picked up by Google to sponsor in 2006 and since then the tesseract software project is developed to become Billy's as far as I'm aware and in my experience the best general use free open source optical character recognition software package out there so that's what we're going to use today and then we'll also find if we go to the page directly and here we are for a search result the Google page for the tesseract project that the Deseret project is available in these separate downloads similar in a sense to how open CV is now the test project is not integrally included with open CV so if we wanted to have an open CV program that performed both open CV functions and also tesseract functions we would need to link not only to the open CV libraries but also to the tesseract libraries in our one program which isn't terribly difficult to do but there's some additional steps fortunately MgO CV includes a wrapper functions not only for open CV as we already know from the previous videos but also for the tesseract objects so as long as we're using MgO CV we only need all actually one additional step is all that's necessary to be able to use the tesseract object in our embassy V program is we're going to see so let's go ahead and fire a visual basic and get to it okay so here we are a visual basic we're going to go ahead and get our project started like normal here windows forms application and we'll call it OCR and that's good we'll choose okay before we save it let's go ahead and rename our form here so we'll call it frm OCR and let's make sure that there we go there's our nice name change to our form and we'll go ahead and save it so we'll save it we'll call it OCR save it to the usual location go ahead and uncheck create directory for a solution so to expand on what I was mentioning a moment ago to start with the first two steps for our linking is going to be the same as the previous video so first we're going to go to add reference then we're going to go to browse see em go I'm going to 686 then and then we're going to choose 1 2 3 4 5 6 7 those 8 DLLs and then ok and just a moment here while Visual Basic adds those and then we're going to choose show all files and expand things a bit so we can see then we're going to go to project and then add existing item and again the steps the same as the previous video so we're going to go to M GU one is x86 been x86 show all and then we're going to include all the dll's that start OpenCV and end in dll so once that include is done there here's where it's going to be a little bit different we're going to go to our M go install open up a new windows explorer here so we're going to go to CM goo I'm goo windows x86 now what we're looking for here is the test data folder so we're going to find that test data folder is in bin here and let's go ahead and expand bin and then it's also in MgO TV OCR now these two test data folders you'll actually find the size of these invoices identical as far as I can tell though these two folders are the same so it doesn't really matter which one you pick but choose one or the other so we'll choose this one and then we're going to go to copy and then we're going to go back to our project and we're going to right click on our project here and then we're going to go to paste and that's going to actually dump the test data folder and all these files that it includes into our project now this next step is especially important here we're going to highlight all our dll's here and then we're going to choose copy to output directory copy always and then we're also going to do the same thing with these test data files here copied output directory copy always if you don't do that step this copy always with the test data files what will happen is when you go to instantiate the tesseract object the program will crash and the error isn't necessarily clear that that's the setting you need to change so definitely bear that in mind and go ahead and set that property okay so at this point we're going to start adding some controls to our form this is going to be extremely similar to the previous video so I'm going to go ahead and fast-forward at this point you forms all set look nice force so we're going to go ahead and have the environment start to write the events for three of the controls for us so for the form we're going to have the environment start to write there were size event for us double click on that and that back to design view and then for the button we're going to have the environment start to write the click event so we'll simply double click button file and then for the text file the text box for the file I should say we're going to choose the text changed event and there we go so bear with me here for a few moments while I need up the code a bit all righty so now we already start coding so what I'm going to do is I'm gonna go ahead and fast-forward through the majority of the code since again it's really pretty similar to the previous projects and then what we'll do is we'll slow down when we actually get to working with the tesseract object so in the member variable section here we're going to declare the tesseract object but then we'll wait until the constructor to instantiate it okay so at this point we're ready to instantiate the tesseract object in this could make a program crash if it's not successful so we're definitely want to put this in a try-catch block so here's what we're going to do try and test is assign new tesseract and we're not going to want to use the blank constructor there because then you just have to call the init function later to set the parameters for it and you notice there's three different options here we don't really need to use this whitelist option here so we'll simply use the the second of the three so here's how that's going to look we're going to put test data now note that's the directory so the fact that we're not specifying a path here is why we have to include the test data folder in our project directly if you want to specify the path where test data the folder is located you could put it here if you like but personally I would suggest that you copy it into your project that way if you distribute your project you can simply include it the same directory and you don't have to worry about which directory it's installed to and then next we're going to put English that'll be our language and then we're going to choose among the options here you can experiment with these options if you like now but for this video we're just sticking to a basic example here so we'll simply use the default option here's how we're going to handle it if that instantiation is not successful we're going to do me dot text so in other words we're going to assign to the title bar the text error instantiating tesseract object and then to make it extra especially clear we're also going to assign the same to the text OCR text box and then to prevent the user from choosing the button to open the image file which would then crash the program if the tesseract object was not instantiated successfully we're going to disable both the file text box and the file button so that's going to do it for our constructor now next we're going to do the form code for the resize event for the form and then the button click and text file text box change event will go ahead and fast forward through that until we get down to the press its image and update GUI portion of the program again we get back to the tesseract object use you okay so now we're actually ready to use the tesseract object to detect the text in the image and this is going to be incredibly easy it's only two lines we're going to do tests that recognize IMG image that's going to be the image with the text in it of course and then we're going to write that out to the textbox so that's going to be text OCR dot text is going to be assigned and our tesseract object get text and that's it now a little bit more code to finish up our form here ready so now we're all ready to go ahead and perform a compile and there we go how about that first go one succeeded so let's go ahead and fire up our program and give it a test run and of course we've made our form code so that the image box in the text box will resize nicely as we change the size of our form here so if we press the button here to choose an image file so what I did when I was preparing for this video is I downloaded a variety of license plate images so here we have some license plates in scenes as they would actually be found on the road and then here we have some license plate images strictly of the plate only and then this last set of images here this is for word processor images actually I typed these up in a word processor and then took screenshots of the word processor and then save those to an image file and then these four here our license plate crops from some of these license plate images here so in future videos what we'll do is we'll take if you have a license plate and a scene how do you narrow down to focus in adjust the license plate and then once you have the license plate for example if you have this New York a Stonewall the Empire State a license plate of course we're only trying to read the actual letters of the license plate to identify the vehicle so that would be Stonewall in this case of course this is not a real license plate but how do we section out New York and the Empire State and then only read the solenoid part well if we go ahead and look at this result here with the tesseract object with no pre-processing of the image we're going to find that New York is not read in Stonewall what we're really trying to read is not read but the Empire State which we're not really trying to read at the bottom is read and also the slots here for the bolts are read as the letter i' which obviously they shouldn't be so in a future video we'll get to the the pre-processing steps so that you're feeding that tesseract object basically perfect characters but for this video since this is just a basic intro to the tesseract function we'll essentially start with perfect text and then we'll cover the other stuff in future videos so let's start out with these word processor generated image files here so here we have Times New Roman font going from a point up to 24 so we can see that 4 8 and 10 point that's pretty small and the tesseract object has a hard time reading that so for the eight it basically doesn't really come up with much for the ten it's closed only it reads point as paint but for everything twelve and up you can see it's dead on so that if we look at the larger sizes will also see pretty similar results here's twenty six through fifty the only imperfection here is that 450 it puts a space in between the five and the zero but that's pretty minor and then four sixty seventy and eighty the only imperfection is that the zero and the 70 and the 80 is read as the letter O and then for a hundred and one hundred and twenty point it's dead on so let's go ahead and take a look at some of our crop license plate here here's the Stonewall plate and that's perfect rendering there and then we have the sample plate and that also came out well and then these last two are a little bit more along lines of what an actual plate would be so here we have this six seven nine plate and then we have this xcg plate also and the tesseract object is able to read all those very nicely so that completes our introduction to using the tesseract object within MgO cv congratulations you now know how to perform optical character recognition on still images using the tesseract functions in MgO cv see everybody next time

Channel: Chris Dahms

Views: 161,406

Keywords: OpenCV, Tutorial, Optical, Character, Recognition, OCR, Emgu, CV

Id: RqvvXJXuRYY

Length: 17min 22sec (1042 seconds)

Published: Sat Oct 13 2012