Python ANPR with OpenCV and EasyOCR in 25 Minutes | Automatic Number Plate Recognition Tutorial

wonder what my uber is uh struggling to find your uber you might need automatic number plate recognition what's happening guys my name is nicholas renaud and in this video we're going to be taking a look at ampr now ampr stands for automatic number plate recognition so we're going to be using a variety of techniques to achieve this namely we're going to be leveraging opencv and easy ocr but enough on that let's take a deeper look as to what we're going to be going through so in terms of what we're going to be doing today as i said we're going to be using opencv to perform some pre-processing then we're going to perform some edge detection so this is going to help us to find the contours which define where our number plate is then last but not least we're going to be using easy ocr to actually go on ahead and extract that text so let's take a look at how this all fits together so first up what we're going to be doing is reading in our image into python using opencv so this is going to be within our jupyter notebook then we'll apply some filtering some edge detection and some contour detection to be able to find out where our number plate actually is within the image then what we'll do is we'll actually perform a masking operation so we'll actually mask out the section that contains our number plate and that's going to make it a whole heap easier when we go and perform our optical character recognition using easy ocr to extract the text ready to do it let's get to it so in order to perform at automatic number plate recognition there's six key steps that we need to go through so first up what we need to do is install and import our dependencies then what we're going to do is read in our image and grayscale it and blur it using opencv we'll then apply a little bit of filtering and some edge detection to actually find out where our number plate actually is and likewise we'll use contour detection and masking to isolate those components then what we're going to do is use easy ocr to actually extract the text from that image so we'll actually be able to get the text from the number plate and then last but not least we're going to render the result so we'll actually be able to see the detected number plate as well as the text from that inside of our image so we'll actually perform an overlay there now in order to do all of this we're going to be leveraging a number of key packages so first and foremost we're going to be using opencv so to grab the documentation for that you can just go to docs.opencv.org forward slash 4.5.0 now all of the links including the completed tutorial will be available in the description below so just check the links there if you want any of this or if you have any questions by all means just drop a comment now on top of opencv we're also going to be using easy ocr so easy ocr actually allows us to extract text from an image so once we've actually detected where our number plate is we're actually going to use easy ocr on the top of that to be able to extract the text from the number plate now in order to facilitate using easy ocr you need to have pytorch installed so to get pytorch all you need to do is go to pytorch.org forward slash get started forward slash locally again links will be in the description and to go and install it all you need to do is scroll on down and what you're basically going to find is that you've got this install wizard here so basically you just need to select which version of pi torch you want so say we wanted the stable version what operating system we're using so whether or not you're using linux mac or windows how you want to install it so say you wanted to install using conda or pip or lib torch or source then you just select that what language you want to install for we're obviously using python today and whether or not you've got cuda so cuda is going to be a acceleration library that you use if you've got a gpu installed in this case this machine's got 10.1 so i'd select 10.1 then all i'd need to do is copy down this command and run it inside of a command prompt or a terminal and that's going to go on ahead and install pi torch now i've already got it installed so we can skip through that step but this is how you'd go about installing it if you're doing it for the first time perfect so now if we head back to our jupiter notebook we're actually going to kick off and start installing and importing our dependencies now we've got two key dependencies that we're actually going to need so aside from opencv we're going to need easy ocr and imutil so let's go ahead and install those okay so those are our two dependencies installed so what we've done is we've used the pip install command so you can see here we've typed in pip install easy ocr so this is going to install easy ocr and then we've also run a second command which is pip install imutil so this just helps us with our contours once we get those detected now what we can do is actually import them into our notebook so let's go ahead and import that as well as the other dependencies that we're going to need alrighty so we've now gone and imported our dependencies so we've got five key dependencies that we've just gone and written in a single cell here so the first one is opencv so to import that we've written import cb2 the second dependency is matplotlib so in order to import that we've just written from matplotlib import pi plot as plot or plt then we've imported numpy so to do that we've just read an import numpy as np and we've also imported imutil so to do that we've written import imutils and last but not least we've imported easyocr so to do that we've written import easy ocr now the next thing that we need to do is actually go on ahead and read in our image and perform some grayscaling now if we actually take a look inside of our directory we've got four images so image one two three and four now again i'll make these images available in the github repo so if you wanna grab these you can use them so in order to read these into our notebook we're just going to use the opencv imread function and then we'll perform some grayscaling so let's go ahead read it in perform that grayscaling and then take a look at it alrighty so that's our image read in grey scout and then visualize so in order to do that we've written three commands so the first one that we've written is cv2.imread and then we've passed through the name of the image that we want to read so in this case we've passed through image2 and you can see that we've got image2 there then what we've done is we've gone and recolored it so to recolor we've used cv2.cvt color so this basically allows us to convert from a single color code to a different one now in this case what we've done is we've passed through our initial image which was stored in a variable called img and then we've passed through the color code conversion that we want to do so in this case we've passed through bgr because when cv2 reads it in it reads it into the format of blue green red and then we've converted it to gray so you can see here that this entire command here is cv2.cvt color we've passed through the image we want to recolor and then the color change code so bgr2gray then what we've done is we've gone and showed it using matplotlib so whenever you're displaying an image using matplotlib matplotlib expects rgb now because our image by default is in bgr we've just gone and applied another color conversion there but likewise we could drop this if we weren't too concerned with what it actually looks like and we're going to be able to see that image here but you can see this looks a little bit weird right now so we're going to make sure that we have that conversion property and you can see we've read it in as great now the next thing that we're going to do is apply a little bit of filtering as well as applying some edge detection so our filtering is basically going to allow us to remove noise from our image and our edge detection is actually going to be able to detect edges within our image now i actually found an absolutely amazing tutorial out there that actually gives a really great explanation of how these work again i'll link to these in the description i believe it's from circuit digest but let's go ahead and actually perform our filtering and our edge detection alrighty so we've now gone and applied our noise reduction and we've done that using the bilateral filter method so to that we've passed through our image that we want to perform our noise reduction on so in this case we've passed through our gray image that we had up here and then we've passed through a number of properties so these allow us to specify how intensely we want our noise reduction and our smoothing so in this case 11 17 17 seem to work pretty well then from that we've actually gone and performed our edge detection so for that we've actually used the kanye algorithm so this allows us to detect edges now again we've got a number of parameters that we can pass through and these can be tuned depending on what you find actually works but ideally you want to be able to see these edges exactly or similar to what you're seeing in the image below then what we've done is we've used a similar line to what we had up here to go and visualize it so again we've applied our cvt color method to make sure it displays appropriately when using matplotlib so now if we take a look at our image you can see that we're better able to see each of those edges within our image so we can start to see where our number plate is so we can see that we've got a definite contour around there we can also see the different edges of our bonnet as well as our grille so you can see that we're applying some filtering so it's starting to become more evident to us as to where our number plate actually is now the next step that we need to perform is actually contour detection so contour detection is all to do with detecting where these lines are and detecting polygons within those lines so ideally shapes within our images now what we're actually looking for is a contour which has four points so ideally we want to be able to see a rectangle because that's most likely going to be the shape of our number plate so let's go ahead and do that hmm alrighty so we've now gone and found our contours so let's take a look at what we've written there because we've written quite a fair bit so the first component to finding out contours is actually finding them so to do that what we've done is we've used cv2 dot find contours so this basically goes through our image and tries to find shapes effectively aka contours so to that what we've passed is our edged image so this is effectively our image that we've applied edge detection to then what we've done is we've specified two arguments so the first up is how we want our results to be returned in this case we've returned a tree so this basically allows us to traverse our tree to find different levels of contours then our next argument is how we want our results returned or what type of results we want so basically by passing through chain approx simple what we're going to get is a simplified version of the contour so ideally if we find a line we don't want to represent every single point on that line we want to approximate the two key points that represent our line so this approx simple algorithm basically allows us to approximate what the actual contour looks like so ideally what you should get is a contour with four points for our number plate section so basically for our fine contours we've used cv2.find contours again we've passed through our edge detection image specified that we want a tree and then specified that we want to approximate what that contour looks like then we've saw that in a variable called key points then what we've done is we've used imutils to go and grab our contours so this basically simplifies how our contours are actually returned and then we've gone through and sorted them and returned the top 10 contours so to do that we've used the python function sorted we've passed through our contours and we've specified how we want to sort so in this case we're sorting by contour area and we're specifying reverse equals true so this ideally should mean we're going descending then we've returned the top 10 contours now the next thing that we want to do is loop through each one of these contours and see whether or not they actually represent a square or a number plate so we're basically going to be checking whether or not there's four key points within those so let's go ahead and do that alrighty so there we go so we've now gone and filtered through our different contours in order to find the location of where our number plate might actually be so in this case what we've done is we've set up a temporary variable called location and then we've looped through each one of our contours that we had up here so this remember was our sorted contours list then what we've done is we've used the opencv method approx polydp so this allows us to approximate the polygon from our contour now in this case we'll pass through our individual contour at a point in time and this next parameter basically allows us to specify how accurate or fine grain our approximation is so say for example we've got a contour that has a whole lot of little dents in it how high we set a polydp is going to round off that contour so say for example you had something which is roughly a rectangle but had a couple of little dents the higher we set this number the more rough the estimation is going to be so basically it might skip over those little dents and specify that that's really a straight line so play around with this if you're not getting appropriate results what i found is setting it to around about 10 ideally make sure that it's fine enough to detect a square but not too fine that it's going to say that it's got more than four key points now that brings us to our next line so what we're basically saying is that if our approximation has four key points then we're going to specify that as most likely our number plate location so and then in this case we've output our location so you can see that these are the coordinates for our number plate within our image now if we actually take a look at this what we're basically saying is that it starts out at around 152 goes down to 153 160 and that basically gives us a reasonably good approximation of this section here so where our number plate actually is but at the moment we're not actually seeing that so it's probably useful to actually apply some masking and really isolate that section out so let's go ahead and do that and there you go so we've now gone and masked out our image and actually shown it back so you can see here that by applying our contour search we've accurately detected where our number plate actually is now in order to do that we've gone and done a couple of key things so first up we've created a blank mask and in order to do that we've used np.0 so this basically creates a mask that's going to be the same shape as our original gray image to that we've passed the shape of our original image so in this case we've done that by passing through gray dot shape and then we've specified how we want to fill it in so in this case we've specified np into eight so basically we're filling it with blank zeros then what we've done is we've gone and drawn our contours within that image so to that we've passed through our temporary image which is our mask we've specified what contour we want to draw in this case it's the location that we had up here and then we've specified how we actually want to draw it and what we want to display in that then what we've done is we've gone and overlayed that mask over our original image so to do that we've used the cv2 bitwise and function and this basically allows us to return our segment or the segment of our image which actually represents our number plate so in this case here you can see that we've extracted covid 19 that's our current number plate now if we were to go and do this on another image we could do that as well so say i passed through image three this is actually going to run through the exact same steps and ideally should detect our number plate again so you can see it's gone and extracted our number plate from that image we'll stick with our second image for now and then go through the rest and then we can test out the entire flow on another image all right so we're still good with covid19 now the next thing that we want to do is just isolate this one segment here so rather than having a whole heap of blank image we want to just isolate this little section because it's going to make it a lot easier when we actually pass it to easy ocr so let's do that alrighty so we've now gone and cropped out our image so you can see here that we've got an image that purely represents the segment which has a number plate in it so in order to do that we've first started out by finding out every single section where our image isn't black so basically what we're doing is we're storing those in variables x and y up here so basically what we're going to get is a set of coordinates which represents all of these sections in here then what we've gone and done is grabbed the minimum x value and the minimum y value to ideally get this point up here so this is going to be stored in a variable x1 y1 and then what we've gone and done is grab the maximum x and y value which should be down here and we've stored that in variables x2 and y2 then what we've done is we've used a little bit of array filtering so we've gone and grabbed point x1 to x2 plus one to give us a little bit of buffer and this has given us this range over here so zero to whatever that number is there and then what we've gone and done is grabbed at points y one to y two plus one to give us a little bit of buffer to get this segment here so ideally when combined we should have our cropped image which represents our number plate and then what we've gone and done is we've gone and visualized it down here so you can see here by typing in plot.imshowcv2.cbtcolor and we've passed through our cropped image you can in fact see that we have gone and displayed our number plate now the next thing that we need to do is use easy ocr to go and read the text within this image so let's go ahead and do that so this is actually relatively straightforward because we've gone and done all the hard work now we just need to read that text so let's do it and there you go so we've now gone and used easy ocr to go and grab the text from that image now you can see that it's gone and returned a colon instead of the sa but overall we've gone and grabbed a reasonably accurate estimation of the number plate so you can see here that it has in fact grabbed covid 19 with a confidence of about 0.47 now in order to do that we've used the easy ocr reader method so this instantiates our reader so first up what we've written is easyocr.reader and then we've passed through the language that we want to use then what we've gone and done is we've used our reader which we specified here and we've used the read text method on our cropped image which is this then what we've gone and done is we've printed out our result and you can see that this is our entire result component here now a key thing to note is that whenever you're performing automatic number plate recognition your results will vary depending on the image that you're working on so in this case here we've gone and extracted those two little dots and that's because we've got some residual text now in terms of how you'd improve this going forward you might actually choose to use deep learning and train a model that's actually able to classify the image and extract the text out of that you might also choose to perform some additional image filtering to strip out those images as well now say for example use this on another image say for example we used it on image one again if we scrolled all the way through so again we've detected our number plate and so in this case because we didn't have any additional text floating around it's performing a little bit better so you can see we've got hr.26.br.904 so it's actually performed pretty well there now if we tried it on image three and again all these images will be available in the github reaper so you should be able to pick them up all right so our number plates bjy 982 and in this case so it's classified the victoria here so v i c as g but in this case it's classified the rest pretty correctly so bjy dot 9 b2 sorry all right so classified the 8 as a b pretty close though in this case what we're going to do is just use image we'll use image 2. so stick with it and the last thing that we need to do is just go ahead and render our result so we're basically going to take our number plate and overlay our detection on the original image so let's go ahead and do that okay and there you go so we've now gone and rendered our result to our image so in order to do this what we've gone and done is first specified our text so we've gone to the result returned by easy ocr and we've grabbed our first instance here and then we've used -2 to go from the end of our array to pick up our result so -1 minus 2 gives us this text here then to go and overlay our results what we've gone and done is specified our font so in this case we've specified font hershey simplex we've then gone and applied our text which is this component here so to that we've passed our image our text and then the location that we want it so in this case we've gone and specified the bottom corner plus 60 so it's a little bit further down specified the font our font scale our font color our thickness which we've set to as well as our line type then the next thing that we've gone and done is drawn our rectangle which is our green rectangle here and so to that we've passed through our image we've passed through our first tuple which gives us our first coordinate and then our second tuple which gives us our bottom coordinate and that draws our image we've then passed through the color that we want so 0 255 0 is going to give us this bright green and then outline thickness so we've set that to 3. and then what we've gone and done is visualize it so again using plot.i am show and then cv2 cvt color so we've gone and converted it from bgr to rgb now in this case you can see that we've accurately detected our number plate now what we could also do with this pipeline is just change our image so in this case say we change it to image one we could actually just step through it all again and ideally what you should get is again another number plate so again it's detected our number plate and printed out our result we could apply it to image i think we add up to image four so let's try that so again it's detected at a number plate you can see it there grabbed our number plate now this should be a pretty good result because you can see that the text is quite clear we don't have any additional stuff so in this case it's detected h982 fkl pretty accurately if we go and visualize that so it's gone and detected our number plate and overlaid our text so it's localized pretty accurately there and again because this is a really clear image and you can really clearly see the number plate but again your results may vary depending on what your images look like but that about wraps it up so we've now gone through a full-blown tutorial on automatic number plate recognition so we started out by installing and importing our dependencies then what we did is we read in our image we grayscaled it as well we then applied a little bit of filtering found some edges for localization then we found our contours and worked with them to go and crop out our image we then used easy ocr to go and grab the text out of it and find what our number plate actually represents and then last but not least we went and rendered our result now again this entire notebook is going to be available on github so just check the description below you'll be able to pick it up and that about wraps it up thanks so much for tuning in guys hopefully you found this video useful if you did be sure to give it a thumbs up hit subscribe and click that bell so you get notified of when i release future videos and let me know how you went about detecting number plates thanks again for tuning in peace you

Channel: Nicholas Renotte

Views: 115,893

Keywords: ANPR, Automatic Number Plate Recognition, ANPR Python Tutorial, anpr opencv python, opencv anpr, anpr using opencv python, detecting license plate, number plate detection using opencv python

Id: NApYP_5wlKY

Length: 25min 3sec (1503 seconds)

Published: Sat Dec 12 2020