Automatic number plate recognition with Python, Easyocr and OpenCV | Computer vision tutorial

hey my name is Felipe and welcome to my channel in this video we are going to be working with automatic number plate recognition and let me show you exactly what you will be able to do with today's tutorial you can see that not only we are detecting exactly where the license plate is located in this car but we are also extracting exactly the value for this license plate we are extracting ony for a and this is exactly the text in this Dyson's plate also we are going to using Eco CR which is a python Library very commonly used to read text or images and in this video I'm going to show you exactly how to use it so this is going to be an amazing project and let's get started and the way we are going to work on this project is by combining two of my previous projects let me show you exactly what are the projects we are going to combine together in order to work on today's tutorial and we are going to use the previous project where I show you how to run Euro V3 from opencv and also a project where I show you how to implement a license plate detector in by pattern these are the two projects we are going to use on today's tutorial and we are going to make some edits on top of these two projects and we are going to build our automatic number played with recognizer and this is a very common practice in software development remember you don't want to invent the wheel when you are solving a problem please make sure you are using all the available resources you have in order to find the easiest solution and the best solution in this case we are going to use two of my previous projects because this is the easiest way to solve this problem with the resources we have available so these are the two projects we are going to use and let me also show you the data we are going to use in order to test this script in order to test this project and these are some images I have selected and that we are going to use in order to test the performance of today's algorithm in in order to show you the performance of today's project you can see that these are all images from cars from completely different cars from completely different countries and these are all the cars and the license plates we are going to detect and recognize on today's tutorial so let's start with this project and the first step will be to clone a one of these two reposters we are going to clone these repositories so I'm going to select these repositories URL and now I will go to pycharm to a pycharm project I created for today's tutorial and the only thing I'm going to do is to type something like git clone and the repository URL you can see that now I have cloned this repository so the first thing we should do if we are going to work on this repository is to clone is to install the requirements for this repository so what I'm going to do now is to CD into this repository and I'm going to do something in the IPP install minus or requirements okay please remember to install all the requirements otherwise nothing is going to work and now I'm just going back to a here and this is the file we are going to use the most this is a file we are going to use in order to build our automatic number plate recognizer and this is the file we are going to execute in order to make this project up and running the idea is that we are going to take this file as a Baseline and we are going to make a few edits in order to fit this file to today's tutorial so the first edit we will do is to edit the model we are going to use to detect objects because remember this previous project this previous video it was about running yellow V3 from opencv and we were using a model which was trained on cocon the crocodile set so in this case we want to detect license plates so we definitely need to edit this model and this is where we are going to use the other repository the other repository I showed you we were going to use in today's tutorial because what we are going to do is to use the model from this repository the model from this project and we are going to run this model from the other project that's basically the idea of what we are going to do so we have three files which Define or model and what I am going to do now is to download these files but from this other project from the project of running the license plate detector so I am going to these repositories GitHub and I am going to download the weights and in order to download the weights the only thing you need to do is to follow this URL and then you just have to download okay this is going to take a few seconds please notice that we will need three files in order to make this edit in order to edit these three files which Define our model so we have already download or we are downloading the weights but let's continue and let's download the other two files uh two and then we just we can just replace everything at once now I am going to the download the configuration file which is another file we need in order to make this to work and I'm just going to select the file I'm going to raw and then I'm just going to do something like the blue red and this file URL okay and then what I'm going to do next is to do exactly the same but with the classes names so I'm going to roll this will be a very easy file to just create because it contains only one word or only one sentence but let's just do it like this because it's going to be easier so I'm going to wet the classes names and that's pretty much all okay so I have downloaded these two files and also the weights file and now let's see how we locate these files into this project structure in order to make this work the first one will be the configuration file and I'm going to put it exactly in this location which will be Model come and then config or actually CFC so I am going to the home directory which is where I have downloaded these files and I'm going to select this configuration file and then go into put it here in the repositories location which is to this tutorial code and then just here CFG okay and I'm going to rename it slightly I'm going to remove this dot 1 and that's pretty much all so I'm going to copy the file name and then I'm going here and then it just replacing the file name so that's pretty much all in order to specify the first one of these three files the file which represents the configuration file now let's do exactly the same but with the weights file I need to go to downloads I'm going to copy this file and now I'm going to wait and it's just paste this file I'm going to rename it so I I mean I'm going to copy the file name so I can just paste it here and that's pretty much all now I have specified what's the location of the weights file now I need to do the same but with the classes names so I am going to home classes names and then it just I am going to put it here and I see that the prison one is called class names so I'm just going to I'm just going to rename it so it's exactly the same okay okay and that's pretty much all for this file too now I have specified the three files which represent or model the configuration file the weights file and the class names and what I'm going to do now is just test how it performs let's see if we can run one of these two repositories but with the model from the other repository from the other project let's see what happens so I'm just going to press play and obviously I need a data so I'm going to select one of the one of the images I show you for today's tutorial I'm just going to use for example the first one of these images which is something like this okay let's see what happens I'm going to press play and we have a I think with the text we can see that the text is huge but it doesn't matter we are not going to use the text anyway so what I'm going to do is just uh I'm going to comment the text because we don't really need we already know that a license plate it's it's the only object we are detecting so we don't really need to write the text of what's what is it that we are detecting because we already know we are only going to detecting license plates so I'm going to execute it again now I am commenting this file and you can see that we are detecting something the detection is not working 100 properly but it doesn't matter we're going to fix it in a few minutes and what we need to do now is to start making some edits in order to make this fit to our today's tutorial in order to fit these scripts to The projecting which we are going to be working today the first edit I am going to do is regarding this um how we are running this file because this file the only thing it's doing is it's running an individual image we are only executing this detector on an individual image but we want to run this file in all the images in this directory so maybe it's a much better idea to just do something like a four and just iterating the same file in all the images at once so I'm going to do something in that four I'm going to define a directory which is input directory and input directory will be the location of my data we list which is this one right something like this okay then I'm going to say 4J or maybe for image name in OS list there the input dear I'm going to say image but will be all spot join input dear an image name okay and then I'm just going to do something like this and I'm just going to delete this sentence okay now everything should work okay now we are iterating in absolutely all the images in our input directory now the next edit we will do is go into this line which is where we are converting our image into our blog because if you remember this previous project we need to convert our image into a blob which is another format for or image and we're going to do this slate change instead of doing 320 times 320 we are going to say this will be 416 times 416. long story short we only need to edit these two numbers and we need to make this for 16 times 416. that and that's basically because that's the way this model works that that's the way this model was trained and let me execute exactly the same process now that we have added this this iteration now that we are iterating in absolutely all the images in this directory and also now that we we are making this change regarding the regarding these two values regarding these two numbers and you can see that now I'm just going to iterate in all the images we are getting a very good detection I'm just going to enlarge it a little so you can see we are getting a very good detection of our license plates I'm going to show you the first license plate I was detecting previously so you can see how it's detecting this license plate now so I'm just going to iterate on all of them until I reach that one you can see everything is detected super super probably and this was the license plate which we originally were not really detecting super super properly we have now fixed this issue and that's basically how we did it we need to make this adjustment here it's just changing the size of the input image so anyway you can see that now we are detecting exactly what's the location of the license plate of absolutely all the images in our input directory so this is a very good news and I'm just going to continue to the next step I'm going to stop this execution and now let's continue adjusting our script we have already removed the text drawing because it was a little messy and we don't really need it and what we are going to do now because um in this script the only thing we're doing is just detecting the objects all the objects we are detecting in this case only license plates and then we are drawing a rectangle but what we need to do is to define a new object which is license plate and this will be a crop from the original image so this will be something like image and I need to specify the uh I mean let me do it first and I'm going to explain in a few minutes I'm going to say this will be from here to here and then from X1 which is this value to this over value and this is three channels so this will be something like this and let's see this okay something like this something like this okay I think this is okay so what I'm doing here is just taking the image the image I am loading from my input directory and I'm just a crop in this image so we keep only with the license plate so let me show you how this works now I am going to plot the original image but I am also going to plot the license plate so I'm going to do something like plot figure and this will be exactly the same but now I'm also going to plot the license plates something like this okay and now let's see what happens I'm just going to press play again and I'm going to execute a couple of um only a couple of images but we have an error let's see why uh xcr something like this now everything should be working fine yeah so this is for example the first one or four images and our license plate you can see that we are cropping exactly the location of the license plate we are also cropping the rectangle maybe something I can do is to do this crop before we are drawing the image so now everything should be cleaner right now we are we should be uh cropping with a rectangle no but what I need to do is to make a copy because otherwise this will not work okay now yeah everything's okay so this is a little confusing but it it doesn't matter just remember that when you are working with images in opencv you are always working with references so for example in this case license plate is a reference to image so whatever it happens to image later on on the script will be reflected on license plate right this is not an object but this is a reference to this object so it doesn't matter it's just uh you have to be a little careful but just copying this file will be enough just a calling copy will be enough for now so anyway this is uh what we have so far which is the original image and then a crop with the license plate let me show you a couple of additional images and you can see that in all cases it's exactly the same in this case we have uh the original image with a rectangle on top and then here we have the license plate which we have corrupt from our image okay now let's continue and now it's time where we are going to start decoding what's the text in our license plates so what I'm going to do is to use an additional Library which is called Eco CR so I am going to the requirements file I have installed the requirements and when we were starting on this tutorial and I'm going to specify a new library which is Eco CR I don't remember quite well what's the version we're going to use so for now I'm just going to specify the library name and later on I will specify the version so I will go to the terminal and now I'm going to install the requirements again and doing so I am going to install the the new library which is Eco CR this is going to take a few minutes but for now let's just continue adding this library and I'm just going to show you how we're going to use it so I'm going to import Eco CR which I need to do here in order to make it be compliant so I'm going to say import Eco CR and then I am going to use it like this the first thing we need to do is to create a new object which is going to be our OCR reader and this object I'm going to call it Reader and reader will be ecocr dot reader and I need to specify the language we are going to use that I'm going to specify English now please notice that we are reading license plates and license plates are basically numbers and characters so in this case it will not be that important what's the language we we use I will say that's going to work pretty much the same way with whatever language we use but I'm just going to use English for Simplicity but we could also select other languages and I think it will work just to find okay so I have created this new object I'm going to put this object here before iterating in all the bonding boxes and what I'm going to do now that we have a crop or image now we have crop or a license plate the only thing I'm going to do now is to call or reader and I'm going to call read text and I need to specify the image we have crop right this is our license plate sorry this is what we need to specify the license plate the first thing I'm going to do is to make sure everything works properly I'm just going to execute this code and let's see if we have an error or not we are having this print now because it's only a message saying hey if you run this in a GPU it's going to be way way faster but it doesn't matter it's going to work anyway so okay we didn't have any errors so I guess everything is okay okay so we are reading the text in license plate and let's see what exactly we are getting here so what I'm going to do now is just printing this output because I want to show you what's the structure from the output we are getting from Eco CR I'm going to show you exactly what's all the information we are getting when we are reading a license plate so let's wait a few seconds now we are going to see the first one or for prints or for figures and we are just going to mind the text which will be this output okay you can see that this is the license plate we are trying to read with Eco CR and this is the output we are getting let me run make this a little uh larger and this is pretty much the output we are getting we are getting some values which seem to be something like uh I don't know some values then we are getting some text and then we are ready a number and the way we are going to read this information the way we are going to understand this information is like this so ecocr not only it's a reading text not only it's telling us what it says in this image but it's also telling us the location of all the text it finds and these are this is the location of the text it has found this is the rounding box of the text it has found you can see that this looks like if it's like a bonding box because we have two coordinates for every one of these elements and we have exactly four elements so this could perfectly be a bounding box and this is actually a bonding box and now let's continue to the next element where we have something that looks like text and if you take a look at the image we are trying to decode you can see that this is very close right because this is BP 199sn and you can see that the text we are decoding is exactly the same but with a question mark instead of the p and then the other value the less value in this output is the coefficients score for this text so Eco CR has detected something has detected some text it it's telling us exactly what's the location for this text and it's also telling us what the confidence value for this detection because remember every time we work with machine learning with different type of machine learning models we are always going to have something that's a confidence value a confidence score and it pretty much means how confident or machine learning model is for the detection or the classification or the prediction it's currently doing right so Eco CRS telling us that it has found this text in this location and this is the confidence value for this detection this is how confident ecocr is regarding this detection okay so now let's continue and this is only to show you the output we are getting from Eco CR what I'm going to do now is to iterate in these objects because in this case for example we have only found one element but if I show you a couple of other images you are going to see that in some cases we have more than one detection because we have a other bonding boxes containing text so let me show you let me show you how it looks like I'm going to print it again I'm going to print output and I'm just going to iterate in all the images there are a few images in which we are going to have more than one output and then we show you exactly how it looks like in this case we only have one bonding box we only have one uh text box but but let's see what happens in other images in this case we are getting something for example in this case you can see that we are getting these which me which is AG 3970725 and if I show you the license plate we are basically reading the license plate information it's not accurate we are going to make it more accurate in a few minutes but this is basically this right and if I scroll a little to the right you're going to see now we have graph automobile Ag and that's basically the text that's below the license plate so this object this output object could contain more than one element depending how many text box are there in our license plate so what I'm going to do is to do something with this 4 out in output I'm going to unwrap all the information I'm going to say something like this um I'm going to say this is bounding box let's say this is text bonding box text and text score right and this will be out okay and now what we are going to do we are going to print the text and also the score we're not going to print the bounding box because we don't really mind where the text is located the only thing we need to we need to know is the what's the text what are what's the value we are reading where the coding and also what's the confidence value this is very important I'm going to show you why in a few minutes so now I'm just going to iterate in all the images and now we are going to mind the performance now let's see what's the performance how accurate it is in each one of these license plates so this is the first one you can see that we are reading B question mark 199sn and you can see that this is it's somehow accurate we already know this that this is not 100 accurate but it's like it's like okay now let's continue let's see exactly how all the license plates are being decoded and and then we are going to continue and we're going to improve the performance a little so in this case this is our license plate and we can see that we are decoding Ag 397.00725 and if we look at the license plate it's pretty much okay it's pretty pretty accurate but we are reading a 5 which is not anywhere in this license plate right we are just it seems we are decoding this symbol as if it were a five so it doesn't matter it means we are making a mistake now let's continue and let's see a couple of our examples let's see how they are being decoded in this case we are decoding these license plate which is wpx-568 and we are getting an mpx568 so this is not accurate at all this is a mistake we are mistaking DW by a name and then we are reading the text below and this is pretty well right it's to Works bill which is pretty well and then hello bargain yeah everything is okay but we are making a mistake here in DW so you can see we could continue looking at our examples and basically you will notice that everything is working pretty well but we have some very very minor mistakes so now we are going to fix it now we are going to improve the performance and this is how we are going to do it so I'm just going to stop this execution and I'm going to show you what are the changes we are going to do so when we what we are doing so far is just taking our image and making a crop and this crop with its or license plate what I'm going to do now is to convert this license plate into grayscale and let's see if this help us in decoding a more accurate text so what I'm going to do is CV2 convert color I'm going to take my license plate and then CV2 um color BGR to gray right and this is let's say this is license plate but uh gray okay and let's see how it performs now let's go one step at a time and let's see how it performs now I'm now I am feeding read text with the gray license plate and let's see how it performs let's see what happens I'm going to press play again I'm printing the text and also the confidence value for all of our detections and let's see what happens and you can see that now for the first 104 license plates you can see that now we are getting a good detection now we are getting BP 199 SN so remember we used to have a question mark here uh in the in the VP and now we have a very accurate detection we have a exactly what's the character we should have but please notice that we are decoding this text with a very low performance we are getting a 0.39 accuracy and that's not really high at all right that's the only thing we should notice from here now let's move to the next one and you can see that now in this license plate we used to have an error regarding the we were decoding an additional five and there's not any five at all and you can see that now we have fixed this issue too so it's AG 397.070072 so everything is working fine and we are also getting a very low performance we are getting a 28 accuracy let's take a look at a couple of four examples uh in this case we are not fixing the issue because we are still decoding MPX instead of wpx so yeah in this case we have not fixed the error and let's see only an additional example let's see how it performs and now then we can continue okay we have we have many things going on here we have many things I'm going to explain exactly what's going on later on but let's focus on the license plate which is this number and you can see this is 2016 320 and actually this is a mistake because it's 2016 390. so we are still having some issues and what we're going to do next in order to improve the performance of this uh reader is to create a new object which will be license plate threshold and we are going to create this object is by calling CV2 threshold so I am going to call the threshold function from from opencv and I'm going to input the grayscale license plate and I need to specify a few numbers one of them is a threshold which I'm going to set in 64 and then another value which is going to be the value at which we are going to uh take all the values which are higher than 64. and then I need to Define another value which is CV2 Trace and let's start with binary I'm going to make it in two steps so the first one will be to set this in CV2 Trace binary okay so remember how threshold works we are thresholding or image into two different values we are specifying a threshold which is in this case is 64. all the values from grayscale license plate which are lower than 64 are going to be 0 and all the values which are higher and 64 are going to be 255 so let's see now how it performs and in order to have like more information available to see exactly what's the transformation we are doing to our license plate I am going to make two additional plots which are the grayscale license plate and also the tourist holder license plate so this is what I'm going to do and let's see now what happens we are doing exactly the same process now we are going to print the uh the text which we are decoding let's see what happens oh this actually we are getting two outputs from here so I'm going I'm going to say something like this I'm going to say something like this because we're not really going to use this or value so yeah this is just the way this function Works which we are getting two outputs we are not getting only the the thresholders image we are getting an additional output so yeah but we are not going to use it so this is going to be just fine so let's see how it performs and you can see that now we are getting a over performance now we are getting a much better performance and the first thing we should notice is that this is the image we are decoding now this is a thresholded image we are decoding you can see that we are this image looks very well if this is the image we are going to use in order to read the text in order to decode the text or for license plates this looks very very very very very very well this looks very very proper I'm sure we are going to have a very good detection from here and if I if we look at the text we are reading you can see that this is actually perfect BP 199 is in so this is exactly what's on our license plate is perfect and now we are doing with a very high accuracy now we are having a very very good performance so in the case of the first license plate we are having a very very good performance and win or threshold was a very good edit um and now let's continue to see how it looks like for other images from other license plates and you can see in this case we are taking this image these lessons played this is the thresholded image and this is a performance we are getting we are mostly going to focus on the license plate itself we are not really going to mind the other text that could be on the license plate so the the numbers and the characters on the license plate itself it's okay it's AG 39 7072 so everything's okay we are detecting this number with a very low performance so that's another good thing but let's continue let's see what else is going on in our other images if we look at the next image at the next license plate you can see that now we are still having some misdictions because we are the code in the license plate as hpx hpx and it's not hpx it's wpx so we are one step closer everything is working I would say everything is working a little better but we are still not getting the performance we are aiming too so the edit I'm going to do now is instead of doing the traditional thresholding instead of doing the truth holding we were just doing I'm going to do the inverse threshold and the way the inverse threshold works is that now every pixel that's lower than 64 we are taking it to 255 and every pixel that's a higher than 64 we are taking it to a zero to Black so everything is going to be black or white every single Pixel it's going to be either black or white but we are reversing the orders of the black and the white now everything that used to be black is going to be White and the other way around that's the only thing that's the only thing we're doing that's the way they inverse threshold Works everything that's lower than 64 is going to be white and everything that's 164 is going to be black and doing this edit you can see that now this is the image we are decoding the thresholded image we are getting exactly the same performance in this case but now the accuracy is a little higher if you remember in the previous case I think it was 86 percent and now is 89 and exactly the same text so in this case it's working a little a little better and now let's see what happens in the next image so let's wait a couple of seconds in this case we can see that we are still decoding the text we are decoding a very proper text so it's AG 397.072 so everything's okay and we are doing it with a 48 accuracy it's not a really high aggressive but it's okay it's it's going to be fine so now let's continue now I'm going to show you exactly how all the license plates look like and let's see the next one the next one is a license plate that we were having issues and now you can see that not only we are not having any issue because we are detecting exactly what's the text right we are detecting exactly the wpx 560a but we are doing it with a very very very very very very high accuracy we are doing it with a 99.6 percent accuracy so that's pretty pretty high and you can see that the text that's like below this license plate the toe Works B feel hella bargain it's also kind of well decoded now it's hella vegan but please remember that we are using the English version of EC Eco CR where is the English model of Eco CR so uh if we were using maybe the German model maybe we were having maybe we will have a better classification here maybe we will have a better detection here so yeah but it doesn't really matter now we're going to focus on the license plate itself and I'm talking about the numbers and the characters in the license plate itself we are not really going to mind all the additional texts okay now moving on to the to this image you can see that now this was another image which we were having some mistakes and in this case we are detecting cool white which is this uh text and we're also detecting two zero one six three ninety it's perfect and we are having some additional detections which seem a little strange we are taking something like course D which doesn't really make any sense and this 499 491 which is not having make any sense either so uh what's going on here is that as the license plate we are detecting you can see it has some text which is not English which is Arabic text we have these Arabic characters you can see that now this noise we are getting these Miss detections we are getting these words which don't make absolutely any sense whatsoever it's easy OCR trying to understand this code these characters as if they were English that's a what's going on and that's why we are having these errors that's why we are having these issues but that's going to be something very easy to fix I'm going to show you later on now let's focus on the license plate itself we are detecting exactly what we should be detecting so this image is working properly as well let's continue I'm just going to iterate in the images we have left and let's see how they perform and at the end of this tutorial I'm going to show you how to fix that noise we were having with that very strange noise regarding the other image now you can see that we are also detecting exactly what's the text in this license plate which is O and Y and 4A so everything is okay and please notice we have an additional character in this license plate which is this car but that's not really affecting or detection we are still decoding everything super super properly so let's continue we have only a couple of images left let's see how they perform in this case we also having a very very good performance we are detecting exactly what the content in this license plate and we are doing it I mean we are detecting 300 sl1957 which is exactly um the content of this license plate and we are doing it with a 99.9 accuracy even higher it's like like a 99.98 percent accuracy so this is a very very very high confidence value and this is a very very good performance for this license plate now let's continue to the next example I don't know exactly how many license plates we have left but it's only going to be a few in this case you can see it's exactly the same situation we are detecting a b two two two eight hm which is exactly the information in our license plate so it is another example of a very good performance and now let's continue to the next example okay in this case maybe I can do something like this so we can always look at the entire information uh I'm just going to do like this so we can see the VCR the grayscale and the on the threshold image and you can see in this case it's a r a r l 333 and that's exactly information we are getting here and also please mind the accuracy it's a very very high accuracy it's a 99.6 percent accuracy for ARL and 99.98 percent accuracy for uh 333 so everything works super super properly now let's continue and I'm just going to close all these windows and let's see how many examples we have left in this other case you can also see we are having a very good performance it's mer 4169 which is exactly what we are getting so I'm just going to close these windows and and we are going to continue so we move a little faster you can see that we have definitely solved this problem we are having a very good performance on all four images and let's see in this one we are getting this is the license plate which is eight one one seven and p7 and if we look at the thresholders image which is this one and you can see that we are getting exactly this is a a picture which is very very easy to read and we are getting by eight and eight one one seven and p7 so this is exactly information we have here and if I'm not mistaken that was the last example and that means we have pretty much completed yeah we have completed completely or iteration and now let's do something in order to fix the issue we were having in one of our images what we're going to do is to filter by the confidence value because if you notice the those words in which we were having some errors or actually they know is we were having in that image we can bear very easy filter that noise by doing something like only printing the texts which are which confidence value is greater than a given threshold which I'm going to set in 0.4 right we are only going to print we are only going to consider valid all the text which has a confidence value which is greater than 0.4 and now let's see super super quickly what's the performance we are having in actually in in that one in that picture we were having some issues so I'm just going to fast forward this until I reach to that image and you can see that now we have a very good performance now we are detecting exactly the words we want to detect which are who white and two one two zero one six three nine zero we are filtering all the other words all the other detections which have a confidence value lower than the threshold we have defined which is 40 so we have an absolutely per perfect performance everything is working super super properly and this is going to be all for today this is how you can build an automatic number plate recognizer using Python and opencv and this is going to work for this tutorial so if you enjoyed this video I invite you to click the like button and I also invite you to subscribe to my channel my name is Felipe I'm a computer vision developer and in this channel I make tutorials calling tutorials which are exactly like this one and I also share my resources and the resources I use as a computer vision developer so if these are the type of videos you are into I also invite you to subscribe to my channel this is going to be all for today and see you on the next video

Channel: Computer vision engineer

Views: 42,984

Keywords:

Id: 73REqZM1Fy0

Length: 44min 31sec (2671 seconds)

Published: Mon Nov 28 2022