OPTICAL MARK RECOGNITION (OMR) MCQ Automated Grading- OpenCV Python

hello everyone welcome to my channel in this video we are going to learn how to create optical mark recognition algorithm in Python using open CV we will write the code from scratch going step by step while discussing the details of each line we will use the webcam to automatically find the grades of different execute I upload videos on a weekly basis so don't forget to subscribe and hit that notification button and if you find the video useful give it a thumbs up and share it along so let's get started [Music] so let's look at the pipeline first this we are going to take our original image and then convert it into grayscale then we will find the edges in this grayscale image next we are going to find all the contours present in this image and then from this we will find the biggest rectangles and their corner parts then we will take the bird's-eye view which is the world perspective later we will apply some threshold and we will find the marks where each of the marks are present lastly we will save our final answer with the score so here we have a blank project and we have the script of om our main and we have three images that we are going to use to detect our correct answers so the first thing we will need is to import our packages so we will go to file settings and we will go to our projects and over here we are going to add our packages so the first one we will need is open CV so we will write open CV and we will click on it and click install the next one we will need is numpy and we will install that as well so the one I am using is the latest one 1.1 8.1 and for open CV let me check it again it is for point to point 0.32 so both of them are latest ones so far so both of them have been installed and we can verify by writing imports CV 2 and then import numpy as NP so the first thing we will do is we will import our image using the I am read function and we will display it in a win so we will say that image is equals to C v2 dot I am Reid and we will define a path though it is right in our directory so we do not need to define a folder or we can write a path here we can say path is equals to 1 dot jpg and we can write our path in here yeah and then we will display it we will say C v2 dot I am show and then we will name the window original and then we will say that we want to display the image and at the end we have to write C v2 dot wait key and we will put it as 0 so let's right-click it and run and there you go oh there you go so we have our image and it is quite big so we are going to rescale it so the first thing we will do after importing is to resize our image so we will say that our image is equals to C be 2.3 size and we will resize our original image and then we will give it a width and a height so width of our image and then height of our image and then we will close the brackets which are already closed so we will go up here and we can define our parameters all these parameters here so let's write that so here and the second one will be width image is equals to let's say 700 and height image let's say also 700 so if we run dots and there you go it's a little squeezed but should work not a big deal ok so next we are going to process our image we call this pre-processing we will convert it into grayscale we will add a little bit of blur and then we will detect our edges so that we can find our rectangles in which we have all the markings so let's start by converting it to grayscale first so image gray is equals to C v2 dot C V T color our source image is image and then we have to say CV two dots color so we will say BG are two gray then we can add a little bit of blur so we will say image blur is equals to C v2 dot Gaussian blur and our source is the image gray and we have to define the size of the kernel and we will say that it is 5 by 5 and then we have to define the Sigma X which will say as 1 now we will we will detect our edges using the image County function so we will say that our image Danny can is equals to C V 2 dots canny I think the C is capital yes C is capital and then we will define we will write our image blur here then we have to define our threshold values so we can start with some small values for example 10 and 50 and then we can change it later on if required now we have created a lot of these images and we haven't actually seen them so it is a good idea to visualize them before we go further so in order to do that we will use one of the functions that I've previously created in one of the previous videos where we can stack a lot of images together to see the pipeline of the project how the project how the images are changing as we go along so let me import that function but what we are going to do in this project we are going to keep all the main functions all the main functionality in our main script which is om our main but we will add a module in which we will put all our supporting functions and we will call that utilities so let me create that new Python file you sillies and that's it so here we are also going to import CV 2 and import numpy as NP so here we will put all our functions that we'll be using during the process so let me import that function so it's called stock images so this is the function so basically if you give it an array of images it will put them together and you can also define a scale and if you want to define you can define some labels as well so in order to use this function we have to write here import and because this is in the main directory we do not have to define from where so we can simply write a little tillis so here instead of the I am show that we are using originally what we are going to do is we are going to use our stacking function to create some array of images and then we will display them using the function so here we can write image array image array is equals to we are going to create an array so let's write that down so first we will write our image then we will display image image gray then image blur and then we can display should we display in the new line or let's display it here image County so we can create another one later on so image canny and so this is our array and now what we have to do is we have to stack them using the function so we will say it's tagged or we should write image stack image tagged is equals to our function is in the utilities module so we will say utilities and dart stack images and here we are going to define our array which is image array and then we have is it not the same spanning okay image array okay so then we have to define the scale which we will say 0.5 and then we have the labels but we are not going to change the labels for now so then we can just take this and put it here instead and we can say here that these are the start images so let's run that and there you go so we have our original image then we have the gray image a little bit of blur added and then we have the canny edge detector so as you can see in the edge detector we are able to detect our rectangles that we need for the markings and we also have the rectangle of the grading that we will add our grade in and we also can see that we are getting quite a bit of difference between the marked what we call the MCQs and the ones that are not marked so that is a good sign so all of this we can put it separate so here we have our let's call this three processing and let's make it capital okay so next next what we are going to do next we are going to find our contours and to find the contours we are going to use the find control function so let's write that down so we are going to say our contours and our hierarchy what are the spellings a are C and then we are going to say C v2 dot find contours we will define our image Connie and then we are going to define the method C V 2 dot we are going to use the external method this will help us find the outer edges and then we are going to say that we do not need any approximation so CP 2 dot chain approximation C which you taught chain approximation as none okay so this will give us all the contours and what we need to do now is we can display them so that we know what kind of contours we are getting so to draw them we have the function draw contours so cv 2 dot draw contours and we have to give it the image so we will say that we want to draw it in a new image so let's go up here and let's create a new image right after we resize it so image contours on tours is equals to image dot copy so now we can use this image contours here and then we have to define which contours we want to display then we have to define the index so we need all of them so we write minus one and then we have to define the color so let's put them green and then we will define the thickness should we define or not let's define it so we'll put it as 10 so it's nice and clear so now we need to add to our image array so that we can see our image and let's create a new one also you will create a blank image just for temporary purposes so that we can compensate for the remaining images what I mean by that is we are creating a new row and over here we are going to write image contours but we need three more images but we don't have anything else so we can just create a blank image so we can say image blank is equals to we can copy it from somewhere so we can say n P dot zeroes like the real image which is image okay so over here we can put image blank a couple of times and that should do it so let's run that and there you go so now we have the black images and we have the image with all our contours so we can see that we are very well able to detect all the contours the rectangles that we are interested in so we need the most the most useful one for us is the one that has all the markings in them and the one that we are going to put the gradient so both of them are detected well so we are good to go without changing any parameters we can move on further so the next step would be to find which of these are actually rectangles so in order to do that we we'll create a new function and we will call it for example a rectangle contour but we will write it in our utilities because we will be calling this function and it will have quite a bit of code so we will put it in our module instead so first we will define our function name which is rectangle contours or contour let's say and then we will request for the argument of contours ok and ok so what we need to do is first of all we need to filter it out using area we don't want any small rectangles and then we need to filter out if it has four corner points or not so if it's if it has four corner points we will declare it as rectangle and then we will use that to find the biggest one and the second biggest one and so on so in order to do that we are going to loop through all the contours and find their area to first filter out the area so we will write for for I in contours we are going to write area is equals to C v2 dot contour area contour area and then we will just define our contour so this will give us the area of our given contour so if you want to see how that looks like we can print it out and we can say area so it will print all the areas of each contour and what we need to do is we need to call this function so we will go back and after drawing it actually we should comment this so we will write here what are we doing we are I'm finding all on corn tours yes and now we are going to find find rectangles so to find the rectangles we are going to write our we are going to call our function little and since we do not have what happened okay Spelling's okay contours so if we run this now it should give us yes it's giving us the area of each contour so you can see some of them are very small so we want to neglect them early on so we will say that what we need is a minimum area of let's say 50 or 100 it depends on the situation but to be on the safe side we'll keep it as 50 so if area is greater than 15 then only we are going to apply our functions so first of all we are going to find the total length of this contour so we will say that our parameter is equals to C v2 dot arc length and then we are going to say that we want to find it for our given contour and we are expecting it to be a closed one so we will write it as true once we have that we are going to approximate how many what type of polygon it is so how many corner points it has so we will write approximation of the polygon we will write CB 2 dot aprox poly and using that we will try to find the poly the poly count or the corner count of our given contour with a resolution so we have to give it a resolution this value you can change and you can play around with so it is based on our total length which is our parameter and then we are also expecting it again to be closed loop so we will say it is true now this will give us an approximation of how many corner points each one of these have so let's print it out and see what that looks like so we can write here print actually let me put that back so if you guys want to check it again you can check it here so we can write here area is this and I will comment it out if anyone wants to check its up till this point so for this one we are going to check the corner points so we are going to write let's start from here we are going to write corner points and then we will write aprox so if we run that again and there you go so for each one of them it is giving us these points but you can see these are the actual points we are not getting how many points we have to get how many we have we have to just check the length of this list so if we write the length don't ask that again and let's go so you can see we have 10 points 8 points 12 points nine points and so on so each of these tell have four is basically our rectangle and that is what we are interested in so we will say that let's comment this out and down below we are going to write if the length of our approximation is equals to four then we are going to put all of these in a list so let's create a new list by at the top so we will say rectangle contours is equals to empty and here we are going to upend our list each time we are getting a rectangle so rectangle contour dot append and we are going to append the contour itself so once that is done we can print out once the loop is done we can print out our what is that correct angle contours so that should be a list of all the what you call yes so there you go that is a list of containing all the corner points of each of the rectangles so now we have a list of all the rectangle contours but what we want to do is we want to arrange them based on their area so that we can say that whichever one is the greatest that should be number one and then the the second biggest is number two and so on so that way it's very easy for us to just grab the first element or the second element of the whole list so what we can do is we can rearrange it we can say rectangle contours is equals two we will use the sorted function sorted and we want to sort our rectangle corner points contour points and we have to define based on what are we what do you call sorting it out so that is known as a key so key is equals to we are going to use C v2 dot contour area on to our area and at the end we we don't want to start it ascending we wanted descending so the first one should be the biggest one and then the second biggest and the third biggest so usually it gives ascending so it will start from the smallest and the second and the third and the fourth so we want the biggest first so we will write reverse as true so this will give us in order of the rectangles of based on their area or the size so once we are done with that we just need to return we turn our contours so now we have all the contours and we can say that the first one will be the biggest the second one is the second biggest so now we can just call this function and we can find the biggest contour so we will save this list in again rectangle contours is equals to utilities and it will find it and send it back so now we can find the biggest contour for our image so we can right biggest biggest contour is equals to the first one that we have so if we take the biggest one and we can just print it out so you can see there is our biggest ones but the thing is that we are getting lots of points it's not just for points we are getting let's let's just see the length of it so you know how many points are we getting there you go so we are getting 1301 points which is a lot so what we need are four points the corner points of our biggest rectangle so what we can do is we can use these control points and we can try to find corner points and we have some functions for that but again we are not going to write it here we can write it in our utilities so we can define that we want to find define our cuts or nerve points and the funny thing is that we have already used that function before so we are going to write here our contour and here we are going to copy this code so basically this is what is giving us the four corner points if you remember our length was four and that is why we are getting this now it is it is a bit redundant I can see that but this is an easy way to do it without getting without making it too complicated so we are going to copy this and instead of I we are going to replace our contour and this will approximate it with the four points and it will return us the approximation so this way we can simply use our function here you tillis dot get corner points and we want to get the corner point of our rectangle and if we run that but before we done that we are going to trends again our biggest contour this time and there you go so now we are getting exactly four points that are our corner points of our biggest rectangle so if we want the second biggest we can simply write one here and that should give us the second biggest control now let's just have a look at this so the biggest one is this which has all the markings and then the second biggest should be our grading so let's call that grading points so we can say that grade points is equals to u2 and this will be at number one so now we have the biggest and the second biggest contour so next we need to see that whether they have been detected or not if they are if they have been detected then we can apply all those functions to get our markings and find the grade and whatever so let's first test if we are getting it so we will say that if biggest contour dot size is not equals to zero and grade points dot size is also not equals to zero then only we will perform the coming functions so once we have our point what we can do is we can we can actually draw them to see what is going on so which one are we actually getting so let's draw them so we can say cb2 dot draw contours we are going to let's create a new image again so we can say here image biggest contour biggest contours so we are getting two so we are writing contours so here we are going to draw our biggest contours on our image so image biggest contours and then we are going to reduce the biggest points and then we are going to define minus one then we have the color so let's put this one as green and then let's define the thickness again as ten and then we are going to do the same thing but this time we will do it for the second-biggest and we will write grade points and then we are going to change the color so that we know that we are on the correct path so 255 and we forgot to display our new image so here we will write dots and there you go so here we can see them okay let me make it bigger so let's put it as 20 and 20 so there you go so we are able to detect our biggest contour as the correct one and then the second biggest as well is correct in our given situation so both of them I have different colors so it's easier to see now once we have these points what we need to do is we need to work them so we need to work the image so we get the bird's-eye view and then we can apply our functions on it to get the markings and create them later on so the first thing we will do is we will reorder our points now if we don't reorder our points we do not know which point refers to origin which is the last one which is the second one so we need to find a way to reorder them so that they are always in the exact same order before we send it to warping because if we are working in the wrong order the image will not come out properly so let's let's create a new function to reorder our points so let's go here and here we are going to define a new function by the name reorder now we need the points so we will save my points and then let me just right here pass for now and let's go back and see what kind of points do we have so these are the points that we are going to send so we need to see what kind of points we have so we'll write prints and we will write the biggest contour dot shape so it is 4 by 1 by 2 so we understand that we have 4 rows we understand that for each row we have two two values the X and the y but this one is attendant so we don't need that so we will reshape it to 4 by 2 so the first thing we will do is we will say that my points is equals to my points dot reshape and we are going to reshape it as 4 by 2 now what we are doing is we are simply going to add and subtract to find our origin point the final the diagonal point and the other two corner points so that we can rearrange them so the first thing we are going to do is that we will sum all of these what we call points so we are going to say add is equals to my points and then dot sum and then we have to define which axis do we need to sum in so we will use axis number one so if I print this you will understand what I mean so prints add let me print the points as well so it's easier for you to see some friends my points and then it will bring that out so let's run that let me remove the shape from here it did not print anything because we did not call the function so we need to call this let's say here so it is in here reorder and we need to reorder our biggest contour and there you go so basically what it's doing is it's adding 2 3 4 by 1 1 0 that gives you 3 4 4 then it's adding 1 7 4 by 3 6 9 which gives you 5 4 3 and then so on it keeps adding now one thing we can tell by simply adding is which of these is the starting or the corner point or the origin point the origin point will have the smallest sum which means it will be 0 0 so in our case 3 4 4 is the origin so the the whichever one is the smallest one from here we will put that as the first element of our contour or the corner so now these are the old points to put them in a new array we are going to declare that new array now if we remember our original array has the shape of 4 by 1 by 2 so we are going to create the same so my points neil is equals to numpy dot zeros and we will define the shape of it so we will say it is 4 by 1 by 2 and then we can define what type are we using so we can say we are using integers so in this now we can store our values so here let me remove this and remove this here we are going to say that our my points nil at number 0 which is our first element should be my points from the old ones and we need to find the index of this array right so we need to find the index so if the smallest one is zero so we will take this here and put it in the new one here so we need to find that index so to find that index we are going to say that numpy dots are G minimum and we will say that we need to find the minimum index of this array so in that case it will be 0 so here we will write ad now note that it will not always be 0 for another image it might be different actually let's let's run with another image so that it gets a clearer picture so let me go here and put it as - there you go the second image and again it's the smallest one again so it might be it might not be so maybe the third one let me try it maybe the third one is a little bit different no it's the same okay there there can be some instances where it's not the first one and so we don't have it to hear right now but anywho we are going to continue so this will be our first element which is our origin 0 0 and then our second one should be our final our final point which is our maximum width and height so that should be the sum that has the maximum value so in this case we are going to say that our last point should always be the maximum value so here our last point which is number 3 should be the maximum of our addition list so this is for the first two and now for the other two we we'll find the difference between them and using difference we are going to define which one comes first so we will write difference is equals to numpy dot def and then my points and we are going to define the axis again like we added them before this time around we are subtracting so that's the difference so we can say the axis is 1 again so next we are going to define our point my points new at number 1 will be my points and it will be the numpy argument minimum of the difference so this is basically our 0 and width so this is what we are referring to now the next one will be the next one sorry it's not - wait it's 0 its width and 0 and the next one will be number 2 and it will be the maximum and it will be height and the 0 so this one is this one is maximum which is your width and height okay so let me just show you what it looks like in the array or the list of differentiation and there you go so this is our list of the the differences so here we have the differences and you can see the lowest one is minus 4 4 4 and then the biggest value is 1 9 5 so both of these we have not used before so we did not use this one and this one so here we are using this and this in the corresponding and the corresponding list so next we are just going to return this so we will say return my points new and we can just print we can comment all of these out so we don't get these as the output so here we are going to reorder our biggest contour and we are also going to reorder our second-biggest contour and so let's undo that so we will say that our biggest contour is equals to once it's reordered we get that now once we have done the reordering we can do it for the second biggest as well so we can say create points and here as well we can say create points now next we are going to apply the word perspective so that we get the bird's eye view now in order to do that we need to define our points first and then we will create a transformation matrix and then we will apply the word perspective so we need to define the points first so we will say points one is equals two so these are the points that we have already got from the what I call the contours so we will write just numpy dot float of 32 and we are going to define our biggest contour then for points number two point two is equals to number dot float32 and here we are going to define our sequence that we reordered them over here so the first one is 0 0 the second one is worth 0 the third one is height 0 and the fourth one is Wilton height so we have to define that order here so to do that we are going to write the first one is 0 0 then we are going to write the second one is width and 0 which will be width of our image that we defined earlier and then it will be height and 0 oh sorry 0 and height I think I did the same mistake at the back it should be 0 and heights so this will be zero and height and then the last one will be working height so these are our points and once we have that we can get our matrix now to get the matrix we will write matrix is equals to C v2 dot get perspective transform so we will say that we need it for the points 1 and point 2 so this will give us our transformation matrix and then we can use this to apply the word perspective so we will say now we will get the actual image so we will say that image let's say work colored is equals to CB 2 dot verb perspective and our source is our image and then we are using our matrix and then we need to define the width and height so we will keep it as the original width and the collisional height now this will give us the verb image the bird view so let's see how that looks like let's rerun that and there you go so now we are getting the bird eye view of our green points which is the biggest one and the second one we need to get it for our the small portion which is off the grade so the process will be the same we will have to copy this and we will paste here instead of biggest we will have great points and for the width and height we do not need the original width and height so we can write any value here for example 335 let's say the height is 150 so we can write here 150 and then 325 and 150 so this we cannot put in our body called image array so we can just but before we do that we need to change the names here so we can write here for example for the grade so G 1 G 2 and we can add here matrix G what happened matrix G + 4 here we can name it image grade grade display and we don't need to change anything here here we need to change this so G 1 and G 2 so just to see if we are getting it right we can just create another I am show so we can say this is the grade part and we will say image grade display okay it's giving us this which is not right so it means we have done some mistake at the back okay we need to use the matrix G not the matrix so let's run that and we still have an issue okay oh yeah the width and the height although it will be three to five in the height will be 150 so there you go so this is what we are getting of our grade and so later on we are going to put some value here based on the grade we are receiving and then we will display it back on our personal image so for now what we need to do next is we need to find these markings so wherever we have the answers and then we will check whether these answers are correct or not and based on that we will give it a grade so the first thing we will do is we will already displaying anything yeah let's let's remove this okay so now we are going to apply some threshold so that we can find the marking points so the the bubbles that do not have any markings will have less amount of pixels non-zero pixels but the values are the bubbles where there we have some markings they will have more amount of pixels so based on that we can defer we can find which pixels are which bubbles are marked and which are not so we are going to apply threshold and to apply the threshold we have to first convert it into gray so we will say that image work gray is equals to C v2 dot CVT color and we will use our image work color and we are going to say that CV two dots color BG are to cry once we have it in gray we are going to apply our threshold so image let's say thrush is equals to see v2 dot threshold and we are going to apply it to our image warp gray and then we have to define some parameters let's put some random parameters threshold this is the maximum value we'll keep it as 255 and then we will write our CV to dot the type so we will write threshold binary but we want to inverse and then we can simply display this out so instead of the plank we can run this and there you go so now we are getting a nice image of our body called image worked and we have converted into threshold binary values so we can easily detect so here for example we will have a lot more pixels white pixels then we have here or here or here so in any of these we can easily detect here we are not getting that much even though we have some good amount of pencil there but let's let's change the value a little bit let's make it a little bit higher there you go it's a little bit better so next we are going to get each of these individual bubbles and see how many pixel values are nonzero to find out which one of them are marked and which one of them are not marked so to find each of these individual bubbles we need to split our image the one we just got into 25 different boxes so if we look at this we need to split it into 25 different regions so we have 1 2 3 4 5 and 1 2 3 4 5 so in order to do that we have a very simple function that is available in numpy package now as you know that images are just matrices so numpy is good with handling matrices so we can use numpy functions to split our image so what we need to do is we will go to our utilities and we will create a new function by the name split let's say split boxes and we will feed it an image now once we have that image we are going to first split it horizontally to get all the rows and then we will split it vertically to get every individual bubble so let's get the rows first so rows is equals to numpy dots vertical splits now we are going to send it an image and we have to define how many cuts or how many splits we want to make so we will say 5 so let's see how that looks like cb2 dots I am sure and we can say split and we can put rows and in the rows we have to get the first one where did it go did I get it no because we did not call it okay so let's go back and paste this here and this is in the utilities and we are going to feed it image threshold image threshold so let's run that and there you go so we are getting now this is our first split so we are getting the first row and now what we need to do is we need to split it again vertically so that we can get each individual bubble so let's split it again we'll go back to our function and here we are going to say that four so four are in rows so for each one of them we are going to say that our column is equals to numpy dot horizontal splits and then we are going to define our image and then how many times we want to section it so we need to do it five times so now this will give us our split again so what we need to do is we need to put each one of these into what you call a new list or a new array so what we can do is we can create here boxes is equal to empty list and then we can save for box in columns for each one of them we want to append boxes not append and we will append the box so it means it will append each image now if you want to see each one of them we can simply print them out so you understand what is happening so before we after we append we can just display it over here and this should give us a lot of images or maybe it will give the last one because we are not changing the names and I'm unable to grab it from the second screen and there you go I grabbed it finally so this is giving us the last one here so we can pick any now because we have them in the list so if we return it we are going to return our and once we have our boxes we can just simply print one or two out to see if we are getting the correct ones so we can say boxes is equals to dilla T's and then C we to dot I am show and then we can say it test and we can say boxes at number let's say two so if we print that out so let me drag it here so there you go so this is 0 1 & 2 so that's why we are getting C so 0 1 2 3 4 5 6 7 up till 25 24 so now we have all these images what we call in a list so we can use them to let me comment this out so we can use them to find whether they are marked or not so as I discussed earlier to find if they are marked or not we are going to count the nonzero pixels of each of these images so for example let's just just to demonstrate I will write here that we got C here right so let's let's find out that the non zero pixels of that so we can write print we will say c v2 dot non zero and then we would say box says at number two I think number one was number one was marked so let's print both of them out so let's run that and there we go so number one yet this is number one that is marked and this is number two that is not marked so you can see the non zero pixels which means the black the white ones are ten thousand in this region and there are only 2,000 in this region so we can clearly see that this is the marked bubble or the region so what we need to do is we need to find out the nonzero pixels for each one of them and once we have them we can put them in an array and we can find the biggest one and we can call it that this is our marked answer so how can we do that so for that we need to first iterate through all our boxes now all our images so let me comment this out so right down here we are going to say for image image in boxes we are going to iterate it and we are going to find for each one of them so we will say that our total let's say pixels is equal to cv 2 dots count non zero pixels of our given image image the one we are getting from here so once we have that we are going to put it in an array so what we need is an array of 5x5 so wherever we have a bubble we will have a value of how many pixels non zero pixels we have so let's create that so here we can write my pixel values is equals to numpy dots zeroes and here we are going to put how many do we need so we need 5 by 5 because we have 5 questions and we have 5 multiple answers actually it's better to write it as a variable here so we can write here questions is equals to 5 and then we have choices is equals to five as one so so first one where did it go so the first one will be our rows so it will be questions and then the columns will be our choices so once we have declared that we are going to put two counters and whenever we have reached five we are going to go down one row and we are going to save it in the coming row so I think it's better to visualize it so I will just write it down and then you can see for yourself how that works so we are going to write count for columns and then count for rows is equal to zero and then we are going to save our values in my pixel value is equals to count for rows and then not equals to count for rows and count for column equals to our current value so total don't is happening total pixels okay so once we have that now we need to iterate so for four columns we are going to iterate count C is equal to plus equals one so every time it will iterate one and whenever we reach the value 5 we are going to go down and iterate again so we will say that if count C is equals to the number of choices so choices which will be 5 in our case and then we have to increase the row count so count row plus equals 1 and we also need to put count of column back to 0 so that it can iterate again so this should run fine hopefully to see if it runs fine we can just print out our my pixel values at the end are we printing anything else no okay so that should work fine there you go so now we have we have an matrix of five by five and for each one of them we are getting how many pixel non zero pixel values each area or each bubble has so the first one is 4,000 then 10,000 2,000 2,000 and 3,000 in the next one mm mm 10,000 mm and mm so it's very clear that this is ten thousand here ten thousand here we have nine thousand here six thousand here and six thousand here so we can easily see that our maximum values are where we have our marks so this works well but the next thing we are going to do we are just going to convert this into zeros and ones so it's a little bit easier to see and it will be easier to match with our final grading so what we will do is we will find out where this maximum point is and we will save the index of this so for example for the first one it will be 0 1 the second one will be 0 1 2 so it will be 2 then it will be 0 then 0 and then 4 so let's do that so we should comment this somehow let's write here what are we doing here we are getting the the pixel values getting pixel values of each let's say non-zero pixel values non zero pixel values of each box so once we have that now we will iterate through our matrix and then wherever we find the maximum we are going to find the index of it so we will declare a new list my index is equals to an empty list and we will say for X in range of 0 to the number of questions we have so we will iterate 5 times so for each one of them 1 2 3 4 5 we will find the maximum and we will save the index of it so the first one will be 1 so hopefully that will work as well so here we are going to write our array is equals to my pixel value at point x ok so then we are going to compare this so this will give us let me just print it out so if you can see what I mean by that so let me write it here okay so let's run that so we are getting each one of these rows one by one so once we have that one row we can use to find the maximum value that's why we are doing this so let's move on so let's write here so my index value is equals to numpy dot where so we are using the wave function which will help us find in this matrix or in this array where is the maximum value so we will say that we need the index number dot argument of marks not argument sorry Emacs and then we are going to find our in our array so if we write that so let's print that out and see what happens prints my index value so let's read on that okay so it is an array so we have to write zero here yeah so there you go so as you can see the the maximum one in the first one is at number one then we have at number two so this is zero one two and then we have at zero then again we have at zero and then we have at four so 0 1 2 3 & 4 so the 4 is the maximum 1 so now we are just going to save it in a list so that it is easier to compare so we are going to write here should we keep it I'll just keep it here and then we are going to write here my index value not sorry my index dot append because we are creating in that list my index value and we want to save our current value so if we print that out friends what I call my index so let me remove this and anything else no so let's run that and there we have a nice what we called list of our index where we have found the markings so next we are going to compare this with our original answers so we need to write down our answers first so let's go up and here we are going to write ants is equals two so we will define what is our original answer so for this image we are going to say that let's say this is the correct answer so all of them are correct or should we put one wrong let's put one wrong so we will put this is 1 2 0 0 4 so let's put 1 2 0 1 4 so that will give us 1 wrong so in the beginning you have to define what is the actual answer and now what we can do is to grade it we can simply compare these two matrices and we can find the final score so we are going to define here we should comment this here pointing index values values of the markings so next let's just okay let's just keep this here and then we are going to do our grading so for grading we are going to create a new list in which we will define how many of these are correct and how many of these are wrong so we will write grading is equals to a new list and then we are going to iterate and compare so for X in range of 0 to our questions so how many questions we have so what we are going to do we are going to say that if our answer at X is equals to my index the markings at X then we are going to append our grading as one grading dot append as one so what this will do is if the answer is correct it will put 1 if the answer is wrong it will put it as 0 so creating dot append and will put it as 0 so we can print out this grading and we can see what happens and there you go so as we discussed we are putting 4 correct and 1 wrong so it gives us a 0 here and the rest of them are correct so now what we can do is we can find the final score let me remove that as well and now we can find the final score so to find the final score we can say score is equals to now we can sum all the gradings so for example we can sum all of these so in this in this case we will get 4 and then we can divide it by total number of questions so it will be 4/5 and we'll multiply it by hundred so when we get 80% so what we can do is we can use no we don't need that sum we will use grading once we sum the grading we can divide it by the number of questions and then we can multiply it by 100 let's put a bracket here we can multiply it by 100 so this will be our final grade how is the problem here okay the indentation okay and now we can just sprint out the score and there you go so we are getting 80% on our test so now comes the part where we are going to take all of this information and put it back to our image so we know how many of these are correct we know where they are correct where there are wrong so we need to put green color in the correct ones we need to put red in the wrong ones and then we need to put the final score which is 80% in our grading area so let's create a new section here so we will say displaying displaying answers so what we will do is we will because it will require a little bit of code so we are going to put it in a function in the utilities so we will say that it will be our show answers so let's go here sorry deaf in answers and we will need a lot of things here but the first thing we will need is the image that we want to display on and we will need all the the index numbers of where we found the markings so we'll write my index we will also need the gradings and we will also need the answers the actual answers and the grading we have done so we will write here grading and we will also need the answers so so we will need the width and the height of the box so we can create rectangles in that region so in order to do that we will use let's say section width is equals to the shape of our image and we will divide it by the number of questions so for example we have image dot shape this is our number one is a horbet and what we will do is we will divide it by the number of questions we have so for example you have questions we have to define here so questions we will define here and then choices as well so and this all of this should be integer to vote for this to work and then we have the section ID let's just copy that equals number zero and then we have choices so basically what this is doing is it's for example let's say we have 500 and the size of our width so the the image size here is 500 by 500 so then it will take that size to width as 500 it will check how many questions we have so let's say we have five questions so then it will divide it by five and we'll get hundreds so the width of our box will be hundred and the height again will be 100 so we will get 100 by hundred as the size of width and height now we know the size of each box so we can create a rectangle or we can create a circle at that exact point wherever we are finding the correct or the wrong answer so for that to work we are going to create for loop so for X in a range of 0 to questions so we will iterate 5 times in our given situation and for each one of them we need to define where our block should be so let me just play so for example this is our grid 1 which is a box 1 which is a then B then C so for each one of them where should we place where did we find the what do you call the marking we need to color this area and whether it's right or wrong so if it's right we will mark it green if it's wrong we'll mark it red and we will show where is the right answer so we will iterate this five times and for each one of them first we need to know what is the answer so my answer is equals to my index at our given X so once we have that we need to know the center position of our body called the box or the area so we will say see X is equals to my answer now this is let's say answer is two let's say so we need to multiply this two with the given number of boxes height and width so we will say section width and then we will add half of our section width to get to the center section width divided by 2 section right so which is y is equals to the X plus and the section height multiplied by this section height and then we will add to the section Heights the division of 2 now this it seems a little complicated but it's very simple it's very intuitive if you just think about it oh we are trying to find the center value of our given box so we have to multiply the X for the height and we have to multiply the the answer index to our total width and to get to the center we have to divide the two wit by 2 and add it and then we have to define the total width the total height by 2 and add it so now all we need to do is we need to color it so we can we can simply write let's create a circle so we can say C v2 dot circle and then we will create it on our given image and we will define our center points as CX and CY and then we will write down the radius of it let's say 50 and then the color let's say it is green B and G and then R and then we want it filled cb2 sorry C v2 dot build so show answers so let me just copy this we can write the values for each one of them and displaying here show answers is in the U so our image will be our which image should we display on so let's display it on the colored worked so let's display it here so we can put it here and then my index or should we copy I think we should copy let's copy this let's copy it here so my self-image results is equals to copy of this and then we will put this here and my index we are already getting from here so my index is this and we have grading we are getting from here we have answer we are getting from here and then the questions and the choices we are already we have already defined so that is pretty much not and we should get a return from here so we are supposed to return that image so a return image what happened here okay so we will go back and we will save it into our image results and we will copy this and paste it oh you're running out of space so let's create a new one and we will put the rest of the math blank and the first one we are going to put as our result what did we forget ah the comma okay that's too big so let's squeeze it down to 0.3 and there you go so now we are getting green at every what we call choice that we selected but it's not supposed to be green when it's wrong it's supposed to be red when it's wrong so we need to check if the answer is red is if the answer is correct or not so before we decide the color so we are going to color it but before we color it we need to write here that if our grading so we will check if it's correct or wrong at our X index X is equals to 1 then we are going to change my color for example my color is equals to green so we can write this here as green or else we are going to write my color as red and we can copy my color and we can put it here Oh run dots and there you go so we have four of them correct so we are getting green and then we have one which is red which is wrong but in the red once it is wrong we also need to define where is the correct one so we will create a new circle so if it is wrong we are going to create a new circle so let's just copy this one so we are going to define let's put this down so we will create this new circle but before we do that we need to find where is the correct answer so we will right here so we will write here the correct hunt's is equals to our answer at this given index we will get that value and based on that we are going to create our circle so in the in the C X what we need to do is instead of this we will create our own now so here we are going to say that our correct answer x our width for each box plus the half of the width the half of the width of each this will be our what I call home where is the bracket this is a bracket for that and this is this and then we will have the wand not to be here okay then here we will have the the height so which will be x x the section height and we will add the section height divided by two so that we are in the center and the color we will not be using red we will be using green so let's just remove that and put green and there you go so it is displaying us where is the right the correct answer let's just put it small so that it is not confusing why we have two of them so let's put 20 and there you go so it's displaying us where is the correct answer when you are getting the wrong answer so this is good so next we are going to take these markings that we have put of green and red and we will put it back in our original image so we have the final result so in order to do that we don't want to take our current image and put it back so we don't want to take this complete image and put it back on our fist image what we want to do is we only want to take these colored parts and the rest of them should be black and then we can overlay it do this image so in order to do that we are going to create a blank image and which will be the same size as our what do you call the word colored image and then we will color all the areas where we have the circles with red and green and then we will put it back into our final image now in order to do that we are going to create a new image so we will say image let's say raw drawings is equals to we will say that we want to copy it in terms of zeros like the image worked colored so this will give us a copy of that image and then we want to display all this that we have done on this as well so we will copy this show answers and we will paste it here we will paste it here and then we can copy this and we can display there so there you go so now we have a blank image which only has the colored parts shown now we can take this image and we can create the inverse perspective and we can apply it on top of this so how can we do that let's find out so here the first thing we will do is we will create a matrix again so if we go back in our matrix so here we have to do a work we need two points and then we need matrix one and then we need the warping so let's copy this now the points will be same but they will be opposite what I mean by that is where do we paste it okay we paste it here now the matrix this time around it will be inverse matrix and we will use point two as one and point 1 as 2 and then we will use inverse matrix here as well and then we are going to use the raw drawing and we will save it as the raw drawing or let's let's save it as the inverse so image inverse word inverse worked so that should give us an inverse so let's copy that and see what we get so there you go so now we are getting it in the in the correct places where with respect to this image all we need to do now is combine them two together to get our final image so here we are going to combine using the added weight so we will say that let's say image final is equals to C v2 dot added is it added or is it at add weight hard weighted and image final image final and we are going to add the image final and the image inverse together so image inverse warp and 1 1 and 0 now image final is not declared so we can copy the original image if we go up here so here we can write image file so there we have it and now if we copy the image final and paste it here and there we go so now we are getting the correct placement of our circles to just just to see it more clearly we can write C v2 dot I am show final result and then we will write image final so if we run that and there you go so now you can see that we are getting it in the correct places and it's telling us that we have done one mistake and four of them are correct and P is the correct one where you have written a now the last thing we have to do is we have to add the grade here so the grade will be in the same manner that we did our what we call the circles so for the grid again we need to first of all put the text we need to create an image a black image and then we need to put the text on it and then we will do the inverse perspective and add it to our original image so we can say that image raw of grade is equals to numpy dot zeros like our image created display and then we need to put the text on this so we will write C v2 dot put text so we will define which image which is image what did we write it as raw great image raw grade and then we will write down what we need to write so we will write a string of our score so let's let's put it as integer so will write it as score and then we are going to add something else with it which will be our percentage sign so let's write here percentage and then we need to write down the origin we can pick any random values and see how they turn out 5500 and then we need to write down the font the font is C v2 dot font we'll just pick a random one and then the scale we can put let's say 3 and then we need to define the color which color let's say 0 255 255 and then the thickness let's say 3 again so because this is not the same size we have to create a new image so CB 2 new window so see with you I am show the window name is let's say grade and then we have to write image role created so let's run that and there you go so this is our final image that we are getting here so all we need to do now is do the inverse perspective and put this over here so it will come over here like this but with perspective of course so we can remove this and then we have to do the inverse perspective and this will be pretty much the same as we have done before so we are going to copy it from the previous one here and we'll place it here and this time around we will write inverse matrix again we will make one as two and two as one and then here we are going to use the grade and [Music] then we have the inverse matrix or matrix and we can define a new image name here image inverse let's say a great display so once we have that now we can put that in our original image like we did before so what happened there we can write our image over here so if we run this now we get an error why do we get an error okay so you this should be able turn Heights okay so let's run that and there you go so now we are getting it in the correct places and we have the grade as well written in to hit salud shifted let's shift it a little bit forward so the word let's say can be 60 yeah it's it's better okay so now we are getting our grade with the inverse perspective and we are getting all our answers properly where we marked them and at the end we are getting the percentage value so next we can do what we can do is we can add labels to our images so we can see which one is which so if we want to do that what we have to two words the stacking okay this is stacking so here we have to put labels and labels we can define here so labels is equals to we have to define for each one of them so we can literally copy this or let's let's just write it down so we can say all right you know and then we have and then we have gray then we have the blur then we have cannae okay okay so in the next one we have contours then we have biggest contour then we have the warp then we have the threshold then we have results we have the raw drawing then we have the inverse warp and at the last we have our final image so we have final so if we run that there we go so now for each one of them we have our labels showing us where what is happening so the last thing we can do is we can add our webcam instead of using the image and whenever we press the S key we can save the image the final result of the image so in order to do that we will go up and here we are going to declare a parameter by the name webcam feed is equals to true in our case and then we are going to define our webcam parameters so here we will say that sure is equals to C v2 dot video capture and then we have to define the camera number so that's right here camera number so the camera number we can put here as one let's say it can be zero if you have only one camera the default camera it will be zero if you have multiple connected then it depends on your camera so next we are going to set the parameter so we can set the brightness of it by using cap dot sets I like to play with this parameter because sometimes it gets it gives better results so we can start off with 150 160 something like that and next we are going to create a while loop now here we will say while true we are going to do all of that and then we will say that if the webcam feed is true then you will take you will use a back cam so we have success and then we have the image itself and then we can read it from our object and if that is not the case then we will say else you will use the image from the path now all of this needs to go in the loop mmm yep that should do it and at the end for the webcam we cannot just put wait key 0 we have to for for the s key we are going to write if CV two dots wait key is a one and zero so if we are getting the s key then we are going to save this image CV to dot I am right and we are going to give it a filename so let's say final results and then we are going to write dot jpg and then we are going to say which image we want to save so it will be our image final so that should work and we can add a delay CV to dot wait key let's say 300 yeah that's pretty much it but there is something else let me display it and you'll see what that is so if we run this now we are getting an error that we are unable to find the biggest contour or we are not getting the corner points or something like that now this can happen if you are not getting rectangles and during a webcam feed there will be lots of time when you're not getting the proper rectangles that you require or the image is not there so to counter any of these issues what we will do is we will ask it to try if it doesn't work then forget it and keep moving so before you start finding any contours we are going to say it try doing this if it doesn't happen just keep moving forward so here mm I think we should do it till here okay so even if it doesn't happen then we can put accept but we need to display something so we can just copy this and we can we can just put the blank image --is everywhere there you go so if you run that now so here we are getting all the blank images and we have the final results so let me put the image or let me put the mcq paper so let's remove the labels and run it again so we will put it we wanna make it a little bit bigger to see so there you go so we are getting our images and we are getting the final result so if I put an mcq paper in front there we go it should give us the correct values so now we can see here it's detecting the contours properly then it's detecting the corner points properly the biggest one and the second biggest one and then we are getting the word perspective and even the threshold we are getting the correct ones that we have marked and it is circling the correct ones and using that information it's putting it back into the final image and if we press s it will should save so if we stop it and go back and there you go this is your final result and it has saved this image so let me show you a few more results so here we have this so let me change that here we can see it's only 20% the answers are correct and I have another one and here you can see the answers are only 80% correct so let us sports and and you can see here that here a is actually it was marked before and then it was rubbed so it's it still did not so this is it for today's tutorial I hope you have learned something new if you like this content don't forget to subscribe and hit that notification bell and if you would like to see something special or something very particular you can comment down below and don't forget to share and I will see you in the next video

Channel: Murtaza's Workshop - Robotics and AI

Views: 36,674

Rating: 4.9740539 out of 5

Keywords: omr reader, omr scanner, recognize sheet, multiple choice, quiz scanner, optical mark recognition, optical mark reader, OPTICAL MARK RECOGNITION, OMR, MCQ Automated Grading, OMR Opencv, omr pyhton, optical mark opencv, optical mark recognition python, optical mark recognition opencv, omr scanner python, how to build omr, how to opencv omr, omr reader opencv, Automated Grading, MCQ Grading, omr software, omr recognize, optical mark recognize

Id: 0IqCOPlGBTs

Length: 113min 25sec (6805 seconds)

Published: Mon Mar 09 2020