Measure the size of an object | with Opencv, Aruco marker and Python

[Music] what is the nice thing about this method that it doesn't matter how far the camera is even if it's closer this marker is going to give a reference point for the size and we can put also multiple objects we have 2.2 centimeters and 6.2 hi there my name is sergio and i help company students influencers to easily efficiently build visual recognition projects we're going to see today the easiest implementation possible of a project which measures in size each object let's say for example that we have this usb pen and we want that our program will automatically say this usb pen is for example five centimeters wide this type of project is generally a complex operation because it will require some calibration of the camera because if you put the object close to the camera this has a specific size if you put the object further because of the perspective this is this looks much smaller but still it has the same size so how can we solve easily this problem in order to solve this i got an idea to use what is an aluco marker if you don't know what this is it's not a problem we will see that later on the video but this is a standard which is used in virtual reality which we will use as a reference point which will give us the size of the object doesn't matter how far the camera is and where the object is positioned so we're going to start first to detect the object on the space once we have the object we can check the size the pixels area that is taking on the frame once we have that with the marker we're going to make the calibrations that we know the size of the object whether it is centimeters inches or the side the measurement size that you use in your country so if you're ready let's start now in order to proceed you need to make sure that you have opencv contrib python library if you don't have that this is the way to install it you go on the terminal so we open the terminal so whether you have windows linux or mac it works the same once you open the terminal you type pip install opencv dash contrib dash python just make a clarification if you have just opencv python it's not the same as opencv country python opencv contrib has some extra libraries that are not not on the official release of opencv which we need today if you have opencv only you're going to get some errors so i want to be clear about this if you if some library is missing it's because you don't have this one so once you do this you press enter and you can install this one i'm not going to do that because i already have opencv contrib and now once you have this we can proceed by creating our script the first step that we have to make is to first of all let's load the image or the camera we start by loading an image which i have right here phone.jpg so that we will work first with an image once we are confident with the image then we will switch to the video camera so that we can detect the size of the object in real time i highly recommend that you download the same images that i have so that you can follow along with what i am doing to download the images and all the files that you see right here you can download for free it's the link below in the description and there will be on the blog post some links to download the source code now now we can finally start first of all let's import cv2 which is opencv then we're going to load the image emg equals dot im read and now we need to put the path of the image and its phone dot jpg how do we make sure that this is loaded correctly we are going to show it if we don't see it it means that we have to change something in show what do we want to show let's show the image so title image or whatever you want and then we want to show this one emg and then cv2.weight key 0 to keep the image on hold open and now we can run this one we see the image so it means that we're loading the image correctly now the first step that i told you about detecting the size of an object will be of course detecting the object and there are many different methods to detect the objects some with basic opencv image processing operations like this one which we can do because there is a white background homogeneous background and there is a black object so it's easy to detect or more advanced audio detection methods using deep learning i'm not going into details about this one if you want to know more about audio detection at a professional level and to implement on your object i have a full course that you can check on my website pysource.com now to make this simple i have created a file which is object detector.pi so you need to download and import that which is going to detect object on a homogeneous background so we're going to import object detector actually from object detector we're going to import everything everything and now we're going to load the detector so right here load object detector detector equals homogeneous background detector homogeneous bg detector so this one is the file that you have to download and it's the and you have to put on the same folder where your main python file is so this is the folder i have the main file where i am writing this and then the audio detector dot pi that you need to download once we have this we can detect our phone and we will do let's first of all it's print detector so what do we get from detector object detector still we don't get anything so yeah there is something that i forgot to put so once we load the audio detector we need to use it to detect the objects on the image so uh contours equals detector dot detect objects where do we want to detect the objects we want to detect the objects on emg right here once we detect this let's print contour and let's see what we get we have also the mask and then later i'm going to hide this to you so you don't need to see this but what we get right here is that we get all the boundaries of this object so these are all the coordinates surrounding the objects to make sure that we have correctly these coordinates i'm going to show what you get when we draw them so using these let's draw a polygon use these coordinates um draw uh of objects boundaries for cnt in contour so if we have multiple objects of course these we have will have multiple coordinates so i'm going to loop through each single object that we have right here so i print cnt and let's run this one to make sure okay that everything is correct we're going to take these coordinates now we will draw the coordinates on the screen to see what we are really getting from them and so cnt uh let's do cv2.polylines where do we want to draw the polygon we want to draw the polygon on emg now here we need to define the points and the points of this we need to put them into brackets this is how it works the polylines function and what do we want to draw we want to draw this point that i was printing before c and t now is the polygon close and he here we need to say true because we want a close polygon now let's choose the color i'm going to make this blue 2055 which is the blue so from zero to 20 55 we give maximum of the blue zero of the green zero of the red and let's make thickness two so that we can clearly see the line of the polygon and now we can run this one as you see here we are correctly detecting the boundaries of the phone once we have the boundaries of course we can later check the size the height and the width here things will get a bit more complex here we have a lot of a lot of coordinates and this is not a specific rectangular object here it has a small curve which will make us hard to detect uh to detect the size because if we had only four points a rectangle it would be easy to attack we could we'll take the longest one will be the the width and the shortest one will be the height for example but here we have a huge amount of points which are going to make this operation complex for us luckily in opencv there is a function which is going to give us a bounding rectangle right from these points and we can extract that this way um okay draw draw polygon right here uh get wrecked cv2 dot wrecked cv2 dot mean our direct is going to give us the rectangle where do we want to get the rectangle from from c and t and i'm going to take this one c and t and here we're going to get a few parameters we're going to get x and y then we're going to get the width and the height and also the angle the angle right this way let's print all of them print x and y let's print width and height and let's print also the angle and let's now run this one so we see x and y 693 we have the x 242 we have the y what is this x and y the x and y is the center point of the object so if i print this if i show this one let's put a circle on x and y save it to that circle where do you want to draw the circle we put the circle on emg the center of the circle is going to be x and y there is one thing that for some reason we're going to get x and y 693 point something when we use coordinates in opencv the coordinate coordinates can never be float numbers so either the coordinate is 693 or 694 it cannot be 693 point something so in order to avoid errors we need to put integers so that we will have only integer numbers then circle where uh the radius of the circle let's make five pixel radius so that we can see what the circle what is the color let's make it red so it's well visible so zero of blue zero of green and 255 of red now finally minus one to fill the circle with the color so we can choose the thickness of the circle i put minus one which will fill the circle with red and let's run this one so i wasn't lying before as i told you this is the center point of the object from this center point we have now also the width and the height of this object and we will have to make some operation to display this rectangle which i will uh i will check one moment and i will show you what we have to do to display everything the idea is that now we are getting from each image we're going to get the minimum rectangle that possible that we can get from the image so that we have a few points to work with and now i want to display the rectangle that we're getting and so here we have the rectangle now i'm going to convert this into points so that we can display this one box equals cv2.box points we're going to get these from let me change something right here from this okay i will put this exactly this way okay so this gives us a rectangle so let's put them rect equals cv2 and then we're going to extract all the information from the rectangle this way and then box points will take the full rectangle this way so if now we print box box here we get these points so now we have the points that we can draw uh to draw the rectangle it is as usual if you see there is 489.68 can never be a float number in opencv so we need to convert them into integers and there is an easy way to do this box equals np so we're going to use the numpy library let let's import numpy uh import numpy asmp and right here box equals np.int 0 off box if we print box now we have only integers and finally let's show the points that we are getting and we are getting exactly we use polylines we don't draw the polygon of the object anymore but we draw the minimum rectangle that we can get from the polygon because that's what we want to use to calculate the size of the object and i'm going to put this right here cnt instead of cnt we're going to draw the box let's run this one and that's what we have here now we have a simple box with four points and we have height and we have width of this object now let's show the size of this object and to show the size of this object at least in pixels it's a quite simple operation because we are we have already the size uh let's let's put some order here this way i'm going to here we draw all the things here we get the rectangle here we uh we display display rectangle and here we have already the size we have width and we have also height of this rectangle so we can show these right here let's show cv2.put text what do we want to draw we want to do the text where we want to draw the text on emg now what is the size uh not the size now the text what do we want to show here we want to show let's say the width width and now dot format the width is this one with w now where do we want to put the text we want to let's show the text exactly on the center of the object uh to show that we have x and y which is the center point so we use x and y and remember when we draw x and y we need to convert them into integers otherwise we'll get an error so into x into y now font face cv2 dot font hair shape we have different font face i don't really care which one we choose let's take the first one front hair shape plane font scale let's make this one of scale then the color let's mix a bit of blue with a lot of green and zero fret thickness two so i i'm not sure how ugly this text will look like but let's see probably we should make this a little bit bigger now what you can see is that we have a long measurement of this object we have 20 47 point and so on let's uh let's round this one only with maximum one number after the dot um so we're going to show this width and then we are going to round weaved with only one number after the dot one other thing i wanted to increase the size and also let's put this a bit higher than the than the center point so the y the y goes zero is the top and then it the more you add the more it goes down on the screen so if we remove some of the y we'll place this a bit more on the top so let's say minus 15. and now we have the width 247.3 of course this is the width in pixels we cannot know any other width because we don't know how far the camera is from the phone but for for the moment let's keep this size 247.3 pixels let's do the same for the height the height of this phone should be around half of the size of the width so let's see if that's correct i'm going to copy this one for the text and put it below and we show the height on the same way height instead of showing w which is the width which shows the height and of course we need to change the position that otherwise the text of width and height they will be overlapping and we cannot read anything so let's put the this one one is -15 minus 15 let's put the other one plus 15. and so we see width and height which probably is not so well visible uh let's put this one a bit on the left side so that we can see this better so let's see minus x minus let's say x minus 50 on both of them or even more let's say -100 of course there's there are better ways to to display the text i'm just doing this to be as quickly as possible not to make things complicated just the only thing is that it's readable and that's enough so we have height 496 with 247. now now it's the trick now we're going to move to the second step and we're going to see how using this marker aruco marker we're going to calibrate make i mean we don't need to make the calibration this would be a reference point for the calibration of the camera what you have to do with this one so i i have printed this one so i'm going to put a link on the blog post where there will be a file so that you can print this one and i will show you after you print it how you can work with this one so this by the way is five centimeters by five centimeters and now i have a second image uh first of all i suggest that you work with the same image right now where i have this marker right here so what i do is i'm going to load the second image which is phone phone mark i don't remember title let me check phone arrow marker okay original phone look marker and this is what we get so in this image i'm going to use this marker and we will see how to use this one what we are going to do right now by using opencv there is some function which can load and detect this specific marker and this will give us some size to calibrate this one and everything so let's do that right now at the beginning of the code we are going to load this one so it doesn't really matter where let's let's load aruka just at the beginning load aluko detector parameters dot detector parameters create so this is what how we load the arrow detector then we need to load the dictionary also dick dick dict equals dot dictionary dictionary get and now there are different types of dictionaries because we have different types of these aruco markers i'm not going to get into details about this one i've printed a specific dictionary for this one so we will be using this as a standard see it so and then i will make some other video where i will go more into details about aruka about vr and all these kind of things for the moment just follow along with what i am doing cv2.rt dict 5x5 underscore 50 which is exactly this one that i have so you can detect only this specific type following this dictionary so be sure about this one because if you print your own rook marker if it doesn't follow this handle right here you probably are not going to detect that one so this is really important that you follow exactly what i am doing once with this we can simply now detect this on the picture so what we do is by using the opencv function okay let me check where we can put this one we can do this i need to make some orders so let's load the image first we load all the detectors aruco detector and object detector after that we work with the image so here we load the image and now even before we detect the object we can detect the arrow marker and we have this corners then underscore underscore equals c c with 2 vitsu.aruco dot detect markers so this is the detector of aruca so this one this function right here is going to detect this one and on corners is going to give us the boundaries the four corners of this marker then we have here some other parameters that we get we don't need them right now so i'm going to put underscore for both of them here we need to input first where do we want to detect this marker we want to detect this on emg emg second where uh the dictionary that from which dictionary you want to attack this one it's on this dictionary arluco dict and now finally the parameters which is equal to parameters this one and after this let's to make sure that this is working correctly let's print corners and let's run this one okay i see some values so we have 777 318 i want to make sure that these are correctly detecting this one so we can also draw these numbers to see that we're detecting correctly this aluco marker and we can do that by using the coordinates that we see here on corners and draw or we can draw a polygon surrounding them so cv2. um let's uh get aluco marker and now let's draw polygon around the marker save it so that poly lines where do we want to draw the polygon on the emg and now the points here we're going to get we're getting float points but each time on opencv we draw something they must be always integers so either the number is five or six it cannot be five point something so only integers so we're going to convert them into corners equals uh np dot int zero i'm going to use the same function used before of corners and now we are going to draw them right here and corners and then is the polygon close yes the polygon is closed true and okay something is missing so what color the polygon let's make this one green zero zero blue twenty fifty five green and zero fret thickness let's make this five pixel thick so that we can see this one it's not working so there is some problem with the coordinates printing corners um sometimes can be a pain when they use different format okay i i managed to make this work with just a really small adjustment you need to remove the the the brackets and it will work that's some magic right there now we have here the aruco marker which will be a reference point now we're going to get to the hot part of this of this tutorial because it is the final part where you will finally get the trick how can we get the size of this so stay tuned each chord each size of these markers is equal this is a square is five centimeters and five centimeters if you don't trust me i guess that you should see this well on the camera i don't know i don't know how well this is visible but it's uh somewhere around five centimeters and for all the sizes what we are going to do if they are all five centimeter it means that the perimeter of this one is 20 centimeters so we know that this green area right here these green lines have a total length of 20 centimeters so what we do we check how many pixels corresponds to 20 centimeters so we check the length of these green lines which will be the boundaries of the aruco marker and how do we do that well we do that this way our uh perimeter equals then we have cv2.arc length we're going to get the length of the corners and then is this close the polygon is closed so we say true now let let's make sure that this is working correctly so i'm going to print aluko perimeter and and of course it's not correct something is not working arc length the formats on how you give here the values i'm going to pause this one until i will find the solution for this i guess i found out the problem so what i see that uh this this function is made in a way that it detects markers plural so if we have more markers in this corner we're going to have multiple uh polygons which one which each single marker so it means that we need to loop through them or if you have only one we need to take the first so this is a tiny mistake that is giving me this problem so corners usually we have many chords the idea they did pro this program is that it will work with just one marker so don't put more than one and we're going to take the first marker so zero for the first and probably probably it should work and finally it works okay so i'm happy with this if you have more markers it just remove one of them because i'm making this one to be as simple as possible to understand so let's not complicate things we'll take the first markers and once with this we have also the perimeter which i was showing then i'm not displaying so perimeter we see 590.53 and so on so what does this mean it means that on this image 20 centimeters correspond to 590 pixels and this is a great value to have once we have this pixel correspond to dot centimeters we have a ratio which we can use to detect any size of any object in this picture so this is the ratio so it's pixel 2 centimeter ratio and this is the magic trick the most important part of this project is the pixel 2 cm ratio and you of course if you have some different unit size in your country you can you can check what's the size i'm going to use 20 centimeters you can check the size of this on your country whether it's inches or if you print this with a different size it's not a problem you can change this right here pixel 2 centimeters ratio is pixel pixel cm ratio equals we're going to divide the aluko perimeter with what with 20. it's 20 centimeters in my case and let's now print this one print pixel cm ratio so it means that one centimeters corresponds to 29.52 pixels now that we have this we're going to divide the width uh we're going to divide both the values of the object detected with this ratio and we will get the size in centimeters at least this is the idea now i'm going to prove this to you so let me check right where we get the size of the objects here we have the width now here is the width in pixel and height in pixels now get let's get get width and height of the objects by applying the ratio the ratio centimeters pixel to cm so what is the width the width will be w uh width of let's say object width equals w divided what divided the ratio pixel centimeter ratio what about the height exactly the same we divide the height object height is the height divided pixel centimeter ratio and height and now we need to display them so the size right here will be object width object height and let's also add centimeters yeah cm and let's now show this one here we have with 600.8 centimeters and height 8.4 centimeters is this correct let me check so here is the phone so we have the width i'm not sure if the camera can get this one but it's around eight centimeters so we will say it's correct probably there there is a mistake of you or of some millimeters that's normal and also this the height is 16.3 and then we have 60.8 so there is of course it's not 100 percent correct but we are really close you might wonder why we have why if this is supposed to be five centimeters we're going to get 4.8 and 4.7 you need to consider that there are many factors that gives us this uh in our case the most important is the distortion of the lens of the camera so each camera has some lens distortion that changes the size of the object based on the perspective so is that the kind of distortion that on the camera you see the objects usually are a bit rounded if you look closely when you are pointing aiming at some uh some strict object if you check and measure with the camera there is a small round of that object and that's why we're going to get these things is possible to solve this yes it's possible of course it will be complex it will require uh some perspective transform and some calibration of the camera that's not the goal of this course of this lesson instead now we're going to complete this one make some change in order that we can do this in real time so i'm going to get the frames from the camera in real time and so we will check the size of some different objects in real time right now let's do that now you don't see me anymore because i i remove the camera from the recording because i need that to attack the object in real time if you want to download the ready source code you can download that for the image and also you can download this one for the video now i'm going to change this one or that it will work with a video we keep everything for the parameters the same we only need to change this one so instead of loading an image here we're going to load cap object it means that we're loading the camera cap equals cv2 dot video capture and here we define the camera camera zero so if you have multiple webcams you can change with one two and so on then we can get the frame so we can get the image in real time from this one cap dot read and let me show what is happening right now so if i run this one it will work exactly the same okay now i'm running the old one let's run this one it will be exactly the same as before instead of loading the image from a file we're loading a single image from the camera so now it takes some time to load the camera but here we have it and here we see we have the objects loading in real time now most most uh more interesting will be of course to get this real time so now we loaded only one image what is the secret load multiple images images well the secret will be to put everything in a loop while true so instead of getting only one image we're going to get one image after the other non-stop so we're going to work with everything that it's right here inside the loop there is some change needed here cv2.wait key 0 is freezing in the frame so it's keeping the frame on hold we need to change this one so that it will work in real time c2 that wait keep one we'll wait only one millisecond and then move to the next frame if the key is equal to 27 we add this one so that we can break the loop 27 is the s key on the keyboard so if we press s we can quit the program we finally release the camera cap dot release and then save it to destroy all windows and we can now run this one another thing that i wanted to make is to make this more higher higher quality is to put these in hd size so by default opencv uses 640 by 480 which is a low resolution for the camera instead i'm going to say cv2 dot cap prop frame width we're going to give 1000 2080 of width and cap dot set cv2.cap height and we're going to give 720 of height not all cameras will support this resolution but most of them will so if you're getting some error just remove these two lines and i'm going to run this one and now we should see the frame much bigger with higher resolution and we should be able to detect the size of the objects in real time it's a bit slow i don't know why with opencv uh in general some version of python with opencv i noticed that it's quite slow to load the camera so you might face the same issue but uh it's not a problem as long as it's loading this one okay we have now it's in real time so you can see ah okay it's in real time and it's interesting so when i put my hand on it we got some error probably if we don't have objects we're going to get the error on line 30 we got okay we are getting the perimeter of aruco even if there is no marker detected so if we try to get the perimeter and there you know market detected we're going to get your the an error so the idea is only if we have the corners detected so if corners only in this case we're going to check everything check the object we check the market and so on otherwise we don't do that so everything this one everything is up to the local market if we have the aruka marker we check for everything we don't have that we're going to skip that one okay now again here it's working real time so we have the size of the phone in real time and i can prove that it's in real time here you can see my hand now the size here it says 600.8 centimeter if we check we okay sometimes it's the width is this one right now it's 16 point something so it's not precise 100 but it's close probably with around five percent of error so the margin is five percent what is the nice thing about this method that it doesn't matter how far the camera is even if it's closer this marker is going to give a reference point for the size and we can put also multiple objects we have 2.2 centimeters and 6.2 let me check 2.2 yeah we're there and six point probably it's around six centimeters six point one so it's quite precise so no matter the distance of the camera here is this one i hope that you enjoyed this tutorial that you got useful information from this one i'm happy about the result because it looks quite precise despite the simplicity of the code and the program itself if you want to learn more about more advanced solutions i recommend to check my website pisource.com

Channel: Pysource

Views: 11,370

Rating: 4.9730339 out of 5

Keywords:

Id: lbgl2u6KrDU

Length: 45min 48sec (2748 seconds)

Published: Fri May 28 2021