Easy Hand Sign Detection | American Sign Language ASL | Computer Vision

hey everyone welcome to my channel today we are going to learn hand sign detection here we are going to learn where to locate the hand and how to classify it for different sites so for example here you can see we have a b then we have an a and then we have a c so these are the three classes that we classified or trained our model on you can use it to classify any signs that you like but for simplicity we are going to use american sign language so which has an a like that b like this and c like that so this will be our project for today it is a very exciting project we are going to learn how to do this and you can add your own alphabets d e f or you can add your own signs whatever you want whatever you like so this will be our project for today it is very exciting and we will go step by step so you know every single part of the process you learn how to collect the data you will learn how to train and you will not install any complicated libraries and it will happen all within a single setting so let's get started this video is sponsored by clearml do you want to know how machine learning and computer vision are done in the industry well we all love running experiments and notebooks to learn really fast but when solving problems in the real world things tend to get really complicated but that's just means there's more to learn clearml and open source toolbox will help you bridge the gap between learning and actually deploying your projects in a way that's scalable to thousands of users if you love machine learning and are looking to make real impact with your skills you can get started for free in 10 minutes check out clearml and let them help you take your projects to the next level if you would like to level up your computer vision skills do check out our premium courses that are available on our cv zone platform links are in the description so without further ado let's get started [Music] so let's start by discussing the project a bit because it can be confusing because we are using a classifier and a detector as well so normally if you wanted to classify hand signs the whole image needs to be of a hand and then it will classify whether it's a b c or whether it's a victory sign or a rock whatever you are classifying it will classify that but what we are trying to achieve is that first of all we will locate the object which will be our hand so it will tell us the position of that hand and from there onwards we are going to classify it uh what exactly is the hand representing so it's not just a classification problem we also have the location part which will be the detection so we will use a detector to detect the hand and then we are going to classify it uh to check which of these classifier which of these classes does it belong to so to achieve this we are going to write two scripts the first one is going to have just the data collection part so whenever we have a hand it's supposed to detect it and then it's supposed to crop the image and then we will get some images that we will use for training the second part of the script will be to actually test the hand so whenever we have a hand there it should locate it and then it should tell us okay this is an a or b or c or a victory sign whatever we classify it as now for classification to keep it easy we could use rock paper scissor or we could use american sign language so in this case we are going to use asl which is the american sign language uh here you can see now if you wanted to classify whatever uh hand gestures or signs you want you can do that so here we will look at let's say abc and if we have time we can do def or maybe that will be your homework let's say to try it by yourself so this is what we are going to do this will be a b and c so these will be relatively easy uh to classify now there is a problem that we are going to face so i want to start with that when we are going to capture the images this is what we will get so for example if you are doing a you will get this but when we are doing b the the image will not be a rectangle uh the image will not be a square it will be a rectangle so the height is much higher than the width so when we crop it it will give us something like this but why why are we cropping when we when we send the data to the classifier we have to crop the images so and it will crop in a square it will not do a rectangle because all the images they have to be of the same size so for example for a it will be fine but for b it will lose a lot of information and even when i look at it i cannot tell this is b so it will be very hard to classify it correctly if we do it like this so the solution for this is very simple all we have to do is we have to add a rectangle at the back uh sorry not a rectangle it's a square so let's say we will use the size of 300 by 300 so this is the fixed pixel size of our image and whatever size we get we are going to add it to the center of the image or or our plane image or this square so in this case we are going to place it in the middle here and the same thing could happen with uh the a as well if it's not if it's not a perfect square then uh whatever margins we have on the left right up and down it will try to manage that and put it in the center so that we have some consistency while collecting the data so for the classifier it will become very easy to classify if we have the images like this so this is a little bit of a tricky part because you're playing with very small numbers and even the mistake of one pixel will give you an error so you have to make sure your numbers your maths is correct otherwise it could give you some difficulties so now that we are on the same page now that we understand hopefully we do so let's go ahead and start with our project so this is our project we are calling it hand sign detection and we have our main dot pi so let's rename it we'll right click and we will rename the file and we are going to call it test or train or let's call it data collection that uh collection so this will be the script for data collection and then we will have another script we will call it test so this will be the main testing part once we are done so this one is data collection we are going to remove everything because we are going to write everything from scratch so it will be easy for us to follow along now what we can do is uh we can create some folders here so we are going to create uh we create a directory and we are going to call it images or you can call it data as well and within this data we are going to right click and create a new directory and we'll call it a then another one we'll call it b and then another one we'll call it c so these are the the folders that will contain all the images so that's why we are writing them over here now the second part is to actually install so we are going to go to file settings and we are going to install everything that we require now there will be another library that we will require at the end when we are testing so we are going to install that later on but for now we will be needing cb zone we will install that and then we will be needing media pipe and we are going to install that as well so cb zone is installed and let's check with media pipe okay so our second package is also installed which is mediapipe and now we are going to start off with the webcam so once we have the webcam ready we can see our hand then we can move on to the next step which is cropping the image so let's start with the webcam we will import cv2 and then we are going to write while true and uh before while true we are going to add our capture object so cb2 dots video capture sometimes it doesn't give you these suggestions this is a problem that we are facing with the newer version of opencv so let's go ahead and downgrade it let's go to six zero and let's install that so now we'll wait a little bit and once it is done we are going to write it again cap equals cv2 cb2 dot video capture so now you can see uh it is giving us the suggestion so it just becomes a little bit easier to work with so we are going to write here zero and then we are going to go down zero is the id number for our webcam and then here we are going to write success and image equals cap dot read and then we will write cb2 dot i am show and we are going to show our image and we will write here img and cb2 dot uh what was it weight e and we are going to write one so this will just give a one millisecond delay so that's the basic idea what did i press wrong there you go okay just formatting a little bit and it should be good to go so let's go ahead and run the data collection parts and let's see if our webcam works and there you go so our webcam i don't know why it's flickering uh maybe it's because the connection is bad but it shouldn't flicker in your case i i'm not sure there are some driver issues or something uh this is happening with my webcam anyways so this is good to go and we can see the image clearly so now the second part would be to crop the image once we get the hand so let's find out the hand first and then we are going to crop it so we are going to write here from cb zone dot hand tracking module import hand detector and we are going to write here detector detector equals um cp zone no we can write directly hand detector and then we are going to write minimum uh tracking confidence let's keep it the same maximum hands let's keep it one because we just want the single hand for our data collection part so here we are going to write hands and then image equals uh detector dot find hands and then we will give in our image and that's pretty much it so now we should get our hand so let's go ahead and try that out and there you go so you are getting the right hand we have all the drawings on it and this is a good part so you want the skeleton as well you want all the joints all the key points because this will help the classifier to better classify it rather than just having the shape of the hand so this is a plus point for us so we are not going to remove it so we will keep that and the next part is to crop so now we are going to crop the image once we get it so here we are going to check if hands if there is something in the hands we are going to say hand equals hands at zero now we know this is the one that is going to be zero because we only have one hand so that's why we are going to write it like that so that's the main idea and if that is the case we are going to get the bounding box information out of that so in the bounding box information we have x y width and height and all we have to do is we have to ask for the dictionary to give us that value so here we can right hand and we will write bounding box and that's pretty much it so it will give us all the values so once we have that we can crop the image and we can call it image prop equals we are going to write what exactly are the dimensions that we want to prop it at so we will write that our main image it is it is just it is basically a matrix so you have to define the height starting height the ending height and the starting width and the ending width so that's what we are going to define so the starting height first one is the height so the starting height will be y and then it will be y plus height then the starting width will be x and the ending width will be x plus w so this will give us exactly the bounding box that we required so now what we can do is we can simply use i am show and we can write here image crop and we can also write here image pro so this will give us the values uh this will give us another image that we will have as the cropped image so again this will only happen once we have our image so right out of the gate you can see it is not giving us the complete hand it is right at the edge of the tip of the finger and that is not very good so what we can do is we can leave a little bit of space so here we can say y minus uh let's say let's let's give a value here so we can write here offset equals let's say 20. so we'll copy that offset and we will write here y minus offset then y plus offset x minus offset and w plus offset so this will make it a little bit bigger than the exact value so it will be uh more you can say appropriate for the classifier so now you can see we are getting the size as well and it is a very good size um try try to avoid going out you can you can give a condition here that try except but we can leave it like that it's just for data collection so we're not going to do that so again try to keep the hand in between you can go back and forth it's fine and it is capturing the image quite well but here comes the problem right now this is kind of a square but if i do this it becomes a rectangle with a very uh big heights compared to the width so we need to fix that and this is vice versa so the width here is much larger than the height so we need to counter that so that all the images are of the same size so what we will do is we are going to create an image by ourselves so we are going to call it let's say um image white so we will have it with a white background so we will call it image white so image white is basically we are going to create it from like a matrix so a matrix we can create by using numpy so we are going to say imports numpy as np and over here we are going to write np dot once so we are going to create a matrix of ones and then what is the size of this matrix uh we will give it a fixed size so here we can define uh image size equals 300 so this will be the size image size will be 300 by 300 so we are going to keep it square 300 by 300 by 3 so this will be a colored image and then we will also give it the the values the type of the values because if we don't give that you will not see the correct colors so in our case our values range from 0 to 255 or you can say it is an 8 bit value so we are going to say numpy dot unsigned integer of 8 bits so that's our data type so again it it just means it is from 0 to 255 so let's run that and wait we didn't do anything so why would it give us an output now sometimes i'm sleeping so we need to write it down here image white image white okay so let's run that and hopefully it will give us both the images this time around and again we have to bring in our hand and the image white is actually black so what exactly did we do wrong so if you think about it uh 0 to 255 is the range of the cars and right now our pixel values are one so of course it's going to be a black so then we just multiply it by 255 and all of them will become 255 that's how simple it is so let's see what do we get and there you go so now we are getting the image white and whatever i do the image white remains the same size so now we need to take this cropped image and we need to put it on the white image now the easiest way is to just start from zero zero and just put it from the beginning so let's go ahead and do that and then we are going to look at a more efficient way to do it so let's go ahead and do it so here we are going to say image white at whatever points we want to decide we want to have the image crop on that so again it's a matrix so you are saying put image crop matrix inside the image white matrix at these values so what are these values now this could be a little bit confusing if we use these variables so we already have the size of that image so we can just tell it to use it from there so our starting point for the height will be 0 and then the starting point the ending point of the height will be the height of the image crop so we can simply write here image crop dot shape and then we are going to write here let's say value number zero so we we can write here as well that the image prop shape equals image crop shape like that and then we can simply copy this part and we can paste it here so whatever the height is uh it will give us that and the same thing we'll do for the width so start from zero and then whatever the width is that will be value number one so this is a matrix of three values so it will be height width and the channels so we don't need the channels we only need the height and the width so this is going to overlay our image on top of the white image so let's go ahead and try that out and there you go so now you can see we have our image on top of the white image there you go so now whatever i do the image size is changing and it doesn't go very big when it does it gives us an error so we'll try to avoid that or you can add the try exception or you can hard code it that don't go above that point but for now i think this is this will become too complicated if we do that but again this is not the most efficient way because let's say if i do this so it's a square but right now we have a lot of empty space and we are wasting all of that white area so for the detector for the classifier it will be a little bit harder because the image will be very small because it's already 300 pixels so if we had it like this maybe like that somehow then it will be much easier for the classifier so in the case of our vertical image we should have something like this but again we can center it so that it is always similar results for every what you call class so how can we do that what we will do is we will check whether the height is bigger than the width if the height is bigger than the width we will make the height 300 if the width is bigger than the height we will stretch the width to 300 and then we will calculate the other value so for example uh we have the height which is bigger so we have the image like this then we are going to calculate the height we are going to stretch the height to 300 and then to keep the proportion we are going to stretch out the width and how much we need to stretch out we are going to calculate and based on that calculation once we have those values we are going to put it in the center of the white image it's a little bit complicated but i think we can do it it should not be a very big deal so let's go ahead and try it out so the first thing we need to do is we need to make sure whether the height is greater or the width so we will start with the height so whenever we are testing we will do this we'll put b so that uh we have uh the height as the bigger value so we will say if um or for let's put aspect ratio so let's say our aspect ratio ratio equals let's say height divided by width so if in this case the value is above one it means height is greater so we are going to say if height is greater than uh sorry if the aspect ratio is greater than one then we are going to do whatever calculation we need to do here so the first thing we need to get is the constant so if we stretch out the value to 300 what will be the value of the width so we need to find that so we will say that our constant is equals to k is for constant equals to uh 300 which is the size of the image actually we have the image size here so we can just copy and paste image size divided by the height so in this case we are making we are stretching the height so we will write it down like that so now we have the constant now we need the calculated width so the width calculated let's write calculated equals we will simply multiplied k by the previous width that's it and we need to change the value so if it's a decimal place we need to round it off uh but we don't want for it to sometimes go higher sometimes go lower so we will give it a mathematical solution so we will write here import math and we will always use ceiling so we will tell it to always go higher so we will write here math dot seal so it if it's 3.5 or it's 3.2 it will always go to 4. so that's the basic idea okay so now we have the calculated width and we have the original width uh sorry the original height the height is 300 and the calculated width is there now we can take this and put it on our white image so we will remove it from here and we'll paste it inside the if statement and over here we are going to write down uh but we need to resize it first so we will write here image resize equals cb2 dot resize and what image do we give we give in image crop we want to resize it to a very specific value what is that very specific value the height is image no the width is before the height so we will write here width calculated and then the height is our image size which is 300 so that's the basic idea so it will give us the exact same image and then instead of writing image crop shape we can write for the image resize so we can copy that and we can write here image crop image resize resize shape we will copy that and we will paste it here there you go and we will make that image resize so now if we have a vertical image the height will always be 300 the width will change but the height will be 300 let's try it out and there is an error of course so it's not going to work the first time so again mostly the issue comes with the pixel values so probably we did some silly mistake uh image crop image size image resize image oh here's the mistake image resize dot shape so let's try that out and there you go so now you can see the height is always 300 but the image is not in the center and to make it more convincing or appealing whatever you want to call it i think it should be in the center so that's what we're going to do so we'll put the image in the center so now you can see it's it's good but let's try to center it now to center it it's a little bit tricky but we are going to try our best so that we can do it easily so the idea is that we have a fixed height so that's 300 and then we have a different width and based on that width we need to shift it a little bit forward so we will have a certain gap that we need to give from the very beginning to push it a little bit forward to center it so we need to find that gap so how can we find it it's very easy we will write here that the width gap equals so the width of our white image is 300 so that is 300 minus whatever the width is that we calculated or that is of the actual image so here whatever the width calculated is we have to subtract it from that then we are going to divide that by two so instead of 300 we can simply write uh what was the name image size yeah image size image size minus what's calculated divided by two and then of course we will add the ceiling part so that the the values are consistent so here we will add that so that will give us the gap so this is the gap that we need to push forward to center the image so here uh for the image whites it will be the height the height is 300 so we can just keep it like that we will say do nothing and for the width we are going to change so the starting position will be our gap so it will be the gap but then the ending position is is not the size of the image it is the size of the image that we calculated plus the gap so it will be width calculated width calculated plus width gap i hope that makes sense so let's try that out there you go so now it's in the center so this is perfect if we move it around but if we do that now the width is greater than the height so nothing is happening because we didn't define that situation but in this case even it's fine actually for our project this should be fine but to make sure we don't get these flickerings we are going to add the code for the width as well so what we will do is we will copy this part and we will just change it for the width so instead of the height we are going to fix the width so we are going to write here else so image size uh divided by width then this will be height calculated then multiplied by height so whatever you find width convert it into height whatever you find height convert it into width as simple as that then we get the image crop width calculated will be the image size which will be 300 so the width is fixed but the height will be calculated so it will be height calculated image resize shape image shape that's fine the height gap will be math ceiling image size minus the height calculated divided by two image white this will be on this side and the width will be the same and over here we will say height gap and then height calculated and then height gap and that's pretty much it so if we do this now it should work as well so let's try that out there you go so now it's centering even if we do this or if we do this all our image images are centered and now it will be much easier for our classifier to classify all these different images so i'm happy with the results so just don't go too far in because it will give an error if i go really again it will give an error so don't do that uh because it's the constrained environment we are the ones stating uh taking the data so it should be fine but if you want you can add a try accept based on your requirements so whatever the case is you can do that okay so that is well and good so now we need to save these images to our folders so that one by one we can collect the data and then we can take it for the training part so it's very very simple what we don't want to do is we don't want to continuously save we only want to save when we click a button so for that we are going to choose the s button so whenever we press the s key it should save where should it save we will define the folder in the beginning so here we will write folder equals what is the name data that uh slash a so whenever you want to change you change the name of this part so we'll convert that into b then c and so on so that is the idea and then here we are going to say e equals this and then we are going to save if key equals ord and then we are going to write uh s that's the s key if we press the s key then only we are going to save so what exactly do we save we save image white so we will write here cv2 dot i am right we will give in the file name let's put an f string so it will be easier so the file name uh will be first of all the folder slash um what should we put here let's call it image underscore time dot time so time.time will give you a unique value so it will not repeat so we can write here imports time and let's go back so here it will write time.time dot jpg or png whatever you want to add you can add here my bad it should be like that okay and then we are saving the image white so we will write that and yeah that's pretty much it do we need to do something else so what we can do is we can add a counter as well so right here counter equals uh one no plus one my bad plus equals one and then we can write here prints print the value of the counter so we know how many images did we save so we can have some consistency when we are saving the data so here we will write counter equals zero so yeah that's the idea let's go ahead and run it and see if it actually saves the data for us or not so let's bring in our hand and then let's start pressing the s key there you go saving now it will get stuck the image itself will not appear very much so that's not a big deal so if we go to a we can see we saved almost 100 images so if we open that up in the explorer there you go a and there you go we have all these images and all of them are saved correctly even if we move around you can see they are saved correctly so the size of all these images is the same it's 300 by 300 so we don't have to worry about that so if i right click go to properties and we go to details you will see it's 300 by 300 we right click here properties details it's 300 by 300 so all of them will be the same so that's good for us so now what we can do is we can actually go back and delete these images so that we can save it properly so we'll delete that and for a if we go back to our sign a is basically like this so that's what we are going to do for a then this is b and this is c so let's go ahead and start with a so let's run that and we are about to start collecting our data so you can just bring in your hand it's not saving the data because we didn't press the s button so now we are going to start with the s button and this is basically our uh sign so we will press the s and we'll go back and forward we'll rotate a little bit and so far we have collected 150 images so we can go and check them out and there you go so we have almost 150 images and you can move you can move around as well so you can move around like this with a little bit of a different background so let's see how many did we get so far 258 let's go for 300 and there you go so our aim is about 300 images for each so let's collect for the b so b is basically like this so i am ready to collect there you go so let's collect so we are going to collect 300 images uh it will get stuck a little bit that's okay it's collecting the separate images there you go so now we have 300 images let's go and check them out this one is for b and there you go so these are all the different images we have collected so this will be good for us so then we are going to go ahead and check out c c is going to be like this so let's try that out we go back and forward we will rotate a little bit so that we get different angles you can try different lighting as well to make it more robust but for now i'm just giving you a demo so that you can do your own preparation on data collection based on your scenario so there you go we have the images for the c and that should be good to go so now we'll take these images to our trainer we are going to go to google teachable machines and there we are going to enter and let the classifier train so that we can get the results so here we are in the teachable machine website uh it is provided by google and we are about to train so we will select an image project and we will select the standard image model and here we are going to write our classes so it will be a b and we add another class which will be c so now we will click on upload and we are going to drag in all our images so this is the first one which is a we're going to drag them in it's loading all the samples it doesn't take a long time we had 300 images and it did it in a few seconds which is pretty good then we will click on the second one which is b uh b and we will select all and we will put it in for training there you go and then we will go to the last one load and we will go to c and we are going to drag that in as well so the default parameters are pretty good so we don't need to change anything all we have to do is we have to click on training and this will start preparing the data and it will start training it will not take a long time but if your internet is slow it might take a while so be patient and do not change the window and do not close the window of course so here you can see it says unresponsive don't worry it will start training there you go it says don't switch tabs the training process has started and this will not take a while it will be very quick it's almost halfway done already so within a few seconds we should have our model trained but but the point here is that uh one thing to note that is you cannot test it once you are done training because you need the same size images with white background and points to actually classify it properly if you try to do it with the webcam it is not going to work so don't try to do that okay the image got stuck because uh of the trained model let me turn that off turn it upon again okay so the camera is working again so we can go ahead and continue so here the model is trained but again we are not going to test it we are simply going to click on export model and we'll go to tensorflow and over here we will choose kiras and we will download the model and that's how simple it is you don't need to do anything with the code here i will tell you what exactly do you need to do so when we start writing the testing code so it will take a while it will convert and then it will download it will have two files uh we only need one file but uh we are going to copy both of them in case you need it in the future so there you go this is our model it is converted eras model dot h5 file with the label dot text so the label dot text will simply contain a b and c uh that's how simple it is so we are going to uh go to our pie chart and within our pycharm actually let's go back to the folder within the folder we are going to create over here we are going to create a new folder and we will call it model m-o-d-e-l model and inside that we are going to paste both of these files that's it so now we have another folder here called model and it has our kira's model and it has label.text so we are good to go so here we are now going to open the test file and what we can do is we can go to data collection and we can copy all of the code and we can we are going to paste it so here we are not going to save the uh save anything we are simply going to test it so we can delete the key part we can remove that and we can run it and see if it works so we'll right click and click on run to see if it works and there you go the test file is working so whatever image we are getting the image white we are going to send it to our model once we send that it will give us a classification so how can we do that it's very simple it's very easy using the cv zone package all you have to do is you have to import the classification module so we'll write here from cv zone dot classification module import classifier so here after the detector we are going to add a classifier class c fire this is spanning i don't know if they are correct classifier equals classifier i think they are correct classifier and then we need to give in the two parameters the model path will be model slash what is the name kiras so i'm writing the same name because whenever you download it will be the name the name will be the same so it will be easier to change the models and all that dot h5 and then the label file will be label model underscore label dot txt so these are the two files that we will be needing and there you go so then we have our classifier ready and now we need to give it the value for it to classify so what is the value so whenever we have this any of these running we need to send in our value so we will say classifier dot get predictions and we will give in the image so this will give us two things in return it will give us uh let me check what it will give us because i forgot it will give us the prediction and the index so predict prediction and the index so for the index we can write our class names here labels equal a b and c so using this index we can get the value so if it has found uh the class to be zero then we can say okay it is the a class if it's one it will be the b class if it's two it will be c so that's why we are creating this otherwise you can get it from here as well you can tell to import it from here and you can do that as well but here we are simply going to do this so for now we're just going to write print the prediction and print the index print both the things so we can see what we are dealing with so in the case of height bigger so first of all we will do that and then we'll go to the next one so we need to keep an eye on the console so that we know what is our prediction it's all running what happened there's an error import oh of course we forgot tensorflow because that is the main library which actually uh has the kiras module as well so we need to import that so we don't need to import skiras we just need to import tensorflow so tensor sir flow and we will simply install it that's how simple it is so this is a very big module if you are installing it for the first time it will take a while and even the media pi package will take a while as well so let's wait for it to install this is version number 2.9.1 so try to use the same one so that you get the same results if you have a later version try that first if it works well and good if it doesn't you can always come back to this value so let's wait for it to finish and we can continue with our project okay so tensorflow has been installed so we can go back and continue with our project and hopefully this time it will work around so again this is a big library so in the beginning it might delay a little bit but then it will start working and it should be fine so let's fade for it to start so it is about to run no such file or directory label dot text oh it's txt let's run that again again we have an error label dot tx really what's the issue oh it's labels i need to i need to read more carefully so let's start hopefully third time's the charm and there you go so if i bring in my hand it will get stuck a little bit it will initialize and there you go so now it's giving us the prediction so if you look at this value this is the prediction and these are the thresholds or you can say the confidence values so you can see this is basically the last one so it was thinking that it is a c whatever we were doing it's uh it thought that it was c and then it thought that it was p so that is good now what we need to do is we need to put it on our image so we can see what exactly is going on so uh at the end result i don't want to see all the points we don't want to show the packet we just want to show that okay we have detected the hand this is the bounding box and then above the bounding box we will show that okay it's an a or a b or a c and that's it we don't want to show what is happening at the back end so the first thing we will do we will stop it to draw we will write here draw equals false and we will remove the image so it will not give us the output as the image so that is our first thing if we draw the uh if we run that now it will not draw anything now we will draw whatever uh drawing we want using our custom parameters so let's wait for it to run there you go so now there is no drawing so but the problem here is that if we do this the drawing removes from these two images as well but for the white image we do need the drawing otherwise the values the prediction will be wrong so this we cannot do this part here we cannot do so we will reverse that and what we will do is we will make a copy of the original image so we'll write here image let's say final equals or final is a weird word let's say output output equals image dot copy so we are going to copy our image and now whatever new we want we are going to paste it on our image output so in the final result instead of image we will put image output and we are going to draw everything on the image output so by the way we need to do the same thing for when the width is greater than the height but let's finish this one first and then we will do the next one so you can do it more efficiently by writing it once and making it a little more dynamic but i'm lazy so i'm going to do it like this you can feel free to enhance it okay so here we are going to write uh the output so we are going to write here cv2 dot put text and we are going to put it on our image output and actually this is this should be outside should not be that lazy this should be outside okay because it's going to happen either way because we have the xyz we have the bounding box so we're going to put that um and then this is our image output then we have to give in the text so the text value is going to come from the index from the index of our labels so we will say so we will write labels at uh index whatever the index is put the labels at that and then we are going to give in the origin the origin is based on the bounding box so we will give in x and y and we can push it a little bit higher or lower so let's say the x is same the y is let's say minus 20 and then we are going to give in face font let's pick uh cb2 dot font hershey complex and then we will give in the scale as to the color let's keep it purple two five five zero two five five and then we are given the value of the color uh sorry the thickness as 2 and that should be it so if we run this now by the way we need to copy this part for the protection here as well so if we run this now it should give us the index above our hand so so if we bring in our hand there you go so it's telling us whatever it thinks it is so whether it's a b or c so if i do this it gives us c uh if i do this it should give us b if i do this it should give us a so we are having some issues with a why is that it's a little bit strange why is a so weird b let's see okay let's see if we are doing it correctly okay that's the problem that's why the predictions are wrong because we are not giving in image whites we need to give it image white instead of the actual image that's why the predictions were so wrong so let's run that again and hopefully this time it will be correct so let's bring in our hand this is b this is a perfect and this is c perfect b a c that's good uh why is it showing here as well that's weird anyways we don't care about that uh it is showing because by default the the classifier is going to draw the image as well on the image so just right here draw false and we are going to copy that parts over here as well so it's not going to draw on that so but again we don't have the bounding box so here we are going to write cb2 dots rectangle rectangle will given our image output and then we have to give in the x y width and height so x y and then x plus width and y plus height and then we are going to give in our color which will be 2 5 5 0 2 5 5 and then we are going to give in the thickness let's say two or let's say four and that's pretty much it so let's run that and see if it works there you go wait what why is it so weird the okay yeah okay i get it so we need to uh given a little bit spacing like we did with the offset so for the offset we have uh the value 20 so we need to give that so here x minus offset y minus offset then plus offset and then plus offset so now it will be much better hopefully so there you go uh let's wait for it to start and there you go so now it's much better so this is b this is c this is a that's very good actually um the the letters are not very visible so what we can do is maybe we can create another rectangle above that so that it's a little more clear what's going on so we can create this above that we can add the text so let's put it at x minus offset and then for the width uh let's put any value let's say 150 for the height uh let's put the value of let's say 50 and then y minus offset minus 50. so this will just give us a little bit of padding and then here for the color we can change this to white so it's clearly visible so let's see if that will work that's that's very weird okay so i did some mistake um first thing is that wait what okay we need to flip the direction we need to make it filled so over here first of all let's make it fill cb2 dot fills then we need to flip the direction for the y so this will be whatever we have here plus 150 so then it should be fine and what else was there an error in the y i think there was so it's the same thing so this plus y so let's start that again and hopefully this time around it will work properly so there you go so now it's much better so but again it's it's visible but it's not very good so let's make it 50 50 instead of 150 which is too big and what else then we can also make the size of this a little bit smaller so one points seven let's say so that it will be a little bit easier to read what's going on okay so um so we have done it 50 let's make it a hundred so this will be 100 let's say or let's say 90. so again you can keep playing with these values as you wish till the point you are satisfied so let's try that let's push push it up a little bit so we will give in the value here minus 30 so it will push it up a little bit more or maybe 25 will be better let's see let's see how it looks like and then we can decide maybe a little bit less uh 26 let's see or maybe 27. um okay you can you can play around with these values as long as you want i'm going to end with this because you can try all these values by yourself so this is b this is c and this is a that's perfect b a and c there you go so did we miss anything or actually we can add some comments we can remove this as well and we can add some comments actually let's say that is your task write the comments and let me know in the comments that what comments did you write okay that's a lot of comments in a single sentence anyway so this was our video for today our tutorial for today i hope you liked it i hope you enjoyed it if you liked it give it a thumbs up if you loved it share it with your friends and let me know what would you like to see next and i will see you in the next

Channel: Murtaza's Workshop - Robotics and AI

Views: 369,384

Keywords:

Id: wa2ARoUUdU8

Length: 63min 48sec (3828 seconds)

Published: Mon Jul 04 2022