Gesture Volume Control | OpenCV Python 2021 | Computer Vision

hey everyone welcome to my channel in this video we are going to learn how to use gesture control to change the volume of a computer we will first look into hand tracking and then we will use the hand landmarks to find the gesture of our hand to change the volume this project is module based which means we will be using a previously created hand module which makes the hand tracking very easy i have created an ecosystem of modules that can be used in various robotics and ai applications to learn more about it check out my premium course on jets and nano in the description below so let's get started so here we are in our pyjama project this is the same one that we used in the previous video so in the previous video we learned about hand tracking minimum code the bare minimum code that is required to run and tracking and then we created a module out of this so that we don't have to write it again and again we can simply import this module and run it as it is now if you haven't checked this video i highly recommend that you go and check that video out first and then you can come back here and we will continue with the hand tracking project so the project that we are working on is the let's call it volume control or let's call it volume hand control so we are going to control the volume of our computer with our hand so that is something very interesting now the first thing is that we will go to file and we will go to settings we will make sure that our libraries or packages are installed so we will write here opencv python so this is the first one that we need we will hit install and then i've already done it that's why i'm not doing it then we are going to write media pipe and we will click on that and we will hit install so these are the two main libraries that we need for now and later on we can check what we need afterwards the first thing you have to do is you have to import your packages so we will write here import cv2 then we will import time and then we have to import numpy numpy as np so these are the basic packages that we will be using later on we will add some more as well so the first thing we want to do is we want to check if the webcam is working and everything is running fine so we will write here that cap is equals to video capture cv2.video capture and we will write the id number so i am using id1 most probably you will need id 0 then we will write while true we are going to check the success of the capture and then we will write image and then we will write cap dot read so this is the main idea and then we will write cv2 dot im show we will write the image and then img and then we will write cv2 dot white key as one so that will give it a one millisecond delay so this is looking good let's run it and see what happens there you go so this is my hand that we will be tracking and we will change the volume from and this is our webcam so what can we do okay one more thing we can do here is we can write here let's say this is the part where we have our parameters so we can write here that our cam width let's write width of our camera and the height of our camera is equals to 640 by 480 so we are basically defining it here and then we can use it here we can say cap dot set prop id number three is width so we will write width cam and then prop id at number four is height cam so we can write height gap so that should be good and let's try it with the different value one two eight zero by seven twenty so this will make it a little bit bigger yeah there you go so but we are going to use 640 by 480 instead okay so that is good what else can we do we can add the frame rate so here we are going to write that our current time is equals to time dot time and then we have the fps is equals to one divided by our current time minus the previous time and then previous time previous time is equals to current time so we can define the previous time as 0 over here and what else so now we can put this fps on our image so we can write here cv2 dot put text and we will write image and then we will write f p s and we will write the value of the fps let's say integer fps because it's decimal we don't want decimal places and then we can write the the location 70 and then we write the font so we can put any font that we want and then we will write the scale and the color so let's write two five five zero two five five and then we will write the thickness so that should be good so let's run this and see if it works oh it does work but looks really bad okay so first of all the color is bad so let's change it to blue so we can remove this and then let's try the blue color first yeah the blue color is much better then we can make it smaller it's really big so scale let's put it as one yeah that seems good thickness thickness is fine i think it's fine we can reduce a little bit but and then we can push it up a little bit let's say 50. yeah now it really looks good and in the correct place okay so now that we are all set now we will do the magic part the magic part here is that we already have our hand tracking module so we don't have to write a lot of code we will just use its functionality and we will be able to get our hand very quickly the landmarks of our hand very quickly so we will write here import hand hand tracking module now if you don't see hand tracking module that is because you need to put it in the same folder as your project so if it's in the it's not in the same folder or if it's not even here then you will not be able to use it so you need to make sure it is in the same folder and we can import it as something else htm let's say hand tracking module because we don't want to write that complete name it's quite big okay so then we will create an object so this is a class inside here we have a class where is it here hand detector so we will create an object from it we will call it detector detector is equals to uh what was the name htm dot hand detector and then we have our default parameters already here so we don't need to write anything for now so we will keep it like that and then what else can we do then after doing this we need to find the location or we need to find the hand so we will write here that detector detector dot find hands so this is the method that we created so this is this method find hands and all we need to do is we need to send in the image so this will give us the hands so we want to draw it so we will not put it as false we will keep the draw as true and it gives us the image back so we will accept that image back again so that is pretty much it so we can we can separate this code not too much we can separate this code so that it is easier to read so now we can run it and see if it works and there you go so now we are getting good detection and it seems quite good so what happens sometimes you can see it detects a little bit on the side and then it detects small hand as well so what we want to do is we want to change the detection confidence here so at default it is 0.5 so the detection confidence is 0.5 we want it to be really sure that it is a hand and then only detect the hand this way when you are changing the volume it will be a little bit smoother because it will not flicker too much so we can write here detection confidence is let's say 0.7 so you can play around with this value if you want but i will keep it at 0.6 yeah seems fine to me and then we are going to go to the next part which is the best part of getting the position so now all we have to do to get this position is to write that our landmark list is equals to detector dot find position and then we just send in our image that's it uh and we will also write draw is equals to false we don't want to draw it because we are already drawing it so we will keep that as false and now let's run it and there you go we are getting it but i forgot to print so print lm list so let's run that and there you go so at the bottom you can see that we are getting that list so here we are getting that list and we have a total of 20 values here uh 21 values because we are counting 0 as well so we have a total of 21 values now if we want to get the value of a particular point then we can write that point itself so here for example we need point number two so this is the landmark number two so if we run that again okay so this is a good point we are getting an error because the index is out of range so before we actually print or before we actually do anything related to the points we have to make sure that there are some points so we will write here if lm list the length of it is not equals to zero not equals to zero then we are going to do this otherwise it will skip it so let's try that and there you go so now i'm only getting landmark number two so this is good but how do we know which landmark do we need so here is the media pipe website and they have given us the landmark model information so these are all the values that we get all the landmarks that we get so here we will need the value number four which is for our thumb and we will need the value number eight which will be for our index so we need the tip of both the thumb and the index so we will go back here and here i can write i need the value number four and i need the landmark list i need the value number eight so this way we will get only these two values there you go so now i have these values and you can see they are changing but to make sure that we are using the correct ones we are going to create a circle around them so we will write here cv2 dot circle and in the circle we are going to say that we want to put it on our image and then we have to given the center value now here what we can do is we can create some variables so that we don't have to write all of this again and again so we can write here x1 and y1 is equals to lm list at number four and we need the first element of it so this is the id number zero then this is the number one element which is the x and this is the second element which is the y so we need the first element as x and then we need the second element as y and then we can copy this and we can do it for the index as well so we will write here this is 8 and this one is 8 as well so now i can simply write x1 and x2 x1 and y1 and then we are going to write the radius let's say 15. and then we will give in the color let's say 255 0 and then 255 and then we will write cb2 dot filled okay so we can copy this for the other one and we can paste it here and we can write x2 and y2 so this will uh create two circles at that point so if we are getting the correct points then it should draw on the thumb and the index and there you go so now you can see that it is drawing on the thumb and the index so this is good now the next thing we can do is we can create a line between them so we can write here cv2.line and within the line we'll give in our image and then we will give an x1 and y1 and then we will give in x2 and x2 and y2 and then we will give in what else do we have we have the color so two five five zero two five five and then we have the thickness so let's try that out there you go so now we are getting the line as well in between okay so next thing what we have to do is we have to get the center of this line so we can do that simply by writing here cx and cy is equals to x1 plus x2 divided by 2 and then y1 plus y2 divided by 2 so we can simply write it like this and this will give us the cx and cy and we can put a circle for that as well so let's write here cx and let's write here cy so let's try that out and there you go so now we are getting a nice circle in between so this is good now next thing we can do is to find the length so this is the most important thing we need to know what is the length between these two points or what is the length of this line when we know the length of this line then we can change the volume based on that so what we can do is we can write our own function and we can do a little bit of maths to square it and then square root but instead we are going to import math and within math we have the hypotenuse function so we will use that so we will write here that our length is equals to math dot hypotenuse and then inside that we are going to write x2 minus x1 and then y2 minus y1 so that will give us our length so we can print it out so let's try that i think we are printing something else as well so we need to remove that first okay let's try it okay so here you can see that when i increase the distance the value increases when i decrease it the value decreases so the maximum you can say is around 300 something so we can say let's say it's 300 and the minimum let's say is 50. okay so this is the maximum and minimum that we have so one thing we can do is we can write here that if our length is less than 50 then we want to change the color of our center circle so let's make it green so this will give it like a button effect so when you're pressing it it changes so here let's let it load okay so here when we come closer there you go so now it actually feels like a button and it is very soothing i don't know why but it's pretty cool and it's quite fast so it is good so this is good and now what we can do is we can change the volume based on this length so how can we do that well we have a couple of libraries that can help us with this the one that i found was pico i don't know why it says car but it is developed by andrey miras so thanks to him for developing this awesome library which allows us to change the volume of our computer so it is uh under the mit license and if you go down you have to just write pip install pico and then you can use this code as the template so i will copy this because i'm lazy and i will paste it here okay so now what we have to do is we have to simply go to file settings and we will add pico by call you can do pip install as well it's the same thing so we can click on pico and we can click on install so that is installed and we're good to go if we go back you will see all the errors are gone so we will copy the imports and we will we will cut them and we will paste it at the top and the rest of it we are going to see what do we need and what we don't so what we can see here is that we have the volume get mute so this seems like the initializations so we are not going to change anything there then here we have volume dot get mute uh we don't want that then we have get master volume level um do we need i don't think we need that we might need the volume range and then we have volume dot master volume so we can set the volume so before we set it let's see what is the range so i can print this and there you go so our range is from minus 65 to zero so this is our range so we are going to use these two parameters so zero will be maximum and 65 will be minimum we will ignore this value so if i set the volume as -20 let's run it and see what happens so right now it is at 26 the volume goes to 26 if i set it as let's say -5 let's see what happens to the volume it goes to 72 and if i put it as 0 then you can see it goes to 100 so this is basically the idea so what we can do is we can get the minimum and maximum range so we can write here that our volume range is equals to volume get volume range and then we will say that or we will keep it like this let's keep it like this we can take the values later on or i think it's better to write so we'll write here minimum volume is equals to volume range at 1 was it yeah at zero and then maximum volume is at one so this is our range okay so now we can use minimum volume and maximum volume instead of using this okay so what is the next thing that we need to do now the next thing is we need to convert our volume ranges so as you saw that our minimum and maximum was 350 so we were getting let's write it after the length so here we know that our our hand range was from 300 was the maximum and the minimum was 50. so it was from 50 to 300. now we need to convert it to our volume range so our volume range is from uh minus 65 to zero so we need to convert this range into this range so in order to do that we have a very simple function in numpy so we have did we include yes we included numpy so in numpy we can write here volume is equals to numpy dot interp and inside that we are going to give in the value that we want to convert so we want to convert the length and now we have to give in the range so our range was 50 to 300 and then now we have to give in the range to which we want to convert so our range here is the minimum minimum volume and this will be the maximum volume so let's print this out so let's print the volume so let's remove that and let's see if this works so right now it's -25 if i go down it goes to -65 so here our volume is zero and if i go up you can see it should go till zero yeah there you go this is the maximum so this is good if you are a little bit confused you can write the length here as well so the length and the volume you can see side by side we can make it let's make it integer that doesn't give weird values so yeah here you can see when we have the minimum volume then it gives us minus 65 and then when the length is maximum it gives us zero so this is exactly what we wanted so now that we have converted this we can simply send it to our master volume so we have the function here we can actually remove it from here and we can go down and after the volume we can paste it here so set master volume level we are going to set it as our volume so let's try this so i will open up my volume and let's try this there you go so if i go to zero it goes to zero as well if i increase the length you can see and now it goes to the maximum i think 300 is a little too far well it's fine you can see i can change the volume from here you can make it a little more smoother as well by changing the range of these two over here and even this you can play around with because i can see it's not very proportional so you can play around with those values to make some changes now the last thing we can do is we can show the volume bar on the side so that it looks a little bit nice to see what is the volume at any given point so what we can do here is that we can create a rectangle so we will go down and we will write here cv2 dot rectangle and we will put it on our image we will give it the initial position and then we will give it the ending position so we will give that and then we will give in the color so let's make it green and then we will write cb2 dot filled so this is the idea so the width of our bar is basically 85 minus 50 which is 35 so if we run this there you go so this is our bar and what we have to do is we have to remove the fill we don't want the fill we want let's say three so let's run that yeah so there is our bar and then the next part will be filled so the next part here we will copy this and now we will give in the volume so we will say that our width is the same but the height will be different so we will write here integer our height so volume of this and do we need to change anything else uh not really so here we need to change cb2 dot filled so if we run this volume is not defined oh yeah volume is not defined here so we can write here volume is equals to zero okay there you go so now you can see that it's going out of the uh what he called image so if i try to change let's try to change does it change no because it is too big for us so what we need to do is we need to convert our range again so in this instance our range is from let's say 400 so 400 let me show you here so 400 is at this point so this is the height 400. so when our volume is 0 it should be 400 and at this point we have what do we have we have 150 when the volume is maximum it should be 150. so our new range so we can write here that this is volume for the bar so we will write here it's again 50 from 300 till 300 but the minimum now is 400 and the maximum is 150 so instead of volume we will send in volume bar so let's run that okay now volume bar is not defined so we need to write here volume bar and let's do volume as well volume is equals to zero okay let's run it still the same why where did i make a mistake volume bar is from so where is the issue let's run it again and see if i put my hand in ah okay so the first value is wrong so we need to go up here and volume bar should be 400 so it is at the first point it should be zero so our zero is at 400 so there you go so now if i bring in my hand and now if i change you can see it changes the value there you go so the last thing we can do is to add a percentage at the bottom so we can copy this and we can paste it here and in the text we are going to given our percentage but again we don't have any percentage so we can create another conversion here we can write here volume percentage is equals to 50 to 300 and now it will be from 0 to 100 so we need the percentage of that and then we can write here that our volume this is our volume should we write volume or let's just keep it like this and then we can write here volume percentage and then we can write percentage in front of that but the location let's put it 440 let's keep it at 40 and then let's put this as 450 and let's keep the rest same or let's change the color let's make it green so it matches that so let's try that out okay volume percent i always forget this volume percentage is not defined we need to define it as zero or should it be zero uh yeah it should be zero okay so zero percent 64. it's hard to see this color um let's change the color completely so let's keep it as blue so i will copy the blue color where is it from the fps and we will put it for all of these okay let's try it again yeah now it's much much clearer so here we can see so 100 is a little bit hard to reach so instead of 300 i can make it 250 for example 275 but now you can see the percentage is changing let me show the volume bar and you can see here that the percentage is changing so that is pretty good so as you can see it is very easy to create a project once we have our module ready so you can import this module into a lot of different projects and you can work with it and all you need to do is you need to create an object where is object this is the object that you have to create and after creating the object you can find out all the different landmarks that you need from the landmark list so this is it for today i hope you have learned something new if you like the video give it a thumbs up and don't forget to share this video with your friends keep spreading the knowledge and hopefully we will have a complete community of computer vision and open cv and whatnot make sure that you subscribe if you haven't subscribed and there will be more videos on post detection and facial mesh and whatnot so make sure you subscribe and i will see you in the next one

Channel: Murtaza's Workshop - Robotics and AI

Views: 1,590,611

Keywords:

Id: 9iEPzbG-xLE

Length: 36min 55sec (2215 seconds)

Published: Tue Mar 30 2021