Build a Personal AI Trainer | OpenCV Python | Computer Vision

hey everyone welcome to my channel in this video we are going to create a personal ai trainer we will use the pose estimation running on cpu to find the correct points and using these points we will get the desired angles then based on these angles we can find many gestures including the number of bicep curves we will write the code in a way that you will be able to find angles between any three points with just a single line of code if you like to create real old computer vision apps do check out my premium course in which we learn how to create apps such as object detection augmented reality document scanner and a lot more the link is in the description below so without further ado let's get started so here we are in our pycharm project and you can see that this is the exact same one that we used in our previous video so we started off with the bare minimum code so you can see this is all of the bare minimum code that we require to run our pose estimation and then we created a module out of this so this is that module which allows us to create these projects very quickly so today will be one of these examples where we create a project very rapidly so then here is our awesome project so this was the demo of how you can actually utilize this to run your module so these were the things or these were the files that we created last time so if you haven't checked that video i highly recommend that you do go through that video before you continue here now today we are going to do the ai trainer so we have a new folder here called the ai trainer let's open that up and let's check out the contents so here we have a test image so basically the idea is that we want to find the angle of any three given points so point number one point number two and point number three so this is actually explained here in the summary so what we will be doing is we will be using these three points and based on these three points we are going to find the angle between these two lines so that will tell us how much angle we are at and based on that we can do some calculations for the gestures or for the poster so that we can tell the person okay you have done this many curls so what we will be doing is we will be counting the number of curls that a person has done you can apply to other techniques as well to see the posture whether they are using correct ones for yoga or something like that so the main idea is that we will do this in two steps the first one will be to find the angles so we will create a method where we we can input any three points and it will give us the angle of these three points so this way we do not have to worry about getting other angles for example if i want for the leg i will have that information as well if i want for the arm and the shoulder i can have that information as well for the elbow and the wrist i can have that so with one single line of code i will be able to have all of these different angles so i will just have to specify the landmark number for example uh for this arm it is 11 13 and 15 so i can say 11 13 15 and it will give me that one if i say 12 14 and 16 it will give me for this one so uh for sorry for this one so this is the idea that we will create this uh method and the second part is where we will try to find the angle and no actually we will have the angle and based on that angle we will see how many curls did the person do so this is our idea so we will start off with this is by the way the video let me bring it here so this is the one we will be using again both of these images and videos i got from pixels.com so you can check it from there or you can find these documents in my website so you will find this folder over there so what we will do is we will right click and we will create a new file and we are going to call this let's say our a ai trainer project so the first thing we will do we will import our packages so we will write here import cv2 then import numpy as np and what else do we need we need time so we will import that so all of these packages we have imported earlier so if you are new and you haven't seen the previous video you can go to settings python interpreter and you can add and here you can write opencv dash python and you can hit install and then you can write media pipe media pipe and there you can install so these are the two libraries that are the most important ones okay so once we have that we are going to import our image and the video so here you can see we have the image uh not this one the test image and the video so we will be using the test image at first for the angles and once we have that then we will use the video for the curls now we will write the while loop anyway so that we don't have to switch at the end we just can remove one line and it will convert to video so we will write here first that we have a video capture device so we will write here uh cb2 dot video capture and we will say this is the ai trainer what is it girls dot mp4 and then we will say while true we are going to check the success and the image and we will say cap dot read and oh and then we will say cv2 dot i am show we will write here image image and then we will write image and cv2 dot wait key and layoff one so if we run this we should get our video running so let's run that and there you go so as you can see this is quite a big video it's quite huge we can resize it we can write here image is equals to cv2 dot resize and we want to give it a specific one so we will write here one two eight zero by seven twenty so let's try that and there you go so now this is good and we are ready for the girls part but as i mentioned before we are going to use the image first and later on we are going to use this so we are going to comment this and here instead we are going to write image is equals to cv2 dot i am read and then we can read our image so it will be a i trainer slash what is it test dot jpg so we can actually put it outside so let's run that and there you go so we are getting our image so we will check the angle of this and we will see how we can calculate that okay so that is good now we need to find our pose so to find the pose what we have to do is we have to import our pose module so here is our pose module that we did in our earlier video so here we have the class pose detector we are going to use that to create an object once we have the object we can use find pose to find the pose and then find position to get all the data in that list so here we are going to say detector the detector is equals to oh i didn't import forgot to import so imports pose module pose module as pm and then we can write here pm dot post detector and we do not need to give any inputs at this point so then we can come down here and we can write here that uh detector dot find pose and do we need to input anything uh we need to put input the image and then we need to tell whether we want to draw or not so by default it's true so we will draw and what else so do we need anything back uh no not really we can yeah we need the image back so let's run that and there you go so now you can see uh okay this is not good we need to put it inside so that it's detecting again and again there you go so now you can see it is detecting the pose and now we can try to find the angle but how do we get the landmark values so we can use the get position so we will write here nlm list is equals to detector dot get position wait why is it not showing detector dot is it oh it's find position okay so find position and then we will write image and we don't want to draw so we will write here false i think there are only two arguments so we can directly write false yeah so that's fine so let's run that actually we need to print to see if we're getting anything lm list and let's do that and there you go so this is our list and we can see we have all these 32 points so that is good okay so now we can uh first of all we need to make sure that we have a list where we have uh the post detected otherwise it will give us an error so we can write here the length of our lm list is not equals to zero then we are going to do something magical so what is the magic that we are going to do so now here the thing is that we can write the code here but then it will be for this project only we can write here pass for now so that will be for this project only but we don't want to do that we want to we want to enhance our pose module by adding a method to it so this pose class which is the pose detector it will have another method that will allow us to get the angle of any three landmarks so instead of giving the points for example 485 and 281 what we will do is we will say we want to find the points between 3 four and five and because it already has this method and it already has this list we will make it an instant list so that it is for that particular object so then we will not have to even input the point value we just have to input the number so we need to know which landmark numbers we are talking about so how can we do that let's start by writing our code so the first thing we will do is we will create a new method we will call it find angle or angle no find the angle because it's just one angle then we will give in our image the image the image is for drawing so we will input that and then we will need the three points so we will call it p1 p2 and p3 so these are the three point uh landmarks that we need and then we can have the flag for drawing as always so we will keep it as true okay so now as i said instead of giving in the points we are just using the values so we are using value number four value number three you can call them index index number two three four so these are basic basically index values so what we need to do is we need to get the values of the points based on our index value so how can we do that so in the find position you can see we have the lm list so what we can do is here we can send this lm list back again to our object but that is not a good way to do it because we already have it we can just use it internally so we can write here self dot and now this is part of that object so we will write here self dot and we will write here self dot so now what we can do is we can write here that our x1 and oh x1 and y1 is equals to self dot self dot lm list at point number one so let's say we want landmark number three so that will give us this landmark number three but the thing is it has three things inside it has number three it has 485 and 281 so what we can do is we can slice it so here we will write we need for from point number one till the end so it will take this and this and it will ignore this so then it will store it in x1 and x in y1 you can also do it like this so you can ignore the first one and you can you can remove this one and you can ignore the first one and you can take just the last two but uh let's do the first method let's do it like this and then we are going to write here x2 y2 and then x3 and white y3 and here we are going to write point number two and point number three now to make sure we are getting the correct points we are going to draw so here we will write if draw we are going to write a circle so let's just copy because we are lazy and we will just change this to x1 and x2 okay so yeah that should be good we can copy this and we can make it x wait what did i do x1 x2 why did i do x1x this should be y1 i always make that mistake y 2 x 2 and then y 2 and then x 3 and then y 3. okay so that should give us but we didn't call it so we need to call it here so we are going to write that detector dot get angle or find angle and we will give in our image so now we need to give in the points so if we go to media pipe and we check for the points you can see that we want point number 11 13 and 15 and then 12 14 and 16 so based on if we want right or left so the left one is the odd one so 11 13 and 15 is the left one so here i think the right one is visible so we will use the right one so 12 14 and 16 and the draw we will keep it as true so now let's run this and see what happens and there you go so now you can see these have turned blue so we know that these are the ones that we are using uh should we decorate it more or should we do it later let's do it now so what we can do is we can make it look a little bit nicer uh just to make sure that we are using these correct points because uh at some point we are going to remove all the other what you call the detection the pose estimates so here we are going to creates a bigger circle and let's put it as 15 and let put this as 10 and we will not fill it and we will put the value of 2 here and for the color let's put it as red so here we are going to write 255 and we are going to write here all of these as red um yeah and then what else so we will copy this we'll paste it here and paste it here we'll make it two and two and then three and three so let's try that there you go uh oh we forgot to remove the filled no we did remove the fill oh we didn't do the size we forgot to change the size okay there you go so these are the three points and what else we can do the line as well so let's do the line before that because uh the circles we want to draw on the line so we will write here cv2 dot line and the first line will be on the image and we will have two points and then we will have we will put the color of white so it's really visible and then we will put the thickness of three so here our first uh what do you call points will be x1 and y1 and then the second one will be x2 and y2 we will copy this and here we will have what happened there here we will have x3 so 2 will be common for both of them 1 and 3 will be changing so let's try this uh nothing happens why didn't anything happen because i put this in the wrong place this should be outside okay there you go so now we have the white line and so now we can go back and we can make this false so that we can just focus on these three points and the rest will be gone so there you go so now we have these three points and we want to know the angle between these points so we will go back to our module and here now we need to find the angle so the angle finding is not actually hard so here we can write for example get the land marks and here we are going to write calculate the angle and then here we will write draw well you know it's written draw here and we're writing draw here very redundant but anyways so we will write here angle is equals to we are going to use math so we will write here imports import math so this is just basic trigonometry so we don't have to worry too much about this so what we can do is we can write here math dot tan two sorry a tan 2 8 and 2 and then we have to give in y 3 minus y 2 and then x 3 minus x 2. then we will subtract and we will write here math dot a tan 2 and then we will write y 1 minus y 2 and then we will write x 1 minus x2 so again i'm missing some brackets so that should be good so this will give us our angle in radians so we can convert it into degrees so math dot degrees and there you go so if you have three points and you want to find the angle between the two lines the this is the method this is the equation that you can use so here we can simply write print angle and we can see what exactly is our angle okay uh what happened there so there is a mistake for sure it's the brackets yeah this is extra so yeah i put one extra my bad okay so here we are getting our angle 87 which makes sense like it is almost 90 degrees you can see that so 87.3 it's not bad so what we can do is we can put this on the actual text or the actual image so that we can see so we can write here cv2 dot put text and we will write image and we will write the angle but let's convert it into integer and then we will convert it into string otherwise it will not accept and then we will write here uh the value so the position so here we will write x2 so this is the center point so we have x1 and x2 these are the further points and x2 is the middle point so we want to write the value near the middle point so we are using x2 but we don't want to write it exactly at that position so we can subtract like 20 from it and then we can write y2 and we can let's say add 50 to it so we can change these values if we are not satisfied okay then we can write cb2 dot font let's pick the plain one and then we will write the scale and the color so color let's put purple and then we will write the thickness so let's run that there you go so we are getting 88.87 so that's good uh the x is bad so let's do we did minus right so minus 50 let's see yeah it's better now maybe let's convert this into blue that's more visible now the background is a little bit black so it's not that visible maybe red will be more visible or green yeah red is not bad so 88 degrees is what we are getting and sometimes what happens is that we get a negative value so for that case we can write here should we actually let's remove this and for those cases we can write if our angle is less than zero then we will say our angle we want to add 360 to it so 360 minus whatever the value will be so that will solve that problem so now what we can do is we can use any three points to find our angle so let's say i want for this is for the right one so we can say here right right arm let's say and then i can do the exact same thing with just one single line of code i can write here left arm and i will just change the values 11 13 and 15 and now you will see it will do for both of them and that is pretty amazing so it is telling the angle for both of them at the same time so uh the last uh the the one that is hidden is not very clear so you cannot rely on that angle but the one that is here is quite good so we can remove this so now we can try this on our video so for the video as i mentioned we want to do it on the left arm rather than the right and by the way you can do it on the legs as well so it it's up to you now which three points you want to take for example you can take 23 25 and 27 so it will tell you the angle between uh these two lines then you can use 24 26 28 it will tell you the angle between these so it is up to you which ones you want to use for your own uh project but for now we will use the left arm for the bicep curls so now we will remove this we will uncomment this and we will remove the image part and let's try to see if we get the angle there you go so now we are getting our angle and as you can see probably it's going to negative and all so that's why it's giving uh this value so what we can do is we can find the minimum and the maximum and based on that we can check if it has reached that point or not so let's try to figure out the minimum and maximum now the good thing is if you click on the window it will stop so we can check our values very easily so here you can see it's 338 but that's not the last one so here i saw 190 something yeah 180 probably or let's say 200 yeah 200 is not bad uh or just to be on the safe side we can keep it a little bit higher like 220 or something like that so that it goes to zero easily and then over here we can see it goes to 300 something so 340 but again might not go always there so 3 30 maybe 20 or 310 i can see it goes again and again to 328 so it went to 340 340 oh it's going to 40. now it's 28 27 20 time okay so to be on the safe side we can we can say that it is um 20 right so or we can say 15. so what we can do is we can convert our range we want it from 0 to 100 we want to know how much curl uh we are at so the percentage of curls so at the zero point or at the hundred percent point so what we will do is we will create a percentage and we will say that uh did we import numpy yes so we will use numpy to convert our range so we will say numpy dot inter interp and then we will given our angle not range angle oh we didn't get it back that's why it's saying this angle is equals to so i think i forgot to return yeah so we need to return return angle so that will return our angle over here and then we can use this here angle is equals to uh what else do we need so we need the first range so our range will be let's say 210 to 310 let's say and we want to convert it into 0 to 100 so this is already 100 so we can easily just subtract but if we had a difficult range you can use this method so that should be good so let's print out this value and we will print out the angle not the angle the percentage so or we can print the angle and the percentage that would be good to see are we printing anything [Music] here no okay that's good so we can run this so here we can see so it's zero going hundred percent then going back to zero then two hundred percent going to zero hundred zero excellent so now we can see that we are getting the correct values so we should not face any issues because we have taken quite safe values if you go too far you might not get good results so you have to check the trade-off between accuracy and taking the risks of getting error okay so then we are going to check when are we reaching the the first curl when are we reaching the second we need to count so here we are going to go up and we are going to define two things the first one will be count which will be zero and the other one will be direction which will be zero now we will have two directions direction number zero and direction number one direction number zero will be when it is going up and direction number 1 will be when it's going down so we will consider a full curve only if it does both of these so it goes up and then it goes down to 0. so from zero to hundred and hundred to zero so if we get that then we will consider it as first curve so we could do the other way around but i want to keep it like this where we have the complete girl if we go back and forth the whole thing so we are going to write here that we want to check check for the [Music] okay so then we are going to write here that if our percentage is equals to 100 and then we are going to check if our direction direction is equals to zero so this is the first direction it means we are going up now you might say why didn't you just write and here if percentage is hundred and if direction is zero i will tell you why later there's a reason so then we are going to say count plus equals 0.5 so we will add 0.5 to the count so if it's going up and it has reached 100 it will be 0.5 and then it's going down and it reaches 0 then it will be 0.5 so that will be a complete curve so what i can do now is i can uh change the direction so direction is equals to 1 and then i can check here that if my percentage is equals to zero uh and my direction is equals to one then i will write count is equals to plus is equals to 0.5 so the same thing that we did earlier and then we will make the direction one uh no we will make the direction zero okay so this will keep adding to our count and this way we will know which direction we are moving and how many counts uh did we have so far so here we can remove this and we can simply print the count so we can write here print count and let's see what do we get so 0.5 down one 1.5 down two two point five down three three point five down four so excellent so if you um okay let's just display it first and then i can discuss so we can display it like this cv2 dot put text and we can write here image and we can write our string so we can write inside that integer so if you want to directly show the decimal places so if you want to show the 0.5 count as well then you can keep it like this so you can write simply count but i don't like that or let's try it now i will show you and then we will change it and then we will write here let's say 50 and 100 and then we will write cb2 dot font let's put the plain one 15 and then we will put 2 5 5 0 0 and then 25 actually these are very big values we will put big values later on but for now we just want to see if we are getting the output properly okay so what is the problem not callable is that the issue oh okay i forgot to write comma okay so here we are getting 1.5 to 2.5 3 3.5 that is good so what i was saying is that if you don't want that you can simply write here int and then you can put it like that so if we see that now it's zero it becomes one two three so it's up to you which one do you prefer okay so now that we have this we can make it a little more appealing so what we can do is we can first of all add our fps so we didn't add that earlier so we can add it now so we can write here c time is equals to time time dot time and then we can write fbs is equals to one divided by current time minus the previous time and then we will write our previous time is equals to current time and then we will put the text so we can copy this and we can paste it here and then we can change it to fps so that should be good now we need to put previous time as zero over here okay so that will give us the time and we can comment this to check the fps so there you go so we are getting good fps now this video is 1080p and we are reducing the size of it and that's why it's giving a lower frame rate otherwise you will have a better frame rate if you because here we are resizing so if you directly use maybe this size or even lower you will get higher frame rate okay so then what we can do is we can put our what you call the number of count in a box so let's create a box so i have already checked the values of this box so i will directly input it so we will write here cv2.rectangle rectangle and we will put in our image and we will put in 0 450 and then 250 and 720 so this is for a 1280 by 720 which is hd image so it is for that so 255 and 0 and then cb2 dot filled so this will give us a green box and what we can do is we can uncomment this and we can format it and then here we need to change the value let's put it as 45 and 670. so let's run that there we go uh okay we need to change the size of this so 15 and 25 25 okay so there you go so now we are getting this nice and big so we can see what is happening that is quite good and it is giving us the angle do we need the angle uh no i don't like the angle let's remove the angle from here so let's run it again there you go so now we are getting without the angle so it looks good now what else can we do okay let's put the bar so if you remember we did it in one of the other videos for the volume gesture control so you can check that out as well it was to control the volume of a computer using your hand gestures so we use the bar in that one as well so here we are going to use the bar again so for the bar first of all we need some values so we will write here bar is equals to np dot interp and we will convert our angle from the range of 220 to 300 uh not 300 310 and we will put it as 650 and 100 so this is the maximum value of our bar no the minimum value of r bar and this is the maximum value of our bar because the opencv convention is opposite so this is the minimum this is the maximum okay so why is not giving a space here okay so then we need to create that rectangle so here we can create that rectangle we will copy this because we are lazy we will copy twice and we will also put the text okay so now we need to change the values here so the starting will be 1100 and then 100 then we have one one seven five and we have 650 then we have uh what he called 1100 here and then we have our bar value so integer bar not bad bar and then we have eleven seven five and then six five zero so for the text we will have our percentage so we will make a string and we will write here integer and then we will write our percentage and at the end we will write percent so this we will display at 1175 and the size will be four and four okay so we can write here that this is for our bar draw bar and let's say we write here uh show or let's try draw uh girl count okay so then let's try it out there you go but [Music] the bar value is not changing what did we do wrong oh so apparently the value of the bar is not changing because it is filled come on so this should be let's say three and let's try that yep so it goes down it goes up it goes down it goes up now one more thing we can do is when it reaches the zero or the maximum position we can change the color so that we know it's it's like when you reach a certain point and or you press a button it changes color so you can you kind of get a feedback so it looks good so that is the reason why we put an if within an if so here we are going to write color is equals to by default we are going to write the color as purple and if it reaches we are going to change the color as green so we will put the color as green so two five five and zero and the same thing we can do for our zero if you want to change the color for zero make it different than the other ones you can do that too but we are going to keep it like that so here we are going to write we are going to write here color and then here we are going to write color and here we are going to write color so all of them are same so now it's green purple purple green purple green green yeah so that's how you can tell if you have reached the correct point or not so as you can see it works very well and again if you want to use a webcam you can simply enter the value or the id number of your webcam and it should run pretty much the same way just make sure that your face is visible uh within the camera and you are at a good distance because a lot of this depends on the face as well so if you are not within the if your face is not in the camera it will not detect properly so that is pretty good so i i had a hard time finding uh these videos with bicep girls so that's why i just had this one but i will try to find some more but yeah so that is the idea the error is because the video ends that's why it gives this error it's not something that we made a mistake here so this is quite good and you can see that how easy it is to create a virtual trainer a personal trainer which is based on ai it will tell you how many push-ups or how many pull-ups or how many curls you have done and it can also help you with the posture again this was just one of the examples you can take it much much further and you can do lots of different things with it so i hope you do that and do remember to share with me tag me in the video whenever you have some output whatever you create something new i will be very glad to see what do you guys come up with so this is it for today's video i hope you have learned something new if you like the video give it a thumbs up if you haven't subscribed do subscribe a lot of you are watching the videos but you haven't subscribed yet so please do subscribe and if you liked it if you loved it share it with your friends help spread the knowledge and i will see you in the next one

Channel: Murtaza's Workshop - Robotics and AI

Views: 79,259

Keywords:

Id: 5kaX3ta398w

Length: 48min 38sec (2918 seconds)

Published: Sat Apr 10 2021