AI Face Body and Hand Pose Detection with Python and Mediapipe

what's happening guys my name is nicolas schruner and in this video we're going to be taking a look at full body pose estimation using media pipe holistic so what we'll be able to do is use our webcam to detect our facial landmarks or all the different landmarks within our face so for example our eyes our lips our nose and our eyebrows for example we'll also be able to detect all of the gestures from our hands so all of the key points within both of our hands as well as our poses from our body now this video forms part of a larger video or larger video series where i want to be able to use this tech to be able to detect different forms of body language so whether or not you're happy whether or not you're sad whether or not you're feeling nervous or anxious so on and so forth so this is just the beginning and let me know if you'd like to see the broader series as well i think it's something that's really interesting but i'd love to hear your thoughts without further ado let's take a deeper look as to what we'll be going through so the first thing that we're going to be doing is installing media pipe now media pipe is a great python library that actually comes with a bunch of pre-trained deep learning models for a whole bunch of use cases now the model that we're going to be using is called media pipe holistic now media pipe holistic is actually made up of three different models one to detect our facial landmarks one to detect all the joints within our hands so we'll get hand tracking as well and the last one to detect the poses within our body so it's actually a full body detection model now this obviously opens up a whole heap of opportunities with this particular model so you could use it for action detection people have used it for workout counting so for example counting the number of bicep curls or squats or push-ups that you might use again these are things that you'd like to see me do tutorials on by all means leave a mention in the comments below i'd love to hear from you so once we've installed media pipe what we'll do is we'll actually go about detecting our different facial landmarks and our hand gestures and our body poses and we'll actually display those results to our screen using nothing but our webcam and python opencv let's take a look as to how this is all going to fit together so first up as i said what we're going to be doing is installing media pipe this is just a straight pip install then we'll leverage opencv and our webcam to actually make real-time detections and we'll be able to render those results to the screen using opencv but again if you'd like to see some of those more alternate use cases so body language decoding or exercise counting i'd love to hear from you so do mention it in the comments below and we can start building that up on that note ready to do it let's get to it alrighty guys so in order to get started with our full body pose detection model there's going to be a bunch of stuff that we need to do now the end goal for this is to be able to have a full-blown pose tracking model that allows us to detect all of our facial landmarks our hand poses as well as our body poses as well now in order to do this we're going to be leveraging a library called media pipe so media pipe actually has a lot of stuff in it so specifically if you actually take a look and again this link will be in the description below there's a bunch of different types of models available within media pipe now the one that we're going to be using is a model called holistic so you can see that over there so this actually gives us a bunch of different types of models that we're able to use in real time and specifically it gives us that facial landmark detection model the hand pose model as well as the full-blown pose tracking model now we're going to keep this particular video quite simple but there's a whole bunch of use cases that you can actually use with this model so say for example you wanted to do exercise tracking like a squat detection or like count the number of squats or bicep curls you're doing you could do it with this model um some of the other cool ones that i've actually seen if we actually step into this model if you actually scroll on a little bit further down there's a guy i might not be on here let's take a look might be on the github so let's take a look over here yeah so it's on this this page so you can see that this guy's actually controlling his screen using media pipes so he's actually got a touchless control demo so here he's actually selecting different points and this video is actually entering in keys using the media pipelistic model so it's pretty cool what you can do with it and now again we're going to keep it simple but you can definitely extend this and do a lot more with it now one of the ideas that i actually had as well is the ability to detect different body language components right so whether or not you're happy or you're sad or you're nervous you're anxious or you're victorious so again this is another thing that you could do with the media pipe holistic model alrighty enough talk let's actually get into the nitty-gritty so the first thing that we're going to go ahead and do is install and import our dependencies now there's two key dependencies that we're going to need here these are media pipe itself so you can see that we've got media pipe here and we're going to be using media pipe for python but again you could try this out in javascript c plus plus android ios whole bunch of other languages the second dependency that we're going to need is opencb so let's go ahead and install those so we can do it using a regular pip install command and hit enter so really all we've written there is exclamation mark pip let's make sure zoomed in install media pipe and then open cv dash python so those are our true dependencies installed so really that's all you really need to do in order to get things up and running now the next thing that we need to do is actually bring these in to our notebook now again as per usual all the code for this particular tutorial is going to be available in the description below just check it out it should be by github so let's go ahead and import our dependencies okay so we've got our two dependencies imported there so we've written import media pipe as mp so this is going to give us our media pipe model specifically our holistic model and then we've also imported opencv so to do that written import cv2 so cv2 is going to be used to access our webcam and do all of our image-based visualizations if you haven't worked with opencv before it basically stands for open computer vision so it's a really powerful open source computer vision library again mediapipe's going to give us a holistic model as well as some drawing utilities so so far what we've done is we've installed our dependencies and we've brought some in the next thing that we need to do is actually set up media pipe so really it's two lines of code we just need to set up our drawing utilities and load our holistic model so let's go on ahead and do that okay so those are both of our components now set up so we've written two lines of code there again so first up what we've done is we've imported our drawing utilities now these are going to help us to draw our different detections from our holistic model to the screen and specifically to opencv once we've made those detections and then the second line is actually importing a holistic model so to do that written mp underscore holistic equals mp.solutions dot holistic the drawing line is mp underscore drawing equals mp.solution start drawing util z now inside of mediapipe you've actually got a bunch of stuff right so if you type in mp.solutions dot there's actually a bunch of different types of models in there so we've got a face detection model face mesh model hands model holistic which is the one that we're going to be using the media pipe base case or base class objectron and pose now if you actually go back to media pipe there's a bunch of different types of models so there's face detection face mesh iris this is one that i'm really interested in so doing like gaze detection that might be cool uh hand tracking post tracking holistic hair segmentation let me zoom in on these object detection box tracking i've just minimized that let's bring it back uh knift autoflipped media sequence as like an nlp model now some of these are only available in certain languages so you can see that there's quite a fair view available in android some available in ios a bunch available in c plus plus few in python a few and js and a couple in coral as well now in this case we're mainly going to be using the python library but again you can play around with a whole bunch of the other models and see how those go as well in this case we're pretty happy with that so we are now going to go in ahead and first up we're going to get a real-time webcam feed using opencv so you'll be able to do it using your webcam but you could just as easily load up an image and apply a list the holistic model to that as well so let's get our webcam feed and then we'll copy that code over here apply our detections and copy that code over here and apply our styling so we should have three different levels of detection so i want to sort of take this step by step and show you how everything's building up so let's get our webcam feed okay so we've now gone and written the code to get our webcam feed so i haven't run this yet so we'll take a look and see whether or not we've got any errors in a sec but i wanted to break this down so if you've ever worked with opencv and worked with a webcam feed so if you've watched any of my object detection videos this code will be very familiar to you so what we're doing is first up we're getting our video capture device now this is this number that we actually pass through to this line here is going to vary computer to computer so you want to test out basically what you're trying to do is find out the device number for your webcam now on most computers it's going to default to zero because you've only got one webcam available on your computer but if you've got others or if you've got a virtual video capture device then you might need to play around with this so normally i'll test 0 1 2 3 4 5 and so on until i get my video capture if you get a weird error where it sort of says the image is empty or image right equals zero then you know that you've probably got the wrong video capture number so in this case play around with it and it should idea and you should ideally be able to find the right number so that first line is cap equals cv2 dot video capture and then we've passed through the device id for our video capture device which is zero then what we're doing is we're basically just looping so written while cap is opened so basically while our video capture device is open then what we're going to do is read that video capture feed so to do that with written ret comma frame so this is just extracting the two values from our feed equals cap.read so cap.read is the call function there so we're basically reading the feed from a webcam then we're going ahead and rendering those results to the screen so to do that we've written cb2 dot i am show and then we've named what we want our frame to read as and in this case we've just written holistic model detections but this could just as easily just be a raw webcam feed to represent our first layer and then comma frame so this is the image that we're actually going to return to the screen now the size of the image is going to be based on your video camera unless you go on ahead and resize it and then what we're basically doing is we're looking for whether or not we're breaking out of that loop so if cv2 dot wait key and then we're passing through 10 and and and then this is basically just looking for whether or not we're hitting q on our keyboard to quit out of that frame this sometimes doesn't work when you're running this code inside of a jupyter notebook but if you're doing it inside of a raw python script should work fine so you can sort of take this these two lines with a pinch of salt if you're running it from a jupiter notebook in this case we just want to have it there in case we wanted to export this later on and then assuming our while loop ends then we're going to release our camera and we're going to destroy all of our cv2 windows so sometimes or when you see this run what you'll see is you get a little white pop-up from python that will pop up sometimes it doesn't want to close so you've got to run a cap.release to release your webcam and then cv2.destroy or windows to actually close that window so if we run this ideally and looks like we've got an error already this should be a single amp send should get a little python pop up might take a little while to start you can see that's running all right so it looks like we've got our webcam feed then you can see we're getting reasonably high frames per second so we've now got a baseline webcam feed and i'll take the green screen down later so we can actually see some appropriate results but no issues there right so we're getting our different feeds looks like it's all working okay so what we're going to do is we're going to stop that cell right now you can see that our raw webcam feed is still open down the bottom there so our cap dot release and cv2 dot destroy or windows didn't run so what we can do is actually copy those two lines paste it here that will release our webcams i've just seen my webcam shut off and you can see it's also closed a little python window down there so that's our baseline use case done so we're now able to get our webcam feed now the next thing that we want to do is actually make some detections from that feed and this is where we're going to start to use our media pipe holistic model so let's go ahead and do this we're basically going to start out by copying this block of code here creating a new cell and then we're going to overlay our media pipe holistic components onto the top of that so let's take a look at all the new lines that we need to write well let's write them and then we'll take a step back and see what we've done okay so before we go on ahead and run that let's take a look at what we've actually done here so we've written three new falling four new lines of code so the first one is opening up our holistic model and setting it up so basically written with we're using with statement and then we're accessing our mp holistic model dot holistic so this is effectively accessing this up here so mp holistic and it's grabbing the actual holistic model and we're setting two parameters so the minimum detection confidence and we're setting that to 50 and we're also specifying the minimum tracking confidence again so we're specifying that to 50 so if you wanted a model that had higher tracking com or higher tracking accuracy results shown to the screen then you would set these both of these components as higher values then we're going to be working with that as holistic so this is sort of just a naming convention so rather than having to write out this entire line we've just written as holistic so we can refer to it as that and then the next two three lines that we've written are over here so the first one that we're doing or the first line that we've written is recoloring our image so basically we're grabbing this frame that we had from over here and we're passing it to the function cv2 dot cvt color passing that frame and then we're recoloring it from bgr to rgb so when we use opencv we get a feed in bgr but when we actually pass it to our holistic model we want the color of that image to be represented in rgb so we perform that recoloring over there then we take that image which is being converted from bgr to rgb and we pass it to our holistic model so we write holistic dot process that particular image so we're getting this model dot process and then we're processing that image to be able to display those results now right now we're not actually drawing anything to the screen we're just going to be printing out what our results look like so we should get this big running feed of random numbers and effectively all about detection so not random they're actually detections from our model so let's run this and we should see our python pop up and then we'll start to see a whole bunch of stuff printed out here okay so you can see we've got our pop-up and now we've got all of our media pipe output so in this case we're just seeing the class again we can stop this and if we actually wanted to see our results or for a specific model we can type in results.face landmarks let's try that first run that again we should have killed our feed that's fine we'll see if it works okay so you can see our feed still running but now we're actually displaying all of the different landmarks for our face model now there's a whole bunch of different types so there's our face landmarks there's also the landmarks for our poses so if we change that so if we write pose landmarks so this is now going to display the landmarks for our poses and this is for our body poses but basically this is accessing the different landmarks for each of our different models remember there's the face model which we just showed there's the pose model and then there's the hands model so there's specifically landmarks for our left hand and our right hand as well you can see all of these displayed here to the screen now right now that's it's nothing all that crazy we actually want to see them drawn to the screen so let's go on about and actually draw first up our facial landmarks and then we'll sort of build up again so if we stop this again then what we can do is rather than printing out our results we can actually start drawing them to the screen so let's go on ahead and do that and let's just release our camera as well cap we can run this cell up here cool all right so let's go on ahead write those couple of lines of code and then we'll start drawing them to the screen okay so that's at next couple of lines of code done now what we've gone and done here is we're basically recoloring our image back to bgr so remember we converted it from bgr to rgb now we're converting it back to rgb now we're converting it back from rgb to bgr to be able to render it using opencv because remember opencv loves bgr and that's how it wants its image is represented otherwise you get some weird colorization when you actually display it inside of the rjb format using opencv so what we've done is we've basically copied this exact same line and just pasted it down here so again we've got image equals cv2 dot cvt color passing through our frame this should actually be our image and then we've written cv2 dots color underscore rgb to bgr so basically we're recreating this line so this line had our frame and now we're taking our processed image and converting it back then what we're doing is actually getting to the good bit so we're actually going on ahead and drawing our different landmarks so in this case we've started out by drawing our face landmark so we've written mp underscore drawing dot draw landmarks and then to that we'll pass through our image which is this over here the results from our face landmarks remember we just printed those out over here so rather than having results up pose landmarks we could have results underscore face landmarks oh god my typing's a bit of a shocker today face landmarks so it's no different to that so we've got face landmarks we've got pose landmarks we've also got so let's just write them out so we should have four out of here so we've got base underscore land marks pose underscore land marks and then we've got left hand landmarks and right hand landmarks so basically you've got all four of these different sets of landmarks that you've got accessible to your model so in terms of how we actually draw these we use our drawing utility so remember we set these up over here and then to that we're passing through three different things also the function that we're actually using is mp underscore drawing dot draw landmarks which is that function there and then two that were passing through three things so our image which is this over here the results dot face landmarks component which was this result set that we had over here and then we're also leveraging the pre-written mp.holistic face connections model so if i actually show you this this actually shows the relationship between all of the different components within our specific detection set so basically it's showing you which landmarks between each set of models actually connect to each other so when you think about a mesh we're actually drawing up a mesh so this is representing which point connects to which other point within those specific sets of detections now there's those available for face connections i believe they're available for pose connections so you can see that the nose connects to the left eye inner nose to the right eye inner should be the same for hand and connections so you can see that there's all of these different sets of connection sets so basically they're showing you which joints connect or which landmarks connect to which other landmark so for our face landmarks we're using our face connections but if we're doing a hand model then we use our hand connections in this particular case all right so so far we've drawn to our image now what we need to do is actually change the image that we're going to be rendering because right now we're rendering our raw frame we want to render our drawn image now so let's change that and if we run that now ideally we should be able to see some results to the screen and there you go so we're now getting all of our different face landmarks drawn to the screen so you can see all of that detections so if i open my mouth move my head and you can see it's tracking pretty well now again i've got a gpu on this model so it is running quite quickly but you could test this out on a machine of your choosing so i can move around and you can see it's appropriately tracking all of the different landmarks in my face even if i move backwards and forth so if i take the green screen down it's still going to work reasonably well pretty cool right so you can see that's our first level now what we want to do is apply the same thing but for our hand landmarks and our pose landmarks as well so our body and our hands so let's go about and do those green screen back up and we're good to go all right so what we're going to do is stop this and then as per usual we're going to release our camera and close our little python window which we've still got down here and now what we're going to do is draw the rest of our landmark so remember we've got our right hand our left hand and then what did we have as well our pose detection all right so basically what we're going to do is repeat this line over here so if we copy that there and copy that there and copy that there what we're effectively going to be doing is of replacing each one of these components so here it's going to be right hand and marks and then rather than using our face connections map we're going to be using our hand connections we're going to use our hand connections for our left hand landmarks so let's change that left and landmarks and then for our pose detection landmarks we're going to have pose landmarks and believe this is pose connections and we're also going to get rid of this printing line over here because otherwise it's going to start using up all of our memory in our notebook and if we hit save so let's quickly take a look at what we've written there so we've now got our right hand landmarks left hand landmarks and our pose detection landmarks so again what we've done there is we've really just reused this same line that we did for our face landmark so rather than having our image our results and face landmarks and our face connections we've now got our right hand landmarks our left hand landmarks and our pose landmarks and we've got the corresponding connection maps as well so here we've got our hand connections which is what you saw down here for our pose connections we're using those ones and this just shows each part of your body that connects to the other parts right so in this case you can see the so say if i take a look at this line so we can see that our right wrist connects to our right pinky so what's effectively saying is that our right wrist is connecting to our right pinky so it's effectively that connection there but again you've got it for the full pose landmark model now in this case we've just gone and applied the same thing so these three new lines are going to give us all of our additional detection components so i believe that's really everything that we needed to draw now so we've got our right hand landmarks our left hand landmarks and our pose landmarks let's go on ahead and run this and see how we go so again i'm just going to run that cell and ideally we should see our python thing and pop up again you can see that we're getting that now so right so we've got our face component so you can see that that's all working but you can also see that i've got the beginnings and i've given it away you can also see that i've got the beginnings of our pose tracking model so you can see it on my shoulder here and on my shoulder here but we've also got our hand components as well now so if i bring up both my hands you can see that we've got all of those by bringing in my elbows you can see the connections to my elbow so we've now got the full model working right so if i actually take a look so you can see that we can move all of those around and the cool thing about this model is it's actually really accurate so it's quite fast and it's also super accurate so if i bring my hand to the screen you can see let's bring it there you can see that it's detecting all of those key points relatively accurately right let's move how do we move out of the way there you go right so it's getting all of those key points moving around pretty quickly the only thing that i don't really like about this is the fact that it's all the same color but we're going to sort that out in a second so you can actually see two components there right so i've got this really specific hand pose model which is moving around but then i've also got these um these lines here so that one that one and those those three little points inside of my hand are actually from the pose model so if i actually disable the hand model and the face model you'll actually see the full results from just the pose model alone because you actually get quite a fair bit out of that so if we stop this and if we comment out our face landmarks model because these are really specific models or they're really granular right so if i comment those out so i'm just commenting out our face landmarks model and our right-handed model and our left-hand model or drawing hit save and then run this ideally what you should see is the result only from the full pose detection model so let's take a look at that right so you can see that now so the pose detection model still has some components of a facial landmark model and again it's in 3d which is pretty cool so it's got the top of my lip it's got my nose it's got my eye to a certain extent and the edge of my what is that my cheekbone but you can see it's also got a relatively basic hand model as well so again it's not as granular as my left hand right hand model but again you can still see that it's sort of detecting those okay now the cool thing about the pose detection model is that it's a whole of body mode if i actually step away you can see that it's actually split out of the way you can see that it's actually detecting my whole body so again pretty accurate can do a whole bunch of stuff with it the only thing that i don't like about this as i was saying the fact that was probably really quiet because i was away from the mic but the only thing that i don't really like about that model or don't like about this right now is the fact that it's all the same color so that's our pose detection model let's comment this out and take a look at each one of these individually so let's take a look at our face landmarks model so again i'm just commenting out pose i'm just commenting out the pose detection model and leaving the face landmarks model available so again you can see our face landmarks model pretty accurate right it's pretty cool so it's doing our lips eyebrows i'm just messing around with but you can see it's actually doing really cool stuff now again we can comment out our face landmarks model and just leave our hand models again we'll save it run that again i think we ne yeah we need to stop this first and then run it sorry my bad okay cool so now you can't see our face model and you can't see our pose model but now if we bring up our hands that's pretty cool right that's my uh a half-assed attempt at actually producing some cinematic sound with my mouth but again so you can sort of see that now i've got some really specific models so again the hand pose model is way more granular and way better than the actual uh the hand post tracking that you're getting from the pose detection model so you can see it's got a lot more key points so it's got the top of our finger all of those different joints so top of our finger this joint this joint this joint so on it's all the way down to our wrist but again pretty cool right so you can start to mess around and if you want to do or start to begin to build different components of your like controlling your screen or controlling games again you could start to do this one awesome integration might be integrating this with pygame or even so i've seen scent decks do some awesome stuff with like game control so you might also be able to integrate some of that stuff as well but again opportunities are endless so again if you've got some awesome ideas let me know in the comments below i'd love to hear about them and maybe start building some of that but in this case again my ocd is getting the better of me and i hate the fact that everything's visualized in the same two sets of colors so what we're going to do rather than displaying it all in just this green and red which is kind of ugly again my ocd kicking in we're going to recolor these and apply a little bit of styling and it's pretty easy to actually apply this styling we can there's a um mp underscore drawing i believe it's a drawing spec which we can actually apply to this to recolor these components so let's stop this as per usual and we're going to uncomment our face landmarks and our pose detections and just to make this clear i'm going to copy this into its own section so again same code we're just going to bring it down to here to apply styling and then what we're going to do is to the end of each one of these components remember we've got our four different components we've got our face landmarks we've got our right hand model we've got our left hand model and then we've got our pose detections model what we want to do now is actually apply our drawing spec so inside of our drawing utility so mp underscore drawing we've got this thing called a drawing spec right question mark question mark you can see that you're through this you're able to pass through three key things so the color the thickness as well as the circle radius again you can tune all of these but basically it's going to give you the ability to specify what types of color thickness and circle radius you want for each one of your components in your detection model so in this case we want to define two key components so we want to define the or whenever we actually go and pass these to these models down here we're going to want to pass through two key components so we want to pass through the line color as well as the circle color so let's go ahead and do that so we're going to set up a drawing spec so really it's just going to be this and then we're going to pass through our color so say for example 0 255 0 is going to be rgb so that's going to be super bright green then we can also pass through our line thickness and our circle radius so let's go ahead and define those so what are they they are keyword arguments we need to say color equals thickness equals and then circle radius equals two again so you can play around with all of these colors but basically what we're now going to do is take this drawing spec rather than leaving it as 255 let's actually set that to zero set that's 255. so this should be super bright red so let's make it blue because we've seen red already right so if we copy that now to our drawing landmarks let's actually take a look at this right so mp underscore drawing dot draw or landmarks so you can see that down here we've got two components that we can pass to our draw landmark so remember we've used our mp underscore drawing draw landmarks function to draw those landmarks right down the bottom you can see that you can pass through two drawing specs so we've got the landmark drawing spec and the connection drawing spec so the first parameter that we're going to be passing through is the actual joint effectively right so the landmark is the joint so or in our hand in this case it's going to be the joint but it's also the dots effectively think of your landmark as the dots that's a better way to think of it so the first parameter that we pass through after our draw landmarks function or the fourth parameter really is going to be the color of our dots and then the second drawing spec that we pass through is going to be the color of our lines or our connections so let's test this out on i think the hand is probably best to test this out because you'd actually see it really clearly so let's do our hand model first up so if we copy this drawing spec to our right hand model so if we just put in a comma here and then a another comma so effectively what we've gotten done is we've got our three baseline parameters our image our landmarks and then our connections then we're passing through our drawing spec and i've just done our right hand for now so in this case they're both going to be blue but we can change it so we could have our so our landmark is going to be 255 and then we might make our line i don't know 240. so again this is just a regular rgb uh color spec right so you could change this and say for example we wanted our circle radius to be four we could change that as well so if we save that now and run this let's just make sure that we release our capture first up so if we run right back up here so again just run cap.release cv2.destroy or windows and now if we go and run this ideally what we should see is that we've got different colors in our right hands our left hand will still be the same color but our right hand will now have different colors and a different circle radius because we've applied this drawing spec so if we run this and wait for our python pop-up as per usual right cool so no change to any of those models so remember only change the right hand so if i bring up my right hand now it looks like we've got different coloring and it's red so rgb oh so it's bgr sorry my bad so if we go change these 255 240 set these to zero i believe it's bgr let's see try that again but we got to stop this yep it's stopped run it let's take a look now cool so that's reopens the left hand no change all right cool so we've now got our change in colors for our right hand model so you can see that we're now getting blue detections and our circle radius is larger cool right so we can play around with these colors so say we were to go and do this let's go ahead and do it for all of our models and then we'll actually see the end result so effectively what i'm going to do is i'm just going to copy these drawing specs to all of our line items to all of our different detections so if we paste that there paste that there and then paste that there so you see these getting the better of me gotta have these lined up so basically what we've gone and done is we've now just gone and applied our drawing spec to our face landmarks to our right hand component to our left hand component and to our pose detection component now again all of this code is going to be available inside of the description below by github so if you want to check this out by all means go ahead and grab it now i've got some colorings that i wanted to apply so it's going to be 80. so let's let me power out and then we'll uh we'll take a look at what we've done okay so we've now gone and applied a slightly different color set so basically i've just gone and changed the colors for each one of these components so again they're just different numbers but you can play around with this so ideally what we should see now is that we no longer have that ugly green and red we've now got these a slightly different colorization so i'm just going to release a capture as per usual because i can see on my webcam that it's still on then what we can go ahead and do is we can actually go and see the results of changing these colors now again what i'm showing right now is really just a simple example of what's possible with this let me know if there's anything specific or if there's any other use cases that you'd like to see me use this to extend out so i'm thinking some of the cool ones might be exercise counting so if we did like a push-up or a squat counter or a body language decoder that might be cool controlling the screen or controlling like a pie game interaction or interface might be awesome as well all right let's go ahead and run this and then see our final result so again we've just changed our colors here so we should be able to see a change model and you can see that we've now got all of our different landmarks so we've got our face landmarks we've got our pose detection landmarks and key thing to note is order as well so our pose land or our pose model is the last one in the list so you can see that's why our pose model is drawing over the top of it again can do this can see all of our different components so we're now tracking all of those i don't think the big circles are all that useful on the face model let's change that let's make that one actually i think that looks a bit better base model looks like it's running rampant let's run that again and you can see just by changing these numbers we've now gone and changed how many points and what the actual visualization looks like so you can see the really small dots on my face right so rather than having really large dots you can actually now see a much more accurate much clearer detection of all of those key points but again you could change all of these you could change what it looks like let's do the full body again you're seeing all of the joints so my knees my ankles if i bring my foot up a little bit further you can see that it's doing your foot detection as well it's doing hand detection pretty cool right so that gives you a baseline idea of just what's possible with media pipelistic and uh on that note that about wraps it up so what we've done now is if we go all the way back so we first up went about installing and importing our dependencies so for that we used a media pipe and opencv we then set up our drawing utility so mp underscore drawing and mp underscore holistic and then we built this up so we built our real-time webcam feed we then made our detections from our feed so remember we've got four different models inside of media pipelistic we've got our face landmarks our right hand component our left hand component and our pose detection component and then we can also apply our styling using this drawing spec over here so remember when we use this draw landmarks function which effectively allows us and we've got our code still running when we use this draw landmarks function you first need to pass through your image your landmark list all of your different connections and then you can pass through your different styling so your landmark drawing spec and your connection drawing spec and then if you go on ahead and run our final code which again will be available inside of the description below via github you're able to get a full blown a model that's able to detect all of those key points now again this is just the beginnings of this you could do a lot more stuff and if you'd like to see me do more tutorials with this by all means do let me know but on that note that about wraps it up thanks so much for tuning in guys hopefully you enjoyed this video if you did be sure to give it a thumbs up hit subscribe and tick that bell so you get notified when i release future videos and let me know again if you'd be interested in the full body language decoder series that will be using media pipe thanks again for tuning in peace

Channel: Nicholas Renotte

Views: 126,052

Keywords: python mediapipe tutorial, mediapipe tutorial, face detection python, pose detection python, hand pose detection, face detection, pose detection, facial landmark detection, facial landmark recognition, pose estimation

Id: pG4sUNDOZFg

Length: 42min 54sec (2574 seconds)

Published: Sun Mar 28 2021