Human Pose Estimation using opencv | python | OpenPose | stepwise implementation for beginners

hello everyone in this video we will be implementing human pose estimation using python as a programming language and for overlaying these lines and points correspondingly we will be using opencv in addition we will use opencv to load already pre-trained deep learning architecture based on tensorflow so actually we have already developed a tensorflow model to train these women pose estimation and in this video we are targeting beginners level so we will not train the human pose estimation rather we can just load the frozen models using opencv at the end of this video we will show you how we can deploy our algorithm uh on pre-stored videos like this is it's a youtube video so in this case you can see that the algorithm successfully estimate the poses right similarly at the end of this video we will also show you that how we can deploy our algorithm using a webcam demo in this case you can see that as long as we have the full view of a body the human pose estimation algorithm can detect all possible 18 points of the body skeleton similarly if we do not have for example occluded position such as some of the points are missing over here so still human pose estimation manages to detect all possible points in the view similarly in in this case we do not have legs and we have very less points body parts and points so in this case still open cv or the with the help of tensorflow deep learning architecture is still it manages to detect all possible points let's get started my name is shanollah and i'm a phd student at in high university south korea my major is computer vision and deep learning let's get started okay let's dive into the implementation of human pose estimation using opencv and the tools we will be requiring in this video is anaconda so anaconda is a great uh package uh which contains many ids like vs code spyder and others but the most preferable uh for this video is jupiter notebook so the beauty of the jupiter notebook is that you can make your notes very easily you will find in the proceeding video if you continue to stay tuned uh similarly the only library we will be using is opencv so in order to download anaconda all you need is to just write in the google search bar and account on download and the first link you will find just click the individual edition anaconda and you will be here and at the down most side of this link you will find the executables to download right so based on your specifications uh like your operating system either 62-bit or 32-bit you can just download right so i assume that it's super simple all you need is to just download the executable and just press next next next and it's super simple to install right i assume that you have already installed the anaconda so once if i write in the search bar of the windows anaconda if i click this you will reach over here this kind of window will be up here right so over here you can observe that some of them are for example blue and some of them are green so the green one are those which i haven't installed in my case i have already installed jupiter notebook so in your case actually all of them will be green right so it's better that you just press the install button and jupyter notebook will be installed now in at this moment i assume that you have already installed the anaconda navigator and jupyter notebook it's always better that you must have a window a folder for example sorry you must have a folder a separate folder to make a project so in my case this is my folder and i assume that in your case it will be empty so let's assume that this is empty and now if i again press anaconda prompt okay so let me zoom in for you guys so just sorry okay like this and if you write jupiter note book okay so i could have opened from using like a launch button over here but it's better that if you once you write over here it will be open jupiter notebook so this kind of window will be appear in your internet explorer right so once it is here we are here just write new python notebook so let me separate this so now i assume that you have just this right so uh you have installed anaconda and jupyter notebook now the most important thing is we need to import cv 2 as cv but in your case it will not be installed so if it if it is not installed then write pip install open open cv dash python right so why is by opencv dash python because opencv is also available for c programming language right so it's better that you just write uh just execute this command so in this case in this case just go to anaconda prompt but this time rather than just clicking it it's better that you just run as administrator okay so it will be here and all you need is to just copy and just paste the command over here once you enter everything will be installed like opencv will be installed in my case it's already installed and one another important thing is for example you need to install pip install not plot clip right so these two important things you need to perform so in my case it's already been installed so i don't need it so import second thing i need is import mat plot lib dot py plot as plt okay so these are the two commands i need to perform okay so after i do it so let me show you some important thing about the human pose estimation if you go to this link i will provide this link in my description so actually this is by uh king hawa 92 like this is the author name and who this is a github link i will provide so actually it is uh inspired by the opencv actually it's opencv example right open post is the project name it's a paper name actually the very first paper on the human pose estimation so pretty much stable these days so this was the example right so this guy what he did is that based on that he provide some easy steps and already tensorflow this is the graph of the tensorflow already been trained so using opencv we can just load the graph of the open uh this tensorflow so all you need is to just press this download the zip so in my case actually i have already downloaded this and so and extracted it so once you extract it the most important thing is this graph right so you just need to copy this graph and paste in the folder where you're working right so i assume that you have already pasted like in my case i have pasted like this okay so i assume that you have already copied and pasted this graph so if i say net for example land and cv dot dnn if i press tab you can just write it over dna then import as this is the the model is from tensorflow okay so all i need is to uh write read and when i press a tab read from tensorflow okay and what i need i need to do is just copy the name of this all right and i what i need to do is this okay so in this case i can just easily uploaded my net right so in now the tensorflow model is already been loaded all the weights right so these are our weights actually right so these are our weights and once we do have our weights so the width uh actually as i showed you just the opencv actual example so all the settings are available over here right so this is actually during the training of the tensorflow all these height and width were set right and all these thresholds and similarly you can see over here the body parts and other things as well so this is again a very important like a body parts and pairs let me discuss again about it so if i just press shift and enter so for in order to execute each line shift enter is the command now i can have a list of the like a body parts and pose pairs in this case you can see that nose is 0 and till 18 we have 18 body parts and the body pairs are like neck and shoulder right shoulder neck and left shoulder okay so for you guys i have again i recommend you that you just visit nano net auto made data capture this is a tutorial uh by this this guy so it's a very uh famous and very good tutorial uh they have written so actually over here this is a skeleton model of the body right so it's starting from nose its nose is zero right in our case so you see over here nose is zero and the neck and right shoulder and then elbow and all 18 body parts we have and these are the pairs right so neck is connected actually so the neck is connected with with what right with the shoulder right shoulder and left left shoulder as well so all of this skeleton how this skeleton is made so one another thing these are there are many applications for this like for human uh action recognition and many of them there are ample of uh applications for this right so this was the initial design all these body parts and joints we had and there was a paper so if you visit this link there are so many like depots and other papers available this is cvpr very famous paper a cvpr deep pose okay because they were the first to uh perform everything using uh the deep learning so prior to them there everything was performed using the computer vision and other techniques but after them they were actually it was inspired by the alexnet backhand again this is image classifier so they use some classifiers uh in order to point out the points right of the of the of the body right then so you've had 2015 there are another localization method right so actually pose estimation is nothing but localization of the body parts right if i say in the one sentence it's the localization of the body parts right so they were it's a good survey like in silver 2016 there so and still there's a lot of papers going on like using heat maps okay and many other revolutionizing day by day right many other deep learning algorithms are coming up and very efficient algorithms are coming up right so uh thanks to opencv that we can easily use the tensorflow model so it's already been so in your case you can just run the uh the deep learning if you're good at deep learning and you can train your deep learning model in tensorflow the good thing about opencv that you can just load the weights right so once i let me execute this and shift enter i press okay so for in my case i have downloaded one of the image from internet so this the name is pose and for example if i check it's png right so in the format is png so let me for example mmg is equal to cv dot i am read is the command and for example pose dot it was png right so let me load this image and if i say plt dot i am show img so you see over here but this is bgr actually so by default the opencv format is bgr blue green and and red right so in order to make it correct like uh to show it correctly it's we need to convert it to rgb so this is my rgb okay image format and okay so let me write a function like pose st mation okay so if for example this is image or frame for example in my case it's frame okay so once i have frame so it should be dynamic the code should be dynamic must be dynamic right so frame shape we can read like width and height we can read right um then for example we can have wait in one wait a minute okay once i do have it okay so this is net so i am now configuring my uh this net right what i have i have uh loaded in my memory i'm just doing the configuration so what are the configuration actually if you just write nat uh like net dot if use s press s this is set input right so it's automatic if you have installed the open cv you will see like this and then cv dot dnn dot blob from image actually we are just grab grabbing the image right so this is the scaling and this is in width and in height okay this is this was the setting actually during the training it was performed and what is it it's a mean right so for example uh 255 divided by 2 is 127.5 right so this is 127.5 so we are averaging all of them and then this is swap rb so in during the um prediction we don't need to swap it again and again right so if we read in the bgr uh it will perform like if we make it true it will perform automatically okay these were some settings okay so okay so these were some settings and for example rather than uh waiting uh i think i should copy all of the code from my nodes and you can just i will share the github links and you can just uh download the code from there uh the the main purpose of this video is to learn right so we are learning these things so length of the body parts uh formula in our case it is 18 right and then we we need to find the points right we need to find the points of this like nose what are the body parts like nose neck or the right shoulder all of the maximum points we need to find it out right so this is the point this is a list right so this is our list and then in the length of the body parts like we you have 18 body parts right we are generating heat map okay and then uh we need to find the local maxima maxima actually right so the discretion is also available and then once we find the points right and x y points and then we need to append in the points so whatever point we find for example we find nose we need to append this point and we know any point we found like nose shoulders all of the points we need to find out right once we find all of the possible points for example uh in the introductory part i told you that first of all if the skeleton is not available so maximum points i can find for my nose my shoulders and other elbows and other points once i find i need to append all of them one by one right once i append all of them now i need to pair them all right i need to join them like first of all the nose is fine and nose to neck and shoulders and all of these things we need to join so this is the joining point right all of them is the joining once we join we need to draw like like for small for line we need to draw for example this is bgr you will see shortly that this is green and this is again bgr this is the red so the points are red in color make sure we will show you um soon and the the lines are in green color right so if i execute this function so it's actually executed but not called yet so let me call this okay so let me call this function for example this is my function so what i need to do is st mated uh image right for example i need to call this function in my case the name is image right i hope so let me check yes this is img and let me call this function now i have executed so if i execute this for example let me copy this okay so this is estimated image um once i estimate it okay now you can see over here as i told you before that all of them are either green like the lines are green right so all of the lines are green and the points are red you can change it for example if you do it 0 0 0 it will be black right if you do it two five five two five five two five it will be white so bgr so whatever you do it one which is two five five so it will be the color right bgr this is in this case it's red and this is is it's red right so i hope uh till this point you are already read over here so now it's time to perform uh this uh demo for video right on video okay so for example if you have already stored some video you were shooting your video a few months ago or many time ago and you want to know what is the estimation of my video so it's better to have it like this so let me uh okay so in my case for example i have already this cv dot video capture is the command both for the webcam and also for the uh image as for the video as well okay so uh once it it will start reading the video and we do have some settings like for example it's optional in your case for example let me change the size so this is my index of the setting and this is my size of the height for example i want to resize it if you don't want to resize it's okay it's up to you it's just optional one right in this case it's four so height and width is now 800 by 800 you can just change it in your case okay so for example let me add some for in my case for example if the video is not open and it is corrupted in uh having any reason and okay so it should show that cannot open the video okay okay so um as the pose estimation takes time so for example by waiting for one second uh we will keep going for infinite loop okay so once for example this cap we need to read this cap right so if it we need to read this video frame by frame because the video is nothing uh usually it is 25 frame per second or up to 50 or 60 for hd videos it is 60 videos per second so video is nothing but a combination of images right so i assume that it it will read image by image the whole video and what if the video is corrupt inside right so we have to find so okay we should go for the next video either we should break or we should go for the next uh sorry frame right once we do have it i can copy like all of my this function like from here okay you can just copy but make sure you have to remove the okay just copy and just return you should remove the return right because it's not function anymore now i need to copy all of them and you can see everything is almost same right so once i have everything same and one important thing i can do it over here is that for example now i let me name a cv dot i am sure okay let me say that it's a video tutorial how to suppose estimation tutorial by the way pose estimation t2 real okay okay and what i need to print over here is frame okay okay if i do it like this let me execute shift enter or you can see you can see over here it's already stored video actually so let me show you this video this is the this one 93 and down so let me show you the video 93 and then you see so this was the video okay this was the video and now it's actually performing the live estimation right so we can easily perform the live estimation right so it's now turn rather than applying our deep learning pose estimation algorithm on pre stored video it's time to apply it on a real-time video demo so in order to deploy this on a real-time webcam demo so let me copy all of the code we just perform the demo on the video already pre-stored video so this time we don't need any video at all if you want if you have a webcam uh okay you can just perform a real-time webcam demo so what are the changes we need to do okay so the one most important thing you need to do is just write one over here i hope that you will not have for example this is webcam okay so i hope you have a webcam so either it will be registered as index zero or one so this is the dynamic code for example if you if it couldn't find the webcam it will throw an error and one another important thing is that cap dot set is equal to cv dot uh is the cap okay if you uh write a tab uh okay i like this cap okay so this is a cap underscore pro again this is not important about this is uh when you press fps okay this is the one so this is again not really important but it's better that mostly the webcam in your case the webcam will have different uh frame per second so it when if it is very fast then definitely it will be unable to perform right so we will have a lot of frame per second so anyhow if you these are the option points right so it depends it varies webcam to webcam right so all you need is to just either write or it's just optional you can just delete it right so these were the changes only uh we started from the very basic uh like pose estimation function then we deploy the function inside our demo video on pre-stored video okay and then now it's time to perform on a real-time webcam so for web game these are the only important changes we just did now let me execute all this okay so okay once i execute uh it's i'm here so it's better to have a whole skeleton of your body right so let me make it the camera in a better place so you see over here all of the points they're just finding all of the possible points right for me so it's better to have the whole skeleton i hope in your case uh your video will be for example you can shoot it at this different level or at a distant up distance apart so it's better to have it so okay so this was our live video demo webcam demo so you can just perform everything uh very easily right so thank you so much for watching the video

Channel: DeepLearning_by_PhDScholar

Views: 95,878

Keywords: Human pose estimation, python, opencv, openpose, tutorial, beginners, deep learning

Id: 9jQGsUidKHs

Length: 25min 44sec (1544 seconds)

Published: Wed Oct 14 2020