AI Emotion Detector Using Mediapipe Facemesh Technology

developers have three modes happy that's if  the code run successfully and everything goes   as expected the second mode is neutral that if  the output is not what they expect or what they   want and finally sad when there is an error  on a line where there is no code at all today   I'm going to be teaching you how to build  your very own mood classifier using computer   vision and machine learning at the end I'll be  sharing my code with you so that you follow the   same guideline and VI your own emotion or mood  classifier ready to do it let's get right into [Music] it all right so before we start the  code including every file will be using in this   particular tutorial will be on my giop repository  and the link will be in the description you can   leverage this code and update it work on it and  also use it as you want so I'm not going to be   coding from scratch I'll just take you through  all the codes so we'll be using three files and   I'll take you through how everything works I've  have done everything so that you can take it on   from there because uh coding all this will take  a lot of time which will make uh the tutorial   boring in my opinion okay so we'll be working  with three files before I even introduce this   files let's go through uh this particular image  so we are going to be using the face meesh data   so face merge Google face merge which is done by  the media pipe team we be using a raer library   cive Zone which will help us to get this fish mesh  data in a few line of code so this contain about   468 landmarks on the human face and uh based on  the action performing by the human pH we collect   this data as to sad or happy and then we'll put  this in a CSV file where we read and perform some   data cleaning and also train a model out of this  uh data so let me show you how the CSV data will   look like before we go into the coding so in  the CSV data what you are going to create is   that because you have 468 landmarks we have the  first uh columns here or our headers here to be   class and this particular rows to take in the  classes we are going to be taking the data for   then this will be X1 y1 all the way to 468 so  you can see up to here 468 because you have 468   landmarks on the human face so that's what we are  going to be doing and for the classes column you   can see we have class for Happy and when I come  down right you can see we have class for sad and   you are not limited by the number of classes you  can actually take many classes as you want you   can do happy um neutral anything at all you can  get from F expression you can add that class to   it the possibilities are endless because of the  flexibility of the code so I close this app and   now uh I have three python script datag gen. Pi  training. Pi and test.py so the data gen dop is   responsible for generating the data for us so for  that we are importing uh these libraries and all   the libraries you need to install are here which  will also be uh on the gab repository you need   opcv python media p Panda CV Zone and nonp then  we use cyc l to also train our model we use some   of their STS to train our model okay so we are  importing CV Zone in order to use the face MH   detector and uh for that I'll take you through  how to get a face M detector or I will even take   the code line after one line after the other so  this is the path to our video we have two video   F here happy and sad and uh happy right here just  contains s of people being happy or people living   in their happy moment so you can see the facial  expression this lady is very happy this man is   happy and so on so I just took some couple of  videos and compile them together the same thing   applies to the S video just some video of people  being in their sad moments which is weird though   but um we have to do it so now that you are  familiar with all the file let's start off by   looking at the data gen. by close this off and  um we imported our libraries once again then we   gave the path to the video file so if you using  your camera you make sure you put uh the index of   your camera here that's one Z and so on so if  it's zero that means it will take your webcam   associated with your computer but if you have a  USB cam you want to use then you should start from   one two until you get your camera to work with so  since I'm using the video side here all I have to   do is to specify the name of the video and that's  video PA because that's video path right here from   there create an object for the face detector so  face detector object so fmd which I'll be using   which stands for face M detector is an instance  here of the class face M detector and over here   you can specify some couple of things such as the  mass faces so the maximum number of faces you want   to uh detect so if you put one here that means  U it only take one person's face when multiple   people are looking into your camera or your webcam  so you can specify or you will leave it so that it   picks everybody in that particular frame then the  next thing is to Define your class name so since   we are doing it for a sad. MP4 then the class name  should be sad so that in our data. CSV it will be   stored there as sad all right let me uncomment  these lines so what we are doing with this Lin   to create our columns in the uh data. CSV or in  our data file so with this first we specify our   class so we create a particular column in our data  call class if you can see that right here at the   top let me scroll at the top here we have class  so the First Column should be class Then followed   by this X1 X2 which are the data points so that's  what we are doing we create our first one which is   class then for the X1 X2 also we look through them  and create them using the noral python formatting   so um we have 468 landm marks then plus our class  name or the class column making one so we add one   to it which is making it 469 in total so that's  what we are doing creating the first header names   right here then after that when we run the code we  want to put that header name in The First Column   of our data set so we do it once that's why it's  not in the loop we do it once here so what we are   doing that first time we run the code this header  names right here this first line or first uh row   of names we put it once because we only need it  once or we need uh down here will be the data and   they corresponding class l so that's what we are  doing it here so that means when you run this code   once it will create this header for you so after  that you need to comment it off because you will   not be creating header anytime you run the code so  once You' run this ones on your side video or any   video at all you need to comment this one off okay  so after this we are doing while cap do is open   to get our video feed from the camera so we are  just getting getting it since it's a video file   we want to get it in the loop we read it we resize  it and then now feed it into our face M detector   so we are calling it frame so we are feeding this  Frame into face Med detor and this will return as   our faces if any faces has been detected and  there is a mesh on it to return return us and   return return us our image to which is the same  as our frame okay so right from here we check if   we are getting some faces if you are getting some  faces we take the faces and then we take the data   so so this face mer data will come in a list so  we are converting it to a list and also we are   making it a onedimensional data or vector that's  why we are using flatten here which is the same   as reshape into one long giant Vector array then  we are inserting the class name into it so we are   taking the phase data here and we are inserting  the class name at the first index that's why in   the CSV file at the first index here you can see  the class so we are inserting the class name here   then what follows will be our data so we take our  phase data and then we insert the class name into   it so that we can um select this class name  if you want to preprocess this data and train   a model out of it so that's what we are doing  and after that um we are writing this to a CSV   file and the format here pay attention we are  appending it that's the mode the we using is we   are appending so that when we take data for one  class and we are coming to take data for another   it will not overwrite that you append that I mean  it will go down we'll be adding to it so we are   just writing to a CSV file called data. CSV and  our delimer we are using comma so comma separated   values after that we are showing our frame so that  we can see it and collect our data so that's all   this data generation script is doing if you have  any to do with it you can uh Advance it to do your   particular tax in hand so let's run this and take  data for the S class so before you run make sure   the class name is sad and the video you are going  to do is also pertaining to sad so right here you   can see we have our video feed we are collecting  data for the S class and you can see we have the   data. CSV file being generated right here so this  will take some time to generate data for the S Class so just like this you are done taking data for the  S class and that's how we take data so the next   time you run this script for another class you  need to comment this to this line of code so that   you don't overwrite the column names again because  we don't need that again so if I want to take data   for happy I can just now change here to happy  here to happy so let me do that so happy. MP4 if   I could spell that correctly and then here should  also be happy so happy. MP4 that's the name of the   video right here and the class name is happy so  that's all that's all I'm doing I'll run this once   again and it will start loading data for the happy  class so this is like a whole data collection uh process and just like that we are done taking  uh dat or collecting data for the happy class   in case I want to take data of myself or from  my webcam all I'll do is to replace it with   one so I'm going to collect data of myself being  happy and let's see how it goes so guys this is   me right here being happy so just as I'm talking  right here I'm collecting data of myself being   happy you can see it right so I'll do some for  sub to as well add data of myself you can miss   the data up it doesn't matter we just use use  the class name to select them from our CSV file   so that it right here I'm collecting more data  and I think it's enough I will do that for sad   to as well so to collect for sad I just have  to change the class name to sad as well and   collect data of myself so you can add your  own data to it if you want or you can even   use your own data from scratch that means  you are not going to touch the video path   variable here all you will do is to change  then uh the names here and then take uh data   for that specific class so you can add your own  class whether you are smiling you are angry you   are do this or that so now let's do it for  sad let me run it so this this is me being sad so um I think it's enough I think it doesn't  look good so um now that it so that's what this   script is for for generating data and we've  generated the data and here is our data being   saved here so we have ton of data we have many  rows so it's about 4,000 plus so I think that's   enough so the next script we'll be working with  will be our training. PI so this is it so over   here we've imported pandas and pandas is for  data manipulation cleaning your data structure   your data rename columns check info about data  and so on so we use pandas to uh structure our   data to clean our data and then use skn which we  are importing so many stuff from here to train   our model split our model into train and test set  and train our model then finally we use the pickle   model from python is inbu to uh save our model so  that we can use it later on okay so over here we   are reading our data data. CSV so pd. read CSV and  we are assigning the data to DF which is uh data   frame then after reading this data there are some  uh stuffs we can perform on this data data do head   this will give you the first five rules of your  data data do columns to also print all the columns   when I run the code you see all of these STS data.  info is going to give you some information about   your data your memory usage and U some additional  information then if you want to perform some   statistical analysis on your data we use the DF  do describe and this is going to give you uh the   main standard deviation the first quarter second  quarter okay since we have a lot of data points we   are using DF drop now here which will uh clean  off all other columns that we have no data so   if there are uh columns which have null values  we are going to clean them off and use um The   Columns that we only have data for then the next  thing is to go ahead and separate this data into   our features and our labels so the labels are our  targets we want to use these features to predict   our targets so um our labels here is our class  names and in this case we have only two class   name happy and sad so that's what we are selecting  here we are taking the data frame in the index of   this particular class and for our features we are  just dropping the class so whatever is left will   be our features when you drop the class then from  X1 up to 4 all the 46 eight landmarks will be our   features which we'll be using to train our model  over we just printing them to show the next thing   I do on this line is to split this data into train  and testing partition so we have our X train y   train X test and Y test and we are using the escal  TR test split function to do that okay so after   splitting this data we create a model pipeline so  we are using standard scaler to normalize our data   and also we are choing using the model to logistic  regression so uh with this particular pipeline   it's very easy to switch between different models  all you need is to type the name of the model here   so let's say you need a random Forest classifier  you just you can just type the name right there to   change this particular model so you can test other  models to see their performance on your data and   then you use the model that best fits this data  for you then uh we use the pipeline fit in order   to train our model then after training the model  we are running test on our test data so model.   predict then uh the next thing is to evaluate this  model to on our test data so we are evaluating it   and we are getting the classification report this  classification report will give you information   such as the F1 score you also get a precision  and the recall you'll get the accuracy and so   much more so uh using the classification report  from skn is very helpful then after that um we've   done training the model so we have to save this  model and that's what we are doing here we are   using PCO to save the model so p. Dum and then  we are writing it so write binary and then we are   damping it the name will be model. pkl so after  running this code we expect our model name model.   PK should be uh save in our directory we are  working in so I'll just go ahead and run this code we just this few steps you can see we now have  our model. pkl here that means this our saved   model which we can use to run inference so let's  go through some of the things it has printed on   the terminal do head which gives us the first five  rows and then also uh we have the info so you can   see over here we have some informations about data  set the memory consumption of this particular data   set and some other informations for the columns  we have all the column names here so the terminal   cannot display all that's why you have the dot  dot here but it gives you from starting to the   end okay so here is where we are doing the data  do describe you see you have your mean for each   and every column so you can see X1 you have your  mean you have your standard deviation after which   we also uh printed the features and the labels  as well so we have the labels features then over   here we are testing our model on a test set so  these are the predictions from the model from   on the test set then we come here this is our  classification report so you can see we have   the Precision recall F1 score and the accuracy  for this particular model is 100% wow that means   our model is very accurate on um our classes so  you can see the happy for the happy class both   Precision recall and F1 score 100 100 set the same  thing that means the distinction between our model   on our classes is very great so let's see whether  this model can generalize in real life where we   are going to run it on our webcam to take our  webcam feeld in order to classify whether we are   happy or we are sad so for that I have the test.  pi here so with the test. we are just loading a   feed from our camera just like the data generator  so field from our camera then over here we are   loading our model then we process this field just  as we did in the data generator we also flatten it   to be a long list then we are feeding that phe  data into our model to give us results which   will display on the screen so that's all we are  doing for the test set I think is s explanatory   when you look through you can see similar to  the data generator but instead of saving the   data pH data we are sending it into our model  for prediction so I'll run it and this will run   and open my webcam so I I set my webcam here one  I have a USB webcam attached so I'm using one if   you using your actual computer webcam then this  should be zero so let me run this and let's see   how this model performs in real world so guys  here am I you can see I'm in a happy moood so   it's the tting that I'm happy um let's try start  so I'm sad I'm happy and you can see so I think   this works this classifier works you guys can go  ahead and add additional class to it to see how   it perform and also I'll be sharing the code and  every other file you see here in this particular   tutorial or my GitHub repository so just check  the description of this video and you can have   access to it guys this is all for today you can  either be happy but don't be sad be happy and   make things work out thanks for watching and  um if today is your first time make sure you   subscribe you share with your friends and up  until then I will see you in the next tutoral

Channel: Tech Watt

Views: 672

Keywords: yolov8, image classfication, custom dataset, ai, machine learning, computer vision, deep learning, pytorch, openvino, model optimization, yolov8 to openvino, #emotion detection, #facemesh detection

Id: x1c8Ja7VTXk

Length: 21min 7sec (1267 seconds)

Published: Mon Mar 04 2024