hey my name is Felipe and welcome to this image
classification course. This is a fully comprehensive course and I'm going to teach you everything
you need to know to get started with image classification which is one of the most important
techniques in computer vision I'm going to cover the fundamentals and I'm also going to show you
three different ways to train your own image classifier: the python Library scikit learn, the most
recent version of YOLO: YOLOV8, and a 100 no code online platform called teachable machine. I'm going
to show you the step-by-step process of how to use an image classifier in super complex projects
we will be building a computer vision API, a pneumonia classification web application and a face
attendance system with liveness detection. By the end of this course you will be more familiar with
one of the most important techniques in computer vision, you will have the skill and knowledge
to train your own image classifier and you will be able to build super amazing computer
vision projects. And now let's get started. So let's get started with this course about
image classification and the first thing we will be discussing is what is image classification I'm
going to share with you a very simple definition and a few examples, image classification is a
computer vision technique to categorize an image into one of several predefined categories
and is very important we will be categorizing an image into one of several pre-defined
categories there are many Technologies to perform image classification these are only
a few of all the available Technologies of all the different ways in which we could do image
classification for example we could use the python Library scikit learn which is a very popular
library for machine learning we could also use yolo V8 which is the most recent version of
YOLO we could use teachable machine which is a very powerful framework provided by Google to
do image classification with absolutely no code whatsoever, its a 100% no code platform we could also
use image classification through a cloud provider for example using AWS rekognition and these are
only a few of all the different ways in which we could do image classification and this is the high
level structure of an image classifier, absolutely every single image classifier needs to comply
with this structure the input will be an image and the output will be the image category and
the category will be given either as an integer value for example 0 1 2 and so on or as a string
for example cat or dog or as a vector but let me give you a very specific example so here we have a
very very simple image classifier and in this case we will be classifying absolutely every single
image into one of these two predefined categories this image classifier is going to take an image
and is going to classify this image it's going to categorize this image either as a cat or as a
dog and in this case this image classifier takes this image of a cat and it Returns the category
cat right this is a very specific example and now let's continue now let's talk about image
classification confidence score, image classifiers use actually provide a measure of confidence
along with their classification, they usually provide a measure of confidence not always this is
something they usually do right remember an image classifier will always return the image category
but they usually also return the confidence score usually not always this confidence score is a
float value in zero one so it ranges from 0 to 1 and it represents how confident the image
classifier is regarding that classification so in the previous example in this case we
are classifying this image as a cat with a 0.95 confidence score we can look at this number
either as a float value and this will be 0.95 or we can look at it as a percentage and in this case
it will be 95% so this means this image classifier is telling us this image is a cat with
a 95% confidence so this means this image classifier is not 100% confident this image is a
cat right this image could be a cat or it could be a dog the image classifier could be making a
mistake is not 100% confident but it's a 95 percent confident right that's exactly what it means the
confidence score we are making a classification and we are providing a value of how confident
we are regarding that classification right now let's continue these are some very specific
examples some very specific applications of image classifiers and these are some examples we
covered in my previous tutorials in this YouTube channel and these are some examples we will
be covering through this course so the first one is a parking spot classifier and this is a
classifier which takes an image of a parking spot and it classifies this image into one of these
two categories so this classifier is going to take an image of a parking spot and it's going to
classify this image as empty if the parking spot is empty or it's going to classify this image as
non-empty if the parking spot is occupied either with a car or any other type of object this is a
very interesting application we will be covering later on this course then another example is
a weather classifier which is going to take an image and is going to classify an image into one
of these four predefined categories which are rain Sunrise cloudy or shine and if this is another
example we will be covering later on then another example is a ship classifier which is going
to take an aerial image a satellite image and is going to classify this image as ship if the image
contains a ship or as no ship if the image does not contain a ship and then the last example and
this is a very very Advanced application of an image classifier which is a spoofing classifier
and the idea with this type of classifier is to detect this type of situations of someone
holding someone else's picture right in this type of application we will be classifying human
faces into one of these two categories we will be classifying every single face either as fake
if the person is doing something like this if it's holding someone else's picture or as real if
it's someone holding its real face right if it's a real face in front of the camera so these
are a few examples of some image classifiers which we will be covering through this course
and now let's continue to the next lesson. Now let's talk about image classification metrics
let's try to answer the question how to measure the performance of an image classifier we will
be discussing three different type of metrics: a confusion Matrix, accuracy and precision and
recall. Precision and recall are two different type of metrics but we will be addressing both of
them at once so let's start with the confusion Matrix It summarizes the performance across
all of our classes it's a very visual way to evaluate a classifier the ideal confusion Matrix
is a diagonal now let me show you this is how a confusion Matrix looks like in the case of only
two classes we will have four numbers which will be arranged as a matrix and the way we will
make sense of these four numbers is exactly like this the number we will have over here
the true positives is how many samples of this class the class number one were classified as the
class number one then this other number the false negatives means how many samples of the class
number one were classified as the class number two then the false positives means how many samples of
this class the class number two were classified as the class number one and the true negatives means
how many elements of the class number two were classified as the class number two this
is going to be much more clear later on because I'm going to show you a couple of examples of how
to compute a confusion Matrix now let's continue let's move on to the accuracy the interpretability
of this metric is very straightforward if classes are perfectly balanced and the higher is better
and this is exactly how we are going to compute the accuracy basically we will need to take the
elements in the diagonal the true positives and the true negatives and we will need to divide
these numbers by the total number of samples right TP means true positives and TN means true
negatives so true positives plus true negatives divided the total number of samples now let's
move on to the Precision and recall and these are the most appropriate metrics if classes are
not perfectly balanced or if misclassifications are too expensive and I'm going to show you an
example in a few minutes of an image classifier where the misclassifications are too expensive
so this is going to be much more clear in a few minutes but for now let's continue recall
measures how effectively we can find objects Precision measures how well we perform once we
find an object and the higher is better for each one of these metrics now let's continue we need
to define a positive class and a negative class precision and recall are computed with respect to
the positive class so going back to the confusion Matrix you can see we have two classes we need to
Define one of our classes as the positive class and the other class as the negative class right
and in the way we have built this confusion Matrix we have over here the class number one will be
the positive class and the class number two will be the negative class going back here once we
have defined these two classes the positive and the negative we can compute the precision as the
number of true positives divided by the number of true positives plus the number of false positives
going back here we will need to take this number the true positives and we will need to divide this
number by the sum of the true positives plus the false positives right we will take this number
and we will divide it by the sum of these two going back here then we will need to compute
recall as the number of true positives divided by the number of true positives plus the number
of false negatives going back to the confusion Matrix it will be like taking this number the true
positives and dividing this number by the sum of these two true positives and false negatives
so we take this number and we divide it by these two by the sum of these two now let's continue
let's take a look at a very specific example let's assume we are working with a cat dog image
classifier so we have an image classifier that classifies every single image either as cat or
dog and now we are going to test this classifier and we have 500 images of dogs and 500 images of
cats and let's assume that 420 images of dogs were classified correctly and 400 images of cats were
classified correctly if we build the confusion Matrix in this problem we will have something like
this right 420 dogs were classified correctly and this is exactly what we have over here and 400
cats were classified correctly and this means that 100 cats were misclassified as dogs and 80
dogs were misclassified as cats in this case the accuracy will involve to sum the elements in the
diagonal in this case is 420 plus 400 divided by the total number of samples which is 1000 and this
is 0.82 then Precision we will need to take this number 420 and then divide this number by the sum
of 420 plus 100 and this is something like 0.81 right then recall we will need to take this number
420 and we will need to divide this number by the sum of 420 plus 80. this is exactly what we have
over here and this is 0.84 so in the case of this very specific example we have these values for the
accuracy the Precision and Recall now let's move to a slightly more advanced problem to a slightly
more complex image classifier which is something like this, this is a spoofing classifier which is
going to classify every single image into either fake or real and these are images of human human
faces right so fake means if the user is doing something like this and real means if it's an user
with its face looking at the camera and without doing anything like this right a real face and
we are going to test this classifier and we have 500 images of real faces and 500 fakes and we get
these values 280 fakes were classified correctly and 480 real faces were classified correctly
if we plot the confusion Matrix we will have something like this 280 fakes were classified as
fakes and 480 real images were classified as real let's continue the accuracy we will need to sum
all the elements in the diagonal 280 plus 480 divided by 1000 and this is 0.76 then for the
precision we will need to take this number and we will need to divide this number by the sum
of these two which is 280 divided 280 plus 20 and it's 0.93 then for the record we will need
to take this number and we will need to divide this number by the sum of these two and this
will be 0.56 so for this image classifier we will have these values for these three metrics
for the accuracy precision and recall and the reason I consider this is a slightly more complex
and a slightly more advanced problem is because if we consider this case if we consider this
image classifier if we assume this is working in something like a face attendance system let's
consider these two situations let's consider we have a real face which is missclassified as fake
so someone is trying to log in into a system and this system this face attendance system tells the
person it's fake so that's the first situation we will need to consider but if you think about
this situation it's not really a big deal right because if a person is classified as fake the
only thing this person will need to do is to try to log in once and again until it's classified
as real and that's it and eventually the only thing this person will need to do is to call its
supervisor or call support or something like that to fix this situation to fix the issue it's
not really a big deal but let's consider the other situation let's consider someone who is holding
someone else's picture someone who is pretending to be someone else and this someone is classified
as real right this someone is not classified as fake as it should be but it's classified as
real this is a huge issue because in this case it means someone has gained unauthorized access
into the system and that's huge that's a huge issue so this is an example in which we have a
misclassification which is too expensive right and in this type of situation the accuracy is
not really that important the Precision is not really that important but the recall is the most
important metric of all because we want to detect this situation as often as possible ideally
we would like to detect absolutely all of these situations and you can see in this case the
recall is 0.56 which means this is a 56% which means 44% of all the people who are trying
to impersonate someone else are getting away with it that's what it means 44% which is a lot that is a
lot of people which means we have a huge problem so this is only to give you an example of a
slightly more complex case of an image classifier of the evaluation of an image classifier and
this is going to be all for this lesson now let's continue to the next lessons where I'm going to
show you how to build your own image classifier. hey my name is Felipe and welcome to my
channel in this video we are going to make an image classifier. Image classification involves
classifying an image into different categories and is one of the most important fields in computer
vision I'm going to show you how to make an image classifier which is very simple, very robust and
it works 100% in Python, I'm going to walk you step by step through the entire process how to prepare
the data how to train the classifier and how to test its performance so following the steps of
this tutorial you will be able to build a very robust image classifier in only a few minutes
so let's get started so this is the process in which we are going to be working today in today's
tutorial you can see that this is a four steps process so training this image classifier will
take only four steps this will be a very easy and a very straightforward process now let me
show you the requirements we are going to use in this project we are going to use scikit learn,
scikit image and numpy and the image classifier we're going to use comes from the library scikit
learn this is the library which is very popular to solve machine learning related problems and
if you want to work with computer vision or with machine learning you will definitely need to be
familiar with scikit learn so this will be a very good example this will be a very good opportunity
in order to get more familiar with scikit learn now let's start with this project let's start with
this tutorial and the first step is preparing the data we are going to use in order to train this
image classifier and now let me show you the data we are going to use in today's tutorial this
is the data we are going to use you can see we have two different categories the categories are
empty or not empty this sounds very strange but let me show you exactly how these two categories
look like or how the data in each one of these categories look like and you can see from the not
empty category we have something that looks like this so basically we have cars these are images
from cars and if I show you... if I show you many many different pictures you can see that in all
of them we can see pretty much the same situation it's like a car that's basically the not
empty category and then if I show you the empty category this is pretty much an empty something
this is pretty much all the images containing empty...ness containing a completely empty... these are
completely empty images in some of them there are very small objects but you can see that all of the
images are mostly empty and these are... this is how these two categories look like this
is how these two categories look like and this data comes from one of my previous videos where
I showed you how to create a parking slot detector and counter using python and computer vision in
this other video I showed you how to take a video exactly like this... a video from a huge parking
lot like this and to make it look like this let me execute the code I showed you how to build in
this tutorial and the idea was to go from here to here the idea was to build something like this
where we detect absolutely all the parking slots and for each one of the parking slots we classify
if the parking slot is empty or not and if it's empty we plot it in green and if it's not empty we
plot it in red so this is exactly where the data comes from the data we are going to use in today's
tutorial you can see that for the not empty category this is how the data looks like these are
images from parking slots containing cars and from the empty category we have images from parking
slots which are completely and absolutely empty so this is the data we are going to use today
and the image classifier I'm going to show you how to build in today's tutorial it's a very
robust classifier in situations like this in situations where the data we have in situations
where the classification we want to make on our data it's very easy it's very simple right where
the different categories we want to tell apart the different categories we want to classify they are
visually super super super distinct they are super different right in this case we have images from
cars and in this other category we have absolutely empty images right, the image classifier I'm
going to show you today it's very robust and it works super super well in situations like this
and the reason I'm saying this is because this may not be a state-of-the-art image classifier
if you look for papers or if you look for the most recent techniques obviously you are not going
to find the classifier we are going to make today we're going to build today... and it's
definitely not the most robust classifier in the machine learning industry but if you want to
solve like a very very simple problem like the one we are going to solve today this image specifier
will be more than enough and I'm going to show you exactly what's the performance we achieve
with this classifier but that's later on in this tutorial now let's get started I already showed you
the data we are going to use and the first thing I'm going to do is to import os which is a library
which we definitely need in order to work with data and to read data and to load data from our
computer so I'm going to define a directory which is input directory and in this case I'm just
going to hardcode where the input directory is located which is here this will be something like
this and then I'm going to Define our categories and I'm also going to hard code it and I'm going
to say something like this our categories are empty and not empty okay and then the only thing I'm
going to do is to Define two lists which are the lists which are going to contain our data one
of them will be data and then the other one will be labels and I'm going to show you exactly how
we're going to use these two objects later on this tutorial and now what we're going to do is
to iterate in absolutely all the images in these two directories and we are going to load these
images we are going to read these images and we are going to format these images... this data in
a way that fits well the classifier we're going to use later today so let's start with it I'm going
to say something like for category in categories and then for file in something
like os listdir input dir and then I'm going to define the
image path like os path join and then input directory and category uh sorry this will
be os dot path dot join input dir and category okay because we are walking in absolutely all
the files for each one of our categories right something like this okay image path join input
dir category and then file and then we have defined exactly what's the location of our images
now the only thing we need to do is to read our images and to save our images into this list so
in order to read our images I am going to import from scikit mage dot IO Import imread in this
channel in all the videos I had made so far we usually used opencv as an image processing library
and obviously opencv it's by far the most popular and the most comprehensive of all image processing
libraries is the one I use the most in my projects but there are also other libraries which are
more convenient in different situations and for example in this case we are going to use scikit
image we're going to use this image processing library in order to read the images from our disk
from our computer so I'm going to call imread from image path and this will be image okay
and now I'm going to resize this image and for that I am going to import another
function from scikit image which is called from scikit image dot transform import
resize so I'm going to call resize I need to input my image and I need to input
the new size... which... the new size I'm going to resize all my images to which is 15 x 15
and this will be my image so I'm walking through absolutely all the images in these two
directories I'm taking these images I'm reading these images from my computer and I'm resizing
these images from this size to 15 x 15 okay now let's continue I have resized all of our
images and what I need to do now is to append this data into the list I have created
in order to contain all my data so data.append and we're not going to append the image... like the
image we have read and we have resized, but we are going to append the flatted image we are going
to take our image and we are going to make it into an array right because currently our image
is something like a matrix right it's something like a matrix of size for example in this
case 61 x 26 and three channels it's BGR we have resized this image now it's 15 x 15
and three channels now we want to take all of that information and we want to make it into
only one very very long array right we want to make it unidimensional so that's why we are
calling flatted and the reason we are doing this is because that's what we need to do in order to
use the image classifier we're going to use today when we are working with machine learning you
will notice different classifiers and different models and different libraries and different so on
different everything require different formats or require we format the data differently in order
to be fed into the machine learning model into the classifier or into the whatever we are using
and in this particular case we need to flat the data we need to make the data into an array
before we input this data into our classifier so that's all for the data and we also need
to append the category for this data the label and in order to do that I'm going to walk through
the categories slightly in a slightly different way I'm going to call this is category
index category in enumerate categories and now I will append category index to these
labels arrray to this label list and that's pretty much all we need to do the only thing I'm
going to do next is to cast each one of these lists into a numpy array so I'm going to say asarray
and then data okay and I'm going to do the same with labels np asarray labels I know what you're
thinking hey Felipe this is not going to work because you have not imported numpy and you're
completely right so I am going to import numpy now import numpy as np okay so that is it that's all we
need to do in order to load our data and in order to prepare our data to be fed into the classifier
or in order to move to the next step that is it we have completed the first step in our process so
congratulations we are one step closer to reach our goal of training our image classifier now in
order to make sure everything works properly and we don't have any error I'm going to execute
the code as it is now and numpy has no flatted um maybe it's flatten I think it's flatten maybe
I made a mistake so let's see what happens now it seems we don't have any errors so yeah I made a
mistake now we have to wait a few minutes because remember we are walking through absolutely all
the images in these two directories and we have among the two directories we have 6090 images so
this is going to take some time but yeah now it's completed and now it's time to move to the next
step where we are going to split our data into two different sets we are going to create a training
set and we are also going to create a test set the training set is the one we are going to use in
order to train our image classifier and then the test set is the one we're going to use to test
the performance of the classifier we are going to train so we definitely need to split all of our
data into these two sets and this is the function we are going to use, we need to import another
function from scikit learn and this function will be something like from scikit learn dot model
selection if I'm not mistaken import train test split yeah so this is how I'm going... we're going
to use this function I'm just going to write some things and then I'm going to explain it so this
will be something like X train X test y train y test... calling our data X and Y it's a very
popular convention in machine learning so we are going to do it too and this will be our train
test split function and we need to input data labels and then there will be other parameters
as test size I'm going to set this in 20... in 0.2 don't worry I'm going to explain exactly why
I'm setting all these parameters in a couple of minutes then another parameter is stratify
this will be according to labels and then Shuffle I'm going to call Shuffle here and this will be
true okay so I have called train test split I have input our data and our labels and then I have
specified a few parameters in order to show you exactly what these parameters mean... I'm going to
start with test size I told you we were splitting our data into two different sets the training
set... and the test set now let me show you an image to show you... what
this means, to show you exactly how this looks like so this is an image I found online this is an
image I found on Google or actually on duckduckgo and basically you can see that we have an array which
is called data which contains all of our data and we're creating two different arrays from it we are
creating X train and X test and we are specifying this size this split size which is 0.2 so what we
are doing is creating two different sets which are training set and test set from the total amount of
our samples from all of our data we are splitting all of our data into two different sets and the
way we decide what's the size of the test set is by specifying this parameter So currently we are
telling train test split to make this split in a way that 20% of all of our samples
are in the test set right now moving to the next argument Shuffle equal true this is
something that we always want to do when we are creating data when we are preparing data in order
to be fed into a machine learning classifier we definitely want to shuffle this data first so we
can avoid absolutely every bias we had when we were reading the data when we were creating
these data arrays... we definitely want to shuffle the data first this is a very good practice
sometimes there are some biases which we are not aware of sometimes the way we are reading
data or the way we are creating these arrays we we are making some mistakes or we are... I don't
know the way this walking is done maybe it's alphabetical order or maybe it's whatever we
always want to shuffle the data first that's a very good practice to avoid any type of bias when
we were reading the data and creating these arrays then stratify equals labels I would say it's always
a good practice when we are splitting our data into two different sets to Define exactly how
we are stratifying this split and in order to show you exactly what stratify means I'm going to
show you this picture which I also found online on duckduckgo and basically you can see that we have a
data set in this case is these 12 people and you can see three of these people are blue three
of these people are green and then six of these people are red and in this example we are taking
a a sample of four elements of four people and the way this sampling is done is by stratifying
by the different labels so if you look at the sample you can see that two of these samples are
red one of the samples is green and the other one is blue so we are keeping the same proportion of
the different labels in our original data set... that's exactly what we are doing when we are
stratifying according to the labels it's like a way to make sure that the all the different labels
are going to be in exactly the same proportion as in the original data set so it's definitely a good
practice... this is something we always always want to do now let's continue and you can
see that we have completed the second step in our process we are one step closer we have only two
steps left two steps to go we are one step closer of training our image classifier and that's going
to be all for splitting our data into a training set and a test set now it's time to train our
image classifier now it's time to do like the the actual training and this is how we are going
to do it I'm going to import another function which is called... if I remember
correctly from sklearn I think it's model selection as well import
gridsearchcv then from sklearn dot svm import SVC okay so the first thing I'm going to do is
to Define an object which is called classifier and this will be something like this this is
the classifier we are going to use uh it's SVC let's see if it's okay is sklearn svm import so I
think it's okay for some reason it didn't found it now I see what's the error SVC should
be in capital letters and now everything should be okay okay so what I'm doing here is creating
a new instance of this object and I'm calling this object classifier and this is the classifier
we are going to use in order to train our image classifier so what I'm going to do next is to
Define another object which is parameters and this will be something like this I'm just going to
write it and I'm going to explain it in a couple of minutes I'm going to define something like
a list of only one element which is going to be a dictionary with two keys one of these keys
will be gamma and then the other key will be C and that's pretty much all and then I'm going
to define a list for each one of these uh items and for gamma it will be something like 0.01
0.001 and 0.0001 okay and for C I'm going to do something like 1 10 100 and 1000 okay and
then it's when I'm going to call grid search I'm going to define a new object which is called grid
search and this will be grid search cv and I'm going to input these two objects I just created the
classifier and the parameters okay and now let me show you exactly what I'm doing I'm creating a
new instance of SVC and I'm calling this instance classifier and this is the classifier we are
going to use and I'm creating this object which is called parameters which is a list containing
only a dictionary and this section has only two keys one of them is gamma and the other one is
C and each one of these keys has a list of values okay now I'm going to to show you the scikit
learn documentation and going to... and going back to scikit learn and I'm going to show you how the
documentation looks like for SVC for the object we are using as our classifier and you may notice
all the different parameters we have in this classifier we have C kernel degree gamma cov0
shrinking probability and many many many other parameters we have many many many many different
parameters to choose from but when we were creating our object we are just calling this empty
Constructor we are not specifying absolutely any parameter we are just using all the default values
for SVC okay and you may notice as well that we are getting this other object and these two keys
we are creating are exactly two parameters from our... from our object from the
object we are going to use as our classifier so what we are going to do is we are not going
to train only one image classifier we are going to train many many many many different image
classifiers and we are going to train an image classifier for each one... for each combination we
have for C and Gamma so we have three values for gamma and we have four values for C this means we
are going to train 12 image classifiers right the way this process is going to work is that we are
not going to train only one image classifier but we are going to train many many and as many as
different combinations we have for C and Gamma in this case we have three values for gamma
four values for C three times four it's twelve so we are going to train 12 image classifiers
that's the process we are going to take in this tutorial and the way it works is that
we are going to choose the best of all of these different classifiers we are going to train so...
and the way we are going to do this training the way we are going to train as many classifiers
at once is by calling grid search by calling this other object I have specified over
here so this is how we are going to use it I have already defined this grid search and then the
only thing we need to do is to call grid search dot fit and I'm going to input X train which
is my training set and then y train which is which are the labels of our training set and
that's pretty much all that's pretty much all in order to train our image classifier that's pretty
much all to train all these 12 image classifiers right let's see what happens when I press play
now remember the way this works is that we are loading the data first and this is going to take a
few minutes because ithis is a lot a lot of data these are a lot of images and then we are taking
this process now the training also takes some time because remember we are training many many many
many different classifiers so this is going to take some time so I'm just going to wait a couple
of minutes and I'm going to see if something happens or if the execution is successful okay
so the execution is now completed we don't have any errors so we can continue and and this was
the third step in our four steps process this means we are pretty much there the only thing we
need to do now is to test the performance of the model we trained but we are almost there we have
almost completed this process and most importantly we have already trained our image classifier, our
image classifier is now trained the only thing we need to do now is to test how it performs to see
if we can use it or not okay and this is how we are going to do I mentioned that we were training
not only... not only one image classifier but we are training 12 different image classifiers one for
each one of... one for each combination of C and gamma so we are training many many different classifiers
and the way we are going to select one of all of these image classifiers if is by calling a member
of grid search which is called grid search dot base_estimator_ so by calling this
member is that we are going to get the best of all the different image classifiers that were
trained right we are training 12 different image classifiers and we are just choosing the
best one and the way we are choosing for the best one is by calling this member of grid search
that's basically what we are doing so this is our model this is our classifier best estimator this
is our model now let's see how it performs let's see if it's really like a good classifier let's
see what's the performance of this classifier on our... test data on the data we have created
in order to test the performance of this algorithm so what I'm going to do is to call base estimator
dot predict and I'm going to input my test data so I'm going to input X test and I'm going to call
the output from this prediction y prediction right and then I need to import another function which
is from sklearn dot metrics import accuracy score and let's see what's our performance
and I'm going to make a print I'm going to Define another value which is score and
score will be accuracy score y prediction and y test right I'm taking this which are the
labels of the test set and I'm just comparing against our predictions and now I'm going to
print something like I'm going to format this super super nicely so we get like a... like a very
clear measure of how well this performs so I'm going to express this as a percentage
and I will say something like of samples were correctly classified something like this and
this will be format string score times 100 right because score is going to give us a number
which ranges between 0 and 1 and it's going to be a measure of our score of how accurate our classifier
is on the test set now this is a value between 0 and 1 and it's very very useful and that contains
absolutely all the information but what I'm going to do is just reshaping where I'm going to do
like a reformat of this number into a percentage right because it's going to be much much cleaner
in order to see how it performs and all I have to do now is to execute this code again and
let's see what happens let's see what's our performance so I'm just going to press play and
I am going to wait a couple of minutes just like before and I will come back with our results
let's see how it performs the execution is now completed and this is the accuracy we got
with the best estimator from all the different image classifiers we trained we are getting a
99.9 percent accuracy this means that this is absolutely perfect this is a pretty much perfect
classifier a 99.9 accuracy is like a super super high performance it's like a very good performance
so we can definitely use this classifier later on we can definitely use this classifier in order
to use it in our project so the only thing we need to do next the only thing we have to do
now in order to complete this tutorial is to save this classifier to save this model because
we want to load this model from another project or we want to load this model in a different...
code or in a different whatever from a different location we definitely want to save the weights
or we want to save exactly all the information which is related to this model so we can use it
later in a different project and the way we are going to save this model is by using pickle which
is another python Library so I'm going to import pickle and then I'm just going to call pickle dot
dump and I'm going to specify the model I want to to save the object I want to save and also I need
to specify the file which is going to be something like model.p and then I need to open this file
as wb okay and that's going to be pretty much all in order to save our model in order to have a
file with our model so we can use it later on, on our project or from our location or we can use
this file in whatever way we want so I'm going to press play and that's going to be pretty much
all after the training process is completed and after everything it's done we should have a file
in exactly the location we have specified which in my case is here... we should have a file which
is called model.p so this is it this is going to be all for this tutorial we have absolutely
completed all these steps in our process we have completed all four steps in our process and
we have trained an image classifier using Python and scikit learn hey my name is Felipe and welcome to my
channel in this video I'm going to show you invite you to write me a message in the comments
section below telling me what do you think about how to make an image classifier using Yolo
V8 on your own custom data I'm going to show this video telling me what do you think about this
tutorial and also telling me your ideas or your you every single step of this process from how
to organize the data so it complies with Yolo V8 recommendations for other videos or other projects
we could work next on this channel my name is how to do the training in your local computer
and also from a Google Colab how to validate the Felipe I'm a computer vision developer and in this
channel I make tutorials coding tutorials exactly performance of the model you trained and finally
how to take the image classifier in order to like this one and I also share my experience and
my resources the resources I use as a computer make new predictions I'm going to show you the
entire process this is going to be an amazing vision developer so if these are the type of
videos you are into I invite you to subscribe tutorial and now let's get started so on today's
tutorial I'm going to show you how to train an to my channel this is going to be all for today
and see you on the next video image classifier using yolo V8 on your own
custom data set so let's get started and the first thing I'm going to do is to show you the
data I am going to use in this tutorial which is a weather related dataset let me show you the
different categories we have and let me show you all the different images how they look like
we have four different categories and they are cloudy, rain, shine and sunrise now let me show you
each one of these categories for example the cloudy category this is how the images look like you
can see that in each one of these images we have a sky which is completely cloudy right we have many
different clouds for each one of these images now the sunrise category it's basically many different
pictures of sunrises so this is how this category look like and now for the shine category we have a
sky which is completely completely clear and with a super super bright sun right you have the sun
in each one of these images and it's super super bright and this is the rainy category and you can
see these are many different pictures of super rainy days so this is basically the dara set I am
going to use in this tutorial but obviously you can apply absolutely everything I'm going to show
you today to absolutely any type of data set you are going to be able to build any type of image
classifier with everything I'm going to say in this tutorial now let me show you the structure
you need for your data because if you're going to train an image specifier or if you're going
to use yolo V8 yes the data is super super important but you also need to structure to give
like a format to all of your data so it complies with the way yolo V8 expects your data to be
right yolo V8 requires your data to be in a given format in a given structure so I'm going
to show you exactly how to structure your file system so everything looks the way it shloud to train
an image classifier using yolo V8 so if I show you I have a directory which is called weather
data set this is going to be the root directory you can call this directory whatever you want but
you need a directory which is going to be your root directory and inside this directory you can
see we have two different folders one of them is called train and the other one is called val and
this is exactly where you are going to have your training dataset and your validation dataset right
it's very important you name these directories exactly like this one of them should be called
train and the other one val now if I show you within the train directory this is where we are
going to have our four directories containing all the different images for all of our categories
basically you need to have as many directories as categories you want to classify with your model so
in my case I want to classify an image into four different categories and this is why I have four
different directories each one of these directories is named as the category
I want to classify my images in right when one of them is called cloudy the ther one
is called rain then shine and then sunrise and these are the categories I want to classify all my
images and then within these directories, these folders is where I have all my data within cloudy
is where I have all my data related to the Cloudy category and so on right the same happens for
the rain and the shine and the sunrise category so this is basically the structure you need for
your data the structure you need for your file system in order to comply with what yolo v8
is expecting for your data and then if I go to the val folder you can see I have exactly the
same structure I have four different directories and they are named under the categories I want
to classify all my images and then if I open this directory it's exactly the same you can
see that I only have different images for that specific category now this is very important
because from now on everything is going to be super super straightforward if you have created
this structure for your file system if your data is exactly in the structure I show you there
is going to be super simple to train an image classifier in yolov8 so this is very very
very important now I'm going to show you three different ways in which you can train an image
classifier using yolo V8 so let's start with the first way which is using a python script we are
going to make a very very simple script in Python in order to train this model and let me show
you how to do it so let's go to pycharm this is a pycharm project I created for todays tutorial and the first thing you should do
if you want to work with yolo V8 is to install a couple of dependencies a couple of python packages
these are two packages we are going to use in this tutorial one of them is ultralytics and the
other one is numpy, ultralytics is very very very very super important because this is exactly the
library you need in order to import yolo, in order to train this model using yolo V8 so you
definitely need these two packages, now in order to install these packages this is how we are going to
do it I'm going to show you a way to install these packages which is going to work with whatever your
OS right if you are a Linux user or if you are a Windows user if you use Mac it doesn't matter it's
going to work anyway so you need to go to file then settings and then you have to select python
interpreter right this is the python interpreter we are going to use you can see that I'm using
python 3.8 and then you need to click on plus and this is where you're going to find... you're
going to search for the packages you want to install in my case I'm going to search for
ultralytics and the version I'm going to use let me copy the version first it's this one so I'm
just going to file setting then Ultralytics again and then the version is this one okay and then I
click on install package in my case I have already installed this dependency so nothing is going
to happen on my computer but please remember to do it on your computer because otherwise you
will not be able to use or you're not going to be able to do anything of what we are going
to be doing today now let's see numpy what's exactly the version we are going to use we're
going to use 1.24.2 so file settings Plus numpy 1.4 24.2 so everything is okay now install package
this is like uh everything it's okay numpy has been installed successfully so now we are ready
to continue once we you have installed these two dependencies these two packages now you're ready
to continue and now you're ready to install your own image classifier using yolo V8 so let's
go to main this is the file we are going to use in order to code everything we need in order
to train this classifier and let me show you exactly what's the code you need to type in order
to do this training I'm going over here to the GitHub repository of yolo V8 and I'm going to
select the classification section right I'm going for the classification and then here I'm going
to click on classification docs this is going to open a new file a new URL a new website a new
page and this is exactly the all the information we need in order to train this image classifier
I'm just going scroll down and I'm going to the train section and this is what we're going to do I'm
going to copy and paste this line which is the one in the middle the one that says load a pre-trained
model recommended for training I'm just going to copy and then I'm going back to Pycharm and I'm
just going to paste it obviously we need to import yolo otherwise this is not going to work
so I'm going to say from ultralytics import YOLO and that's pretty much all you can see now we
are creating our model, we are creating the object we are going to use as our model and
then I'm just going to copy and paste this last line which is model.train I'm going to paste it
here and then I'm going to make a few edits I'm going to leave this value I'm going to leave
the image size in 64 but then for the number of epochs I'm going to set it in 1 right because
the first thing we're going to do is we're going to do a very very dummy training in order to make
sure everything works as expected in order to make sure everything works properly and once we are
completely and 100% sure everything is okay we are going to move forward with a more deeper training
and with a more real training right but for now let's just do the training for one Epoch and let's
see how it goes then for data this is where you're going to specify the absolute part to the data
you are going to train this model with right in my case it's going to be this weather dataset so I'm
just going to copy and paste the absolute path of this data set which is this... I'm going to copy this
path and I'm going to paste it here right this is the data I am going to use remember that you need to
specify the absolute path to the root directory of your data and remember you need to structure
your data into the exact format that I already mentioned right otherwise this is not going to
work and that's everything we need in order to train this image classifier so the only thing I'm
going to do is to press play I'm going to run this script so let's see what happens remember we are
running this training we are doing this process for only one Epoch because we need to make sure
everything works properly and once everything is working properly we are just going to edit
this value we're going to make this training for more epochs but you can see everything seems
to be working properly so everything seems to be okay and everything seems to be completed and
everything seems to be ready so that's it and you can see that the results have been saved here
in run/classify/train12 so let me show you exactly where this directory where this location
is in my file system if I go to the project the pycharm project I created into my file system
this is exactly the project I created this is the file we are currently working in the main.py file
and this is where my data is located and this is where the runs directory the runs folder will be
located this is where it will be created you can see that within runs we have another directory
which is called classify and here is where you will have many many many folders for each one of
your training processes and you can see that in my case I have trained this classifier many many
many different times while I Was preparing this video so there are many directories for me but
this is exactly the one which was just created the train12 right train12 this is exactly
the directory which was just created and if I open this directory you can see we have another
directory and then we have two files I'm going to explain what exactly all these different files and
all these different folders are and exactly what's the information we have in all these files but
I'm going to do it later on this tutorial when we are validating this training process right
for now just remember all the results will be saved here will be saved within
this folder within the runs folder and then within classify and then a new directory a new folder
will be created for the training process you have just executed right this is something you need to
remember for now but later on this tutorial I'm going to show you exactly how you can validate the
training using the information that's within this directory but for now let's continue I'm going
to show you now a different way in which you can train this image classifier using yolo V8 I'm
going to do... I'm going to show you how to do it using the command line using this utility and this
is actually like a very very straightforward way to do this training let me show you you can see that
we have three different examples I'm just going to select this one I'm going to copy and paste this
instruction this line and I'm going to show you how it I'm just going to paste it here and you can
see that we have many different parameters right in the first word is yolo this is the utility
we are going to execute then classify this is the task we are going to execute we are going to
train an image classifier and then we are going to train it so we need... we have another keyword
which is train and then we have these arguments data model and epochs and also image size I'm
going to do exactly the same with image size I'm just going to leave this value in 64 but then
I'm going to edit all the other values so actually I'm going to the number of epochs and I'm
also going to edit data for the number of epochs let's do something similar I'm just going to do
it for one epoch so we make sure everything runs smoothly and everything runs properly and then
we can do like a more serious training a more real training for more epochs this is exactly
the model I'm going to use so I'm not going to edit this keyword either and then I'm going to
edit this argument and I'm just going to say this is the absolute path to my data so this is
going to be exactly the same as I have over here something like this okay and that's pretty much
all the only thing I need to do now I'm going to copy and paste this sentence and I'm just going
to a terminal and I'm going to do something like this right I'm just copying I have just copy and
paste that sentence and you can see that that's all we need to do in order to train this image
classifiers using yolov8 you can see that the model... the training process has started and everything
is running super super smoothly so everything is going super super well that's all right that's a
very very quick way and a very straightforward way to do this training you can see the training has
just been completed and this is exactly where the results have been saved to runs/classify/train13,
so everything is completed everything is ready you can see how simple how fast is to train an
image classifier just by running this command now I'm going to show you another way to do this
training which is using a google colab we are going to use a Jupiter notebook we are going to use a
notebook in a Google collab in order to train this model and this is also like a very good way to do
it so let me show you how to do it so basically you need to go to google drive you need to go to
your Google Drive you need to select new then more google collaboratory and this is going to
open a new notebook this is going to open a new notebook in Google Colab and
is exactly what you need to do in order to use this notebook to train yolo v8
now I'm going to show you a notebook I have already created in order to train
this model which is this one is called train.ipymb and obviusly I'm going to give you
exactly this notebook in the GitHub repository of today's video of today's tutorial so you can
just use this notebook if you want now I'm going to show you all these different cells everything
that's already writen on this notebook so you can... so you understand how exactly to use it
and how it works and what exactly you are doing at each step so let's start with the first step
another thing you need to do if you want to train this image classifier is to upload all
the data with all the images and with all your categories into Google Drive obviously for
example in my case this is where I have my weather data set you can see that this directory
is exactly this same directory I have over here weather data set within weather data set there
are two directories which are train and val if I if I open this directory you can see we
also have traiin and val so this is exactly exactly the same data as in my local computer now
this is something very important because remember to do it because you need the data in your Google
Drive in order to train this model using a Google collab this is a very very important step please
remember to upload your data into Google Drive now once your data is in Google Drive then you
need to be able to access your data from the Google collab and in order to do that you need
to execute this cell if I click enter you can see that now I'm going to be asked if I want
to connect Google collab with Google Drive and the only thing I need to do is to say I accept there
you can see that it's requesting for my permission I say connect to Google Drive and then
I select my account and then basically is to scroll down to the bottom of this page and to
click allow and it's going to allow Google collab to access all the data you have in your Google
Drive so this is a very very very important step now something that's very important is that
you need to be able to access your data so you need to know where your data is located in the Google
drive right you need to know exactly what's the path what's the location of your data in Google Drive
in my case let me show you my Google Drive you can see that my data is located into a directory
this is my root directory which is my drive then I have another directory which is called computer
vision engineer then another directory which is image classification yolo V8 and then data and
then this is where my weather data set is located in your case it's going to be different obviously
it depends on where exactly you have uploaded your data so something you may want to do is just to
click this... ls you can say something like ls and then you say something like content my Gdrive
my drive right you execute this command and if I execute this command you're going to see a very
very long list of files which are basically all the files which are in my root directory in
Google Drive and for example this is where I have the directory which is called computer
video engineer and if I do ls you're going to see all these different directories if I say
something like image classification yolo V8 then this is data train.ipymb which is
exactly this notebook and then if I say data this is exactly where the weather data set is
located right so do something like that because you definitely need to know what is the path of
your data in Google collab right you definitely need to do it in order to continue to The
Next Step this is very important because if you haven't set your data properly if your data
location is not set properly then yolo V8 will not be able to train your model this is very very
important so in my case this is exactly where the data the weather data set is located right this is
the path to the weather dataset so this is the the cell I am going to execute and this is the value I'm
going to save in this value in data dir now I'm going to continue then we need to pip install
ultralytics which is the library we need in order to train this model in order to use yolo V8 now
the only thing you need to do is to execute this cell and everything will run super smoothly you
can see that we have already completed this process now I'm going to continue and the only
thing we need to do now is to execute this cell and you can see that the code we have in this
cell is very very similar to the code we have over here right basically we are running a python
script from a Google collab that's all we're doing so you can see we are importing OS and also we
are importing the YOLO Library we are importing from ultralytics we're importing yolo and
then we are doing exactly the same as we are doing before and this is where we are using the
data directory the data dir variable we have defined over here right so this is why it's
very very important you set this variable properly so the only thing I'm going to do... I'm going to
do exactly the same as before I'm just going to do this training for only one Epoch so we make
sure everything's okay I'm going to press enter and that should be it in order to do all
this training the first time you execute this training it may take a little longer because
you are downloading all the weights and you're downloading the models and everything but uh after
that everything should be much much quicker okay so you can see that now the training process is
in progress everything is going super super well and from now on the only thing we need to do
is to edit the number of epochs so we do like a more deeper training but I will say everything is
working super super properly so now let's move to the other cells so I show you what exactly you
need to do once everything is completed once everything is completed the only thing you need to
do is to run this cell so you are copying you are going to copy all your results which were saved
on this directory you're going to copy everything on your Google drive right because remember you
are working on a Google colab you're working on an environment which is your Google collab
environment if you don't do something like this it's going to be super super hard for you to get
the data you have just trained right to get your results to get your model your weights is going to
be super super hard because everything is located in your Google collab environment and long story
short is going to be much much simpler and much much better if you just do something like this
and you just copy everything all the results which were saved in this directory into your Google
Drive it's going to be much much better because it's going to be much easier to download the weights
to download the results and so on so now I'm just going to wait a couple of minutes so everything is
completed over here and then I can show you how to copy the results into your Google Drive okay now
the training process has been completed and you can see that the results have been saved into runs
classify train so this has a very similar output to the one we just noticed when we were training on
our local environment now the only thing we need to do is to copy everything into our Google Drive
so everything is much much simpler if you want to download these results or to do whatever we
want so the only thing I'm going to do is to run this cell and everything will be copied into this
directory which is the same directory where I have my data and where I have my my Google collab right
now you can see that everything has been copied already this is the directory I have just copied
this is the time this is the current time so this is the result of the cell I have just executed and
if I go to runs classify train you can see that these are all the results we have generated this
is the CSV file containing many different results which I'm going to show you in a few minutes
and these are the weights and so on so from now on if we want to get this data or if we want to
analyze this data the only thing we need to do is to select runs and then we just need to click
download and it is going to download all this directory into your local drive right you can see
everything is being zipping and once everything is zipped this directory will be downloaded
into my local computer and you can see that this directory has just been downloaded so everything
is working just fine now this is pretty much all in order to show you three different ways in which
you can train an image classifier using yolo V8 and now let's do the deeper training right I'm
just going to take this script and I'm going to edit the number of epochs so we do this training
for something like 20 epochs I have already been doing some tests and 20 epochs is just enough for
this dataset for the data set I am using in this tutorial so 20 will be just fine now the only
thing we need to do is to click on run I'm just going to run this script as it is and everything
will be exactly the same as before everything will be exactly the same right we are just we just need
to wait until this process is completely we don't need to do anything from now on but this process
will be executed for 20 epochs so the only thing I'm going to do is to wait until this process is
completed and once everything is completed we are going to validate this training process I'm going
to show you how to analyze if all this process was done successfully or not if you have successfully
trained a good image classifier or not so I'm just going to pause the recording here and I'm
going to fast forward until this is completed okay so the training process has been completed
and now let me show you all the results which were saved here into runs classify and train14
now let me show you this directory this folder in my local computer if I go to runs
classify and then train14 this is where all the results have been saved and this is everything
we are going to analyze now, now we are going to decide if the model we have trained is a good
model or not we are going to decide if this is a model we can use or not so you can see that
there are two files args.yaml and results.csv and another directory called weights let's start
by args.yaml if I open this file you can see that this is something like a config file and
this is exactly the entire configuration file which we have just used in order to train this
model this is very important because this is a super super comprehensive list of all the hyper
parameters we have used in order to train this model and for example the only parameters we have
specified are image size number of epochs and then data the location of the data we have just used
and you can see that we have a keyword which is data then epochs then image size and then we
have many many many other keywords as well this is very important because these are
absolutely all the keywords we have used we have used all these default values which were
set for all these different keywords and this is important in case we want to train a new model
and we want to make some changes into some of these hyper parameters now let me show you the
other file which is the results.csv file I would say this is much more important this is like the
file containing all the information we need in order to decide if this is a good model or not
and you can see that we have many different rows each row for one of our training epochs right we
have trained this model for 20 epochs and you can see that we have 20 rows for each one of these
epochs and for each one of these rows we have all this different information and we are going to
focus on these three values on the training lose the accuracy, this is the accuracy of the
validation set and then also the validation loss right these are the three keywords in which
we are going to focus on this tutorial in order to validate this model and I'm going to give you
like a very very quick tip like a very quick way in order to analyze this training process which
is make sure the training loss and the validation loss are going down through this training process
and also make sure the accuracy goes up and I know you're thinking hey this is a very simple way
to analyze this process felpe yeah I agree with you this is a very simple way but at the same
time it's very robust this is like a a very simple but at the same time very powerful way to
decide if you have a good model or not now we can analyze all these numbers but I think it's going
to be much much better and it's going to be much much prettier if we make a plot with all these
numbers right because we have epochs in this um in this column in this coordinate and
we also have all these different values and we can definitely plot these values across all
these different epochs so let me show you a python file I have created and this is exactly what
this python file does this file is called plot_metrics and if I open this file you can see that
it basically we need to set the path to our results.csv file in our case I'm going to set it to
train14 and you can see this is run/classify/train14 and thenresults.csv and then this is only
like some logic some very simple logic to take all the data from these results.csv file and to do
some plots with it right that's all we are doing we're just taking the data and doing some plots
and this file will be available in the GitHub repository of this project of this tutorial so you
can definitely take this file and you can just use it to to plot your functions as well all I'm going
to do now is just press play and you can see that if we wait only a few seconds we get all these
two plots right and this is all the information in our CSV file right everything I showed you
over here it's summarized on these two plots so this is exactly what I mean with make sure your
loss is going down this is your loss in the training set and in the validation
set in the training set we are plotting the loss in blue and in the validation set is red and you
can see that in both cases the loss is going down right which is exactly what we expect it's exactly
what we want now this is a very very simple way to analyze this process but trust me this is also a
very powerful way right this is something that's very very healthy something that looks like
this it's very healthy and then for this other plot which is how the validation accuracy evolves
through this training process you can see that the evaluation accuracy goes up when we increase the
number of epochs right you can see that starting from the 10th Epoch or so everything starts to be
like somehow iddle right we are not really gaining a lot of accuracy from here but we are not losing
accuracy either right we are just in something like a plateau and this this is exactly how a
validation accuracy plot should look like right we are starting from a very low value and then we
are just increasing our accuracy until we reach a very high value of accuracy right this is like
a very healthy training process now obviously we could make this process even better if we just
tune if we just change some of these parameters and if we do like a more customized training I'm
sure we are... we will be able to have a better model right because remember we are using all the default
values so as it usually goes if we make like a more customized training and
we try different parameters and so on we should be able to get like a better model but obviously
we're not going to do it in this tutorial because I just wanted to show you like the end-to-end of
how to train this image classifier but remember you could do it even better than this if you make
like a more custom model so this is pretty much all for analyzing these plots which are the
validation accuracy and the loss function in order to validate your training and then it's like
this directory which is the weights directory you can see that this directory is called weights
and this is exactly where the models will be saved this is very important because you have trained
a model and now obviously you want this model in order to use it in your images in your data and
this is exactly where you are going to find this model and you can see that you have two different
files one of them is called last.pt another one is called best.pt now let me explain exactly what
these two files are and exactly what they mean so remember how this training process works right
remember that you have a model you have a deep learning model which is comprised of many many
many different weights and the way it goes is that at the end of every Epoch right at the end of
the first Epoch of the second epoch of the third epoch and so on you are updating the weights
of your model you are updating the weights of your architecture of your deep learning model so
the way it works is that at the end of every Epoch you have a model available which is a model
you have trained so far with all the process you have followed so far so last.pt means that
you are taking the model which was the result of the last Epoch of your training process right
remember at the end of absolutely every single Epoch you have a model available which you can
definitely use if you want to in order to produce your inferences and so on, so last.pt only
means that you are taking the last Model the model which was produced at the end of your training
process at the end of the last Epoch in your training process so at the end of the 20th Epoch
in our training process we are producing this model Which is last.pt but you may Wonder hey Felipe
yeah it's great because at the end of our training process our accuracy is something
like a 93% right a 93% it's a very good accuracy but if we take the accuracy
if we take the model at the end of the 16th Epoch for example our accuracy it's higher it's a
94.9 % maybe it makes more sense to take that model instead right because we have an even
better accuracy we have an even higher accuracy and if you ask me something like that I would say
yeah you're perfectly right you're you're super super right that's a very valid argument and
that's exactly what the best.pt model is right we are saving the weights of the best model in
our entire training process so if we look at our data the best model in our training process is
this one if I'm not mistaken right it's the model we produced at the end of the 16th Epoch and our
accuracy our validation accuracy was 94.9% so this is definitely higher than the accuracy
we got at the end of this training process which has which was a 93.5 and if we will take
the best model we have produced in the entire process in the entire training process then
we will definitely need to take this model so this is exactly what best.pt represents is the
best training the best model you have trained in your training process and if you ask me what I
usually do is take in the model which was produced at the end of the training process right what
I usually do is take in the last,pt file because I consider that if this is a model we
have produced at the end of the training process in this model we are summarizing much more
information right because we are considering much more data we are considering much more
everything in all this training process many things are going on many many things are going
on and remember there's a lot of Randomness in this training process so I, me, personally
I consider that if I take the model which was trained at the end of this process is a much
better option that if I choose a another one if I choose like the best model or the model which
got the highest accuracy but it's not the last Model that's what I usually do I usually take
the last model which was produced at the end of the training process but if you want to take the
best model if you want to take best.pt it also makes sense because you are taking the model
which produced the highest accuracy right so you can do either one of them and I think it's
a very a good option that's why you have these two files because you can use one of them or you
can use the other one and I would say that making like a very very like the best decision on which
mode to use depends on many different variables depends on many different things depends on your
data depends on your problem depends on your use case depends on your training process
depends on many many different things which is the best option right so remember you have these
two models and it's all up to you it's all up to your specific project and it's all up to your
preferences which model you want to use right if the best model which you have produced through
the entire training process or if you want to use the last Model the model which you have produced
at the end of your training process so now let's go back to pycharm because now it's time to make
our inferences now it's time to predict new samples right and we are going to input an image
and we're going to use our image classifier in order to predict which category this image belongs to so
let me show you how to do it I'm going to import from ultralytics import YOLO and then
let's go back to this page because now we are going to move to the predict section and
the only thing I'm going to do is to copy this sentence... going to paste it here and then
I'm going to specify the path the absolute path to the model which we have trained right
we don't really need to make it like the absolute path we can use the relative path
so I'm going to do something like this right sorry something like this so this is the path to
the model we have just trained right this is the last model which we produce at the end of this
training process and this is the model I'm going to use in order to show you how this works and
now let's copy this additional sentence which is results = model and the model path the image
path right you can see that you can use an image in your local computer in your file system or you
can also use something like an URL for example in this case in this example which is in the
yolo V8 website you can see that the example is using an URL and this is also going to work so
in my case I'm going to use an image in my local computer I'm going to use one of the images I used
for training because I only want to show you how this works but obviously you can use whatever
data whatever image you want so this is the image I am going to use I'm just going to use I'm
just going to inference this image right which is the first image in my Sunrise category data so
this is going to be something like sunrise1.jpg and this is pretty much all so these are
the results the first thing I'm going to do is just trying to run this code and let's see
what happens everything should run smoothly but this is where we are going to see if we have an
error or something like that we may need to wait a couple seconds and everything seems to be working
fine because we didn't get an error so what I'm going to do now is I'm going to print results
because I want to show you a couple of things so this is the entire information we are getting
when we are printing results right you can see that this is a lot of information we have these
probabilities which is the inferences we are making this is exactly the result of applying
our image classifier and then we have a lot of information another object or another result
which is very important is this one which are the names of the categories we have just trained
our image classifier on right you can see this is cloudy rain shine sunrise and also you can see
that we have different integer values for each one of these categories so this is something like a
dictionary because we are going to have a result from applying our image classifier and then
with this result which is going to be an integer we are going to call this dictionary we're
going to call this object because we want to know exactly what's the name of the category we
have just inferenced right so this is how we're going to do it I'm going to call another variable
which is going to be names something like names dictionary names_dict and this is results
zero because results is a list in this case we only want to access the first element because
we are only predicting an individual image so this is the element we want and then we are
going to call Dot names and that's pretty much all then I'm going to Define another variable
which is props and this is results 0 dot props and this is the probability Vector of all the
different categories we are trying to classify right so we are going to have a length 4 array
with the probabilities of the different classes we are classifying right so let me show you how
props looks like I'm going to print props and I'm going to do something else I'm going to say to
list so we make this object into a list we are using yolo which is based on pytorch so if we
don't do this if we don't call this method we will be working with a torch object right with a tensor
so we don't really want to do that so that's why I'm doing this tolist now I'm going to print
props so I show you how it looks like and I'll show you how to continue from here okay you can
see that this is a result we got from applying from printing props and you can see that this is a list
with four elements one two three and four and each one of these elements are the probabilities
of this image to be one of these categories right let's print the names too so we have all
the information in our screen I want to show you I want to show you something so I'm going to
print sorry this wasn't names this was names dict and now let's wait a couple of seconds I want
to show you not only the probabilities but also the class names so it's a little more clear what
exactly I'm going to show you now so this means that this number is the probability for this
image to be cloudy right this other number is the probability for this image to be rain this
other number is probability to be shine and then this last number is the probability to be
sunrise and you can see by the values that we are definitely classifying this image as Sunrise
right because this is almost a one this is almost like a super super confident and absolutely
confident classification so this is exactly the category we are classifying for this image and
this is how to make sense of this information so what I'm going to do now is to print names dicts
and then I'm going to call np dot arg max and then I'm going to input the probability list
I just showed you and obviously I need to import numpy as np otherwise is not going to work and
basically what we are doing here is that we are looking at this list the one containing all
four probabilities we are taking a look at the maximum number which in this case is this one and
we are taking the index of this maximum number so in this case this is the first element so this is
the index 0 this is one this is two and this is three right so from this um from calling np dot
arg max props we are getting three and then we are calling the third element of the names_dicts
object so we go here and we see that 3 belongs to the sunrise category and if we look at this image
again we are going to see we are in fact plotting a sunrise let me show you so everything seems to
be working fine and this is going to be all for today this is exactly how you can train an image
classifier using yolo V8 in your own custom data and this is going to be all for this tutorial
so this is going to be all for today if you enjoyed this video remember to click the like button
and also remember to subscribe to my channel my name is Felipe I'm a computer vision engineer
and these are exactly the type of projects I make in this channel so this is going to be
all for today and see you on my next video hey my name is Felipe and welcome to my
channel in this video we are going to train an image classifier using tensorflow 100 % online
without using any code whatsoever we're going to use teachable machine which is a tool made by
Google and we're going to train a super super robust image classifier in only a few minutes
now let's get started so let's start with this tutorial and this is exactly the website we
are going to use in order to train this image classifier 100 % online with no code whatsoever now
let me show you the data we are going to use today we're going to use exactly the same data set we
used in one of my previous videos where I show you how to train an image classifier using yolo
V8 we are going to use a word data set and these are the four categories we have in this dataset the
first category is cloudy and you can see how the images look like now if I go to the over category
it's rain and you can see that these are pictures of super super rainy days now if I go to shine
which is the third category in this dataset you can see that in each one of these pictures we
have the sun which is super super bright super up in the sky and super huge and now the last
category is sunrise where we have pictures of many many many many sunrises so this is exactly
thea data we are going to use today and remember we have already used this set in one of my previous
videos now let's go back here you need to go to teachablemachine.withgoogle.com this is exactly
the URL for this website and once you are here you need to press get started let me show you
how simple and how straightforward this process is it's going to be super super quick super
simple super 34 and in only a few minutes we will have a model an image classifier which we
trained 100 online with no code whatsoever now you can see that you have three different options
and we're going to select image project and then standard image model the first thing you're
going to do is I'm going to name make light my glasses I'm just going to name them correctly
one of them is cloudy the other one is rain then I'm going to add two additional classes
the classes are named Shane and sunrise and once I have created my four classes and I
have just named them all properly I am going to upload my images I'm going to click in upload
choose images and now I'm going here to the word set the train set I'm going to select all my
cloudy images and then I'm just going to wait and that's it that's how simple and how easy
is to load all the data into each one of these categories I'm going to do exactly the same for
the Rainy category I'm just going to select all the Rainy images I'm going to press enter
and that's it now I'm going to do exactly the same for the other two categories for
the shine category it's something like this and then for the sunrise category we are going
to do exactly the same and that's pretty much all that's pretty much all that's pretty much all it
takes to lose all of our data to name all of our categories properly and to take all of our data
into this platform that's exactly what we need to do and that's how simple this process is now the
only thing we need to do in order to train this model is to press this button train model you
get the idea right you get the idea how simple and how straightforward everything will be today
but before trading this model I'm going to make a very very small adjustment I'm going to press this
tab over here which is Advanced and I'm going to change the number of epochs I'm going to train it
for 10 epochs so we can see how we get how we go from there right I'm going to train for only 10
epochs I'm going to press train model and that's it the only thing I'm going to do now is I'm going
to wait I'm going to wait until this training process is completed and this is going to be super
super super amazingly quick right this is going to be a very very quick process a very quick training
process now it's raining and you can see that this is the first Epoch the second Epoch the third
Epoch the fourth Epoch and so on right the fifth the sixth the sixth the seventh and you can see
how fast everything is going in only 15 seconds we have trained our model in only 15 seconds we
have trained our image classifier now it's asking for permission to use my webcam because that's one
of the ways in which you can make predictions by using your webcam but we are not going to use the
webcam in this tutorial now the first thing you're going to do before making our predictions before
making our inferences is look under the hood I'm going to press this button because if we have
trained an image classifier the first thing we need to do if we have trained absolutely any type
of machine learning model the first thing we need to do is to validate this model is to take
a look how this model performs right that's exactly the everything we need to do with the
machine learning model before using this mode now if I look under the hood if I look at these
plots this is the accuracy per Epoch and this is the loss per Epoch take a look that now we
have two functions right we have a function in blue and a function in Orange and we have exactly
the same here for the loss functions and although we have uploaded only the training data we
have uploaded only one that said we haven't specified anything as a validation asset or as
a test does it it seems teachable machine it seems this website internally somehow splits
the data into a training set and a test set it seems this is done internally because we
haven't specified absolutely anything because you can see we have two different functions
one of them is accuracy and the other one is test accuracy and for the loss function we have
exactly the same loss and then we have test loss so it seems this website internally splits the
data set somehow and now we have two assets the training does it and the test does it and let's do
like a very very simple validation process let's validate this training process in a very very
simple way the same way we did in my previous tutorials if you watched my previous tutorials you
will remember I always always say make sure your loss function is going down if your loss function
is going down it's likely everything is going well make sure the loss function is going down both in
the training set and also in your validation set and if we look at this loss function you can see
that in the training set everything seems to be going well everything it's it's going down so
everything seems to be going super super well but in the validation set or actually in the
test set you can see that the loss function seems to be somehow stuck and also it seems to be
going up slightly up right it seems to be going slightly in the up Direction and this is something
I don't like about this training process there is something this is a situation that I don't like
now if we look at the accuracy you can see that although both accuracies the the training accuracy
and also they test accuracy both of them are going up you can see that the tree in accuracy is
going up a lot we are getting a 100 accuracy in the training set and that's hardly ever a good
situation because if you are getting that high of an accuracy in your training set it's it's usually
a sign that you are overfeeding your training to your training data so that's hardly ever a good
sign that you don't really like having such a high performance in the training set and if I look at
these two plots I can see that we are definitely overfeeding to the training data because the last
function in the validation data is going up and also we get a 100 accuracy in the training data so
this means the ball is overfeeding to our training data so what I'm going to do now is I'm going
to do a much shorter training and I'm going to stop this training around here which is around the
moment around the time the validation laws started to go down right it started to go up so I'm going
to do this training for only three epochs and let's see what happens so I'm going to press train
model again and now we're going to wait until we train a new model and let's see how this new model
performs okay so this is a performance on the new model we trained you can see that for the loss
function now both of our loss functions are going down right the training loss and the validation
loss or the test loss and for the accuracy we got a very very similar accuracy in both of these
sets and for the training set we are not getting a 100 accuracy so everything seems to be better now
now we are just going to keep this mode and now I'm going to show you how to make predictions
right how to take this small in order to make predictions we have two ways to do it we could
do it on the same platform we could do it 100 online the same way we train this model we could
do it using our webcam but what I'm going to do is I'm going to do it I'm going to upload a file
this is one of the ways in which we could make our predictions so this is how I'm going to show
you how to do it first I'm going to my validation data this is my data which is my local computer
this is completely unseen data for my algorithm for my model so the model haven't seen this data
before and you can see that we are predicting this image as rain with a 99 score so everything
seems to be going well with this model we just trained now this is one of the ways in which you
can make predictions using this model the other way is by going to export model and here you
have three different ways to use this model you can use it in tensorflow.js you can use it as a
tensorflow.js model so you can just put this model into something like a browser using JavaScript
and you can definitely use it from there you can take this small as a tensorflow model and
use it from a python script for example I will say this is the way we are most familiar with in
this channel because we usually work with python and you can also take this model as a tensorflow
lead model and use it from a mobile device you have these three ways to make predictions with
your model and I'm going to show you how to do it using tensorflow you can see that not only we
can export this mode into these three formats but we also have something like a code we can use in
order to make these predictions so we have already all the code we need and we know exactly what we
need to do in order to make our predictions this is how simple and how straightforward everything
is using teachable machine now let me show you I'm going to download my model I'm going to download
this model as a Keras model and then this says convertible once everything is converted and it's
just going to be downloaded and in the meanwhile I'm going to copy and paste I'm going to copy this
code and I'm going to paste it in a python project I created for this tutorial so the only thing I'm
going to do is go into main.pi this is a a file a Python's file I created in a python project and
I'm going to press paste and that is it and that's absolutely all I'm going to do I'm just press
copy and then I press paste so I'm pretty much not doing absolutely anything you may notice and this
is how simple everything is using this website now if I go back here you can see that now
and we have two files one of them is core is called kerasmall.h5 and the other one is called
labels.txt now this is a mode we have downloaded and what I'm going to do is I'm going to take
these two files and I'm just going to locate this file in the in the file system in my fault system
for the python project I have just created right you can see that this is a mine the main.pi file
I however here we we have just space is absolutely all my code and this is where I'm going to take
these two files which represent the model visual strength using digital machine now this is just
going to take a couple of seconds you can see that now these two files are in my file system are
in my local computer and now let's go to pycharm because everything is pretty much ready to be
used but we need to do like a very very small edit which is you can see this file name here it says
kerasmall.h5 everything is okay but you can see that this is a capital m the m is in uppercase we
need a m we need an M which is lowercase right I don't know why we have this typo but we do if you
take a look at the mall you have just downloaded you can see that this file it says kerasmall.h5
but it's in lowercase so that's the only edit we need to do and everything else should
work just fine as it is now I'm going here where it says image pad and here I'm just
going to copy and paste the part to one of my images in the validation set right so
I'm going here ball I'm just going to choose one of the shine images I'm going to choose
chain one so I'm going to do something like these unshine1.jpg and now the only thing I'm
going to do is I'm going to press play and that's it and that's all we need to do in
our core predictions using python with that third floor model we trained use this website that's
how simple everything is now obviously while we are waiting for our predictions everything is just
completed you can see that we are getting shine exactly as we should with a 99.4 confidence score
so everything is amazing and what I was going is that remember to install this project dependencies
remember to install all these packages because otherwise it's not going to work you can see here
at the top we are using Keras peel and numpy so please remember to install all these packages
which are pillow numpy Keras tensorflow and also seek it learn because we are going to do another
edit because I think it's very very interesting we have this code in order to make predictions
or indeed on individual images so we get an idea of how this performs on an individual image but
let's see how it performs on the entire validation set we have over here right remember in one of
my previous tutorials I show you how to make an image classifier using yellow V8 and we used
exactly the same data so let's just compute what's the accuracy on the entire validation set so we
can compare this more we trained with the mall we trained in one of my previous tutorials and in
order to do so I'm going to just make a few small edits the first one is going to be the final new
variable which is root directory and this is where I'm going to specify the path to the validation
that's it so I'm just going to press paste and that should be all okay so this is root directory
and I'm going to iterate in all the directories within my validation data and then I'm going to
iterate also in all the images within each one of these directories and this is how I'm going to do
four deer in OS list deal root directory obviously need to import Os or OSD is not going to work
so import OS and then for J in US dot list there root directory dear and that's pretty much
all okay and then I'm going to Define another variable which is image path and This Is Us Part
join root directory deer and J and that's pretty much all now I'm going to edit this value over
here and I'm going to change it by image part and now we are reading the image path we are reading
absolutely every single image in each one of four directories in the validation data so everything
I'm going to do is to make sure this executes properly and once we make sure everything it runs
super support smoothly I'm going to make some additional edits but for now let's see if this
is just okay if this runs super super smoothly okay we are getting predictions so everything
is just fine and now what I'm going to do is I'm going to save all of my predictions into a list
so I'm going to create a list which is called predictions and then I'm going to create another
list which is called label tools and this is where I'm going to save all my predictions and this
is where I'm going to save all my labels right now I need to import an additional library and an
additional function which is from a skill learn dot matrix is key learn dot matrix import
accuracy score right because I am going to call this function in order to get the
accuracy score of this model in absolutely all my validation data and by calling
this function it's going to be super super straightforward now I'm going back here
and what I'm going to do is I'm going to append this is the index so this is the value we need
this is the the integer we need in order to make or in order to have our classification or
to have our predictions so I'm just going to say predictions.append index now I need to
append the labels the label for these images and in order to do so let me show you something
first these are all class names if I go back here to this project um file system to this project
directory you can see that if I open this file is the labels.txt file we have just downloaded
from digital machine and you can see that we have four values zero cloudy one range true shine
and three Sunrise so these are the category names and these are the indexes right so once we are
saving all the data into predictions and into labels we need to make the predictions and the
labels into these format right we need to append these values now all labels currently the way we
are iterating all these directories our label is here it's in the directory name right if I go
back here to the validation data you can see that the validation data to the root directory of
the validation data you can see that all labels are actually the directories names so we need to
make some process we need to make some function or something to take this name and to return this
Index right it's very straightforward it's very easy but we need to do it first now this is how
I'm going to do I'm going to print class names I'm going to print this variable which is actually
the um this variable this this variable contains all the information we can see over here but let's
see exactly how this information looks like so I'm just going to press play I'm going to press
print this variable and then I'm going to continue okay and now it was just going to press stop you
can see that this is the variable we print and this is how it looks like right so we have four
elements this is a list with four elements they are called the elements are called the zero cloudy
then one brain to shine three sunrise and then for each one of these elements we also have a new
line character right that's very very important so I'm going to create a variable I'm going
to create a dictionary which is something like Category 2 Index right this is going to be the
name and this is going to be a dictionary then what I'm going to do is I'm going to iterate in
all these elements in this list so this is going to be something like 4 class name in class names
and then I'm going to say something like index uh sorry first I'm going to say something like
class name Dot split and I'm going to split it by the space right because I want to get these two
values the index in one in one variable and then I want the category name in the lower variable
so I'm going to say something like this and this is going to be index and name right so I'm going
to say something like this and then I'm going to say that name is equal to name except the last
element right because we we get that the last character is a new line character and we don't
really care about that character and now I'm going to say this Category 2 index name equal to Index
right so now with this new variable we have just defined it's super super easy to do something like
this I'm going to labels append category to index dear because this is the level of the image we
have just read over here so this is how simple this will be now I'm just going to press play
to make sure everything executes properly and to make sure we'll have an error and then it's
going to be super super easy because the only thing we need to call accuracy score predictions
labels right but let's take it one step at a time let's see if this executes probably everything
seems to be okay and now let's see what happens I'm going to print the accuracy score on the
validation data with the mole which was trained using teachable machine let's see what happens
okay so we got an error so I think I know what's the problem and this is the problem that we
need to say this is int right because we need to compare and we need to compare integers to
integers or why this is not going to work now I'm going to press play again and if that was
a problem okay so now I didn't get any error and this is the value we are looking for we are
getting a 98 98.6 percent of accuracy in your validation data with the model we trained and
remember this is completely completely unseen data for this model within use these images in
order to train this model so this is amazing this is an amazing amazing performance so
it's going to be your for today my name is Felipe I'm a computer vision engineer and these
are exactly the type of projects I make in this channel so if you enjoyed this video I invite
you to click the like button and I also invite you to subscribe to my channel this is going to
be all for today and see you on my next video so this is exactly the web application in which
we are going to be working today on today's tutorial this is a pneumonia classifier and let me
show you exactly how it works so I'm just going to drag and drop an image from my computer and now
let's see what happens and you can see that we are classifying pneumonia with a 99.9% confidence
so this is exactly the project in which we are going to be working today on today's tutorial and
I'm going to show you every single step of this process from how to set up this web application
up and running using Python and Streamlit how to use the image classifier and how to analyze
its confusion Matrix in order to validate its performance and most importantly I'm going to
share some super amazingly Pro tips in order to improve the performance of your model if something
is not working as expected if you are expecting everything to be super super smooth and super
straightforward then think again because when you are training an image classifier in real life or
any type of machine learning model things happen and whatever thing happens whatever problem
it arises with your model with your classifier you need to provide a solution you need to fix the
problem and in order to fix the problem you need to be creative you need to think outside the box
exactly as we are doing over here so this is what I want you to take from this video this is by far
the most important part of this tutorial my name is Felipe welcome to my channel welcome
to this video and now let's get started and now let's get started with this tutorial this
exactly the data that we are going to be using today we are going to work with a pneumonia
dataset and we will be classifying chest x-ray images into one of these two categories we are
going to be classifying absolutely every single image either as pneumonia or normal these are the
two categories of today's tutorial and now let me show you a few examples of each one of these
categories so you see exactly how they look like these are a few examples of the normal category
and you can see that we are looking at some chest x-ray images and this is exactly these are a few
examples of the normal category now let me show you a few examples of the pneumonia category you
can see that in this case we are also looking at some chest x-ray images and I don't know about you
but in my case I have to be honest and these two categories they look pretty much the same right I
am not a doctor and I have absolutely no medical background whatsoever so these two categories
and all of these images they look pretty much the same for me and that's exactly why we are
going to be using an image classifier in order to classify all of our images into one of these
two categories that's exactly why we're going to be using an image classifier to tell these two
categories apart so this is exactly a date that we are going to be using in today's tutorial and
I'm not going to show you the process of building this image classifier because this is something
I have already built before star starting with this tutorial I built the Evers classifier using
teachable machine using this website and if you're curious to know the process of building an image
classifier using digital machine then I invite you to take a look at all of my previous videos where
I show you the entire process of building this image classifier so if you're curious to know how
exactly I built the image classifier we are going to use today please take a look at that video I'm
going to be posting somewhere over there now let's continue this is a classifier we are going to use
today this is a classifier I have already trained pneumonia classifier.h5 it's already trained it's
ready and we're just going to use it today and now let's continue let's go to pycharm because let
me show you the python project we are going to work on to this tutorial so you can see that
they have three files over here one of them is requirements.txt the other one is main.pa and the
other one is udl.pi so these are the three files we are going to use today requirements.txt is the
list of requirements you need for this project and you can see we have numpy streamlit Pillow colors
and tensorflow so please remember to install these requirements before starting with this tutorial
otherwise nothing is going to work and then if we move to main.pi you can see that this is
the entire process we are going to be cooling onto this tutorial and this is exactly everything
we're going to be doing today this is exactly what it takes to make this image classifier available
through a web application using streamlit so the first step will be setting up the title then this
the header then we are going to create a widget in order to upload a file then we're going to load
the classifier and so on right the entire process is described as a sequence of steps right it's one
two three four five six seven eight steps that's all it takes to complete this tutorial right
that's how Sim simple this will be so this is main.pi and now let's move to udil.pi you can see
that here we have two functions one of them is set background and the other one is classified for set
background this is a function we are going to use in order to change the background of the website
we are going to be building today so this function is only a detail like a very meaningless detail in
order to make everything look much nicer and much prettier but it's not really that important
because it's not going to affect the entire functionality we are going to be defining here
in this process so that's why I thought it was a much better idea to just use this function from
a utils file so we don't really lose time code in this function and we can just use it at the end
of this tutorial at the end of this process now let me show you this or function which is classify
and this is a very very important function and you can see we have a very comprehensive description
here in this doc string you can see we have the this function exactly what it does and we have
all the description for all these parameters but then if I scroll down you can see this function
is completely and absolutely empty right the only thing we have is these comments over here these
inline comments and this is also like a sequence of steps the same way we had over here so the idea
is that we are going to be good in this function together on this tutorial but we are going to do
it at the end of everything else we're going to focus on the entire Pipeline and once everything
is ready and once we are happy with this pipeline then we're going to get back here to the util.pi
file and then we are going to call this function and we're going to make the real classification
the actual classification right but that's the process we are going to follow in this tutorial
we are going to start here with this process and then we are going to classify an animals we're
going to build the code for this classification and then at the end we're just going to change
the background because that's going to be like the least important of all the details we are
going to be doing today so that's exactly what we are going to cover on this tutorial and now
let's get started with the main.pi file okay and the first thing we need to do is to import
streamlit as St that's very very important because this is a python package we are going to use in
order to make this process available through a web application using streamlit so streamlit is
definitely important and then the first step is to set up the title and this is exactly how we're
going to do we're going to call St dot title and we're just going to set whatever title we want
I'm just going to write something like pneumonia classification right pneumonia classification
everything is okay yeah so this is going to be the title of our web application and now let's
set the header and this is something we're going to do call in St dot header and here is going to
be something like please upload a picture please upload an image of a chest x-ray something like
that or maybe we can rephrase it on something like please upload a chest x-ray right this is
going to be the header of our web application so we're just instructing the user to upload a
chest x-ray image in order to continue in order to classify this image right now let's continue
now we are going to create the widget we need in order to upload a file and this is something
we are going to do call in St file uploader we're going to input two parameters one of them
is going to be an empty string completely empty and then we are going to specify what are the
types we are going to accept in this widget which I'm just going to do something like jpeg jpg and
PNG right which are some very very popular formats for images so I think we're going to be just fine
with these three formats then we need to load the classifier we're going to use in order to make
our predictions and remember I already show you the classifier we're going to use today which
is this one I already show you the model the weights of the classifier we're going to be using
today which is pneumoniaclassifier.h5 and this is where it's located within this small directory so
that's exactly what we are going to load now but first we need to make an import we're going to
save from keras.models import load model and now I'm just going to call load model and I'm going
to specify the location of my model which is model and then um pneumonia classifier.h5 and this
is my model and that's it that's how simple it is and now let's load the class names and in
order to do so we are going to use this file we have over here within the moles directory which
is called labels.txt and if you're curious to know where this file comes from please remember
to watch my previous video on how to train an image classifier using digital machine because
in that video I explained the entire process I explained the output I explained absolutely
everything so by watching that video you're going to understand where these labels.txt
file comes from now going back to pycharm this is exactly how we're going to load this
file we're going to do something like with open and this is going to be model
labels.txt we're going to open this as r as if and then we're going to
say something like class names is equal to then we're going to do something
like we're going to for a for a in F read lines and obviously we are not going to mine the last
character because that's a new line character and that's basically all but if I show you this
file again you can see that this is exactly how or how this file looks like we have a zero and
pneumonia and we have a one a normal right so now the only thing we care about are the class
names so now the only thing we care about are pneumonia and normal so the way I'm going to do
it I'm going to do it in a very very generic way so I'm going to do like this this value dot split the space right remember we have a space between
the number and the word on the class name and then I'm just going to keep the first value right so
this is exactly how we are going to get all the class names from this file it doesn't matter how
many they are this is exactly how we're going to get all the class names and that's going to be
pretty much all in order to make sure everything works properly I'm just going to print class
names and let's see what happens I'm going to execute this file and you can see that these are
the two categories we are printing pneumonia and normal so everything seems to be just fine and now
something we could do to start working on this web application to see how it looks like we can just
run this file and we should be looking at these two Fields right the title the header and also the
file uploader so let's see how everything looks like so far I'm going to my terminal I'm going to
clear this and I'm just going to call streamlit and then I'm going to call Ron main.pi this
is exactly how we are going to execute the code we have here and exactly how we're going to
launch this web application this exactly the text we have so far which is so please upload a chest
x-ray image and then this widget we have defined over here as the file uploader so so far so good
so far everything looks just fine and now let's continue and now you can see that the next step
in this process is to display the image the user has selected right so this is how we are going
to do it you can see that we are calling this value and we need to say this is equal to file
and then we're going to say if file is not known we are going to do something like this we
are going to find image as image dot open file right an image we are going to import
them from peel from peel import image this is a function we're going to use in order to
read the image the user has selected and this is very important we need to convert this image
into RGB now let's continue now the next step is to visualize this image is to display this image
to the user and we're going to call St dot emerge we're going to input this image and then we are
going to call this parameter which is use column width equal to True right this is exactly what
we need to do in order to visualize this image in order to display this image I in order to move one
step by time let's see how everything looks like so far and if we go back to the browser you can
see that if I refresh we are going to see exactly the same pneumonia classification please upload
a chest x-ray image and instead of selecting a file over there I'm just going to drop I'm just
going to select and drag one of the images in my computer so this is exactly what I'm going to do
and it is going to take a few seconds now it's reading the image it's processing everything and
now we are displaying this image in the browser and the user is now super super happy because
it's visualizing the image it has just chosen from its file system so everything is okay so far
now let's go back to pycharm because now we need to follow additional steps now is the time we are
going to make our classification we are not going to make the real classification but we are just
going to call this very empty function in order to get these dummy values and in order to just move
on right so what I'm going to do is from YouTube import classify and I'm just going to call this
function I'm going to call classify I'm going to input the image I'm going to input the model
and then I'm going to input the class names and this is going to be equal to if I go back here
this is going to return a tuple of the predicted class name and the confidence score for that
prediction so this is going to be equal to I had an S missing over here this is going to be class
name and then conf score equal to classify image model class names exactly what we need to specify
according to this documentation we have over here and remember we are doing nothing everything is
fully and absolutely empty we're just going to get these two zeros but it doesn't matter let's
just take these two zeros and let's move on now I'm going back here to main.pi and now let's just
write the classification back to the user so let's continue on something that's very important
we need to make this within this if always we are going to have issues and now we are going to
call St grind and we're going to do something like we are going to control the size of the text
we are going to print by adding some um these values right these characters by adding these
characters we are going to control the size of the text we are going to write here this is a very
similar syntax I think it have if I'm not mistaken as in the readme file of a GitHub repository so
this is where we are going to say something like uh we're just going to print the class name so
this is going to be format class name this is where we're going to write the category we are
classifying and then we're we're also going to write the confidence value right so we are going
to make this a little smaller so we are going to add an additional character and then we are just
going to say something like score and this is going to be the conf score right so this is this
exactly what we need to do in order to write the classification and so far we are returning a zero
zero but actually if this is a class name maybe it makes more sense to return something which looks
like a class name we're just going to return dummy class name and a zero right so we are detecting
the class name called dummy class name and with a confidence value of zero right it doesn't make
any sense whatsoever this is only for debugging and in order to make sure everything is working
properly so now let's go back to the browser and let's see what happens so I'm going to refresh
I'm going to load the same image again and now we should be getting no we should be displaying
the image and we should also be visualizing this new output which is dummy class name score zero
so everything works properly so far and if you know this we have completed the web application
the web application is completed I mean the the entire structure of the web application is just
completed and the only thing we need to do now is to code the classify function we have over here
but this process is done it's completed we have completed this web application this is exactly why
I split I divided the process into these two files because this way is going to be much simpler to
just focus on everything in the main.pi file and then just make the classify function right we are
just organizing the code this is a very very good practice and I don't know about you but I'm super
happy because this is done this is completed I do notice how C simple this is right so now let's
continue now we are going to code this function which is the classify function and you can see
that we also have a sequence of one two three four five steps the first one is to convert the image
we are getting over here to this size to a 224 times 224 so this is exactly how we're going to
do it we need to make an import which is from peel import image Ops because this is the function
we're going to use in order to convert the image we're getting into this size so we're
going to call Image Ops and we are going to input sorry we are going to call Image Ops
dot fit and we are going to input the image we are getting and we are also going to input
the size which is 224 times 224 and then we are going to input a constant which is image dot
resampling dot Lan c c the OS right this is only a constant a parameter we need to input into this
function but the long story short remember we are just converting the size we're resizing the
image we are getting into this size 224 times 224 and obviously we need to make another import
which is image otherwise it is not going to work and that's pretty much all this is how we
are converting the image and the output is also called image we're just going to overgrade
the image we are getting and now let's continue now the next step is to convert image as to an
ampere right so I'm just going to Define your variable which is image RI and this is going
to be numpy dot as Ry e mesh right so we are getting the image which currently is a preload
object right it's an image but in the pillow format and we're just converting this image into
an ampere right right and we are using the numpy library which obviously I need to import important
MP SMP okay and this is pretty much all in order to complete the second step in this process which
is convert the image to one on PRI and now we need to normalize this image so what we need to do I'm
going to define a new variable which is normalize and this is equal to we are going to take
the immature right we are going to cast it us type MP dot float32 we're going to cast it
first into this into this value we are going to take the image and we are going to convert it
as floats flow 32 and then we're going to divide it by 127.5 right remember so far the image goes
from every single Pixel goes from 0 to 255 so if we divide it we are making absolutely every single
Pixel goes from 0 to 2 right so far every single Pixel goes from 0 to 2 but if we lose this if
we subtract one now every single Pixel goes from -1 to 1 right we have normalized or image all
right and now let's continue now we are going to set the model input and in order to do so
we are going to define a new variable which is called data and this is going to be MP ndri and
this is going to be a shape equal to 1 because we are going to input one image at a time then the
size will be to 24 to 224 224 and then we will have three channels for every single image right
so this is exactly a size we need an image only one image 224 times 224 and three channels and
the type this is very important is numpy float32 okay the same type we have
converter image over here and that's pretty much all and now the only thing
we need to do is to Define that the first element and the only element of data is the normalized
image array we have over here and that's pretty much all we need to do in order to set the model
input the data we are going to input the model and now we have only one step left which is making
our prediction let's see how we can do that we take the more less input is one of our
parameters the model so the only thing we're going to do is calling model and we're just
going to input we're going to call mole.predict and we're going to input the data we have defined
over here and this is our prediction right but now we need to unwrap all the information we need from
this prediction right we need to make we need to get the class number we are predicting and also
the class name and also the confidence value and so on now let's define a new variable which is
index and this is going to be the index of the class we are detecting and we are going to make
it like this and later on on this tutorial I'm going to show you another way to get the index of
the class we are detecting I'm going to show you a very Pro way to do it I'm going to show you a
much better way to do it as what we're going to do now right from now let's just do it in a
very very classical way which is ARG Max and we are just going to input prediction right this
way we are just going to get the argument of the maximum value of prediction prediction is going
to be a vector two values and we're just going to get the argument the index of the maximum number
and that's going to be the index of the class we are detecting so from now on class name is going
to be class names and we are going to input the index remember class names is an old parameter
we take as input here and then confuse core is just getting the prediction zero because we
only have one element and then e Index right so this is exactly what we need to do in
order to get a class name and in order to get the confidence score of the prediction we
are getting over here and that's pretty much all and now the only thing we need to do is to replace
this value and to replace this or value and that's all and that's it so you can see how simple
making this classify function is and now you can see why I have divided the entire code into
these two files right because the first step is to complete the process in main.pi which was very
very very simple very straightforward and then it's just to code these few lines in the classify
function of the util.pi file right and in only a few minutes we have a an entire pipeline which
is up and running and we are just serving this image classifier into a web application using
streamlit and that's it right so now let's test how everything works let's see what happens and
now let's go back to the browser I am going to refresh and I'm going to choose a couple of images
open the set to see how it performs so I'm going back to my file system and I am going back until
my test said because obviously I'm going to choose a couple of images which are completely unseen
data from my model so we know exactly how the mole performs I'm just going to drag and drop
an image in the pneumonia Theory and let's see what happens so we can see that we are detecting
pneumonia we are classifying ammonia with a 99.9 confidence so this is a an amazing performance
this is amazing so so far the model is performing super super super well now let's see what happens
with an image in the normal category so let's go back here I'm going to normal and I'm just going
to select an image from the normal theory of my test set I am going to drag and drop the image
here and I'm going to wait a couple of seconds and you can see that now we are also detecting
pneumonia we are classifying pneumonia too and with a super amazingly high confidence you can see
that the confidence value of this classification of this prediction is 99.4 percent which is
amazingly high so something is going on obviously something is going on and now let me show you
something I'm going to show you the confusion Matrix of this classifier this is the confusion
Matrix I am sure you are familiar with the confusion Matrix remember how it works in the why
the axis we have the true labels and in the x-axis we have the predicted labels and for example this
number over here 384 means that 384 images in the pneumonia category were classified correctly as
pneumonia and only six images in the pneumonia category were classified incorrectly as normal
but if we look at the normal category you can see that we have these numbers over here and it means
that only 70 mm images of the normal category were classified correctly as normal and 164 images of
their normal category were classified incorrectly as pneumonia so something is going on with
this classifier and I have been expecting many many different images and I have been inspecting
this a confusion Matrix and so on and I realized I noticed that this image classifier is super
super biased to the pneumonia category so long story short it's most likely for for a given
image is most likely to predict pneumonia than normal so pretty much is classifying pretty much
all images or most of the images as pneumonia and is not very it's not performing very well for the
normal category right if we look at one category at the time you can see that the accuracy in the
pneumonia category is something like a 98 right something like a 98 percent but if we look at the
normal category it's only a 30 something percent or around a 30 percent so that's what's going on
right and I think this is very interesting because this is something that sometimes happens in real
life when you are training them all sometimes it happens that you have a bias and the classifier
it's classifying one of the categories which are much more likelihood that the other category and
this is something that sometimes happens when you are building a mold is something that sometimes
happens in the industry and sometimes it's not very clear why this is going on sometimes it's not
clear why this is happening remember that we are predicting images which are chest x-rays and as I
mentioned when I was starting with this tutorial when I look at these images they all look pretty
much the same because I have absolutely no medical background whatsoever so it's very challenging
for me to know what's going on with these images to know what's going on with this data maybe if
I will have some medical training or something I that maybe it will be a little more clearer for
me but it doesn't matter what I want you to take from here is that this is something that sometimes
happens when you are training a mall and sometimes it may not be very clear the reason why it is
going on if you ask me I'm from the top of my head I will say that one of the issues or something
that may be going on with this data with this classifier is that the pneumonia category maybe
all the images are more similar between them maybe don't we don't really have a lot of diversity and
all the images which were classified as pneumonia they are like very similar between them but for
the normal category the normal category I guess that maybe we have a lot of diversity in this
category because Normal in this context means that it's not pneumonia but the patient could be
completely and absolutely healthy or it could be sick but with a different condition right so I
guess we have a lot of diversity in this category and maybe that's the reason why the classifier is
not learning this class properly maybe that's the reason I know that's from the top of my head but
remember this is something that sometimes happens when your training mode and sometimes it may not
be very clear the reason why this is going on but we are going to see how we can fix this issue
obviously we are going to fix this issue I'm not going to give you a bias classifier I will never
do something like that so I'm going to give you a solution it may not be the best solution ever
but it's a good solution I would say it's an okay solution I'm going to show you exactly what we are
going to do the way we are getting the prediction we are currently getting the prediction is the
way it usually goes right the way we usually do it which is just getting the maximum number from
the predictions Vector right we are getting two numbers one of the numbers is the confidence value
in the pneumonia category and the order number is the confidence value in the normal category and
we are just getting the index of the maximum of these two numbers right this is what we are doing
over here and this is the way it usually goes when we are we are getting a prediction but this is
not going to work in this case because this is going to give us a very very biased classifier
what we are going to do instead is this we are going to classify as pneumonia which is the
index 0 if the confidence value is greater then 0.95 and in any other case we are going to
classify as one this is going to be for index equal to 0 right so we're going to classify as
pneumonia if the confidence value of getting pneumonia is greater than 0.95 and in any other
case we are going to classify as normal and the reason for this is that I have been inspecting
many many many classifications many many many predictions and I noticed that for all of those
cases or for most of those cases for which we are classifying pneumonia we are getting a super
super high confidence value we are classifying pneumonia with a super super high confidence but
I noticed that in many of the cases in which we are classifying as pneumonia an image which
was labeled as normal in many of those cases the confidence value for classifier is not really
that high it's not as super super convenient value as it usually happens when we are classifying as
pneumonia something that it's pneumonia you see what I mean and you notice that in many cases for
the misclassifications over here for many of these Miss classifications many of these 164 images the
confidence value of our classifier wasn't really that high right not in all cases because we also
have cases like this where we are classifying as pneumonia something that's normal and with a very
very high confidence value this this is happening too but in many other cases the confidence value
is not really that high so if we make this change we are going to be classifying as pneumonia only
those cases those images for which the classifier is super super comfy in this pneumonia and
in any other case we are just going to say is normal this is a very very a typical way to
make a prediction this is a very a typical way to make a classification right the way it usually
goes you should take the maximum number of this prediction vector and that's it but now we are
doing something that's very very very atypical but let me show you what happens if we make this
change let me show you the new confusion Matrix if we do this change and this is how it looks
like you can see that for the pneumonia category we are getting a very very very high accuracy is
not really that high as it used to be right now we are getting more mistakes we are getting more
misclassifications but nevertheless this is a very good accuracy in total we have something like 390
images which were labeled as pneumonia and 353 of those images are classified as pneumonia are still
classified as pneumonia so this is something like 90 percent of the entire data of the entire data
which was labeled as pneumonia this is Allah this is a very very high accuracy but now if we look at
the normal category you can see it's not perfect this is not as good as the pneumonica theory we
are still getting many many misclassifications but nevertheless this looks much much better
than before in the older confusion Matrix in the other one I showed you a few minutes ago we had
something like a 30 percent accuracy in the normal category right and now you can see we are getting
something like a 60 percent if we make this sum 162 plus 72 and now I divide 162 divided
234 we are getting a 69 accuracy so this is much higher this is almost a 70 accuracy so it's
not as high as the accuracy we get here from the for the pneumonica theory but it's very high and
effortless so this is a much better Improvement and you can see that by doing this very very
small change and now we are getting a much much better classifier and we're getting a much much
more robust classifier so this is definitely a very good Improvement and this is what I want you
to take from this video this is by far the most important part of this tutorial sometimes when
you are training an image classifier when you are training a machine learning model sometimes
things happen and the way usually goes things happen if you're expecting everything to be super
super smooth and super straightforward then think again because when you are training an English
classifier in real life or any type of machine learning model things happen and whatever thing
happens whatever problem it arises with your model with your classifier you need to provide
a solution you need to fix the problem and in order to fix the problem you need to be creative
you need to think outside the box exactly as we are doing over here so this is what I want you
to take from this video this is by far the most important part of this tutorial so this is the
change we are going to make in order to improve this classifier in order to make this classifier
way more robust and that's pretty much all and that's speed that's that's all so the only thing
we need to do now before I forget is to change the background right we are going to change the
background for something a little more nicer for something prettier and let's do it before I
forget because this is such a detail and this is such a meaningless detail that I almost forget
so let's just import set background and now let me show you the images I have in my local computer
which we are going to use in order to change this background I have a directory which is called
bgs and if I open this directory you can see I have many different many different images these
are some images I have been testing When I Was preparing this tutorial and let's just use this
one which is bg5.png so let's go back to pycharm and I'm just going to say set background
and I'm going to specify the path to this image which is bg5.png and that's pretty
much all now I'm just going to refresh and you can see that now it looks much much better
it looks much nicer it looks much much prettier I'm just going to test it again let's try with
another normal image maybe some something like this we should be getting something like a 70
accuracy so we should be getting this image classify correctly as normal with a 93 percent
accuracy so you can see that everything looks pretty pretty well and that's going to be all for
this tutorial my name is Felipe I'm a computer vision engineer and these are exactly the type
of videos and the type of tutorials I make in this channel so if you enjoyed this video I invite
you to click delay button and I also invite you to subscribe to my channel this is going to be all
for today and see you on my next video now I'm going to try to login into this system but I'm not going to log in with my own face but I'm going to hold my picture to the camera let's see what happens I press login and I get a welcome Felipe so you can see I am logging in into this face attendance system holding a picture of me there are many cases where people are trying to spoof these systems and they are trying to do something like I'm just doing right now right they are trying to hold a picture of someone else and they are just pretending to be someone else this is a very very very important problem in the industry in the computer vision industry and there is a lot of research involved and there is a lot of resources assigned in order to solve this problem in order to come up with different solutions in this case it's not really a problem because in this case it's me holding a picture of me but you get the idea right in many other situations there will be other people holding pictures of other people so this is definitely something we need to solve so what we are going to do today is that we are going to use this software this face attendance system but we are going to integrate it with this spoofing detectors so we are only going to log in or log out our users if they are really users if they are people if they are like people in front of a camera and we are going to take this type of situations of someone just holding someone else picture and we are going to deny the login or the logout if the user is doing something like that so let me show you the three repositories we are going to use in today's tutorial the first repository we are going to use today and actually the most important repository is this one which is one of my previous projects an easy project regarding the face attendance system I show you how to make in one of my previous videos and this is very important because absolutely everything we are going to do today into this tutorial we are going to build it on top of this previous project so this repository is super super important and is one of their posters we are going to use today then I mentioned we were going to work with spoofing detection we are going to detect smooth things and this is exactly the repository we are going to use in order to detect or spoofers it's called silent face anti-spoofing and this is exactly the repository we are going to use you may notice this is Chinese if I'm not mistaken everything is written in Chinese and I'm not sure about you guys but I I definitely don't speak Chinese and if any of you don't speak Chinese either don't worry because we have another rhythmic file which is reading incomplete English absolutely everything is written in English so this is another repository we are going to use today and then the third repository which is also very very important is exactly the same repository we used in my previous project right we are going to use the same face recognition Library we used in my previous project so we are really familiar with this repository but I am just showing you that this is another posture we are going to use today so the in order to get started with today's tutorial the first thing I want to mention is that I have noticed many of you have had some issues have had some challenges when you were trying to execute this a project right if I'm not mistaken it was basically many of you Windows users were having some issues we're having some challenges so I have been doing a lot of research so I can see exactly how I can help you all of you in order to solve this problem and I have come up with a very good solution so let me show you the first thing all of you Windows users will need to do in order to in order to work on today's tutorial in order to execute the face attendance system I showed you in my previous video is to follow a few steps and the first one is described in this video I am going to post a link to this video in the description of the video you're currently watching and going to post a link to this video so you know exactly what you have to do in order to work on today's tutorial in order to work with the face recognition Library some of the issues you guys were having were due to the face recognition libraries and some issues or some incompatibilities which is related to Windows so this is exactly the first step you should follow you should take in order to in order to solve the issue then another thing which is going to help you a lot is to use a different set of requirements you may notice that now in the repository of this project we have two files two requirements files one of them is called requirements txt and the other one is called requirements windows.txt and this is exactly the requirements file all of you Windows users will need to install right this is a very specific set of requirements which is exactly the requirements you will need to install in order to work on today's tutorial and now let's start working on today's tutorial and the first thing we will need to do is to clone this repository let's clone the face attendance system repository so I'm going to copy this repository URL and I'm going to pycharm this is a python project I created for today's tutorial these are completely and absolutely empty project you can see that the only thing it's in this project is a virtual environment that's the only thing and it's a completely completely empty virtual environment so I am going to the terminal of this python project and I'm going to say something like it clone under repositories URL so the repository is cloned and then if I go here I should be able to see it here yes okay so the first thing I'm going to do is to install the requirements for this project remember if you are running this in a Windows computer please remember to install the windows requirements right but in our case I am running this in Linux so I'm going to PP install the requirements file which is in face attendance system requirements oh sorry that's not going to work because I forgot the txt okay so this is going to take a couple of minutes and in the meanwhile let me show you a few edits I have done to this code and I invite you to take a look at each one of these commits so you can see exactly what I have changed in this project but it's basically a few changes regarding how the face recognition Library it's executed and also the an additional functionality I have a little this repository so this is for example some changes in how the face recognition is executed and I I invite you to take a look at it and invite you to become familiar with how to read a commit basically you can see that everything that's new is green and everything that has been removed is red and this is not really that hard but it takes some time to become more familiar how to read a commit so I invite you to take a look at it in the repository of this project and also uh yeah I invite you to take a look at absolutely all the changes I have done now let's go back to pycharm let's see if it's completed everything is completed so we can start working on these projects so I'm going to Main and the only thing I'm going to do is to execute this file and let's see what happens so I'm executing this file I have already connected my webcam and you can see it's taking my webcam so let me show you how it works let me remind you how it works the idea is that each new user is going to register so for example in this case I'm going to register into the system so I'm going to look at the camera I'm going to try to smile and see what happens okay yeah that's okay that's a good picture let's say we can use this this picture my name is Felipe so I'm going to accept and then I am going to try to login into this system so I'm just going to press the login button and I have been testing this face recognition Library I have been testing this software and I am amazed with how well it performs for example in this case I'm not looking at the camera the camera is right here and I'm not looking at the camera and take a look what happens if I try to log in even if I'm not looking at the camera I am recognized with a absolutely no problem so the face recognition library is super super robust and that's very important now I'm going to continue testing the other feature which is the log out button and if I just press this button I am not going to look at the camera I'm just going to look straight to the computer and I press logout and I get with by Felipe so everything seems to be working well in order to show you like a more comprehensive test of how it works remember that in my previous video I show you a few tests uh logging in a picture of Rolo and a picture of capital Picard now I have other pictures which I'm also going to show you which I'm also going to use in order to show you how this works so in order to remember how this works so I'm going to register more views into this system you can see it is a picture of Morpheus I'm going to Press Register new user let me uh hold it well something like this register new user input a user name and this will be more fuse except a user was resistor successfully and I'm going to do the same with data I'm going to use a picture of new tenant commandant data if I'm not mistaken so I'm going to register an user username will be data and I'm going to accept user was resistor successfully and now let's see what happens if data tries to log in into the system we get welcome data so everything works properly and now let's try to do the same with more views I'm going to Press login and welcome more views everything is okay and I have also a picture of me this is a picture of me so this is me holding a picture of me now I'm going to try to login into this system but I'm not going to log in with my own face but I'm going to hold my picture to the camera let's see what happens I press login and again welcome Felipe so you can see I am logging in into this face attendance system holding a picture of me and this is exactly why we are going to use this spoofing detector because this is a very well known problem in the face recognition industry because once we are developing software in order to recognize a person's face or in order to recognize a face in a picture there are many cases where people are trying to spoof these systems and they are trying to do something like I'm just doing right now right they are trying to hold a picture of someone else and they are just pretending to be someone else this is a very very very important problem in the industry in the computer vision industry and there is a lot of research involved and there is a lot of resources assigned in order to solve this problem in order to come up with different solutions in this case it's not really a problem because in this case it's me holding a picture of me but you get the idea right in many other situations there will be other people holding pictures of other people so this is definitely something we need to solve so what we are going to do today is that we are going to use this software this face attendance system but we are going to integrate it with this uh spoofing detector so we are only you going to login or log out or users if they are really users if they are people if they are like people in front of a camera and we are going to take these type of situations of someone just holding someone else picture and we are going to deny the login or the logout if the user is doing something like that so in order to continue with this tutorial what I'm going to do first is I'm going to show you how this spoffy detector works so the next step will be to clone this repository and I'm just going to show you how it works for now right I'm just going to clone it in a random location in a random directory I'm going to close it in my desktop because for now the only thing I'm going to do is I'm going to show you how it works and then in the next step I'm going to show you how to integrate this spoofing detector into the face attendance system so the first thing I'm going to do is to create a virtual environment remember that the only thing you're doing now is to test this repositories I can show you how it works and then we are going to integrate it into the face attendance so forth so I'm going to call virtual M bam I'm going to use pylon 3.8 and then I'm going to activate this virtual environment so now what I need to do is to install these requirements but I have been doing some tests already and it's going to be a much better idea to change some of the versions of these packages so I'm going to install the same exactly the same packages but with different versions and I'm going to post exactly what are the packages I am using obviously but this is only to let you know that I'm doing a few changes regarding the um the requirements which are in this repository so I'm going to PP install this file and then I'm just going to wait let's see what happens PV install minus r requirements I have to wait a few seconds but this will be enough in order to install all these requirements and something I have to say is that I have been doing many tests of this repository of this spoofing detector and I would say it works very very very very well and we can definitely use it in our project but I can also say that there are some situations where I tested this algorithm this detector and it's um it's not perfect so we are going to use a spoofing detector which works very well but it's not 100 perfect that's something that you should be aware of and I'm going to say it's not perfect it means that in some of the images I have used this repository this project is not really detecting the spool things and some pictures which are actually real pictures it's misclassifying them as spoofs so the but nevertheless I think it's a very very good repository and we can perfectly use it for now and this is going to be a very very good example because when you are creating a project when you are building a solution you definitely want to take a look at all the available Solutions and everything you can just use out of the box in order to build your product in order to build your solution but some of these Solutions may not be 100 fit to your specific use case or to your specific needs so this is a very good example that when you are building a solution you definitely want to use all the available resources you can find but you can just use it like a way to start working on that project or to start working on your solution right if we will want to make something that's like a 100 accurate spoofing detector for the specific images or the specific use case or the specific environment we are going to use this system in what we will have to do what we will need to do ideally is to collect datam to annotate data to train a detector to do something right if we are using something that's publicly available it means it's a very good starting point it means that's going to be a very good solution in order to start working on this project right so that's something we should be aware of the delivery we are going to use the project we are going to use it's very it's functional and it works very well but there are some cases in which we are going to have some Mis detections so let's continue other repositories where all the packages were successfully installed and what I'm going to do now is to test how it performs and I'm going to use one of the images in their examples right they provide a few examples you can see for example in this case it's someone called in a picture in this or case it's someone holding a cell phone with a with a purse on it and in this case it's a real person so these are a few examples which they provide and what I'm going to do is just to use these pictures in order to show you how to execute it the idea is that we are going to call the test script we are going to call this script and we are going to just run it and what this script does is to call anore function which is a function which is also called test and the only thing we need to provide for this script is the image we want to inference the image we are going to we want to attack if it's a spoofing or not and also we want to detect the we need to input where the models are located right so let me show you how it works I'm going to input one of the images I show you in these samples so I'm going to test dot pi uh sorry python test.vi and I'm going to input image F1 there's something I'm just going to there are a few changes we need to do obviously but let's see how this is used okay so I'm just going to leave this empty as I mentioned this is a perfect and amazing repository we can use as a starting point but obviously we have to make a few changes if we if we want to fit it into our purpose right so the image name will be the full parts and then image f1.gpg okay let's see how it works with this image I'm just going to run it and it says this is a fake and it also provides a score and this is exactly the output which is actually the same image but with a bounding box on top and it says a fake underscore right so it's basically that's what we we are getting so I'm going to execute it for this one as well because these are real image and we get this enemy C mesh T1 so I am going to replace this by image T1 uh okay and you can see in this case we get real and the score is 0.99 right and then we have an output which is uh exactly the same image but with a bonding box on top so this is exactly how it works and you can see that this is very very easy to use the this is very very straightforward the only thing I did was cloning the repository I install the requirements I had to change the versions of the libraries but that's only like a detail and then in order to execute the test in order to execute the inference of this spoofing detector the only thing I had to do is to execute the test file and now it's the best part of this tutorial because it's when we are going to integrate this buffing detector into our face attendance app so this is how we are going to do we are going to follow a very similar process as we just did when we were testing the spoofing detector but we are going to do it from here and so I am in the in my python project I go into the terminal and I am going to 2 CD into face attendance system okay and now I am going to clone this spoofing detector here so something like git clone okay now we are going to clone the entire repository and I am going to take the requirements file and I'm going to make exactly the same edit as I did here so I'm going to copy this versions and I'm just going to paste it here and now obviously we need to do something like pip install minus r and this requirements file which is here okay so requirements are now installed and now let's go to test because I am going to use exactly the same script as before but now I am going to use it as a function I'm going to use it I'm going to import this function from my main script so we are going to use this test file and I'm going to do a few changes into this file but the first thing I'm going to do is go into settings and then project structure because we need to add this directory into my sources because remember we want to import a directory we want to import a file which is in this directory and otherwise it will be a little hard to do it so that's going to be the easiest way to do it so now I'm going to Main uh let me do something like this in Maine I'm going to say something like from test import test right we are going to import the test function in this files so we are going to import this function and now it's the time where we are going to make a few edits into this function because this is a very good starting point but we need to do a few edits the first thing we are going to edit is that we no longer need to write the output as we were writing it before remember I show you we every time I execute the test file we generated an image with a bonding box on top and the result and so on and we don't need to do that anymore so I'm going to delete all of these also the rectangles and also the print ok and something else I'm going to do is that we don't really need the we are not going to input an image path but we're going to input an image so what I'm going to do is this will be image and this will be something like this right we are not going to read an image from our hard drive but we are going to input an image that's another change and then another thing we need to do we will need to resize all of our images so they comply with this right and we need to do something like image equal to CV2 resize so this size we need for our image will be something like image.shape I'm just going to write it first and I'm going to explain it in a couple of minutes times 3 divided by 4 and then image dot shape zero right and I need to cast this as an integer or what is not going to work and this is a requirement in order to execute this script absolutely all the images we are going to inference absolutely all the images we want to know if they are spoof or not we need them to comply with this image size right when in this relationship regarding the width and the height of our images this is related to how this model was trained that's what that's what I think and this is exactly what we need to do and this one already changes we need to do and then we need to do over changes if I look at this function that's pretty much all we have already removed the last Parts something we need to do we don't really need this anymore just going to read it and then what I'm going to do is I'm going to return the label right the label information is the one containing if the image is a spoof or not and if I look at this uh if label is 1 means it's a real face and if it's zero means it's a fake okay let's remember those values for now the only thing I'm going to do is to return label and I need to do an additional edit which is in another function let me take a look this is a function we need to edit so this is under source and this is called anti-spoofpredict.pi and basically you you can tell that we have all of these hard-coded values we have all these hard-coded paths and we are going to make an edit which is exactly this one I already prepared the um the sentences we need to in order to replace these two lines and these are the sentences we are going to use and another edit I need to import traceback and that's pretty much all right long story short this is it's never a good practice to hard code anything and especially something like a path to a pad to a file so this is basically what we have done we have made the what used to be something hard-coded into something that's relative to the user's pad location so it will users current directory and this is a very good edit that's pretty much all if I remember if I remember correctly and now let's go back to main because we need to add some additional logic because remember that we only want to log in or log out our users if they are real users so this is where we are going to do so I'm going to take my login function and I'm going to do exactly the same with my logout function and I'm going to do something like if test I need to input three parameters image and let's see what are the other two parameters moldier and device ID so this will be something like this and something like this and the image remember this is the actual image this is the numpy array so I am going to input most recent capture then model directory this is the model directory of my let's see I have so many directories code this is exactly this directory right this directory into inputs let's do like the absolute part okay and then device ID I'm just going to leave it in zero um and that's pretty much all so if this remember is going to Output a number which is zero or one one means that it's a real image and zero means is a fake so let's do it a little better and let's do something like this remember this will be labeled and then I am going to ask if label is one and if it is I'm just going to proceed as usual and if it's not I'm going to print something so I can say something to the user because if it's not one if it's zero means the user is actually ah it's trying to spoof this system it's trying to do something that is he's not supposed to be doing so I am going to Output a message box saying something like hey you are a spoofer and then um I don't know something like you are fake absolutely it doesn't really matter the idea is just that we are going to see this message box every time the label is zero so we are only going to use the data test but you get the idea we're just going to tell the user that he's an imposter that he's a fake so now let's continue now this is pretty much all we have to do I'm going to do exactly the same in the logout function you can notice that we are repeating code and that's never never never a good idea but let's just let's just do it like this for now and we can do it better in the future let's see how this works or if we have to make an additional edit I'm going to run this process again so now I'm going to try to log in I'm going to try to do it with my own face I mean I'm going to do it with my real face and then I'm also going to test with all these different pictures I have over here so I'm going to start uh I'm going to do it first I'm going to be super super still because I noticed that if I move remember that we are taking frames continuously and we are plotting these frames and yeah so it looks like it's a video it looks like it cons it's continuous but remember we are just plotting one frame after the other so if I move it's going to be blurred and the the spoofing detector may think it's a spoof so I'm just going to be super super still as a real person which is like a very good thing to do because I am a real person and now let's see what happens if I try to log in with all these different pictures so this will be a picture of Morpheus and it says you are fake which is uh exactly the case because this is a picture so let's see what happens with my picture of data something like this in the uses Enterprise will have been amazing right so something like this and it says you are fake which is also uh true because this is only a picture and then I'm going to try to log in but with my face uh in this case it's it's completely unimportant because it's just me holding my picture but remember it could be another person which is holding my picture right now so this needs to work super super properly and Express login and it says you're fake so you can see everything works super super purpley and uh yeah so this is going to be pretty much all for today's tutorial this is exactly how you can how you can add a spoofing detector feature into a face attendance system and this is going to
