Confusion Matrix | Machine Learning with MATLAB

hello friends welcome to our channel knowledge amplifier so in my this particular video i am going to discuss one very interesting concept related to validation that is where the model which you have trained using the data which you have how much accurate that is okay one kind of that parameter i am going to discuss today right so all the data which i am going to use in my this particular video will be provided in the description box or in the comment section if you want you can go through that so first clc clearance closure warning of as you generally write clc to clear the command window clear all is used to clear the workspace previously used variables all those things close all is used to close the figure window and warning off is used to suppress all the warnings arriving in the programming part okay now what is the data set let me just discuss about that little bit so here i am reading one particular data set m okay m dot text is present let me just run this particular piece of code and here let me write m okay and see there are basically four columns where one verb to var three fair four okay all four are basically you can see continuous data there is nothing like categorical data present in this case right now what is this what are these values what are the significance of these columns let us try to understand the fair one is basically indicating the time where 2 is basically indicating the x coordinate value okay that 3 is basically indicating y coordinate value and fare 4 is basically indicating amount of pressure at that particular instant so basically this data set is nothing but some features or some data points we can say while writing the character m okay when you are writing any character that time time is required okay at each time what is the x and y coordinate of the point using which you are writing and how much pressure you are giving that is basically loaded in these four variables first one is time next one is my x coordinate value next one is my y coordinate value and last one is the amount of pressure okay so x coordinate y coordinate values are stored in fact two and fair three so if we simply plot vector in x direction factor in y direction we should get the corresponding shape so i am using plot m dot that two comma m dot fat three because you know using dot we can extract the table columns okay and see you can clearly understand this is m to observe in a better way we can give axis equal and let's run this particular code once more and you can clearly see that this is nothing but our english alphabet m right then what we are doing we are reading similar kind of uh data present as text file in the current directory okay now this time it is j okay and we are plotting our j variable also giving access equal in a new figure window okay similarly like that sort i am uh plotting v okay where the handwritten digit v is present what is the time at each instant at those time instant what is the x coordinate y coordinate value and moreover what is that the last feature is how much pressure is appearing while writing and i am plotting the second and third feature which is x coordinate and y coordinate and giving axis equal so these three are my major data let us just run this particular sector code will be posted in the description box or in the comment section no need to worry about the code so you can see this is our english character m this is j see this is the body part and this one is the top circular part present in the character j right and this is our v okay right okay all right up to this i hope it is clear to you now we cannot simply pass these values like the law format for training we need to extract certain features okay a feature is simply a value calculated from a signal that signal can be uh normal one dimensional signal maybe audio signal or something like that sort or that may be images or that may be videos okay whatever right so this step is for feature extraction from this particular signal so obviously if you observe these three uh handwritten characters you can understand there are some important features we can easily extract from this particular data what is that obviously one can be aspect ratio okay height is to with okay that is max value of y minus mean value of y divided by max value of x minus of mean value of x okay that is nothing but height height is what max of y minus mean of y with this nothing but what max of x minus mean of x right so if you take this ratio this is called aspect ratio and aspect ratio for m will be differing from j because j is tall okay right but whereas our m is kind of square shape okay the span along y direction is almost equal to span along x direction but for our this particular character j you can understand this is tall that is in y direction this spread is more compared to spreading x direction right similarly for v it will be different right then one more thing can be considered as feature that how much time it is taken for writing these characters okay the timing required for writing the character m will be differing for uh the timing required for writing other characters like j or v right so obviously as you know we have this timing feature as first column so we can extract this easily so duration for m duration for j duration for v all this sorry yup duration for m duration for j duration for v all these things we can calculate if you just take our sorry m data you can see the timing is starting from zero and it is going to this particular value so duration is nothing but the uh last value because for the first value it is taking always zero because at time instant zero the character uh someone has started writing the character okay so always first l first time value will be zero so the duration is not nothing but going to be the last value for the time column okay so we can extract the end values for the var one that is first column we are extracting the last value okay and this is giving us the duration required for writing the particular characters then what is that i have told you aspect ratio aspect ratio can be interesting feature and what i have told you the formula here i am writing again aspect ratio equal to max of y direction minus min of y direction that is nothing but this will be representing kind of height okay by max of x minus of mean of x okay this is our aspect ratio now this max of y minus mean of y instead of writing like this you can use direct range function present in matlab right and y value is stored in our third column right first column is ah time instance second column is x direction value third uh column is y direction value fourth column is basically the pressure okay so we are calculating aspect ratio like this range of y by range of x similarly for j similarly for v we are extracting so using these two way we are already achieved two very important features for our this particular signal one is basically aspect ratio another one is basically duration okay now we can use these two to train our model okay before going ahead let us first consider one particular data set so this one i have just shown you with respect to different different three data set m j and v okay which is taken here now similarly like this suppose uh from different different people we have uh just uh taken lighting for m j and v variable and we have already extracted the features like aspect ratio duration and all this okay and that is basically stored in my features dot text okay this data also i'll be providing in the description box no need to okay so features equal to read table features dot text let us check how the data looks like actually so if i just run features here so here you can see that this particular features data is having basically three columns where first column is indicating the aspect ratio for each character second one is indicating duration and third one is the what is the corresponding character so we have extracted the aspect ratio and duration first column is aspect ratio second column is duration and third column is basically the character which is written okay you can simply check size of features and it will be getting 470 cross three okay there is 470 data present okay right now this is nothing but now becoming one classification problem right because we have the levels now we can classify and this is not continuous the outcome is not continuous obviously this is classification not regression right before going ahead to train our machine learning model let us check that how the distribution looks okay so the features the input features we are going to plot and we are going to group the features which are belonging to same class okay so we are using group scatter g scatter okay we are plotting aspect ratio in x direction and duration in y direction and features dot fair three this is basically our uh classes okay based on this we are going to group right so let's give here one figure and let's run this particular piece of code so basically you can see that this is our group wise scatter plot for our this 470 different handwritten digits okay which is containing v j and n right now in x direction we are plotting aspect ratio in y direction we are plotting our duration okay one thing is clear i have told you just recall j shape is tall okay in y direction span is much more than span in x direction right so obviously aspect ratio for j is going to be more so you can see from the visualization also we can conclude that for this green data points the aspect ratio is going to very high value and this green data points are actually belonging to the class j okay but here basically our v and j are placed in in a closed area that is overlapping each other these two features so obviously using these two extracted features one is expectation otherwise duration it is difficult to train a good machine learning model because here you can see some overlapping is happening but still we will try to do that okay so here observation from this particular visualization is it is not possible to make a reasonably accurate model that uses these two features to distinguish between between these three letters because we have seen overlapping is there but still let us have a try so first we are going to train our k n model using feed c k n as you know we have to pass the features and what is our outcome okay which is stored that is classes output labels which is stored in our last column fair three okay we are writing this and then here i am taking one small test data set which is again having three column first column is basically our uh aspect ratio second one is duration and third one is actual level so this one we can use for validation okay so what i can do uh here i have i am training my k n model with default uh value of k as you know for matlab the default value of k infinity k n is one that is one nearest neighbor it is going to implement okay right and then here we are predicting that first we are passing the model and then passing the input feature from test data because no need to pass the label that is last column so what i am doing all rows and first two columns i am passing and observe in observation matrix what we are storing we are storing first the actual labels that is the last column of my test data which is storing the actual labels of the test data and what our model is predicting okay so let's run this particular piece of code okay and let's check the observation uh yeah so we have taken up to here let us check our observation so this particular for the observation as you can see first one is storing actual labels and second one is storing predictions okay right so if you see that jj matching mm matching vv matching this one matching this one matching vj mismatch test actual level and prediction mismatch okay no problem here matching here matching here matching here matching most of the things are matching with k value one itself but logically if you think we should not take k equal to one because if you recall our discussion related to k nearest neighbor i have told you if kfl is very small like here k equal to 1 that time it is highly affected by noise or outliers and you can see that this particular data has some outliers right if you check this particular red point is coming actually it seems it is coming under area of this green region right similarly this particular green data point also coming under this blue region okay so basically what is happening that outliers present and to overcome the effect of outlets we need to increase our k value right so let's go through that and here what i am doing to reduce the effect of outlier i am increasing my number of neighbors from one to five okay fits again in features where three number of neighbors five and then again applying prediction and calculating the observation okay right now let's run this particular piece of code so up to this we have computed let us check the observation so j match match v match j match mv this one is mismatching vj this one two mismatch happening okay actually with k equal to one we are getting a lesser amount of misclassification but whereas k equal to five is generating more number of misclassification but no problem logically this should be the value because you can see that there are lots of outliers present right okay this might happen that for this particular tested it is happening like this but generally as outlier is present so it is suggested not to take very less k value okay now let's go to accuracy which is our two days actual discussion okay so accuracy if you check very simple uh just we will compare our prediction with respect to our test data actual level and if match then correct if mismatch then wrong right so is correct we are storing in this particular array we are comparing prediction with our last column of test data which is storing actual labels okay we are converting to string because those are categorical and directly categorical comparison using equal to equal to not possible then we are computing accuracy which is sum of is correct okay divided by total number of data points present in the particular test data and number of uh data points present in the test data is nothing but number of data points presenting is correct right obvious so using this we can calculate accuracy so each correct will be one if prediction and test data actual level match is correct will be zero if test return prediction mismatch right if you want to calculate miss classification rate then just take compliment of is correct and apply the same formula okay right let us run this particular piece of code and let us check the accuracy and this classification so it will be taking some time accuracy is zero point eight miss classification is zero point two as we have seen also that two misclassification eight correct classification our test data is having size 10 so 80 percent accuracy 20 percent misclassification right but this one are giving a single single value okay rather what we can do we can plot confusion matrix what is that let me discuss so here confusion matrix in matlab the syntax is confusion chart okay first input parameter will be y true that is vector which is basically known classes actual levels and next one is prediction what our model is predicting in new figure window i am plotting that confusion chart testdata.3 var 3 is nothing but what correct actual labels of my test data and next one we are passing prediction let's run this particular piece of code and let's check how it is showing so this is our confusion matrix what this one is indicating c j m v similarly in y axis j m v this is true class this is predicted class right see j here also j here also j that means there are four j which are classified properly okay right then here this one is blank this one is black monitoring okay this one is also blank nolan dory coming to this one true class is m predicted class is m that means prediction and true class matched three times for m class but here one is coming true class is m but predicted class is v okay that means actual class is m but prediction our model is telling no the uh actual class for that particular test data is v like that how many scenarios appearing only one case okay this one is indicating c true class v predicted class j this is indicating true class for our one of data point is v but our model is predicting that as j okay this is miss classification this one blank leave it this one true class v predicted class v that is correct classification so correct classification c is stored diagonally if you observe j j m m v v that is true class predicted class same and here if you see miss classification is this one and this one okay right two miss classification i hope you have understood this this is one very interesting thing confusion chart which is basically generates confusion matrix and what is confusion matrix i hope it is clear to you this is all for my this video i will continue this kind of discussion in my upcoming videos till then stay tuned happy machine learning thank you

Channel: Knowledge Amplifier

Views: 1,868

Rating: 5 out of 5

Keywords: Confusion Matrix, Machine Learning using MATLAB, MATLAB, Machine Learning, Accuracy MATLAB, ML, Data Science, K Nearest Neighbor MATLAB, KNN

Id: eYSJGItwa-w

Length: 20min 23sec (1223 seconds)

Published: Sun Feb 07 2021