Tutorial 31- Hypothesis Test, Type 1 Error, Type 2 Error

hello my name is crushed I got my loom to my youtube channel today in this particular video we'll be discussing about hypothesis testing now this is the question that are asked by many of your subscribers and again trust me guys there for the lot of confusion regarding this because this concept is not much clear with respect to most of the learners so I'm taking this particular portion to explain you and this is just not be one video I'm going to explain about hypothesis testing then in the next video we'll be discussing about be values P tests and then what X Z test which is also called as X Co and apart from that I'm also be discussing about choice practice so this also all the types of hypothesis testing which you can see over here t-test and what is Z test drive square test and many more and we'll also be discussing about the p-values in the later classes here I will just be giving you the introduction about hypothesis testing so basic knowledge about p-values and we will also be discussing about type 1 and type 2 error so in the upcoming videos in around 3 to 4 videos will complete this whole thing about hypothesis testing so that you'll not be having any questions furthermore so let us go ahead what exactly is hypothesis testing see this statistics is all about data you know there's a huge amount of data and that data will only be useful if you have work to analyze it if you're going to find out some conclusions from it right so it is very much important and to find out such an important interpretation and conclusion we basically use hypothesis testing now generally generally what is exactly hypothesis testing will discuss about that what is hypothesis testing order different types of hypothesis testing what are the steps in hypothesis testing everything will try to discuss so in hypothesis testing we actually evaluate two mutually exclusive statement and this is just an example we can evaluate three to four mutual statement also for that we usually use ANOVA test okay but just understand in the basic terminology we try to evaluate two mutually exclusive statements on a population using a sample of data okay just try to understand this we evaluate you mutually exclusive statement okay so there are two things either this one can be true or this one can be true okay it may be any statement on operation data you sample data okay and then we finally conclude that this population belongs it is true for a particular statement or the other state just think in this particular way so I may say that I am eating a very good example in the court of law and you say that the defendant is guilty the defendant is innocent so these are two statements right and one of the statement can be to how it will be true and just discuss all the steps and for testing that again this for testing that we need evidence right we need evidence so let us go ahead and take a particular example what are the steps of hypothesis testing so here the first step says that we have to make an initial assumption now if I take an example of you know whether the defendant is guilty or is innocent so let me consider that I'm taking the initial assumption as H 0 and H 0 basically says me it is basically called as null hypothesis okay and this is V oh I can make an assumption saying that the defendant is innocent okay so I am adamantly assigning that particular first statement to this so I'm making an initial assumption then I will start collecting the data now what is this particular data this particular data will not be but it will be just evidences you know now for the criminal cases and it take the fingerprints annotate the DNA prints and lot of things try to find out whether the defendant is guilty or whether he is innocent right so it start collecting the data which is also called as evidences okay now I gather this evidences and then I will try to analyze whether whether we will check this is null hypothesis or whether should I accept the null hypothesis now in this null hypothesis what whether the defendant is innocent or not okay and suppose if I just take an example of court of law if the evidence is against him I will actually reject this and I'll say that right this is just an example but in case of data when we have huge amount of data we will try to analyze that particular data we try to experiment goes to particular data consider the sample data not the population but sample data and then we can actually come to a conclusion all the compilation whole data itself okay so that is what these are very initial six guys this is called a null hypothesis okay so to defend like this you know say okay this becomes a null hypothesis there is also one more hypothesis which is called as a great hypothesis so we indicate it as x1 the h1 will be the opposite of n0 so if the defendant is you know sent over here oh here the defendant will be guilty okay so if we are able to prove this then this becomes true if we are not able to prove this due to the lack of evidences we will actually reject this and accept this side the right hypothesis pretty much simple you know just considering in this particular way just understand this basic concept guys how to do it how to do it mathematically how to do with the help of code and then you explaining in the coming codes but this is very important to understand what is null hypothesis what is the okay this is just an experiment the experiment has two outcomes either one can be do the other can be true and for that you have evidences whichever evidences supports which either hypothesis so either problem one will be actually true and I can be required a lot of evidences for a lot of tests usually there for that okay now you have understood this okay now if I just consider if I just consider this particular stuff and if I just consider that suppose if a null hypothesis is actually true okay I am saying that here the defendant is innocent and saying but I don't have much evidence I know somewhere or the other way this null hypothesis is true but I don't have enough evidence then what will happen this will get rejected right this is null hypothesis will get rejected because I don't have much evidence so instead it there the different will be placed guilty but I know of vino suppose this defendant is innocent and me since would not have much evidence we are treating him as guilty okay and if I go to this confusion matrix I hope you remember if you do over classification any algorithms with respect to class reason you have heard of confusion matrix now even the confusion matrix in the top side you have told in the left side you basically say that do we have to reject this assumption that is your null hypothesis or we should reject or we should not reject okay now in this particular case I am saying that s 0 my dependent is you know same but I do not have you know I do not have evidence because of that this was rejected and this got selected now this scenario will come over here which is basically called as my type 1 error okay do you do the lack of evidence I had to reject the null hypothesis and because of that this is basically an error and because of the time that you can see over here this is naoetsu right so this situation may lead to a lot of effect you know since we know that this person was you know same but due to lack of evidence we had to go over here and this is what I 1 error is actually belonging to we know that this is true positive to negative false positive or negative right I hope you all know this this is completion matrix but what is exactly here let me have a blue body I only have a null hypothesis here we have that current hypothesis here we have should we reject this null hypothesis or what should be and it should be rejected we not reject the null hypothesis ok so pretty much simple pretty much easy to understand whatever type winner let me take an example suppose I say that my null hypothesis says h0 the market is going to crash the market is going to crash the market is not going to crash ok now what will happen suppose I do not have enough evidence now I do not have enough evidence because of this I was not able to prove this market is going to crash or not my isolate hypothesis suppose I say that the market is not going to crash market is not going to crash ok and I collected this evidence I collected various evidence and from that I found out that my h1 became too but suppose due to some of lack of evidence this is becoming through the market is going to crash over there so in this particular scenario I will be reading the type 2 error okay and you can see that with respect to this type 2 error I do not reject that right so this is basically a type 2 error and because of this we so what to or we will not manage to capture whether the market is going to crash on and this is catastrophic types because if we can understand if the market is going to crash it can save a whole lot of money can save many people like right so this type 1 and type 2 error can play a major role in hypothesis testing so whenever you solve the classification problems whenever you solve with the help of any machine learning algorithms this always makes sure that you see this type 1 and type 2 error pretty much simple guys again I'm going to repeat what is exactly type 2 error okay now if you can see that I do not reject this h1 statement which is the truth and considering this market is not going to crash I am going to approve this considering that I do not have much evidence to support this okay then I do not have much evidence to support this all that because there are two going select this but it may happen in such a way that due to lack of evidence in the future this was the original statement over there and because I had I predicted this I came to a conclusion with respect to this right but when I try to combine it with the truth value this became my type 2 error so this import this is very very important despite what I try to to understand okay and you should understand with respect to the confusion matrix in considering the real world Raman state right so this is pretty much super good guys this is what is all the what should you put this is testing okay and it depends whether type 1 error can be more harmful and type 2 error can be more harmful based on the problem statement that you're solving okay and now in my next video I'll be showing you how you can go ahead and put this is testing how to perform hypothesis testing I will be solving a simple math problem statement like how we did in schools in colleges so that you will be able to understand how to solve hypothesis testing with the help of p-values what is P values what is T test and what s dentist now whenever I am saying that I am NOT able to prove this null hypothesis will be basically considered a p-value less than point zero five if the values less than point zero five we reject the null hypothesis go with alternate hypothesis now you may be considering what is this point zero five this is called as significance panel I'll be telling you the importance of this p-value because I have to draw a normal distribution and in that particular curve have to show where this p-value indicates what what the speeds less than or equal to point zero five indicate so I hope you like this particular video please do subscribe my channel if you're not already subscribe and please make sure guys you start watching this playlist every day I'll be coming up with other videos so that you will not be having any questions related to it and this is pretty much important right so as well in the next video have a great day thank you wonder

Channel: Krish Naik

Views: 150,100

Keywords: upgrad, appliedaicourse, hypothesis testing, data science

Id: Y3QzLIBp64M

Length: 11min 45sec (705 seconds)

Published: Mon Jan 06 2020