Statistics 101: Visualizing Type I and Type II Error

hello thank you for watching and welcome to the next video in my series on basic statistics now as usual a few things before we get started number one if you're watching this video because you are struggling in a class right now I want you to stay positive and keep your head up if you're watching this it means you've accomplished quite a bit already you're very smart and talented and you may have just hit a temporary rough patch now I know with the right amount of hard work practice and patience you can get through it I have faith in you many other people around you have faith in you so so should you number two please feel free to follow me here on YouTube on Twitter on Google+ or on LinkedIn that way when I upload a new video you'd know about it and it's always nice to connect with people who watch my videos online life is far too short the world is much too big to waste the opportunity to connect with each other number three if you liked the video please give it a thumbs up share it with classmates or colleagues or put it on a playlist because that does encourage me to keep making them on the flipside if you think there is something I can do better please leave a constructive comment below the video and that will try to take those ideas into account when I make new ones for you and finally just keep in mind that these videos are meant for individuals who are relatively new to stats so I'm just going over basic concepts and I will be doing so in a very slow deliberate manner not only do I want you to understand what's going on but also why and how to apply it so all that being said let's go ahead and get started so this video is the next in our series on hypothesis formulation and eventually hypothesis testing so in the videos leading up to this one we talked about the concept of type 1 and type 2 errors and that can seem like a very confusing topic and for many students it is so in this video what I'm going to try to do is make it visual I'm going to show you some graphs so you can visualize what type 1 and type 2 errors are and actually sort of what's going on behind the scenes when we talk about type 1 and type 2 errors so let's go ahead and dive right in so remember that we have the standard normal curve and you should probably know that by now the area under the curve equals 1 which is the same thing as the probability under the curve is equal to 1 so if we split the curve into sections you know everything to the left of the midpoint would be 50% everything to the right would be the other 50% so you kind of see how that works now this is the sampling distribution of means so the curve we're looking at here is actually a distribution of many many many sample means so we call that X bar so if we go into a population maybe it has a million people in it or a million things and we take many samples over and over again of the same size we could put those samples into their own distribution and it would look something like this so that's called the sampling distribution just to remind you now of course we're interested in putting that concept into the idea of hypothesis formulation and hypothesis testing so in the previous video as we talked about what the null hypothesis is and the alternative hypothesis so in this video I'm going to show them in a little bit different way because it's going to prepare us for hypothesis testing in the next video so you can see here we have mu and mu sub 0 now mu is the actual mean of the population under analysis and mu sub 0 is the hypothesized mean of the population under analysis so what we're talking about here is does the actual mean a line with the hypothesized mean of course we will test that question using sample means and confidence intervals when we do hypothesis testing proper in the next video we have a hypothesized mean that's given to us in our problem then we're going to go out and take a sample of that population and we're going to test whether or not that sample is from a population that aligns with our hypothesized population that's all we're doing now that seems sort of a circular way of talking about things I wouldn't be surprised but again the next couple of videos are meant to actually show you visually what we're talking about so let's go ahead and take a look at the first one so let's say we have a hypothesized distribution here so here we have mu sub 0 and again that's our hypothesized mean that's what we're going to be given in the problem now we can put that in a confidence interval and again we talked about those in previous videos so I won't go back into that and we can set up boundaries now we're going to choose an alpha of 0.05 that's called our significance level or our type 1 error rate which we'll talk about here in a bit but basically what we're saying is that 95% of all the sample means we take are hypothesized to be in this blue region now remember our null and alternative hypothesis so our null says if we take a sample and we take many samples and we have a sampling distribution that that distribution of samples should basically be right on top of this distribution that's why the equal sign is there now the alternative says that that's not the case so if we take samples and we have a sampling distribution that that sampling distribution will be either off to the side to the left below or off to the side to the right as compared to this hypothesized distribution let's look at how that would actually work so let's say we take our first sample and that's X bar sub 1 here that's our first sample mean and as you can see it's right there in the middle now how would this affect how we interpret our null and alternative hypothesis well in this case we would fail to reject the null hypothesis why would we reject the null hypothesis because that sample mean is right smack in the middle of our hypothesized mean so therefore it seems like the null hypothesis is supported it holds quote true so we would not reject it we would fail to reject the null hypothesis so let's say we take another sample and it ends up right there what's a little bit higher than sample one but it's still within that blue region so again we would fail to reject the null hypothesis so maybe we take another sample and it ends up here again we would fail to reject the null hypothesis because it's in that region sample four same story let's say we come to sample five and look where that is well that is outside outside of our interval what would we do then in that case we would reject the null hypothesis so that's so far outside of our hypothesized mean that we would reject the null hypothesis that this came from the same population as our hypothesized population there must be a difference between the two that's how we take another sample sample six well again we would fail to reject that null hypothesis and sample seven we would fail to reject that one so what do we make of all these samples now we can say is probably that our actual population mean is about right or that Green mu is so it's pretty close to the hypothesized mean but you can see it's kind of the middle of all the green regions or the green samples in this graphic so if we took a sample and it was by chance like sample five we would incur correctly reject the null hypothesis so you can think of that sample as kind of the one oddball out of all the samples we could take and again remember statistics is never perfect so there's always that chance we're going to take the oddball sample that is either well below or well above the hypothesized mean and in that case we would end up rejecting the null hypothesis when we should not and that is type 1 error so that is classic type 1 error we reject the null hypothesis because by chance we get a sample that is too far away from the hypothesized population mean therefore we reject the null but we do it incorrectly because we would take many more samples and we would fail to reject the null and that's always a risk we take when we're doing sampling and testing hypotheses now remember alpha is called the level of significance and this is our tolerance for making a type 1 error so in this case our alpha is point zero 5 so what we're saying is that if we did this process a hundred times we would expect about five of these sample means to be outside of that blue region and that is how we interpret graphically type 1 error now what about type 2 error so we repeat the same process so here is a hypothesized population mean and we still have the same things we did before so now we take a sample a sample one and now it's outside of the hypothesized population mean so in this case we would reject the null hypothesis because obviously that's outside the boundary our null hypothesis says it should be within that region and therefore we would reject it just take another sample again we would reject the null hypothesis let's do another one again we would reject the null hypothesis take an the sample now what about this one notice that this is within our region so what will we do we would fail to reject the null hypothesis because our null hypothesis says that this sample comes from a population that is the same or aligns with a hypothesized population so according to this sample that is the case so we look up at our null it says they're equal so therefore we would fail to reject it take another sample well again that one's outside so we would reject the null again we would reject the null and again we would reject the null now what sense can we make of this well it would appear that our actual mean or actual population mean is outside of that region so you can see the Green mu symbol there sort of is in the middle of all those green arrows those that green region so it would appear that those samples are coming from a population that's different statistically different than the hypothesized population except for sample four so if we took a sample and it was by chance like sample four we would incorrectly accept the null hypothesis or fail to reject the null hypothesis and that is type 2 error it's the case when we should have rejected the null but we did not because again we got an oddball sample that in this case is sort of the higher end of the actual population and therefore it put it into that region and we failed to reject the null hypothesis when we should have now this is called beta and it is the probability of committing a type 2 error now unfortunately it's not as easy to compute as alpha is it contains it what varies depending on things like sample size and alpha level in things like that so we're not going to go into calculating beta at least in this one but just keep in mind that alpha our significance level is the probability of committing type 1 error and beta is the probability of committing type 2 error let's talk about the two-tailed test rejection region so everything we have right now is the same as we have before our null is the same an alternative is the same and our alpha or significance level is the same so we'll go ahead and put our curve here and what do we notice well our hypothesized population mean is there in the middle which we expect and we call this blue area the non rejection region and when we go to an iPod is testing that will have a very concrete meaning but for now this we call it just keep in mind that we call it the non rejection region on the ends in the tails we call these the rejection region now realize that this is called the two-tailed test rejection region well why is it called that if you look at our null hypothesis it states that the actual population mean should equal the hypothesized population mean the alternative therefore is that it does not but it could be either below or it could be above if the alternative is the one that's supported so we have to take into account both possibilities that's why it is two-tailed because we don't know which side it may fall on if the alternative is the hypothesis we're going to support based on our analysis now in this case the tails combine to account for an alpha of 0.05 now that point 0 5 or 5% is evenly divided among both tails so in the bottom tail we have alpha divided by 2 which is 0.025 or 2.5% in the upper tail we have alpha over 2 or 2.5% so we're just splitting that alpha on both tails now the point that where we go from the non rejection region over to the rejection region is called the critical value it's sort of the threshold between those two regions so it sort of sets boundaries on whether or not a mean is within the non rejection region or the rejection region so what we're saying is that if the null hypothesis is correct then alpha times 100% that's a fancy way of saying 95% of the sample means should be in this non rejection region the blue in the middle now the critical value is determined by alpha and if we are using the Z or the T distribution now we will not go into that here in this video but we will be talking about it in future videos when we actually do hypothesis tests and we'll talk about when to use the Z the standard normal curve or when to use the T so what are we really asking in this process did our sample come from the same population we assume is underlying the null hypothesis that's what we're asking if so then we expect our sample means to be inside the critical region 90% 95% or 99% of the time depending on what we choose for our alpha now remember whatever our alpha is we choose so for an alpha of 0.1 or 0.1 0 we would have 90% in the blue there we'll show that here in a second for an alpha of 0.05 we would have 95% in the middle for an alpha of 0.01 we'd have 99% in the middle so as keep in mind that what we're asking is did our sample come from the same population we assume is underlying the null hypothesis or the hypothesized population mean so let's take a look at a few curves here now if you notice we're going to change the Alpha level for each one so in this first curve we have alpha is point one zero now look at the blue region there in the middle now how do you compare that to the next curve so look at our alpha of 0.05 now look at the blue region in that curve what do you notice well the blue region in the middle the non rejection region went outward it got wider and this should make sense in the first case we're saying 90% of our sample means should be in that non rejection region in the second curve we're saying 95% of our example means should be in that non rejection region well to get 95% of them in that region the region's going to have to be wider now look at the Alpha of 0.001 now the non rejection region is actually further out than the 0.05 so our critical values on the ends go even further outward so what we're saying is that 99% of our sample means should be in that non rejection region so in order to obtain that we're going to have to widen our non rejection region or after widen our critical values so what we're saying there is as alpha decreases as it shows in these graphs so point one point zero five and point zero one as alpha decreases so does the chance of type one error so as alpha decreases the chance of type 1 error also decreases the critical value to reject the null hypothesis moves outward thus capturing capturing more sample means so it's kind of like moving the goal posts on a football pitch outward the wider they are the more kicks on average are going to go in so to capture more of the sample means the area widens out and that's as our alpha decreases now what does that mean for type one error now that error decreases is because since the area is wider more means are going to fit into that so therefore our null hypothesis will be supported more of the time and we are less likely to reject it incorrectly now is that we have a sample mean over here on our first curve how would we interpret the null and alternative hypothesis well here we would reject the null hypothesis because it's outside of our non rejection region it's in our rejection region in the lower tail now what about this Veen well notice it's in the exact same place the location of the mean has not changed it's in the exact same place but what did change the Alpha change so that means our critical values when outward so how will we interpret the mean under this alpha level well it's close but we would still reject the null hypothesis it's just outside that region so it's in the rejection region just on the other side of the critical value just barely but it is now same mean same place now what will we do in terms of our hypothesis well in this case we would fail to reject the null hypothesis because see it fell within our non rejection region so you can see that by decreasing the Alpha we were able to include that mean but again the question is should we have so in the first two cases we rejected the null hypothesis and in the third case we failed to reject the null hypothesis but the question always is should we have now notice again the mean didn't change location only the alpha levels did now however the move outward of their critical values may also capture a mean from a different population that's kind of off to the side like we showed in those previous two videos we would fail to reject the null hypothesis when indeed we should reject it thus the chance of type 2 error increases as alpha decreases so there is an inverse relationship between alpha and beta given the same sample size so as we decrease the chance of type 1 error we increase the chance of type 2 error and that's the delicate balance that's why we always sort of have to keep in mind the chance that we may incorrectly reject the null hypothesis or fail to reject a null hypothesis let's talk about one tailed test very quickly so basically everything here is the same now a hypothesis changed a little bit so you can see that our null says mu is greater than or equal to mu sub 0 and our alternative is that mu is less than mu sub 0 same alpha level alpha of 0.05 let's go ahead and draw a curve now what do we notice about how the curve relates to our hypotheses so notice our null says that the actual population is greater than or equal to they hypothesized population so if we look down here we can see that we have our Apophis sized mean there in the middle and if our null is supported then our mean that we find should be in this blue region all the way up to and including the entire right side it's a directional hypothesis now if we reject the null then it's going to be down in that rejection region over there on the left hand side now in this case all the Alpha is in one tail remember in the two-tailed test we split the alpha between both tails so we have point zero to five in the left tail and point zero to five in the right tail or the upper tail now because this is a directional hypothesis we put all the Alpha in one tail or the other in this case the lower tail now in a one-tailed hypothesis test all the Alpha is in one tail or the other depending on our alternative hypothesis now the trick to this is that the alternative hypothesis points direction to the tail where the critical value and rejection regions are not just a quick trick but hopefully you sort of understand it conceptually from the actual hypothesis so let's say we have a mean that falls over here that's obviously in the rejection region so we would reject our null hypothesis up there and the right remember our null says that mu should be greater than or equal to mu sub 0 or hypothesized population mean well this sample mean is way down below it's in our rejection region therefore we would reject the null hypothesis and then proceed on to the alternative now the alternative says that mu is less than mu sub 0 and that appears to be the case based on this sample down here in the lower left so we're saying that this sample mean is from a population with a mean less than our hypothesized population mean so what we are positing is that this sample is from some other population with a mean less than our hypothesized population mean now what about this sample over here well we would fail to reject the null hypothesis because remember our null says that this actual population mean is greater than or equal to they are pathi sized population mean well this sample seems to support that so we would not reject that null hypothesis so it appears that this mean is from a population with a mean greater than or equal to the hypothesized population mean so again because our sample was in that non rejection region we would fail to reject the null hypothesis let's go ahead and quickly look at the other one tailed test and that's the upper region so in this case our hypotheses are switched so the null says that our actual population data we're working with has a mean that is less than or equal to they hypothesized population mean the alternative says that the population mean of the data we're working with is greater than they hypothesized population mean and again we're using an alpha of 0.05 so we go ahead and put this in our curve and again it looks just like the lower tail but it's flipped on the other side so again we have a hypothesized population mean there in the middle and then our alpha is all in the right side all in the upper tail so in a one tailed test we already went over this all the Alpha is in the one tail or the other in this case the upper tail so again the trick is that the alternative points to the tail where the critical value and rejection region are so you can see that the greater than sign in the alternative points to the right so that is where our alpha is so let's say we get a mean a sample mean and it falls here on the left hand side how would that affect our hypotheses well in this case we would fail to reject the null hypothesis this is why we call this the non rejection region because if the sample mean falls here we fail to reject the null hypothesis so according to our null hypothesis the population mean of the data we're working with is less than or equal to the hypothesized population mean given in our problem now what if this sample Falls here well in this case we would reject the null hypothesis and that should make sense this sample seems to be coming from another population that has a mean that's higher than that in our hypothesized population mean now remember we could so be committing a error so it just depends now it also means we could be committing a type 1 error that's always the risk but based on what we see here this mean appears to be beyond our hypothesized population mean so just a few causes of type 1 and type 2 errors I went over these in the previous video but I'll just go over them really quickly in this one as well so remember that when selecting samples we are always subject to the laws of chance we may by random chance alone select a sample that is not representative of the population we may select a sample of under field or over filled water bottles that was the example in our previous video we may select a sample of very small or very large farms again that's from an example in our previous video or just in general the sample is in the far-out tails of the sampling distribution because remember the sampling distribution is a distribution of many many many samples now some of those samples are going to be out in the very far tails of that sampling distribution and just by chance we could get one of those when we make our sample just by chance now our sampling techniques baby flow that's why it's a possibility the assumptions and our null hypothesis may be flawed so if we make a null hypothesis based on some hypothesized population mean and that hypothesis is wrong or the number in that is wrong then that's really going to mess up our apophysis test but that could happen and it does happen so in the previous video we talked about USDA data in terms of the farm size so what if the USDA went out and did something wrong in the research and therefore they come up with a hypothesized population mean that we're going to use but we go out and do our analysis and our number is drastically different so did they make a mistake in their sampling or do we is their hypothesis about the farm size correct or is ours so you can see that the Assumption underlying that null hypothesis may be flawed but the most common cause is chance and chance alone and that's the easiest way of saying it so if you have proper sampling techniques and you are confident that the population underlying your null hypothesis is indeed correct then the most common type the most common cause of type 1 and type 2 error is just chance and that's why we have type 1 and type 2 errors okay so let's put all this information into a nice easy handy table so it goes like this remember we can reach two conclusions with respect to our hypothesis we can not reject the null hypothesis or we reject the null hypothesis those are the two conclusions we can reach based on our analysis now the question always is how do those conclusions correspond to the real world to the actual state of the population were interested in now the two conditions of the actual population can be that the null is quote true or supported or the alternative hypothesis is true or supported and again I put true in quotes because I don't mean truth as an eternal truth I just mean supported or upheld so let's take a look at the case where we do not reject the null hypothesis so let's say we do not reject the null hypothesis and the actual condition of the population is that the null hypothesis is true or it is supported so our conclusion matches the actual reality well in that case our sample is correctly inside that non rejection region sort of inside that blue part in the graphs we have been using now let's say we do not reject the null hypothesis but the reality is that the alternative is true or is the real state of the world now in that case we've committed type 2 error so the sample is inside that non rejection region due to chance so that's sort of the second graph we looked at before remember we had all those samples that were well below the hypothesized population mean but we had that one sample sample for that was actually inside that blue region and that was our type 2 error so that's this case we do not reject the null hypothesis when in fact we should have or we get that one sample that's the oddball and it's inside the non rejection region when the other 95% of them are outside that region now look at the conclusion where we reject the null hypothesis down at the bottom so we reject the null hypothesis but the actual condition of the population is that the null hypothesis should be supported or is true so in this case we have our sample mean outside of the non rejection region due to some chance so that was the first graph we looked at so with all those means that were sort of in that blue region as we went down then we had sample 5 that was outside of that non rejection region now of course the other 95% or whatever of those sample means would be inside so in that case just by chance we had a sample outside the non rejection region therefore we rejected the null hypothesis but we should not have it was done in error because we had that one oddball sample and again that's just the chance that's just a probability we have to deal with when we are doing statistics and that's why again we have type 1 and type 2 error and we have to recognize and control for them okay so that wraps up our video on the visualization of type 1 and type 2 errors and this will really prepare us for the next video where we start actually doing hypothesis testing but actually wanted you to see the mechanics of what's going on how the samples relate to the actual curves so when we talk about hypothesis testing proper as we go forward in your mind you'll actually be able to see the curve you'll be able to see what's going on and that helps make it more concrete ok so just a few reminders if you're watching the video because you're struggling in a class stay positive and keep your head up I know you're smart and talented everyone else around you does as well so you should believe so too feel free to follow me here on YouTube on Twitter on Google+ or on LinkedIn that way when I upload a video you know about it and it's always nice for me to connect with people who watch my videos online the world is much too big and life is much too short not to connect with people when we have the chance if you like the video please give it a thumbs up share it with classmates or colleagues you put it on the playlist that does encourage me to keep making them for you on the flip side if you think there is something I can do better please leave a constructive comment below the video and they will try to take those ideas into account when I make new ones for you and finally just keep in mind that the fact that you're on here trying to learn trying to improve yourself as a student or as a business person or just learn something new in general that's what really matters I firmly believe that if you have the right learning process in place the results will take care of themselves so thank you very much for watching I wish you the best of luck in all your studies and in your work and I look forward to seeing you again next time when we talk about hypothesis testing you

Channel: Brandon Foltz

Views: 161,445

Rating: 4.9595823 out of 5

Keywords: statistics type 1 and 2 error, type I error, type 1 and type 2 error, type1 and type 2 error, type 1 and type 2 errors, type 1 error and type 2 error, type 1 error, type i and type ii errors, type i error and type ii error, type II error, type 1 type 2 error, type 1 and type 2 errors statistics, type 2 error, null hypothesis, alternative hypothesis, brandon foltz, brandon c foltz, statistics 101, hypothesis test, statistics, hypothesis formuation, hypothesis testing

Id: k80pME7mWRM

Length: 37min 42sec (2262 seconds)

Published: Fri Mar 01 2013