Statistics 101: Type I and Type II Errors

hello thank you for watching and welcome to the next video in my series on basic statistics now as usual a few things before we get started number one if you're watching this video because you were struggling in a class right now I want you stay positive and keep your head up if you're watching this it means you've accomplished quite a bit already you're very smart and talented and you may have just hit the temporary rough patch now I know with the right amount of hard work practice and patience you can get through it I have faith in you many of the people around you have faith in you so so should you number two please feel free to follow me here on YouTube on Twitter on Google+ or on LinkedIn that way when uploading a video you know about it and it's always nice for me to connect with people who watch my videos online wherever in the world you may happen to be on the topic of the video if you like it please give it a thumbs up share it with classmates or colleagues or put it on a playlist because that does encourage me to keep making them for you on the flipside if you think there is something I can do better please leave the constructive comment below the video and I will try to take those ideas into account when to make new ones for you and finally just keep in mind that these videos are meant for individuals who are relatively new to stats so I'm just going over basic concepts and I will be doing so in a very slow deliberate manner not only they want you to know what's going on but also why and how to apply it so all that being said let's go ahead and get started so this video is the next in our series on hypothesis formulation and eventually hypothesis testing and like the last videos it's sometimes a concept that can give intro stat students headaches it can drive them crazy and the reason is that this concept runs counter to how we usually think about our everyday lives and of course this is the concept of type 1 and type 2 errors and hypothesis formulation and testing now remember in previous video two talked about what hypotheses are we talked about how to set them up pull data out of problems and then set up our null hypothesis and/or alternative hypothesis now remember the null hypothesis is what we assume to be true it's the status quo it's the information we are given or we again we assume to be the case the alternative is where we go next if we have to reject the null hypothesis so if we have to reject our assumption then we proceed on to the alternative now one thing we know by this point is that stats is never certain it's never 100% perfect so the way this usually works as far as hypotheses go we set up our hypothesis we go out and collect the sample we do some analysis on that sample and then we reach some conclusion about our hypothesis so the question is does our conclusion from our analysis match the actual state of reality so remember we're basing that conclusion of some sample we took and then we either reject or fail to reject our null hypothesis but the question is does our conclusion match the actual state of reality now we know it's not going to happen 100 percent of the time in that in a nutshell is type 1 and type 2 error does our conclusion and hypothesis test do our analysis match the actual state of reality now this is not just some academic distinction here in real life in real studies in real research the type 1 and type 2 errors can have literally life or death consequences and I'm not over exaggerating so in the example I'll use in this video I'm going to show you that as startling as I can now the way I'm going to proceed is to show you a couple of non statistical reward examples that exhibit the idea of type one and type two error and then we will proceed into more formal statistical ways of putting it so all that being said let's go ahead and look at our first non statistical real world example now I call this the fire alarm hypothesis and unfortunately it's something a lot of us have to deal with at some point in our lives or in something similar you know similar situation then it goes something like this let's say you're going down the hallway at work or school sort of doing your own thing everything is normal as usual but you encounter a sudden smell of smoke like something's burning now you know that may mean a serious fire is taking place in the building or it could be nothing serious maybe someone burned popcorn or something else in the microwave in the kitchen or cafeteria or wherever else you might be but the question is what do you do next you have to make some decision on what you're going to do just take a look at what might happen if you think the smoky smell is nothing serious you may decide that your assumption that everything is normal is correct and therefore you will not pull the smoke alarm or the fire alarm so maybe someone burned something in the kitchen of course that's annoying but it's not a serious fire so you would just proceed on like everything was okay and of course you will not pull the fire alarm for that now if you think the smoky smell is due to a serious fire you may reject your assumption that everything is normal and then you will pull the fire alarm so you have to reach some conclusion based on the evidence around you about the state of reality it's either a not serious fire or it's a very serious fire and then you have to decide what you're going to do so let's look at what might go wrong in this situation so of course you smell the smoke and you think to yourself this is not one so you reject the assumption that everything is okay you reject your null hypothesis therefore you go ahead and pull the fire alarm so you rejected your null hypothesis that everything is normal and then you went ahead and pulled the fire alarm so the building is evacuated and the fire department arrives to investigate now after the investigation it is determined that there was no serious fire you quote falsely pulled the fire alarm now here's what we interpret that when you rejected your assumption that everything was okay so you look to the situation you said this is not normal I'm going to reject my assumption and everything is normal and then you proceeded so when he rejected your assumption that everything was okay when it really was okay because there was no fire you committed type 1 error or in this case a false alarm so let's talk about type 1 error more formally so type one error is rejection of the assumption or rejection of the null hypothesis when it should not have been rejected so in the fire case you rejected your assumption that everything was normal but the reality was everything was okay so type 1 error is rejecting the null hypothesis when it should not be rejected you can think of it as incorrectly rejecting the null hypothesis in this case a false alarm and again this is a real-world situation but as a perfect example of what type 1 error is rejecting the null when it should not have been rejected now stick of what else might go wrong let's say you smell smoke and you think oh it's probably someone who burned their lunch and the microwave no big deal so you go ahead and just proceed along your way therefore you did not reject your assumption that everything is okay you upheld your null hypothesis you just sort of concluded that's not nothing serious so you are not something serious so you went ahead and just went on your way but let's say there is indeed a serious fire now no one is injured luckily but the entire building does burn to the ground so when you failed to reject your assumption that everything is okay when it really was not okay you committed type two error so let's formally define this and then we'll compare the two so type two error failure to reject the assumption or failure to reject the null hypothesis when it should have been rejected so incorrectly not rejecting the null hypothesis so think about type 1 type 1 is when we reject the null hypothesis incorrectly type 2 is when we fail to reject the null hypothesis incorrectly so in the first case we pull the fire alarm when there was no fire in the second case we did not pull the fire alarm even though there was a fire so again we'll do some other examples but this sort of sets the stage for basically what type 1 and type 2 error as relates to our conclusion we reach and the actual state of reality now we can set this up as a little hypothesis so our null hypothesis was the smoke is annoying but not serious everything is okay as usual there's no serious fire the alternative would be that smoke is evidence of a serious fire everything is not okay as usual there is a serious fire now we can set up a little hypothesis table here so on the left hand side we have the conclusion we reach and then on the top we have the actual condition or the actual state of reality so let's go ahead and fill this in with some information now in the conclusion side we have we do not reject the null hypothesis so no serious fire or we reject our null hypothesis that everything is okay and there is some serious fire so those are the two conclusions we could reach in this situation now the actual condition of reality is there is no serious fire or there is a serious fire now we notice that there are two correct conclusions here so if we do not reject the whole null hypothesis and there's no serious fire and the state of reality is that there is no serious fire that is a correct conclusion now if we reject the null hypothesis that everything is normal and there is a serious fire based on our evidence and reality is that there is a serious fire that is also a correct conclusion so you can see our two correct conclusions there now the question is where is the type 1 and type 2 error in this chart well they go here let's look at the type 1 error situation first so in the type 1 error we reach a conclusion where we reject the null hypothesis so in this case we reject the assumption and everything is normal and we think there is a serious fire going on but the fire department arrives and they determine the actual condition is that there is no serious fire so therefore our conclusion does not match the actual reality now let's look at type 2 error so in type 2 error our conclusion was that we did not reject our null hypothesis and therefore we don't think there's a serious fire going on but the reality is there is a serious fire going on so we have a disconnect between our conclusion and the actual state of things so you can see that we have two correct conclusions and two errors type 1 error and type 2 error now in general the real-world consequences of a type 2 error are much greater in this case a type 2 error may mean the loss of property or even lives so think about this what happens if we commit a type 1 error in this case well we think there's a really serious fire so we pull the fire alarm that gets everyone out of the building and it brings the fire department to the location but it turns out there's no serious fire well pulling the fire alarm is going to be a very big inconvenience to everyone in the building but we pulled it we pulled the fire alarm because we'd rather be safe than sorry right so the minor inconvenience of the fire drill is no match of the consequences of type 2 error look at type 2 we don't reject our null hypothesis we think there's no serious fire going on when in fact there is a serious fire so we don't pull the fire alarm what can what can the consequences be here well if no one catches it in time the building could continue to burn because there is a serious fire and it could mean the loss of course property and lives so while the type 1 error in this case is a real-world inconvenience the type 2 error can cost property and lives and that pattern tends to repeat itself throughout these examples so let's look at another example and I call this the end on hypothesis now you're probably thinking what is that well here's what it is in a manufacturing system such as the Toyota Production system so Toyota the car company and other lean systems line workers have the power and indeed the responsibility to stop the entire production line when they notice a serious problem and this is done by pulling what's called an end on cord so you can see in these pictures all along the production line there's a cord like you might see on a bus or even a train and if they notice something is seriously wrong the worker is supposed to is actually required in this environment to pull it that will stop the entire production line dead in its tracks until that situation is resolved now the assumption is that the production line is running correctly this is the minute-by-minute null hypothesis so as things run the assumption is everything is running smoothly everything is running correctly now when a worker notices a defect he or she has a choice to make should they pull the cord or not if the end on cord is pulled the worker is rejecting the assumption that the process is error-free they are rejecting the null hypothesis they are rejecting the assumption then everything is okay if the cord is not pulled then the worker is not rejecting the assumption that the process is error-free does maintaining a null hypothesis and the production line continues to run so it's all about rejecting or not rejecting the assumption that everything is okay now what if the cord is pulled which is a rejection of the null hypothesis but nothing is actually wrong with the production process again this is sort of a false alarm situation in this case the worker committed a type 1 error they reject the null hypothesis incorrectly they rejected the null hypothesis that everything was okay incorrectly so type 1 error again is a wrongful rejection of the null hypothesis when it should not be rejected now something about this in the Toyota Production system and in similar lean manufacturing systems this is actually encouraged this is perfectly okay so if the worker thinks there is something wrong they are supposed to they are required to reject the null hypothesis reject the Assumption and everything is going ok pull the cord and stop the production line they are not supposed to worry about whether or not they are correct or incorrect and that goes to the idea of the difference in the real-world consequences between type 1 and type 2 error which log about here in a minute but actually this type 1 error is OK in this type of production system now what if the cord is not pulled thus supporting the null hypothesis but something actually is wrong with the production process so oftentimes a worker or sometimes a worker might be afraid to pull that cord because they don't to be blamed for stopping production and this is especially true in American manufacturing systems where at least previously to Toyota coming on the scene the idea was to produce as much as possible as fast as possible so what if the cord is not pulled but something is wrong in this case the worker committed a type 2 error so they failed to reject the null hypothesis they failed to reject their assumption that everything is wrong when something was actually wrong so type 2 error is wrongful support of the null hypothesis when it actually should be read acted now note in this case in this case a type 2 error can be catastrophic it may mean that hundreds or thousands of cars have that defect built into them and see this is why in the Toyota Production system and other lean manufacturing systems pulling that and on cord is encouraged if not actually required because the type 1 error while an annoyance is okay if they stop the line for a couple minutes it's not that big a deal it's going to mean a few cars off the end of the line that didn't produce but think about the reverse if a worker sees something that they don't think is right but don't pull the cord and let that defect keep going and going and going and then all these cars at the end of the line have that defect build in that's why type 1 error in this case is really okay but type 2 error is very very bad very bad and if it's a part on the car that's a safety issue it can be very very bad so again think of the difference between type 1 and type 2 error in these two cases type 1 was kind of a false alarm so the alarm was pulled when there really wasn't a fire the andon cord was pulled when there really wasn't a major problem that's annoying but it's not catastrophic in terms of the consequences but type 2 is in the fire case if we don't pull the fire alarm and there is a fire our building may burn down and people may lose their lives in this case if we don't pull the end on cord and there is a problem that defect may get repeated hundreds or thousands of times and that's the real-world consequence of type 2 error as related to type 1 error now we can set this up as a couple of hypotheses like we did the last time so our null hypothesis is that the problem is annoying but not serious everything is okay no major defect so the normal running of the production line our null hypothesis is that the problem is serious everything is not okay as usual there is a major defect so we can set up our table just like this so we have two conclusions we can make we can accept our null hypothesis or you think of it as not rejecting I should have changed that to not rejecting our null hypothesis and that is no serious defect we could reach the conclusion of rejecting our null hypothesis that there is some serious defect going on so that's the conclusion we can make on the line in the moment now the actual condition may be when the supervisor walks over could be that there is no serious defect or there is some serious defect now if we if we cannot reject our null and there is no serious defect that's the correct conclusion and actually that's the way the production line works now the vast majority of the time we assume everything is going okay and there are no serious defects that's a conclusion that is correct that's a match now if we reject the null hypothesis we think there is a serious defect and the reality is there is a serious defect that's also correct so our correct conclusion now let's look at the type 1 situation very quickly so let's say we reject the null hypothesis we reject our assumption that production is going okay without serious defect we're going to reject that because we think there's a problem now the actual condition is that there is no serious problem or defect so that disconnect between conclusion and reality is the type 1 error so the production line might stop for a few minutes or even maybe less than the minute to check it and then if we see there's no serious problem or defect it just starts up again the type 2 error that's when we accept or fail to reject the null hypothesis so we think everything is going smoothly when in fact there is a serious defect going on and that is type 2 error and again that gets replicated over and over and over into each successive car so in general the real-world consequences a type 2 error much greater in this case type 2 error would mean hundreds or thousands of defective cars and maybe even unsaved cars as we proceed come on down the line

Channel: Brandon Foltz

Views: 131,000

Rating: 4.961165 out of 5

Keywords: type i and type ii errors examples, type one and type two errors, type i and type ii errors, type 1 and type 2 errors statistics, type 1 error statistics, type 1 and type 2 errors, type i errors, type II error, type I error, type 1 and type 2 error, type 1 type 2 error, type 1 error and type 2 error, null hypothesis, brandon foltz, alternative hypothesis, brandon c foltz, hypothesis testing, statistics 101, hypothesis, logistic regression, hypothesis test, statistics

Id: VFMcGdWp0MQ

Length: 24min 54sec (1494 seconds)

Published: Wed Feb 27 2013