Hypothesis Testing and The Null Hypothesis, Clearly Explained!!!

stat question the mornin stat quest at night stat quest in the afternoon it's alright stat quest hello I'm Josh stormer and welcome to stat quest today we're gonna talk about hypothesis testing and the null hypothesis I'm not going to name names but imagine there was a virus and we had two drugs we could use to treat it so we give drug a to three people and measure how long it takes each person to recover from the virus the first thing we notice is that not everyone recovered in the exact same amount of time person number one recovered the fastest in person number two recovered the slowest it's possible that person number one eats healthy food and exercises and already has a strong immune system and that helped them recover quickly and maybe person number two doesn't get as much exercise or maybe person number two has a stressful job or lives where there is a lot of air pollution the point is is that even though all three people had the same virus and took the same drug they did not all recover in the exact same amount of time and that might be due to a lot of random things like exercise or job stress that we cannot control now let's give drug b to three different people that have the virus and measure how long it takes them to recover again we see that even though three people had the same virus and took the same drug they did not all recover in the exact same amount of time and this is probably due to random stuff that we can't control like how much exercise each person gets or how much candy they eat overall it looks like people taking drug a took less time to recover than people taking drug B and when we calculate the mean or average value for drug a and the mean value for drug B we see that on average there is a 15-hour difference between drug a and drug B so after seeing this preliminary data it might seem reasonable to form the following hypothesis people taking drug a need on average 15 fewer hours to recover than people taking drug B and now that we have this hypothesis we can test it by repeating the experiment now when we calculate the means we see that on average people taking drug a need 35 more hours than people taking drug be compared to our preliminary data this result is very unexpected in fact it is the opposite of the original hypothesis but it is also possible that all three people that took drug a in the second experiment have super stressful jobs and unhealthy lifestyles and maybe that's why it took them so long to recover and maybe everyone taking drug be was well rested and super healthy to begin with and maybe that's why they recovered so quickly but it is also possible that we mislabeled drug a and drug B and did the wrong experiment so we repeat the experiment and again the results are totally backwards from the preliminary experiment and totally backwards from the hypothesis that we made so again just to make sure we didn't miss label things we redo the experiment and again these results are the opposite of the original hypothesis so we just keep repeating the experiment each time double-checking every little detail and every time we do the experiment we get the opposite result of the original hypothesis so after doing all of these repeated experiments where we double-checked every little step we can confidently reject this hypothesis that we came up with after doing the preliminary experiment BAM now let's imagine we had two more drugs C and D just like before we gave drug C to three people and measured how long it took each person to recover from the virus then we gave drug D to three different people and measured how long it took them to recover from the virus and based on this data we can create a hypothesis about drug C and drug D people taking drugs C need on average 13 fewer hours to recover than people taking drug D now just like before we decide to test this hypothesis by repeating the experiment only this time instead of getting something that's the exact opposite of what we expected we get something that is only slightly different in this case the difference is in the same direction but it is only 12 hours then we repeat the experiment again and again we get something slightly different from the preliminary experiment and hypothesis the difference is in the same direction but this time it is 13.5 hours the good news is that we probably didn't mislabel the drug like we did last time in the differences between the three experiments might be due to random things we cannot control like maybe these people exercised a lot and had relatively healthy diets compared to these people who took longer to recover but regardless the hypothesis says that people taking drug C needed 13 fewer hours to recover but when we repeated the experiment the first replicate said the difference between averages was 12 which is different from the hypothesis and the second replicate said the difference was 13.5 which is also different from the hypothesis and let's be honest the only reason the hypothesis says 13 fewer hours is because that was the result from the first experiment however we could have just as easily put 12 fewer hours in the hypothesis because that's what we got the second time or we could have put 13.5 fewer hours in the hypothesis because that's what we got the third time so if we just pick one experiment like the first one and use that to define the hypothesis then we have two experiments that are not different enough to give us confidence to reject the hypothesis but because there is just as much data suggesting that the difference is 12 hours and there is just as much data suggesting that the difference is 13.5 hours these experiments don't make a super confident that the hypothesis of 13 fewer hours is correct again maybe drug a reduces recovery by 13 fewer hours but maybe it reduces recovery by 12 hours or 13.5 because the results from the repeated experiments are not different enough to cause us to reject the hypothesis and because they don't convince us that the hypothesis is correct either the best we can do is fail to reject the hypothesis small BAM to summarize what we've covered so far we can create a hypothesis and if data gives us strong evidence that the hypothesis is wrong then we can reject the hypothesis but when we have data that is similar to the hypothesis but not exactly the same then the best we can do is fail to reject the hypothesis because it's unclear if the hypothesis should be based on this result or this other slightly different result or this result or any other possible outcome double bam now let's take a closer look at the hypothesis itself you may remember that the only reason the hypothesis is 13 fewer hours is that it was the first result but we could have just as easily gotten a 12 hour difference or a 13.5 hour difference and ended up with a different hypothesis and if 12 and 13.5 are reasonable hypotheses then so is 12.25 or 13.1 in other words there are a lot of reasonable hypotheses how do we know which one to test since the goal is to see if drug C is different from drug D we simply test to see if there is no difference between the drugs oh no it's the dreaded terminology alert the hypothesis that there is no difference between things is called the null hypothesis so let's take a look at two examples of the null hypothesis in action now imagine we are testing two new drugs E and F and this time we only get a 0.5 hour difference this person recovered the fastest but it is easy to imagine that if they had exercised a little less or had a slightly worse diet then they might have taken a little longer to recover likewise if this person was just a little healthier to begin with then they might have recovered a little more quickly these small random differences give us a slightly different result now instead of drug f being slightly better by 0.5 hours drug ii is slightly better by 0.25 hours because these small random differences give us slightly different results we can use the null hypothesis so we don't have to worry about whether or not the difference is exactly 0.25 or 0.5 hours instead we simply see if the data convinces us to reject the hypothesis that there is no difference between drug II and drug f in this case the original result was zero point five hours in favor of drug f but small random things could have easily changed the result to be a 0.25 hour difference in favor of drug II and thus the data does not overwhelmingly convince us to reject the null hypothesis so we fail to reject the null hypothesis that there is no difference between the drugs in contrast if we tested the drugs on a lot of people and little random things would not change the results very much then we could confidently reject the null hypothesis that there is no difference between drug II and drug f BAM note without the null hypothesis we need preliminary data in order to make a statement that we can test and follow-up experiments this is because we don't know if we should test if the difference is 13 hours or 13,000 hours until we get some data in contrast the null hypothesis does not require preliminary data because the only value that represents no difference is zero triple bam in summary rather than get stressed out over a large number of possible hypotheses that we could test to see if drug C is different from drug D we use the null hypothesis to determine if there is a difference if we do an experiment with a bunch of people and a lot more people taking drug C had shorter recovery times than people taking drug D so many that it would be hard to imagine that the results were due to random things like everyone taking drug C had better diets or got more exercise than the people taking drug D then we could reject the null hypothesis and then we know that there is a difference between drug C and drug D alternatively if little random things could easily shift the result from one drug to the other and then back again then we would fail to reject the null hypothesis BAM but wait what about the alternative hypothesis because the alternative hypothesis is super important it has its own quest so check it out and if you don't already know about p-values they would make a wonderful follow-up lastly if you want to review statistics and machine learning offline check out the stat quest study guides at stat questo RG there's something for everyone hooray we've made it to the end of another exciting stat quest if you liked this stack quest and want to see more please subscribe and if you want to support stack quest consider contributing to my patreon campaign becoming a channel member buying one or two of the stack quest study guides or a t-shirt or a hoodie or just donate the links are in the description below alright until next time quest on

Channel: StatQuest with Josh Starmer

Views: 106,949

Rating: 4.9425135 out of 5

Keywords: Josh Starmer, StatQuest, Machine Learning, Statistics, Data Science, Hypothesis Testing, Null Hypothesis

Id: 0oc49DyA3hU

Length: 14min 40sec (880 seconds)

Published: Sun Jul 05 2020