Distinguished future physicians welcome to
Stomp on Step 1 the only free videos series that helps you study more efficiently by focusing
on the highest yield material. I’m Brian McDaniel and I will be your guide
on this journey through Null Hypothesis, Alternative Hypothesis, Type I and Type II Error, p-Value,
alpha, beta, power & Statistical Significance. This is the 11th video in my playlist covering
all of biostatistics and Epidemiology for the USMLE Step 1 Medical Board Exam. There is a lot to cover but, we will try to
move through things quickly and break them down into bite sized pieces. We will start with the Null Hypothesis which
is represented by H subscript zero. The null hypothesis states that there no difference
between the groups being studied. In other words there is no relationship between
the risk factor or treatment being studied and occurrence of the health outcomes. For example, if we are comparing a placebo
group to a group receiving a new diabetes medication then then null hypothesis states
that the blood sugars or medical complications would be roughly the same in each group. We will talk about this more in a second,
but by default you assume the null hypothesis is correct until you have enough evidence
to support rejecting this hypothesis. If you are the researcher it is usually kind
of a bummer when the null hypothesis is valid, because it means you didn’t find a treatment
that works or that the risk factor you are studying isn’t as important as you were
hoping. The Alternative Hypothesis is denoted by H
subscript a or H1. As you might expect it is the opposite of
the null hypothesis. This hypothesis states that there is a difference
between groups. The research groups are different with regard
to what is being studied. In other words there is a relationship between
the risk factor or treatment and occurrence of the health outcome
Obviously, the researcher wants the alternative hypothesis to be true. If the Ha is true it means they discovered
a treatment that improves patient outcomes or identified a risk factor that is important
in the development of a health outcome. However, you never prove the alternative hypothesis
is true. You can only reject a hypothesis (say it is
false) or fail to reject a hypothesis (could be true but you can never be totally sure). So a researcher really wants to reject the
null hypothesis, because that is as close as they can get to proving the alternative
hypothesis is true. In other words you can’t prove a given treatment
caused a change in outcomes, but you can show that that conclusion is valid by showing that
the opposite hypothesis (or the null hypothesis) is highly improbable given your data. Anytime you reject a hypothesis there is a
chance you made a mistake. This would mean you rejected a hypothesis
that is true or failed to reject a hypothesis that is false. Type 1 Error is when you incorrectly rejecting
the null hypothesis. The researcher says there is a difference
between the groups when there really isn’t. It can be thought of as a false positive study
result. Usually we focus on the null hypothesis and
type 1 error, because the researchers want to show a difference between groups. If there is any intentional or unintentional
bias it more likely exaggerates the differences between groups based on this desire. The probability of making a Type I Error is
called alpha. You can remember this by thinking that alpha
is the first letter in the greek alphabet so it goes with type 1 error. I’m gonna hold off on talking about alpha
and p-value for a few slides. Type 2 Error is when you fail to reject the
null when you should have rejected the null hypothesis. The researcher says there is no difference
between the groups when there is a real difference. It can be thought of as a false negative study
result. The probability of making a Type II Error
is called beta. You can remember this by thinking that β
is the second letter in the greek alphabet. Power is the probability of finding a difference
between groups if one truly exists. It is the percentage chance that you will
be able to reject the null hypothesis if it is really false. Power can also be thought of as the probability
of not making a type 2 error. In equation form, Power equals 1 minus beta. It is good for a study to have high power. A cutoff for differentiating high from low
power would be roughly around 0.8 or 80%. In other words, having a beta less than 20%
for a given study is good. Where power comes into play most often is
while the study is being designed. Before you even start the study you may do
power calculations based on projections. That way you can tweak the design of the study
before you start it and potentially avoid performing an entire study that has really
low power since you are unlikely to learn anything. Power increases as you increase sample size,
because you have more data from which to make a conclusion. Power also increases as the effect size or
actual difference between the group’s increases. If you are trying to detect a huge difference
between groups it is a lot easier than detecting a very small difference between groups. Increasing the precision (or decreasing standard
deviation) of your results also increases power. If all of the results you have are very similar
it is easier to come to a conclusion than if your results are all over the place. p-value is the probability of obtaining a
result at least as extreme as the current one, assuming that the null hypothesis is
true. Imagine we did a study comparing a placebo
group to a group that received a new blood pressure medication and the mean blood pressure
in the treatment group was 20 mm Hg lower than the placebo group. Assuming the null hypothesis is correct the
p-value is the probability that if we repeated the study the observed difference between
the group averages would be at least 20. Now you have probably picked up on the fact
that I keep adding the caveat that this definition of the p-value only holds true if the null
hypothesis is correct (AKA if is no real difference between the groups). However, don’t let that throw you off. You just assume this is the case in order
to perform this test because we have to start from somewhere. It is not as if you have to prove the null
hypothesis is true before you utilize the p-value. The p-value is a measurement to tell us how
much the observed data disagrees with the null hypothesis. When the p-value is very small there is more
disagreement of our data with the null hypothesis and we can begin to consider rejecting the
null hypothesis (AKA saying there is a real difference between the groups being studied). In other words, when the p-value is very small
our data suggests it is less likely that the groups being studied are the same. Therefore, when the p-value is very low our
data is incompatible with the null hypothesis and we will reject the null hypothesis. When the p-value is high there is less disagreement
between our data and the null hypothesis. In other words, when the p-value is high it
is more likely that the groups being studied are the same. In this scenario we will likely fail to reject
the null hypothesis. You may be wondering what determines whether
a p-value is “low” or “high.” That is where the selected “Level of Significance”
or Alpha comes in. As we have already discussed Alpha is the
probability of making a Type I Error (or the probability of incorrectly rejecting the null
hypothesis). It is a selected cut off point that determines
whether we consider a p-value acceptably high or low. If our p-value is lower than alpha we conclude
that there is a statistically significant difference between groups. When the p-value is higher than our significance
level we conclude that the observed difference between groups is not statistically significant. Alpha is arbitrarily defined. A 5% level of significance is most commonly
used in medicine based only on the consensus of researchers. Using a 5% alpha implies that having a 5%
probability of incorrectly rejecting the null hypothesis is acceptable. Therefore, other alphas such as 10% or 1%
are used in certain situations. So here is the key that you need to understand. In most cases in medicine, if the p value
of a study is less than 5% then there is a statistically significant difference between
groups. If the p-value is more than 5% than there
is not a statistically significant difference between groups. There are a couple caveats that complicate
things a bit. Both are related to how you can’t take statistics
out of context to make conclusions. Statistical significance is not the same things
as clinical significance. Clinical Significance is the practical importance
of the finding. There may be statistically significant difference
between 2 drugs, but the difference is so small that using one over the other is not
a big deal. For example, you might show a new blood pressure
medication is a statistically significant improvement over an older drug, but if the
new drug only lowers blood pressure on average by 1 more mm Hg it won’t have a meaningful
impact on the outcomes that are important to patients. It is also often incorrectly stated (by students,
researchers, review books etc.) that “p-Value can be used to determine that the observed
difference between groups is due to chance (or random sampling error).” In other words, “if my p-Value is less than
alpha then there is less than a 5% probability that the null hypothesis is true.” While this may be easier to understand and
perhaps may even be enough of an understanding to get test questions right it is a misinterpretation
of p-value. For a number of reasons p-Value is a tool
that can only help us determine the observed data’s level of agreement or disagreement
with the null hypothesis and cannot necessarily be used for a bigger picture discussion about
whether our results were caused by random error. The p-Value alone cannot answer these larger
questions. In order to make larger conclusions about
research results you need to also consider additional factors such as the design of the
study and the results of other studies on similar topics. It is possible for a study to have a p-value
of less than 0.05, but also be poorly designed and/or disagree with all of the available
research on the topic. Statistics cannot be viewed in a vacuum when
attempting to make conclusions and the results of a single study can only cast doubt on the
null hypothesis if the assumptions made during the design of the study are true. A simple way to illustrate this is to remember
that by definition the p-value is calculated using the assumption that the null hypothesis
is correct. Therefore, there is no way that the p-Value
can be used to prove that the alternative hypothesis is true. Another way to show the pitfalls of blinding
applying p-Value is to imagine a situation where a researcher flips a coin 5 times and
gets 5 heads in a row. If you performed a one-tailed test you would
get a p-value of 0.03. Using the standard alpha of 0.05 this result
would be deemed statically significant and we would reject the null hypothesis. Based solely on this data our conclusion would
be that there is at least a 95% chance on subsequent flips of the coin that heads will
show up significantly more often than tails. However, we know this conclusion is incorrect,
because the studies sample size was too small and there is plenty of external data to suggest
that coins are fair (given enough flips of the coin you will get heads about 50% of the
time and tails about 50% of the time). In actuality the chance of the null hypothesis
being true is not 3% like we calculated, but is actually 100%. Lastly we have Statistical hypothesis testing
which is how we test the null hypothesis & determine statistical significance. For the USMLE Step 1 Medical Board Exam all
you need to know when to use the different tests. You don’t need to know how to actually perform
them. When you are comparing the mean or average
of 2 groups you use the t-Test. When you are comparing the mean of 3 or more
groups you use an ANOVA test. When you are using categorical variables instead
of numerical variables you use a chi-squared test. When using categorical values rather than
having a continuous numerical value that is measurable you have categories such as gender
or the presence or absence of a disease. That brings us to the end of the video. I’d like to give a big thanks to Brittany
Hale & dave carlson for going to my website StompOnStep1.com and making donations which
helped to fund this video. If you found this video useful please comment
below as it really helps me out. And if you would like to be taken directly
to the next video in the series which will cover confidence intervals you can click on
this black box here if you are watching on a computer. That video will be very much related to this
one so I definitely suggest checking it out. Thank you so much for watching and good luck
with the rest of your studying.
