We are ready to learn about how to compare more than two means, or how to do hypothesis testing when we have more than two groups. The test we will use is called ANOVA or a one-way between subjects analysis of variance. Now that you've learned about t tests and how they can be used to compare the means of samples, you may wonder why would we ever need anything else? Anytime we need to compare means, we'll use a t-test! But hold up, there's something more you need to know. T tests are fine when you're comparing two means, but if we have three or more groups, the t tests have a flaw. and here is why we need more than just t-tests. A t-test can be used to compare two independent means. Remember "Tea for Two"? But the t-test has some limitations. The t-test is limited to only one independent variable with two groups, and only one dependent variable. So if we want to compare multiple independent variables, or multiple dependent variables, we're out of luck. And the t-test can only compare two means. But often we would like to compare means from three or more groups. But then again, if we want to compare three means, why not just use three t tests? And here's where the problem comes in. We cannot use multiple t tests because using multiple tests on the same data causes the Alpha level to skyrocket and this means that we are more likely to make Type I errors, or to find erroneously statistically significant results where no result truly exists. So let me illustrate this problem with an example about exercise. The Mayo Clinic recommends exercising 2.5 hours per week with moderate aerobic activity for your health. Well, what if we wanted to compare people who exercise 2.5 hours per week with a control group that does not exercise at all? But while we are at it, why not add another group that exercises for five hours per week, doubling the recommended time. Will that double the health benefit? Now, this might seem like a good time to use our T tests. We have three groups: A, B, and C, so we could run one T test comparing group A and B, and then we could run a second T test comparing groups B and C, and then we would run a third T test comparing groups A and C. But with our alpha level set at .05, that means that each comparison has a 5% error rate. So what's the problem? Well, let's examine this using the analogy of a lottery. Imagine that you are going to play the lottery. Now, normally I would not recommend this because the lottery is a tax on people who are bad at math and don't understand probability, but this is a special lottery: only 20 tickets are going to be sold for ten dollars each, and the prize is one thousand dollars. This is a good lottery to play! So what would you do? Well, if you were smart, you would buy all 20 tickets. Why is that? In this analogy, every lottery ticket has a 1 in 20 chance (a 5% chance) of winning. The probability of any ticket winning is p = .05, but even though any one ticket has a 5% chance of winning, if you hold all of them, you have a 100% chance of winning. If you hold all 20 tickets, you will spend two hundred dollars but one of them will certainly be a winner and you will win one thousand dollars, although you will not know ahead of time which one will be the winner. On the other hand, you won't really care which one was the winner, but that's not the point. The point is that this same logic applies with hypothesis testing. Every T test has a p = .05 chance of being a Type I error, of erroneously finding a difference where none exists. For any single t-test, you have a five percent chance of being wrong. But if you keep running t tests on the same data, one of them is going to be a Type I error, just like that lottery. And you don't have to run 20 t-tests to have this problem. For instance, if you reject the null in just five groups, your alpha level is functionally about a .40 One of your tests will identify an effect that does not exist. One of them will be a Type I error, but you will not know which one was in error. Only now, you will care, because one of your published findings is going to be a lie and you won't know which one. So the solution is analysis of variance. Analysis of variance is abbreviated as ANOVA. Analysis of variance analyzes the variances of the groups to assess differences in means between the groups. ANOVA is actually an extension of regression, called the general linear model, or GLM. In SPSS will be one of five models called GLM1 through GLM5. We will be talking about the GLM1 model. The GLM1 ANOVA model can be used with one independent variable with any number of conditions or levels. But you have options! The general linear model can be used with both independent groups or related groups, called the repeated measures ANOVA. General linear model can be used with multiple independent variables or multiple dependent variables, called a MANOVA. The GLM can include covariance as a control called an ANCOVA or a MANCOVA with multiple variables. The general linear model is actually a special form of regression, so in reality, everything that we are doing with hypothesis testing from Z tests, to t-tests, to correlation, to ANOVA, is actually all regression. It's all regression under the hood. So how would the ANOVA work with our now familiar hypnosis example of hypothesis testing? Using our research question of "Is recall better under hypnosis or without?", we would begin with an existing memory test. We would randomly select participants and then we would randomly assign participants to one of three groups. What might those groups be? The first group of our independent variable would be those who are hypnotized. We would compare their memory scores to a second group that was not hypnotized, and perhaps a third group, who was drunk. The dependent variable for all three groups would be number of words recalled. In the end, we could answer the question: "Do hypnotized participants remember more words than the unhypnotized or the drunk participants?" If the ANOVA was significant, then one of the groups was different from at least one other group, but we would need a post-hoc tests to figure out which group differed from which other groups.
