Pretest and Posttest Analysis Using SPSS

hello this is dr. Todd grande welcome to my video on pretest and posttest statistics I'll be talking about how to set up a basic research design and collecting pretest and posttest data and then testing that data using spss so I have a the SPSS data view up and I have a fictitious experiment or data from a fictitious fictitious experiment already entered into this spreadsheet so I'm going to put forth some assumptions about this fictitious study and then show you how to run some statistics that will give you some value information about this data so let's start with the experimental design let's assume for the purposes of this video that you are at an agency a mental health agency and you want to you have a brief intervention that treats anxiety okay so you have a let's say a four session intervention it takes a couple weeks so it's quite brief and you believe that this intervention great greatly reduces symptoms of anxiety you have here now you can see the way I've configured this this ID number would be the participant number ID number for each participant and you can see there's 80 total participants so you have 80 clients that have agreed to become participants in the study and what you would do in the situation to test your was is one way you could you could test your treatment for anxiety is you would screen these participants to see if they are in fact coming in with symptoms of anxiety and you'd have to have inclusion criteria for this type of experiment and of course exclusion criteria for example you may not want participants in your study who have anxiety but also have other severe symptoms you want to try to get the participants whose primary complaint is anxiety so in the real world doesn't always work that smoothly but for the purposes of this video let's let's assume that you you have 80 participants that come in and they have they present with mostly anxiety not many other symptoms mixed in there and you want to test out this new force for session treatment and maybe this this special treatment modality involves other activities as well perhaps they go to a group maybe they watch a series of psycho-educational videos you know that the treatment can be however you want to design it okay but the important part for demonstrating the statistics here is that it is a special treatment that you're testing so you'd put you would randomly assign the 80 participants into two groups and I have a grouping variable here and you can see that half or forty of the participants I've labeled with a one and a half with the two so one would be the control group so since you're your special treatment for anxiety only takes two weeks let's assume that this control group is simply put on a waiting list now it's important to recognize that I use the word control group and the word waiting list for the purposes of this video to more or less mean the same thing but they are in fact different constructs a control group receives no treatment now a wait list a participant on a waiting list doesn't receive treatment but they have the hope of receiving treatment the future which is in you know of itself sort of a treatment so yeah I have to be a little careful there but again just for the purpose of this video let's just treat the control group and the wait list is exactly you know more or less the same thing anyway and you have 40 participants in that group and you have 40 participants in the treatment group that is these participants the one labeled with the grouping variable labeled 2 and grouping variable they're going to receive your treatment for two weeks now you have a pretest score here so when the participants come in you are going to give them an assessment that measures their level of anxiety right so this group this grouping variable is an independent variable okay and this pretest category is a dependent variable it's an observed score so I created these numbers these are fictitious numbers that I entered into an Excel spreadsheet and then imported into SPSS but let's let's for argument's sake say that this particular assessment this lower scores indicate lower anxiety and let's say that any score in the 40s is fairly severe anxiety and the 30s would be moderate and in saying the 20s would be low anxiety so if we look at record number three for example this this participant had a pretest score of 41 which would be which we classifies severe anxiety will be on the low end of severe anxiety whereas this participant number 847 that would be at the higher end of severe anxiety so you have a pretest which is administered and you collect all the scores for all 40 participants then you administer your treatment to the selected and randomly assigned participants and then after that treatment is complete of course the other group just waits they they don't receive any treatment the waitlist then receive any treatment then you administer the same exact instrument again and this is called the post-test so the pretest occurs before the treatment and the post-test occurs after the treatment I also have another column here which I labeled difference which is simply the pretest value minus the post-test value so you can see here for the first record of course in zero but if you look down a ways like for at record 25 this participant dropped from 41 to 28 so that's 13 points is the diff and I'll I'll it'll make sense a little bit later in the video why I created the difference column you can do this I do this in Excel you can also create a column like this in SPSS and it's actually quite easy I'll show you it's transform compute variable as you can see I've already set this up different I'll call this one difference too so I type in difference two and we clear this away now take pretest drag that over subtraction symbol - and then post-test and drag that over and then hit OK it's going to execute that command and you can see that even though it has the decimal places they are in fact the same values all right so a couple different ways to do that I prefer to do the calculations in Excel and then import them over but if you are still in the midst of data collection and you have some cells that are blank you may want to set the variable up like this and just remember though unlike Excel this will not automatically update when you change a value so you have to go to transform compute variable and hit OK and run it again if you make any changes in variables that would affect the new variables value so I'm just going to clear that to get the data set back to the way I had it ok so there's a few different elements components at work here that I want to talk about and a few different ways to test this data what you really want to know is does your anxiety treatment work so the result that you would like to see from the statistics that I'll be running is that the well first of all you'd like to see the anxiety decreased for your your treatment group so you have that special treatment you administer and you want that anxiety to decrease the second thing that you want is to that anxiety to decrease so much as to make it statistically significantly different than the control group right so you're looking for statistical significance you want your your treatment group to have a great improvement as compared to your control group and the way we define that is the way we test that is to say that we look at the two groups and say what are the chances that the differences that we observed occurred to random error alone and if those chances are less than 5% typically in social sciences we use 5% we say that has met statistical significance because there's a less than 5% chance that random error explained the difference which means that you were your experimental treatment in this case the treatment to help reduce anxiety seems to work it seems it seems to do what you wanted to do and that is reduced anxiety better than in a meaningful way better than the wait list group so as you can see here each of these ID numbers represents a participant so let's just take this participant for example 1 0 0 7 this participant had a score of 41 not a score 40 on the post-test so improvement of 1 this is the same participant being tested multiple times so this is called a within subjects design so you've eliminated the error of it being a different person that would be between subjects all right by using the same participant and pre testing and post testing you've eliminated the error that would occur because of using another participant that's why it's called within subjects all right so there are specific statistics you can apply to this type of setup we have within subjects design and they're not going to give you all the information I'm just going to show you a few statistics not going to be all the information you need but in within subject design say that you want to know in general did the did was the pretest significantly different than the post or the post-test significantly lower than the pretest across all eighty records okay so there you'd want to use within subjects because you can you can do that because the way this is set up and the way it was recorded she would go to the analyze menu selection and go to compare means and you'd select paired samples t-test so paired samples is essentially within subjects and independent samples is essentially between subjects just a question of what language is used to describe the different elements in SPSS so for within subjects design you'd want to paired samples t-test so I'm going to clear out clear out these variables I had here before and show you how to set up a paired-samples t-test you take pretest put in variable one post-test variable two and then okay that's it and this gives you some different information remember this is across all 80 of the records this doesn't compare the control group to the treatment group all right this just compares the pretest to the post-test and you can see here that the pretest the mean value was 37 which would be moderate anxiety a little on the high side of moderate and post-test which would be right around moderate 34 and a half roughly the pretest and posttest variables are highly correlated no surprise there then you have the actual statistic the T statistic and you can see that this is highly significant Oh point zero zero zero which I would if I was writing some a paper I would say less than point zero zero one that's how I would record that you can configure SPSS to give you more digits to the right of the decimal place but typically we leave it set at default which is three so we know that's less than point zero zero one which is significant so there is a significant difference between the pretest scores in general and the post-test meaning across all 80 records this doesn't give you any information about specifically about whether your treatment group treatment group worked better I mean your treatment work better than the waiting list I'm moving back to this view I'd like to run the analysis out of this view although you can actually run it out of either view so let's let's look at a one way to run a between subjects let's take a look at an ANOVA again I'll clear this so this is a this is an ANOVA and this is a way to see if there's a difference between the groups and I mentioned earlier that it would become clear why I included the difference column and this is why this is this is one way you can run the statistic so you have difference and that becomes of course your dependent variable and then the fixed factor again they use term fix factor you could also say grouping variable which in this case I've labeled group of that over there's also some other useful information that you can pull up through these buttons on the side for example I'll add a profile plot based on that group variable there are different post hoc tests that can be run that's beyond the scope of this video but just want to show you and there's options here now you can see I'm going to display the means for group I'll have the homogeneity test display descriptives estimates of effect size and observed power okay and then once you've populated the variables and set the these settings that you want you hit OK and you can see it's analysis of variance here you can see have 40 in each group one is the control two as the treatment as I mentioned before you have some descriptives here it gives you the mean for the control and for the treatment and the standard deviation now this is remember this is the difference so this is the mean difference you can see the treatment group had a much larger mean difference now this is Levine's test here and you actually just describes it fairly well this is exactly what it is this tests the null hypothesis that the error variance of the dependent variable is equal across groups and that's what we that's what we want we want homogeneity and you can see it's point nine nine is the p-value which of course we would accept the null hypothesis at that level it has to be below 0.05 for us to reject the null so we're going to accept the null which says so we're going to assume that the error variance of the dependent variable is equal across groups that's a that is an important statistic that is run automatically as part of an analysis of variance so now this table in this table can be confusing what I want to draw your attention is that you're going to be looking at the group row here group not the corrected model not the intercept but the group now and I realize that they're identical values but this is where you want this is where you want to be so you can see it returns an F value analysis variance uses an F test and it has a significance of 0.47 which means it's barely significant but it is significant so so the difference between the treatment and the control group is statistically significant point zero four seven is below point zero five and I'll draw your attention here to a partial a two squared which is the effect size this is five percent point zero five and that means that five percent what a partially squared tells you what effect size tells you is how much movement in one variable can be explained by movement another okay so five percent is fairly low as a as an effect size okay so the the effect size of group of your treatment is actually fairly low only explaining five percent of the variance in the dependent variable and then we have the estimated marginal means and then this little plot and this is fairly straightforward you have the control group and difference that was observed and treatment so you can see just from here it does seem like the treatment worked much better but again it's important recognize that significance although it was met it's very close that's very close to five so that's how to interpret an ANOVA now in this situation you could also run an independent samples t-test and this is already configured so I'll move this out the way to set this up is your test variable it would be the difference between the two and the grouping variable would be group and then you have to define the groups and of course Group one would be one in group to be two all right because that's how I arranged control and treatment all right and hit OK and it gives you less output but you can still see the you know the main difference here and the end which is the sample size Levine's test is the same there and you can see that we would assume equal variances because Levine's if if it was if this was below point zero five we would use this this row here the equal equal variances not assumed row but we're going to use equal variances assumed and you can see that the using this other test the t-test that you still get point zero four seven you get the same result all right so just just two different ways to really accomplish the same goal when you get into more complicated research designs the ANOVA has much more flexibility than the t-test but in this case because the number of variables in the type of variables you do get the same result so that is some information on how to design and record data for a simple experiment and how to run some statistics using that data to give you some meaningful results I hope this video is helpful and as always if you have any questions feel free to ask me and I'll be happy to help you thank you

Channel: Dr. Todd Grande

Views: 183,789

Rating: 4.7146702 out of 5

Keywords: SPSS (Software), pretest, posttest, grande

Id: WQZXzYI_8WI

Length: 25min 56sec (1556 seconds)

Published: Mon Jun 02 2014