Multilevel modeling using STATA (updated 2/9/18)

okay so the purpose of this video is to provide some illustrations of how you can carry out multi-level modeling using the Stata program we're going to focus on carrying out a two-level multi-level model using data that was downloaded from this website right here they're the datasets and the example is coming from this book basically just downloaded the SPSS data file and converted it to Stata for the purposes of the illustration the data is school level data are actually a student data with students nested within schools so the variables that are associated with the data file are over here you'll have school code ID gender SES and so forth if we open up the data file and take a look at the data you'll see that in this in this file right here we have variable for school code there are about 419 schools we have students nested within those schools these are a student identifier some the data set these are the identifiers for students within schools so these first twelve students that we see right here are associated with school one the next 13 are associated with school two and so forth we have our primary outcome variable at level one is math achievement so it's we're using this variable right here is our level one outcome and we're going to use SES student level SES as a predictor of math achievement in the data file we also have you know there's gender coded zero and one for male and female respectively so that's another level one variable but we're not going to focus on that in this particular illustration we also have SES mean and basically these are the average SES levels for the students within each school so you can see right here these values are all exactly the same and it's reflecting the average of these students SES levels so this is basically going to be treated as a contextual predictor at level two so we have level one student SES and level two school level composite SES as predictors of math achievement so we're gonna start off our analysis with a random intercept model and basically the random intercept model is is utilized in order to improve health determinants as to whether or not HLM is an appropriate strategy and whether you know might we also consider something like least squares regression and so the basic idea is is that if you're drawing random samples out of a population and there's no clustering kind of effects no dependent observations within clusters then OLS regression makes more sense but you know given that school that students are nested within schools those students within the individual schools are probably going to be more similar to each other than they are between schools and so that clustering effect produces a lack of independence in terms of the the observations on our on our dependent variable and so so it makes sense to consider multi-level modeling with this particular data and for as a first step we typically start with a baseline model which is a random intercept model and base and what that means is there's no predictors that are incorporated in the model we're only studying variation in the intercepts across the level two units and what that translates into in a in this kind of model is that with the intercepts are basically the school means on the dependent variable when we start incorporating predictors in the model the intercepts are still means but they're adjusted for the presence of the predictors that you include in the model so we're still fundamentally looking at variation and it means but we we talk about in terms of varying intercepts so we can run our random intercept model in a couple ways through Stata we can use drop down menu or we can use syntax so this is the syntax that we're gonna use for our analysis but I thought I would illustrate both just to give you an idea about some of the menus as well so to run our analysis we'll go to statistics we'll go down to MIT multi-level mixed effects models to linear regression here you'll see that we have two estimation methods that are available to us maximum likelihood restricted maximum likelihood the example in the text from this example right here in Chapter three uses restricted maximum likelihood but we're gonna stick with maximum likelihood in this demonstration okay so in terms of the where it says fixed effects model what we're gonna do is we are going to highlight our dependent variable which is math for math achievement and we're not going to include any independent variables in the model so because we don't have any fixed effects at that level that we are interested in studying but we do want to model the random variation and intercepts across the schools so I'm gonna click on create level variable for equation I'm gonna select school code right here and then leave everything else as it was so now this this set of specifications will model the random variation across schools so I'm gonna click on OK and so now you'll see our output and so here in terms of the output the coefficient that you see right here this is basically the grand mean across the schools and this is basically a gamma I'm not very good at drawing these things but gamma and this is the grand mean or on for the intercept and what that translates into is that we can say well across the schools the average math achievement was fifty-seven point six seven four we have a significance test right here which is really not terribly useful under many circumstances because we're really just testing whether this coefficient is significantly different from zero and so it's not terribly useful under many circumstances down here we have the variance components we have the residuals at level one and the very and the variation intercepts at level two so these right here we are essentially variance components so you know you can think about this way at level 1 we have prediction errors for each observation reflecting the deviation between a given observation and the and the mean for the given school so that's just the difference between each student's score on math achievement and the the school mean and what we want to do is to model or study the variation in those school means are in the residuals at level 1 so that's level one and then at level 2 we have the variation and intercepts and so each intercept is denoted as you know reflecting mu 0 J and we want to study the variation of those intercepts okay or this is actually kind of reflecting the the deviation between a given school mean and the grand mean so we're essentially studying the variation and and those as school effects if you will so down here we have the variance estimate for the residuals and the variance estimate for the intercepts and what we want to do is to test these to see if they're statistically significantly different from 0 and so you you obviously we have our standard errors right here and interestingly we we have no printout of any kind of Z test or anything of that nature and so it's a little bit confusing when it comes to trying to make a judgment using the confidence intervals so I'm I kind of created a little calculator to run a test of these effects to see if they are significantly different from zero so this is it right here and basically all we do is just put the the variance estimate here standard error here and we can generate a p-value for essentially a one tailed z test so basically what I'll do is I'll just going to highlight the variance of the residuals and put it here where it says level one residuals and the standard error for the residuals it over as well and we have the variance of the intercepts and the variance assist in the standard error associated with that variance estimate okay and so then we have and observe the Z value and we have a essentially for the variance components test we have it's a one tailed z test just kind of for comparative purposes I want to just show you what's going on in SPSS and and show you how this was derived so I'm gonna run a intercept only model in SPSS and you know I've already there's already specified the random effect and everything and asked for all the statistics that I'm asking for through the through Stata and so here you can see down below there's our grand mean across the schools and these are the variance components so these are the estimates and you'll notice that those estimates are exactly the same as what we see in in SPSS the standard errors are the same and then we have the wall Z so the the Z value for the residuals is fifty six point eight oh two which is you know kind of going back to our calculator you have the fifty six point eight oh two and for the intercepts the variance of the intercepts it was ten point three four eight and you can see there's a ten point three four eight so that's how it was derived so basically just taking a ratio the estimate to the standard error it gets us a Z value and we basically are going to use a one tailed test so the interesting thing in SPSS is is that the p value is actually printed out for a two-tailed test but it really should be one tailed because you can't have a negative variance so we're only testing in the upper tail of the Z distribution and so so that's why this predict or value right here it's kind of noted that we're using a one-tailed test if we you know if we want it to model the covariance between the intercepts and slopes later on then we would use the two-tailed tests and so there's an adjustment for that for this particular test so at any rate that is the random intercept model if I want to use syntax I will just go down to the command line I'll type in mixed so that's the the command to to run the multi-level model I'll type in math which is the name of the dependent variable two vertical bars and then type in the name of the the level two predictor which is school code and then a colon just make sure that the the variables in terms of the caps and you know whether the upper and lower case match what's the variable names in your data set so when I run the analysis this is it right here and it's the same information as what we had had before oftentimes you want to follow up the multi-level model by by generating the intraclass correlation coefficient or various fit indices so we're gonna use the e-stat command here you can use it through the drop-down menus through the post estimation drop-down menus in Stata but it's just a little quicker if you can just do it this way so we're gonna use we're gonna type in East at ICC and what the ICC is the intraclass correlation coefficient is giving us an index of the proportion of variation in the level one outcome that is occurring between the level two units in this case between schools so we can interpret this at to mean that about thirteen point seven percent of the variation and math achievement is occurring between schools and that's really a non-trivial amount so it actually suggests and that multi-level modeling is really probably the best way to go I can also type in East at IC and get information criteria such as you know the AIC akaike information criterion bayesian information criteria and this can be useful for making model comparisons you only want to utilize you know these kinds of criteria though if you're using a maximum likelihood estimation now let's try another model where we incorporate a fixed level one predictor with randomly varying intercepts so in this case I'm gonna add in socioeconomic status as a level 1 predictor and the basic idea is is that you know if I'm if I'm laying out this predictor with such a economic status I'm essentially saying that I'm modeling to allow the slopes are basically fixing the slopes to be the same across the school context but I'm still allowing the intercepts to randomly vary later on we might wish to allow the slopes to randomly vary as well as the intercepts but in this case if I if I am generating this kind of model I'm fixing the slope to be equal across groups but then allowing the intercepts to randomly vary so let's do this through through our two routes so if I go to statistics multi-level mixed effects models linear regression in this case I'm going to click on this button go down to SES and leave everything else exactly the same so now when I click on OK we have our output you'll notice that in this case SES this is our regression coefficient for SES and it is positive and I would interpret this to mean for every one unit increase on SES as a predicted increase a three-point roughly 3.8 eight units on math achievement so there's a positive predictive relationship we have a Z test here and the p-value that's printed out is for a two-tailed test so if I wanted to carry out a one tailed test testing a directional hypothesis I could certainly do that by just splitting the p-value in half but it looks like SES is a significant positive predictor of math achievement meaning that students who tended to have higher levels of SES also tended to demonstrate Guardi levels of math achievement the intercept now is math achievement adjusted for the presence of SES so in other words this is the grand mean on math achievement adjusted for the level and predictor in the model now we have the variance components still at level one and level two so we're essentially want to test whether the residuals at level one are they statistically significantly different from zero and also are the intercepts at level two is that variance significantly different from zero so I can again I can just use my little handy dandy calculator and I will just put in the level one residual variance estimate here and the standard error and we'll do the same for the intercepts okay there you go and so we can see we still have we have significant variation in the residuals at level one and significant variation the intercepts again just by virtue of comparison just I'll run the same model through SPSS with SES incorporated as a fixed predictor and you will see then that in this case the wold Z values fifty six point six four two fifty six point six four two and six point four two two and six point four two two so exactly the same so that you know so we still that's indicating then that we have significant variation at level one and level two that may be accounted for by adding in additional predictors if I want to follow up with the intraclass correlation coefficient I'll just type east at ICC and you can see now the ICC is at point zero five two so in other words after controlling for socioeconomic status you know our clustering effect actually is decreasing so basically 5.2 percent of the variation in math achievement is occurring between schools after we control for the socioeconomic status variable so and then you know again if I want to model or use syntax let me go back here then essentially this is what I would do so you can see it says mixed math is a dependent variable and now I'm adding in the name of the predictor variable the fixed predictor variable at level 1 which is SES and then we have the school code as as our level two identifier so rather than just typing in and I'm just going to go ahead and copy and paste and so there you go there's our same information as what we had before now let's say we want to model incorporate a level two predictor which is take the school mean for those students and as a predictor of variation at level two so in this case when we add in that the school mean on SES we're essentially modeling variation in the intercepts across schools for modeling variation the intercepts which are again adjusted means on math achievement and we're modeling that variation so so we're indirectly then modeling variation in individual level math achievement through the study of variation the intercepts so to do this in this case we're just gonna do it through both routes again just for illustration purposes and so now where it says independent variables I'm gonna click on SES mean and there you go and so now I'm gonna click on OK I get the coefficients and you can see right here we've got SES mean so that's a level two predictor and SES is our level 1 predictor so you can see that both coefficients are positive and both of them are statistically significant in the model so and so the level 1 predictor SES we are basically infer that student level SES is a significant positive predictor of math achievement and then school level SES is a positive predictor of of math achievement as well through the as it's positively related to the variation in the in the intercepts ok so in terms of looking at level 1 and level 2 variances again we can again test whether these are statistically significantly different from 0 and so we'll just plug this information into our little calculator so there's the level 1 variance standard error level two variance estimate and the standard error and so now you can see that you know both are still statistically significant so there's still variation to be explained at at both levels of compute the ICC and you can see now it's actually drops to point zero three eight so basically there's still there's still clustering effects that are unaccounted for by adding in the level one level two predictors so at any rate there there you go now let's try another example where we have a fixed level two predictor with a rather fixed level one and fixed level two predictors with randomly varying intercepts and slopes so we're basically building on this model but allowing the slopes for SES at level one to randomly vary so in this case using our menu option find it again there is in this case what we're going to do is we are you know we're allowing we're going to maintain these independent variables as it were but now you know this equation right here this is allowing random variation in the intercept so if I want to add random randomly varying slopes then I have to create create the opportunity to do that so I'm going to click on create go back put school code in right here and then the independent variables in the equation I'm gonna highlight SES because I want to allow the slopes for SES to vary across the groups notice that we have a couple of options you know in this particular case if I stick with independent what that translates into is that you know we had the variance in the intercepts if you can kind of imagine a variance covariance matrix where we have the variance in the intercepts and then we have the variance in the slopes and I'm denote variation slopes as mu I should say one J right there so we have the variance in the intercepts variance in the slopes if we leave it as independent then the covariance between those those two coefficients is zero if I specify it as unstructured then what that translates into is that we have the variance for the intercepts Barents for the slopes and then we have the covariance between those two which is mu o J mu 1 J right here and then it's repeated up there and our covariance matrix so at any rate things can get a little bit more complicated if we start allowing random variation in the in the slopes because then we have to make a decision as to whether we just focus on the variation in the the intercepts and in the variation in the slope or do we also build in the Co variation between them so if we leave it as independent what's going to happen is that when we run the model you know you'll still see that we have down here we've still got the variance in in SES and the slope for SES we have the variance in the intercepts and then we also have the variance at level one so again this is the variance and the epsilon IJ s then we have the variance in the mule J's oops and then we have the variance in the mu1 J's which is for the slopes so we have then a variance estimate for the slopes intercepts and again the level 1 residuals so here we can test this out again same way by using our little calculator so at level 1 and then the intercepts variation the intercepts put in here and and now we have the slope so now I can't I can put in the variance estimate for the slope there you go so you can see then that the slopes appear to be exhibiting significant variation across school contexts so what that you know what that suggests then is that not only do the intercepts vary but now you know we have different slopes so in other words just kind of to give you a little bit of a visual of what I'm talking about you know if you imagine that you know each of these lines has a slope associated with them and they also have their variances so we're talking about the variation and both and so if there's significant variation in the slopes and the next question is why what is it about the school context that might be producing variation and so then we might add in level two predictors of that variation and that's where you move into modeling cross level interactions so you'll notice that in this particular case again we you know this is essentially by the way this is the syntax for running this particular analysis so again but it's kind of plug this in down here you'll notice that that there's no specification concerning the covariance matrix for the coefficients the default is essentially a variance components an analysis or approach which you know if I was an SEO in SPSS what that would translate into is that I would have again there's mean here I would have my affects my fixed effects there and then the random effects would include include SES and then the the covariance type is variance so you know if I run that analysis you know these are the the you know essentially the same values that what we had when we generated a results using Stata so that is essentially using variance components I might also choose an unstructured covariance matrix which would allow the estimation of the covariance between the slopes and the intercepts so if I'm doing this through through the menu option and Stata essentially I would go to equation two down here and where it says and click on edit and I would move down and click on unstructured so then when I click on ok and then on ok here then you can see that now what is estimated I have the variance for int for SES so that's the variance estimate for the the variation in slopes and the standard error variation in the intercepts so right here and then we have the Co variation between the slopes and the intercepts and so you can see us- and what that basically means then is that you know in in those situations where basically that the that that there's a negative relationship between the slopes for SES and the and their intercepts so without going into too much detail I will just illustrate this again using my little calculator and and we can test whether or not that covariance is statistically significant so I'm just gonna incorporate each of this information here there's a variance of the intercepts there it's for the slopes and then excuse me the variance of the intercepts and standard errors for the intercepts and we have the slopes right here standard error and then we have the covariance and so what I was saying earlier was that this test is actually going to be a two-tailed test because we're unlikely to really have a a priori hypothesis about the covariance between the slopes and and the intercepts so basically you know what the negative coefficient would just mean that in schools where that has that have a higher means the slopes are more negative so there that's basically what it means so you know again by way of comparison with SPSS if I wanted to do the serious SPSS using the unstructured option you know this is it right here and so you can see you've got the variance for the intercepts that's the first coefficient so then you've got the variance for the slope the second coefficient and then you've got the covariance between the the two and so these are all the test values so once that's basically done once we've kind of identified what type you know it whether or not does if if the slopes are varying across the school context then we might want to build in predictors of that variation and so that's where we can model cross level interactions so in other words you know the idea is is that you know if if it's the case that you know are our slopes and you know are varying across school contexts we want to know why what is it about the relationship between our school contexts that altar is a relationship between SES and math achievement and so then we can build in cross level interactions using the state of program so just kind of I've already actually run this through Stata and what I thought I would do is I'm just going to take this syntax right here and and plug it in to this box the command box down here and so and what I've done also I actually ran it you in the unstructured matrix and we end up with problems with model convergence so so allowing the Co variation between the slopes and intercepts produced basically in an invincible solution so I in this case what I used was a covariance type in it which is just essentially the variance components and so at any rate that's just kind of an illustration there were some problems with collinearity and so forth but that was just kind of the highlight the basic idea and so you know what we were basically testing is if there is a cross level interaction then that would signal that there are some certain contextual factors that may be producing variation in the slopes between SES and math achievement so that's that's just a really quick follow up I didn't really intend to go too far in into that but that's just to highlight a typical follow-up to the question of you know it's a slopes of randomly varying than Y and what is it about those school context and so if we model cross level interaction that we're looking at how contextual factors are altering the relationships observed at level one so that concludes this this illustration and hopefully you find it useful

Channel: Mike Crowson

Views: 41,051

Rating: 4.932961 out of 5

Keywords: STATA, multilevel model, HLM, school, achievement

Id: ZI8d1EPeIvU

Length: 33min 20sec (2000 seconds)

Published: Fri Feb 09 2018