SEM Boot Camp 2018 Confirmatory Factor Analysis

I don't think I understand the question I'm not familiar with this model okay alright there we go let's see yeah sorry my knowledge is limited okay I think I've heard the name before but yeah any other lingering questions from yesterday for EFA or anything about data screening yeah I I wouldn't if you have a bead you on there this does cooks distance yeah there's a video on cooks distance I would show it today tomorrow tomorrow will show it but if you want go to youtube and type Gascon d and oh yeah it's in this one SCM series 2016 but we'll do it again today as well okay yeah oh yeah sure yeah real quick follow up on making correlations in Excel I'll just bring up this stats practice data I put this in the data folder let's see so in correlation let's go over here so a bunch of data just organized as you'd normally organize it in spss how we can do this with our SPSS data as well but two different ways you can do equals Correll with two columns like this one and this one and I'll give you the exact correlation start and zoom the way out okay so equals Correll and give it a couple columns and you do it with two or if you want to do a full-blown correlation matrix you need the data analysis toolpak this thing do you need to know how to get that so you're asking you okay so once you have it then click on it and use correlation here correlation and hit a hide yeah okay and then the input range is just all of your variables you're gonna correlate yeah so I'll just I'll just grab all this here control shift over and down and then I'll say that I have labels in the first row give it an output range just where I want to put it let's just put it up here okay and it'll it'll produce this correlation matrix for you and now it's recorded in the video if you ever wonder on the video we are recording right now it's just a correlation matrix you can do in any program you want as long as the formula is correct I assume it will be oh yeah that's nice and easy to hit the one in SPSS is all muddled with a bunch of extra rows and columns and it's messy this is great especially if you then go like this and go like this and go like this there you go that's nice yeah and you can conditionally format it to just show you the big ones I give me the top 10 items yeah in green and okay oh we should ignore those oh well you see the big correlations there okay any other questions from yesterday yeah [Music] do does it make sense to do correlations on the package for us and the answer is yes let me open up that dataset real quick you don't want anything over like point eight - yeah because that would imply that they're really the same thing that's good that data folder so everything from yesterday is saved and I've saved one additional thing last night which we use later today it's the EFA pattern matrix there's the final pattern matrix from our final esa that we've conducted but let me open up it is on the Google Drive folder that's where it is it's in the data folder yes the final EFA yeah so let me open up this data set from yesterday and this is the data set will be using today the 2018 clean take moment do little dance waiting it's so slow SPSS we do have all day did anyone do something awesome last night in Provo aside from go to the Bombay house this is the actual name of the place I didn't make this one yeah it's good Indian food in free advertising yeah all right so um yesterday we produced these these variables down here these factor scores which oh I ran another EFA it produce factor scores again let me click those earlier so we produce these right here and we named them and so we can now use these in a correlation matrix if we want I would go to analyze correlate bivariate and then go to the very bottom and we can get all those variables we produce using factor scores throw them in here into the correlation and hit ok and this is the ugly correlation matrix that comes up with but we'll see that it is the same if we were to correlate these over in Excel you get the same values but they're valid if we look in I'm just gonna copy this out this kind of just the wrong size here Excel Word ok and opposed paste this in here here we go we can see correlations are significant a lot of them but none of them are like massive which is what we're looking for we want significant non massive massive would be anything over 0.82 don't ask me why but that is the right number yeah that looks pretty good so they were yeah these are factor scores imputed from the EFA yeah the correlations will be the same just different format in the end oh we've never done this okay yeah yeah doing a correlation matrix in SPSS joycie again real quick oh you don't know me doing that ok this is an important thing ok we can go over that again real quick um so you're good you're good ok so you go to do your factor analysis just like before analyze dimension reduction factor and throw in all that stuff we had which is all this we don't need to do all the usual stuff we know it's a good we know it's a good factor analysis so we don't think we need to do is make sure we have Promax on and in the scores this is the part you missed and the scores click on save as variable and what that'll do is for every column in our pattern matrix it'll produce a new variable so we'll have eight new variables and each one represents the factor that is loading most highly on that if you want to you don't have to actually this is just notch and if you want to then do a correlation matrix with your variables or check your like multicollinearity or something like that for your regressions final parameters not the exactly don't save fact don't save as variables throughout this whole time because you'll be saving variables every time you run the EFA we ran it probably about 25 times yesterday so you end up with a lot of factors it's not required so in during the EFA you don't need to produce a correlation matrix it's important to look at it in the EFA so if i were to run this link uncheck this for a second create more factor scores but in your EFA zoom out ah in your EFA let me run this it's important to look at the correlation matrix at the bottom just to make sure there aren't any strong strong correlations above 0.7 but it during the CFA will produce your final correlation matrix it's the one you actually report yeah the plugin will produce it but you don't need to report this correlation matrix because we're just going to produce a final correlation matrix during the CFA so there's more of an FYI okay anything else from yesterday let's say we do get that final patent matrix and we decide I don't think I understand so you see how we're not the correlation matrix II how the correlation to up there yep yep will it do the same no it's not gonna put those up there and every time you do a pattern matrix the correlation matrix that comes out just uses numbers as the labels yeah anything else before you confirmatory factor analysis okay so yesterday I went through the slides for today I updated them a bit but honestly we're not gonna spend a lot of time in the slide sort of like yesterday they're here but we're just gonna be in Amos most of the day I'll cover a couple things real quick CFA is confirmatory factor analysis has anyone here never used Amos before great this would be a painful experience Amos Amos has many student names which I will not repeat in this classroom with a passion you'll see in a moment Amos is not user-friendly but it's more user friendly than any other user interface for statistics out there which is sad anyway so we're gonna do confirmatory factor analysis in Amos today and it's good for doing a ton of stuff in fact the confirmatory factor analysis is the longest portion of your of your analysis you do your data screening is pretty quick right check normalities and outliers and stuff do your EFA that could take you a few weeks and then you do your cfa and just where you get stuck in the rabbit holes and you're not sure if you're doing it right and there's so many little little things so we're gonna cover all those little things today and hopefully set some solid guidelines down I added a few slides in here that give solid guidelines I've also updated the stat wiki include these guidelines and now we're gonna have it in video form today so we'll be good well we're gonna start with is opening Amos and cringing a little bit so let's do that open up Amos if you have it Amos graphics is anyone using a version earlier than 2122 good oh you are what version yard oh yeah 25 is good oh you don't need to have this join when you start this is what we're gonna end up working with what a mess yeah it should intimidate you all right it's open up famous and make sure everyone on this end is doing good yep good remote side who will be following along in Amos just so I know how fast slow to go most everybody okay you try we have time at the end of the day we should just do your model in Amos that'll be cool type search - Amos practice in a space was it mr. doback just type Amos Amos I just there it is in us hopefully if he gives you a license error okay so hopefully it works it might be limited I'm not sure how many variables or sample size it allows anyone else we got a Miss okay okay this is Davis um it is you have your canvas here you have all your tools over here on the left and the tools are kind of fun like a firetruck here a magic wand and this set of balloons or something and it's candelabra and I'll show you how it all works with it um in Amos we can use Amos for a couple things one is testing a measurement model like with CFA the others processing a structural model like testing your hypotheses so in order to test your measurement model your CFA you have to build and don't do this with me just watch for a second you have to build your model control Z which involves creating latent variables with indicators manually and rotating them don't do this with me just watch and then bringing your data into them and naming them so like this would be I use fulness or something like that and you'd have to bringing all your variables and all your error terms which is stupid it takes forever so there's a plugin for that they ain't nobody got time for that you race off there we go um we need to get plugins for several people do this go to stat wiki oh no you don't even need to do that it's in the folder here they are go to the SEM goo camp folder and go to estimates and plugins and if you're in version 24 or 25 use the version 24 or higher folder if you're in version 23 or lower you're the 23 or lower holder and we have a bunch of plugins in here the one you need for right now is pattern matrix builder this thingy you're gonna just copy it and then you're gonna go find this super hidden location to stick it in so the hidden location is open up a new Explorer folder that windows e to do that or or just go to your desktop or something like that if I find a folder you need to see the folder tree is what you need and you're gonna go to this PC operating system see OSC or local disk C and then users and then your user name mine happens to be J Gaston don't ask who Apollo is J Gaskin and now we're in a folder it's not gonna look too the same for everybody this is the hard part you gotta go to the view spot go to view and check this box for hidden items we want to see hidden items so check that box and once you do that you'll be able to see all these other folders including one called apt data click on app data and then local and then your yeah we got unblock it in this egg yeah local and then Amos development and and Amos and then your version number for me it's 25 and then plugins how we made it and then paste it in here and once you've pasted it in here go down to it and right click it go to its properties and if it says unblock check the box for unblock I don't have an unblocked here cuz my security rules are very lacks but if you have a block or unblock make sure you check the unblock yeah check unblock if that's an option and then hit OK and now in Amos check to see if you have this plug-in showing up it should be called pattern matrix model builder is it show up for you ok we'll do it again I went through a lot of steps there manually okay oh you totally build it manually and for today's at least I'll save all the models I produce and so if it's going too slow you can just pull in my model that I'll make so do you see the pattern matrix better yeah restart Amos and that might be okay for those who can follow along as I went really fast here's again how you did it go copy the plug-in right right-click and copy from the data folder or from the bootcamp folder and then go find this location it's operating system or local disk do you have that you're on a Mac yeah yeah like like a you can usually get students citrus does anyone have a Mac and they're using my plugins alright for those remotely as well if you're using a Mac and it's not you know I'm Amos on your windows side if you Mac assuming you have a windows side of your Mac I don't think the plugins that worked actually or if you're using Citrix server I don't know it'll work so you'll have to follow along let me show you quickly manually how to do this I'm not gonna build the whole thing but you get the right idea but I'll save my models that I create as we go along so I'm gonna do everything we do today I'll do it the old-fashioned way and then the new way that'll only go a little bit down the old-fashioned way just to show you how to do it yeah he got it fixed yeah yeah if you do run into issues with the plugins there is a troubleshooting guide if you go to stat wiki and go to the plugins page you can find a troubleshooting guide so this is plugins let's see plugins info right here click on that go to troubleshooting and here's how to fix every issue that is most prominent pattern matrix model builder all your using Citrix one that's why hmm I'll check it out later so for now I just have to follow along watching hello you found the plugins folder nice yeah what yeah once you get it in there are you gonna right-click it and so you're right click it go to properties stop ah right click go to properties and then at the bottom there might be an unblocked checkbox if there is check that box and then hit OK yeah and then hopefully it'll show up in your Amos plugins if not restart Amos it should be there ok ok let's go back to Amos oh you want to see it again that's right sorry so to find the location so the location is operating system C local disk C users and then your user name and then you have to view hidden items so go to the View tab up top and check the box for hint items and then you should see a folder called app data open that up and then local and then Amos development and then Amos and then the highest number there and then plugins and there you are they hit it real good they even made it a hidden folder it also changed the structure around the plugins so that my old plugins don't work on the new versions of the software we had to recreate all the plugins it was it's easier I don't know if it's easy but yeah it's easier for sure I remember I was sitting in a class of stats classes using Amos and we were doing everything like I'm about to show you the slow way stupid gotta be a better way all lots of difference between EQs neighbors but the algorithms are almost the same Amos uses maximum likelihood whereas EQs uses what's it called a different one I can't remember cheek grazing but would come up with roughly the same results okay so if you need to build the model manually here's how you do it those with the plugin don't don't bother doing this but those who need to build it manually what you got to do is look at your pattern matrix which I saved over here in the data folder if you go back to the data folder in chat let's see if anyone needs help getting the question where can move please feel free to contact me yeah and there's a troubleshooting guide for it too okay the pattern matrix is this SPV file in the data folder so you'll want to open that up it's just our latest EFA run this is what you want to see and you're gonna build your CFA based on this those with the plug-in hold tight for a sec those without the plug-in follow along if you're following along it's the way you do it is you want to convert convert you're gonna transfer every latent variable over to Amos so everything variable is every column and so the first column is usefulness and so we're going to put usefulness 1 through 7 in aletan factor over here so we click on the candelabra just click don't click and drag and then click and drag here to get the right size latent factor and then it had seven items seven oh that's eight whoops and then oh we got to pull our data in as well so everybody has to do this even if you have the plug-in pull your data in goes like this go to file data files things gonna be understandably slow as we get into the new software so file data files and then click on file name to go find your data set the data set I'm using is the clean 2018 data set this one boot camp original trimmed 2018 clean oh I'll show ya okay so it's this candelabra if you just if you click click click click click click just keep clicking on the latent variable yeah okay so we pulled in the data that's file data files click file name go find your data and then pull it in and then your data will show up in this white stachy thing which is the list of variables in the data set this white stachy thing if you click that and you'll just pull variables click and drag pull it into the observed variable box oh dang some of you may be experiencing this where pulls in the entire label let's fix that you would go to view interface properties Y that's the default go to miss miscellaneous tab and uncheck the display variable labels so that's again view interface properties miscellaneous tab uncheck display variable labels okay and then it'll just put the variable name in there we got to put useful twos in here and three so you manually drag these out one more time imagine if you have a hundred variables yeah that's one factor we also need to rotate it here's a rotating tool right here under the firetruck Lake who is doing this manually okay slow me down I'm gonna do best once you have a rotated you want to double click the latent factor to name it this be useful its variable name we getting Joe how do I have one called useful already well we'll find out fine I like living on the edge okay and so essentially you're gonna do this for every factor in your data set I'm just gonna pretend I have factors here you move this over here the way to move things by the way is with the fire truck and if you wanted to move all together you have to use the balloons to keep it together is the balloons yeah so that moves things all together if you don't have the balloons checked it goes like this is not what you want and then so let's pretend we have a few of these that's not the one this is the one let's pretend we have a few of these Rosie once we have a few of these you'll want to Co vary all the dependent all the latent factors and so you can do this you can do this using the finger selector and clicking each latent factors so they're blue and then this plug-in should work because it's a built-in plugin to drop covariances as plugins Drago variances those who are doing manually do you have that plug-in are you can'tyou the data file in well bang what are using a citrix server version puppy all right give me a sec okay walk around those who couldn't get the plug-in to work for this you can actually just draw these covariances with the double-headed arrow you're gonna click and drag from one to another yeah click and drag okay [Music] [Applause] [Music] you click and drag so no you're good so there's click and then there's click and drag click would do this click and drag lets you set how big it's going to be so click and drag over here but this is a siave - all the data so those were doing this manually you see why i create a plugin before we move forward and i save a new model that you can just download for yourself do you have any questions before I move forward okay then I'm gonna use the plug-in and make a model that you can just download so you're not to build it all yourself okay yeah okay so I'm gonna delete this real quick the plug-in allows you to take the pattern matrix here and just paste it into Amos and it creates the model for you so to do that assuming you have the plugin that works you right-click the pattern matrix and copy and then over in Amos assuming you've pulled the data in so file data files and you have the data set already in here assuming your date is in here you can go to plugins and then pattern matrix builder paste your pattern matrix in there just hit create diagram and it creates it all pretty and symmetric for you know just negotiating with terrorists or with third-party developers or whatever we're called yeah yeah no interested in implementing this at all okay so that does that once you do that though you still have to rename some things and move things around so if you've done this if you have a model I strongly recommend you rename your variables by double clicking them and changing their variable names so they're not numbers so this would be useful and then this one would be anxiety playful you know we already have factors called these don't we from the factor scores so we're gonna have to name this like playful X or something like that anxiety x useful X so it's a new unique name decision wall X Y as we name those factor scores these things already so since we're I have an observed variable named anxiety we can't also have an observe variable name anxiety Amos gets confused info acquisition X and comp use X use thanks so Dec D asks this is positive pause X and social desirability neg X I'm going to save this so anyone who's doing it manually you can just use this file let me just do one more thing I strongly recommend when you start your CFA to move the parameter constraint you see this one here and one here one here there's it has to be one parameter constraint per latent factor in order for Amos to run it part of the algorithm it requires that value to be constraint but you can move it around I recommend you do move it from a path up to the latent variable level so I'm just going to double click the latent variable go to surrenders add a 1 and click here 1 1 that's why it in order for Amos to run it has to have a parameter constraint somewhere on the latent variable whether that's on a path and I'm thinking on these paths and deleting that constraint or whether it's on the latent variable itself because if some of the analyses we're about to run it makes more sense to have the constraint on the latent variable not on the path is we need to compare paths you can't compare a path when it's constrained to be one so we move that constraint around and then while I you're ready to roll so I'm going to save this so everybody can use it let's see or so everybody who wasn't able to use the plugin can use it I'm gonna save this in the models folder demo models and I'll call this a CFA initial 2018 okay that model should be available for you you'll probably have to reset where the data is coming from so once you open it up make sure you go back to file data files and relink the data if if you need to whoa using Amos is exhausting and the sad thing is it's still better than the alternative so it's an improvement okay I did that model comes through for anyone who's trying to use it it's called CFA initial 2018 it should be easy it's in the demo models folder and it's called CFA initial 2018 yeah it'll say it's an audio file because your computer doesn't recognize a Mis files yet like I said it's gonna be a little slow going as we get started but once we get going we'll be able to go a little faster this is why we spend the whole day on CFA though it's just getting to know Amos we're recording right now so I don't respond to that do me good is he bringing each other all those miss a two-way arrow between everything in fact it's pretty slow and then also all the error terms need to be named which you can do with the plug-in you can do plug-in named unobserved variables and that one name all of your errors consistently as e1 e2 e3 to make this it was about 350 ish not it wasn't too long we should put the variance constraint on if we should for for metric and variance okay how we doing did he come in okay we good we good ish okay all right moving forward so this is the CFA once you get here there's a heap of stuff we can do um first off make sure you're good and saved make sure it's saved outside of that shared Google Drive folder because I'm going to be manipulating the one in the Google Drive folder but the first thing you want to do is click on the colored abacus which is right here because that makes perfect sense colored abacus what this is is it's the properties of the analysis we're about to run so first thing I'm going to do go to the output tab and do a little TurboTax here we go we want standardized estimates in that output tab if analysis properties output tab standardized estimates and that's probably all we need right now actually you can keep minimization history that's fine and then there's no ok button you just hit X because they don't follow conventions you eye conventions and save again and we're gonna run this model the way to run it is with the non colored abacus right here under the magic wand just click on that it thinks for a sec tells you no good you have no license I have our license give me a sec I'm just gonna close a Miss reopen it a Miss gives us all sorts of weird errors one of which is you have no license might be it does but I ran this time no problem I have a license you shall pass I click in the code abacus I went to the output tab and then just check the box for standardized estimates yeah and then just click the X and then you click the unstandardized or not you click on the uncolored Appius black and white app because under the magic wand and then it runs and nothing happens so you have to click the up arrow to make something happen there it is the up arrow up here up top so this up arrow displays data on the model and the data it displays is unstandardized so we actually also go down here and click on standardized estimates instead of unstandardized and that's gonna switch what's displayed to be standardized and let's walk through what all this is way about chat real quick let's see I know I asked this before at least twice but I want to make totally sure for the CFA we include moderators and control variables that are on a Likert scale latent variables yes that is correct yeah in the CFA we include everything that was in the EFA and what was in the FA all of our reflective latent factors whether they were workers or moderators or mediators or dependent variables or control variables if their latent and we're gonna use them in a latent fashion and the reflective they belong here in the CFA do formative factors belong in the CF age we excluded them from the iya thing in the CFA it's also built on the covariance matrix and so no Forman factors do not belong in Amos at all because Amos is a what's called covariance based structural equation modeling application and Forman factors aren't built on the assumption of covariance okay so let's look at what this all means if you zoom in just by scrolling up you'll see that the paths here have regression weights displayed we want these weights roughly averaging out above 0.7 sort of like in our pattern matrix and then the covariance is here these are actually correlations not covariances there's a slight difference it's the square anyway we want the correlations here to be not enormous I'm not above 0.8 and we can see the strongest correlation is this one here 0.69 that's pretty high all the rest are pretty good so the reason you don't want them high is because you don't want to have what's called discriminant validity issues you want to make sure these variables are unique the same thing okay but that's just a rough look what we need to do now is we need to assess whether the model is good or whether the factors are valid and whether we can proceed with a causal model to do that in correlation yeah if you have a point correlation either you're experiencing two factors that are actually measuring the same thing or two factors that are measuring two components or dimensions of the same thing so it might actually just be a second-order factor these are just its dimensions oh yeah um so first things first if you don't know where to start go to the stat wiki that's why I made it and go to general guidelines order of operations go find CFA here's the CFA what do i do first obtain a roughly decent model quickly check model fit and validity okay let's do that for those we can use the plugins there's a plug-in that does all of this for us but let's start manually manually if you uncheck validity essentially you do what you just did you look at each factor make sure it has roughly averaging out at point 7 or above loadings and then make sure to code correlations aren't 2i you can also use a stats tool that's a require you could do this as well do this if you can't use the plugins go find the stats tools package excel file which is also in the SEM bootcamp right here stats tools package xlsm and make sure you grab that if you open that up we've automated some things well is it's a slower automation but you can still use it if you can't use the plugins no I already have it open Ross there it is when you open it you should see something like this bunch of tabs bunch of tools and you want to go to the validity master tab it's the first tab how many you're going this route so I know how best to go okay only if you're not using the plugins if you use the plugins don't do it this route there's a better way okay so what are you able to find the file we're good not good mine you sir are you able to find the file you sir because you find are you doing it this way yeah you found the file oh good you get very good that's what I'm here alright now go to your model click on the output which is not that one it is this one it's right next to the save floppy disk is a view text and it's gonna pop up this thingy hopefully and then you want to go to this is all the output for the model you wanna go to the estimates area and the scalars eggsy produces regression wait stan has really experienced correlations and variances yeah that's assuming that's really tiny okay estimates scalars and then all this stuff well we want in the stats tools baggage is exactly what it says paste correlations table in a two so we're gonna go the correlations table right here by clicking on correlations you're gonna left click outside of it and then right click to copy this table then you're gonna go paste it over here like that I'm gonna do the same with the standardized regression wait stable and f2 standardized regression weights copy it paste it over an f2 once you've done that click on this button and this button runs a bunch of stuff for you and produces a correlation matrix and as well as the composite reliability score the average variance tracted score and a few other things that we're not gonna pay attention to here's the correlation matrix as well as the square root of the AV e on the diagonal so this is a standard table you need to stick in any quantitative article [Music] [Music] [Music] [Music] all right sorry folks [Music] [Music] [Music] [Music] [Music] okay so what is the statuses of will how we doing that is good when it works so let me explain we have here what we have here is a brief assessment just sort of quick overview of the validity of the model of factors so we have all the latent factors here and there CR which is like cronbach's alpha which you went above 0.7 so all of these should be about 0.7 if they're not they turn red and so social desirability positive and negative oh it's red they're also both pretty close so I'm not too worried it's a marker variable it's not a key theoretical variable so I'm not too miffed about that hey ve should be above 0.5 again all of them are except the social desirability so not a big deal the square root of that a ve is on the diagonal here and this has to be greater than any correlation with another latent factor so for example the square root of a VE for social desirability neg X is 0.55 to 3 it's got to be greater than its correlation with any other latent variable here and it is so we're good this implies discriminant validity CR implies reliability a B implies convergent validity so we're actually pretty good why are we so good it's because we did a good EF a if we hadn't spent time in the EFA this would have been a mess point 5 pretty low so it's a little questionable but you could try to write it off say it's just a market variable anyway but if it's a little low yeah point three yeah it's a little low but write it up as a marker variable yeah okay so that's validity you also got to check the model fit roughly only two factors that doesn't work as some of the squares I have a tool that'll do it for just two yeah yeah also do the plugins work for you or no no plugins don't work for you okay so that I have a tool for you if you go if you only have two Layton factors go to the stats tools package old yeah yeah it'll do all the calculations for you and in the sad sauce package old go to the validity tab and then end to your standardized loadings here and your correlation here so let's say we only had only one correlation it still produces CR and a VE for you you're welcome okay also those who are using the plug-in this is how it looks better plugins validity and reliability test and it'll think about it and produce this same findings but it also gives you recommendations this one so it says well these two are having problems you might consider dropping social desirability four and ten why would it suggest that so we go look here it's such a desirability for at the bottom here is men go down there's ten and four here's ten a loading of 0.36 and here's for loading a point four eight so those are the two now we're not so concerned about our mark variable being super strong and valid it's really just a tool for method bias which we'll talk about later so I'm actually gonna just leave these as is right we could have chosen one or the other but two is better because it makes it a more robust tool for extracting just method bias rather than shared trait variance so the more the larger your marker variable the better and let's say if we remove four and ten you're still good it you're making it less powerful but it still should work oh it's in the folder it's called yeah model validity Oh mine's called master validity master validity and the most recent ones called Amos - master validity so Ryan if you were looking for the newest version weird okay okay yes pressure you should be the same okay so that's that we also have to test the model fit roughly just to make sure there's no major issue with model fit oh sorry there's text chat let's see bird [Music] what's the deal with the P next to the D F is it supposed to be a point oh we haven't gotten there yet I'll tell you in a minute if you remove four or ten do you have to go back and look at the model fit or even go back and redo the EFA you so what this is a really good and valid question um let's say we remove items during the CFA do we have to go back and redo the EFA answer is no we we're done with the DFA it was a tool for exploring our factor structure now that we have it we're good we're done we don't have to go back to DFA at all okay my heart is skipping a lot let's run through this last bit and then it'll take a two-minute break every time Rob takes me ma my heart skips you make my heart flutter Rob we get the means and there it goes again man my heart just skipped again okay sorry Rob area we also have to test model fit so you can do this by looking at the put again in the output if you go to the model fit section it's not in the estimate section it's below it down here called model fit over here you get a bunch of output since we're just doing a cursory glance right now we don't have to like do a full analysis here's what you're looking for Rob to answer your question the p-value for the semen which is the chi-square it should be above point O five but it is a really really really strict measure of model fit and highly biased by model complexity and sample size the more complex your model meaning the more variables that are in it and the higher your sample size the higher your chi-square look at my chi-square it's 1,500 it's pretty big and so the p-value is very susceptible to deflation due to complexity in example size so we may never reach a point oh five with this kind of complex model so this variable this metric has since lost credibility in the literature and we don't typically use this we report it we say but look at the CFI the SRM are in our BCA which are considered the more recent and again this depends on your field this is in information systems and more business school disciplines other disciplines like that like the NFI the tli some of the other older measures but in business we use the CFI which we won about 0.9 ideally about 0.9 50.9 will do and then the if we scroll down yeah GFI the old ignore GFI look at rmsea it should be less than depending which literature you cite less than 0.06 this one is so we're good and the P close should be greater than 0.05 so we're good this model is actually pretty good it's not perfect but it's good in fact if we wanted to make it perfect which I don't recommend by the way we're not optimizing mother but if you wanted to see where could I improve it you can click on this modification indices option here in the analysis properties I'm gonna bump mine up to 40 because we have a complex model sorry I went faster I'm just showing you you don't the following with this part because I'm not recommending it I'm just showing it but if you look at the output now with modification indices run there's a marketing modification indices portion and it says well II nineteen eighty twenty are really related so our e3 and e4 we go back and look at the variables there nineteen twenty three and four here's three and four right here it's useful three and useful for if we were to go look at the wording for useful three and useful for to do here is useful three let me zoom in using Excel in my work increases my productivity using Excel enhances my work effectiveness those are the same thing of course there are there now what do we do let's say we didn't have model fit as high as we wanted and we needed to improve it well we have an indication here that three and four are causing a problem with model fit they're too strongly strongly related and yet they are not connected directly with the line the best approach when you have this many items on a reflective Leighton factor is to simply lock one off so I take the one with the lower loading which is here's ninety three years ninety one I'd get rid of three just take it out of the family but since we already have good fit we're good we don't have to do that don't try to optimize your model you want to err on the side of not removing items if you can help it I put it back in yeah if if we took one out here for those who can here on remote if we took out three here and our model fit didn't improve I just stick three right back in the other one was 23 and 24 right I think so decision quality 8 and 9 which are no you don't have to rerun the matter matrix we're done with the EFA no you can just add them manually like that I use the candelabra I'll show you I'll just show you one moment you're good yeah Excel helps me make higher quality decisions accepts me in my decision analysis process so these are sort of the same yeah let's take one out here I'll show you if I were to take one out using this x marks the spot thing after hitting the down arrow because you can't mess with the model while the up arrows up click on the X and you get rid of three I click on the three itself and also get rid of the III which was connected to it let's go look what happens to our CFI it was what nine three seven a nine to seven or something that model fit see if I is now nine thirty so it was night two seven before us and I guess um I really didn't make much of difference if I want to add usefulness three back in what I'm news I mean click on the candelabra here and just click on usefulness again and it creates another one down there so it created a new empty one down here yeah and then I just put use for this three back in use for this three oops too big usefulness three put it back and it's ugly but it'll do and we name this whatever it was e3 okay there's that so we have a good model that's what we have to do first just check make sure things are valid if they're not address them how do we address them I the the model fit or the validity master plugin tells you how to address them but essentially if you have discriminant validity problems what you need to do is try to try to find a way to decrease the correlation between the variables that have the discriminability problem for example let me run this again at the up-arrow yeah a barrow standardized the highest correlation we had was right here zoom in right here between decision quality and information acquisition there are two ways we can address the discriminant validity problem one is to try to increase convergent validity because as we increase the AV e our correlation between two factors can be higher and still be good because what it is discriminability we go back here it's just the comparison of the square root of the AV e to the actual correlations so if Rab E is higher we can have higher correlations and still be discriminant so one way to do that is to drop some of the lower loading items like information acquisition v if I drop this my AV e would increase probably and then that correlation between these two is assuming it didn't move would now not be such a problem the other option is to take these two into Excel like iron onto Excel into SPSS again and run EF a with just these two and let all the cross loadings show and see where the biggest cross loadings are and then try to pull them apart even further by removing the items without greatest cross loadings and that will decrease the correlation between these two factors and then change the CFA to match that new those new findings is that questions about that I sort of glossed over that quickly good discriminability problems trying to decrease the correlation conversion deleted problems trying to increase the loading average value I mean one way to do that is to just drop the lowest loading item so there you have it yeah and the modification indices certainly would show some of the ways so you just don't address it so for those on the remote site who can hear the question was how much do we try to fix an apple when we're just sort of doing a rough high-level pass before we go and do complicated things like scaler and variants and method bias all that junk the answer is unless I see a real red flag I don't do anything now if model fits good enough validities are close enough I don't mess with that and most of what you do in the EFA really sets you up nicely to have a clean CFA so that you don't have to mess with it much you'll notice in this model that we have we didn't have to do any adjusting to reach a good fit or good validity I'm sayin this diagram like a new report yes oh no no correlation matrix with the CR and the AV e that's sufficient and model fits you don't need a picture of your model the exception to that is if you had a second-order factor it would be helpful to see that okay yeah if you had a second-order factor because because that's a little different and visual would help in that case tables a little harder to read second-order it could be a cuts in the measurement model as well yeah yeah I'll show you that in a little bit or if we had abused Marie's model later we can do with hers all those little log files and block files and know it's gonna clutter up your folder yeah Amos creates a bunch of remnants as well as little artifacts you can delete them though the only only files you do need are the a MW file an essay P file yeah okay anything else before we start doing crazy hard stuff because this was easy yeah here is the second sample range of stage so I think that my trouble is that okay so for the remote folks in other fields and even in business the more rigorous approach is to do an EF a with a random sample of your data set and then do the CFA with a different random sample of the data set or a separate data set entirely same variables just different people different respondents I that's considered more rigorous unfortunately in many cases the Liberty to do that is not available because you don't have enough data and you don't want to run it with let's say you have a hundred in your sample size you don't want to cut your sample size in half because then you lose stability so in most at least in business contexts we just use the whole data set for the whole analysis all the way through if you are splitting your data set it's really important to run a Levine's Majin ad of variance test in SPSS prior to using that other sample in CFA do you know that have you seen the Levine's test before yeah let me show you real quick this is important for anyone to consider who's gonna be doing this real quick let me answer Robert if one was working on an academic paper and the general allowed twenty pages of appendices would you put the measurement model in the appendix a picture of it no the reports about it sure if you want to put a picture of it go for it you don't have to when the my favorite hypothetical capstone last year had no appendices it was a 20-page report no appendices really anyway all right let me okay let me do the Levine's test real quick so when you have multiple groups weather like this this often happens when you have multiple data collection runs so I collect data from let's say this company and then you collect the same data from this company or this group in that group whenever you do that you have different data collection phases you need to run what's called a homogeneity of variance test which I'm just gonna make up for a moment I'm just gonna do I'm gonna create a new variable just to represent this thing group no something like that and I'm gonna create data for this so everybody and the first half is gonna be gluten um zero okay these people are grouping them zero and then group num to one group num one is gonna be a Rios really you do this based on actual data collection I'm just making up groups right now so there's like phase one phase two or Group one group two company one committee to you go to analyze the commitment compare means one way ANOVA and in this what you want is in the post hoc where'd you go where'd you do options here it is in the options homogeneity of variance test it's in the options button margin ad of variance test and what this does is it tests whether all of the items in your factor analysis so I'm gonna throw all these items in here based on this group num in the factoring box are those items variance is the same or different across these two groups in your case would be sample 1 sample - assuming you have them in the same data set you probably have to combine data sets if you hit okay the test runs here's the margin a variance test what we want to see is oh they added to this that makes it way more complicated but basically you want to see let me go up here you want to see non significant differences as when you were C greater than 0.05 and so anxiety were good except here dang anxiety - was answered differently by my two groups even though they were just randomly yeah well so anxiety fives good but the rule of thumb is if at least one variable in a latent factor is invariant not different between the two groups then you're good to go so in this case we have anxiety five totally not different we're good then I go into comp use computes one totally good even though a couple of the others are not you get the idea but if you find it for a whole set of indicators there's complete difference between the two groups you values less than 0.05 then you're in trouble then you got to figure out what to do next because those samples did not understand the questions released did not respond to the questions in a similar fashion and so either it's a limitation or you got to drop that factor which is not ideal obviously but any any analysis you run going forward will be subject to that limitation okay back to CFA I wouldn't have taken it some time to do that if it weren't important that's a Levine's homogeneity of variance test I think there's important stuff whenever you have multiple groups or multiple data collection phases it is essentially a metric invariance test in an IFA and we're gonna be the same thing in a CFA the dimensionality sorry we're so we're gonna very obsessed yeah the scalar invariance in dimensionality we're gonna be that's gonna be probably next right let's see if we go to the order of operations let's see CFA here it is configurable metric and scalar invariance dimensionality if that's what you're talking about I think that's me tell you matter okay if not we can do something else so the next thing is if you have multiple groups and you're testing hypotheses based on those multiple groups and in our in our model we are we go back to our model here whoops here the model were pretending to use during this bootcamp is right down here we said there is multiple groups male and female we're saying this model differs between male and female and if we're gonna make causal claims based on multiple groups we also need to test our measurement model to make sure it is the same for both groups because if it's not the same than any findings we have based on those multiple groups are suspect we can have confidence and then because we don't know that the measures used in each group are the same it'd be like me giving apples to the men in the room and oranges the women in the room and then asking how much they liked fruit the fruit I gave them and they had totally different fruit and so the whatever hypotheses I made about men and women liking fruit the same or different would be invalid because they didn't have the same measure that's what this is for okay so this is unnecessarily complicated um what you gotta do and we tried to simplify it a little bit but I don't have a plugin for this yet but Amos does so we'll be okay what you got to do let's save this as a different model real quick save as in Baron of CFA invariants 2018 and now this model is available if you want to wait till I add the groups you can wait or you can do it with me but now we're gonna add groups to this data Oh actually we're gonna take its three minute break yes okay so we minute break rule books and I'm gonna eat a gummy bear [Music] did you see the mountains at eight o'clock from eight to 8:45 the mountains glow orange and pink because the Sun is over there [Music] [Music] [Music] I [Music] [Music] [Music] [Music] [Music] [Music] [Music] sorry good here okay invariants so there are three types of invariants actually or more but there are three that I care there's zero that but there are three there's three which is probably covered that's what I'm finding I'm recording this right who actually watches these only tens of thousands who sympathize with me and who who disagree don't watch it okay oh maybe alright anything so there are three types of invariance one is scalar one is configure one is metric if you want to know what they are if you go to the stat wiki let me go here stat wiki and if you were to go to the CFA section there's a section on measurement model and variance and you can see definitions of each and videos on how to do it all and what to do if things fail so let's do this first off we got to bring our groups in to do that double-click group number one or whatever your group number is our moderating variable is gender so I'm gonna put male here and then make a new model our new group sorry and say female female and then hit close if you accidently hit new again then just click on your extra model that you created and hit delete so you should just have two groups here male and female if you're using that data and then we have to add the data to those groups so if you go tour you add data file data files you'll see that we have data already added for the first group males we need to add the same data set for females or if you have them in separate data sets Amos can actually handle that as long as it's the same variables so gonna double-click female and pull in the same data set oh not the speedrun data and go back and get the right data data is this trimmed 2018 clean and then once you have that you actually have to separate these the data into groups here so you click on grouping variable for my top one for the male group I'll go find gender I need okay same with female click on the female group go to grouping variable gender okay and then it would say well what value for gender so you click on group group value for males it's in this data set males we have 287 of them ok and then we click on the female group group value and to is 491 to that point so now we have our datasets split by males and females and so all the analyses we run in Amos from here forward will have this data split to run two separate models of model for males and model for females hit OK and I'm gonna save this so that you guys will have it it is now saved in the Google Drive if you need it CFA very in variants 2018 so once you do that we need to assess configural metric and scalar invariance to to assess configural invariance is super easy you just test the model fit again so you gonna have to plug in or you can do it the old way I showed you you should show a model fit here real quick this is with the data separated the question is is model fit still good it's running and the model fit is still good or acceptable at least even when the data is separated so this says we have configural invariants the two models run fine separately they're appropriate for each gender essentially that's configurable and variants so what would you report you'd report we ran the model with the data split unconstrained and the mall that was good seaman DF CFI armsie assumed or something like that in parentheses with the values yeah if your mother wasn't good at this point um you was it good during the initial run so if it was good during the initial run and not good now what you'd want to do is go look at the output and figure out with modification indices if possible where the biggest issues were for each group because it will differ notice in this group it's a 1980 20 this is the male group down bottom left here I'll show you there's a group area one for male and female if I click on female or male it'll show me the modification indices for just that group so I'm a male right now and it's a 1980 20 or the problem female no problem hmm because here's why well yeah the data sets small and so the chi-square is small and so the modification indices threshold was too big so what you got to do is in your analysis properties color abacus go to the output tab in the modification indices area change this threshold here instead of 40 change down to like 10 why because our chi-square for the female group is very small ah because it's a smaller number of females than males so close that run it again wait for it here's it never finishes as unfortunate slocum peter don't worry about X's on the default model for now but if you were to look at modification indices again for the female group here are the big ones e 9 and 12 and e 22 in useful weird let's go look at that e 22 and useful where is e 20 to eat when T 2 is down here decision quality 7 and useful I wonder how his decision quality 7 worded here's we know it useful means right it's a useful software here's decision quality 7 Excel helps me make more effective decisions sounds pretty useful so so our problem is decision quality 7 is very similar to the useful measures so we do yeah if we gotta meet this configural invariance test we'd have to delete decision quality seven we have four more so we're good the other option is throw decision quality and usefulness into an EF a see where the cross loadings are and try to separate them not to the originally FA yeah you go back to a newly fa it's just a tool you gotta pick up your hammer again yeah so but that's what I would do is either go back to a new EF a with just decision quality usefulness or just drop the offending item we might not have a problem at this point yes vias decision quality 7 wasn't identified in our initial EF a our CFA as a problem yeah it's only a problem when you consider that groups separately not across groups but we did we did it in the initial survey let's real quick let me save this we'll go back to the initial CFA 2018 it is if we go look at the modification indices here you'll see that the usefulness was at 7 usefulness 7 shouldn't be a problem which is where we go usefulness 7 hello play 7 sorry - I have a question please yes just a moment I'm going to say e 22 isn't listed here [Music] it's so it's not a major one is what I'm trying to say let me go here and set this to 20 we go off gauging indices and it was e 22 there's this but that isn't a relationship with usefulness eat 22 just shows up here with itself so we may have ended up addressing it there right yes mojo yes I'm assuming we had or you call it a bad fit or relatively good people can we can we go to the modification this is an identify I mean the key ones and co-vary maybe just a few 102 at maxima so there's there are different schools of thought on whether you should just go very error terms if you co berry error terms what it does is it accounts for the discrepancy between the chi-square the two covariance matrices that produce the chi-square but what you're doing is you're adding a totally artificial relationship in your model that doesn't exist in theory and so while some say it's okay on occasion when the two variables you're you're connecting are highly related theoretically others would say well no it's still an artificial relationship he shouldn't do that I tend to agree more and more these days with those who would say not to co-vary errors a introduces problems later in your model makes things unstable and it is totally artificial especially when you have reflective model the reflective model those items are redundant and so if you're seeing a strong covariance between their errors it's because they're redundant and so you might as well just remove one because it's redundant may be better to remove that item the only exception of course is if you have three items and you don't remove and end up with two items in your latent factor and then you might just accept a lower threshold of model fit rather than covary the errors cool okay okay so we dressed what to do if you don't get good model fit and but that's configural invariance and again we're doing this because we have multiple groups we're going to test in our causal model and we have to make sure that they understood the measures the same way should we report see if I in an academic journal heck yes see if I should always be reported according to this scholar if someone didn't report see if I I'd ask them to please include it if they didn't include GFI I wouldn't care okay hypothetically yeah if you include SRM RCF i-rms EAP close you're good also report the chi-square degrees of freedom of course oh yeah so P less is different yeah we'll talk about that Saturday right we'll talk about that Saturday yeah okay from Chet okay what about if see if I is the only low score and it was because of a low loading on a something of social desirability which won't transfer over to the structural model anyway ah good question hypothetical question was what if you only have bad model fit because of your marker variable yeah if it were me I would with all confidence delete the marker variable during my model fit assessment and then move forward and just parenthetically note that you this is the model fit - the marker variable and I'd say it constantly and write it confidently and then wait to be reprimanded okay here we go I hope you have a kind reviewer I you know it's funny I review can you cite me sure I get asked to review papers constantly because of the statistics stuff and the problem is when I review papers I ignore all of their statistics because I assume they did it right I I focus more on theory and motivation and positioning and framing and methods I assume you are correct which is probably not the editors want to tell you want me to look at the methods anyway okay Factory so that's configural invariance we next need to do metric and scalar invariance which there is a tool for this little multi group tool down here next to the copy machine the fax machine is that fax machine yeah okay printer is still somewhat relevant okay so click on the multiple groups tool here and it's gonna say there's only one group oops I need to go back to my other model that we invariants where you here make sure in your in the invariance model ok then click on the multiple groups it's gonna say we're gonna wipe all stuff youth included say yes that's fine okay okay what we're doing in this is we're allowing Amos to run some chi-square different tests between unconstrained and constrained models where the constraint models are forcing the male and female models to be equal to each other and then comparing that to an unconstrained model where they're allowed to be freely estimated why are we doing this and also what is the chi-square difference test chi-square is in for lack of a better description is a measurement of the amount of error produced when you compare the covariance matrix of all these variables including in the model to the covariance matrix you're producing by modeling the variables in this way it's the observed model versus the predicted model yours is the predicted the observed is the natural core of correlations between variables so when you compare those two you literally just take a difference between those two matrices it produces a chi-square value the size of the difference between the two matrices is a little bit more involved but roughly that'll do the bigger the difference the less your model fits the data and if the model you're proposing doesn't fit the data then there's probably a better model that you're not proposing and so your model fit is poor that's the model fit chi-square a chi-square difference test says we have this proposed model and this proposed model the unconstrained in the constrained models let's compare those now is the size of the chi-square compared to its degrees of freedom significant or is it a nominal difference is the chi-square being produced again a chi-square is just the difference between two matrices is that chi-square being produced a significant difference or does it represent something no different from zero if it's no different from zero there's no difference between these two models therefore they're equivalent that's a chi-square difference test conceptually these I should make some slides for that anyway so what we're gonna do is we're gonna run some chi-square different sets we're gonna say here's male and female unconstrained that's a model with the covariance majors heirs male and female constrained to be equal are these two types of reality compatible if not we don't have invariance if they are if the chi-square different test turns up non significant they work fine so just gonna hit OK here once what do whatever it proposes here hit OK it's going to label a bunch of stuff for you and then you're going to run the model as is and it's gonna take a moment to run as it's doing a lot of comparisons once it runs you can go look at the output and there's a new section called model comparisons well here it is right under model fit it's called model comparisons click on that and you have some new output that is not what I was expecting oh I clicked an execution time okay Auto comparisons new output this output means it's soon out for sake here it is okay this output assumes the unconstraint mob would be correct assumes the measurement weights to be correct assumes the structural variance is to be correct let's look at metric and variance first which is the measurement weights this model compared to the unconstrained model you zoom in so when we compare the measurement model to the unconstraint model the p-value for the chi-square is non significant meaning the chi-square produced by comparing constrained regression weights to unconstrained regression weights across these two groups is no different from 0 so these are the same so we meet the test for metric and variance next is to look at the structural covariances which is actually worked out cool I wasn't expecting that we want the when we constrained structural covariances and other constrain them and compare them we want the chi square to be non significant this means we have scalar and variance I didn't include the intercepts here whoops we need to include the intercepts because scalar invariance includes both covariances and intercepts so to do that what you have to do is go to your analysis properties and uncheck modification indices and go to the estimation and make sure you check this box right here estimate means and intercepts if we don't estimate means intercepts intercepts aren't estimated and so we can't constrain them and and unconstrained them for these comparisons so estimate means intercepts zoom out close this run again actually we might have to recreate the model actually not the whole model recreate them all the multi group assessment yeah it didn't produce it so what you have to do is once you've estimated means intercepts to make sure still check to get go back to the model the multiple group analysis click on that it's gonna say I'm gonna wipe everything yep that's fine hit OK and now it's gonna produce five models I believe if five models there we go hit okay it now is going to constrain the intercepts as well yeah unconstrained is non significant we wouldn't run do you have covariant errors because that would do it I might throw it off do you have the common weighting factor mixed in there somewhere yeah you have a really small sample size in one group hmm I'm not sure that's busy the unconstraint model try running it without the multiple groups assessment and it should because it's the same model as as these okay if we run this there are now usually measurement inter intercepts are listed here we can look at those so yeah if you left the regression we constraint on one of these indicators the only issue is you can't compare that path across groups so if you're still good if you're people I use end up more than 0.05 who cares you made it you're good but if they didn't then you might want to try moving it and doing it again um here's the output and we see some differences now measurement weights were still good metric invariants score we've got it the p-values the same as it was before but the intercepts no good and covariances actually includes intercepts with it no good so what do we do we answer the chat let's see what does the zero next to the one on the factor mean that means you're estimating means and intercepts the mean is zero that's what it means because it's being estimated as zero okay so we have a problem our intercepts are not invariant so we have some options first option social desirability the marker variable we don't care if it's invariant across groups because it's not part of our causal theory right it's just a control variable it doesn't need to is we're not testing any hypotheses with it so for this particular test we can unconstraint those constraints so let's do that real quick you'll notice here the social desirability the intercept is i-35 onward so I'm gonna zoom out here go into my model here's covariances model I'm gonna intercepts model sorry double-click the intercepts model and it brings up the manage models dialog I'm gonna scroll all the way down to eye 35 and get rid of everything from i-35 onward deleted I'm gonna do the same in my structural covariances model I 35 to the end of the eyes I'm gonna get rid of don't get rid of those C's though okay so those I 3,500 I got rid of the constraints between both the social desirability so we're not including them in this analysis let me close that I'm gonna run it again see if that just fixed it if you didn't meet it then yes I would unconstrained that path yeah okay yeah I did in both and notice things did change the p-values are getting higher yeah but they're not fixed there's more we need to do so next is to go see where are the differences right now we clearly do not have scaler invariance and so what we need to do is find out if we have partial scaler invariants and to do that we're gonna go to the estimates here and Maius scalars and intercepts where they are estimate scalars intercepts and what we want to do is we want to find out where the big differences between our two groups because that's what's causing this invariance is the different intercepts because it's forcing them to be equal and the extent to which they're different is inflating the Chi square okay so for the intercepts I'm gonna click here copy take it over to excel and just paste it in here this is for the male group I'm gonna do the same for the female group to switch the female just down on the bottom left you click on the female label and then copy this over page student here and I want to see what is the difference give me a delta equals absolute of the estimate minus the estimate I'll zoom in sorry so it's just the absolute value of the difference between the two estimates because I want to see which ones have the biggest difference so I can unconstrained those and see if I have at least partial scalar and very invariance where most of the things most of things are invariant but maybe they're a couple that aren't so I'm gonna fill that down and oh I double clicked the little handle here there is this itty-bitty little handle as well like square if you double click that little square it fills down okay we want to find the big ones so just to make my life easy other conditional formatting top bottom give me the top 10 items yeah top 10 items here they are looks like it's anxiety our two groups males and females we are not treating anxiety differently yeah and I'm totally over it so turns out we treat it differently same with social desirability but honestly we don't care about those you already unconstrained those so it's mainly anxiety and then there are these two others playfulness five and six anxiety playfulness and for anxiety oh we last look at this there's one anxiety 7 that is probably okay we're gonna leave that one constrained but the rest we're gonna unconstraint so at least while partial invariants hopefully so anxiety 1 through 7 we can see right here that those are I 7 through 12 so we're gonna remove I 7 through 12 in our model oh but this isn't the test so this test doesn't say oh this one's not invariant I just said give me the top ten big ones yeah so I 7 through 12 in measurement intercepts go find I 7 through 12 7 through 12 deleted and make sure you do the same in the structural covariances I 7 through 12 you may think yourself how on earth am I ever gonna remember all this you would be right that's why we make videos so you don't have to all right and then the other one was I 18 and I 1918 and I bet this will do it and in the other model here stressful covariances I'm so excited to run this I save this is the new model run it hope a little please look at the output model comparison oh-ho-ho look at that we did it yeah right on the murder our intercepts right on the border all right invariant our covariance is no problem we have partial scalar and variance that's what we report and what do we say we say we have partial scalar and variance all factors were invariant except anxiety which was partially invariant and playfulness I guess platelets scalar I D Li so this one it's looking at the model as a whole but we looked at it individually and said well but actually it was just anxiety and a little bit of playfulness so we report both who say globally we have partial invariance but if you were to look at each factor individually we're a full invariance except for anxiety and playfulness at least there's a different distribution is what we observe and so we look what we interpret from that is that when we are asking about anxiety let's say we have a hypothesis in our causal model this as anxiety leads to less enjoyment of using Excel right the issue we run into is anxiety is apples for men and oranges for females women so we can't say that anxiety leads to something because it's not anxiety it's anxiety male and it's anxiety female choose if I ever run for office all of these videos gonna be you're exactly correct this is exactly what I'm trying to say remote folks did that did you did you hear any of that okay so fairly nice um so the issue of invariance this is a good example I'm just gonna repeat it in different cultures such as in in Spanish Spanish speaking the word pride the constructive pride is always a negative connotation whereas in english-speaking like in America we're always saying about much pride we have in our team and in our children or whatever and it's a very positive thing and so if you would ask the same question on a survey to Spanish Spanish speaking and English speaking you're asking about two separate constructs not the same construct that's what we see with anxiety here men and women interpret anxiety differently yeah there's a cultural bias in this case there's a gender bias yeah so that's why we have to test for marriage because if let's say we ignored invariance because it's just such a pain in the rear right and we went straight toward causal model and we tested our hypothesis that anxiety leads to better decisions or worse decisions or something in Excel and we come out with nothing or we come out that there is a positive effect we actually couldn't have confidence in those results because anxiety isn't just anxiety it's two separate things from when and for women so that's what we have to test it but now we can be confident we at least have partial invariance it's also possible that they're not just interpreting it differently yes the whole the whole recording of the data for that group was different for them for this group like the measures we got for them oh yeah we could actually go into an interesting study let me try something real quick just thanks for amusing me um or humoring me that humoring mean I mean you also amuse me let's go to SPSS real quick and run a one way ANOVA on just anxiety and instead of group num let's use gender would you go gender you are we have a Levene's homogeneity variants test running right now is it gonna be significantly different or the same it's gonna be some different some the same overall it says different different different different different different different and look at number seven the one we saw was okay a is the same under if you can see these sorry this small but the p-values for all the differences between men and women different except for number seven or it is this so we have partial invariance for anxiety interesting okay invariance is an advanced topic and often you don't report that you did an invariance test you just assumed that those who did the analysis we're doing it sorry I've been ignoring these let's see so is it insignificant non significant at point O five or greater or just greater than point five because it is greater than 0.05 then it is at level nope is it the same construct of that's oh sorry Rob is it the same construct are the two cultures treating it or is it or the Google hopes are treating differently it's a different construct I mean it's the same calm struct overall it's a we labeled it the same but it's treated differently cuz I just said yes to both sides of your question I know they're statistically it's different I'm gonna say that did you say yesterday that a constant can be considered invariant if one of the regression lines unfactored is invariant correct we have partial invariance if one is correct how come my questions don't sound as good when you read them Thanks we appreciate you contribution to the levity of this conference all right James yes from more roots yeah from more instability isn't there a minimum required sample size for each group yes there is a minimum sample size so this is an issue if we are running an invariance test on a small group there will be a lot of error and so naturally the chi-square will be inflated and so the invariance test might be confounded by that small sample size the larger the sample sizes also the chi-squared can inflate but we have less error so is there a sweet spot probably I don't know what that number is but is that what you're asking sorry yes um I wondered if the sample size for females was adequate that's all right it is smaller so how many items do we have here sorry what oh so there's a yeah I was a Fisher said you need at least 30 to run an ANOVA or t-test for SEM it's way more do becomes it depends on the number of degrees of freedom but when we calculate it is 50 plus 5 times the number of observed variables yeah we have a lot of observes burials here we have what 7 14 21 30 50 variables here 47 variables so that's a lot we need 300 we have threader but not in the female group or in the male group so we're limited scaler birds dang straight it's gonna take to his dad to skip it no just oh that that's not you're saying transferred over to excel oh that's important yeah so if you can't find it the invariance at one level what you're gonna do if it's at the metric level with measurement weights are you gonna transfer over the regression weights on the standardized regression rates if it's with intercepts you're gonna transfer over the intercepts and I'll show you how to do that if it's what the covariance is you transfer the covariances let's which one would you like to do intercepts so if you go to the output and you go to the estimates and scalars there is a section for intercepts and so you just left click over here in the white space then right click and copy and push this over to excel just create a new sheet here paste that in and then go back and you have to switch down on the bottom left you have to switch it to the other group and then copy this over and paste it in here and then you're gonna create a Delta between the two which is just the absolute difference a equals a BS between estimate 1 minus estimate 2 and then drag that down and for me I did a conditional formatting here just to highlight the top 10 items and since it's absolutely differents they're all positive absolute values me anyway and that highlight the big items and then I went and removed do you wanna see that part okay yeah so just remove those from the constraints so I have an invariance plugin I shouldn't have named it invariance sorry it's actually a multi group comparisons plug-in so it doesn't apply to a CFA oh yeah okay okay but we achieved at this point we achieved partial scalar invariants full metric invariants and full configurable invariants so we'd report that just like that we ran variance tests configurable metric and scalar and we found full variance for first two and then partial for the last one due to anxiety and playfulness being interpreted slightly different between men and women that's it I wouldn't spend paragraphs on this should be a couple lines okay I think I need some more gummy bears my heart keeps skipping please that's my excuse I'm gonna switch over here real quick okay thanks pretend that sugar helps but medically speaking it actually doesn't at all but it gets me excited so maybe the adrenaline yeah okay um we've done this next run a full validity test yep honestly I would actually go in a different order I wouldn't do that yet you guys are watching me edit this thing I'm gonna change us yeah this one I'm gonna switch there we go because things might change during the method by his test so you might as well we save here we go response bias cool this is believe it or not the most painful part don't see you BAE we didn't just finish the most painful work we had to do it yeah method bias should talk about this briefly method bias is the assumption that the way you've collected is the assumption that you didn't get exactly what you were expecting to get your the responses to your questions on your survey were somehow biased by some some other factor some compounds this could be in our case social desirability it could be in other cases loyal to to company or confidence optimism things like this that a third and they call in dodge an ad a third confounding factor that just raises everything or dampens everything a good example of this is I did a study in Nigeria or we're studying corruption in business and we were asking businessman the extent which they lie cheat steal bribe and engage in truck practices there's a way to answer those questions of course I do know I do not cheat I do not steal bribing on the other end ah no but um so we had to collect with that data we had to collect social desirability bias questions to see to what extent they inflate all of their responses based on the socially desirable way to answer a question and then because we collected social desirability we were able to then what's the word account for it we were able to adjust that's where I was like just their scores based on the social desirability inflation and that's what we've done here do we think that there is a socially desirable way to answer questions about usefulness and anxiety and playfulness maybe anxiety probably there's some negative connotations in anxiety decision quality definitely people are always gonna say I make better decisions than worse decisions right who makes worse decisions so there is some sources as I have a way to answer these it's probably not the best marker variable for this data set may be a better marker variable would be something like computer self-efficacy or optimism or self was the word like ego something like that self appraisal okay so we need to control for that as well I'm trying to say if you don't control for remebers then you could end up with a lot of positive relationships between variables that aren't real they're false positives because the positive relationship is actually due to shared bias variance not shared trait variance so we need to fix that to do that I'm gonna save this but I'm gonna go back to my CFA initial 2018 model my ungrouped model and then we have plug-ins for this hmm dang it if you don't have a plugin for this it gets really painful I'll show you real quick manually but the whole thing because it would just take too long the answer is you don't want to know but you want to know I have a slide on this and says exactly what to do but it's only to get really nitpicky the way we ran it will work and it's sufficient but if you want to know like that the true true true method here's the slide on it I'll show you the method on the group I sorry what was that what I'm saying now is you said now at this moment we have to do the method bias on the ungroup more than so your ungroup tomorrow on I mean do they method bias on on the group model can we do that no I would strongly recommend you do it on the ungroup tomorrow okay yeah especially since we have a lower sample size for the female group to answer your question then here's the bullet point you need when you're doing scalar invariants it is best to keep the constraint same except for each factor one of the groups make mean make the variance constraint equal to 1 and but the path constraint on the other group so for male group have it be a variance constraint for female group have to be a path constraint but like I said it runs just fine the way we did it but this is the true like the true approach also find if yeah if you ran it for both groups or the path constraint yes is better new bias testing with your constraint on the factory also what if we had changed our CFA during the invariance tests let's say we had deleted what was it a decision quality 7 now when we go back to this initial CFA we should probably delete decision quality 7 moving forward because we realize we are invariant without doing that so we didn't in our model but if we had deleted an item during the America tests we'd have to delete it now in our server running a master measurement model ok yeah if we deleted an item during our invariance tests then when we go back now and test for method bias we'd want to test for method bias without that variable in there so I'm yeah without that variant variable at triple-negative yeah so are we just be adjusting our master CFA to match our most recent edits only if you couldn't reach invariance partial invariance if you had to delete a variable during the Americas leave it deleted the rest of the time if you didn't delete any variables don't delete any now you're good yes we have invariance even if it's just partial okay model bias we have 20 minutes let's see what you do but I think we're doing time we saw three hours this afternoon and I've covered 80% of the material so we're doing great this afternoon we may run through Murray's model in a CFA which would be so cool and then we may even jump into causal models so that we can spend more time tomorrow doing pls and may be finished by lunch tomorrow who knows get out early over Aiko here we go play in snow plugins to do it manually to do it manually you're gonna move the whole model for a moment I'm gonna select all with a big open hand use the fire truck but don't have the balloons selected then I'm gonna move the whole model over to the right just so it's out of the way of it and then I'm gonna deselect with the closed hand and then I'm going to add a common latent factor I'm gonna use this ellipse tool and click and drag just make a nice big common latent factor here it's size actually doesn't matter but it helps when you're drawing things move it a little bit okay this is our common latent factor I'm going to name it CLF and in order to test the extent to which all variables were inflated by some external cause what you have to do is relate some common cause to every single observed item one at a time you can see why we made a plugin for this because that's not all folks you also have to then run it unconstrained and then constrained and then constrained equals zero so there's a tool for this if you run if you create this common latent factor this and then select it just like that nothing else and then run the common Layton nope nope not that one specific bias test yeah run the specific bias test what is gonna do is it's gonna connect all those for you and then just gonna run several tests in a row and so you just gotta keep clicking proceed because it's gonna run whoops and evening that variable it's Kenneth rope errors sorry I mean I was supposed to name it my bad even I don't know how to use my plugins so double click this call it the CLS CLS or call it Bob whatever you want and then select it so it is blue and then run the plug-in specific bias test here we are it's gonna run proceed and it's gonna run again so that ran the unconstrained model now it's constraining all paths to be equal to zero and that runs all paths constrained to be equal to each other it's thinking one more time now they're all equal to each other and what it's doing is it's comparing these all doing cut models doing chi-square difference tests and to see is is their bias and if there is is evenly distributed and here's what it comes out with says chi-square difference test was significant the unconstrained model versus a model where we assume there is zero bias that's what that means 0 constraint model this says no bias at all is there difference between a model where we assume no bias and where we let bias sort of inflate itself and the answer is yeah there's a difference there is bias you have method bias and then equal constraint says what was that bias evenly distributed across the items or is it unevenly distributed and in this case it is unevenly distributed the bias or the test is significant and so here you go the chi-square test for the zero constrained model was significant there was measurable by us therefore biased just biased distribution test was made of equal constraints the chi-square test is significant on this test as well unevenly distributed bias you should retain the social the specific bias construct that's social desirability for subsequent causal analyses and I make note of this so we we failed the test we do have method bias but that's fine what this means is we need to go back to our model and proceed I should have left those connected thing when we when we proceed I mean go back to this when we proceed to our causal model we need to account for this bias that's what this means and so I'll show you that in a moment after I answer this check when I run specific bias test plugin I get the error message please only select the specific bias weighting factor I haven't selected anything can you run through that part again um yeah so let's run this again I'm gonna select all move it over deselect all with the closed hand that's important create a common lean factor name it CLF or Bob I'm do Bob there we go name it Bob and then you have two single finger point select Bob or the CLF nothing else should be blue only Bob should be blue and then you can run a specific bias test brew seat so hopefully that Alvia okay so moving forward we need to account for this bias and bike by calculating factors scores like we did in the EFA by calculating factor scores in the cfa while this is connected to everything we are essentially parceling out all of the shared variances due to this confound but we need to also make sure that isn't breaking our model so let's check that real quick go back here and I'm gonna reconnect Bob to everything um me save and reopen but this one there we go real quick let me just connect Bob to everything again there we go okay Bob's connected our CLF is connected and what I'm gonna do is I'm gonna run this just as regular model and see if my validities and model fitter just destroyed model fit it should actually be fine but my validity is I want to see if like these loadings here are messed up lo and behold they are look at this you see negative loadings here negative loadings here the CLF broke my model well dang this is not uncommon when you add a commonly in factor and the trouble is when you don't go back and check this and you proceed you have a broken model so any causal analysis you make after this are completely flawed so you got to go back and check and make sure your model is not totally broken now we can take two approaches to trying to fix this let's take the first simpler approach which would be to look and see if we can tell what it is that's actually being broken forego the estimates and we look at these p-values for all the non CLF ones huh well it looks like there's no problem so you look at the p-values and usually when you have a broken model some of the p-values become non significant for the from the latent factor to its indicators and so that's a good sign of where the breakage is happening and so what you can do is move around constraints play around with it a little bit in our case they're all still significant they're just mostly negative look there is a pattern though ooh anyone see the pattern there's two positive ones so should I really and anxiety stupid anxiety um dang I bet if we not that we want to but I bet if we removed anxiety this would totally work I just for kicks and giggles can I try it let me just I'm gonna save this as I'll save as CF a CMB 2018 with anxiety and now let me do another one save as without anxiety now I'm gonna delete anxiety oops almost there okay let's run this again and see if it works proceed and nope even worse cool alright so that didn't fix it so we're gonna try the other approach I'm gonna go back to this with anxiety we answer this text here let's see if CMU breaks the model are you more likely to remove an offending factors such as hypothetical anxiety if the vector is either controller moderator answer's no I was just doing that fun um I wouldn't remove a factor at all that'd be silly good what I do is what I'm about to do check this out so when the CLF breaks your model here's what is strongly recommend if you have a social desirability or some sort of marker specific bias variable to fall back on you're gonna do it I'm about to do if you don't have any marker variable at all then you report method bias was detected but including any sort of unmeasured leighton factor made the model completely unstable so it is a limitation of our study that's it's just a limitation we have method class so sorry but if you have marker veritable do this delete the comment latent vector and delete all the cool bearings bless you whoever sneezed you're not muted raising your hand though I didn't know you do that that's kind of cool dad give him a question no no that was earlier thanks oh cool lower hand interesting I'm learning stuff yeah if a factor stays red it won't get unread just reopen the model you don't have to close Amos just save and reopen them all okay so what you do is you delete all the covariances with your marker variables or variable if you only have one so I'm gonna delete all these covariances here and we're gonna treat our specific bias variable as our common latent factor like this I do want to covary these two actually okay I'm gonna move these over here zoom out you go over here you go over here and with the plug-in you can just select these two here and say specific bias test hmm drive somebody else selected deselect everything real quick just like these two plugins specific bias test why or how with the X here just manually what am I'm gonna delete this one too it seems to be throwing it up there we go is it red the thing you're trying to delete if it's red it's just inactive and it's not responding and so you have to reopen the model you can save it and reopen it it won't stay red if you save it okay so this just treats each of these to like specific like the marker or like the leighton factor commonly in factor and so it's gonna test whether this specific bias social desirability has some shared variance with all the items see if this works oh yeah like this one this one says since it's not breaking the model this one says no social desirability is not impacting your model you're good to go you can move on to causal modeling but let's see make sure to retain specific bias variable as a control variable that's actually optional at this point but we have it so we might will include it what we're gonna do is we're gonna impute factor scores assuming this didn't break your model let's look at it up arrow aha see no broken it's not broken nice all these values look valid and strong this is great in fact we could wonder if our plugin here will work validity and reliability proceed here we go lots of issues there but that's because they're not correlated with anything but look our validities are still good with the specific bias markers in there which ones would I report I would report this as my final correlation matrix ignoring social desirability here but this is my final set of measures here and I have no validity concerns because social desirability is not one of my factors of interests it's not just a marker variable and this is the validity I report and then we have three minutes and then from here this is the next step you just go save but then you do data imputation right here analyze data imputation this will create a factor scores for you adjusted for any potential bias observed being parceled out by social desirability if we do that we don't even have to include social side body as a control variable because it's already accounted for thanks it's accounted for it's accounted for in all the measures because right here what we're doing were parceling out all parceling out all of the social desirability shared by us from all items and so when we impute vector scores the extent to which social desirability was impacting all variables is already being adjusted for in the new factors for trust me on that one all right there's a texture but could there be other biases not related social desirability dang straight yeah they're good like optimism ego and vengeance that could be impacting the model but is not detected with our successful test but in your model SD is also a control variable I was controlling for it because I thought it might be an issue and if I were to have these Co varied with all other factors then I would still have to control for social desirability in my cause model but since I'm using it as a specific bias marker variable here I do I no longer need to control for it in my causal model because I'm already controlling for it in all the measures that make sense that means that would mean that there is no difference it's unlikely there is yeah okay so that's our model that's our final CFA model the next step we'll do after lunch but essentially you just impute factor scores and then you have a new set of variables you can use to run your causal analysis yep or even if you now it takes few seconds data tution oh yeah proceed alright I'm appearing now but we can view that for lunch as well I'm just gonna hit impute and in a buted whoo what this did is it created a new dataset for me right here data here it is this is my new dataset and it has new factor scores each or new variables each called the same as the latent factor yes and they're down here variable view at the very bottom and you're they are my new factors I can use these just like they use factor scores from eBay we'll get to that after lunch to our lunch I'm gonna do the cannon Center again because it's so good yeah yeah if you want to put your stuff in my office I'll go to my office first let's see and stop this stop share

Channel: James Gaskin

Views: 6,373

Rating: 5 out of 5

Keywords: SEM, statistics, CFA, invariance, model fit, validity, reliability, AMOS, structural equation

Id: 4_ZvpU8wu3Q

Length: 174min 39sec (10479 seconds)

Published: Fri May 11 2018