Effect size calculation in meta analysis

regression the result is exactly the same because the underlying statistical model is exactly the same which gives us a lot of benefits right and the benefit is that we can start playing around and that we can start converting data from one particular statistic to another one right and if you have means and standard deviations well you can actually calculate a correlation based on that available information you can convert a p-value into a correlation right those are more advanced conversion so we stay away from them right we will explain you the basics the most commonly used conversions in this section so coming to our example right imagine that we have four studies examining the relationship between controllability and negative emotions right in the papers that you coded there were many more relationships that were tested right but we're focusing on one for the reason of simplicity and we only focus on four studies for the reason of simplicity right and reality we have bunch more so imagine the first one sample size of over 600 has T value we have reliabilities the second study smaller sample size reports a correlation right so this is the first paper that you coded third study reports only a regression coefficient only reports a standardized beta no correlation matrix right if you ever report a regression please list the correlation matrix right it's much easier for us but so they only report a beta coefficient and then the fourth one is the F value which you also found in one of the articles so we have four studies reporting different statistics and we need to convert those into a common metric we talked about the correlation right we talked about correlation but they're actually a lot of different effect size metrics right correlation is only one that you typically use for linear relationships but you also have Cohen's D which is often used inside in psychology right and that's actually more an effect science metric that is more about it's more commonly used with experiments and with means and standard deviations that you use so in service and marketing in management you mainly see the correlation as the effect size metric why because everybody's familiar with it write your first stats one on one course you see the correlation coefficient right so everybody knows how interpret how to interpret it it's easy goes from minus 1 to 1 right Cohen's D who ever heard about a coin C right just a few people right not everybody's familiar with that so that's the reason why we focus on that correlation coefficient everybody probably knows these cut-offs as well right if you have correlation of point 10 it's a small relationship a weak relationship point 30 is medium and then point 50 is a large really or strong relationship point 70 is a very strong relationship so just to give you an idea about thinking in terms of effect sizes right we're not talking about is this significant or is this not significant we're just looking at the effect size how strong is the effect that we are observing how strong is variable a related to variable B right and it's a standardized metric from minus 1 to 1 right and if we can convert all statistics into that metric then we have the perfect solution because then we can start comparing the studies and I we can start integrating all those studies right in order to do that we need to find a way to convert it and the easiest thing is when the correlation coefficient is reported right that's the easiest of all if a correlation coefficient is reported take that one right away it becomes more problematic when it's not reported right because then we need to do all sorts of magic tricks in order to turn statistics that we observe into the correlation so for example standardized beta coefficient right a regression result of standardized coefficient or a coefficient in the SEM model right there are some simulation studies that actually show that adding or subtracting point zero five comes close to the correlation so if you observe a beta coefficient if it's positive you add point five if it's negative you subtract point five and then that's the close approximation of the correlation coefficient it's not a perfect conversion but it's enough proximation in case of a t-test right if the researchers do their job they report the T value they report the degrees of freedom right T in between squares the degrees of freedom right if you take the T value you square it and so in the numerator and the denominator you take the square of the T value and you add the degrees of freedom you take the square root if you do that you arrive at the correlation coefficient and it's an exact conversion right so it's an exact conversion of the correlation coefficient with the knife value right people who had a little bit more advanced statistics know that and that value is simply the square of a T value all right so you do the same procedure right you put the F value in the numerator the denominator you add the degrees of freedom and you take the square root so basically every statistic that we observe in a paper can be converted into a correlation coefficient right and these are the most commonly used statistics so you're only showing those but if you have a chi-square test right you can convert a high square into a correlation coefficient right you can convert a p-value into correlation coefficient right and if you ever encounter that problem right just send an email because then in that case right we can provide you with the exact formula right because a little bit more complicated yes yeah you take that one because it's the closest approximation approximation of the strength of a relationship right so it's not a perfect conversion in terms of the beat equation it's a standardized beta not the unstandardized if you take the standardized beta and you simply put it in this formula it's according to simulation studies it's the closest approximation possible so it's when we don't have another option we take that one right if we have the correlation coefficient report it in the paper then we go for that one right away but if we see that authors only report a regression model then that's our best available evidence so we take that one and we kind of discard the fact that there's shared variance and so on so it's some some give-and-take should we include it should we exclude it it's a piece of information so we should include it somehow right and this is this is one way to do it right so basically when we apply these formula to the statistics that we observed to those four studies that I listed just a few minutes ago right it's just doing some basic mathematics then we can convert every statistic like the ones that you encountered in these articles into a correlation coefficient right and then we get my weak correlation for study one a very weak for study three very strong for study two and a medium two strong for study four all right so we can get some insights into the strengths of the relationship right we don't care at this point in time we don't care whether it's significant or not we just want to know about the strength of the relationship this clear because this is pretty crucial right the main message every statistic that you encounter but really every statistic can be converted into a correlation coefficient in some cases in the article of Walton and Hume right if you read that one you encounter a situation where the relationship between controllability and some outcome variables was not significant what does that mean and the order simply say it's not significant they don't provide tests to stick with test statistics what do you do with that case we're not interested in p-values the otters simply say it's not significant at a p-value or an at an alpha of 0.5 but it's not significant what does it mean if something is not significant there is no relationship right so it means that basically the correlation is zero right so you can can't include that kind of information as well because the correlation is zero it might deviate a little bit right but you take the best available information that you have and that's what we also see is that otters are not really transparent in their reporting right so if you ever write a paper be very transparent right and report everything we will not hate you for it right so it's very important to think about those issues as well some journals explicitly ask for correlation matrices right just to make our life a little bit easier all right so this is basically our input data right now we have a standardized metric first study showing the strength of the relationship between negative controllability and negative emotions then we can start

Channel: SERVSIG

Views: 5,824

Keywords: LTAS, Meta-Analysis, Research, service

Id: CF05obuNis4

Length: 12min 1sec (721 seconds)

Published: Thu Dec 20 2018