{metafor} variance explaination for paired pre-test/posttest

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

{metafor} variance explaination for paired pre-test/posttest

John Williams
In a previous post https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html, the following calculation was given for imputing the variance of change scores for paired studies:

// begin quote

2) Often, the dependent variable is not the same in each study. Then you will have to resort to a standardized outcome measure. There are two options:

a) standardization based on the change score standard deviation

Then yi = (m1i - m2i) / sdi with sampling variance vi = 1/ni + yi^2 / (2*ni).

// end quote

I used the sampling variance equation above in a paper that is being reviewed by a coauthor, who is a biostatistician.

He commented that he has never seen this equation for variance before, and it looks strange to him. To put my knowledge into perspective, I am an undergraduate taking my first statistics course. I imputed the t-statistic from two-sided p-values reported in the paper, and used that to get the sdi (as in the previous post).

I consulted the Cochrane Handbook and The Handbook of Research Syntheses and Meta-analysis 2nd Ed (Cooper, Hedges, Valentine 2009) and couldn't find that equation anywhere.

Would Prof. Viechtbauer, or anyone else knowledgeable, mind explaining the sample variance above? I need to be able to defend my choice of equation. Since it's the only method that I found that doesn't rely on a correlation coefficient (which are not included in the papers), I'd like to be able to justify it and not redo calculations for 23 studies if possible.

Thank you very much,

John

~~~~
John Williams
ALB Candidate, Harvard University (Expected May 2014)
johnwilliams@fas.harvard.edu
jawilliamsjr@gmail.com
Reply | Threaded
Open this post in threaded view
|

Re: {metafor} variance explaination for paired pre-test/posttest

Wolfgang Viechtbauer-2
The standardized mean change using 'change score standardization' is described in this article:

Gibbons, R. D., Hedeker, D. R., & Davis, J. M. (1993). Estimation of effect size from a series of experiments involving paired comparisons. Journal of Educational Statistics, 18(3), 271-279.

For a comparison of the standardized mean change using change versus raw score standardization, see:

Morris, S. B., & DeShon, R. P. (2002). Combining effect size estimates in meta-analysis with repeated measures and independent-groups designs. Psychological Methods, 7(1), 105-125.

Viechtbauer, W. (2007). Approximate confidence intervals for standardized effect sizes in the two-independent and two-dependent samples design. Journal of Educational and Behavioral Statistics, 32(1), 39-60.

These articles also provide equations for the sampling variance of the standardized mean change. The equation 1/ni + yi^2/(2*ni) is the estimate based on the asymptotic variance of the standardized mean change using change score standardization.

Best,
Wolfgang

--  
Wolfgang Viechtbauer, Ph.D., Statistician  
Department of Psychiatry and Psychology  
School for Mental Health and Neuroscience  
Faculty of Health, Medicine, and Life Sciences  
Maastricht University, P.O. Box 616 (VIJV1)  
6200 MD Maastricht, The Netherlands  
+31 (43) 388-4170 | http://www.wvbauer.com   

> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]]
> On Behalf Of John Williams
> Sent: Tuesday, April 08, 2014 02:30
> To: [hidden email]
> Subject: [R] {metafor} variance explaination for paired pre-test/posttest
>
> In a previous post
> https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html
> <https://stat.ethz.ch/pipermail/r-help/2012-April/308946.html>  , the
> following calculation was given for imputing the variance of change
> scores
> for paired studies:
>
> // begin quote
>
> 2) Often, the dependent variable is not the same in each study. Then you
> will have to resort to a standardized outcome measure. There are two
> options:
>
> a) standardization based on the change score standard deviation
>
> Then yi = (m1i - m2i) / sdi with sampling variance vi = 1/ni + yi^2 /
> (2*ni).
>
> // end quote
>
> I used the sampling variance equation above in a paper that is being
> reviewed by a coauthor, who is a biostatistician.
>
> He commented that he has never seen this equation for variance before,
> and
> it looks strange to him. To put my knowledge into perspective, I am an
> undergraduate taking my first statistics course. I imputed the t-
> statistic
> from two-sided p-values reported in the paper, and used that to get the
> sdi
> (as in the previous post).
>
> I consulted the Cochrane Handbook and The Handbook of Research Syntheses
> and
> Meta-analysis 2nd Ed (Cooper, Hedges, Valentine 2009) and couldn't find
> that
> equation anywhere.
>
> Would Prof. Viechtbauer, or anyone else knowledgeable, mind explaining
> the
> sample variance above? I need to be able to defend my choice of equation.
> Since it's the only method that I found that doesn't rely on a
> correlation
> coefficient (which are not included in the papers), I'd like to be able
> to
> justify it and not redo calculations for 23 studies if possible.
>
> Thank you very much,
>
> John
>
> ~~~~
> John Williams
> ALB Candidate, Harvard University (Expected May 2014)
> [hidden email]
> [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: {metafor} variance explaination for paired pre-test/posttest

John Williams
Prof. Viechtbauer, thanks for the articles. I appreciate your help.

Yours,

John
 
~~~~
John Williams
ALB Candidate, Harvard University (Expected May 2014)
johnwilliams@fas.harvard.edu
jawilliamsjr@gmail.com