Metafor: How to integrate effectsizes?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

Metafor: How to integrate effectsizes?

Verena Weinbir
Hello!

I am using the metafor package for my master's thesis as an R-newbie. While
calculating effectsizes from my dataset (mean values and
standarddeviations) using "escalc" shouldn't be a problem (I hope ;-)), I
wonder how I could at this point integrate additional studies, which only
state conhens d (no information about mean value and sds available), to
calculate an overall analysis.  I would be very grateful for your support!

Best regards,

Verena

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: How to integrate effectsizes?

Michael Dewey
At 13:15 24/04/2014, Verena Weinbir wrote:
>Hello!
>
>I am using the metafor package for my master's thesis as an R-newbie. While
>calculating effectsizes from my dataset (mean values and
>standarddeviations) using "escalc" shouldn't be a problem (I hope ;-)), I
>wonder how I could at this point integrate additional studies, which only
>state conhens d (no information about mean value and sds available), to
>calculate an overall analysis.  I would be very grateful for your support!

You mean Cohen's d I think.

You will need some more information to enable you to calculate its
standard error. Have a look at Rosenthal's chapter in
@book{cooper94,
    author = {Cooper, H and Hedges, L V},
    title = {A handbook of research synthesis},
    year = {1994},
    publisher = {Russell Sage},
    address = {New York},
    keywords = {meta-analysis}
}
(There is an updated edition)
This gives you more information about converting effect sizes and
extracting them from unpromising beginnings.

It often requires some ingenuity to get the information you need so
have a go and then get back here with more details if you run into problems


>Best regards,
>
>Verena
>
>         [[alternative HTML version deleted]]

Michael Dewey
[hidden email]
http://www.aghmed.fsnet.co.uk/home.html

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: How to integrate effectsizes?

Michael Dewey
At 12:33 25/04/2014, you wrote:
>Thank you very much for your reply and the book recommendation, Michael.
>
>Yes, I mean Cohen's d - sorry for the typo :-)
>
>Just to make this sure for me: There is no
>possibility to integrate stated Cohens' ds in an
>R-Metaanalysis (or a MA at all), if there is no
>further information traceable regarding SE or the like?

If there is really no other information like
sample sizes, significance level, value of some
significance test then you would have to impute a
value from somewhere. That would seem a last resort.

I have cc'ed this back to the list, please keep
it on the list so others may benefit and contribute.


>best regards,
>
>Verena
>
>
>On Fri, Apr 25, 2014 at 1:21 PM, Michael Dewey
><<mailto:[hidden email]>[hidden email]> wrote:
>At 13:15 24/04/2014, Verena Weinbir wrote:
>Hello!
>
>I am using the metafor package for my master's thesis as an R-newbie. While
>calculating effectsizes from my dataset (mean values and
>standarddeviations) using "escalc" shouldn't be a problem (I hope ;-)), I
>wonder how I could at this point integrate additional studies, which only
>state conhens d (no information about mean value and sds available), to
>calculate an overall analysis. Â I would be very grateful for your support!
>
>
>You mean Cohen's d I think.
>
>You will need some more information to enable
>you to calculate its standard error. Have a look at Rosenthal's chapter in
>@book{cooper94,
>Â  Â author = {Cooper, H and Hedges, L V},
>Â  Â title = {A handbook of research synthesis},
>Â  Â year = {1994},
>Â  Â publisher = {Russell Sage},
>Â  Â address = {New York},
>Â  Â keywords = {meta-analysis}
>}
>(There is an updated edition)
>This gives you more information about converting
>effect sizes and extracting them from unpromising beginnings.
>
>It often requires some ingenuity to get the
>information you need so have a go and then get
>back here with more details if you run into problems
>
>
>Best regards,
>
>Verena
>
>Â  Â  Â  Â  [[alternative HTML version deleted]]
>
>
>Michael Dewey
><mailto:[hidden email]>[hidden email]
>http://www.aghmed.fsnet.co.uk/home.html
>

Michael Dewey
[hidden email]
http://www.aghmed.fsnet.co.uk/home.html

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: How to integrate effectsizes?

Wolfgang Viechtbauer-2
If you know the d-value and the corresponding group sizes for a study, then it's possible to add that study to the rest of the dataset. Also, if you only know the test statistic from an independent samples t-test (or only the p-value corresponding to that test), it's possible to back-compute what the standardized mean difference is.

I added an illustration of this to the metafor package website:

http://www.metafor-project.org/doku.php/tips:assembling_data_smd

Best,
Wolfgang

--  
Wolfgang Viechtbauer, Ph.D., Statistician  
Department of Psychiatry and Psychology  
School for Mental Health and Neuroscience  
Faculty of Health, Medicine, and Life Sciences  
Maastricht University, P.O. Box 616 (VIJV1)  
6200 MD Maastricht, The Netherlands  
+31 (43) 388-4170 | http://www.wvbauer.com   

> -----Original Message-----
> From: [hidden email] [mailto:[hidden email]]
> On Behalf Of Michael Dewey
> Sent: Friday, April 25, 2014 16:23
> To: Verena Weinbir
> Cc: [hidden email]
> Subject: Re: [R] Metafor: How to integrate effectsizes?
>
> At 12:33 25/04/2014, you wrote:
> >Thank you very much for your reply and the book recommendation, Michael.
> >
> >Yes, I mean Cohen's d - sorry for the typo :-)
> >
> >Just to make this sure for me: There is no
> >possibility to integrate stated Cohens' ds in an
> >R-Metaanalysis (or a MA at all), if there is no
> >further information traceable regarding SE or the like?
>
> If there is really no other information like
> sample sizes, significance level, value of some
> significance test then you would have to impute a
> value from somewhere. That would seem a last resort.
>
> I have cc'ed this back to the list, please keep
> it on the list so others may benefit and contribute.
>
>
> >best regards,
> >
> >Verena
> >
> >
> >On Fri, Apr 25, 2014 at 1:21 PM, Michael Dewey
> ><<mailto:[hidden email]>[hidden email]> wrote:
> >At 13:15 24/04/2014, Verena Weinbir wrote:
> >Hello!
> >
> >I am using the metafor package for my master's thesis as an R-newbie.
> While
> >calculating effectsizes from my dataset (mean values and
> >standarddeviations) using "escalc" shouldn't be a problem (I hope ;-)),
> I
> >wonder how I could at this point integrate additional studies, which
> only
> >state conhens d (no information about mean value and sds available), to
> >calculate an overall analysis. Â I would be very grateful for your
> support!
> >
> >
> >You mean Cohen's d I think.
> >
> >You will need some more information to enable
> >you to calculate its standard error. Have a look at Rosenthal's chapter
> in
> >@book{cooper94,
> >Â  Â author = {Cooper, H and Hedges, L V},
> >Â  Â title = {A handbook of research synthesis},
> >Â  Â year = {1994},
> >Â  Â publisher = {Russell Sage},
> >Â  Â address = {New York},
> >Â  Â keywords = {meta-analysis}
> >}
> >(There is an updated edition)
> >This gives you more information about converting
> >effect sizes and extracting them from unpromising beginnings.
> >
> >It often requires some ingenuity to get the
> >information you need so have a go and then get
> >back here with more details if you run into problems
> >
> >
> >Best regards,
> >
> >Verena

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: How to integrate effectsizes?

Michael Dewey
At 20:34 25/04/2014, Viechtbauer Wolfgang (STAT) wrote:

>If you know the d-value and the corresponding
>group sizes for a study, then it's possible to
>add that study to the rest of the dataset. Also,
>if you only know the test statistic from an
>independent samples t-test (or only the p-value
>corresponding to that test), it's possible to
>back-compute what the standardized mean difference is.
>
>I added an illustration of this to the metafor package website:
>
>http://www.metafor-project.org/doku.php/tips:assembling_data_smd

Verena might also like to look at the compute.es
package available from CRAN to see whether any of
the conversions programmed there do the job.


>Best,
>Wolfgang
>
>--
>Wolfgang Viechtbauer, Ph.D., Statistician
>Department of Psychiatry and Psychology
>School for Mental Health and Neuroscience
>Faculty of Health, Medicine, and Life Sciences
>Maastricht University, P.O. Box 616 (VIJV1)
>6200 MD Maastricht, The Netherlands
>+31 (43) 388-4170 | http://www.wvbauer.com
>
> > -----Original Message-----
> > From: [hidden email] [mailto:[hidden email]]
> > On Behalf Of Michael Dewey
> > Sent: Friday, April 25, 2014 16:23
> > To: Verena Weinbir
> > Cc: [hidden email]
> > Subject: Re: [R] Metafor: How to integrate effectsizes?
> >
> > At 12:33 25/04/2014, you wrote:
> > >Thank you very much for your reply and the book recommendation, Michael.
> > >
> > >Yes, I mean Cohen's d - sorry for the typo :-)
> > >
> > >Just to make this sure for me: There is no
> > >possibility to integrate stated Cohens' ds in an
> > >R-Metaanalysis (or a MA at all), if there is no
> > >further information traceable regarding SE or the like?
> >
> > If there is really no other information like
> > sample sizes, significance level, value of some
> > significance test then you would have to impute a
> > value from somewhere. That would seem a last resort.
> >
> > I have cc'ed this back to the list, please keep
> > it on the list so others may benefit and contribute.
> >
> >
> > >best regards,
> > >
> > >Verena
> > >
> > >
> > >On Fri, Apr 25, 2014 at 1:21 PM, Michael Dewey
> > ><<mailto:[hidden email]>[hidden email]> wrote:
> > >At 13:15 24/04/2014, Verena Weinbir wrote:
> > >Hello!
> > >
> > >I am using the metafor package for my master's thesis as an R-newbie.
> > While
> > >calculating effectsizes from my dataset (mean values and
> > >standarddeviations) using "escalc" shouldn't be a problem (I hope ;-)),
> > I
> > >wonder how I could at this point integrate additional studies, which
> > only
> > >state conhens d (no information about mean value and sds available), to
> > >calculate an overall analysis. Â I would be very grateful for your
> > support!
> > >
> > >
> > >You mean Cohen's d I think.
> > >
> > >You will need some more information to enable
> > >you to calculate its standard error. Have a look at Rosenthal's chapter
> > in
> > >@book{cooper94,
> > >Â  Â author = {Cooper, H and Hedges, L V},
> > >Â  Â title = {A handbook of research synthesis},
> > >Â  Â year = {1994},
> > >Â  Â publisher = {Russell Sage},
> > >Â  Â address = {New York},
> > >Â  Â keywords = {meta-analysis}
> > >}
> > >(There is an updated edition)
> > >This gives you more information about converting
> > >effect sizes and extracting them from unpromising beginnings.
> > >
> > >It often requires some ingenuity to get the
> > >information you need so have a go and then get
> > >back here with more details if you run into problems
> > >
> > >
> > >Best regards,
> > >
> > >Verena

Michael Dewey
[hidden email]
http://www.aghmed.fsnet.co.uk/home.html

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: How to integrate effectsizes?

Verena Weinbir
Thank you very much for your illustration, Wolfgang! It helped me a lot.
And also thank you for the package-hint, Michael!

Now, I have re-checked the respective studies, and there still are a couple
of studies left, only stating cohens d, and the respective t-value and
p-value - sample and group sizes are not addressed (its data from an older
meta-analysis). Is there a way to embed these studies in my sample?
Wolfgangs illustration addresses only cases in which group sizes are
stated, if I understand you correctly...

Many thanks in advance,

Verena


On Sat, Apr 26, 2014 at 1:38 PM, Michael Dewey <[hidden email]>wrote:

> At 20:34 25/04/2014, Viechtbauer Wolfgang (STAT) wrote:
>
>> If you know the d-value and the corresponding group sizes for a study,
>> then it's possible to add that study to the rest of the dataset. Also, if
>> you only know the test statistic from an independent samples t-test (or
>> only the p-value corresponding to that test), it's possible to back-compute
>> what the standardized mean difference is.
>>
>> I added an illustration of this to the metafor package website:
>>
>> http://www.metafor-project.org/doku.php/tips:assembling_data_smd
>>
>
> Verena might also like to look at the compute.es package available from
> CRAN to see whether any of the conversions programmed there do the job.
>
>
>
>  Best,
>> Wolfgang
>>
>> --
>> Wolfgang Viechtbauer, Ph.D., Statistician
>> Department of Psychiatry and Psychology
>> School for Mental Health and Neuroscience
>> Faculty of Health, Medicine, and Life Sciences
>> Maastricht University, P.O. Box 616 (VIJV1)
>> 6200 MD Maastricht, The Netherlands
>> +31 (43) 388-4170 | http://www.wvbauer.com
>>
>> > -----Original Message-----
>> > From: [hidden email] [mailto:[hidden email]
>> ]
>> > On Behalf Of Michael Dewey
>> > Sent: Friday, April 25, 2014 16:23
>> > To: Verena Weinbir
>> > Cc: [hidden email]
>> > Subject: Re: [R] Metafor: How to integrate effectsizes?
>> >
>> > At 12:33 25/04/2014, you wrote:
>> > >Thank you very much for your reply and the book recommendation,
>> Michael.
>> > >
>> > >Yes, I mean Cohen's d - sorry for the typo :-)
>> > >
>> > >Just to make this sure for me: There is no
>> > >possibility to integrate stated Cohens' ds in an
>> > >R-Metaanalysis (or a MA at all), if there is no
>> > >further information traceable regarding SE or the like?
>> >
>> > If there is really no other information like
>> > sample sizes, significance level, value of some
>> > significance test then you would have to impute a
>> > value from somewhere. That would seem a last resort.
>> >
>> > I have cc'ed this back to the list, please keep
>> > it on the list so others may benefit and contribute.
>> >
>> >
>> > >best regards,
>> > >
>> > >Verena
>> > >
>> > >
>> > >On Fri, Apr 25, 2014 at 1:21 PM, Michael Dewey
>> > ><<mailto:[hidden email]>[hidden email]> wrote:
>> > >At 13:15 24/04/2014, Verena Weinbir wrote:
>> > >Hello!
>> > >
>> > >I am using the metafor package for my master's thesis as an R-newbie.
>> > While
>> > >calculating effectsizes from my dataset (mean values and
>> > >standarddeviations) using "escalc" shouldn't be a problem (I hope ;-)),
>> > I
>> > >wonder how I could at this point integrate additional studies, which
>> > only
>> > >state conhens d (no information about mean value and sds available), to
>> > >calculate an overall analysis. Â I would be very grateful for your
>> > support!
>> > >
>> > >
>> > >You mean Cohen's d I think.
>> > >
>> > >You will need some more information to enable
>> > >you to calculate its standard error. Have a look at Rosenthal's chapter
>> > in
>> > >@book{cooper94,
>> > >  Ã‚ author = {Cooper, H and Hedges, L V},
>> > >  Ã‚ title = {A handbook of research synthesis},
>> > >  Ã‚ year = {1994},
>> > >  Ã‚ publisher = {Russell Sage},
>> > >  Ã‚ address = {New York},
>> > >  Ã‚ keywords = {meta-analysis}
>> > >}
>> > >(There is an updated edition)
>> > >This gives you more information about converting
>> > >effect sizes and extracting them from unpromising beginnings.
>> > >
>> > >It often requires some ingenuity to get the
>> > >information you need so have a go and then get
>> > >back here with more details if you run into problems
>> > >
>> > >
>> > >Best regards,
>> > >
>> > >Verena
>>
>
> Michael Dewey
> [hidden email]
> http://www.aghmed.fsnet.co.uk/home.html
>
>
        [[alternative HTML version deleted]]


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: How to integrate effectsizes?

Wolfgang Viechtbauer-2
Without the sample size of a study (i.e., either the group sizes or the total sample size), you cannot convert the p-value to a t-value or a t-value to a d-value. And for studies where you have the d-value but no sample size, you cannot compute the corresponding sampling variance. So, without additional information, you cannot include these studies. Maybe studies where a d-value is directly reported also report a CI for the d-value? Then the sampling variance can be back-calculated (since a 95% CI for d is typically computed with d +- 1.96 sqrt(vi), where vi is the sampling variance).

Best,
Wolfgang

> -----Original Message-----
> From: Verena Weinbir [mailto:[hidden email]]
> Sent: Tuesday, May 06, 2014 15:09
> To: Michael Dewey
> Cc: Viechtbauer Wolfgang (STAT); [hidden email]
> Subject: Re: [R] Metafor: How to integrate effectsizes?
>
> Thank you very much for your illustration, Wolfgang! It helped me a
> lot.  And also thank you for the package-hint, Michael!
>
> Now, I have re-checked the respective studies, and there still are a
> couple of studies left, only stating cohens d, and the respective t-value
> and p-value - sample and group sizes are not addressed (its data from an
> older meta-analysis). Is there a way to embed these studies in my sample?
> Wolfgangs illustration addresses only cases in which group sizes are
> stated, if I understand you correctly...
>
> Many thanks in advance,
>
> Verena
>
> On Sat, Apr 26, 2014 at 1:38 PM, Michael Dewey <[hidden email]>
> wrote:
> At 20:34 25/04/2014, Viechtbauer Wolfgang (STAT) wrote:
> If you know the d-value and the corresponding group sizes for a study,
> then it's possible to add that study to the rest of the dataset. Also, if
> you only know the test statistic from an independent samples t-test (or
> only the p-value corresponding to that test), it's possible to back-
> compute what the standardized mean difference is.
>
> I added an illustration of this to the metafor package website:
>
> http://www.metafor-project.org/doku.php/tips:assembling_data_smd
>
> Verena might also like to look at the compute.es package available from
> CRAN to see whether any of the conversions programmed there do the job.
>
>
> Best,
> Wolfgang
>
> --
> Wolfgang Viechtbauer, Ph.D., Statistician
> Department of Psychiatry and Psychology
> School for Mental Health and Neuroscience
> Faculty of Health, Medicine, and Life Sciences
> Maastricht University, P.O. Box 616 (VIJV1)
> 6200 MD Maastricht, The Netherlands
> +31 (43) 388-4170 | http://www.wvbauer.com
>
> > -----Original Message-----
> > From: [hidden email] [mailto:r-help-bounces@r-
> project.org]
> > On Behalf Of Michael Dewey
> > Sent: Friday, April 25, 2014 16:23
> > To: Verena Weinbir
> > Cc: [hidden email]
> > Subject: Re: [R] Metafor: How to integrate effectsizes?
> >
> > At 12:33 25/04/2014, you wrote:
> > >Thank you very much for your reply and the book recommendation,
> Michael.
> > >
> > >Yes, I mean Cohen's d - sorry for the typo :-)
> > >
> > >Just to make this sure for me: There is no
> > >possibility to integrate stated Cohens' ds in an
> > >R-Metaanalysis (or a MA at all), if there is no
> > >further information traceable regarding SE or the like?
> >
> > If there is really no other information like
> > sample sizes, significance level, value of some
> > significance test then you would have to impute a
> > value from somewhere. That would seem a last resort.
> >
> > I have cc'ed this back to the list, please keep
> > it on the list so others may benefit and contribute.
> >
> >
> > >best regards,
> > >
> > >Verena
> > >
> > >
> > >On Fri, Apr 25, 2014 at 1:21 PM, Michael Dewey
> > ><<mailto:[hidden email]>[hidden email]> wrote:
> > >At 13:15 24/04/2014, Verena Weinbir wrote:
> > >Hello!
> > >
> > >I am using the metafor package for my master's thesis as an R-newbie.
> > While
> > >calculating effectsizes from my dataset (mean values and
> > >standarddeviations) using "escalc" shouldn't be a problem (I hope ;-
> )),
> > I
> > >wonder how I could at this point integrate additional studies, which
> > only
> > >state conhens d (no information about mean value and sds available),
> to
> > >calculate an overall analysis. Â I would be very grateful for your
> > support!
> > >
> > >
> > >You mean Cohen's d I think.
> > >
> > >You will need some more information to enable
> > >you to calculate its standard error. Have a look at Rosenthal's
> chapter
> > in
> > >@book{cooper94,
> > >Â  Â author = {Cooper, H and Hedges, L V},
> > >Â  Â title = {A handbook of research synthesis},
> > >Â  Â year = {1994},
> > >Â  Â publisher = {Russell Sage},
> > >Â  Â address = {New York},
> > >Â  Â keywords = {meta-analysis}
> > >}
> > >(There is an updated edition)
> > >This gives you more information about converting
> > >effect sizes and extracting them from unpromising beginnings.
> > >
> > >It often requires some ingenuity to get the
> > >information you need so have a go and then get
> > >back here with more details if you run into problems
> > >
> > >
> > >Best regards,
> > >
> > >Verena
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: How to integrate effectsizes?

Michael Dewey
At 14:23 06/05/2014, Viechtbauer Wolfgang (STAT) wrote:

>Without the sample size of a study (i.e., either
>the group sizes or the total sample size), you
>cannot convert the p-value to a t-value or a
>t-value to a d-value. And for studies where you
>have the d-value but no sample size, you cannot
>compute the corresponding sampling variance. So,
>without additional information, you cannot
>include these studies. Maybe studies where a
>d-value is directly reported also report a CI
>for the d-value? Then the sampling variance can
>be back-calculated (since a 95% CI for d is
>typically computed with d +- 1.96 sqrt(vi), where vi is the sampling variance).

Verena,
What Wolfgang says is true of course but if you
have _both_ the t value and the p value you can
backcalculate the number of degrees of freedom
and then if you are willing to assume equal arms you have the sample sizes.

finddf <- function(t, pval) {
    helper <- function(df) {res <- pval - pt(t, df, lower.tail = FALSE); res}
    res <- uniroot(helper, interval = c(5, 10000))
    res
}

If you call finddf with the value of t and the
_one-sided_ p-value (divide by 2 if two-sided) it
should give you a return value which, if you look
at the element of the list called root is its
estimate of the degrees of freedom. If you get
errors from uniroot the interval supplied in the call may need to be widened.

I would suggest that when you have your final
dataset it would be a really good idea to do some
model checks using plot.influence to see whether
the studies for which you have imputed values are
fundamentally different for some reason. This
will also check your calculations as a bonus.


>Best,
>Wolfgang
>
> > -----Original Message-----
> > From: Verena Weinbir [mailto:[hidden email]]
> > Sent: Tuesday, May 06, 2014 15:09
> > To: Michael Dewey
> > Cc: Viechtbauer Wolfgang (STAT); [hidden email]
> > Subject: Re: [R] Metafor: How to integrate effectsizes?
> >
> > Thank you very much for your illustration, Wolfgang! It helped me a
> > lot.  And also thank you for the package-hint, Michael!
> >
> > Now, I have re-checked the respective studies, and there still are a
> > couple of studies left, only stating cohens d, and the respective t-value
> > and p-value - sample and group sizes are not addressed (its data from an
> > older meta-analysis). Is there a way to embed these studies in my sample?
> > Wolfgangs illustration addresses only cases in which group sizes are
> > stated, if I understand you correctly...
> >
> > Many thanks in advance,
> >
> > Verena
> >
> > On Sat, Apr 26, 2014 at 1:38 PM, Michael Dewey <[hidden email]>
> > wrote:
> > At 20:34 25/04/2014, Viechtbauer Wolfgang (STAT) wrote:
> > If you know the d-value and the corresponding group sizes for a study,
> > then it's possible to add that study to the rest of the dataset. Also, if
> > you only know the test statistic from an independent samples t-test (or
> > only the p-value corresponding to that test), it's possible to back-
> > compute what the standardized mean difference is.
> >
> > I added an illustration of this to the metafor package website:
> >
> > http://www.metafor-project.org/doku.php/tips:assembling_data_smd
> >
> > Verena might also like to look at the compute.es package available from
> > CRAN to see whether any of the conversions programmed there do the job.
> >
> >
> > Best,
> > Wolfgang
> >
> > --
> > Wolfgang Viechtbauer, Ph.D., Statistician
> > Department of Psychiatry and Psychology
> > School for Mental Health and Neuroscience
> > Faculty of Health, Medicine, and Life Sciences
> > Maastricht University, P.O. Box 616 (VIJV1)
> > 6200 MD Maastricht, The Netherlands
> > +31 (43) 388-4170Â | http://www.wvbauer.com
> >
> > > -----Original Message-----
> > > From: [hidden email] [mailto:r-help-bounces@r-
> > project.org]
> > > On Behalf Of Michael Dewey
> > > Sent: Friday, April 25, 2014 16:23
> > > To: Verena Weinbir
> > > Cc: [hidden email]
> > > Subject: Re: [R] Metafor: How to integrate effectsizes?
> > >
> > > At 12:33 25/04/2014, you wrote:
> > > >Thank you very much for your reply and the book recommendation,
> > Michael.
> > > >
> > > >Yes, I mean Cohen's d - sorry for the typo :-)
> > > >
> > > >Just to make this sure for me: There is no
> > > >possibility to integrate stated Cohens' ds in an
> > > >R-Metaanalysis (or a MA at all), if there is no
> > > >further information traceable regarding SE or the like?
> > >
> > > If there is really no other information like
> > > sample sizes, significance level, value of some
> > > significance test then you would have to impute a
> > > value from somewhere. That would seem a last resort.
> > >
> > > I have cc'ed this back to the list, please keep
> > > it on the list so others may benefit and contribute.
> > >
> > >
> > > >best regards,
> > > >
> > > >Verena
> > > >
> > > >
> > > >On Fri, Apr 25, 2014 at 1:21 PM, Michael Dewey
> > > ><<mailto:[hidden email]>[hidden email]> wrote:
> > > >At 13:15 24/04/2014, Verena Weinbir wrote:
> > > >Hello!
> > > >
> > > >I am using the metafor package for my master's thesis as an R-newbie.
> > > While
> > > >calculating effectsizes from my dataset (mean values and
> > > >standarddeviations) using "escalc" shouldn't be a problem (I hope ;-
> > )),
> > > I
> > > >wonder how I could at this point integrate additional studies, which
> > > only
> > > >state conhens d (no information about mean value and sds available),
> > to
> > > >calculate an overall analysis. Â I would be very grateful for your
> > > support!
> > > >
> > > >
> > > >You mean Cohen's d I think.
> > > >
> > > >You will need some more information to enable
> > > >you to calculate its standard error. Have a look at Rosenthal's
> > chapter
> > > in
> > > >@book{cooper94,
> > > >Â  Â author = {Cooper, H and Hedges, L V},
> > > >Â  Â title = {A handbook of research synthesis},
> > > >Â  Â year = {1994},
> > > >Â  Â publisher = {Russell Sage},
> > > >Â  Â address = {New York},
> > > >Â  Â keywords = {meta-analysis}
> > > >}
> > > >(There is an updated edition)
> > > >This gives you more information about converting
> > > >effect sizes and extracting them from unpromising beginnings.
> > > >
> > > >It often requires some ingenuity to get the
> > > >information you need so have a go and then get
> > > >back here with more details if you run into problems
> > > >
> > > >
> > > >Best regards,
> > > >
> > > >Verena

Michael Dewey
[hidden email]
http://www.aghmed.fsnet.co.uk/home.html

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: How to integrate effectsizes?

Verena Weinbir
Hello,

thank you very much for your replies. I am almost done :-) but theres one
study left, where I only have sample size (not group size),  mean values
and standarddeviations. Is there a way to compute cohens d from this data?

I thought it was correct to use measure="SMDH" in the escalc () function to
compute cohens d? In your illustration, Wolfgang, you use SMD as measure -
now I am confused ;-)

Thank you very much in advance!

Best,
Verena


On Tue, May 6, 2014 at 7:14 PM, Michael Dewey <[hidden email]>wrote:

> At 14:23 06/05/2014, Viechtbauer Wolfgang (STAT) wrote:
>
>> Without the sample size of a study (i.e., either the group sizes or the
>> total sample size), you cannot convert the p-value to a t-value or a
>> t-value to a d-value. And for studies where you have the d-value but no
>> sample size, you cannot compute the corresponding sampling variance. So,
>> without additional information, you cannot include these studies. Maybe
>> studies where a d-value is directly reported also report a CI for the
>> d-value? Then the sampling variance can be back-calculated (since a 95% CI
>> for d is typically computed with d +- 1.96 sqrt(vi), where vi is the
>> sampling variance).
>>
>
> Verena,
> What Wolfgang says is true of course but if you have _both_ the t value
> and the p value you can backcalculate the number of degrees of freedom and
> then if you are willing to assume equal arms you have the sample sizes.
>
> finddf <- function(t, pval) {
>    helper <- function(df) {res <- pval - pt(t, df, lower.tail = FALSE);
> res}
>    res <- uniroot(helper, interval = c(5, 10000))
>    res
> }
>
> If you call finddf with the value of t and the _one-sided_ p-value (divide
> by 2 if two-sided) it should give you a return value which, if you look at
> the element of the list called root is its estimate of the degrees of
> freedom. If you get errors from uniroot the interval supplied in the call
> may need to be widened.
>
> I would suggest that when you have your final dataset it would be a really
> good idea to do some model checks using plot.influence to see whether the
> studies for which you have imputed values are fundamentally different for
> some reason. This will also check your calculations as a bonus.
>
>
>  Best,
>> Wolfgang
>>
>> > -----Original Message-----
>> > From: Verena Weinbir [mailto:[hidden email]]
>> > Sent: Tuesday, May 06, 2014 15:09
>> > To: Michael Dewey
>> > Cc: Viechtbauer Wolfgang (STAT); [hidden email]
>> > Subject: Re: [R] Metafor: How to integrate effectsizes?
>> >
>> > Thank you very much for your illustration, Wolfgang! It helped me a
>> > lot.  And also thank you for the package-hint, Michael!
>>
>> >
>> > Now, I have re-checked the respective studies, and there still are a
>> > couple of studies left, only stating cohens d, and the respective
>> t-value
>> > and p-value - sample and group sizes are not addressed (its data from an
>> > older meta-analysis). Is there a way to embed these studies in my
>> sample?
>> > Wolfgangs illustration addresses only cases in which group sizes are
>> > stated, if I understand you correctly...
>> >
>> > Many thanks in advance,
>> >
>> > Verena
>> >
>> > On Sat, Apr 26, 2014 at 1:38 PM, Michael Dewey <[hidden email]
>> >
>> > wrote:
>> > At 20:34 25/04/2014, Viechtbauer Wolfgang (STAT) wrote:
>> > If you know the d-value and the corresponding group sizes for a study,
>> > then it's possible to add that study to the rest of the dataset. Also,
>> if
>> > you only know the test statistic from an independent samples t-test (or
>> > only the p-value corresponding to that test), it's possible to back-
>> > compute what the standardized mean difference is.
>> >
>> > I added an illustration of this to the metafor package website:
>> >
>> > http://www.metafor-project.org/doku.php/tips:assembling_data_smd
>> >
>> > Verena might also like to look at the compute.es package available from
>> > CRAN to see whether any of the conversions programmed there do the job.
>> >
>> >
>> > Best,
>> > Wolfgang
>> >
>> > --
>> > Wolfgang Viechtbauer, Ph.D., Statistician
>> > Department of Psychiatry and Psychology
>> > School for Mental Health and Neuroscience
>> > Faculty of Health, Medicine, and Life Sciences
>> > Maastricht University, P.O. Box 616 (VIJV1)
>> > 6200 MD Maastricht, The Netherlands
>> > +31 (43) 388-4170Â | http://www.wvbauer.com
>>
>> >
>> > > -----Original Message-----
>> > > From: [hidden email] [mailto:r-help-bounces@r-
>> > project.org]
>> > > On Behalf Of Michael Dewey
>> > > Sent: Friday, April 25, 2014 16:23
>> > > To: Verena Weinbir
>> > > Cc: [hidden email]
>> > > Subject: Re: [R] Metafor: How to integrate effectsizes?
>> > >
>> > > At 12:33 25/04/2014, you wrote:
>> > > >Thank you very much for your reply and the book recommendation,
>> > Michael.
>> > > >
>> > > >Yes, I mean Cohen's d - sorry for the typo :-)
>> > > >
>> > > >Just to make this sure for me: There is no
>> > > >possibility to integrate stated Cohens' ds in an
>> > > >R-Metaanalysis (or a MA at all), if there is no
>> > > >further information traceable regarding SE or the like?
>> > >
>> > > If there is really no other information like
>> > > sample sizes, significance level, value of some
>> > > significance test then you would have to impute a
>> > > value from somewhere. That would seem a last resort.
>> > >
>> > > I have cc'ed this back to the list, please keep
>> > > it on the list so others may benefit and contribute.
>> > >
>> > >
>> > > >best regards,
>> > > >
>> > > >Verena
>> > > >
>> > > >
>> > > >On Fri, Apr 25, 2014 at 1:21 PM, Michael Dewey
>> > > ><<mailto:[hidden email]>[hidden email]> wrote:
>> > > >At 13:15 24/04/2014, Verena Weinbir wrote:
>> > > >Hello!
>> > > >
>> > > >I am using the metafor package for my master's thesis as an R-newbie.
>> > > While
>> > > >calculating effectsizes from my dataset (mean values and
>> > > >standarddeviations) using "escalc" shouldn't be a problem (I hope ;-
>> > )),
>> > > I
>> > > >wonder how I could at this point integrate additional studies, which
>> > > only
>> > > >state conhens d (no information about mean value and sds available),
>> > to
>> > > >calculate an overall analysis. Â I would be very grateful for your
>>
>> > > support!
>> > > >
>> > > >
>> > > >You mean Cohen's d I think.
>> > > >
>> > > >You will need some more information to enable
>> > > >you to calculate its standard error. Have a look at Rosenthal's
>> > chapter
>> > > in
>> > > >@book{cooper94,
>> > > >Â  Â author = {Cooper, H and Hedges, L V},
>> > > >Â  Â title = {A handbook of research synthesis},
>> > > >Â  Â year = {1994},
>> > > >Â  Â publisher = {Russell Sage},
>> > > >Â  Â address = {New York},
>> > > >Â  Â keywords = {meta-analysis}
>>
>> > > >}
>> > > >(There is an updated edition)
>> > > >This gives you more information about converting
>> > > >effect sizes and extracting them from unpromising beginnings.
>> > > >
>> > > >It often requires some ingenuity to get the
>> > > >information you need so have a go and then get
>> > > >back here with more details if you run into problems
>> > > >
>> > > >
>> > > >Best regards,
>> > > >
>> > > >Verena
>>
>
> Michael Dewey
> [hidden email]
> http://www.aghmed.fsnet.co.uk/home.html
>
>
        [[alternative HTML version deleted]]


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: How to integrate effectsizes?

Michael Dewey
At 13:10 22/05/2014, Verena Weinbir wrote:
>Hello,
>
>thank you very much for your replies. I am
>almost done :-) but theres one study left, where
>I only have sample size (not group size),  mean
>values and standarddeviations. Is there a way to
>compute cohens d from this data?

You could assume that the split was equal. If you
are worried you could, as a sensitivity analysis,
assume a plausible range of splits and see if
that affects the outcome. If these are
experiments or trials it is unlikely people would
have arranged anything too asymmetrical.


>I thought it was correct to use measure="SMDH"
>in the escalc () function to compute cohens d?
>In your illustration, Wolfgang, you use SMD as measure - now I am confused ;-)

People usually use SMD I think, the other relaxes
the assumption about homogeneity of variances.


>Thank you very much in advance!
>
>Best,
>Verena
>
>
>On Tue, May 6, 2014 at 7:14 PM, Michael Dewey
><<mailto:[hidden email]>[hidden email]> wrote:
>At 14:23 06/05/2014, Viechtbauer Wolfgang (STAT) wrote:
>Without the sample size of a study (i.e., either
>the group sizes or the total sample size), you
>cannot convert the p-value to a t-value or a
>t-value to a d-value. And for studies where you
>have the d-value but no sample size, you cannot
>compute the corresponding sampling variance. So,
>without additional information, you cannot
>include these studies. Maybe studies where a
>d-value is directly reported also report a CI
>for the d-value? Then the sampling variance can
>be back-calculated (since a 95% CI for d is
>typically computed with d +- 1.96 sqrt(vi), where vi is the sampling variance).
>
>
>Verena,
>What Wolfgang says is true of course but if you
>have _both_ the t value and the p value you can
>backcalculate the number of degrees of freedom
>and then if you are willing to assume equal arms you have the sample sizes.
>
>finddf <- function(t, pval) {
>Â  Â helper <- function(df) {res <- pval - pt(t, df, lower.tail = FALSE); res}
>Â  Â res <- uniroot(helper, interval = c(5, 10000))
>Â  Â res
>}
>
>If you call finddf with the value of t and the
>_one-sided_ p-value (divide by 2 if two-sided)
>it should give you a return value which, if you
>look at the element of the list called root is
>its estimate of the degrees of freedom. If you
>get errors from uniroot the interval supplied in
>the call may need to be widened.
>
>I would suggest that when you have your final
>dataset it would be a really good idea to do
>some model checks using plot.influence to see
>whether the studies for which you have imputed
>values are fundamentally different for some
>reason. This will also check your calculations as a bonus.
>
>
>Best,
>Wolfgang
>
> > -----Original Message-----
> > From: Verena Weinbir [mailto:[hidden email]]
> > Sent: Tuesday, May 06, 2014 15:09
> > To: Michael Dewey
> > Cc: Viechtbauer Wolfgang (STAT);
> <mailto:[hidden email]>[hidden email]
> > Subject: Re: [R] Metafor: How to integrate effectsizes?
> >
> > Thank you very much for your illustration, Wolfgang! It helped me a
> > lot.  And also thank you for the package-hint, Michael!
>
> >
> > Now, I have re-checked the respective studies, and there still are a
> > couple of studies left, only stating cohens d, and the respective t-value
> > and p-value - sample and group sizes are not addressed (its data from an
> > older meta-analysis). Is there a way to embed these studies in my sample?
> > Wolfgangs illustration addresses only cases in which group sizes are
> > stated, if I understand you correctly...
> >
> > Many thanks in advance,
> >
> > Verena
> >
> > On Sat, Apr 26, 2014 at 1:38 PM, Michael
> Dewey <<mailto:[hidden email]>[hidden email]>
> > wrote:
> > At 20:34 25/04/2014, Viechtbauer Wolfgang (STAT) wrote:
> > If you know the d-value and the corresponding group sizes for a study,
> > then it's possible to add that study to the rest of the dataset. Also, if
> > you only know the test statistic from an independent samples t-test (or
> > only the p-value corresponding to that test), it's possible to back-
> > compute what the standardized mean difference is.
> >
> > I added an illustration of this to the metafor package website:
> >
> >
> <http://www.metafor-project.org/doku.php/tips:assembling_data_smd>http://www.metafor-project.org/doku.php/tips:assembling_data_smd
> >
> > Verena might also like to look at the
> <http://compute.es>compute.es package available from
> > CRAN to see whether any of the conversions programmed there do the job.
> >
> >
> > Best,
> > Wolfgang
> >
> > --
> > Wolfgang Viechtbauer, Ph.D., Statistician
> > Department of Psychiatry and Psychology
> > School for Mental Health and Neuroscience
> > Faculty of Health, Medicine, and Life Sciences
> > Maastricht University, P.O. Box 616 (VIJV1)
> > 6200 MD Maastricht, The Netherlands
> > +31 (43) 388-4170Â | <http://www.wvbauer.com>http://www.wvbauer.com
>
> >
> > > -----Original Message-----
> > > From:
> <mailto:[hidden email]>[hidden email]
> [mailto:<mailto:r-help-bounces@r->r-help-bounces@r-
> > <http://project.org>project.org]
> > > On Behalf Of Michael Dewey
> > > Sent: Friday, April 25, 2014 16:23
> > > To: Verena Weinbir
> > > Cc: <mailto:[hidden email]>[hidden email]
> > > Subject: Re: [R] Metafor: How to integrate effectsizes?
> > >
> > > At 12:33 25/04/2014, you wrote:
> > > >Thank you very much for your reply and the book recommendation,
> > Michael.
> > > >
> > > >Yes, I mean Cohen's d - sorry for the typo :-)
> > > >
> > > >Just to make this sure for me: There is no
> > > >possibility to integrate stated Cohens' ds in an
> > > >R-Metaanalysis (or a MA at all), if there is no
> > > >further information traceable regarding SE or the like?
> > >
> > > If there is really no other information like
> > > sample sizes, significance level, value of some
> > > significance test then you would have to impute a
> > > value from somewhere. That would seem a last resort.
> > >
> > > I have cc'ed this back to the list, please keep
> > > it on the list so others may benefit and contribute.
> > >
> > >
> > > >best regards,
> > > >
> > > >Verena
> > > >
> > > >
> > > >On Fri, Apr 25, 2014 at 1:21 PM, Michael Dewey
> > > ><<mailto:[hidden email]><mailto:in
> [hidden email]>[hidden email]> wrote:
> > > >At 13:15 24/04/2014, Verena Weinbir wrote:
> > > >Hello!
> > > >
> > > >I am using the metafor package for my master's thesis as an R-newbie.
> > > While
> > > >calculating effectsizes from my dataset (mean values and
> > > >standarddeviations) using "escalc" shouldn't be a problem (I hope ;-
> > )),
> > > I
> > > >wonder how I could at this point integrate additional studies, which
> > > only
> > > >state conhens d (no information about mean value and sds available),
> > to
> > > >calculate an overall analysis. Â I would be verry grateful for your
>
> > > support!
> > > >
> > > >
> > > >You mean Cohen's d I think.
> > > >
> > > >You will need some more information to enable
> > > >you to calculate its standard error. Have a look at Rosenthal's
> > chapter
> > > in
> > > >@book{cooper94,
> > > >Â  Â author = {Cooper, H H and Hedges, L V},
> > > >Â  Â title = {A handbook ok of research synthesis},
> > > >Â  Â year = {1994},
> > > > >Â  Â publisher = {Russellell Sage},
> > > >Â  Â address = {New York}rk},
> > > >Â  Â keywords = {meta-anaanalysis}
>
> > > >}
> > > >(There is an updated edition)
> > > >This gives you more information about converting
> > > >effect sizes and extracting them from unpromising beginnings.
> > > >
> > > >It often requires some ingenuity to get the
> > > >information you need so have a go and then get
> > > >back here with more details if you run into problems
> > > >
> > > >
> > > >Best regards,
> > > >
> > > >Verena
>
>
>Michael Dewey
><mailto:[hidden email]>[hidden email]
>http://www.aghmed.fsnet.co.uk/home.html
>

Michael Dewey
[hidden email]
http://www.aghmed.fsnet.co.uk/home.html

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.