Metafor: Strange Trim and Fill Outcome?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Metafor: Strange Trim and Fill Outcome?

Verena Weinbir
Hey guys,

I have tested the metafor trim and fill function (y:SD, x:SMD)on my data
set and yielded the following result:

1. missing studies on the right: 34

2. open circles on the rights side appear to be the number of additional
effects

3. adjusted d would be higher than observed d.

Since normally, as I understand, those parameters are the other way round
(black dots indicating missing studies on the left, which would reduce the
effect size), I wonder:

Is there a mistake I have done? Or, if this is an actual outcome how can I
interpret this?  That its not a publication bias that influences my data
set, but a lack of precision (studies missing that are precise -small SD-
and have big SMD)?

Many thanks in advance!

Verena

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: Strange Trim and Fill Outcome?

Michael Dewey
At 09:34 26/05/2014, Verena Weinbir wrote:
>Hey guys,
>
>I have tested the metafor trim and fill function (y:SD, x:SMD)on my data
>set and yielded the following result:
>
>1. missing studies on the right: 34

That seems a lot of missing studies unless you have a very large set
of primary studies.


>2. open circles on the rights side appear to be the number of additional
>effects
>
>3. adjusted d would be higher than observed d.

Implying that the mechanism is suppressing studies which found a
large effect. This might happen if the dominant view is that there is
no effect and so when people find one they worry about their results.


>Since normally, as I understand, those parameters are the other way round
>(black dots indicating missing studies on the left, which would reduce the
>effect size), I wonder:
>
>Is there a mistake I have done? Or, if this is an actual outcome how can I
>interpret this?  That its not a publication bias that influences my data
>set, but a lack of precision (studies missing that are precise -small SD-
>and have big SMD)?

Sorry but that bit is not very clear to me.

>Many thanks in advance!
>
>Verena
>
>         [[alternative HTML version deleted]]

Michael Dewey
[hidden email]
http://www.aghmed.fsnet.co.uk/home.html

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: Strange Trim and Fill Outcome?

Verena Weinbir
thanks for your reply, Michael!

ad primary studies:
yup, I have a large set of studies: At the moment I consider 126 data sets
in my analysis.

ad interpretation:
thats an interesting information. But usually there should be found an
effect.

ad "bit" :-):
I am irritiated, because in the plots I have studied so far, I always found
that the observed studies are open circles and the additional "mirrored"
effects are black dots. In my case, it is the other way round.

Also, most of the mirrored circels appear in the right upper corner,
indicating, that there are studies missing in my data set, which have
smaller SDs (i.e. are more precise?) and higher effect sizes?

best,

Verena




On Mon, May 26, 2014 at 12:26 PM, Michael Dewey <[hidden email]>wrote:

> At 09:34 26/05/2014, Verena Weinbir wrote:
>
>> Hey guys,
>>
>> I have tested the metafor trim and fill function (y:SD, x:SMD)on my data
>> set and yielded the following result:
>>
>> 1. missing studies on the right: 34
>>
>
> That seems a lot of missing studies unless you have a very large set of
> primary studies.
>
>
>
>  2. open circles on the rights side appear to be the number of additional
>> effects
>>
>> 3. adjusted d would be higher than observed d.
>>
>
> Implying that the mechanism is suppressing studies which found a large
> effect. This might happen if the dominant view is that there is no effect
> and so when people find one they worry about their results.
>
>
>
>  Since normally, as I understand, those parameters are the other way round
>> (black dots indicating missing studies on the left, which would reduce the
>> effect size), I wonder:
>>
>> Is there a mistake I have done? Or, if this is an actual outcome how can I
>> interpret this?  That its not a publication bias that influences my data
>> set, but a lack of precision (studies missing that are precise -small SD-
>> and have big SMD)?
>>
>
> Sorry but that bit is not very clear to me.
>
>  Many thanks in advance!
>>
>> Verena
>>
>>         [[alternative HTML version deleted]]
>>
>
> Michael Dewey
> [hidden email]
> http://www.aghmed.fsnet.co.uk/home.html
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: Strange Trim and Fill Outcome?

Wolfgang Viechtbauer-2
>From help(forest.rma):

pch -- plotting symbol to use for the observed effect sizes or outcomes. By default, a solid circle is used. Can also be a vector of values. See points for other options.

pch.fill -- plotting symbol to use for the effect sizes or outcomes filled in by the trim and fill method. By default, a circle is used. Only relevant when plotting an object created by the trimfill function.

The defaults are pch=19 and pch.fill=21. So, by default, a solid circle is used for the observed outcomes and an open circle is used for the outcomes that are filled in.

An example is shown here:

http://www.metafor-project.org/doku.php/plots:funnel_plot_with_trim_and_fill

>From help(trimfill.rma.uni):

side -- either "left" or "right", indicating on which side of the funnel plot the missing studies should be imputed. If left undefined, the side is chosen within the function depending on the results of Egger's regression test (see regtest for details on this test).

The argument is left undefined by default, so the side is chosen based on the results of the regression test (essentially, whether the slope is positive or negative). If you think the suppression occurred on the other side than the one that is chosen, specify the side via this argument.

Best,
Wolfgang
________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Verena Weinbir [[hidden email]]
Sent: Monday, May 26, 2014 1:15 PM
To: Michael Dewey
Cc: r-help
Subject: Re: [R] Metafor: Strange Trim and Fill Outcome?

thanks for your reply, Michael!

ad primary studies:
yup, I have a large set of studies: At the moment I consider 126 data sets
in my analysis.

ad interpretation:
thats an interesting information. But usually there should be found an
effect.

ad "bit" :-):
I am irritiated, because in the plots I have studied so far, I always found
that the observed studies are open circles and the additional "mirrored"
effects are black dots. In my case, it is the other way round.

Also, most of the mirrored circels appear in the right upper corner,
indicating, that there are studies missing in my data set, which have
smaller SDs (i.e. are more precise?) and higher effect sizes?

best,

Verena

On Mon, May 26, 2014 at 12:26 PM, Michael Dewey <[hidden email]>wrote:

> At 09:34 26/05/2014, Verena Weinbir wrote:
>
>> Hey guys,
>>
>> I have tested the metafor trim and fill function (y:SD, x:SMD)on my data
>> set and yielded the following result:
>>
>> 1. missing studies on the right: 34
>>
> That seems a lot of missing studies unless you have a very large set of
> primary studies.
>
>  2. open circles on the rights side appear to be the number of additional
>> effects
>>
>> 3. adjusted d would be higher than observed d.
>>
> Implying that the mechanism is suppressing studies which found a large
> effect. This might happen if the dominant view is that there is no effect
> and so when people find one they worry about their results.
>
>  Since normally, as I understand, those parameters are the other way round
>> (black dots indicating missing studies on the left, which would reduce the
>> effect size), I wonder:
>>
>> Is there a mistake I have done? Or, if this is an actual outcome how can I
>> interpret this?  That its not a publication bias that influences my data
>> set, but a lack of precision (studies missing that are precise -small SD-
>> and have big SMD)?
>>
> Sorry but that bit is not very clear to me.
>
>  Many thanks in advance!
>>
>> Verena
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: Strange Trim and Fill Outcome?

Verena Weinbir
thanks for your reply, Wolfgang!

Regarding the dots/circle -"problem" I am relieved now :-)

Regarding the Trim and Fill outcome I don't really think that it's the
wrong side, I just dont know how to interpret the result.

regtest () reveals a negative z, which is not significant.
ranktest () reveals a negative tau, which is not significant.

Any ideas? :-)

best,

Verena


On Mon, May 26, 2014 at 2:00 PM, Viechtbauer Wolfgang (STAT) <
[hidden email]> wrote:

> From help(forest.rma):
>
> pch -- plotting symbol to use for the observed effect sizes or outcomes.
> By default, a solid circle is used. Can also be a vector of values. See
> points for other options.
>
> pch.fill -- plotting symbol to use for the effect sizes or outcomes filled
> in by the trim and fill method. By default, a circle is used. Only relevant
> when plotting an object created by the trimfill function.
>
> The defaults are pch=19 and pch.fill=21. So, by default, a solid circle is
> used for the observed outcomes and an open circle is used for the outcomes
> that are filled in.
>
> An example is shown here:
>
>
> http://www.metafor-project.org/doku.php/plots:funnel_plot_with_trim_and_fill
>
> From help(trimfill.rma.uni):
>
> side -- either "left" or "right", indicating on which side of the funnel
> plot the missing studies should be imputed. If left undefined, the side is
> chosen within the function depending on the results of Egger's regression
> test (see regtest for details on this test).
>
> The argument is left undefined by default, so the side is chosen based on
> the results of the regression test (essentially, whether the slope is
> positive or negative). If you think the suppression occurred on the other
> side than the one that is chosen, specify the side via this argument.
>
> Best,
> Wolfgang
> ________________________________________
> From: [hidden email] [[hidden email]] On
> Behalf Of Verena Weinbir [[hidden email]]
> Sent: Monday, May 26, 2014 1:15 PM
> To: Michael Dewey
> Cc: r-help
> Subject: Re: [R] Metafor: Strange Trim and Fill Outcome?
>
> thanks for your reply, Michael!
>
> ad primary studies:
> yup, I have a large set of studies: At the moment I consider 126 data sets
> in my analysis.
>
> ad interpretation:
> thats an interesting information. But usually there should be found an
> effect.
>
> ad "bit" :-):
> I am irritiated, because in the plots I have studied so far, I always found
> that the observed studies are open circles and the additional "mirrored"
> effects are black dots. In my case, it is the other way round.
>
> Also, most of the mirrored circels appear in the right upper corner,
> indicating, that there are studies missing in my data set, which have
> smaller SDs (i.e. are more precise?) and higher effect sizes?
>
> best,
>
> Verena
>
> On Mon, May 26, 2014 at 12:26 PM, Michael Dewey <[hidden email]
> >wrote:
>
> > At 09:34 26/05/2014, Verena Weinbir wrote:
> >
> >> Hey guys,
> >>
> >> I have tested the metafor trim and fill function (y:SD, x:SMD)on my data
> >> set and yielded the following result:
> >>
> >> 1. missing studies on the right: 34
> >>
> > That seems a lot of missing studies unless you have a very large set of
> > primary studies.
> >
> >  2. open circles on the rights side appear to be the number of additional
> >> effects
> >>
> >> 3. adjusted d would be higher than observed d.
> >>
> > Implying that the mechanism is suppressing studies which found a large
> > effect. This might happen if the dominant view is that there is no effect
> > and so when people find one they worry about their results.
> >
> >  Since normally, as I understand, those parameters are the other way
> round
> >> (black dots indicating missing studies on the left, which would reduce
> the
> >> effect size), I wonder:
> >>
> >> Is there a mistake I have done? Or, if this is an actual outcome how
> can I
> >> interpret this?  That its not a publication bias that influences my data
> >> set, but a lack of precision (studies missing that are precise -small
> SD-
> >> and have big SMD)?
> >>
> > Sorry but that bit is not very clear to me.
> >
> >  Many thanks in advance!
> >>
> >> Verena
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.