Metafor: Differences between two categories of a moderator

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Metafor: Differences between two categories of a moderator

Holger Steinmetz
Hi there,

when using the metafor package for testing mixed effects models with categorical moderators, I get a regression table reporting betas. These betas are mean differences between a certain category in the effect size to the reference category (intercept). In addition, the QM-tests allows testing if the differences between all the categories TO the reference are significant or not.

The problem is that if there are differences AMONG the categories (but not to the reference), the QM-test will not notice that. Hence, my question is if there is an option to test differences among specific categories (e.g., simple subgroup comparison).

Best,
Holger
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: Differences between two categories of a moderator

Viechtbauer, Wolfgang (SP)
Dear Holger,

Actually, the omnibus test ("QM-test") will give you the same result regardless of which category you make the reference category (try it out!). So, it will pick up all of the differences, whether they are to the reference category or between any of the other categories.

If the omnibus test turns out significant, then you can examine the pairwise differences. Essentially, that is what the individual betas are. You will have to switch the reference category to get all of the pairwise comparisons (the relevel function comes in handy here).

Here is an example to illustrate all of this:

data(dat.bcg)
dat <- escalc(measure="RR", ai=tpos, bi=tneg, ci=cpos, di=cneg, data=dat.bcg, append=TRUE)

res <- rma(yi, vi, mods=~factor(alloc), data=dat)
res

res <- rma(yi, vi, mods=~relevel(factor(alloc),"random"), data=dat)
res

Best,

--
Wolfgang Viechtbauer
Department of Psychiatry and Neuropsychology
School for Mental Health and Neuroscience
Maastricht University, P.O. Box 616
6200 MD Maastricht, The Netherlands
Tel: +31 (43) 368-5248
Fax: +31 (43) 368-8689
Web: http://www.wvbauer.com
________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Holger Steinmetz [[hidden email]]
Sent: Tuesday, May 31, 2011 12:52 PM
To: [hidden email]
Subject: [R] Metafor: Differences between two categories of a moderator

Hi there,

when using the metafor package for testing mixed effects models with
categorical moderators, I get a regression table reporting betas. These
betas are mean differences between a certain category in the effect size to
the reference category (intercept). In addition, the QM-tests allows testing
if the differences between all the categories TO the reference are
significant or not.

The problem is that if there are differences AMONG the categories (but not
to the reference), the QM-test will not notice that. Hence, my question is
if there is an option to test differences among specific categories (e.g.,
simple subgroup comparison).

Best,
Holger


--
View this message in context: http://r.789695.n4.nabble.com/Metafor-Differences-between-two-categories-of-a-moderator-tp3562778p3562778.html
Sent from the R help mailing list archive at Nabble.com.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: Differences between two categories of a moderator

Holger Steinmetz
Hi Wolfgang

that's good news. One further small follow-up question: When I conduct multiple comparisons via the relevel-command: should I adjust the p-value?

Thanks in advance,
Holger
Reply | Threaded
Open this post in threaded view
|

Re: Metafor: Differences between two categories of a moderator

Viechtbauer, Wolfgang (SP)
Well, that's a good question. It actually applies to many different contexts (not just meta-analysis). Think of the ANOVA F-test and post-hoc/planned contrasts. It's essentially the same situation. And if you would ask 10 different statisticians about this, you may get 11 different answers.

My suggestion would be this: If you have no particular hypotheses in mind and are just screening for group differences, then adjust. If you have a priori hypotheses, then test them without adjustment (and in that case, you may even ignore the omnibus test).

Best,

--
Wolfgang Viechtbauer
Department of Psychiatry and Neuropsychology
School for Mental Health and Neuroscience
Maastricht University, P.O. Box 616
6200 MD Maastricht, The Netherlands
Tel: +31 (43) 368-5248
Fax: +31 (43) 368-8689
Web: http://www.wvbauer.com
________________________________________
From: [hidden email] [[hidden email]] On Behalf Of Holger Steinmetz [[hidden email]]
Sent: Wednesday, June 01, 2011 10:10 AM
To: [hidden email]
Subject: Re: [R] Metafor: Differences between two categories of a moderator

Hi Wolfgang

that's good news. One further small follow-up question: When I conduct
multiple comparisons via the relevel-command: should I adjust the p-value?

Thanks in advance,
Holger

--
View this message in context: http://r.789695.n4.nabble.com/Metafor-Differences-between-two-categories-of-a-moderator-tp3562778p3565210.html
Sent from the R help mailing list archive at Nabble.com.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.