Re: How to extract coefficients from sequential (type 1), ANOVAs using lmer and lme

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Re: How to extract coefficients from sequential (type 1), ANOVAs using lmer and lme

Phillip Alday-2
(This time with the r-help in the recipients...)

Be careful when mixing lme4 and lmerTest together -- lmerTest extends
and changes the behavior of various lme4 functions.

From the help page for lme4-anova (?lme4::anova.merMod)

>      ‘anova’: returns the sequential decomposition of the contributions
>           of fixed-effects terms or, for multiple arguments, model
>           comparison statistics.  For objects of class ‘lmerMod’ the
>           default behavior is to refit the models with ML if fitted
>           with ‘REML = TRUE’, this can be controlled via the ‘refit’
>           argument. See also ‘anova’.

So lme4-anova will give you sequential tests; note, however, that lme4
won't calculate the denominator degrees of freedom for you and thus
won't give p-values. See the FAQ
(https://cran.r-project.org/doc/FAQ/R-FAQ.html#Why-are-p_002dvalues-not-displayed-when-using-lmer_0028_0029_003f)

From the help page for lmerTest-anova (?lmerTest::anova.merModLmerTest):
> Usage:
>
>      ## S4 method for signature 'merModLmerTest'
>      anova(object, ... , ddf="Satterthwaite",
>      type=3)
>      
> Arguments:
>
...
>     type: type of hypothesis to be tested. Could be type=3 or type=2 or
>           type = 1 (The definition comes from SAS theory)


So lmerTest-anova by default gives you Type III ('marginal', although
Type II is what actually gives you tests that respect the Principle of
Marginality; see John Fox's Applied Regression Analysis (book) or
Venables' "Exegeses on Linear Models"
(https://www.stats.ox.ac.uk/pub/MASS3/Exegeses.pdf) for more information
on that. Type I tests are the sequential tests, so with anova(model,
type=1), you will get the sequential tests you want. lmerTest will
approximate the denominator degrees of freedom for you (using
Satterthwaite method by default, or the more computationally intensive
Kenward-Roger method), so you'll get p-values if that's what you want.

Finally, it's important to note two things:

1. The "type"-argument for nlme::summary doesn't actually do anything
(see ?nlme::summary.lme). It's just passed onto the 'print' method,
where it's silently ignored. The 'type' of sum of squares is an
ANOVA-thing; the closest correspondence in terms of model coefficients
is the coding of your categorical contrasts. See the literature
mentioned above for more details as well as Dale Barr's discussion on
simple vs. main effects in regression models
(http://talklab.psy.gla.ac.uk/tvw/catpred/).

(?nlme::anova.lme does have indeed have a 'type' argument.)

2. It is possible for the sequential tests and the marginal tests to
yield the same results. Again, see the above literature. You have no
interactions in your model and continuous (i.e. not-categorical)
predictors, so if they're orthogonal, then the sequential and marginal
tests will be numerically the same, even if they test different
hypotheses. (See section 5.2, starting on page 14; the sequential tests
are the "eliminating" tests, while the marginal tests are the "ignoring"
tests in that explanation.)

Best,
Phillip


On 28/11/17 12:00, [hidden email] wrote:
> I wantto run sequential ANOVAs (i.e. type I sums of squares), and trying to getresults including ANOVA tables and associated coefficients for predictive variables(I am using the R 3.4.2 version). I think ANOVA tables look right, but believecoefficients are wrong. Specifically, it looks like that the coefficients arefrom ANOVA with ?marginal? (type III sums of squares). I have tried both lme (nlmepackage) and lmer (lme4 + lmerTEST packages). Examples of the results arebelow:
>


<snip>


> Ibelieve the results from summary() are for ?marginal? instead of ?sequential?ANOVA because the p-value (i.e., 0.237 for narea) in summary are identical tothose in tables from ?marginal?. I also used lmer in the lme4 pacakge to findthe same results (summary() results look like from ?marginal?).
>
>
> Cananybody tell me how to get coefficients for ?sequential? ANOVAs? Thank you.
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: How to extract coefficients from sequential (type 1), ANOVAs using lmer and lme

Phillip Alday-2
Please reread my point #1: the tests of the (individual) coefficients in
the model summary are not the same as the ANOVA tests. There is a
certain correspondence between the two (i.e. between the coding of your
categorical variables and the type of sum of squares; and for a model
with a single predictor, F=t^2), but they are not the same in general.
The t-test in the model coefficients is simply the ratio of the estimate
to the standard error (i.e. a Wald test), and the standard errors, like
the estimates, are all calculated at the same time. So in that sense,
the t-tests are always marginal (cf. Pinheiro and Bates 2000, pp. 90-91).

I would also encourage you to focus more on your estimates (the Value
column) and less on p-values.

All that said, it seems you only care about the significance of model
terms and not the estimates, so there are two possibilities for
sequential tests:

1. Nested model comparison (likelihood-ratio test) with either the
anova() function or drop1(model,test='Chisq')

2. use the p-values from the ANOVA results

(see Pinheiro and Bates 2000, pp. 90-91, for some notes on which test is
preferred as well as the GLMM FAQ:
https://bbolker.github.io/mixedmodels-misc/glmmFAQ.html)

But please please note that it would be misleading to say that these are
the p-values for the coefficients in your model. These issues are the
same for both mixed and 'normal' regression models.

Phillip

On 30/11/17 16:56, Akihiro Koyama wrote:

> Hi Phillip,
>
> Thank you very much for informative comments. But I still cannot find a
> way to extract coefficients from sequential ANOVAs.
>
> I have many data sets which all give different p-values for "sequential"
> and "marginal" options in anova(). But summary() command looks only
> provide me coefficients associated with "marginal" ANOVAs.
>
>
>
>
> On Tuesday, November 28, 2017 6:51 AM, Phillip Alday
> <[hidden email]> wrote:
>
>
> Be careful when mixing lme4 and lmerTest together -- lmerTest extends
> and changes the behavior of various lme4 functions.
>
> From the help page for lme4-anova (?lme4::anova.merMod)
>
>>      ‘anova’: returns the sequential decomposition of the contributions
>>          of fixed-effects terms or, for multiple arguments, model
>>          comparison statistics.  For objects of class ‘lmerMod’ the
>>          default behavior is to refit the models with ML if fitted
>>          with ‘REML = TRUE’, this can be controlled via the ‘refit’
>>          argument. See also ‘anova’.
>
> So lme4-anova will give you sequential tests; note, however, that lme4
> won't calculate the denominator degrees of freedom for you and thus
> won't give p-values. See the FAQ
> (https://cran.r-project.org/doc/FAQ/R-FAQ.html#Why-are-p_002dvalues-not-displayed-when-using-lmer_0028_0029_003f)
>
> From the help page for lmerTest-anova (?lmerTest::anova.merModLmerTest):
>> Usage:
>>
>>      ## S4 method for signature 'merModLmerTest'
>>      anova(object, ... , ddf="Satterthwaite",
>>      type=3)
>>    
>> Arguments:
>>
> ...
>>    type: type of hypothesis to be tested. Could be type=3 or type=2 or
>>          type = 1 (The definition comes from SAS theory)
>
>
> So lmerTest-anova by default gives you Type III ('marginal', although
> Type II is what actually gives you tests that respect the Principle of
> Marginality; see John Fox's Applied Regression Analysis (book) or
> Venables' "Exegeses on Linear Models"
> (https://www.stats.ox.ac.uk/pub/MASS3/Exegeses.pdf) for more information
> on that. Type I tests are the sequential tests, so with anova(model,
> type=1), you will get the sequential tests you want. lmerTest will
> approximate the denominator degrees of freedom for you (using
> Satterthwaite method by default, or the more computationally intensive
> Kenward-Roger method), so you'll get p-values if that's what you want.
>
> Finally, it's important to note two things:
>
> 1. The "type"-argument for nlme::summary doesn't actually do anything
> (see ?nlme::summary.lme). It's just passed onto the 'print' method,
> where it's silently ignored. The 'type' of sum of squares is an
> ANOVA-thing; the closest correspondence in terms of model coefficients
> is the coding of your categorical contrasts. See the literature
> mentioned above for more details as well as Dale Barr's discussion on
> simple vs. main effects in regression models
> (http://talklab.psy.gla.ac.uk/tvw/catpred/).
>
> (?nlme::anova.lme does have indeed have a 'type' argument.)
>
> 2. It is possible for the sequential tests and the marginal tests to
> yield the same results. Again, see the above literature. You have no
> interactions in your model and continuous (i.e. not-categorical)
> predictors, so if they're orthogonal, then the sequential and marginal
> tests will be numerically the same, even if they test different
> hypotheses. (See section 5.2, starting on page 14; the sequential tests
> are the "eliminating" tests, while the marginal tests are the "ignoring"
> tests in that explanation.)
>
> Best,
> Phillip
>
>
> On 28/11/17 12:00, [hidden email]
> <mailto:[hidden email]> wrote:
>> I wantto run sequential ANOVAs (i.e. type I sums of squares), and
> trying to getresults including ANOVA tables and associated coefficients
> for predictive variables(I am using the R 3.4.2 version). I think ANOVA
> tables look right, but believecoefficients are wrong. Specifically, it
> looks like that the coefficients arefrom ANOVA with ?marginal? (type III
> sums of squares). I have tried both lme (nlmepackage) and lmer (lme4 +
> lmerTEST packages). Examples of the results arebelow:
>>
>
>
> <snip>
>
>
>> Ibelieve the results from summary() are for ?marginal? instead of
> ?sequential?ANOVA because the p-value (i.e., 0.237 for narea) in summary
> are identical tothose in tables from ?marginal?. I also used lmer in the
> lme4 pacakge to findthe same results (summary() results look like from
> ?marginal?).
>>
>>
>> Cananybody tell me how to get coefficients for ?sequential? ANOVAs?
> Thank you.
>>
>
>
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.