Quantcast

Bartlett's Test of Sphericity

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Bartlett's Test of Sphericity

thibault grava-3
Hello Dear R user,

I want to conduct a Principal components analysis and I need to run two
tests to check whether I can do it or not. I found how to run the KMO
test, however i cannot find an R fonction for the Bartlett's test of
sphericity. Does somebody know if it exists?

Thanks for your help!

Thibault

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bartlett's Test of Sphericity

Daniel Malter
The formula for the chi-square value is:

-( (n-1) - (2*p-5)/6 )* log(det(R))

where n is the number of observations, p is the number of variables, and R is the correlation matrix. The chi square test is then performed on (p^2-p)/2 degrees of freedom. So you can compute it by hand. Or you can use the function below (no warranty) where you supply the data as data frame to the function bartlett.sphere()

example:

x<-rnorm(100)
y<-x+rnorm(100,0,0.1)

bartlett.sphere<-function(data){chi.square=-( (dim(data)[1]-1) - (2*dim(data)[2]-5)/6 )* log(det(cor(data,use='pairwise.complete.obs')));cat('chi.square value ', chi.square , ' on ', (dim(data)[2]^2-dim(data)[2])/2, ' degrees of freedom.' , ' p-value: ', 1-pchisq(chi.square,(dim(data)[2]^2-dim(data)[2])/2))}

bartlett.sphere(data.frame(x,y))

HTH,
Daniel



thibault grava-3 wrote
Hello Dear R user,

I want to conduct a Principal components analysis and I need to run two
tests to check whether I can do it or not. I found how to run the KMO
test, however i cannot find an R fonction for the Bartlett's test of
sphericity. Does somebody know if it exists?

Thanks for your help!

Thibault

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bartlett's Test of Sphericity

Jeremy Miles-2
In reply to this post by thibault grava-3
cortest.bartlett() in the psych package.

I've never seen a non-significant Bartlett's test.

Jeremy



On 17 June 2011 12:43, thibault grava <[hidden email]> wrote:

> Hello Dear R user,
>
> I want to conduct a Principal components analysis and I need to run two
> tests to check whether I can do it or not. I found how to run the KMO
> test, however i cannot find an R fonction for the Bartlett's test of
> sphericity. Does somebody know if it exists?
>
> Thanks for your help!
>
> Thibault
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bartlett's Test of Sphericity

ted.harding-3
To add to Jeremy's comment below: The Bartlett test is very
sensitive to non-normality in the data, so can readily give
"significant" results even for non-correlated data.

Ted.

On 18-Jun-11 06:47:52, Jeremy Miles wrote:

> cortest.bartlett() in the psych package.
>
> I've never seen a non-significant Bartlett's test.
>
> Jeremy
>
>
>
> On 17 June 2011 12:43, thibault grava <[hidden email]> wrote:
>> Hello Dear R user,
>>
>> I want to conduct a Principal components analysis and I need to
>> run two tests to check whether I can do it or not. I found how
>> to run the KMO test, however i cannot find an R fonction for the
>> Bartlett's test of sphericity. Does somebody know if it exists?
>>
>> Thanks for your help!
>>
>> Thibault

--------------------------------------------------------------------
E-Mail: (Ted Harding) <[hidden email]>
Fax-to-email: +44 (0)870 094 0861
Date: 18-Jun-11                                       Time: 09:48:13
------------------------------ XFMail ------------------------------

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bartlett's Test of Sphericity

Peter Dalgaard-2

On Jun 18, 2011, at 10:48 , (Ted Harding) wrote:

> To add to Jeremy's comment below: The Bartlett test is very
> sensitive to non-normality in the data, so can readily give
> "significant" results even for non-correlated data.

Hmm, I wouldn't bet on that. Correlation tests are usually fairly robust.

More likely, it's that the null hypothesis of complete independence is rather extreme, especially in a context where you are contemplating PCA or FA. (I.e., "Of _course_ they are correlated, dummy!").

>
> Ted.
>
> On 18-Jun-11 06:47:52, Jeremy Miles wrote:
>> cortest.bartlett() in the psych package.
>>
>> I've never seen a non-significant Bartlett's test.
>>
>> Jeremy
>>
>>
>>
>> On 17 June 2011 12:43, thibault grava <[hidden email]> wrote:
>>> Hello Dear R user,
>>>
>>> I want to conduct a Principal components analysis and I need to
>>> run two tests to check whether I can do it or not. I found how
>>> to run the KMO test, however i cannot find an R fonction for the
>>> Bartlett's test of sphericity. Does somebody know if it exists?
>>>
>>> Thanks for your help!
>>>
>>> Thibault
>
> --------------------------------------------------------------------
> E-Mail: (Ted Harding) <[hidden email]>
> Fax-to-email: +44 (0)870 094 0861
> Date: 18-Jun-11                                       Time: 09:48:13
> ------------------------------ XFMail ------------------------------
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

--
Peter Dalgaard
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bartlett's Test of Sphericity

David Cross-2
In reply to this post by Jeremy Miles-2
Yes, Bartlett's is not a good way to "justify" a PCA.

David Cross
[hidden email]
www.davidcross.us




On Jun 18, 2011, at 1:47 AM, Jeremy Miles wrote:

> cortest.bartlett() in the psych package.
>
> I've never seen a non-significant Bartlett's test.
>
> Jeremy
>
>
>
> On 17 June 2011 12:43, thibault grava <[hidden email]> wrote:
>> Hello Dear R user,
>>
>> I want to conduct a Principal components analysis and I need to run two
>> tests to check whether I can do it or not. I found how to run the KMO
>> test, however i cannot find an R fonction for the Bartlett's test of
>> sphericity. Does somebody know if it exists?
>>
>> Thanks for your help!
>>
>> Thibault
>>
>>        [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bartlett's Test of Sphericity

Tal Galili
In reply to this post by thibault grava-3
Hi Thibault,
Not that I think you'll use this after the above responses, but for the
record, have a look at:
?bartlett.test


Cheers,
Tal


----------------Contact
Details:-------------------------------------------------------
Contact me: [hidden email] |  972-52-7275845
Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) |
www.r-statistics.com (English)
----------------------------------------------------------------------------------------------




On Fri, Jun 17, 2011 at 10:43 PM, thibault grava
<[hidden email]>wrote:

> Hello Dear R user,
>
> I want to conduct a Principal components analysis and I need to run two
> tests to check whether I can do it or not. I found how to run the KMO
> test, however i cannot find an R fonction for the Bartlett's test of
> sphericity. Does somebody know if it exists?
>
> Thanks for your help!
>
> Thibault
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bartlett's Test of Sphericity

Daniel Malter
In reply to this post by ted.harding-3
I do not understand why that would be the case as the only input involving the relationship of the data is the determinant of the correlation matrix. For what you suggest to be true, the non-normality of the data  would have to introduce correlation.

If what you are saying is true, we would expect the p.value in uncorrelated samples to be significant more or less often than 5 percent of the times (dependent on what exactly you mean by sensitivity) when we use a random sample of highly skewed gamma and Poisson distributions as data. At least for two data columns the example below does not suggest that this is the case.

Daniel


set.seed(4785)

cor.data<-NULL
chi.square<-NULL
p.value<-NULL

for(i in 1:1000){
y<-rgamma(100,1,2)
x<-rpois(100,2)

data=data.frame(x,y)

cor.data[i]=cor(data,use='pairwise.complete.obs')[1,2]
chi.square[i]=-( (dim(data)[1]-1) - (2*dim(data)[2]-5)/6 )* log(det(cor(data,use='pairwise.complete.obs')))
p.value[i] = 1-pchisq(chi.square[i],(dim(data)[2]^2-dim(data)[2])/2)
}

hist(p.value)
quantile(p.value,prob=0.95)

ted.harding-3 wrote
To add to Jeremy's comment below: The Bartlett test is very
sensitive to non-normality in the data, so can readily give
"significant" results even for non-correlated data.

Ted.

On 18-Jun-11 06:47:52, Jeremy Miles wrote:
> cortest.bartlett() in the psych package.
>
> I've never seen a non-significant Bartlett's test.
>
> Jeremy
>
>
>
> On 17 June 2011 12:43, thibault grava <[hidden email]> wrote:
>> Hello Dear R user,
>>
>> I want to conduct a Principal components analysis and I need to
>> run two tests to check whether I can do it or not. I found how
>> to run the KMO test, however i cannot find an R fonction for the
>> Bartlett's test of sphericity. Does somebody know if it exists?
>>
>> Thanks for your help!
>>
>> Thibault

--------------------------------------------------------------------
E-Mail: (Ted Harding) <[hidden email]>
Fax-to-email: +44 (0)870 094 0861
Date: 18-Jun-11                                       Time: 09:48:13
------------------------------ XFMail ------------------------------

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bartlett's Test of Sphericity

Peter Dalgaard-2
In reply to this post by Tal Galili

On Jun 18, 2011, at 15:48 , Tal Galili wrote:

> Hi Thibault,
> Not that I think you'll use this after the above responses, but for the
> record, have a look at:
> ?bartlett.test

That's the "other" Bartlett's test, namely the one for comparison of variances. That test is well known to rely on higher moments of the normal distribution.

--
Peter Dalgaard
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bartlett's Test of Sphericity

Tal Galili
Sorry for the confusion, thank you for the correction and explanation.


Tal

----------------Contact
Details:-------------------------------------------------------
Contact me: [hidden email] |  972-52-7275845
Read me: www.talgalili.com (Hebrew) | www.biostatistics.co.il (Hebrew) |
www.r-statistics.com (English)
----------------------------------------------------------------------------------------------




On Sat, Jun 18, 2011 at 7:38 PM, peter dalgaard <[hidden email]> wrote:

>
> On Jun 18, 2011, at 15:48 , Tal Galili wrote:
>
> > Hi Thibault,
> > Not that I think you'll use this after the above responses, but for the
> > record, have a look at:
> > ?bartlett.test
>
> That's the "other" Bartlett's test, namely the one for comparison of
> variances. That test is well known to rely on higher moments of the normal
> distribution.
>
> --
> Peter Dalgaard
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Email: [hidden email]  Priv: [hidden email]
>
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bartlett's Test of Sphericity

Daniel Malter
In reply to this post by Peter Dalgaard-2
Thanks for the clarification. That makes sense.

To summarize, bartlett.test() in the base distribution of R is not the sphericity test, and it relies on the higher moments of the normal distribution. The sphericity test can be computed with the formula provided or the one implemented in the psych package, and it does not rely on the higher moments of the normal distribution, but only on the correlation structure between the data.

Peter Dalgaard-2 wrote
On Jun 18, 2011, at 15:48 , Tal Galili wrote:

> Hi Thibault,
> Not that I think you'll use this after the above responses, but for the
> record, have a look at:
> ?bartlett.test

That's the "other" Bartlett's test, namely the one for comparison of variances. That test is well known to rely on higher moments of the normal distribution.

--
Peter Dalgaard
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Bartlett's Test of Sphericity

Fernando Henrique Ferraz Pereira da Rosa
In reply to this post by thibault grava-3
You could also check this function I implemented awhile back:

http://www.fernandohrosa.com.br/en/P/sphericity-test-for-covariance-matrices-in-r-sphericity-test/

On Fri, Jun 17, 2011 at 4:43 PM, thibault grava <[hidden email]>wrote:

> Hello Dear R user,
>
> I want to conduct a Principal components analysis and I need to run two
> tests to check whether I can do it or not. I found how to run the KMO
> test, however i cannot find an R fonction for the Bartlett's test of
> sphericity. Does somebody know if it exists?
>
> Thanks for your help!
>
> Thibault
>
>        [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>


--
"Though this be randomness, yet there is structure in't."
Fernando H Rosa - Statistician
http://www.fernandohrosa.com.br / http://www.feferraz.net - Estatística,
Matemática e Computação
BankReview.com.br <http://www.bankreview.com.br/> - Escolha melhor seus
serviços financeiros!
AprendaAlemao.net <http://aprendaalemao.net/> - Seu ponto de partida para
melhorar seu Alemão!
@fhrosa

        [[alternative HTML version deleted]]


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Loading...