Quantcast

Normal distribution (Lillie.test())

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Normal distribution (Lillie.test())

Bosken
Hi all,

I have a dataset of 2000 numbers ( it's noise measured with a scoop )

Now i want to know of my data is normal distributed (Gaussian distribution).

I did already:

- 68-95-99.7 test
- Q-Q-plot

and now i used "nortest library" and the Lilli.test()

However i don't understad the output?

lillie.test(z)

        Lilliefors (Kolmogorov-Smirnov) normality test

data:  z
D = 0.0218, p-value = 0.0278


I read wiki, but still can understand it..

Can anyone, give an explanation of my output D and p-value?

Thanks in advance

Gr. Bosken


Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Normal distribution (Lillie.test())

Moshe Olshansky-2
Hi,

As far as I understand, D is the value of (Kolmogorov-Smirnov) statistic and p-value is the probability to get that (or greater) value for normally distributed variables (so in your case you would most probably reject the hypothesis that your data is normal).

--- On Tue, 23/2/10, Bosken <[hidden email]> wrote:

> From: Bosken <[hidden email]>
> Subject: [R] Normal distribution (Lillie.test())
> To: [hidden email]
> Received: Tuesday, 23 February, 2010, 7:22 AM
>
> Hi all,
>
> I have a dataset of 2000 numbers ( it's noise measured with
> a scoop )
>
> Now i want to know of my data is normal distributed
> (Gaussian distribution).
>
> I did already:
>
> - 68-95-99.7 test
> - Q-Q-plot
>
> and now i used "nortest library" and the Lilli.test()
>
> However i don't understad the output?
>
> lillie.test(z)
>
>     Lilliefors (Kolmogorov-Smirnov)
> normality test
>
> data:  z
> D = 0.0218, p-value = 0.0278
>
> I read wiki, but still can understand it..
>
> Can anyone, give an explanation of my output D and
> p-value?
>
> Thanks in advance
>
> Gr. Bosken
>
>
>
> --
> View this message in context: http://n4.nabble.com/Normal-distribution-Lillie-test-tp1565083p1565083.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email]
> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained,
> reproducible code.
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Normal distribution (Lillie.test())

Bosken
Hi,

Thanks for your reaction;

How do you come to the decision that my data not is normal distributed?

With the 69-95-99.7 test and Q-Q plot seems it ok! But these test are not exact, they only give you an image.
 
Gr. Bosken
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Normal distribution (Lillie.test())

Greg Snow-2
You should probably read fortune(117) and fortune(234) (and possibly some of the original discussions that lead to the fortunes).  Reading the help page for the SnowsPenultimateNormalityTest function (TeachingDemos package) may also help.  If you are happy with the plots, but still feel the need for a "test" of some sort, then you should investigate using the vis.test function in the TeachingDemos package.

Hope this helps,

--
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
[hidden email]
801.408.8111


> -----Original Message-----
> From: [hidden email] [mailto:r-help-bounces@r-
> project.org] On Behalf Of Bosken
> Sent: Tuesday, February 23, 2010 4:13 AM
> To: [hidden email]
> Subject: Re: [R] Normal distribution (Lillie.test())
>
>
> Hi,
>
> Thanks for your reaction;
>
> How do you come to the decision that my data not is normal distributed?
>
> With the 69-95-99.7 test and Q-Q plot seems it ok! But these test are
> not
> exact, they only give you an image.
>
> Gr. Bosken
> --
> View this message in context: http://n4.nabble.com/Normal-distribution-
> Lillie-test-tp1565083p1565762.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Normal distribution (Lillie.test())

Bosken
Hi,

Thanks for your reaction.

The purpose of my test is to check if my NoiseGenerators really are Normal Distributed en witch circuit is the best!

So I need some good test to do this.

But what with: Fortune(117) and fortune(234), can't find anything about it..

Thanks for the help!

Bosken
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Normal distribution (Lillie.test())

Bert Gunter
In reply to this post by Greg Snow-2
...

But,quoting Pogo, "We have met the enemy, and he is us."

Normality tests are standard fare in a host of statistical texts.

Bert Gunter
Genentech Nonclinical Biostatistics
 
 

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On
Behalf Of Greg Snow
Sent: Thursday, February 25, 2010 9:00 AM
To: Bosken; [hidden email]
Subject: Re: [R] Normal distribution (Lillie.test())

You should probably read fortune(117) and fortune(234) (and possibly some of
the original discussions that lead to the fortunes).  Reading the help page
for the SnowsPenultimateNormalityTest function (TeachingDemos package) may
also help.  If you are happy with the plots, but still feel the need for a
"test" of some sort, then you should investigate using the vis.test function
in the TeachingDemos package.

Hope this helps,

--
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
[hidden email]
801.408.8111


> -----Original Message-----
> From: [hidden email] [mailto:r-help-bounces@r-
> project.org] On Behalf Of Bosken
> Sent: Tuesday, February 23, 2010 4:13 AM
> To: [hidden email]
> Subject: Re: [R] Normal distribution (Lillie.test())
>
>
> Hi,
>
> Thanks for your reaction;
>
> How do you come to the decision that my data not is normal distributed?
>
> With the 69-95-99.7 test and Q-Q plot seems it ok! But these test are
> not
> exact, they only give you an image.
>
> Gr. Bosken
> --
> View this message in context: http://n4.nabble.com/Normal-distribution-
> Lillie-test-tp1565083p1565762.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Normal distribution (Lillie.test())

Greg Snow-2
The t-test is even more standard fare, but I am not familiar with any textbooks that suggest that you use it for categorical data.  I just want people to think about what their question really is and if the test they are tempted to use really answers the right question.

Recommendations change, but they change much slower if nobody pushes for the change.  When I took my first intro stat class the standard procedure for a comparison of 2 means was to first test for equality of variance (sometimes using an alpha=0.2), then do a pooled t-test or an approximate t-test based on the results.  Current intro text books have switched to just suggesting the use of the approximate test (with better approximations now that we have computers to do the ugly formulas) and not bother with a variance test, or maybe use a rule of thumb variance test.  In some cases the pooled test is not even mentioned.

I am not saying to never use a test of normality, but I think many cases of doing normality tests (especially with large sample sizes and on residuals from a regression) are like the drunk looking for his keys under the light post rather than near where he dropped them because the light is better.  Doing a normality test is easy, but often it is answering the wrong question.

My hope is that the textbooks someday will move to suggesting the methodology implemented in the vis.test function (TeachingDemos package, note that the credit for this idea should go much more to the authors listed in the references section than to the function author).  I think that test answers a much more useful question of close enough.  And even the process of getting a significant p-value from that test may give the user confidence to conclude close enough.

--
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
[hidden email]
801.408.8111


> -----Original Message-----
> From: Bert Gunter [mailto:[hidden email]]
> Sent: Thursday, February 25, 2010 10:24 AM
> To: Greg Snow; 'Bosken'; [hidden email]
> Subject: RE: [R] Normal distribution (Lillie.test())
>
> ...
>
> But,quoting Pogo, "We have met the enemy, and he is us."
>
> Normality tests are standard fare in a host of statistical texts.
>
> Bert Gunter
> Genentech Nonclinical Biostatistics
>
>
>
> -----Original Message-----
> From: [hidden email] [mailto:r-help-bounces@r-
> project.org] On
> Behalf Of Greg Snow
> Sent: Thursday, February 25, 2010 9:00 AM
> To: Bosken; [hidden email]
> Subject: Re: [R] Normal distribution (Lillie.test())
>
> You should probably read fortune(117) and fortune(234) (and possibly
> some of
> the original discussions that lead to the fortunes).  Reading the help
> page
> for the SnowsPenultimateNormalityTest function (TeachingDemos package)
> may
> also help.  If you are happy with the plots, but still feel the need
> for a
> "test" of some sort, then you should investigate using the vis.test
> function
> in the TeachingDemos package.
>
> Hope this helps,
>
> --
> Gregory (Greg) L. Snow Ph.D.
> Statistical Data Center
> Intermountain Healthcare
> [hidden email]
> 801.408.8111
>
>
> > -----Original Message-----
> > From: [hidden email] [mailto:r-help-bounces@r-
> > project.org] On Behalf Of Bosken
> > Sent: Tuesday, February 23, 2010 4:13 AM
> > To: [hidden email]
> > Subject: Re: [R] Normal distribution (Lillie.test())
> >
> >
> > Hi,
> >
> > Thanks for your reaction;
> >
> > How do you come to the decision that my data not is normal
> distributed?
> >
> > With the 69-95-99.7 test and Q-Q plot seems it ok! But these test are
> > not
> > exact, they only give you an image.
> >
> > Gr. Bosken
> > --
> > View this message in context: http://n4.nabble.com/Normal-
> distribution-
> > Lillie-test-tp1565083p1565762.html
> > Sent from the R help mailing list archive at Nabble.com.
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-
> > guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Normal distribution (Lillie.test())

David Winsemius
In reply to this post by Bosken

On Feb 25, 2010, at 12:20 PM, Bosken wrote:

>
> Hi,
>
> Thanks for your reaction.
>
> The purpose of my test is to check if my NoiseGenerators really are  
> Normal
> Distributed en witch circuit is the best!
>
> So I need some good test to do this.
>
> But what with: Fortune(), can't find anything about it..

# This might work, but if not, you should get the idea.

install.packages(pkgs="fortunes")
require(fortunes)
fortune(117)
fortune(234)


>
> Thanks for the help!
>
> Bosken
> --
> View this message in context: http://n4.nabble.com/Normal-distribution-Lillie-test-tp1565083p1569361.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius, MD
Heritage Laboratories
West Hartford, CT

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Normal distribution (Lillie.test())

Greg Snow-2
In reply to this post by Bosken
Install and load the fortunes package first, then run fortune(117), etc.  Then run fortune() quite a few times for possible enlightenment (or at least mild entertainment).

Do your NoiseGenerotors need to generate exactly normal data (they don't, see SnowsPenultimateNormalityTest), or is there a level of close enough?  If I remember correctly, you were testing 2000 values, with that sample size most normality tests will find very small differences to be significantly different, even if those small differences are practically meaningless.

--
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
[hidden email]
801.408.8111


> -----Original Message-----
> From: [hidden email] [mailto:r-help-bounces@r-
> project.org] On Behalf Of Bosken
> Sent: Thursday, February 25, 2010 10:21 AM
> To: [hidden email]
> Subject: Re: [R] Normal distribution (Lillie.test())
>
>
> Hi,
>
> Thanks for your reaction.
>
> The purpose of my test is to check if my NoiseGenerators really are
> Normal
> Distributed en witch circuit is the best!
>
> So I need some good test to do this.
>
> But what with: Fortune(117) and fortune(234), can't find anything about
> it..
>
> Thanks for the help!
>
> Bosken
> --
> View this message in context: http://n4.nabble.com/Normal-distribution-
> Lillie-test-tp1565083p1569361.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Normal distribution (Lillie.test())

Ravi Varadhan
In reply to this post by Bert Gunter
May be you should have said:

"Normality tests are standard farce in a host of statistical texts."

Ravi.

----------------------------------------------------------------------------
-------

Ravi Varadhan, Ph.D.

Assistant Professor, The Center on Aging and Health

Division of Geriatric Medicine and Gerontology

Johns Hopkins University

Ph: (410) 502-2619

Fax: (410) 614-9625

Email: [hidden email]

Webpage:
http://www.jhsph.edu/agingandhealth/People/Faculty_personal_pages/Varadhan.h
tml

 

----------------------------------------------------------------------------
--------


-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On
Behalf Of Bert Gunter
Sent: Thursday, February 25, 2010 12:24 PM
To: 'Greg Snow'; 'Bosken'; [hidden email]
Subject: Re: [R] Normal distribution (Lillie.test())

...

But,quoting Pogo, "We have met the enemy, and he is us."

Normality tests are standard fare in a host of statistical texts.

Bert Gunter
Genentech Nonclinical Biostatistics
 
 

-----Original Message-----
From: [hidden email] [mailto:[hidden email]] On
Behalf Of Greg Snow
Sent: Thursday, February 25, 2010 9:00 AM
To: Bosken; [hidden email]
Subject: Re: [R] Normal distribution (Lillie.test())

You should probably read fortune(117) and fortune(234) (and possibly some of
the original discussions that lead to the fortunes).  Reading the help page
for the SnowsPenultimateNormalityTest function (TeachingDemos package) may
also help.  If you are happy with the plots, but still feel the need for a
"test" of some sort, then you should investigate using the vis.test function
in the TeachingDemos package.

Hope this helps,

--
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
[hidden email]
801.408.8111


> -----Original Message-----
> From: [hidden email] [mailto:r-help-bounces@r-
> project.org] On Behalf Of Bosken
> Sent: Tuesday, February 23, 2010 4:13 AM
> To: [hidden email]
> Subject: Re: [R] Normal distribution (Lillie.test())
>
>
> Hi,
>
> Thanks for your reaction;
>
> How do you come to the decision that my data not is normal distributed?
>
> With the 69-95-99.7 test and Q-Q plot seems it ok! But these test are
> not
> exact, they only give you an image.
>
> Gr. Bosken
> --
> View this message in context: http://n4.nabble.com/Normal-distribution-
> Lillie-test-tp1565083p1565762.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-
> guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Normal distribution (Lillie.test())

Bosken
In reply to this post by Greg Snow-2
Hi Greg,

I'm making NoiseGenerators with different noise sources and components, the meaning of my tests with R is to know which NoiseGenerator approached most the Normal distribution function...

Thanks, for all the reactions.

Bosken

Greg Snow-2 wrote
Do your NoiseGenerotors need to generate exactly normal data (they don't, see SnowsPenultimateNormalityTest), or is there a level of close enough?  If I remember correctly, you were testing 2000 values, with that sample size most normality tests will find very small differences to be significantly different, even if those small differences are practically meaningless.

--
Gregory (Greg) L. Snow Ph.D.
Statistical Data Center
Intermountain Healthcare
greg.snow@imail.org
801.408.8111
Loading...