How to compute the power of a wilcoxon test?

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

How to compute the power of a wilcoxon test?

tn85
Hello All,

I posted a similar question before, but the direction was driven to whether my case is suitable for a wilcoxon test. After research about the appropriateness, I am pretty sure that a wilcoxon test is the right tool for my case. But how to compute the power of the test is still an unanswered question bothering me.

The basic stats of my two paired samples are:
mean1 = 0.0032,   sd1 = 0.012
mean2 = 0.00042, sd2 = 0.0016
sample size = 366

Could anyone help? Thank you in advance.
Reply | Threaded
Open this post in threaded view
|

Re: How to compute the power of a wilcoxon test?

Peter Dalgaard-2

On Sep 15, 2011, at 15:47 , tn85 wrote:

> Hello All,
>
> I posted a similar question before, but the direction was driven to whether
> my case is suitable for a wilcoxon test. After research about the
> appropriateness, I am pretty sure that a wilcoxon test is the right tool for
> my case. But how to compute the power of the test is still an unanswered
> question bothering me.
>
> The basic stats of my two paired samples are:
> mean1 = 0.0032,   sd1 = 0.012
> mean2 = 0.00042, sd2 = 0.0016
> sample size = 366
>
> Could anyone help? Thank you in advance.

Thing is, you can't really do that. Nonparametic tests are only distribution-free under the null hypothesis. For power, you need the distribution of the test statistic under the alternative.

First: Do you really mean "paired"? If so, there would seem to be some information missing, namely the magnitude of the correlation. Also, do you really mean that the sd's differ by a factor of 10?? (And, if those are empirical quantities, don't do power calculations at all).

Assuming that you have two independent samples of size 366, and parameters as above, you might do

> pval <- replicate(1000, wilcox.test(rnorm(366,.0032,.012), rnorm(366,.00042,.0016))$p.value)
> summary(pval)
     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
0.0000000 0.0000004 0.0000185 0.0064520 0.0003877 0.8997000
> sum(pval < .05)
[1] 979

suggesting a power of around 98%. Adjust as necessary.  


>
> --
> View this message in context: http://r.789695.n4.nabble.com/How-to-compute-the-power-of-a-wilcoxon-test-tp3815616p3815616.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

--
Peter Dalgaard, Professor
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

RE: How to compute the power of a wilcoxon test?

tn85

Thank you, Peter. They are empirical quantities, that’s why I felt confused as being requested to give the power of my test. Could you please clarify the reason why power calculations are not fit for empirical quantities? So that I can defend my test for such request.

 

BTW, the correlation of the two samples is -0.053.

 

Thanks,

Tianchan

 

From: Peter Dalgaard-2 [via R] [mailto:ml-node+[hidden email]]
Sent: Thursday, September 15, 2011 11:04 AM
To: Tianchan Niu
Subject: Re: How to compute the power of a wilcoxon test?

 


On Sep 15, 2011, at 15:47 , tn85 wrote:


> Hello All,
>
> I posted a similar question before, but the direction was driven to whether
> my case is suitable for a wilcoxon test. After research about the
> appropriateness, I am pretty sure that a wilcoxon test is the right tool for
> my case. But how to compute the power of the test is still an unanswered
> question bothering me.
>
> The basic stats of my two paired samples are:
> mean1 = 0.0032,   sd1 = 0.012
> mean2 = 0.00042, sd2 = 0.0016
> sample size = 366
>
> Could anyone help? Thank you in advance.


Thing is, you can't really do that. Nonparametic tests are only distribution-free under the null hypothesis. For power, you need the distribution of the test statistic under the alternative.

First: Do you really mean "paired"? If so, there would seem to be some information missing, namely the magnitude of the correlation. Also, do you really mean that the sd's differ by a factor of 10?? (And, if those are empirical quantities, don't do power calculations at all).

Assuming that you have two independent samples of size 366, and parameters as above, you might do

> pval <- replicate(1000, wilcox.test(rnorm(366,.0032,.012), rnorm(366,.00042,.0016))$p.value)
> summary(pval)
     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
0.0000000 0.0000004 0.0000185 0.0064520 0.0003877 0.8997000
> sum(pval < .05)
[1] 979

suggesting a power of around 98%. Adjust as necessary.  


>
> --
> View this message in context: http://r.789695.n4.nabble.com/How-to-compute-the-power-of-a-wilcoxon-test-tp3815616p3815616.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.


--
Peter Dalgaard, Professor
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.


If you reply to this email, your message will be added to the discussion below:

http://r.789695.n4.nabble.com/How-to-compute-the-power-of-a-wilcoxon-test-tp3815616p3815852.html

To unsubscribe from How to compute the power of a wilcoxon test?, click here.

Reply | Threaded
Open this post in threaded view
|

Re: How to compute the power of a wilcoxon test?

Peter Dalgaard-2

On Sep 15, 2011, at 17:15 , tn85 wrote:

> Thank you, Peter. They are empirical quantities, that's why I felt confused as being requested to give the power of my test. Could you please clarify the reason why power calculations are not fit for empirical quantities? So that I can defend my test for such request.

Well, in two words: "Too late". Power calculations belong before the data are collected, once you have them, p-values and confidence intervals provide all relevant information. You can Google for "post hoc power" and find discussions involving more than two words...

One exception could be when it is considered useful to know what size of effect the study was designed to find. If authors "forgot" to do so in advance, they might want to reconstruct the calculation later. This verges on scientific misconduct (of a relatively minor sort), but even in those cases, I'd say that the observed differences are irrelevant and you should do the calculation AS IF you were starting from scratch: decide on the minimum relevant difference, etc.

>
> BTW, the correlation of the two samples is -0.053.

So they really are paired? If so, the code I gave should probably have paired=TRUE in the test, but simulating as uncorrelated is defensible.

>
> Thanks,
> Tianchan
>
> From: Peter Dalgaard-2 [via R] [mailto:[hidden email]]
> Sent: Thursday, September 15, 2011 11:04 AM
> To: Tianchan Niu
> Subject: Re: How to compute the power of a wilcoxon test?
>
>
> On Sep 15, 2011, at 15:47 , tn85 wrote:
>
>> Hello All,
>>
>> I posted a similar question before, but the direction was driven to whether
>> my case is suitable for a wilcoxon test. After research about the
>> appropriateness, I am pretty sure that a wilcoxon test is the right tool for
>> my case. But how to compute the power of the test is still an unanswered
>> question bothering me.
>>
>> The basic stats of my two paired samples are:
>> mean1 = 0.0032,   sd1 = 0.012
>> mean2 = 0.00042, sd2 = 0.0016
>> sample size = 366
>>
>> Could anyone help? Thank you in advance.
>
> Thing is, you can't really do that. Nonparametic tests are only distribution-free under the null hypothesis. For power, you need the distribution of the test statistic under the alternative.
>
> First: Do you really mean "paired"? If so, there would seem to be some information missing, namely the magnitude of the correlation. Also, do you really mean that the sd's differ by a factor of 10?? (And, if those are empirical quantities, don't do power calculations at all).
>
> Assuming that you have two independent samples of size 366, and parameters as above, you might do
>
>> pval <- replicate(1000, wilcox.test(rnorm(366,.0032,.012), rnorm(366,.00042,.0016))$p.value)
>> summary(pval)
>     Min.   1st Qu.    Median      Mean   3rd Qu.      Max.
> 0.0000000 0.0000004 0.0000185 0.0064520 0.0003877 0.8997000
>> sum(pval < .05)
> [1] 979
>
> suggesting a power of around 98%. Adjust as necessary.
>
>>
>> --
>> View this message in context: http://r.789695.n4.nabble.com/How-to-compute-the-power-of-a-wilcoxon-test-tp3815616p3815616.html
>> Sent from the R help mailing list archive at Nabble.com.
>>
>> ______________________________________________
>> [hidden email]</user/SendEmail.jtp?type=node&node=3815852&i=0> mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> --
> Peter Dalgaard, Professor
> Center for Statistics, Copenhagen Business School
> Solbjerg Plads 3, 2000 Frederiksberg, Denmark
> Phone: (+45)38153501
> Email: [hidden email]</user/SendEmail.jtp?type=node&node=3815852&i=1>  Priv: [hidden email]</user/SendEmail.jtp?type=node&node=3815852&i=2>
>
> ______________________________________________
> [hidden email]</user/SendEmail.jtp?type=node&node=3815852&i=3> mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>
> ________________________________
> If you reply to this email, your message will be added to the discussion below:
> http://r.789695.n4.nabble.com/How-to-compute-the-power-of-a-wilcoxon-test-tp3815616p3815852.html
> To unsubscribe from How to compute the power of a wilcoxon test?, click here<
>
>
> --
> View this message in context:
http://r.789695.n4.nabble.com/How-to-compute-the-power-of-a-wilcoxon-test-tp3815616p3815893.html
> Sent from the R help mailing list archive at Nabble.com.
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: How to compute the power of a wilcoxon test?

biobee
This post has NOT been accepted by the mailing list yet.
In reply to this post by Peter Dalgaard-2
Hello Peter, Thanks for sharing your reply on how to compute power for wilcoxon test. I have a similar question but with different input. If I want to compute the power of a wilcoxon test for two groups with unequal sample size and I know only the effect size, and that the groups are drawn from normal distribution.How can I modify the code your code: pval <- replicate(1000, wilcox.test(rnorm(366,.0032,.012), rnorm(366,.00042,.0016))$p.value) Thanks, biobee