Power Analysis

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Power Analysis

Schatzi
I am trying to do a power analysis to get the number of replicas per treatment.

If I try to get the power it works just fine:
setn=c(2,3)
sdx=c(1.19,4.35)
power.t.test(n = setn, delta = 13.5, sd = sdx, sig.level = 0.05,power = NULL)

If I go the other way to obtain the "n" I have problems.
sdx=c(1.19,4.35)
power.t.test(n = NULL, delta = 13.5, sd = sdx, sig.level = 0.05, power = 0.8)

Is there any way to do this? Thank you.
In theory, practice and theory are the same. In practice, they are not - Albert Einstein
Reply | Threaded
Open this post in threaded view
|

Re: Power Analysis

jones-2
First, note that you are doing two separate power calculations,
one with n=2 and sd = 1.19, the other with n=3 and sd = 4.35.
I will assume this was on purpose.  Now...

> power.t.test(n = 2, delta = 13.5, sd = 1.19, sig.level = 0.05)

     Two-sample t test power calculation

              n = 2
          delta = 13.5
             sd = 1.19
      sig.level = 0.05
          power = 0.9982097
    alternative = two.sided

Now, with n=2, the power is already .99.  With n=1, there are zero df.
So, what n corresponds to a power of .8?  

> power.t.test(n = 1.6305, delta = 13.5, sd = 1.19, sig.level = 0.05)

     Two-sample t test power calculation

              n = 1.6305
          delta = 13.5
             sd = 1.19
      sig.level = 0.05
          power = 0.8003734
    alternative = two.sided

It looks like 1.63 subjects will do the job :-)

Finally, look at the power.t.test function, there is a line that explains
your error message:

 else if (is.null(n))
        n <- uniroot(function(n) eval(p.body) - power, c(2, 1e+07))$root

power.t.test() is making the sensible assumption that we only care about
sample sizes of at least n = 2....

albyn

On Mon, Apr 18, 2011 at 02:31:19PM -0700, Schatzi wrote:

> I am trying to do a power analysis to get the number of replicas per
> treatment.
>
> If I try to get the power it works just fine:
> setn=c(2,3)
> sdx=c(1.19,4.35)
> power.t.test(n = setn, delta = 13.5, sd = sdx, sig.level = 0.05,power =
> NULL)
>
> If I go the other way to obtain the "n" I have problems.
> sdx=c(1.19,4.35)
> pow=c(.8,.8)
> power.t.test(n = NULL, delta = 13.5, sd = sdx, sig.level = 0.05, power =
> 0.8)
>
> Is there any way to do this? Thank you.
>
> --
> View this message in context: http://r.789695.n4.nabble.com/Power-Analysis-tp3458786p3458786.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

--
Albyn Jones
Reed College
[hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Power Analysis

David Cross-2
It seems to me, with deltas this large (relative to the SD), that a significance test is a moot point!

David Cross
[hidden email]
www.davidcross.us




On Apr 18, 2011, at 5:14 PM, Albyn Jones wrote:

> First, note that you are doing two separate power calculations,
> one with n=2 and sd = 1.19, the other with n=3 and sd = 4.35.
> I will assume this was on purpose.  Now...
>
>> power.t.test(n = 2, delta = 13.5, sd = 1.19, sig.level = 0.05)
>
>     Two-sample t test power calculation
>
>              n = 2
>          delta = 13.5
>             sd = 1.19
>      sig.level = 0.05
>          power = 0.9982097
>    alternative = two.sided
>
> Now, with n=2, the power is already .99.  With n=1, there are zero df.
> So, what n corresponds to a power of .8?  
>
>> power.t.test(n = 1.6305, delta = 13.5, sd = 1.19, sig.level = 0.05)
>
>     Two-sample t test power calculation
>
>              n = 1.6305
>          delta = 13.5
>             sd = 1.19
>      sig.level = 0.05
>          power = 0.8003734
>    alternative = two.sided
>
> It looks like 1.63 subjects will do the job :-)
>
> Finally, look at the power.t.test function, there is a line that explains
> your error message:
>
> else if (is.null(n))
>        n <- uniroot(function(n) eval(p.body) - power, c(2, 1e+07))$root
>
> power.t.test() is making the sensible assumption that we only care about
> sample sizes of at least n = 2....
>
> albyn
>
> On Mon, Apr 18, 2011 at 02:31:19PM -0700, Schatzi wrote:
>> I am trying to do a power analysis to get the number of replicas per
>> treatment.
>>
>> If I try to get the power it works just fine:
>> setn=c(2,3)
>> sdx=c(1.19,4.35)
>> power.t.test(n = setn, delta = 13.5, sd = sdx, sig.level = 0.05,power =
>> NULL)
>>
>> If I go the other way to obtain the "n" I have problems.
>> sdx=c(1.19,4.35)
>> pow=c(.8,.8)
>> power.t.test(n = NULL, delta = 13.5, sd = sdx, sig.level = 0.05, power =
>> 0.8)
>>
>> Is there any way to do this? Thank you.
>>
>> --
>> View this message in context: http://r.789695.n4.nabble.com/Power-Analysis-tp3458786p3458786.html
>> Sent from the R help mailing list archive at Nabble.com.
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> --
> Albyn Jones
> Reed College
> [hidden email]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Power Analysis

jones-2
Yes, Richard Savage used to call this "inter ocular data";
the answer should leap up and strike you right between the eyes...

albyn

On Mon, Apr 18, 2011 at 05:23:05PM -0500, David Cross wrote:

> It seems to me, with deltas this large (relative to the SD), that a significance test is a moot point!
>
> David Cross
> [hidden email]
> www.davidcross.us
>
>
>
>
> On Apr 18, 2011, at 5:14 PM, Albyn Jones wrote:
>
> > First, note that you are doing two separate power calculations,
> > one with n=2 and sd = 1.19, the other with n=3 and sd = 4.35.
> > I will assume this was on purpose.  Now...
> >
> >> power.t.test(n = 2, delta = 13.5, sd = 1.19, sig.level = 0.05)
> >
> >     Two-sample t test power calculation
> >
> >              n = 2
> >          delta = 13.5
> >             sd = 1.19
> >      sig.level = 0.05
> >          power = 0.9982097
> >    alternative = two.sided
> >
> > Now, with n=2, the power is already .99.  With n=1, there are zero df.
> > So, what n corresponds to a power of .8?  
> >
> >> power.t.test(n = 1.6305, delta = 13.5, sd = 1.19, sig.level = 0.05)
> >
> >     Two-sample t test power calculation
> >
> >              n = 1.6305
> >          delta = 13.5
> >             sd = 1.19
> >      sig.level = 0.05
> >          power = 0.8003734
> >    alternative = two.sided
> >
> > It looks like 1.63 subjects will do the job :-)
> >
> > Finally, look at the power.t.test function, there is a line that explains
> > your error message:
> >
> > else if (is.null(n))
> >        n <- uniroot(function(n) eval(p.body) - power, c(2, 1e+07))$root
> >
> > power.t.test() is making the sensible assumption that we only care about
> > sample sizes of at least n = 2....
> >
> > albyn
> >
> > On Mon, Apr 18, 2011 at 02:31:19PM -0700, Schatzi wrote:
> >> I am trying to do a power analysis to get the number of replicas per
> >> treatment.
> >>
> >> If I try to get the power it works just fine:
> >> setn=c(2,3)
> >> sdx=c(1.19,4.35)
> >> power.t.test(n = setn, delta = 13.5, sd = sdx, sig.level = 0.05,power =
> >> NULL)
> >>
> >> If I go the other way to obtain the "n" I have problems.
> >> sdx=c(1.19,4.35)
> >> pow=c(.8,.8)
> >> power.t.test(n = NULL, delta = 13.5, sd = sdx, sig.level = 0.05, power =
> >> 0.8)
> >>
> >> Is there any way to do this? Thank you.
> >>
> >> --
> >> View this message in context: http://r.789695.n4.nabble.com/Power-Analysis-tp3458786p3458786.html
> >> Sent from the R help mailing list archive at Nabble.com.
> >>
> >> ______________________________________________
> >> [hidden email] mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-help
> >> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> >> and provide commented, minimal, self-contained, reproducible code.
> >>
> >
> > --
> > Albyn Jones
> > Reed College
> > [hidden email]
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
>
>

--
Albyn Jones
Reed College
[hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Power Analysis

Schatzi
"Inter ocular data"
Quite amusing :)
Thank you for the help. For some reason I was thinking that I could get the n values for the combined test, but that doesn't make sense as there could be an infinite number of combinations of n values.
Thanks again for the replies.
In theory, practice and theory are the same. In practice, they are not - Albert Einstein
Reply | Threaded
Open this post in threaded view
|

Re: Power Analysis

Marc Schwartz-3
On Apr 19, 2011, at 8:43 AM, Schatzi wrote:

> "Inter ocular data"
> Quite amusing :)
> Thank you for the help. For some reason I was thinking that I could get the
> n values for the combined test, but that doesn't make sense as there could
> be an infinite number of combinations of n values.
> Thanks again for the replies.


Just a quick follow up comment, specifically pertaining to the description of function arguments in R's help files. If a function argument can take more than one value, the argument description in the help file will typically (albeit, not absolutely always) include the word 'vector'. For example:

 x    a numeric vector ...

and frequently, the examples in the help files would include such use. The Details section of the help file will also typically elaborate on this, where appropriate, perhaps based upon context.

What you were observing, is an unintended consequence of the vector cycling of your 'n' and 'sd' arguments internally in the function, enabling the result to include two values in the output, where in fact, only one is intended. This is, as was noted by Albyn, the result of two independent power analyses.

Also, there was a post some time ago with a proposal from Claus Ekstrøm, describing a modification of power.t.test() to handle different sample sizes and variances:

  http://tolstoy.newcastle.edu.au/R/devel/03b/0492.html

which might be worth a look. I don't see a reply and needless to say, the modification has not been made to the core R function.

HTH,

Marc Schwartz

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.