Extracting p value from bootstrap in R

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Extracting p value from bootstrap in R

skate17
I've been trying to get the pvalue of my samples from a bootstrap rsquared test in R. I'm not very good with statistics so could someone please take a look at my below code and point me in the right direction with regards to how I can extract the p-values per sample (basically input a Descriptor_Score value and output a p-value based on the bootstrap model)? If my understanding is incorrect, please let me know. This code is mostly from API.

# Bootstrap 95% CI for R-Squared

library(boot)

# function to obtain R-Squared from the data

rsq <- function(formula, data, indices) {

d <- data[indices,] # allows boot to select sample

fit <- lm(formula, data=d)

return(summary(fit)$r.square)

}

###vdw beta bootstrap

`results[b]_vdw` = boot(data=rescored_beta, statistic=rsq,

R=10000, formula=Descriptor_Score~vdw)

# get 95% confidence interval

confb_vdw=boot.ci(`results[b]_vdw`)

cib_vdw = confb_vdw$bca[ , c(4, 5)]

Thanks!

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Extracting p value from bootstrap in R

Rui Barradas
Hello,

Your code is correctly extracting the bootstrapped r-squared confidence
intervals calculated using the adjusted bootstrap percentile (BCa) method.

If you want the p-values, do

summary(fit)$coefficients[, 4]

in the function.

Also, function boot calls a function statistic that expects data and
indices as 1st and 2nd arguments. Argument formula should go after these
ones. In your case that doesn't make any difference because you are
passing *named* arguments but it's better to keep to the rules, you'll
be avoiding potential/future problems. Rewrite the function as


rsq <- function(data, indices, formula) {
   d <- data[indices, ] # allows boot to select sample
   fit <- lm(formula, data = d)
   summary(fit)$r.square
}


Hope this helps,

Rui Barradas


Às 18:18 de 02/01/20, Saaim Khan escreveu:

> I've been trying to get the pvalue of my samples from a bootstrap rsquared test in R. I'm not very good with statistics so could someone please take a look at my below code and point me in the right direction with regards to how I can extract the p-values per sample (basically input a Descriptor_Score value and output a p-value based on the bootstrap model)? If my understanding is incorrect, please let me know. This code is mostly from API.
>
> # Bootstrap 95% CI for R-Squared
>
> library(boot)
>
> # function to obtain R-Squared from the data
>
> rsq <- function(formula, data, indices) {
>
> d <- data[indices,] # allows boot to select sample
>
> fit <- lm(formula, data=d)
>
> return(summary(fit)$r.square)
>
> }
>
> ###vdw beta bootstrap
>
> `results[b]_vdw` = boot(data=rescored_beta, statistic=rsq,
>
> R=10000, formula=Descriptor_Score~vdw)
>
> # get 95% confidence interval
>
> confb_vdw=boot.ci(`results[b]_vdw`)
>
> cib_vdw = confb_vdw$bca[ , c(4, 5)]
>
> Thanks!
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Extracting p value from bootstrap in R

Patrick (Malone Quantitative)
Also, note that the p-values will probably not exactly correspond to
conclusions drawn from the bootstrapped confidence limits. They are
two different approaches to the research question. Broadly speaking,
the bootstrapped limits avoid the assumption of a symmetric sampling
distribution that underlies the coefficient/standard error testing
strategy. Using the bootstrap gets you bootstrapped standard errors,
which have less stringent assumptions than normal-theory standard
errors, but the p-value will still reflect the symmetry assumption.
The BCa method allows asymmetric confidence intervals.


On Fri, Jan 3, 2020 at 2:52 PM Rui Barradas <[hidden email]> wrote:

>
> Hello,
>
> Your code is correctly extracting the bootstrapped r-squared confidence
> intervals calculated using the adjusted bootstrap percentile (BCa) method.
>
> If you want the p-values, do
>
> summary(fit)$coefficients[, 4]
>
> in the function.
>
> Also, function boot calls a function statistic that expects data and
> indices as 1st and 2nd arguments. Argument formula should go after these
> ones. In your case that doesn't make any difference because you are
> passing *named* arguments but it's better to keep to the rules, you'll
> be avoiding potential/future problems. Rewrite the function as
>
>
> rsq <- function(data, indices, formula) {
>    d <- data[indices, ] # allows boot to select sample
>    fit <- lm(formula, data = d)
>    summary(fit)$r.square
> }
>
>
> Hope this helps,
>
> Rui Barradas
>
>
> Às 18:18 de 02/01/20, Saaim Khan escreveu:
> > I've been trying to get the pvalue of my samples from a bootstrap rsquared test in R. I'm not very good with statistics so could someone please take a look at my below code and point me in the right direction with regards to how I can extract the p-values per sample (basically input a Descriptor_Score value and output a p-value based on the bootstrap model)? If my understanding is incorrect, please let me know. This code is mostly from API.
> >
> > # Bootstrap 95% CI for R-Squared
> >
> > library(boot)
> >
> > # function to obtain R-Squared from the data
> >
> > rsq <- function(formula, data, indices) {
> >
> > d <- data[indices,] # allows boot to select sample
> >
> > fit <- lm(formula, data=d)
> >
> > return(summary(fit)$r.square)
> >
> > }
> >
> > ###vdw beta bootstrap
> >
> > `results[b]_vdw` = boot(data=rescored_beta, statistic=rsq,
> >
> > R=10000, formula=Descriptor_Score~vdw)
> >
> > # get 95% confidence interval
> >
> > confb_vdw=boot.ci(`results[b]_vdw`)
> >
> > cib_vdw = confb_vdw$bca[ , c(4, 5)]
> >
> > Thanks!
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.



--
Patrick S. Malone, Ph.D., Malone Quantitative
NEW Service Models: https://malonequantitative.com

He/Him/His

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.