# Why missing values are not allowed in 'poly'?

7 messages
Open this post in threaded view
|

## Why missing values are not allowed in 'poly'?

 Dear all, I'm a bit surprised by this behavior in poly: x <- c(NA, 1:10) poly(x, degree = 2, raw=TRUE) ## Error in poly(x, degree = 2, raw = TRUE) : ##   missing values are not allowed in 'poly' x^2 ## [1] NA 1 4 9 16 25 36 49 64 81 100 As you can see, poly() will fail if the vector contains NAs, whereas it is perfectly possible to obtain the square of the vector manually. Is there a reason for this limitation in poly? Regards, Liviu -- Do you think you know what math is? http://www.ideasroadshow.com/issues/ian-stewart-2013-08-02Or what it means to be intelligent? http://www.ideasroadshow.com/issues/john-duncan-2013-08-30Think again: http://www.ideasroadshow.com/library______________________________________________ [hidden email] mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.
Open this post in threaded view
|

## Re: Why missing values are not allowed in 'poly'?

 I think the worst aspect of this restriction in poly() is that when you use poly in the formula of a model-fitting function you cannot have any missing values in the data, even if you supply na.action=na.exclude.   > d <- transform(data.frame(y=c(-1,1:10)), x=log(y))   Warning message:   In log(y) : NaNs produced   > fit <- lm(y ~ poly(x, 3), data=d, na.action=na.exclude)   Error in poly(x, 3) : missing values are not allowed in 'poly' Thus people are pushed to using a less stable formulation like   > fit <- lm(y ~ x + I(x^2) + I(x^3), data=d, na.action=na.exclude) Bill Dunlap TIBCO Software wdunlap tibco.com On Wed, Mar 23, 2016 at 12:59 PM, Liviu Andronic <[hidden email]> wrote: > Dear all, > I'm a bit surprised by this behavior in poly: > > x <- c(NA, 1:10) > poly(x, degree = 2, raw=TRUE) > ## Error in poly(x, degree = 2, raw = TRUE) : > ##   missing values are not allowed in 'poly' > x^2 > ## [1] NA 1 4 9 16 25 36 49 64 81 100 > > As you can see, poly() will fail if the vector contains NAs, whereas > it is perfectly possible to obtain the square of the vector manually. > > Is there a reason for this limitation in poly? > > Regards, > Liviu > > > -- > Do you think you know what math is? > http://www.ideasroadshow.com/issues/ian-stewart-2013-08-02> Or what it means to be intelligent? > http://www.ideasroadshow.com/issues/john-duncan-2013-08-30> Think again: > http://www.ideasroadshow.com/library> > ______________________________________________ > [hidden email] mailing list -- To UNSUBSCRIBE and more, see > https://stat.ethz.ch/mailman/listinfo/r-help> PLEASE do read the posting guide > http://www.R-project.org/posting-guide.html> and provide commented, minimal, self-contained, reproducible code. >         [[alternative HTML version deleted]] ______________________________________________ [hidden email] mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.
Open this post in threaded view
|

## Re: Why missing values are not allowed in 'poly'?

 On Wed, Mar 23, 2016 at 9:29 PM, William Dunlap <[hidden email]> wrote: > I think the worst aspect of this restriction in poly() is that when > you use poly in the formula of a model-fitting function you cannot > have any missing values in the data, even if you supply > na.action=na.exclude. > >   > d <- transform(data.frame(y=c(-1,1:10)), x=log(y)) >   Warning message: >   In log(y) : NaNs produced >   > fit <- lm(y ~ poly(x, 3), data=d, na.action=na.exclude) >   Error in poly(x, 3) : missing values are not allowed in 'poly' > > Thus people are pushed to using a less stable formulation like >   > fit <- lm(y ~ x + I(x^2) + I(x^3), data=d, na.action=na.exclude) > My difficulty precisely. What's more, I inspected the code for `poly` and at least for the simple case of raw=TRUE it seems trivial to support NAs. It suffices to change line 15 of the function: if (anyNA(x)) stop("missing values are not allowed in 'poly'") to: if (!raw && anyNA(x)) stop("missing values are not allowed in 'poly'") This way for raw polynomials estimation continues unimpeded. With the change above, I get this: > poly(x, degree = 2, raw=TRUE)        1 2 [1,] NA NA [2,] 1 1 [3,] 2 4 [4,] 3 9 [5,] 4 16 [6,] 5 25 [7,] 6 36 [8,] 7 49 [9,] 8 64 [10,] 9 81 [11,] 10 100 attr(,"degree") [1] 1 2 attr(,"class") [1] "poly" "matrix" Regards, Liviu > > Bill Dunlap > TIBCO Software > wdunlap tibco.com > > On Wed, Mar 23, 2016 at 12:59 PM, Liviu Andronic <[hidden email]> > wrote: >> >> Dear all, >> I'm a bit surprised by this behavior in poly: >> >> x <- c(NA, 1:10) >> poly(x, degree = 2, raw=TRUE) >> ## Error in poly(x, degree = 2, raw = TRUE) : >> ##   missing values are not allowed in 'poly' >> x^2 >> ## [1] NA 1 4 9 16 25 36 49 64 81 100 >> >> As you can see, poly() will fail if the vector contains NAs, whereas >> it is perfectly possible to obtain the square of the vector manually. >> >> Is there a reason for this limitation in poly? >> >> Regards, >> Liviu >> >> >> -- >> Do you think you know what math is? >> http://www.ideasroadshow.com/issues/ian-stewart-2013-08-02>> Or what it means to be intelligent? >> http://www.ideasroadshow.com/issues/john-duncan-2013-08-30>> Think again: >> http://www.ideasroadshow.com/library>> >> ______________________________________________ >> [hidden email] mailing list -- To UNSUBSCRIBE and more, see >> https://stat.ethz.ch/mailman/listinfo/r-help>> PLEASE do read the posting guide >> http://www.R-project.org/posting-guide.html>> and provide commented, minimal, self-contained, reproducible code. > > -- Do you think you know what math is? http://www.ideasroadshow.com/issues/ian-stewart-2013-08-02Or what it means to be intelligent? http://www.ideasroadshow.com/issues/john-duncan-2013-08-30Think again: http://www.ideasroadshow.com/library______________________________________________ [hidden email] mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.
Open this post in threaded view
|

## Re: Why missing values are not allowed in 'poly'?

 I don't know what is in R's poly(), but if it is like S+'s or TERR's then one could do             if (anyNA(x))  {                 nax <- na.exclude(x)                 px <- poly(x = nax, degree = degree, coefs = coefs, raw = raw, simple = simple)                 px <- structure(naresid(attr(nax, "na.action"), px), coefs = attr(px, "coefs"), degree = attr(px, "degree"), class = attr(px, "class"))                 return(px)             } and get nice results in the usual raw=FALSE case as well.  Similar stuff could be done in the multivariate cases. Bill Dunlap TIBCO Software wdunlap tibco.com On Wed, Mar 23, 2016 at 1:41 PM, Liviu Andronic <[hidden email]> wrote: > On Wed, Mar 23, 2016 at 9:29 PM, William Dunlap <[hidden email]> wrote: > > I think the worst aspect of this restriction in poly() is that when > > you use poly in the formula of a model-fitting function you cannot > > have any missing values in the data, even if you supply > > na.action=na.exclude. > > > >   > d <- transform(data.frame(y=c(-1,1:10)), x=log(y)) > >   Warning message: > >   In log(y) : NaNs produced > >   > fit <- lm(y ~ poly(x, 3), data=d, na.action=na.exclude) > >   Error in poly(x, 3) : missing values are not allowed in 'poly' > > > > Thus people are pushed to using a less stable formulation like > >   > fit <- lm(y ~ x + I(x^2) + I(x^3), data=d, na.action=na.exclude) > > > My difficulty precisely. What's more, I inspected the code for `poly` > and at least for the simple case of raw=TRUE it seems trivial to > support NAs. It suffices to change line 15 of the function: > if (anyNA(x)) stop("missing values are not allowed in 'poly'") > > to: > if (!raw && anyNA(x)) stop("missing values are not allowed in 'poly'") > > This way for raw polynomials estimation continues unimpeded. With the > change above, I get this: > > poly(x, degree = 2, raw=TRUE) >        1 2 > [1,] NA NA > [2,] 1 1 > [3,] 2 4 > [4,] 3 9 > [5,] 4 16 > [6,] 5 25 > [7,] 6 36 > [8,] 7 49 > [9,] 8 64 > [10,] 9 81 > [11,] 10 100 > attr(,"degree") > [1] 1 2 > attr(,"class") > [1] "poly" "matrix" > > > Regards, > Liviu > > > > > > Bill Dunlap > > TIBCO Software > > wdunlap tibco.com > > > > On Wed, Mar 23, 2016 at 12:59 PM, Liviu Andronic <[hidden email] > > > > wrote: > >> > >> Dear all, > >> I'm a bit surprised by this behavior in poly: > >> > >> x <- c(NA, 1:10) > >> poly(x, degree = 2, raw=TRUE) > >> ## Error in poly(x, degree = 2, raw = TRUE) : > >> ##   missing values are not allowed in 'poly' > >> x^2 > >> ## [1] NA 1 4 9 16 25 36 49 64 81 100 > >> > >> As you can see, poly() will fail if the vector contains NAs, whereas > >> it is perfectly possible to obtain the square of the vector manually. > >> > >> Is there a reason for this limitation in poly? > >> > >> Regards, > >> Liviu > >> > >> > >> -- > >> Do you think you know what math is? > >> http://www.ideasroadshow.com/issues/ian-stewart-2013-08-02> >> Or what it means to be intelligent? > >> http://www.ideasroadshow.com/issues/john-duncan-2013-08-30> >> Think again: > >> http://www.ideasroadshow.com/library> >> > >> ______________________________________________ > >> [hidden email] mailing list -- To UNSUBSCRIBE and more, see > >> https://stat.ethz.ch/mailman/listinfo/r-help> >> PLEASE do read the posting guide > >> http://www.R-project.org/posting-guide.html> >> and provide commented, minimal, self-contained, reproducible code. > > > > > > > > -- > Do you think you know what math is? > http://www.ideasroadshow.com/issues/ian-stewart-2013-08-02> Or what it means to be intelligent? > http://www.ideasroadshow.com/issues/john-duncan-2013-08-30> Think again: > http://www.ideasroadshow.com/library>         [[alternative HTML version deleted]] ______________________________________________ [hidden email] mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-helpPLEASE do read the posting guide http://www.R-project.org/posting-guide.htmland provide commented, minimal, self-contained, reproducible code.
Open this post in threaded view
|