arima() bug

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

arima() bug

Ray.Brownrigg
I guess this is more r-devel than r-help.

Note, I am just the messenger - I have no idea what the user is trying to model here.

arima() crashes R (segfault) with Linux R-2.7.0, Solaris R-2.6.0:

 *** caught segfault ***
address 42400000, cause 'memory not mapped'

Traceback:
 1: .Call(R_getQ0, phi, theta)
 2: makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa)
 3: arima(x, c(1, 0, 1), c(1, 0, 1))

Under rw-2.7.0 or R version 2.8.0 Under development (unstable) (2008-06-10 r45893)
it gets:
Error: cannot allocate vector of size 1010.9 Mb
In addition: Warning messages:
1: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
  Reached total allocation of 447Mb: see help(memory.size)
2: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
  Reached total allocation of 447Mb: see help(memory.size)
3: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
  Reached total allocation of 447Mb: see help(memory.size)
4: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
  Reached total allocation of 447Mb: see help(memory.size)

Reproduce by:

# 2 years of daily temperature data
set.seed(1); x <- ts(20*sin((1:731)*2*pi/365) + 10 + rnorm(731, 0, 4), freq=365)
arima(x, c(1, 0, 1), c(1, 0, 1))

Ray Brownrigg

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: arima() bug

Antonio, Fabio Di Narzo
No segfault with my r-patched version on linux-i686:

> set.seed(1); x <- ts(20*sin((1:731)*2*pi/365) + 10 + rnorm(731, 0, 4), freq=365)
> arima(x, c(1, 0, 1), c(1, 0, 1))
Errore: cannot allocate vector of size 1010.9 Mb

F.

> R.version
               _
platform       i686-pc-linux-gnu
arch           i686
os             linux-gnu
system         i686, linux-gnu
status         Patched
major          2
minor          7.0
year           2008
month          05
day            29
svn rev        45820
language       R
version.string R version 2.7.0 Patched (2008-05-29 r45820)


2008/6/12 Ray Brownrigg <[hidden email]>:

> I guess this is more r-devel than r-help.
>
> Note, I am just the messenger - I have no idea what the user is trying to model here.
>
> arima() crashes R (segfault) with Linux R-2.7.0, Solaris R-2.6.0:
>
>  *** caught segfault ***
> address 42400000, cause 'memory not mapped'
>
> Traceback:
>  1: .Call(R_getQ0, phi, theta)
>  2: makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa)
>  3: arima(x, c(1, 0, 1), c(1, 0, 1))
>
> Under rw-2.7.0 or R version 2.8.0 Under development (unstable) (2008-06-10 r45893)
> it gets:
> Error: cannot allocate vector of size 1010.9 Mb
> In addition: Warning messages:
> 1: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>  Reached total allocation of 447Mb: see help(memory.size)
> 2: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>  Reached total allocation of 447Mb: see help(memory.size)
> 3: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>  Reached total allocation of 447Mb: see help(memory.size)
> 4: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>  Reached total allocation of 447Mb: see help(memory.size)
>
> Reproduce by:
>
> # 2 years of daily temperature data
> set.seed(1); x <- ts(20*sin((1:731)*2*pi/365) + 10 + rnorm(731, 0, 4), freq=365)
> arima(x, c(1, 0, 1), c(1, 0, 1))
>
> Ray Brownrigg
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>



--
Antonio, Fabio Di Narzo
Ph.D. student at
Department of Statistical Sciences
University of Bologna, Italy

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: arima() bug

Prof Brian Ripley
On Thu, 12 Jun 2008, Antonio, Fabio Di Narzo wrote:

> No segfault with my r-patched version on linux-i686:
>
>> set.seed(1); x <- ts(20*sin((1:731)*2*pi/365) + 10 + rnorm(731, 0, 4), freq=365)
>> arima(x, c(1, 0, 1), c(1, 0, 1))
> Errore: cannot allocate vector of size 1010.9 Mb

Yes, you need a lot of memory to reproduce it.  It's a nonsensical
calculation but nevertheless we need to track down the cause.

>
> F.
>
>> R.version
>               _
> platform       i686-pc-linux-gnu
> arch           i686
> os             linux-gnu
> system         i686, linux-gnu
> status         Patched
> major          2
> minor          7.0
> year           2008
> month          05
> day            29
> svn rev        45820
> language       R
> version.string R version 2.7.0 Patched (2008-05-29 r45820)
>
>
> 2008/6/12 Ray Brownrigg <[hidden email]>:
>> I guess this is more r-devel than r-help.
>>
>> Note, I am just the messenger - I have no idea what the user is trying to model here.
>>
>> arima() crashes R (segfault) with Linux R-2.7.0, Solaris R-2.6.0:
>>
>>  *** caught segfault ***
>> address 42400000, cause 'memory not mapped'
>>
>> Traceback:
>>  1: .Call(R_getQ0, phi, theta)
>>  2: makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa)
>>  3: arima(x, c(1, 0, 1), c(1, 0, 1))
>>
>> Under rw-2.7.0 or R version 2.8.0 Under development (unstable) (2008-06-10 r45893)
>> it gets:
>> Error: cannot allocate vector of size 1010.9 Mb
>> In addition: Warning messages:
>> 1: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>>  Reached total allocation of 447Mb: see help(memory.size)
>> 2: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>>  Reached total allocation of 447Mb: see help(memory.size)
>> 3: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>>  Reached total allocation of 447Mb: see help(memory.size)
>> 4: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>>  Reached total allocation of 447Mb: see help(memory.size)
>>
>> Reproduce by:
>>
>> # 2 years of daily temperature data
>> set.seed(1); x <- ts(20*sin((1:731)*2*pi/365) + 10 + rnorm(731, 0, 4), freq=365)
>> arima(x, c(1, 0, 1), c(1, 0, 1))
>>
>> Ray Brownrigg
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
>
>
> --
> Antonio, Fabio Di Narzo
> Ph.D. student at
> Department of Statistical Sciences
> University of Bologna, Italy
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: arima() bug

Simone Giannerini
In reply to this post by Antonio, Fabio Di Narzo
I get the same behaviour on R version 2.7.0 Patched (2008-06-05 r45857),
Opensuse 10.3 x86_64 (32 Gb RAM)

> set.seed(1); x <- ts(20*sin((1:731)*2*pi/365) + 10 + rnorm(731, 0, 4),
freq=365)
> arima(x, c(1, 0, 1), c(1, 0, 1))

 *** caught segfault ***
address 0x2aafb83e9f50, cause 'memory not mapped'

Traceback:
 1: .Call(R_getQ0, phi, theta)
 2: makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa)
 3: arima(x, c(1, 0, 1), c(1, 0, 1))


the problem seems to show up when the seasonal AR part is included


> arima(x, c(2, 1, 3), c(0, 0, 0))

Call:
arima(x = x, order = c(2, 1, 3), seasonal = c(0, 0, 0))

Coefficients:
         ar1     ar2      ma1      ma2     ma3
      0.0363  0.9477  -0.9434  -0.9468  0.9060
s.e.  0.0291  0.0289   0.0258   0.0397  0.0233

sigma^2 estimated as 18.41:  log likelihood = -2100.79,  aic = 4213.59

> arima(x, c(0, 0, 0), c(0, 0, 1))

Call:
arima(x = x, order = c(0, 0, 0), seasonal = c(0, 0, 1))

Coefficients:
        sma1  intercept
      0.9999     9.9293
s.e.  0.0724     0.5674

sigma^2 estimated as 78.47:  log likelihood = -2832.41,  aic = 5670.81


> arima(x, c(0, 0, 0), c(1, 0, 0))

 *** caught segfault ***
address 0x2b00bf41eae0, cause 'memory not mapped'

Traceback:
 1: .Call(R_getQ0, phi, theta)
 2: makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa)
 3: arima(x, c(0, 0, 0), c(1, 0, 0))


> arima(x, c(0, 0, 0), c(2, 0, 0))
Errore in makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
  cannot allocate memory block of size 137438953465.2 Gb

> arima(x, c(0, 0, 0), c(3, 0, 0))
Errore in optim(init[mask], armaCSS, method = "BFGS", hessian = FALSE,  :
 initial value in 'vmmin' is not finite



> R.version
               _
platform       x86_64-unknown-linux-gnu
arch           x86_64
os             linux-gnu
system         x86_64, linux-gnu
status         Patched
major          2
minor          7.0
year           2008
month          06
day            05
svn rev        45857
language       R
version.string R version 2.7.0 Patched (2008-06-05 r45857)



On Thu, Jun 12, 2008 at 9:27 AM, Antonio, Fabio Di Narzo <
[hidden email]> wrote:

> No segfault with my r-patched version on linux-i686:
>
> > set.seed(1); x <- ts(20*sin((1:731)*2*pi/365) + 10 + rnorm(731, 0, 4),
> freq=365)
> > arima(x, c(1, 0, 1), c(1, 0, 1))
> Errore: cannot allocate vector of size 1010.9 Mb
>
> F.
>
> > R.version
>               _
> platform       i686-pc-linux-gnu
> arch           i686
> os             linux-gnu
> system         i686, linux-gnu
> status         Patched
> major          2
> minor          7.0
> year           2008
> month          05
> day            29
> svn rev        45820
> language       R
> version.string R version 2.7.0 Patched (2008-05-29 r45820)
>
>
> 2008/6/12 Ray Brownrigg <[hidden email]>:
> > I guess this is more r-devel than r-help.
> >
> > Note, I am just the messenger - I have no idea what the user is trying to
> model here.
> >
> > arima() crashes R (segfault) with Linux R-2.7.0, Solaris R-2.6.0:
> >
> >  *** caught segfault ***
> > address 42400000, cause 'memory not mapped'
> >
> > Traceback:
> >  1: .Call(R_getQ0, phi, theta)
> >  2: makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa)
> >  3: arima(x, c(1, 0, 1), c(1, 0, 1))
> >
> > Under rw-2.7.0 or R version 2.8.0 Under development (unstable)
> (2008-06-10 r45893)
> > it gets:
> > Error: cannot allocate vector of size 1010.9 Mb
> > In addition: Warning messages:
> > 1: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
> >  Reached total allocation of 447Mb: see help(memory.size)
> > 2: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
> >  Reached total allocation of 447Mb: see help(memory.size)
> > 3: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
> >  Reached total allocation of 447Mb: see help(memory.size)
> > 4: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
> >  Reached total allocation of 447Mb: see help(memory.size)
> >
> > Reproduce by:
> >
> > # 2 years of daily temperature data
> > set.seed(1); x <- ts(20*sin((1:731)*2*pi/365) + 10 + rnorm(731, 0, 4),
> freq=365)
> > arima(x, c(1, 0, 1), c(1, 0, 1))
> >
> > Ray Brownrigg
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >
>
>
>
> --
> Antonio, Fabio Di Narzo
> Ph.D. student at
> Department of Statistical Sciences
> University of Bologna, Italy
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>



--
______________________________________________________

Simone Giannerini
Dipartimento di Scienze Statistiche "Paolo Fortunati"
Universita' di Bologna
Via delle belle arti 41 - 40126 Bologna, ITALY
Tel: +39 051 2098262 Fax: +39 051 232153
http://www2.stat.unibo.it/giannerini/
______________________________________________________

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: arima() bug

Bill Dunlap
In reply to this post by Ray.Brownrigg
On Thu, 12 Jun 2008, Ray Brownrigg wrote:

> arima() crashes R (segfault) with Linux R-2.7.0, Solaris R-2.6.0:
>
> Reproduce by:
>
> # 2 years of daily temperature data
> set.seed(1); x <- ts(20*sin((1:731)*2*pi/365) + 10 + rnorm(731, 0, 4), freq=365)
> arima(x, c(1, 0, 1), c(1, 0, 1))

I put a breakpoint in Rf_allocVector when its length argument
was more than 10 million and it stopped when library/stats/src/arima.c:getQ0
asked for a vector of nrbar=132497980 doubles, a number proportional
to the 4th power of max(length(phi),length(theta)) from the R arima():upARIMA()
function:

SEXP getQ0(SEXP sPhi, SEXP sTheta)
{
    ...
    int  p = LENGTH(sPhi), q = LENGTH(sTheta);
    ...
    int r = max(p, q + 1), np = r * (r + 1) / 2, nrbar = np * (np - 1) / 2;
    ...
    rbar = (double *) R_alloc(nrbar, sizeof(double));

(gdb) print nrbar
$1 = 132497980
(gdb) print np
$2 = 67528
(gdb) print r
$3 = 367
(gdb)  print q
$4 = 366

Trying to recover from running out of memory probably
causes the crash.

rbar is a scratch array.

----------------------------------------------------------------------------
Bill Dunlap
Insightful Corporation
bill at insightful dot com

 "All statements in this message represent the opinions of the author and do
 not necessarily reflect Insightful Corporation policy or position."

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: arima() bug

Prof Brian Ripley
In reply to this post by Prof Brian Ripley
On Thu, 12 Jun 2008, Prof Brian Ripley wrote:

> On Thu, 12 Jun 2008, Antonio, Fabio Di Narzo wrote:
>
>> No segfault with my r-patched version on linux-i686:
>>
>>> set.seed(1); x <- ts(20*sin((1:731)*2*pi/365) + 10 + rnorm(731, 0, 4),
>>> freq=365)
>>> arima(x, c(1, 0, 1), c(1, 0, 1))
>> Errore: cannot allocate vector of size 1010.9 Mb
>
> Yes, you need a lot of memory to reproduce it.  It's a nonsensical
> calculation but nevertheless we need to track down the cause.

Which is an integer overflow in an integer calculation of a dimension.
The current code is only able to code with lags up to 350, and this is
attempting a fit with lag 366.  I've added an explicit check.

>
>>
>> F.
>>
>>> R.version
>>               _
>> platform       i686-pc-linux-gnu
>> arch           i686
>> os             linux-gnu
>> system         i686, linux-gnu
>> status         Patched
>> major          2
>> minor          7.0
>> year           2008
>> month          05
>> day            29
>> svn rev        45820
>> language       R
>> version.string R version 2.7.0 Patched (2008-05-29 r45820)
>>
>>
>> 2008/6/12 Ray Brownrigg <[hidden email]>:
>>> I guess this is more r-devel than r-help.
>>>
>>> Note, I am just the messenger - I have no idea what the user is trying to
>>> model here.
>>>
>>> arima() crashes R (segfault) with Linux R-2.7.0, Solaris R-2.6.0:
>>>
>>>  *** caught segfault ***
>>> address 42400000, cause 'memory not mapped'
>>>
>>> Traceback:
>>>  1: .Call(R_getQ0, phi, theta)
>>>  2: makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa)
>>>  3: arima(x, c(1, 0, 1), c(1, 0, 1))
>>>
>>> Under rw-2.7.0 or R version 2.8.0 Under development (unstable) (2008-06-10
>>> r45893)
>>> it gets:
>>> Error: cannot allocate vector of size 1010.9 Mb
>>> In addition: Warning messages:
>>> 1: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>>>  Reached total allocation of 447Mb: see help(memory.size)
>>> 2: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>>>  Reached total allocation of 447Mb: see help(memory.size)
>>> 3: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>>>  Reached total allocation of 447Mb: see help(memory.size)
>>> 4: In makeARIMA(trarma[[1]], trarma[[2]], Delta, kappa) :
>>>  Reached total allocation of 447Mb: see help(memory.size)
>>>
>>> Reproduce by:
>>>
>>> # 2 years of daily temperature data
>>> set.seed(1); x <- ts(20*sin((1:731)*2*pi/365) + 10 + rnorm(731, 0, 4),
>>> freq=365)
>>> arima(x, c(1, 0, 1), c(1, 0, 1))
>>>
>>> Ray Brownrigg
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>
>>
>>
>>
>> --
>> Antonio, Fabio Di Narzo
>> Ph.D. student at
>> Department of Statistical Sciences
>> University of Bologna, Italy
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
> --
> Brian D. Ripley,                  [hidden email]
> Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
> University of Oxford,             Tel:  +44 1865 272861 (self)
> 1 South Parks Road,                     +44 1865 272866 (PA)
> Oxford OX1 3TG, UK                Fax:  +44 1865 272595
>

--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel