Precision error in time index of ts objects

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Precision error in time index of ts objects

Andrea Altomani
I have a time series x, and two other series obtained from it:

x <- structure(2017, .Tsp = c(2017.41666666667, 2017.41666666667, 12),
class = "ts")
y <- floor(x)
z <- x-y

I would expect the three series to have exactly the same index.
However I get the following

> time(x)-time(y)
     Jun
2017   0

as expected, but

> time(x)-time(z)
integer(0)
Warning message:
In .cbind.ts(list(e1, e2), c(deparse(substitute(e1))[1L],
deparse(substitute(e2))[1L]),  :
  non-intersecting series

and indeed, comparing the indices gives:

> time(x)[1]-time(z)[1]
[1] 3.183231e-12

Is this a bug in R, or is it one of the expected precision errors due
to the use of limited precision floats?

I am using R 3.4.0 (2017-04-21) on Windows (64-bit).

Thaks!

Andrea Altomani

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Precision error in time index of ts objects

Jeff Newmiller
You already know the answer. Why ask?
--
Sent from my phone. Please excuse my brevity.

On September 1, 2017 7:23:24 AM PDT, Andrea Altomani <[hidden email]> wrote:

>I have a time series x, and two other series obtained from it:
>
>x <- structure(2017, .Tsp = c(2017.41666666667, 2017.41666666667, 12),
>class = "ts")
>y <- floor(x)
>z <- x-y
>
>I would expect the three series to have exactly the same index.
>However I get the following
>
>> time(x)-time(y)
>     Jun
>2017   0
>
>as expected, but
>
>> time(x)-time(z)
>integer(0)
>Warning message:
>In .cbind.ts(list(e1, e2), c(deparse(substitute(e1))[1L],
>deparse(substitute(e2))[1L]),  :
>  non-intersecting series
>
>and indeed, comparing the indices gives:
>
>> time(x)[1]-time(z)[1]
>[1] 3.183231e-12
>
>Is this a bug in R, or is it one of the expected precision errors due
>to the use of limited precision floats?
>
>I am using R 3.4.0 (2017-04-21) on Windows (64-bit).
>
>Thaks!
>
>Andrea Altomani
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Precision error in time index of ts objects

Andrea Altomani
I should have formulated my question in a more specific way.

1. I suspect this is a floating point precision issue. I am not very
knowledgeable about R internals, can someone else confirm it?

2. Should this be considered a bug or not, because it is "just a precision
issue"? Should I report it?

3. How can it happen? From a quick review of ts.R, it looks like the values
of the time index are never modified, but only possibly removed. In my case:
   - x and y have the same index.
   - the subtraction operator recognizes this, and create a new ts with one
entry
   - the result of the subtraction has an index which is different from the
input.
  This is very surprising to me, and I am curious to understand the problem.


--
Andrea Altomani



On Fri, Sep 1, 2017 at 5:53 PM Jeff Newmiller <[hidden email]>
wrote:

> You already know the answer. Why ask?
> --
> Sent from my phone. Please excuse my brevity.
>
> On September 1, 2017 7:23:24 AM PDT, Andrea Altomani <
> [hidden email]> wrote:
> >I have a time series x, and two other series obtained from it:
> >
> >x <- structure(2017, .Tsp = c(2017.41666666667, 2017.41666666667, 12),
> >class = "ts")
> >y <- floor(x)
> >z <- x-y
> >
> >I would expect the three series to have exactly the same index.
> >However I get the following
> >
> >> time(x)-time(y)
> >     Jun
> >2017   0
> >
> >as expected, but
> >
> >> time(x)-time(z)
> >integer(0)
> >Warning message:
> >In .cbind.ts(list(e1, e2), c(deparse(substitute(e1))[1L],
> >deparse(substitute(e2))[1L]),  :
> >  non-intersecting series
> >
> >and indeed, comparing the indices gives:
> >
> >> time(x)[1]-time(z)[1]
> >[1] 3.183231e-12
> >
> >Is this a bug in R, or is it one of the expected precision errors due
> >to the use of limited precision floats?
> >
> >I am using R 3.4.0 (2017-04-21) on Windows (64-bit).
> >
> >Thaks!
> >
> >Andrea Altomani
> >
> >______________________________________________
> >[hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Precision error in time index of ts objects

Achim Zeileis-4
On Fri, 1 Sep 2017, Andrea Altomani wrote:

> I should have formulated my question in a more specific way.
>
> 1. I suspect this is a floating point precision issue. I am not very
> knowledgeable about R internals, can someone else confirm it?

Yes. If you represent a series with increment 1/12 it depends on how you
do it. As a simple example consider the following two descriptions of the
same time point:

2 - 1/12
## [1] 1.916667

1 + 11/12
## [1] 1.916667

However, both are not identical:

(2 - 1/12) == (1 + 11/12)
## [1] FALSE

The difference is just the .Machine$double.eps:

(2 - 1/12) - (1 + 11/12)
## [1] 2.220446e-16

> 2. Should this be considered a bug or not, because it is "just a
> precision issue"? Should I report it?

I don't think it is a bug because of the (non-standard) way how you
created the time series.

> 3. How can it happen? From a quick review of ts.R, it looks like the values
> of the time index are never modified, but only possibly removed. In my case:
>   - x and y have the same index.
>   - the subtraction operator recognizes this, and create a new ts with one
> entry
>   - the result of the subtraction has an index which is different from the
> input.
>  This is very surprising to me, and I am curious to understand the problem.

The object 'x' and hence the object 'y' have the same time index. But in
'z' a new time index is created which is subtly different from that of
'x'. The reason for this is that R doesn't expect an object like 'x' to
exist.

You should create a "ts" object with ts(), e.g.,

x <- ts(2017, start = c(2017, 6), freqency = 12)

But you created something close to the internal representation...but not
close enough:

y <- structure(2017, .Tsp = c(2017.416667, 2017.416667, 12), class = "ts")

The print functions prints both print(x) and print(y) as

       Jun
2017 2017

However, aligning the two time indexes in x - y or ts.intersect(x, y) does
not work...because they are not the same

as.numeric(time(x)) - as.numeric(time(y))
## [1] -3.333332e-07

The "ts" code tries to avoid these situations by making many time index
comparisons only up to a precision of getOption("ts.eps") (1e-5 by
default) but this is not used everywhere. See ?options:

     'ts.eps': the relative tolerance for certain time series ('ts')
           computations.  Default '1e-05'.

Of course, you could ask for this being used in more places, e.g., in
stats:::.cbind.ts() where (st > en) is used rather than ((st - en) >
getOption("ts.eps")). But it's probably safer to just use ts() rather than
structure(). Or if you use the latter make sure that you do at a high
enough precision.

hth,
Z


> On Fri, Sep 1, 2017 at 5:53 PM Jeff Newmiller <[hidden email]>
> wrote:
>
>> You already know the answer. Why ask?
>> --
>> Sent from my phone. Please excuse my brevity.
>>
>> On September 1, 2017 7:23:24 AM PDT, Andrea Altomani <
>> [hidden email]> wrote:
>>> I have a time series x, and two other series obtained from it:
>>>
>>> x <- structure(2017, .Tsp = c(2017.41666666667, 2017.41666666667, 12),
>>> class = "ts")
>>> y <- floor(x)
>>> z <- x-y
>>>
>>> I would expect the three series to have exactly the same index.
>>> However I get the following
>>>
>>>> time(x)-time(y)
>>>     Jun
>>> 2017   0
>>>
>>> as expected, but
>>>
>>>> time(x)-time(z)
>>> integer(0)
>>> Warning message:
>>> In .cbind.ts(list(e1, e2), c(deparse(substitute(e1))[1L],
>>> deparse(substitute(e2))[1L]),  :
>>>  non-intersecting series
>>>
>>> and indeed, comparing the indices gives:
>>>
>>>> time(x)[1]-time(z)[1]
>>> [1] 3.183231e-12
>>>
>>> Is this a bug in R, or is it one of the expected precision errors due
>>> to the use of limited precision floats?
>>>
>>> I am using R 3.4.0 (2017-04-21) on Windows (64-bit).
>>>
>>> Thaks!
>>>
>>> Andrea Altomani
>>>
>>> ______________________________________________
>>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>>> https://stat.ethz.ch/mailman/listinfo/r-help
>>> PLEASE do read the posting guide
>>> http://www.R-project.org/posting-guide.html
>>> and provide commented, minimal, self-contained, reproducible code.
>>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Precision error in time index of ts objects

Andrea Altomani
Thanks for the very detailed explanation.

I did not create the series using structure(), that was the result of
dump() on an intermediate object created within tsdisagg::ta(), which is
where I found the error in the first place. ta() indeed manipulates .Tsp
directly, rather than using ts. I guess this is a bug in tsdisagg then.

Thanks!


--
Andrea Altomani


On Sat, Sep 2, 2017 at 12:31 AM Achim Zeileis <[hidden email]>
wrote:

> On Fri, 1 Sep 2017, Andrea Altomani wrote:
>
> > I should have formulated my question in a more specific way.
> >
> > 1. I suspect this is a floating point precision issue. I am not very
> > knowledgeable about R internals, can someone else confirm it?
>
> Yes. If you represent a series with increment 1/12 it depends on how you
> do it. As a simple example consider the following two descriptions of the
> same time point:
>
> 2 - 1/12
> ## [1] 1.916667
>
> 1 + 11/12
> ## [1] 1.916667
>
> However, both are not identical:
>
> (2 - 1/12) == (1 + 11/12)
> ## [1] FALSE
>
> The difference is just the .Machine$double.eps:
>
> (2 - 1/12) - (1 + 11/12)
> ## [1] 2.220446e-16
>
> > 2. Should this be considered a bug or not, because it is "just a
> > precision issue"? Should I report it?
>
> I don't think it is a bug because of the (non-standard) way how you
> created the time series.
>
> > 3. How can it happen? From a quick review of ts.R, it looks like the
> values
> > of the time index are never modified, but only possibly removed. In my
> case:
> >   - x and y have the same index.
> >   - the subtraction operator recognizes this, and create a new ts with
> one
> > entry
> >   - the result of the subtraction has an index which is different from
> the
> > input.
> >  This is very surprising to me, and I am curious to understand the
> problem.
>
> The object 'x' and hence the object 'y' have the same time index. But in
> 'z' a new time index is created which is subtly different from that of
> 'x'. The reason for this is that R doesn't expect an object like 'x' to
> exist.
>
> You should create a "ts" object with ts(), e.g.,
>
> x <- ts(2017, start = c(2017, 6), freqency = 12)
>
> But you created something close to the internal representation...but not
> close enough:
>
> y <- structure(2017, .Tsp = c(2017.416667, 2017.416667, 12), class = "ts")
>
> The print functions prints both print(x) and print(y) as
>
>        Jun
> 2017 2017
>
> However, aligning the two time indexes in x - y or ts.intersect(x, y) does
> not work...because they are not the same
>
> as.numeric(time(x)) - as.numeric(time(y))
> ## [1] -3.333332e-07
>
> The "ts" code tries to avoid these situations by making many time index
> comparisons only up to a precision of getOption("ts.eps") (1e-5 by
> default) but this is not used everywhere. See ?options:
>
>      'ts.eps': the relative tolerance for certain time series ('ts')
>            computations.  Default '1e-05'.
>
> Of course, you could ask for this being used in more places, e.g., in
> stats:::.cbind.ts() where (st > en) is used rather than ((st - en) >
> getOption("ts.eps")). But it's probably safer to just use ts() rather than
> structure(). Or if you use the latter make sure that you do at a high
> enough precision.
>
> hth,
> Z
>
>
> > On Fri, Sep 1, 2017 at 5:53 PM Jeff Newmiller <[hidden email]>
> > wrote:
> >
> >> You already know the answer. Why ask?
> >> --
> >> Sent from my phone. Please excuse my brevity.
> >>
> >> On September 1, 2017 7:23:24 AM PDT, Andrea Altomani <
> >> [hidden email]> wrote:
> >>> I have a time series x, and two other series obtained from it:
> >>>
> >>> x <- structure(2017, .Tsp = c(2017.41666666667, 2017.41666666667, 12),
> >>> class = "ts")
> >>> y <- floor(x)
> >>> z <- x-y
> >>>
> >>> I would expect the three series to have exactly the same index.
> >>> However I get the following
> >>>
> >>>> time(x)-time(y)
> >>>     Jun
> >>> 2017   0
> >>>
> >>> as expected, but
> >>>
> >>>> time(x)-time(z)
> >>> integer(0)
> >>> Warning message:
> >>> In .cbind.ts(list(e1, e2), c(deparse(substitute(e1))[1L],
> >>> deparse(substitute(e2))[1L]),  :
> >>>  non-intersecting series
> >>>
> >>> and indeed, comparing the indices gives:
> >>>
> >>>> time(x)[1]-time(z)[1]
> >>> [1] 3.183231e-12
> >>>
> >>> Is this a bug in R, or is it one of the expected precision errors due
> >>> to the use of limited precision floats?
> >>>
> >>> I am using R 3.4.0 (2017-04-21) on Windows (64-bit).
> >>>
> >>> Thaks!
> >>>
> >>> Andrea Altomani
> >>>
> >>> ______________________________________________
> >>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >>> https://stat.ethz.ch/mailman/listinfo/r-help
> >>> PLEASE do read the posting guide
> >>> http://www.R-project.org/posting-guide.html
> >>> and provide commented, minimal, self-contained, reproducible code.
> >>
> >
> >       [[alternative HTML version deleted]]
> >
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> > https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> > and provide commented, minimal, self-contained, reproducible code.
> >
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Precision error in time index of ts objects

Achim Zeileis-4
On Sat, 2 Sep 2017, Andrea Altomani wrote:

> Thanks for the very detailed explanation.
> I did not create the series using structure(), that was the result of dump()
> on an intermediate object created within tsdisagg::ta(),

There is no tsdisagg package on CRAN, just tsdisagg2. But this does not
have a function ta(). So I guess it's tempdisagg you are using?

> which is where I found the error in the first place. ta() indeed
> manipulates .Tsp directly, rather than using ts. I guess this is a bug
> in tsdisagg then.

I just grabbed the latest version of tempdisagg from CRAN and this does
not seem to have ".Tsp" anywhere in the code. It employs ts() in a couple
of places so I'm not sure which part of the code you are referring to
exactly.

> On Sat, Sep 2, 2017 at 12:31 AM Achim Zeileis <[hidden email]>
> wrote:
>       On Fri, 1 Sep 2017, Andrea Altomani wrote:
>
>       > I should have formulated my question in a more specific way.
>       >
>       > 1. I suspect this is a floating point precision issue. I am
>       not very
>       > knowledgeable about R internals, can someone else confirm it?
>
>       Yes. If you represent a series with increment 1/12 it depends on
>       how you
>       do it. As a simple example consider the following two
>       descriptions of the
>       same time point:
>
>       2 - 1/12
>       ## [1] 1.916667
>
>       1 + 11/12
>       ## [1] 1.916667
>
>       However, both are not identical:
>
>       (2 - 1/12) == (1 + 11/12)
>       ## [1] FALSE
>
>       The difference is just the .Machine$double.eps:
>
>       (2 - 1/12) - (1 + 11/12)
>       ## [1] 2.220446e-16
>
>       > 2. Should this be considered a bug or not, because it is "just
>       a
>       > precision issue"? Should I report it?
>
>       I don't think it is a bug because of the (non-standard) way how
>       you
>       created the time series.
>
>       > 3. How can it happen? From a quick review of ts.R, it looks
>       like the values
>       > of the time index are never modified, but only possibly
>       removed. In my case:
>       >   - x and y have the same index.
>       >   - the subtraction operator recognizes this, and create a new
>       ts with one
>       > entry
>       >   - the result of the subtraction has an index which is
>       different from the
>       > input.
>       >  This is very surprising to me, and I am curious to understand
>       the problem.
>
>       The object 'x' and hence the object 'y' have the same time
>       index. But in
>       'z' a new time index is created which is subtly different from
>       that of
>       'x'. The reason for this is that R doesn't expect an object like
>       'x' to
>       exist.
>
>       You should create a "ts" object with ts(), e.g.,
>
>       x <- ts(2017, start = c(2017, 6), freqency = 12)
>
>       But you created something close to the internal
>       representation...but not
>       close enough:
>
>       y <- structure(2017, .Tsp = c(2017.416667, 2017.416667, 12),
>       class = "ts")
>
>       The print functions prints both print(x) and print(y) as
>
>              Jun
>       2017 2017
>
>       However, aligning the two time indexes in x - y or
>       ts.intersect(x, y) does
>       not work...because they are not the same
>
>       as.numeric(time(x)) - as.numeric(time(y))
>       ## [1] -3.333332e-07
>
>       The "ts" code tries to avoid these situations by making many
>       time index
>       comparisons only up to a precision of getOption("ts.eps") (1e-5
>       by
>       default) but this is not used everywhere. See ?options:
>
>            'ts.eps': the relative tolerance for certain time series
>       ('ts')
>                  computations.  Default '1e-05'.
>
>       Of course, you could ask for this being used in more places,
>       e.g., in
>       stats:::.cbind.ts() where (st > en) is used rather than ((st -
>       en) >
>       getOption("ts.eps")). But it's probably safer to just use ts()
>       rather than
>       structure(). Or if you use the latter make sure that you do at a
>       high
>       enough precision.
>
>       hth,
>       Z
>
>
>       > On Fri, Sep 1, 2017 at 5:53 PM Jeff Newmiller
>       <[hidden email]>
>       > wrote:
>       >
>       >> You already know the answer. Why ask?
>       >> --
>       >> Sent from my phone. Please excuse my brevity.
>       >>
>       >> On September 1, 2017 7:23:24 AM PDT, Andrea Altomani <
>       >> [hidden email]> wrote:
>       >>> I have a time series x, and two other series obtained from
>       it:
>       >>>
>       >>> x <- structure(2017, .Tsp = c(2017.41666666667,
>       2017.41666666667, 12),
>       >>> class = "ts")
>       >>> y <- floor(x)
>       >>> z <- x-y
>       >>>
>       >>> I would expect the three series to have exactly the same
>       index.
>       >>> However I get the following
>       >>>
>       >>>> time(x)-time(y)
>       >>>     Jun
>       >>> 2017   0
>       >>>
>       >>> as expected, but
>       >>>
>       >>>> time(x)-time(z)
>       >>> integer(0)
>       >>> Warning message:
>       >>> In .cbind.ts(list(e1, e2), c(deparse(substitute(e1))[1L],
>       >>> deparse(substitute(e2))[1L]),  :
>       >>>  non-intersecting series
>       >>>
>       >>> and indeed, comparing the indices gives:
>       >>>
>       >>>> time(x)[1]-time(z)[1]
>       >>> [1] 3.183231e-12
>       >>>
>       >>> Is this a bug in R, or is it one of the expected precision
>       errors due
>       >>> to the use of limited precision floats?
>       >>>
>       >>> I am using R 3.4.0 (2017-04-21) on Windows (64-bit).
>       >>>
>       >>> Thaks!
>       >>>
>       >>> Andrea Altomani
>       >>>
>       >>> ______________________________________________
>       >>> [hidden email] mailing list -- To UNSUBSCRIBE and
>       more, see
>       >>> https://stat.ethz.ch/mailman/listinfo/r-help
>       >>> PLEASE do read the posting guide
>       >>> http://www.R-project.org/posting-guide.html
>       >>> and provide commented, minimal, self-contained, reproducible
>       code.
>       >>
>       >
>       >       [[alternative HTML version deleted]]
>       >
>       > ______________________________________________
>       > [hidden email] mailing list -- To UNSUBSCRIBE and more,
>       see
>       > https://stat.ethz.ch/mailman/listinfo/r-help
>       > PLEASE do read the posting guide
>       http://www.R-project.org/posting-guide.html
>       > and provide commented, minimal, self-contained, reproducible
>       code.
>       >
>
>
>
______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.