Resampling 1 time series at another set of (known) irregularly spaced times

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Resampling 1 time series at another set of (known) irregularly spaced times

John Hillier
Dear All,


I would appreciate a quick pointer in the right direction (e.g. www page I could look at, or indicator of which function within a package).


The problem: I have a regular time series of values x at times t (i.e. t, x). I would like to sample them at irregular, known times - this is a second time series (T).


I can move these data between formats as required (i.e. file, vector, matrix, ts etc ....)


I have been searching around for a while and found many packages to regularise time-series (e.g. xts, lubricate, ..... ), but not the reverse as I want to.


Before you ask, I know it might seem a bit odd, but it is necessary for the particular question I'm asking.


Thank you for your time,


John


-------------------------
Work days: Mon-Thurs
Web page: <http://homepages.lboro.ac.uk/~gyjh5/> <http://www.lboro.ac.uk/departments/geography/staff/john-hillier/> http://www.lboro.ac.uk/departments/geography/staff/john-hillier/
Latest research: http://publications.lboro.ac.uk/publications/all/collated/gyjh5.html<https://lb-public.lboro.ac.uk/cgi-bin/personcite?username=gyjh5&dobranding=1&hits=10>

Dr John Hillier
Senior Lecturer & NERC Knowledge Exchange Fellow (Insurance Sector)
Geography and Environment
Loughborough University
01509 223727

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Resampling 1 time series at another set of (known) irregularly spaced times

Jeff Newmiller
The key to accomplishing this is to clarify how you want to address selecting values between the existing points, but there are many base R functions and packages that address this problem. In general the methods fall into two categories: interpolation and smoothing. Interpolation includes piecewise linear interpolation, splines, last-observation-carried-forward, and  first-order-extrapolation, all of which yield the same values of applied only at the original independent values. Smoothing methods such as regression, loess, kriging, and kernel interpolation may not have this identity property but you don't need unique input values at each independent variable value either.

Read some Task Views, e.g.

https://cran.r-project.org/web/views/NumericalMathematics.html

https://cran.r-project.org/web/views/TimeSeries.html

https://cran.r-project.org/web/views/MissingData.html



On January 9, 2019 2:55:04 PM PST, John Hillier <[hidden email]> wrote:

>Dear All,
>
>
>I would appreciate a quick pointer in the right direction (e.g. www
>page I could look at, or indicator of which function within a package).
>
>
>The problem: I have a regular time series of values x at times t (i.e.
>t, x). I would like to sample them at irregular, known times - this is
>a second time series (T).
>
>
>I can move these data between formats as required (i.e. file, vector,
>matrix, ts etc ....)
>
>
>I have been searching around for a while and found many packages to
>regularise time-series (e.g. xts, lubricate, ..... ), but not the
>reverse as I want to.
>
>
>Before you ask, I know it might seem a bit odd, but it is necessary for
>the particular question I'm asking.
>
>
>Thank you for your time,
>
>
>John
>
>
>-------------------------
>Work days: Mon-Thurs
>Web page: <http://homepages.lboro.ac.uk/~gyjh5/>
><http://www.lboro.ac.uk/departments/geography/staff/john-hillier/>
>http://www.lboro.ac.uk/departments/geography/staff/john-hillier/
>Latest research:
>http://publications.lboro.ac.uk/publications/all/collated/gyjh5.html<https://lb-public.lboro.ac.uk/cgi-bin/personcite?username=gyjh5&dobranding=1&hits=10>
>
>Dr John Hillier
>Senior Lecturer & NERC Knowledge Exchange Fellow (Insurance Sector)
>Geography and Environment
>Loughborough University
>01509 223727
>
> [[alternative HTML version deleted]]
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

--
Sent from my phone. Please excuse my brevity.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Resampling 1 time series at another set of (known) irregularly spaced times

Bert Gunter-2
John:

Clarification: Do you mean you just want an "irregular" subset of your
*given* data values/times, or do you want times randomly over the series
duration for which you will construct values, which is what Jeff described.

The former is trivial: see ?sample with the "replace" argument = FALSE :
you're actually just sampling from the integer vector of time indices here,
so sample.int would even do. For the latter, I would presume you could use
?runif to sample arbitrary times over the time series duration and then
follow Jeff's suggestions to fill in values for these times using methods
to which he referred you.

Or have I misunderstood completely?

Cheers,
Bert

Bert Gunter

"The trouble with having an open mind is that people keep coming along and
sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Wed, Jan 9, 2019 at 3:17 PM Jeff Newmiller <[hidden email]>
wrote:

> The key to accomplishing this is to clarify how you want to address
> selecting values between the existing points, but there are many base R
> functions and packages that address this problem. In general the methods
> fall into two categories: interpolation and smoothing. Interpolation
> includes piecewise linear interpolation, splines,
> last-observation-carried-forward, and  first-order-extrapolation, all of
> which yield the same values of applied only at the original independent
> values. Smoothing methods such as regression, loess, kriging, and kernel
> interpolation may not have this identity property but you don't need unique
> input values at each independent variable value either.
>
> Read some Task Views, e.g.
>
> https://cran.r-project.org/web/views/NumericalMathematics.html
>
> https://cran.r-project.org/web/views/TimeSeries.html
>
> https://cran.r-project.org/web/views/MissingData.html
>
>
>
> On January 9, 2019 2:55:04 PM PST, John Hillier <[hidden email]>
> wrote:
> >Dear All,
> >
> >
> >I would appreciate a quick pointer in the right direction (e.g. www
> >page I could look at, or indicator of which function within a package).
> >
> >
> >The problem: I have a regular time series of values x at times t (i.e.
> >t, x). I would like to sample them at irregular, known times - this is
> >a second time series (T).
> >
> >
> >I can move these data between formats as required (i.e. file, vector,
> >matrix, ts etc ....)
> >
> >
> >I have been searching around for a while and found many packages to
> >regularise time-series (e.g. xts, lubricate, ..... ), but not the
> >reverse as I want to.
> >
> >
> >Before you ask, I know it might seem a bit odd, but it is necessary for
> >the particular question I'm asking.
> >
> >
> >Thank you for your time,
> >
> >
> >John
> >
> >
> >-------------------------
> >Work days: Mon-Thurs
> >Web page: <http://homepages.lboro.ac.uk/~gyjh5/>
> ><http://www.lboro.ac.uk/departments/geography/staff/john-hillier/>
> >http://www.lboro.ac.uk/departments/geography/staff/john-hillier/
> >Latest research:
> >http://publications.lboro.ac.uk/publications/all/collated/gyjh5.html<
> https://lb-public.lboro.ac.uk/cgi-bin/personcite?username=gyjh5&dobranding=1&hits=10
> >
> >
> >Dr John Hillier
> >Senior Lecturer & NERC Knowledge Exchange Fellow (Insurance Sector)
> >Geography and Environment
> >Loughborough University
> >01509 223727
> >
> >       [[alternative HTML version deleted]]
> >
> >______________________________________________
> >[hidden email] mailing list -- To UNSUBSCRIBE and more, see
> >https://stat.ethz.ch/mailman/listinfo/r-help
> >PLEASE do read the posting guide
> >http://www.R-project.org/posting-guide.html
> >and provide commented, minimal, self-contained, reproducible code.
>
> --
> Sent from my phone. Please excuse my brevity.
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide
> http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Resampling 1 time series at another set of (known) irregularly spaced times

John Hillier
Thank you Jeff and Bert,


You provided the pointer to reliable starting place that I needed after my initial searching.  I now better understand the terminology/approach used in R for this sort of thing. The answer is indeed trivial when you know how e.g. use the 'xout' option in approx().


Now I've got my eye-in, and I have been able to do similar in a range of other packages/functions.


Thank you!


John


p.s. - For the record, some more detail clarifying question and answer.


# A regular time series
s1 <- as.data.frame(c(0,1,2,3,4,5,6,7,8,9,10))
s1[,2] <- c(0,0.1,0.2,0.3,0.4,0.5,0.4,0.3,0.2,0.4,0)
names(s1)[1] <- "t"
names(s1)[2] <- "x"

# irregular, known times to sample this at
s2 <- as.data.frame(c(0.3,2.1,2.6,4,6.5,8.8,9.3))
names(s2)[1] <- "T"


# Solution - e.g. for approx() in RBase use the 'xout' option.

# approx(x, y, xout, method="linear", n=50, yleft, yright, rule=1, f=0)

out <- approx(s1[,1],s1[,2],s2[,1], method="linear")

out$x[]

out$y[]


-------------------------
Work days: Mon-Thurs
Web page: <http://homepages.lboro.ac.uk/~gyjh5/> <http://www.lboro.ac.uk/departments/geography/staff/john-hillier/> http://www.lboro.ac.uk/departments/geography/staff/john-hillier/
Latest research: http://publications.lboro.ac.uk/publications/all/collated/gyjh5.html<https://lb-public.lboro.ac.uk/cgi-bin/personcite?username=gyjh5&dobranding=1&hits=10>

Dr John Hillier
Senior Lecturer & NERC Knowledge Exchange Fellow (Insurance Sector)
Geography and Environment
Loughborough University
01509 223727
________________________________
From: Bert Gunter <[hidden email]>
Sent: 09 January 2019 23:37:01
To: Jeff Newmiller
Cc: R-help; John Hillier
Subject: Re: [R] Resampling 1 time series at another set of (known) irregularly spaced times

John:

Clarification: Do you mean you just want an "irregular" subset of your *given* data values/times, or do you want times randomly over the series duration for which you will construct values, which is what Jeff described.

The former is trivial: see ?sample with the "replace" argument = FALSE : you're actually just sampling from the integer vector of time indices here, so sample.int<http://sample.int> would even do. For the latter, I would presume you could use ?runif to sample arbitrary times over the time series duration and then follow Jeff's suggestions to fill in values for these times using methods to which he referred you.

Or have I misunderstood completely?

Cheers,
Bert

Bert Gunter

"The trouble with having an open mind is that people keep coming along and sticking things into it."
-- Opus (aka Berkeley Breathed in his "Bloom County" comic strip )


On Wed, Jan 9, 2019 at 3:17 PM Jeff Newmiller <[hidden email]<mailto:[hidden email]>> wrote:
The key to accomplishing this is to clarify how you want to address selecting values between the existing points, but there are many base R functions and packages that address this problem. In general the methods fall into two categories: interpolation and smoothing. Interpolation includes piecewise linear interpolation, splines, last-observation-carried-forward, and  first-order-extrapolation, all of which yield the same values of applied only at the original independent values. Smoothing methods such as regression, loess, kriging, and kernel interpolation may not have this identity property but you don't need unique input values at each independent variable value either.

Read some Task Views, e.g.

https://cran.r-project.org/web/views/NumericalMathematics.html

https://cran.r-project.org/web/views/TimeSeries.html

https://cran.r-project.org/web/views/MissingData.html



On January 9, 2019 2:55:04 PM PST, John Hillier <[hidden email]<mailto:[hidden email]>> wrote:

>Dear All,
>
>
>I would appreciate a quick pointer in the right direction (e.g. www
>page I could look at, or indicator of which function within a package).
>
>
>The problem: I have a regular time series of values x at times t (i.e.
>t, x). I would like to sample them at irregular, known times - this is
>a second time series (T).
>
>
>I can move these data between formats as required (i.e. file, vector,
>matrix, ts etc ....)
>
>
>I have been searching around for a while and found many packages to
>regularise time-series (e.g. xts, lubricate, ..... ), but not the
>reverse as I want to.
>
>
>Before you ask, I know it might seem a bit odd, but it is necessary for
>the particular question I'm asking.
>
>
>Thank you for your time,
>
>
>John
>
>
>-------------------------
>Work days: Mon-Thurs
>Web page: <http://homepages.lboro.ac.uk/~gyjh5/>
><http://www.lboro.ac.uk/departments/geography/staff/john-hillier/>
>http://www.lboro.ac.uk/departments/geography/staff/john-hillier/
>Latest research:
>http://publications.lboro.ac.uk/publications/all/collated/gyjh5.html<https://lb-public.lboro.ac.uk/cgi-bin/personcite?username=gyjh5&dobranding=1&hits=10>
>
>Dr John Hillier
>Senior Lecturer & NERC Knowledge Exchange Fellow (Insurance Sector)
>Geography and Environment
>Loughborough University
>01509 223727
>
>       [[alternative HTML version deleted]]
>
>______________________________________________
>[hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

--
Sent from my phone. Please excuse my brevity.

______________________________________________
[hidden email]<mailto:[hidden email]> mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.