

I have two identically sized matrices of data that represent time series (I am storing the data in zoo objects, but the idea should apply to any matrix of data). The time series in the second matrix extend further than in the first matrix, and I would like to use the data in matrix 2 to extrapolate the data in matrix 1. In other words, if mat1[i,j] == NA, then mat1[i,j] < mat1[i1, j]*mat2[i,j]/mat2[i1,j]. Of course, before we can calculate mat1[i,j] we may need to calculate mat1[i1,j], and that in turn may require the computation of mat1[i2,j], etc. This could all clearly be done with loops, but I am wondering if anyone can think of a vectorized expression or other efficient method that would work.
Thanks very much.
Abiel Reinhart
This communication is for informational purposes only. It is not
intended as an offer or solicitation for the purchase or sale of
any financial instrument or as an official confirmation of any
transaction. All market prices, data and other information are not
warranted as to completeness or accuracy and are subject to change
without notice. Any comments or statements made herein do not
necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
and affiliates.
This transmission may contain information that is privileged,
confidential, legally privileged, and/or exempt from disclosure
under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or
use of the information contained herein (including any reliance
thereon) is STRICTLY PROHIBITED. Although this transmission and any
attachments are believed to be free of any virus or other defect
that might affect any computer system into which it is received and
opened, it is the responsibility of the recipient to ensure that it
is virus free and no responsibility is accepted by JPMorgan Chase &
Co., its subsidiaries and affiliates, as applicable, for any loss
or damage arising in any way from its use. If you received this
transmission in error, please immediately contact the sender and
destroy the material in its entirety, whether in electronic or hard
copy format. Thank you.
Please refer to http://www.jpmorgan.com/pages/disclosures for
disclosures relating to European legal entities.
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/rhelpPLEASE do read the posting guide http://www.Rproject.org/postingguide.htmland provide commented, minimal, selfcontained, reproducible code.


Abiel X Reinhart < [hidden email]> writes:
> I have two identically sized matrices of data that represent time series (I am storing the data in zoo objects, but the idea should apply to any matrix of data). The time series in the second matrix extend further than in the first matrix, and I would like to use the data in matrix 2 to extrapolate the data in matrix 1. In other words, if mat1[i,j] == NA, then mat1[i,j] < mat1[i1, j]*mat2[i,j]/mat2[i1,j]. Of course, before we can calculate mat1[i,j] we may need to calculate mat1[i1,j], and that in turn may require the computation of mat1[i2,j], etc. This could all clearly be done with loops, but I am wondering if anyone can think of a vectorized expression or other efficient method that would work.
If you use tis objects and all of the columns of mat1 become NA on the same row, i.e.,
mat 1 looks like this:
20100101 1 2 3
20100102 4 5 6
20100103 NA NA NA
20100104 NA NA NA
then something like this should work:
mat1 < naWindow(mat1) ## drops the ending NA rows
rate < growth.rate(window(mat2, start = end(mat1)))
growth.rate(mat1) < rate

Jeff
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/rhelpPLEASE do read the posting guide http://www.Rproject.org/postingguide.htmland provide commented, minimal, selfcontained, reproducible code.


Try this using the zoo package. See ?na.approx for more and note that
this functionality requires zoo 1.63 or later.
.> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
> na.approx(m[, 1], x = m[, 2])
19700102 19700103 19700104 19700105 19700106 19700107 19700108
1.000000 2.000000 2.714286 3.714286 5.000000 5.916667 7.000000
> na.approx(m[, 1])
19700102 19700103 19700104 19700105 19700106 19700107 19700108
1 2 3 4 5 6 7
On Tue, May 11, 2010 at 4:48 PM, Abiel X Reinhart
< [hidden email]> wrote:
> I have two identically sized matrices of data that represent time series (I am storing the data in zoo objects, but the idea should apply to any matrix of data). The time series in the second matrix extend further than in the first matrix, and I would like to use the data in matrix 2 to extrapolate the data in matrix 1. In other words, if mat1[i,j] == NA, then mat1[i,j] < mat1[i1, j]*mat2[i,j]/mat2[i1,j]. Of course, before we can calculate mat1[i,j] we may need to calculate mat1[i1,j], and that in turn may require the computation of mat1[i2,j], etc. This could all clearly be done with loops, but I am wondering if anyone can think of a vectorized expression or other efficient method that would work.
>
> Thanks very much.
>
> Abiel Reinhart
> This communication is for informational purposes only. It is not
> intended as an offer or solicitation for the purchase or sale of
> any financial instrument or as an official confirmation of any
> transaction. All market prices, data and other information are not
> warranted as to completeness or accuracy and are subject to change
> without notice. Any comments or statements made herein do not
> necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
> and affiliates.
>
> This transmission may contain information that is privileged,
> confidential, legally privileged, and/or exempt from disclosure
> under applicable law. If you are not the intended recipient, you
> are hereby notified that any disclosure, copying, distribution, or
> use of the information contained herein (including any reliance
> thereon) is STRICTLY PROHIBITED. Although this transmission and any
> attachments are believed to be free of any virus or other defect
> that might affect any computer system into which it is received and
> opened, it is the responsibility of the recipient to ensure that it
> is virus free and no responsibility is accepted by JPMorgan Chase &
> Co., its subsidiaries and affiliates, as applicable, for any loss
> or damage arising in any way from its use. If you received this
> transmission in error, please immediately contact the sender and
> destroy the material in its entirety, whether in electronic or hard
> copy format. Thank you.
>
> Please refer to http://www.jpmorgan.com/pages/disclosures for
> disclosures relating to European legal entities.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/rhelp> PLEASE do read the posting guide http://www.Rproject.org/postingguide.html> and provide commented, minimal, selfcontained, reproducible code.
>
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/rhelpPLEASE do read the posting guide http://www.Rproject.org/postingguide.htmland provide commented, minimal, selfcontained, reproducible code.


Gabor,
This comes close to solving my problem, but I am still left with the problem of how I can extrapolate, not just interpolate. In our example, if I define m as,
m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
instead of
m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
then I will only get five values back, when I really want m[,1] to be fully extrapolated so that there are seven values. Is there a workaround?
Abiel Reinhart
Original Message
From: Gabor Grothendieck [mailto: [hidden email]]
Sent: Wednesday, May 12, 2010 8:37 AM
To: Abiel X Reinhart
Cc: [hidden email]
Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
Try this using the zoo package. See ?na.approx for more and note that
this functionality requires zoo 1.63 or later.
.> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
> na.approx(m[, 1], x = m[, 2])
19700102 19700103 19700104 19700105 19700106 19700107 19700108
1.000000 2.000000 2.714286 3.714286 5.000000 5.916667 7.000000
> na.approx(m[, 1])
19700102 19700103 19700104 19700105 19700106 19700107 19700108
1 2 3 4 5 6 7
On Tue, May 11, 2010 at 4:48 PM, Abiel X Reinhart
< [hidden email]> wrote:
> I have two identically sized matrices of data that represent time series (I am storing the data in zoo objects, but the idea should apply to any matrix of data). The time series in the second matrix extend further than in the first matrix, and I would like to use the data in matrix 2 to extrapolate the data in matrix 1. In other words, if mat1[i,j] == NA, then mat1[i,j] < mat1[i1, j]*mat2[i,j]/mat2[i1,j]. Of course, before we can calculate mat1[i,j] we may need to calculate mat1[i1,j], and that in turn may require the computation of mat1[i2,j], etc. This could all clearly be done with loops, but I am wondering if anyone can think of a vectorized expression or other efficient method that would work.
>
> Thanks very much.
>
> Abiel Reinhart
> This communication is for informational purposes only. It is not
> intended as an offer or solicitation for the purchase or sale of
> any financial instrument or as an official confirmation of any
> transaction. All market prices, data and other information are not
> warranted as to completeness or accuracy and are subject to change
> without notice. Any comments or statements made herein do not
> necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
> and affiliates.
>
> This transmission may contain information that is privileged,
> confidential, legally privileged, and/or exempt from disclosure
> under applicable law. If you are not the intended recipient, you
> are hereby notified that any disclosure, copying, distribution, or
> use of the information contained herein (including any reliance
> thereon) is STRICTLY PROHIBITED. Although this transmission and any
> attachments are believed to be free of any virus or other defect
> that might affect any computer system into which it is received and
> opened, it is the responsibility of the recipient to ensure that it
> is virus free and no responsibility is accepted by JPMorgan Chase &
> Co., its subsidiaries and affiliates, as applicable, for any loss
> or damage arising in any way from its use. If you received this
> transmission in error, please immediately contact the sender and
> destroy the material in its entirety, whether in electronic or hard
> copy format. Thank you.
>
> Please refer to http://www.jpmorgan.com/pages/disclosures for
> disclosures relating to European legal entities.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/rhelp> PLEASE do read the posting guide http://www.Rproject.org/postingguide.html> and provide commented, minimal, selfcontained, reproducible code.
>
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/rhelpPLEASE do read the posting guide http://www.Rproject.org/postingguide.htmland provide commented, minimal, selfcontained, reproducible code.


Use rule = 2 as in the extrapolation examples in the na.approx help file.
On Wed, May 12, 2010 at 10:10 AM, Abiel X Reinhart
< [hidden email]> wrote:
> Gabor,
>
> This comes close to solving my problem, but I am still left with the problem of how I can extrapolate, not just interpolate. In our example, if I define m as,
>
> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>
> instead of
>
> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>
> then I will only get five values back, when I really want m[,1] to be fully extrapolated so that there are seven values. Is there a workaround?
>
> Abiel Reinhart
>
> Original Message
> From: Gabor Grothendieck [mailto: [hidden email]]
> Sent: Wednesday, May 12, 2010 8:37 AM
> To: Abiel X Reinhart
> Cc: [hidden email]
> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>
> Try this using the zoo package. See ?na.approx for more and note that
> this functionality requires zoo 1.63 or later.
>
> .> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>> na.approx(m[, 1], x = m[, 2])
> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
> 1.000000 2.000000 2.714286 3.714286 5.000000 5.916667 7.000000
>> na.approx(m[, 1])
> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
> 1 2 3 4 5 6 7
>
>
> On Tue, May 11, 2010 at 4:48 PM, Abiel X Reinhart
> < [hidden email]> wrote:
>> I have two identically sized matrices of data that represent time series (I am storing the data in zoo objects, but the idea should apply to any matrix of data). The time series in the second matrix extend further than in the first matrix, and I would like to use the data in matrix 2 to extrapolate the data in matrix 1. In other words, if mat1[i,j] == NA, then mat1[i,j] < mat1[i1, j]*mat2[i,j]/mat2[i1,j]. Of course, before we can calculate mat1[i,j] we may need to calculate mat1[i1,j], and that in turn may require the computation of mat1[i2,j], etc. This could all clearly be done with loops, but I am wondering if anyone can think of a vectorized expression or other efficient method that would work.
>>
>> Thanks very much.
>>
>> Abiel Reinhart
>> This communication is for informational purposes only. It is not
>> intended as an offer or solicitation for the purchase or sale of
>> any financial instrument or as an official confirmation of any
>> transaction. All market prices, data and other information are not
>> warranted as to completeness or accuracy and are subject to change
>> without notice. Any comments or statements made herein do not
>> necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
>> and affiliates.
>>
>> This transmission may contain information that is privileged,
>> confidential, legally privileged, and/or exempt from disclosure
>> under applicable law. If you are not the intended recipient, you
>> are hereby notified that any disclosure, copying, distribution, or
>> use of the information contained herein (including any reliance
>> thereon) is STRICTLY PROHIBITED. Although this transmission and any
>> attachments are believed to be free of any virus or other defect
>> that might affect any computer system into which it is received and
>> opened, it is the responsibility of the recipient to ensure that it
>> is virus free and no responsibility is accepted by JPMorgan Chase &
>> Co., its subsidiaries and affiliates, as applicable, for any loss
>> or damage arising in any way from its use. If you received this
>> transmission in error, please immediately contact the sender and
>> destroy the material in its entirety, whether in electronic or hard
>> copy format. Thank you.
>>
>> Please refer to http://www.jpmorgan.com/pages/disclosures for
>> disclosures relating to European legal entities.
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/rhelp>> PLEASE do read the posting guide http://www.Rproject.org/postingguide.html>> and provide commented, minimal, selfcontained, reproducible code.
>>
>
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/rhelpPLEASE do read the posting guide http://www.Rproject.org/postingguide.htmland provide commented, minimal, selfcontained, reproducible code.


Gabor,
Maybe I am doing this wrong, but rule=2 does not look like it is growing the series out, but rather just carrying the last value forward. It looks like na.approx() followed by na.locf(). For instance:
m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
na.approx(m[, 1], x = m[, 2], rule=2)
19700102 19700103 19700104 19700105 19700106 19700107 19700108
1.0 2.0 2.7 3.7 5.0 5.0 5.0
Abiel Reinhart
Original Message
From: Gabor Grothendieck [mailto: [hidden email]]
Sent: Wednesday, May 12, 2010 10:40 AM
To: Abiel X Reinhart
Cc: [hidden email]
Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
Use rule = 2 as in the extrapolation examples in the na.approx help file.
On Wed, May 12, 2010 at 10:10 AM, Abiel X Reinhart
< [hidden email]> wrote:
> Gabor,
>
> This comes close to solving my problem, but I am still left with the problem of how I can extrapolate, not just interpolate. In our example, if I define m as,
>
> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>
> instead of
>
> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>
> then I will only get five values back, when I really want m[,1] to be fully extrapolated so that there are seven values. Is there a workaround?
>
> Abiel Reinhart
>
> Original Message
> From: Gabor Grothendieck [mailto: [hidden email]]
> Sent: Wednesday, May 12, 2010 8:37 AM
> To: Abiel X Reinhart
> Cc: [hidden email]
> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>
> Try this using the zoo package. See ?na.approx for more and note that
> this functionality requires zoo 1.63 or later.
>
> .> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>> na.approx(m[, 1], x = m[, 2])
> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
> 1.000000 2.000000 2.714286 3.714286 5.000000 5.916667 7.000000
>> na.approx(m[, 1])
> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
> 1 2 3 4 5 6 7
>
>
> On Tue, May 11, 2010 at 4:48 PM, Abiel X Reinhart
> < [hidden email]> wrote:
>> I have two identically sized matrices of data that represent time series (I am storing the data in zoo objects, but the idea should apply to any matrix of data). The time series in the second matrix extend further than in the first matrix, and I would like to use the data in matrix 2 to extrapolate the data in matrix 1. In other words, if mat1[i,j] == NA, then mat1[i,j] < mat1[i1, j]*mat2[i,j]/mat2[i1,j]. Of course, before we can calculate mat1[i,j] we may need to calculate mat1[i1,j], and that in turn may require the computation of mat1[i2,j], etc. This could all clearly be done with loops, but I am wondering if anyone can think of a vectorized expression or other efficient method that would work.
>>
>> Thanks very much.
>>
>> Abiel Reinhart
>> This communication is for informational purposes only. It is not
>> intended as an offer or solicitation for the purchase or sale of
>> any financial instrument or as an official confirmation of any
>> transaction. All market prices, data and other information are not
>> warranted as to completeness or accuracy and are subject to change
>> without notice. Any comments or statements made herein do not
>> necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
>> and affiliates.
>>
>> This transmission may contain information that is privileged,
>> confidential, legally privileged, and/or exempt from disclosure
>> under applicable law. If you are not the intended recipient, you
>> are hereby notified that any disclosure, copying, distribution, or
>> use of the information contained herein (including any reliance
>> thereon) is STRICTLY PROHIBITED. Although this transmission and any
>> attachments are believed to be free of any virus or other defect
>> that might affect any computer system into which it is received and
>> opened, it is the responsibility of the recipient to ensure that it
>> is virus free and no responsibility is accepted by JPMorgan Chase &
>> Co., its subsidiaries and affiliates, as applicable, for any loss
>> or damage arising in any way from its use. If you received this
>> transmission in error, please immediately contact the sender and
>> destroy the material in its entirety, whether in electronic or hard
>> copy format. Thank you.
>>
>> Please refer to http://www.jpmorgan.com/pages/disclosures for
>> disclosures relating to European legal entities.
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/rhelp>> PLEASE do read the posting guide http://www.Rproject.org/postingguide.html>> and provide commented, minimal, selfcontained, reproducible code.
>>
>
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/rhelpPLEASE do read the posting guide http://www.Rproject.org/postingguide.htmland provide commented, minimal, selfcontained, reproducible code.


Yes, that is what it does. Note that na.approx interpolates and does
not work precisely as you discussed but its easy, does use m[,2] and
may be good enough. If you really do want something precisely as you
discussed try this. It NAs out the rows of m for which column 1 is NA
and then uses na.locf to move the prior nonNA into it. Then we apply
the formula:
> library(zoo)
> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>
> # mm will hold result; m.locf
> m.locf < mm < m
> ix < is.na(mm[,1])
> m.locf[ix,] < NA
> m.locf < na.locf(m.locf)
> mm[ix, 1] < m.locf[ix, 1] * mm[ix,2] / m.locf[ix,2]
> mm
19700102 1.0 1
19700103 2.0 4
19700104 4.5 9
19700105 8.0 16
19700106 5.0 25
19700107 7.2 36
19700108 9.8 49
19700102 1.0 1
19700103 2.0 4
19700104 4.5 9
19700105 8.0 16
19700106 5.0 25
19700107 7.2 36
19700108 9.8 49
Yes, that is what it does. Please read the help file for na.approx and
approx. If you want something different you will have to special case
the end values.
On Wed, May 12, 2010 at 11:11 AM, Abiel X Reinhart
< [hidden email]> wrote:
> Gabor,
>
> Maybe I am doing this wrong, but rule=2 does not look like it is growing the series out, but rather just carrying the last value forward. It looks like na.approx() followed by na.locf(). For instance:
>
> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
> na.approx(m[, 1], x = m[, 2], rule=2)
>
> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
> 1.0 2.0 2.7 3.7 5.0 5.0 5.0
>
> Abiel Reinhart
>
> Original Message
> From: Gabor Grothendieck [mailto: [hidden email]]
> Sent: Wednesday, May 12, 2010 10:40 AM
> To: Abiel X Reinhart
> Cc: [hidden email]
> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>
> Use rule = 2 as in the extrapolation examples in the na.approx help file.
>
> On Wed, May 12, 2010 at 10:10 AM, Abiel X Reinhart
> < [hidden email]> wrote:
>> Gabor,
>>
>> This comes close to solving my problem, but I am still left with the problem of how I can extrapolate, not just interpolate. In our example, if I define m as,
>>
>> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>>
>> instead of
>>
>> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>>
>> then I will only get five values back, when I really want m[,1] to be fully extrapolated so that there are seven values. Is there a workaround?
>>
>> Abiel Reinhart
>>
>> Original Message
>> From: Gabor Grothendieck [mailto: [hidden email]]
>> Sent: Wednesday, May 12, 2010 8:37 AM
>> To: Abiel X Reinhart
>> Cc: [hidden email]
>> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>>
>> Try this using the zoo package. See ?na.approx for more and note that
>> this functionality requires zoo 1.63 or later.
>>
>> .> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>>> na.approx(m[, 1], x = m[, 2])
>> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
>> 1.000000 2.000000 2.714286 3.714286 5.000000 5.916667 7.000000
>>> na.approx(m[, 1])
>> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
>> 1 2 3 4 5 6 7
>>
>>
>> On Tue, May 11, 2010 at 4:48 PM, Abiel X Reinhart
>> < [hidden email]> wrote:
>>> I have two identically sized matrices of data that represent time series (I am storing the data in zoo objects, but the idea should apply to any matrix of data). The time series in the second matrix extend further than in the first matrix, and I would like to use the data in matrix 2 to extrapolate the data in matrix 1. In other words, if mat1[i,j] == NA, then mat1[i,j] < mat1[i1, j]*mat2[i,j]/mat2[i1,j]. Of course, before we can calculate mat1[i,j] we may need to calculate mat1[i1,j], and that in turn may require the computation of mat1[i2,j], etc. This could all clearly be done with loops, but I am wondering if anyone can think of a vectorized expression or other efficient method that would work.
>>>
>>> Thanks very much.
>>>
>>> Abiel Reinhart
>>> This communication is for informational purposes only. It is not
>>> intended as an offer or solicitation for the purchase or sale of
>>> any financial instrument or as an official confirmation of any
>>> transaction. All market prices, data and other information are not
>>> warranted as to completeness or accuracy and are subject to change
>>> without notice. Any comments or statements made herein do not
>>> necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
>>> and affiliates.
>>>
>>> This transmission may contain information that is privileged,
>>> confidential, legally privileged, and/or exempt from disclosure
>>> under applicable law. If you are not the intended recipient, you
>>> are hereby notified that any disclosure, copying, distribution, or
>>> use of the information contained herein (including any reliance
>>> thereon) is STRICTLY PROHIBITED. Although this transmission and any
>>> attachments are believed to be free of any virus or other defect
>>> that might affect any computer system into which it is received and
>>> opened, it is the responsibility of the recipient to ensure that it
>>> is virus free and no responsibility is accepted by JPMorgan Chase &
>>> Co., its subsidiaries and affiliates, as applicable, for any loss
>>> or damage arising in any way from its use. If you received this
>>> transmission in error, please immediately contact the sender and
>>> destroy the material in its entirety, whether in electronic or hard
>>> copy format. Thank you.
>>>
>>> Please refer to http://www.jpmorgan.com/pages/disclosures for
>>> disclosures relating to European legal entities.
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/rhelp>>> PLEASE do read the posting guide http://www.Rproject.org/postingguide.html>>> and provide commented, minimal, selfcontained, reproducible code.
>>>
>>
>
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/rhelpPLEASE do read the posting guide http://www.Rproject.org/postingguide.htmland provide commented, minimal, selfcontained, reproducible code.


Thanks Gabor, this looks like it serves my needs. I've extended the code to work with an example where we have two multicolumn zoo objects, one with the original data and another that has the growth rates.
# mat1 = zoo object to extend
# mat2 = zoo object whose growth rate is used to extend mat1
mergeGrowth < function(mat1, mat2)
{
ix < is.na(mat1)
mat1.locf < na.locf(mat1, na.rm=F)
mat2.locf < mat2
mat2.locf[ix] < NA
mat2.locf < na.locf(mat2.locf, na.rm=F)
coredata(mat1)[ix] < coredata(mat1.locf * mat2 / mat2.locf)[ix]
mat1
}
Abiel Reinhart
Original Message
From: Gabor Grothendieck [mailto: [hidden email]]
Sent: Wednesday, May 12, 2010 12:00 PM
To: Abiel X Reinhart
Cc: [hidden email]
Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
Yes, that is what it does. Note that na.approx interpolates and does
not work precisely as you discussed but its easy, does use m[,2] and
may be good enough. If you really do want something precisely as you
discussed try this. It NAs out the rows of m for which column 1 is NA
and then uses na.locf to move the prior nonNA into it. Then we apply
the formula:
> library(zoo)
> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>
> # mm will hold result; m.locf
> m.locf < mm < m
> ix < is.na(mm[,1])
> m.locf[ix,] < NA
> m.locf < na.locf(m.locf)
> mm[ix, 1] < m.locf[ix, 1] * mm[ix,2] / m.locf[ix,2]
> mm
19700102 1.0 1
19700103 2.0 4
19700104 4.5 9
19700105 8.0 16
19700106 5.0 25
19700107 7.2 36
19700108 9.8 49
19700102 1.0 1
19700103 2.0 4
19700104 4.5 9
19700105 8.0 16
19700106 5.0 25
19700107 7.2 36
19700108 9.8 49
Yes, that is what it does. Please read the help file for na.approx and
approx. If you want something different you will have to special case
the end values.
On Wed, May 12, 2010 at 11:11 AM, Abiel X Reinhart
< [hidden email]> wrote:
> Gabor,
>
> Maybe I am doing this wrong, but rule=2 does not look like it is growing the series out, but rather just carrying the last value forward. It looks like na.approx() followed by na.locf(). For instance:
>
> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
> na.approx(m[, 1], x = m[, 2], rule=2)
>
> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
> 1.0 2.0 2.7 3.7 5.0 5.0 5.0
>
> Abiel Reinhart
>
> Original Message
> From: Gabor Grothendieck [mailto: [hidden email]]
> Sent: Wednesday, May 12, 2010 10:40 AM
> To: Abiel X Reinhart
> Cc: [hidden email]
> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>
> Use rule = 2 as in the extrapolation examples in the na.approx help file.
>
> On Wed, May 12, 2010 at 10:10 AM, Abiel X Reinhart
> < [hidden email]> wrote:
>> Gabor,
>>
>> This comes close to solving my problem, but I am still left with the problem of how I can extrapolate, not just interpolate. In our example, if I define m as,
>>
>> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>>
>> instead of
>>
>> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>>
>> then I will only get five values back, when I really want m[,1] to be fully extrapolated so that there are seven values. Is there a workaround?
>>
>> Abiel Reinhart
>>
>> Original Message
>> From: Gabor Grothendieck [mailto: [hidden email]]
>> Sent: Wednesday, May 12, 2010 8:37 AM
>> To: Abiel X Reinhart
>> Cc: [hidden email]
>> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>>
>> Try this using the zoo package. See ?na.approx for more and note that
>> this functionality requires zoo 1.63 or later.
>>
>> .> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>>> na.approx(m[, 1], x = m[, 2])
>> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
>> 1.000000 2.000000 2.714286 3.714286 5.000000 5.916667 7.000000
>>> na.approx(m[, 1])
>> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
>> 1 2 3 4 5 6 7
>>
>>
>> On Tue, May 11, 2010 at 4:48 PM, Abiel X Reinhart
>> < [hidden email]> wrote:
>>> I have two identically sized matrices of data that represent time series (I am storing the data in zoo objects, but the idea should apply to any matrix of data). The time series in the second matrix extend further than in the first matrix, and I would like to use the data in matrix 2 to extrapolate the data in matrix 1. In other words, if mat1[i,j] == NA, then mat1[i,j] < mat1[i1, j]*mat2[i,j]/mat2[i1,j]. Of course, before we can calculate mat1[i,j] we may need to calculate mat1[i1,j], and that in turn may require the computation of mat1[i2,j], etc. This could all clearly be done with loops, but I am wondering if anyone can think of a vectorized expression or other efficient method that would work.
>>>
>>> Thanks very much.
>>>
>>> Abiel Reinhart
>>> This communication is for informational purposes only. It is not
>>> intended as an offer or solicitation for the purchase or sale of
>>> any financial instrument or as an official confirmation of any
>>> transaction. All market prices, data and other information are not
>>> warranted as to completeness or accuracy and are subject to change
>>> without notice. Any comments or statements made herein do not
>>> necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
>>> and affiliates.
>>>
>>> This transmission may contain information that is privileged,
>>> confidential, legally privileged, and/or exempt from disclosure
>>> under applicable law. If you are not the intended recipient, you
>>> are hereby notified that any disclosure, copying, distribution, or
>>> use of the information contained herein (including any reliance
>>> thereon) is STRICTLY PROHIBITED. Although this transmission and any
>>> attachments are believed to be free of any virus or other defect
>>> that might affect any computer system into which it is received and
>>> opened, it is the responsibility of the recipient to ensure that it
>>> is virus free and no responsibility is accepted by JPMorgan Chase &
>>> Co., its subsidiaries and affiliates, as applicable, for any loss
>>> or damage arising in any way from its use. If you received this
>>> transmission in error, please immediately contact the sender and
>>> destroy the material in its entirety, whether in electronic or hard
>>> copy format. Thank you.
>>>
>>> Please refer to http://www.jpmorgan.com/pages/disclosures for
>>> disclosures relating to European legal entities.
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/rhelp>>> PLEASE do read the posting guide http://www.Rproject.org/postingguide.html>>> and provide commented, minimal, selfcontained, reproducible code.
>>>
>>
>
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/rhelpPLEASE do read the posting guide http://www.Rproject.org/postingguide.htmland provide commented, minimal, selfcontained, reproducible code.


I have managed to reduce it to two statements using the trick that the
elements of xx are NA or 0 according to whether the corresponding
element of x is NA or not. You could probably extend this to your
situation too.
> library(zoo)
> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>
> r < replace(m[,1], TRUE,
+ na.locf(m[,1]) * m[,2] / na.locf(m[,2] + (m[,1]m[,1])))
> cbind(V1 = r, V2 = m[,2])
V1 V2
19700102 1.0 1
19700103 2.0 4
19700104 4.5 9
19700105 8.0 16
19700106 5.0 25
19700107 7.2 36
19700108 9.8 49
On Wed, May 12, 2010 at 4:58 PM, Abiel X Reinhart
< [hidden email]> wrote:
> Thanks Gabor, this looks like it serves my needs. I've extended the code to work with an example where we have two multicolumn zoo objects, one with the original data and another that has the growth rates.
>
> # mat1 = zoo object to extend
> # mat2 = zoo object whose growth rate is used to extend mat1
> mergeGrowth < function(mat1, mat2)
> {
> ix < is.na(mat1)
> mat1.locf < na.locf(mat1, na.rm=F)
> mat2.locf < mat2
> mat2.locf[ix] < NA
> mat2.locf < na.locf(mat2.locf, na.rm=F)
> coredata(mat1)[ix] < coredata(mat1.locf * mat2 / mat2.locf)[ix]
> mat1
> }
>
>
> Abiel Reinhart
>
> Original Message
> From: Gabor Grothendieck [mailto: [hidden email]]
> Sent: Wednesday, May 12, 2010 12:00 PM
> To: Abiel X Reinhart
> Cc: [hidden email]
> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>
> Yes, that is what it does. Note that na.approx interpolates and does
> not work precisely as you discussed but its easy, does use m[,2] and
> may be good enough. If you really do want something precisely as you
> discussed try this. It NAs out the rows of m for which column 1 is NA
> and then uses na.locf to move the prior nonNA into it. Then we apply
> the formula:
>
>> library(zoo)
>> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>>
>> # mm will hold result; m.locf
>> m.locf < mm < m
>> ix < is.na(mm[,1])
>> m.locf[ix,] < NA
>> m.locf < na.locf(m.locf)
>> mm[ix, 1] < m.locf[ix, 1] * mm[ix,2] / m.locf[ix,2]
>> mm
>
> 19700102 1.0 1
> 19700103 2.0 4
> 19700104 4.5 9
> 19700105 8.0 16
> 19700106 5.0 25
> 19700107 7.2 36
> 19700108 9.8 49
>
> 19700102 1.0 1
> 19700103 2.0 4
> 19700104 4.5 9
> 19700105 8.0 16
> 19700106 5.0 25
> 19700107 7.2 36
> 19700108 9.8 49
>
>
>
>
>
> Yes, that is what it does. Please read the help file for na.approx and
> approx. If you want something different you will have to special case
> the end values.
>
> On Wed, May 12, 2010 at 11:11 AM, Abiel X Reinhart
> < [hidden email]> wrote:
>> Gabor,
>>
>> Maybe I am doing this wrong, but rule=2 does not look like it is growing the series out, but rather just carrying the last value forward. It looks like na.approx() followed by na.locf(). For instance:
>>
>> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>> na.approx(m[, 1], x = m[, 2], rule=2)
>>
>> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
>> 1.0 2.0 2.7 3.7 5.0 5.0 5.0
>>
>> Abiel Reinhart
>>
>> Original Message
>> From: Gabor Grothendieck [mailto: [hidden email]]
>> Sent: Wednesday, May 12, 2010 10:40 AM
>> To: Abiel X Reinhart
>> Cc: [hidden email]
>> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>>
>> Use rule = 2 as in the extrapolation examples in the na.approx help file.
>>
>> On Wed, May 12, 2010 at 10:10 AM, Abiel X Reinhart
>> < [hidden email]> wrote:
>>> Gabor,
>>>
>>> This comes close to solving my problem, but I am still left with the problem of how I can extrapolate, not just interpolate. In our example, if I define m as,
>>>
>>> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, NA), seq(7)^2), as.Date(1:7))
>>>
>>> instead of
>>>
>>> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>>>
>>> then I will only get five values back, when I really want m[,1] to be fully extrapolated so that there are seven values. Is there a workaround?
>>>
>>> Abiel Reinhart
>>>
>>> Original Message
>>> From: Gabor Grothendieck [mailto: [hidden email]]
>>> Sent: Wednesday, May 12, 2010 8:37 AM
>>> To: Abiel X Reinhart
>>> Cc: [hidden email]
>>> Subject: Re: [R] Vectorized expression to extrapolate matrix columns with columns of another matrix
>>>
>>> Try this using the zoo package. See ?na.approx for more and note that
>>> this functionality requires zoo 1.63 or later.
>>>
>>> .> m < zoo(cbind(c(1, 2, NA, NA, 5, NA, 7), seq(7)^2), as.Date(1:7))
>>>> na.approx(m[, 1], x = m[, 2])
>>> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
>>> 1.000000 2.000000 2.714286 3.714286 5.000000 5.916667 7.000000
>>>> na.approx(m[, 1])
>>> 19700102 19700103 19700104 19700105 19700106 19700107 19700108
>>> 1 2 3 4 5 6 7
>>>
>>>
>>> On Tue, May 11, 2010 at 4:48 PM, Abiel X Reinhart
>>> < [hidden email]> wrote:
>>>> I have two identically sized matrices of data that represent time series (I am storing the data in zoo objects, but the idea should apply to any matrix of data). The time series in the second matrix extend further than in the first matrix, and I would like to use the data in matrix 2 to extrapolate the data in matrix 1. In other words, if mat1[i,j] == NA, then mat1[i,j] < mat1[i1, j]*mat2[i,j]/mat2[i1,j]. Of course, before we can calculate mat1[i,j] we may need to calculate mat1[i1,j], and that in turn may require the computation of mat1[i2,j], etc. This could all clearly be done with loops, but I am wondering if anyone can think of a vectorized expression or other efficient method that would work.
>>>>
>>>> Thanks very much.
>>>>
>>>> Abiel Reinhart
>>>> This communication is for informational purposes only. It is not
>>>> intended as an offer or solicitation for the purchase or sale of
>>>> any financial instrument or as an official confirmation of any
>>>> transaction. All market prices, data and other information are not
>>>> warranted as to completeness or accuracy and are subject to change
>>>> without notice. Any comments or statements made herein do not
>>>> necessarily reflect those of JPMorgan Chase & Co., its subsidiaries
>>>> and affiliates.
>>>>
>>>> This transmission may contain information that is privileged,
>>>> confidential, legally privileged, and/or exempt from disclosure
>>>> under applicable law. If you are not the intended recipient, you
>>>> are hereby notified that any disclosure, copying, distribution, or
>>>> use of the information contained herein (including any reliance
>>>> thereon) is STRICTLY PROHIBITED. Although this transmission and any
>>>> attachments are believed to be free of any virus or other defect
>>>> that might affect any computer system into which it is received and
>>>> opened, it is the responsibility of the recipient to ensure that it
>>>> is virus free and no responsibility is accepted by JPMorgan Chase &
>>>> Co., its subsidiaries and affiliates, as applicable, for any loss
>>>> or damage arising in any way from its use. If you received this
>>>> transmission in error, please immediately contact the sender and
>>>> destroy the material in its entirety, whether in electronic or hard
>>>> copy format. Thank you.
>>>>
>>>> Please refer to http://www.jpmorgan.com/pages/disclosures for
>>>> disclosures relating to European legal entities.
>>>>
>>>> ______________________________________________
>>>> [hidden email] mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/rhelp>>>> PLEASE do read the posting guide http://www.Rproject.org/postingguide.html>>>> and provide commented, minimal, selfcontained, reproducible code.
>>>>
>>>
>>
>
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/rhelpPLEASE do read the posting guide http://www.Rproject.org/postingguide.htmland provide commented, minimal, selfcontained, reproducible code.

