Using attachSymbols with non-default argument values (Was: the ZigZag function)

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Using attachSymbols with non-default argument values (Was: the ZigZag function)

gsee
Gordon,

The way I did it was by storing other arguments/values in the DDB object.

I did this for other getSymbols methods, but not for getSymbols.yahoo.
 Since you asked, I mashed together some code that will do what you
ask.  (see attached).

For this solution, I replaced create.binding, attachSymbols, and
DDB_Yahoo.  I don't know that all of that is necessary.  Anyway, if
you source the attached script, then you can do this.

> attachSymbols2(DB=DDB_Yahoo2(from=Sys.Date()-150))
> head(DIA)
DIA not in memory. fetching....
           DIA.Open DIA.High DIA.Low DIA.Close DIA.Volume DIA.Adjusted
2011-11-28   114.78   115.48  114.31    115.10    6666100       113.96
2011-11-29   115.32   116.18  115.11    115.56    5325400       114.41
2011-11-30   118.41   120.39  118.41    120.19   11581500       119.00
2011-12-01   120.01   120.55  119.66    120.13    5437700       118.94
2011-12-02   121.22   121.40  120.00    120.06    5584400       118.87
2011-12-05   121.77   121.85  120.21    120.95    6285400       119.75


Best,
Garrett

On Thu, Apr 26, 2012 at 3:55 PM, Gordon Erlebacher
<[hidden email]> wrote:

> Hi,
>
> I'd like to use attachSymbols() to create an on-demand database. I have two
> questions:
>
> 1) I'd like to restrict the stocks to the past 150 days. I found that
> quantmod:::create.binding  uses the function "getSymbols", which allows a
> range. But how to change the function quantmod:::create.binding to use a
> version of getSymbols() with a different date range? Alternatively, how to
> replace "quantmod:::create.binding" with my own version such that any
> function in quantmod that uses create.binding will use my version. Thanks.
>
> Or there is an easier way to restrict the range of symbols retrieved on
> demand.
>
> Thanks!
>
>    Gordon
>
>        [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

attachSymbolsWithArgs.R (7K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Using attachSymbols with non-default argument values (Was: the ZigZag function)

Gordon Erlebacher
Thanks Garrett. I thought about this solution on my way home from work.
Glad I did not have to do it. So here is a harder problem, but one you
might have solved, or might need to solve.

I wish to keep the last 150 days of 7000 stocks on disk. Every day, I would
like to run an update that removes the earliest entry of every stock, and
stores the latest entry of every stock. So cache size remains constant, but
contents change.
Any ideas? Am I clear?

   thanks,

    Gordon

On Thu, Apr 26, 2012 at 5:49 PM, G See <[hidden email]> wrote:

> Gordon,
>
> The way I did it was by storing other arguments/values in the DDB object.
>
> I did this for other getSymbols methods, but not for getSymbols.yahoo.
>  Since you asked, I mashed together some code that will do what you
> ask.  (see attached).
>
> For this solution, I replaced create.binding, attachSymbols, and
> DDB_Yahoo.  I don't know that all of that is necessary.  Anyway, if
> you source the attached script, then you can do this.
>
> > attachSymbols2(DB=DDB_Yahoo2(from=Sys.Date()-150))
> > head(DIA)
> DIA not in memory. fetching....
>           DIA.Open DIA.High DIA.Low DIA.Close DIA.Volume DIA.Adjusted
> 2011-11-28   114.78   115.48  114.31    115.10    6666100       113.96
> 2011-11-29   115.32   116.18  115.11    115.56    5325400       114.41
> 2011-11-30   118.41   120.39  118.41    120.19   11581500       119.00
> 2011-12-01   120.01   120.55  119.66    120.13    5437700       118.94
> 2011-12-02   121.22   121.40  120.00    120.06    5584400       118.87
> 2011-12-05   121.77   121.85  120.21    120.95    6285400       119.75
>
>
> Best,
> Garrett
>
> On Thu, Apr 26, 2012 at 3:55 PM, Gordon Erlebacher
> <[hidden email]> wrote:
> > Hi,
> >
> > I'd like to use attachSymbols() to create an on-demand database. I have
> two
> > questions:
> >
> > 1) I'd like to restrict the stocks to the past 150 days. I found that
> > quantmod:::create.binding  uses the function "getSymbols", which allows a
> > range. But how to change the function quantmod:::create.binding to use a
> > version of getSymbols() with a different date range? Alternatively, how
> to
> > replace "quantmod:::create.binding" with my own version such that any
> > function in quantmod that uses create.binding will use my version.
> Thanks.
> >
> > Or there is an easier way to restrict the range of symbols retrieved on
> > demand.
> >
> > Thanks!
> >
> >    Gordon
> >
> >        [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R questions
> should go.
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Using attachSymbols with non-default argument values (Was: the ZigZag function)

braverock
On Thu, 2012-04-26 at 18:22 -0400, Gordon Erlebacher wrote:
> I wish to keep the last 150 days of 7000 stocks on disk. Every day, I
> would like to run an update that removes the earliest entry of every
> stock, and stores the latest entry of every stock. So cache size
> remains constant, but contents change.
> Any ideas? Am I clear?

Disk is cheap, and getSymbols supports from and to dates.

Buy more disk.


--
Brian G. Peterson
http://braverock.com/brian/
Ph: 773-459-4973
IM: bgpbraverock

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Using attachSymbols with non-default argument values (Was: the ZigZag function)

gsee
In reply to this post by Gordon Erlebacher
I suggest re-downloading all 150 days each day instead of downloading
1 day, and deleting a day.

I doubt it will take any longer to download 150 days from yahoo than
it would to download 1 day.  To download 7,000 symbols, you'll most
likely have to add a Sys.sleep call between your calls to getSymbols.
You'll also probably want to wrap those calls in a while loop that
retries a number of times if there were errors downloading.  I don't
think you want to use attachSymbols for this.

Also, to delete the first day, you have to load all data from disk first.

Garrett

On Thu, Apr 26, 2012 at 5:22 PM, Gordon Erlebacher
<[hidden email]> wrote:

> Thanks Garrett. I thought about this solution on my way home from work. Glad
> I did not have to do it. So here is a harder problem, but one you might have
> solved, or might need to solve.
>
> I wish to keep the last 150 days of 7000 stocks on disk. Every day, I would
> like to run an update that removes the earliest entry of every stock, and
> stores the latest entry of every stock. So cache size remains constant, but
> contents change.
> Any ideas? Am I clear?
>
>    thanks,
>
>     Gordon

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Using attachSymbols with non-default argument values (Was: the ZigZag function)

Gordon Erlebacher
In reply to this post by braverock
Hi Brian,

I have huge amounts of disk. The issue is time efficiency. Why download 150
days of 7000 stocks each day, when I need only download 1 day of each
stock. Of course, I can write my own caching system, but why duplicate what
somebody may have done already. I am relatively new to R, but I have done
similar things in other computer languages (not in Finance though).

On an aside, using attachSymbols, I can load on demand. I would like to
cycle through the list of 7000 stocks. I can get the list, but given "AAPL"
for example, I need to execute AAPL. I do not know how to do this yet.

   Thanks for the input!

     Gordon




On Thu, Apr 26, 2012 at 6:39 PM, Brian G. Peterson <[hidden email]>wrote:

> On Thu, 2012-04-26 at 18:22 -0400, Gordon Erlebacher wrote:
> > I wish to keep the last 150 days of 7000 stocks on disk. Every day, I
> > would like to run an update that removes the earliest entry of every
> > stock, and stores the latest entry of every stock. So cache size
> > remains constant, but contents change.
> > Any ideas? Am I clear?
>
> Disk is cheap, and getSymbols supports from and to dates.
>
> Buy more disk.
>
>
> --
> Brian G. Peterson
> http://braverock.com/brian/
> Ph: 773-459-4973
> IM: bgpbraverock
>
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Using attachSymbols with non-default argument values (Was: the ZigZag function)

gsee
get("AAPL")

But, see my last e-mail about not using attachSymbols for this. ;-)

Garrett

On Thu, Apr 26, 2012 at 5:46 PM, Gordon Erlebacher
<[hidden email]> wrote:

> Hi Brian,
>
> I have huge amounts of disk. The issue is time efficiency. Why download 150
> days of 7000 stocks each day, when I need only download 1 day of each
> stock. Of course, I can write my own caching system, but why duplicate what
> somebody may have done already. I am relatively new to R, but I have done
> similar things in other computer languages (not in Finance though).
>
> On an aside, using attachSymbols, I can load on demand. I would like to
> cycle through the list of 7000 stocks. I can get the list, but given "AAPL"
> for example, I need to execute AAPL. I do not know how to do this yet.
>
>   Thanks for the input!
>
>     Gordon
>
>
>
>
> On Thu, Apr 26, 2012 at 6:39 PM, Brian G. Peterson <[hidden email]>wrote:
>
>> On Thu, 2012-04-26 at 18:22 -0400, Gordon Erlebacher wrote:
>> > I wish to keep the last 150 days of 7000 stocks on disk. Every day, I
>> > would like to run an update that removes the earliest entry of every
>> > stock, and stores the latest entry of every stock. So cache size
>> > remains constant, but contents change.
>> > Any ideas? Am I clear?
>>
>> Disk is cheap, and getSymbols supports from and to dates.
>>
>> Buy more disk.
>>
>>
>> --
>> Brian G. Peterson
>> http://braverock.com/brian/
>> Ph: 773-459-4973
>> IM: bgpbraverock
>>
>>
>
>        [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Using attachSymbols with non-default argument values (Was: the ZigZag function)

gsee
I'd do something like this, which is effectively the same as using
attachSymbols with a cache directory. (But I'm pretty sure this will
fail at some point in the data download process because of yahoo 404
errors)

library(FinancialInstrument)
library(quantmod)

s <- quantmod:::getSymbolsDB()
getSymbols(s, src='yahoo', from=Sys.Date()-150)
saveSymbols.common(s, "/path/to/storage/dir", extension='RData')

# Then, to load from disk:
getSymbols(s, src='FI', dir="/path/to/storage/dir/",
split_method='common', extension='RData')

Garrett

On Thu, Apr 26, 2012 at 5:49 PM, G See <[hidden email]> wrote:

> get("AAPL")
>
> But, see my last e-mail about not using attachSymbols for this. ;-)
>
> Garrett
>
> On Thu, Apr 26, 2012 at 5:46 PM, Gordon Erlebacher
> <[hidden email]> wrote:
>> Hi Brian,
>>
>> I have huge amounts of disk. The issue is time efficiency. Why download 150
>> days of 7000 stocks each day, when I need only download 1 day of each
>> stock. Of course, I can write my own caching system, but why duplicate what
>> somebody may have done already. I am relatively new to R, but I have done
>> similar things in other computer languages (not in Finance though).
>>
>> On an aside, using attachSymbols, I can load on demand. I would like to
>> cycle through the list of 7000 stocks. I can get the list, but given "AAPL"
>> for example, I need to execute AAPL. I do not know how to do this yet.
>>
>>   Thanks for the input!
>>
>>     Gordon

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Using attachSymbols with non-default argument values (Was: the ZigZag function)

Patrick Burns-2
In reply to this post by Gordon Erlebacher
One reason to duplicate is adjustments --
the adjusted close changes over time for
the same day.

Pat

On 26/04/2012 23:46, Gordon Erlebacher wrote:

> Hi Brian,
>
> I have huge amounts of disk. The issue is time efficiency. Why download 150
> days of 7000 stocks each day, when I need only download 1 day of each
> stock. Of course, I can write my own caching system, but why duplicate what
> somebody may have done already. I am relatively new to R, but I have done
> similar things in other computer languages (not in Finance though).
>
> On an aside, using attachSymbols, I can load on demand. I would like to
> cycle through the list of 7000 stocks. I can get the list, but given "AAPL"
> for example, I need to execute AAPL. I do not know how to do this yet.
>
>     Thanks for the input!
>
>       Gordon
>
>
>
>
> On Thu, Apr 26, 2012 at 6:39 PM, Brian G. Peterson<[hidden email]>wrote:
>
>> On Thu, 2012-04-26 at 18:22 -0400, Gordon Erlebacher wrote:
>>> I wish to keep the last 150 days of 7000 stocks on disk. Every day, I
>>> would like to run an update that removes the earliest entry of every
>>> stock, and stores the latest entry of every stock. So cache size
>>> remains constant, but contents change.
>>> Any ideas? Am I clear?
>>
>> Disk is cheap, and getSymbols supports from and to dates.
>>
>> Buy more disk.
>>
>>
>> --
>> Brian G. Peterson
>> http://braverock.com/brian/
>> Ph: 773-459-4973
>> IM: bgpbraverock
>>
>>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.
>

--
Patrick Burns
[hidden email]
http://www.burns-stat.com
http://www.portfolioprobe.com/blog
twitter: @portfolioprobe

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.