precision of data download in rbbg/rbloomberg

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

precision of data download in rbbg/rbloomberg

Aidan Corcoran
hi folks,

I was wondering if it is possible to limit the precision of the data being
downloaded via the bdh function in rbbg, in the interest of speed. I
haven't seen any mention of such an option but thought I would ask just in
case I missed something.

thanks in advance
Aidan

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: precision of data download in rbbg/rbloomberg

John Laing
Aidan,

I don't think the Bloomberg API offers such an option. However, I also
don't think that precision is a big cause of slowness. Can you send a
sample query to demonstrate your issue? E.g.:

> system.time(ty1 <- bdh(conn, "TY1 Comdty", "PX_LAST",
start_date=as.Date("1900-01-01"), end_date=Sys.Date()))
   user  system elapsed
  0.428   0.024   2.772
> dim(ty1)
[1] 7795    2


-John


On Fri, Apr 19, 2013 at 3:52 AM, Aidan Corcoran
<[hidden email]>wrote:

> hi folks,
>
> I was wondering if it is possible to limit the precision of the data being
> downloaded via the bdh function in rbbg, in the interest of speed. I
> haven't seen any mention of such an option but thought I would ask just in
> case I missed something.
>
> thanks in advance
> Aidan
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: precision of data download in rbbg/rbloomberg

Aidan Corcoran
hi John,

thanks for your reply. my results were:
 user  system elapsed
   1.00    0.06    6.43

This is using win7 64bit, 12gb of RAM. I get roughly 6mbps download speed
from the internet where I am in case that has any bearing.

The speed is probably not bad, I just wanted to make sure I wasn't missing
an easy option to get a bit more speed.

Thanks for your help!


On Fri, Apr 19, 2013 at 12:04 PM, John Laing <[hidden email]> wrote:

> Aidan,
>
> I don't think the Bloomberg API offers such an option. However, I also
> don't think that precision is a big cause of slowness. Can you send a
> sample query to demonstrate your issue? E.g.:
>
> > system.time(ty1 <- bdh(conn, "TY1 Comdty", "PX_LAST",
> start_date=as.Date("1900-01-01"), end_date=Sys.Date()))
>    user  system elapsed
>   0.428   0.024   2.772
> > dim(ty1)
> [1] 7795    2
>
>
> -John
>
>
> On Fri, Apr 19, 2013 at 3:52 AM, Aidan Corcoran <
> [hidden email]> wrote:
>
>> hi folks,
>>
>> I was wondering if it is possible to limit the precision of the data being
>> downloaded via the bdh function in rbbg, in the interest of speed. I
>> haven't seen any mention of such an option but thought I would ask just in
>> case I missed something.
>>
>> thanks in advance
>> Aidan
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R questions
>> should go.
>>
>
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: precision of data download in rbbg/rbloomberg

Jeffrey Ryan
Aidan,

There is an awful lot of additional processing happening inside of bdh in
terms of data extraction/manipulation. In the past I've extracted out the
important .java type calls and return types to get substantial speed-ups,
especially when dealing with a common/more constrained search (e.g. one
name at a time)

Of course, YMMV

Best,
Jeff


On Fri, Apr 19, 2013 at 7:34 AM, Aidan Corcoran
<[hidden email]>wrote:

> hi John,
>
> thanks for your reply. my results were:
>  user  system elapsed
>    1.00    0.06    6.43
>
> This is using win7 64bit, 12gb of RAM. I get roughly 6mbps download speed
> from the internet where I am in case that has any bearing.
>
> The speed is probably not bad, I just wanted to make sure I wasn't missing
> an easy option to get a bit more speed.
>
> Thanks for your help!
>
>
> On Fri, Apr 19, 2013 at 12:04 PM, John Laing <[hidden email]> wrote:
>
> > Aidan,
> >
> > I don't think the Bloomberg API offers such an option. However, I also
> > don't think that precision is a big cause of slowness. Can you send a
> > sample query to demonstrate your issue? E.g.:
> >
> > > system.time(ty1 <- bdh(conn, "TY1 Comdty", "PX_LAST",
> > start_date=as.Date("1900-01-01"), end_date=Sys.Date()))
> >    user  system elapsed
> >   0.428   0.024   2.772
> > > dim(ty1)
> > [1] 7795    2
> >
> >
> > -John
> >
> >
> > On Fri, Apr 19, 2013 at 3:52 AM, Aidan Corcoran <
> > [hidden email]> wrote:
> >
> >> hi folks,
> >>
> >> I was wondering if it is possible to limit the precision of the data
> being
> >> downloaded via the bdh function in rbbg, in the interest of speed. I
> >> haven't seen any mention of such an option but thought I would ask just
> in
> >> case I missed something.
> >>
> >> thanks in advance
> >> Aidan
> >>
> >>         [[alternative HTML version deleted]]
> >>
> >> _______________________________________________
> >> [hidden email] mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> >> -- Subscriber-posting only. If you want to post, subscribe first.
> >> -- Also note that this is not the r-help list where general R questions
> >> should go.
> >>
> >
> >
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>



--
Jeffrey Ryan
[hidden email]

www.lemnica.com

R/Finance 2013: Applied R in Finance
May 17, 18 Chicago, IL
www.RinFinance.com

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: precision of data download in rbbg/rbloomberg

John Laing
Jeff is right, Rbbg's result processing is not, shall we say, optimized.
Whit Armstrong has been working on a more lightweight C++-based interface
to Bloomberg that is also more graceful with its data type handling. I
don't think he's released it publicly yet.

But in this case his version is roughly the same speed as Rbbg. I think the
slowness is on Bloomberg's side.

-John


On Fri, Apr 19, 2013 at 8:52 AM, Jeff Ryan <[hidden email]> wrote:

> Aidan,
>
> There is an awful lot of additional processing happening inside of bdh in
> terms of data extraction/manipulation. In the past I've extracted out the
> important .java type calls and return types to get substantial speed-ups,
> especially when dealing with a common/more constrained search (e.g. one
> name at a time)
>
> Of course, YMMV
>
> Best,
> Jeff
>
>
> On Fri, Apr 19, 2013 at 7:34 AM, Aidan Corcoran
> <[hidden email]>wrote:
>
> > hi John,
> >
> > thanks for your reply. my results were:
> >  user  system elapsed
> >    1.00    0.06    6.43
> >
> > This is using win7 64bit, 12gb of RAM. I get roughly 6mbps download speed
> > from the internet where I am in case that has any bearing.
> >
> > The speed is probably not bad, I just wanted to make sure I wasn't
> missing
> > an easy option to get a bit more speed.
> >
> > Thanks for your help!
> >
> >
> > On Fri, Apr 19, 2013 at 12:04 PM, John Laing <[hidden email]>
> wrote:
> >
> > > Aidan,
> > >
> > > I don't think the Bloomberg API offers such an option. However, I also
> > > don't think that precision is a big cause of slowness. Can you send a
> > > sample query to demonstrate your issue? E.g.:
> > >
> > > > system.time(ty1 <- bdh(conn, "TY1 Comdty", "PX_LAST",
> > > start_date=as.Date("1900-01-01"), end_date=Sys.Date()))
> > >    user  system elapsed
> > >   0.428   0.024   2.772
> > > > dim(ty1)
> > > [1] 7795    2
> > >
> > >
> > > -John
> > >
> > >
> > > On Fri, Apr 19, 2013 at 3:52 AM, Aidan Corcoran <
> > > [hidden email]> wrote:
> > >
> > >> hi folks,
> > >>
> > >> I was wondering if it is possible to limit the precision of the data
> > being
> > >> downloaded via the bdh function in rbbg, in the interest of speed. I
> > >> haven't seen any mention of such an option but thought I would ask
> just
> > in
> > >> case I missed something.
> > >>
> > >> thanks in advance
> > >> Aidan
> > >>
> > >>         [[alternative HTML version deleted]]
> > >>
> > >> _______________________________________________
> > >> [hidden email] mailing list
> > >> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > >> -- Subscriber-posting only. If you want to post, subscribe first.
> > >> -- Also note that this is not the r-help list where general R
> questions
> > >> should go.
> > >>
> > >
> > >
> >
> >         [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R questions
> > should go.
> >
>
>
>
> --
> Jeffrey Ryan
> [hidden email]
>
> www.lemnica.com
>
> R/Finance 2013: Applied R in Finance
> May 17, 18 Chicago, IL
> www.RinFinance.com
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: precision of data download in rbbg/rbloomberg

Whit Armstrong-3
So, as JL was saying, in this case, implementation speed isn't the issue.

Before someone makes a fuss about re-inventing the wheel...

The thinking behind the c++ version is that as much as I love Java (ha
ha), we need to use this in ruby, so the c++ bit will eventually be
generalized into something independent from this R package.

One of the bigger issues to deal with is a c++ implementation of
DataFrames.  It's a bit of a pain in the ass.

Anyway, the link is here if anyone wants to experiment.  I've set this
up to compile on linux, but I haven't done anything about Windows, so
you're on your own if you want that to work.

https://github.com/armstrtw/Rblpapi

-Whit

On Fri, Apr 19, 2013 at 9:05 AM, John Laing <[hidden email]> wrote:

> Jeff is right, Rbbg's result processing is not, shall we say, optimized.
> Whit Armstrong has been working on a more lightweight C++-based interface
> to Bloomberg that is also more graceful with its data type handling. I
> don't think he's released it publicly yet.
>
> But in this case his version is roughly the same speed as Rbbg. I think the
> slowness is on Bloomberg's side.
>
> -John
>
>
> On Fri, Apr 19, 2013 at 8:52 AM, Jeff Ryan <[hidden email]> wrote:
>
>> Aidan,
>>
>> There is an awful lot of additional processing happening inside of bdh in
>> terms of data extraction/manipulation. In the past I've extracted out the
>> important .java type calls and return types to get substantial speed-ups,
>> especially when dealing with a common/more constrained search (e.g. one
>> name at a time)
>>
>> Of course, YMMV
>>
>> Best,
>> Jeff
>>
>>
>> On Fri, Apr 19, 2013 at 7:34 AM, Aidan Corcoran
>> <[hidden email]>wrote:
>>
>> > hi John,
>> >
>> > thanks for your reply. my results were:
>> >  user  system elapsed
>> >    1.00    0.06    6.43
>> >
>> > This is using win7 64bit, 12gb of RAM. I get roughly 6mbps download speed
>> > from the internet where I am in case that has any bearing.
>> >
>> > The speed is probably not bad, I just wanted to make sure I wasn't
>> missing
>> > an easy option to get a bit more speed.
>> >
>> > Thanks for your help!
>> >
>> >
>> > On Fri, Apr 19, 2013 at 12:04 PM, John Laing <[hidden email]>
>> wrote:
>> >
>> > > Aidan,
>> > >
>> > > I don't think the Bloomberg API offers such an option. However, I also
>> > > don't think that precision is a big cause of slowness. Can you send a
>> > > sample query to demonstrate your issue? E.g.:
>> > >
>> > > > system.time(ty1 <- bdh(conn, "TY1 Comdty", "PX_LAST",
>> > > start_date=as.Date("1900-01-01"), end_date=Sys.Date()))
>> > >    user  system elapsed
>> > >   0.428   0.024   2.772
>> > > > dim(ty1)
>> > > [1] 7795    2
>> > >
>> > >
>> > > -John
>> > >
>> > >
>> > > On Fri, Apr 19, 2013 at 3:52 AM, Aidan Corcoran <
>> > > [hidden email]> wrote:
>> > >
>> > >> hi folks,
>> > >>
>> > >> I was wondering if it is possible to limit the precision of the data
>> > being
>> > >> downloaded via the bdh function in rbbg, in the interest of speed. I
>> > >> haven't seen any mention of such an option but thought I would ask
>> just
>> > in
>> > >> case I missed something.
>> > >>
>> > >> thanks in advance
>> > >> Aidan
>> > >>
>> > >>         [[alternative HTML version deleted]]
>> > >>
>> > >> _______________________________________________
>> > >> [hidden email] mailing list
>> > >> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> > >> -- Subscriber-posting only. If you want to post, subscribe first.
>> > >> -- Also note that this is not the r-help list where general R
>> questions
>> > >> should go.
>> > >>
>> > >
>> > >
>> >
>> >         [[alternative HTML version deleted]]
>> >
>> > _______________________________________________
>> > [hidden email] mailing list
>> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> > -- Subscriber-posting only. If you want to post, subscribe first.
>> > -- Also note that this is not the r-help list where general R questions
>> > should go.
>> >
>>
>>
>>
>> --
>> Jeffrey Ryan
>> [hidden email]
>>
>> www.lemnica.com
>>
>> R/Finance 2013: Applied R in Finance
>> May 17, 18 Chicago, IL
>> www.RinFinance.com
>>
>>         [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R questions
>> should go.
>>
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: precision of data download in rbbg/rbloomberg

Aidan Corcoran
thanks for all the info guys.
after reading your replies bdh doesn't seem so slow after all!
:)


On Fri, Apr 19, 2013 at 2:35 PM, Whit Armstrong <[hidden email]>wrote:

> So, as JL was saying, in this case, implementation speed isn't the issue.
>
> Before someone makes a fuss about re-inventing the wheel...
>
> The thinking behind the c++ version is that as much as I love Java (ha
> ha), we need to use this in ruby, so the c++ bit will eventually be
> generalized into something independent from this R package.
>
> One of the bigger issues to deal with is a c++ implementation of
> DataFrames.  It's a bit of a pain in the ass.
>
> Anyway, the link is here if anyone wants to experiment.  I've set this
> up to compile on linux, but I haven't done anything about Windows, so
> you're on your own if you want that to work.
>
> https://github.com/armstrtw/Rblpapi
>
> -Whit
>
> On Fri, Apr 19, 2013 at 9:05 AM, John Laing <[hidden email]> wrote:
> > Jeff is right, Rbbg's result processing is not, shall we say, optimized.
> > Whit Armstrong has been working on a more lightweight C++-based interface
> > to Bloomberg that is also more graceful with its data type handling. I
> > don't think he's released it publicly yet.
> >
> > But in this case his version is roughly the same speed as Rbbg. I think
> the
> > slowness is on Bloomberg's side.
> >
> > -John
> >
> >
> > On Fri, Apr 19, 2013 at 8:52 AM, Jeff Ryan <[hidden email]>
> wrote:
> >
> >> Aidan,
> >>
> >> There is an awful lot of additional processing happening inside of bdh
> in
> >> terms of data extraction/manipulation. In the past I've extracted out
> the
> >> important .java type calls and return types to get substantial
> speed-ups,
> >> especially when dealing with a common/more constrained search (e.g. one
> >> name at a time)
> >>
> >> Of course, YMMV
> >>
> >> Best,
> >> Jeff
> >>
> >>
> >> On Fri, Apr 19, 2013 at 7:34 AM, Aidan Corcoran
> >> <[hidden email]>wrote:
> >>
> >> > hi John,
> >> >
> >> > thanks for your reply. my results were:
> >> >  user  system elapsed
> >> >    1.00    0.06    6.43
> >> >
> >> > This is using win7 64bit, 12gb of RAM. I get roughly 6mbps download
> speed
> >> > from the internet where I am in case that has any bearing.
> >> >
> >> > The speed is probably not bad, I just wanted to make sure I wasn't
> >> missing
> >> > an easy option to get a bit more speed.
> >> >
> >> > Thanks for your help!
> >> >
> >> >
> >> > On Fri, Apr 19, 2013 at 12:04 PM, John Laing <[hidden email]>
> >> wrote:
> >> >
> >> > > Aidan,
> >> > >
> >> > > I don't think the Bloomberg API offers such an option. However, I
> also
> >> > > don't think that precision is a big cause of slowness. Can you send
> a
> >> > > sample query to demonstrate your issue? E.g.:
> >> > >
> >> > > > system.time(ty1 <- bdh(conn, "TY1 Comdty", "PX_LAST",
> >> > > start_date=as.Date("1900-01-01"), end_date=Sys.Date()))
> >> > >    user  system elapsed
> >> > >   0.428   0.024   2.772
> >> > > > dim(ty1)
> >> > > [1] 7795    2
> >> > >
> >> > >
> >> > > -John
> >> > >
> >> > >
> >> > > On Fri, Apr 19, 2013 at 3:52 AM, Aidan Corcoran <
> >> > > [hidden email]> wrote:
> >> > >
> >> > >> hi folks,
> >> > >>
> >> > >> I was wondering if it is possible to limit the precision of the
> data
> >> > being
> >> > >> downloaded via the bdh function in rbbg, in the interest of speed.
> I
> >> > >> haven't seen any mention of such an option but thought I would ask
> >> just
> >> > in
> >> > >> case I missed something.
> >> > >>
> >> > >> thanks in advance
> >> > >> Aidan
> >> > >>
> >> > >>         [[alternative HTML version deleted]]
> >> > >>
> >> > >> _______________________________________________
> >> > >> [hidden email] mailing list
> >> > >> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> >> > >> -- Subscriber-posting only. If you want to post, subscribe first.
> >> > >> -- Also note that this is not the r-help list where general R
> >> questions
> >> > >> should go.
> >> > >>
> >> > >
> >> > >
> >> >
> >> >         [[alternative HTML version deleted]]
> >> >
> >> > _______________________________________________
> >> > [hidden email] mailing list
> >> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> >> > -- Subscriber-posting only. If you want to post, subscribe first.
> >> > -- Also note that this is not the r-help list where general R
> questions
> >> > should go.
> >> >
> >>
> >>
> >>
> >> --
> >> Jeffrey Ryan
> >> [hidden email]
> >>
> >> www.lemnica.com
> >>
> >> R/Finance 2013: Applied R in Finance
> >> May 17, 18 Chicago, IL
> >> www.RinFinance.com
> >>
> >>         [[alternative HTML version deleted]]
> >>
> >> _______________________________________________
> >> [hidden email] mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> >> -- Subscriber-posting only. If you want to post, subscribe first.
> >> -- Also note that this is not the r-help list where general R questions
> >> should go.
> >>
> >
> >         [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R questions
> should go.
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.