Reading MetaStock data format in R

classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|

Reading MetaStock data format in R

Peter Fertig
Hello,

I am new at R and need some help because I am stuck in a project. I have financial data, which is stored in the MetaStock data format and is updated daily. For reading this data, I started to write a R-script. So far, I managed to read the Master file, which contains the information about the various securities in a directory, as well as reading the Fxxx.Dat data files. The data is stored in the Microsoft Binary Format, which was used for Basica under DOS. Thus, it has to be converted into the IEEE format for the floating point numbers. But here my problems start.

I have a DLL-file with functions to convert the 4byte MBF format into IEEE format as well as from IEEE to MBF format. As recommended, this DLL is in the Windows/System32 directory. I use this DLL with Visual Basic and it works fine. In the R script, I included the following line

dyn.load("MBFIEE32", DLLpath="C:/Windows/System32").

However, testing if the DLL had been loaded with is.loaded("MBFIEE32") returns FALSE. I have read the manual, but have not figured out what went wrong. Thus, my question is, what might I have overlooked, which could cause this error?

I also have a file with the functions for the format conversions in C. After making slight modifications to ensure that the functions return void as requested, I wanted to compile the file with the command (using the shell feature of RStudio) R CMD SHLIB [-d] -o MBF2IEEE.DLL MBF2IEEE.c. I called this command from the directory where the C-file is stored. However, the file was not compiled. Does the C-file has to be stored in a particular directory? After reading the online help, I can not figure out, what went wrong. 

Thanks for any help

Peter  
        [[alternative HTML version deleted]]


_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Reading MetaStock data format in R

Michael Weylandt


On Sep 30, 2013, at 3:28, Peter Fertig <[hidden email]> wrote:

> Hello,
>
> I am new at R and need some help because I am stuck in a project. I have financial data, which is stored in the MetaStock data format and is updated daily. For reading this data, I started to write a R-script. So far, I managed to read the Master file, which contains the information about the various securities in a directory, as well as reading the Fxxx.Dat data files. The data is stored in the Microsoft Binary Format, which was used for Basica under DOS. Thus, it has to be converted into the IEEE format for the floating point numbers. But here my problems start.

Godspeed -- nothing quite like non-standard binary formats before the first morning coffee.


>
> I have a DLL-file with functions to convert the 4byte MBF format into IEEE format as well as from IEEE to MBF format. As recommended, this DLL is in the Windows/System32 directory. I use this DLL with Visual Basic and it works fine. In the R script, I included the following line
>
> dyn.load("MBFIEE32", DLLpath="C:/Windows/System32").

Don't think it works like that -- anything you want exposed within R will need to return R objects or wrapper classes (SEXP's generally) -- if your DLL works for / was written for VB, it won't work un-modified in R.

>
> However, testing if the DLL had been loaded with is.loaded("MBFIEE32") returns FALSE. I have read the manual, but have not figured out what went wrong. Thus, my question is, what might I have overlooked, which could cause this error?
>
> I also have a file with the functions for the format conversions in C. After making slight modifications to ensure that the functions return void as requested, I wanted to compile the file with the command (using the shell feature of RStudio) R CMD SHLIB [-d] -o MBF2IEEE.DLL MBF2IEEE.c. I called this command from the directory where the C-file is stored. However, the file was not compiled. Does the C-file has to be stored in a particular directory? After reading the online help, I can not figure out, what went wrong.
>

Is your R development environment set up properly? R is not officially supported to build under VS and I'd imagine the same applies to packages.

To test that, I'd try building the 'xts' package locally -- it's an easy (enough) build and relevant for finance work anyways.

Depending on the scale of your data and the frequency you'll be doing this, it might be worth writing a standalone program which makes a csv file (or similar) and then just reading that into R so you can get on with your work.

Cheers,
Michael


> Thanks for any help
>
> Peter  
>    [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Reading MetaStock data format in R

Peter Fertig
Hi Michael,

thank you for your comments. I have learnt now that using the DLL file, will not work as I can not modify the DLL.

Creating csv or Excel files, another proposal I got, is what I have to do already for another commercial econometric software (RATS from Estima) I use now for a couple of years. Thus, I limited this to weekly data for only a few instruments. However, I would like to run the analysis on a daily basis for more instruments. Therefore, I still look for a solution in R by using C-code. Unfortunately, I have only experience in programming in Visual Basic and Pascal/Delphi. Maybe having only the code for the functions is not sufficient and the C-file needs a few more lines? Something, I still have to explore.

Best regards
Peter


________________________________
 Von: "R. Michael Weylandt <[hidden email]>" <[hidden email]>

CC: "[hidden email]" <[hidden email]>
Gesendet: 14:45 Montag, 30.September 2013
Betreff: Re: [R-SIG-Finance] Reading MetaStock data format in R






> Hello,
>
> I am new at R and need some help because I am stuck in a project. I have financial data, which is stored in the MetaStock data format and is updated daily. For reading this data, I started to write a R-script. So far, I managed to read the Master file, which contains the information about the various securities in a directory, as well as reading the Fxxx.Dat data files. The data is stored in the Microsoft Binary Format, which was used for Basica under DOS. Thus, it has to be converted into the IEEE format for the floating point numbers. But here my problems start.

Godspeed -- nothing quite like non-standard binary formats before the first morning coffee.


>
> I have a DLL-file with functions to convert the 4byte MBF format into IEEE format as well as from IEEE to MBF format. As recommended, this DLL is in the Windows/System32 directory. I use this DLL with Visual Basic and it works fine. In the R script, I included the following line
>
> dyn.load("MBFIEE32", DLLpath="C:/Windows/System32").

Don't think it works like that -- anything you want exposed within R will need to return R objects or wrapper classes (SEXP's generally) -- if your DLL works for / was written for VB, it won't work un-modified in R.

>
> However, testing if the DLL had been loaded with is.loaded("MBFIEE32") returns FALSE. I have read the manual, but have not figured out what went wrong. Thus, my question is, what might I have overlooked, which could cause this error?
>
> I also have a file with the functions for the format conversions in C. After making slight modifications to ensure that the functions return void as requested, I wanted to compile the file with the command (using the shell feature of RStudio) R CMD SHLIB [-d] -o MBF2IEEE.DLL MBF2IEEE.c. I called this command from the directory where the C-file is stored. However, the file was not compiled. Does the C-file has to be stored in a particular directory? After reading the online help, I can not figure out, what went wrong.
>

Is your R development environment set up properly? R is not officially supported to build under VS and I'd imagine the same applies to packages.

To test that, I'd try building the 'xts' package locally -- it's an easy (enough) build and relevant for finance work anyways.

Depending on the scale of your data and the frequency you'll be doing this, it might be worth writing a standalone program which makes a csv file (or similar) and then just reading that into R so you can get on with your work.

Cheers,
Michael


> Thanks for any help
>
> Peter 
>    [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.
        [[alternative HTML version deleted]]


_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Reading MetaStock data format in R

Frank-2
Have you looked at getSymbols() in the quantmod package? It can download
symbols into the R environment from Yahoo Finance. I also save the data to a
CSV file. I then download the dividends from Yahoo Finance to calculate
current yield. It is so fast I don't bother to save the data and append new
only data.

Best,

Frank
Chicago, IL

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Peter Fertig
Sent: Monday, September 30, 2013 8:55 AM
To: R. Michael Weylandt <[hidden email]>
Cc: [hidden email]
Subject: Re: [R-SIG-Finance] Reading MetaStock data format in R

Hi Michael,

thank you for your comments. I have learnt now that using the DLL file, will
not work as I can not modify the DLL.

Creating csv or Excel files, another proposal I got, is what I have to do
already for another commercial econometric software (RATS from Estima) I use
now for a couple of years. Thus, I limited this to weekly data for only a
few instruments. However, I would like to run the analysis on a daily basis
for more instruments. Therefore, I still look for a solution in R by using
C-code. Unfortunately, I have only experience in programming in Visual Basic
and Pascal/Delphi. Maybe having only the code for the functions is not
sufficient and the C-file needs a few more lines? Something, I still have to
explore.

Best regards
Peter


________________________________
 Von: "R. Michael Weylandt <[hidden email]>"
<[hidden email]>

CC: "[hidden email]" <[hidden email]>
Gesendet: 14:45 Montag, 30.September 2013
Betreff: Re: [R-SIG-Finance] Reading MetaStock data format in R






> Hello,
>
> I am new at R and need some help because I am stuck in a project. I have
financial data, which is stored in the MetaStock data format and is updated
daily. For reading this data, I started to write a R-script. So far, I
managed to read the Master file, which contains the information about the
various securities in a directory, as well as reading the Fxxx.Dat data
files. The data is stored in the Microsoft Binary Format, which was used for
Basica under DOS. Thus, it has to be converted into the IEEE format for the
floating point numbers. But here my problems start.

Godspeed -- nothing quite like non-standard binary formats before the first
morning coffee.


>
> I have a DLL-file with functions to convert the 4byte MBF format into IEEE
format as well as from IEEE to MBF format. As recommended, this DLL is in
the Windows/System32 directory. I use this DLL with Visual Basic and it
works fine. In the R script, I included the following line
>
> dyn.load("MBFIEE32", DLLpath="C:/Windows/System32").

Don't think it works like that -- anything you want exposed within R will
need to return R objects or wrapper classes (SEXP's generally) -- if your
DLL works for / was written for VB, it won't work un-modified in R.

>
> However, testing if the DLL had been loaded with is.loaded("MBFIEE32")
returns FALSE. I have read the manual, but have not figured out what went
wrong. Thus, my question is, what might I have overlooked, which could cause
this error?
>
> I also have a file with the functions for the format conversions in C.
After making slight modifications to ensure that the functions return void
as requested, I wanted to compile the file with the command (using the shell
feature of RStudio) R CMD SHLIB [-d] -o MBF2IEEE.DLL MBF2IEEE.c. I called
this command from the directory where the C-file is stored. However, the
file was not compiled. Does the C-file has to be stored in a particular
directory? After reading the online help, I can not figure out, what went
wrong.
>

Is your R development environment set up properly? R is not officially
supported to build under VS and I'd imagine the same applies to packages.

To test that, I'd try building the 'xts' package locally -- it's an easy
(enough) build and relevant for finance work anyways.

Depending on the scale of your data and the frequency you'll be doing this,
it might be worth writing a standalone program which makes a csv file (or
similar) and then just reading that into R so you can get on with your work.


Cheers,
Michael


> Thanks for any help
>
> Peter  
>    [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
should go.
        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Reading MetaStock data format in R

Michael Weylandt
In reply to this post by Peter Fertig


On Sep 30, 2013, at 9:55, Peter Fertig <[hidden email]> wrote:

> Hi Michael,
>
> thank you for your comments. I have learnt now that using the DLL file, will not work as I can not modify the DLL.

You don't need to modify it -- just wrap it in something which 'speaks R' rather than trying to call it directly.

M


>
> Creating csv or Excel files, another proposal I got, is what I have to do already for another commercial econometric software (RATS from Estima) I use now for a couple of years. Thus, I limited this to weekly data for only a few instruments. However, I would like to run the analysis on a daily basis for more instruments. Therefore, I still look for a solution in R by using C-code. Unfortunately, I have only experience in programming in Visual Basic and Pascal/Delphi. Maybe having only the code for the functions is not sufficient and the C-file needs a few more lines? Something, I still have to explore.
>
> Best regards
> Peter
>
> Von: "R. Michael Weylandt <[hidden email]>" <[hidden email]>
> An: Peter Fertig <[hidden email]>
> CC: "[hidden email]" <[hidden email]>
> Gesendet: 14:45 Montag, 30.September 2013
> Betreff: Re: [R-SIG-Finance] Reading MetaStock data format in R
>
>
>
> On Sep 30, 2013, at 3:28, Peter Fertig <[hidden email]> wrote:
>
> > Hello,
> >
> > I am new at R and need some help because I am stuck in a project. I have financial data, which is stored in the MetaStock data format and is updated daily. For reading this data, I started to write a R-script. So far, I managed to read the Master file, which contains the information about the various securities in a directory, as well as reading the Fxxx.Dat data files. The data is stored in the Microsoft Binary Format, which was used for Basica under DOS. Thus, it has to be converted into the IEEE format for the floating point numbers. But here my problems start.
>
> Godspeed -- nothing quite like non-standard binary formats before the first morning coffee.
>
>
> >
> > I have a DLL-file with functions to convert the 4byte MBF format into IEEE format as well as from IEEE to MBF format. As recommended, this DLL is in the Windows/System32 directory. I use this DLL with Visual Basic and it works fine. In the R script, I included the following line
> >
> > dyn.load("MBFIEE32", DLLpath="C:/Windows/System32").
>
> Don't think it works like that -- anything you want exposed within R will need to return R objects or wrapper classes (SEXP's generally) -- if your DLL works for / was written for VB, it won't work un-modified in R.
>
> >
> > However, testing if the DLL had been loaded with is.loaded("MBFIEE32") returns FALSE. I have read the manual, but have not figured out what went wrong. Thus, my question is, what might I have overlooked, which could cause this error?
> >
> > I also have a file with the functions for the format conversions in C. After making slight modifications to ensure that the functions return void as requested, I wanted to compile the file with the command (using the shell feature of RStudio) R CMD SHLIB [-d] -o MBF2IEEE.DLL MBF2IEEE.c. I called this command from the directory where the C-file is stored. However, the file was not compiled. Does the C-file has to be stored in a particular directory? After reading the online help, I can not figure out, what went wrong.
> >
>
> Is your R development environment set up properly? R is not officially supported to build under VS and I'd imagine the same applies to packages.
>
> To test that, I'd try building the 'xts' package locally -- it's an easy (enough) build and relevant for finance work anyways.
>
> Depending on the scale of your data and the frequency you'll be doing this, it might be worth writing a standalone program which makes a csv file (or similar) and then just reading that into R so you can get on with your work.
>
> Cheers,
> Michael
>
>
> > Thanks for any help
> >
> > Peter  
> >    [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R questions should go.
>
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Reading MetaStock data format in R

Paul Gilbert-2
Peter

(No doubt you are working in a context where you don't have control or
influence over everything, but ... )

I think you are really attacking the wrong part of your problem. If you
are going to all the trouble of downloading and updating daily data on
an inhouse db, why don't you put it in a modern open format that is
generally usable, or even put it in a real database?

Best,
Paul


On 13-09-30 10:19 AM, R. Michael Weylandt <[hidden email]>
wrote:

>
>
> On Sep 30, 2013, at 9:55, Peter Fertig <[hidden email]> wrote:
>
>> Hi Michael,
>>
>> thank you for your comments. I have learnt now that using the DLL
>> file, will not work as I can not modify the DLL.
>
> You don't need to modify it -- just wrap it in something which
> 'speaks R' rather than trying to call it directly.
>
> M
>
>
>>
>> Creating csv or Excel files, another proposal I got, is what I have
>> to do already for another commercial econometric software (RATS
>> from Estima) I use now for a couple of years. Thus, I limited this
>> to weekly data for only a few instruments. However, I would like to
>> run the analysis on a daily basis for more instruments. Therefore,
>> I still look for a solution in R by using C-code. Unfortunately, I
>> have only experience in programming in Visual Basic and
>> Pascal/Delphi. Maybe having only the code for the functions is not
>> sufficient and the C-file needs a few more lines? Something, I
>> still have to explore.
>>
>> Best regards Peter
>>
>> Von: "R. Michael Weylandt <[hidden email]>"
>> <[hidden email]> An: Peter Fertig <[hidden email]>
>> CC: "[hidden email]" <[hidden email]>
>> Gesendet: 14:45 Montag, 30.September 2013 Betreff: Re:
>> [R-SIG-Finance] Reading MetaStock data format in R
>>
>>
>>
>> On Sep 30, 2013, at 3:28, Peter Fertig <[hidden email]> wrote:
>>
>>> Hello,
>>>
>>> I am new at R and need some help because I am stuck in a project.
>>> I have financial data, which is stored in the MetaStock data
>>> format and is updated daily. For reading this data, I started to
>>> write a R-script. So far, I managed to read the Master file,
>>> which contains the information about the various securities in a
>>> directory, as well as reading the Fxxx.Dat data files. The data
>>> is stored in the Microsoft Binary Format, which was used for
>>> Basica under DOS. Thus, it has to be converted into the IEEE
>>> format for the floating point numbers. But here my problems
>>> start.
>>
>> Godspeed -- nothing quite like non-standard binary formats before
>> the first morning coffee.
>>
>>
>>>
>>> I have a DLL-file with functions to convert the 4byte MBF format
>>> into IEEE format as well as from IEEE to MBF format. As
>>> recommended, this DLL is in the Windows/System32 directory. I use
>>> this DLL with Visual Basic and it works fine. In the R script, I
>>> included the following line
>>>
>>> dyn.load("MBFIEE32", DLLpath="C:/Windows/System32").
>>
>> Don't think it works like that -- anything you want exposed within
>> R will need to return R objects or wrapper classes (SEXP's
>> generally) -- if your DLL works for / was written for VB, it won't
>> work un-modified in R.
>>
>>>
>>> However, testing if the DLL had been loaded with
>>> is.loaded("MBFIEE32") returns FALSE. I have read the manual, but
>>> have not figured out what went wrong. Thus, my question is, what
>>> might I have overlooked, which could cause this error?
>>>
>>> I also have a file with the functions for the format conversions
>>> in C. After making slight modifications to ensure that the
>>> functions return void as requested, I wanted to compile the file
>>> with the command (using the shell feature of RStudio) R CMD SHLIB
>>> [-d] -o MBF2IEEE.DLL MBF2IEEE.c. I called this command from the
>>> directory where the C-file is stored. However, the file was not
>>> compiled. Does the C-file has to be stored in a particular
>>> directory? After reading the online help, I can not figure out,
>>> what went wrong.
>>>
>>
>> Is your R development environment set up properly? R is not
>> officially supported to build under VS and I'd imagine the same
>> applies to packages.
>>
>> To test that, I'd try building the 'xts' package locally -- it's an
>> easy (enough) build and relevant for finance work anyways.
>>
>> Depending on the scale of your data and the frequency you'll be
>> doing this, it might be worth writing a standalone program which
>> makes a csv file (or similar) and then just reading that into R so
>> you can get on with your work.
>>
>> Cheers, Michael
>>
>>
>>> Thanks for any help
>>>
>>> Peter [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance --
>>> Subscriber-posting only. If you want to post, subscribe first. --
>>> Also note that this is not the r-help list where general R
>>> questions should go.
>>
>>
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance --
> Subscriber-posting only. If you want to post, subscribe first. --
> Also note that this is not the r-help list where general R questions
> should go.
>

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Reading MetaStock data format in R

alexios
In reply to this post by Peter Fertig
The Metastock conversion topic was covered here before:

https://stat.ethz.ch/pipermail/r-sig-finance/2009q4/005368.html

Have a look at this open source C++ project for converting
Metastock to csv, after which you can import into R.

https://bitbucket.org/rudimeier/atem/overview


-Alexios

On 30/09/2013 14:55, Peter Fertig wrote:

> Hi Michael,
>
> thank you for your comments. I have learnt now that using the DLL file, will not work as I can not modify the DLL.
>
> Creating csv or Excel files, another proposal I got, is what I have to do already for another commercial econometric software (RATS from Estima) I use now for a couple of years. Thus, I limited this to weekly data for only a few instruments. However, I would like to run the analysis on a daily basis for more instruments. Therefore, I still look for a solution in R by using C-code. Unfortunately, I have only experience in programming in Visual Basic and Pascal/Delphi. Maybe having only the code for the functions is not sufficient and the C-file needs a few more lines? Something, I still have to explore.
>
> Best regards
> Peter
>
>
> ________________________________
>  Von: "R. Michael Weylandt <[hidden email]>" <[hidden email]>
>
> CC: "[hidden email]" <[hidden email]>
> Gesendet: 14:45 Montag, 30.September 2013
> Betreff: Re: [R-SIG-Finance] Reading MetaStock data format in R
>
>
>
>
>
>
>> Hello,
>>
>> I am new at R and need some help because I am stuck in a project. I have financial data, which is stored in the MetaStock data format and is updated daily. For reading this data, I started to write a R-script. So far, I managed to read the Master file, which contains the information about the various securities in a directory, as well as reading the Fxxx.Dat data files. The data is stored in the Microsoft Binary Format, which was used for Basica under DOS. Thus, it has to be converted into the IEEE format for the floating point numbers. But here my problems start.
>
> Godspeed -- nothing quite like non-standard binary formats before the first morning coffee.
>
>
>>
>> I have a DLL-file with functions to convert the 4byte MBF format into IEEE format as well as from IEEE to MBF format. As recommended, this DLL is in the Windows/System32 directory. I use this DLL with Visual Basic and it works fine. In the R script, I included the following line
>>
>> dyn.load("MBFIEE32", DLLpath="C:/Windows/System32").
>
> Don't think it works like that -- anything you want exposed within R will need to return R objects or wrapper classes (SEXP's generally) -- if your DLL works for / was written for VB, it won't work un-modified in R.
>
>>
>> However, testing if the DLL had been loaded with is.loaded("MBFIEE32") returns FALSE. I have read the manual, but have not figured out what went wrong. Thus, my question is, what might I have overlooked, which could cause this error?
>>
>> I also have a file with the functions for the format conversions in C. After making slight modifications to ensure that the functions return void as requested, I wanted to compile the file with the command (using the shell feature of RStudio) R CMD SHLIB [-d] -o MBF2IEEE.DLL MBF2IEEE.c. I called this command from the directory where the C-file is stored. However, the file was not compiled. Does the C-file has to be stored in a particular directory? After reading the online help, I can not figure out, what went wrong.
>>
>
> Is your R development environment set up properly? R is not officially supported to build under VS and I'd imagine the same applies to packages.
>
> To test that, I'd try building the 'xts' package locally -- it's an easy (enough) build and relevant for finance work anyways.
>
> Depending on the scale of your data and the frequency you'll be doing this, it might be worth writing a standalone program which makes a csv file (or similar) and then just reading that into R so you can get on with your work.
>
> Cheers,
> Michael
>
>
>> Thanks for any help
>>
>> Peter  
>>     [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R questions should go.
> [[alternative HTML version deleted]]
>
>
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.
>

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Reading MetaStock data format in R

Peter Fertig
Hi Alexios,

thank you for your answer. Converting the MetaStock data into a csv file is not a problem. I can do this with Visual Basic easily. I want to use R for some time series analysis and want to use the results for trading systems in MetaStock or TradeStation. Thus, I would have two conversions for each security. This could be time consuming. Therefore, I look for a way to read/write the data directly from R. The problem I run into is just the conversion of the binary formats. Here, the solutions I have for Visual Basic don't work in R.

Best regards

Peter


________________________________
 Von: alexios ghalanos <[hidden email]>

CC: "R. Michael Weylandt <[hidden email]>" <[hidden email]>; "[hidden email]" <[hidden email]>
Gesendet: 16:36 Montag, 30.September 2013
Betreff: Re: [R-SIG-Finance] Reading MetaStock data format in R


The Metastock conversion topic was covered here before:

https://stat.ethz.ch/pipermail/r-sig-finance/2009q4/005368.html

Have a look at this open source C++ project for converting
Metastock to csv, after which you can import into R.

https://bitbucket.org/rudimeier/atem/overview


-Alexios

On 30/09/2013 14:55, Peter Fertig wrote:

> Hi Michael,
>
> thank you for your comments. I have learnt now that using the DLL file, will not work as I can not modify the DLL.
>
> Creating csv or Excel files, another proposal I got, is what I have to do already for another commercial econometric software (RATS from Estima) I use now for a couple of years. Thus, I limited this to weekly data for only a few instruments. However, I would like to run the analysis on a daily basis for more instruments. Therefore, I still look for a solution in R by using C-code. Unfortunately, I have only experience in programming in Visual Basic and Pascal/Delphi. Maybe having only the code for the functions is not sufficient and the C-file needs a few more lines? Something, I still have to explore.
>
> Best regards
> Peter
>
>
> ________________________________
>  Von: "R. Michael Weylandt <[hidden email]>" <[hidden email]>
>
> CC: "[hidden email]" <[hidden email]>
> Gesendet: 14:45 Montag, 30.September 2013
> Betreff: Re: [R-SIG-Finance] Reading MetaStock data format in R
>
>
>
>
>
>
>> Hello,
>>
>> I am new at R and need some help because I am stuck in a project. I have financial data, which is stored in the MetaStock data format and is updated daily. For reading this data, I started to write a R-script. So far, I managed to read the Master file, which contains the information about the various securities in a directory, as well as reading the Fxxx.Dat data files. The data is stored in the Microsoft Binary Format, which was used for Basica under DOS. Thus, it has to be converted into the IEEE format for the floating point numbers. But here my problems start.
>
> Godspeed -- nothing quite like non-standard binary formats before the first morning coffee.
>
>
>>
>> I have a DLL-file with functions to convert the 4byte MBF format into IEEE format as well as from IEEE to MBF format. As recommended, this DLL is in the Windows/System32 directory. I use this DLL with Visual Basic and it works fine. In the R script, I included the following line
>>
>> dyn.load("MBFIEE32", DLLpath="C:/Windows/System32").
>
> Don't think it works like that -- anything you want exposed within R will need to return R objects or wrapper classes (SEXP's generally) -- if your DLL works for / was written for VB, it won't work un-modified in R.
>
>>
>> However, testing if the DLL had been loaded with is.loaded("MBFIEE32") returns FALSE. I have read the manual, but have not figured out what went wrong. Thus, my question is, what might I have overlooked, which could cause this error?
>>
>> I also have a file with the functions for the format conversions in C. After making slight modifications to ensure that the functions return void as requested, I wanted to compile the file with the command (using the shell feature of RStudio) R CMD SHLIB [-d] -o MBF2IEEE.DLL MBF2IEEE.c. I called this command from the directory where the C-file is stored. However, the file was not compiled. Does the C-file has to be stored in a particular directory? After reading the online help, I can not figure out, what went wrong.
>>
>
> Is your R development environment set up properly? R is not officially supported to build under VS and I'd imagine the same applies to packages.
>
> To test that, I'd try building the 'xts' package locally -- it's an easy (enough) build and relevant for finance work anyways.
>
> Depending on the scale of your data and the frequency you'll be doing this, it might be worth writing a standalone program which makes a csv file (or similar) and then just reading that into R so you can get on with your work.
>
> Cheers,
> Michael
>
>
>> Thanks for any help
>>
>> Peter 
>>     [[alternative HTML version deleted]]
>>
>> _______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R questions should go.
>     [[alternative HTML version deleted]]
>
>
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.
>
        [[alternative HTML version deleted]]


_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Reading MetaStock data format in R

alexios
I think you can also achieve what you want (i.e. 3rd party calling R for
analysis without leaving the 3rd party environment) using
http://www.omegahat.org/RDCOMServer/

A similar 'plug-in' was developed for amibroker doing just that:
https://stat.ethz.ch/pipermail/r-sig-finance/2008q3/002859.html

-Alexios

On 30/09/2013 16:00, Peter Fertig wrote:

> Hi Alexios,
>
> thank you for your answer. Converting the MetaStock data into a csv file
> is not a problem. I can do this with Visual Basic easily. I want to use
> R for some time series analysis and want to use the results for trading
> systems in MetaStock or TradeStation. Thus, I would have two conversions
> for each security. This could be time consuming. Therefore, I look for a
> way to read/write the data directly from R. The problem I run into is
> just the conversion of the binary formats. Here, the solutions I have
> for Visual Basic don't work in R.
>
> Best regards
>
> Peter
>
> ------------------------------------------------------------------------
> *Von:* alexios ghalanos <[hidden email]>
> *An:* Peter Fertig <[hidden email]>
> *CC:* "R. Michael Weylandt <[hidden email]>"
> <[hidden email]>; "[hidden email]"
> <[hidden email]>
> *Gesendet:* 16:36 Montag, 30.September 2013
> *Betreff:* Re: [R-SIG-Finance] Reading MetaStock data format in R
>
> The Metastock conversion topic was covered here before:
>
> https://stat.ethz.ch/pipermail/r-sig-finance/2009q4/005368.html
>
> Have a look at this open source C++ project for converting
> Metastock to csv, after which you can import into R.
>
> https://bitbucket.org/rudimeier/atem/overview
>
>
> -Alexios
>
> On 30/09/2013 14:55, Peter Fertig wrote:
>> Hi Michael,
>>
>> thank you for your comments. I have learnt now that using the DLL
> file, will not work as I can not modify the DLL.
>>
>> Creating csv or Excel files, another proposal I got, is what I have to
> do already for another commercial econometric software (RATS from
> Estima) I use now for a couple of years. Thus, I limited this to weekly
> data for only a few instruments. However, I would like to run the
> analysis on a daily basis for more instruments. Therefore, I still look
> for a solution in R by using C-code. Unfortunately, I have only
> experience in programming in Visual Basic and Pascal/Delphi. Maybe
> having only the code for the functions is not sufficient and the C-file
> needs a few more lines? Something, I still have to explore.
>>
>> Best regards
>> Peter
>>
>>
>> ________________________________
>>  Von: "R. Michael Weylandt <[hidden email]
> <mailto:[hidden email]>>" <[hidden email]
> <mailto:[hidden email]>>
>>
>> CC: "[hidden email] <mailto:[hidden email]>"
> <[hidden email] <mailto:[hidden email]>>
>> Gesendet: 14:45 Montag, 30.September 2013
>> Betreff: Re: [R-SIG-Finance] Reading MetaStock data format in R
>>
>>
>>
>>
>>
>>
>>> Hello,
>>>
>>> I am new at R and need some help because I am stuck in a project. I
> have financial data, which is stored in the MetaStock data format and is
> updated daily. For reading this data, I started to write a R-script. So
> far, I managed to read the Master file, which contains the information
> about the various securities in a directory, as well as reading the
> Fxxx.Dat data files. The data is stored in the Microsoft Binary Format,
> which was used for Basica under DOS. Thus, it has to be converted into
> the IEEE format for the floating point numbers. But here my problems start.
>>
>> Godspeed -- nothing quite like non-standard binary formats before the
> first morning coffee.
>>
>>
>>>
>>> I have a DLL-file with functions to convert the 4byte MBF format into
> IEEE format as well as from IEEE to MBF format. As recommended, this DLL
> is in the Windows/System32 directory. I use this DLL with Visual Basic
> and it works fine. In the R script, I included the following line
>>>
>>> dyn.load("MBFIEE32", DLLpath="C:/Windows/System32").
>>
>> Don't think it works like that -- anything you want exposed within R
> will need to return R objects or wrapper classes (SEXP's generally) --
> if your DLL works for / was written for VB, it won't work un-modified in R.
>>
>>>
>>> However, testing if the DLL had been loaded with
> is.loaded("MBFIEE32") returns FALSE. I have read the manual, but have
> not figured out what went wrong. Thus, my question is, what might I have
> overlooked, which could cause this error?
>>>
>>> I also have a file with the functions for the format conversions in
> C. After making slight modifications to ensure that the functions return
> void as requested, I wanted to compile the file with the command (using
> the shell feature of RStudio) R CMD SHLIB [-d] -o MBF2IEEE.DLL
> MBF2IEEE.c. I called this command from the directory where the C-file is
> stored. However, the file was not compiled. Does the C-file has to be
> stored in a particular directory? After reading the online help, I can
> not figure out, what went wrong.
>>>
>>
>> Is your R development environment set up properly? R is not officially
> supported to build under VS and I'd imagine the same applies to packages.
>>
>> To test that, I'd try building the 'xts' package locally -- it's an
> easy (enough) build and relevant for finance work anyways.
>>
>> Depending on the scale of your data and the frequency you'll be doing
> this, it might be worth writing a standalone program which makes a csv
> file (or similar) and then just reading that into R so you can get on
> with your work.
>>
>> Cheers,
>> Michael
>>
>>
>>> Thanks for any help
>>>
>>> Peter
>>>    [[alternative HTML version deleted]]
>>>
>>> _______________________________________________
>>> [hidden email] <mailto:[hidden email]>
> mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>>> -- Subscriber-posting only. If you want to post, subscribe first.
>>> -- Also note that this is not the r-help list where general R
> questions should go.
>>     [[alternative HTML version deleted]]
>>
>>
>>
>> _______________________________________________
>> [hidden email] <mailto:[hidden email]>
> mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
>> -- Subscriber-posting only. If you want to post, subscribe first.
>> -- Also note that this is not the r-help list where general R
> questions should go.
>>
>
>
>

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.