PairTrading package

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
15 messages Options
Reply | Threaded
Open this post in threaded view
|

PairTrading package

Alec Schmidt-2
I used to have R version 3.6.0 and tried to install PairTrading but got a message that the package is not available for that version.
Now I've updated R to 4.0.2. but still have the message:

package �PairTrading� is not available (for R version 4.0.2)

I wonder if anything can be done about it or there may be other packages with similar functionality.

Thanks! Alec

        [[alternative HTML version deleted]]


_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: PairTrading package

Daniel Cegiełka
Hi Alec,

$ R --version
R version 4.0.0 (2020-04-24) -- "Arbor Day"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin17.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://www.gnu.org/licenses/.

$ git clone https://github.com/cran/PairTrading
Cloning into 'PairTrading'...
remote: Enumerating objects: 39, done.
remote: Counting objects: 100% (39/39), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 39 (delta 11), reused 39 (delta 11), pack-reused 0
Unpacking objects: 100% (39/39), done.

$ ls
PairTrading

$ R CMD INSTALL PairTrading/
* installing to library ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library’
* installing *source* package ‘PairTrading’ ...

(…)

** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (PairTrading)


Best regards,
Daniel


> On 21 Sep 2020, at 22:38, Alec Schmidt <[hidden email]> wrote:
>
> I used to have R version 3.6.0 and tried to install PairTrading but got a message that the package is not available for that version.
> Now I've updated R to 4.0.2. but still have the message:
>
> package �PairTrading� is not available (for R version 4.0.2)
>
> I wonder if anything can be done about it or there may be other packages with similar functionality.
>
> Thanks! Alec
>
> [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: PairTrading package

Jasen Mackie
Hi Alec

Thanks Daniel. I will add assuming you are looking for something to merely
test a strategy as opposed to something specifically using cointegration to
build your signal, there is also quantstrat
<https://github.com/braverock/quantstrat> and you can find a pair trading
demo here
<https://github.com/braverock/quantstrat/blob/master/demo/pair_trade.R>.

Regards
Jasen

On Mon, 21 Sep 2020 at 17:01, Daniel Cegiełka <[hidden email]>
wrote:

> Hi Alec,
>
> $ R --version
> R version 4.0.0 (2020-04-24) -- "Arbor Day"
> Copyright (C) 2020 The R Foundation for Statistical Computing
> Platform: x86_64-apple-darwin17.0 (64-bit)
>
> R is free software and comes with ABSOLUTELY NO WARRANTY.
> You are welcome to redistribute it under the terms of the
> GNU General Public License versions 2 or 3.
> For more information about these matters see
> https://www.gnu.org/licenses/.
>
> $ git clone https://github.com/cran/PairTrading
> Cloning into 'PairTrading'...
> remote: Enumerating objects: 39, done.
> remote: Counting objects: 100% (39/39), done.
> remote: Compressing objects: 100% (24/24), done.
> remote: Total 39 (delta 11), reused 39 (delta 11), pack-reused 0
> Unpacking objects: 100% (39/39), done.
>
> $ ls
> PairTrading
>
> $ R CMD INSTALL PairTrading/
> * installing to library
> ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library’
> * installing *source* package ‘PairTrading’ ...
>
> (…)
>
> ** building package indices
> ** testing if installed package can be loaded from temporary location
> ** testing if installed package can be loaded from final location
> ** testing if installed package keeps a record of temporary installation
> path
> * DONE (PairTrading)
>
>
> Best regards,
> Daniel
>
>
> > On 21 Sep 2020, at 22:38, Alec Schmidt <[hidden email]> wrote:
> >
> > I used to have R version 3.6.0 and tried to install PairTrading but got
> a message that the package is not available for that version.
> > Now I've updated R to 4.0.2. but still have the message:
> >
> > package �PairTrading� is not available (for R version 4.0.2)
> >
> > I wonder if anything can be done about it or there may be other packages
> with similar functionality.
> >
> > Thanks! Alec
> >
> >       [[alternative HTML version deleted]]
> >
> > _______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R questions
> should go.
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: PairTrading package

Alec Schmidt-2
In reply to this post by Daniel Cegiełka
Daniel and Jasen, thanks much!

Daniel, sorry, I'm on Windows PC, so I used dir instead of ls and I do see PairTrading but your command
R CMD INSTALL PairTrading/ doesn't work:

'R' is not recognized as an internal or external command,operable program or batch file.

Thanks again, Alec

________________________________
From: Daniel Cegiełka <[hidden email]>
Sent: Monday, September 21, 2020 5:01 PM
To: Alec Schmidt <[hidden email]>
Cc: [hidden email] <[hidden email]>
Subject: Re: [R-SIG-Finance] PairTrading package

Hi Alec,

$ R --version
R version 4.0.0 (2020-04-24) -- "Arbor Day"
Copyright (C) 2020 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin17.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under the terms of the
GNU General Public License versions 2 or 3.
For more information about these matters see
https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gnu.org%2Flicenses%2F&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Ce85102d9ef0f4cea022e08d85e718b7e%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363189046435726&amp;sdata=OfQqu78x4j3D6D1oHM4KrPcqTha0Y%2FMLVPUESTCQH4k%3D&amp;reserved=0.

$ git clone https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcran%2FPairTrading&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Ce85102d9ef0f4cea022e08d85e718b7e%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363189046440727&amp;sdata=RqrpSGXxOiJgM5I05DV0fPmRFUmkBnYHD5kge1Id3AE%3D&amp;reserved=0
Cloning into 'PairTrading'...
remote: Enumerating objects: 39, done.
remote: Counting objects: 100% (39/39), done.
remote: Compressing objects: 100% (24/24), done.
remote: Total 39 (delta 11), reused 39 (delta 11), pack-reused 0
Unpacking objects: 100% (39/39), done.

$ ls
PairTrading

$ R CMD INSTALL PairTrading/
* installing to library ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library’
* installing *source* package ‘PairTrading’ ...

(…)

** building package indices
** testing if installed package can be loaded from temporary location
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (PairTrading)


Best regards,
Daniel


> On 21 Sep 2020, at 22:38, Alec Schmidt <[hidden email]> wrote:
>
> I used to have R version 3.6.0 and tried to install PairTrading but got a message that the package is not available for that version.
> Now I've updated R to 4.0.2. but still have the message:
>
> package �PairTrading� is not available (for R version 4.0.2)
>
> I wonder if anything can be done about it or there may be other packages with similar functionality.
>
> Thanks! Alec
>
>        [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fr-sig-finance&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Ce85102d9ef0f4cea022e08d85e718b7e%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363189046440727&amp;sdata=yLrwHpI16stt%2F0aYnQjIJgSgIegnuOtmLHYlAe8DQkQ%3D&amp;reserved=0
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.


        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: PairTrading package

Enrico Schumann-2
On Mon, 21 Sep 2020, Alec Schmidt writes:

> Daniel and Jasen, thanks much!
>
> Daniel, sorry, I'm on Windows PC, so I used dir instead of ls and I do see PairTrading but your command
> R CMD INSTALL PairTrading/ doesn't work:
>
> 'R' is not recognized as an internal or external command,operable program or batch file.
>
> Thanks again, Alec

It's more of a question for R-help, but:

If you follow the URL https://cran.r-project.org/package=PairTrading ,
you'll find that the package has been archived. You can still follow the
archive link, download the package and then install (on Windows) from
the R-GUI by selecting "install from local files" in the packages menu.
(Keep in mind that packages are archived for a reason, usually.)

If you want to try Daniel's suggestion: That 'R' is not found tells you
it's not in the search path.  You can set it on Windows in the
system/environment variable settings; just add 'C:/Program Files/R
<....>/bin ' (or whereever R is installed on your system) to the path.


kind regards
    Enrico

> ________________________________
> From: Daniel Cegiełka <[hidden email]>
> Sent: Monday, September 21, 2020 5:01 PM
> To: Alec Schmidt <[hidden email]>
> Cc: [hidden email] <[hidden email]>
> Subject: Re: [R-SIG-Finance] PairTrading package
>
> Hi Alec,
>
> $ R --version
> R version 4.0.0 (2020-04-24) -- "Arbor Day"
> Copyright (C) 2020 The R Foundation for Statistical Computing
> Platform: x86_64-apple-darwin17.0 (64-bit)
>
> R is free software and comes with ABSOLUTELY NO WARRANTY.
> You are welcome to redistribute it under the terms of the
> GNU General Public License versions 2 or 3.
> For more information about these matters see
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gnu.org%2Flicenses%2F&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Ce85102d9ef0f4cea022e08d85e718b7e%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363189046435726&amp;sdata=OfQqu78x4j3D6D1oHM4KrPcqTha0Y%2FMLVPUESTCQH4k%3D&amp;reserved=0.
>
> $ git clone
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcran%2FPairTrading&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Ce85102d9ef0f4cea022e08d85e718b7e%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363189046440727&amp;sdata=RqrpSGXxOiJgM5I05DV0fPmRFUmkBnYHD5kge1Id3AE%3D&amp;reserved=0
> Cloning into 'PairTrading'...
> remote: Enumerating objects: 39, done.
> remote: Counting objects: 100% (39/39), done.
> remote: Compressing objects: 100% (24/24), done.
> remote: Total 39 (delta 11), reused 39 (delta 11), pack-reused 0
> Unpacking objects: 100% (39/39), done.
>
> $ ls
> PairTrading
>
> $ R CMD INSTALL PairTrading/
> * installing to library ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library’
> * installing *source* package ‘PairTrading’ ...
>
> (…)
>
> ** building package indices
> ** testing if installed package can be loaded from temporary location
> ** testing if installed package can be loaded from final location
> ** testing if installed package keeps a record of temporary installation path
> * DONE (PairTrading)
>
>
> Best regards,
> Daniel
>
>
>> On 21 Sep 2020, at 22:38, Alec Schmidt <[hidden email]> wrote:
>>
>> I used to have R version 3.6.0 and tried to install PairTrading but got a message that the package is not available for that version.
>> Now I've updated R to 4.0.2. but still have the message:
>>
>> package �PairTrading� is not available (for R version 4.0.2)
>>
>> I wonder if anything can be done about it or there may be other packages with similar functionality.
>>
>> Thanks! Alec
>>

[...]


--
Enrico Schumann
Lucerne, Switzerland
http://enricoschumann.net

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: PairTrading package

Alec Schmidt-2
Thanks a lot, Enrico.

________________________________
From: Enrico Schumann <[hidden email]>
Sent: Tuesday, September 22, 2020 1:29 AM
To: Alec Schmidt <[hidden email]>
Cc: Daniel Cegiełka <[hidden email]>; [hidden email] <[hidden email]>
Subject: Re: [R-SIG-Finance] PairTrading package

On Mon, 21 Sep 2020, Alec Schmidt writes:

> Daniel and Jasen, thanks much!
>
> Daniel, sorry, I'm on Windows PC, so I used dir instead of ls and I do see PairTrading but your command
> R CMD INSTALL PairTrading/ doesn't work:
>
> 'R' is not recognized as an internal or external command,operable program or batch file.
>
> Thanks again, Alec

It's more of a question for R-help, but:

If you follow the URL https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fpackage%3DPairTrading&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Cc6a153bd55104ef7910208d85eb87a52%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363493716373736&amp;sdata=bsmkUniLSnS6ofp73MaG0DYToVii7VEZbzIgzZcAjR8%3D&amp;reserved=0 ,
you'll find that the package has been archived. You can still follow the
archive link, download the package and then install (on Windows) from
the R-GUI by selecting "install from local files" in the packages menu.
(Keep in mind that packages are archived for a reason, usually.)

If you want to try Daniel's suggestion: That 'R' is not found tells you
it's not in the search path.  You can set it on Windows in the
system/environment variable settings; just add 'C:/Program Files/R
<....>/bin ' (or whereever R is installed on your system) to the path.


kind regards
    Enrico

> ________________________________
> From: Daniel Cegiełka <[hidden email]>
> Sent: Monday, September 21, 2020 5:01 PM
> To: Alec Schmidt <[hidden email]>
> Cc: [hidden email] <[hidden email]>
> Subject: Re: [R-SIG-Finance] PairTrading package
>
> Hi Alec,
>
> $ R --version
> R version 4.0.0 (2020-04-24) -- "Arbor Day"
> Copyright (C) 2020 The R Foundation for Statistical Computing
> Platform: x86_64-apple-darwin17.0 (64-bit)
>
> R is free software and comes with ABSOLUTELY NO WARRANTY.
> You are welcome to redistribute it under the terms of the
> GNU General Public License versions 2 or 3.
> For more information about these matters see
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gnu.org%2Flicenses%2F&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Cc6a153bd55104ef7910208d85eb87a52%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363493716383730&amp;sdata=55EEnTvuJH429ccclZvkkx1%2BWWejUD5Qy2b17UzGAis%3D&amp;reserved=0.
>
> $ git clone
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcran%2FPairTrading&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Cc6a153bd55104ef7910208d85eb87a52%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363493716383730&amp;sdata=erwDXdwA44XFD93I1vBXTtmm819Mb9PUQvGvo0kdm%2FM%3D&amp;reserved=0
> Cloning into 'PairTrading'...
> remote: Enumerating objects: 39, done.
> remote: Counting objects: 100% (39/39), done.
> remote: Compressing objects: 100% (24/24), done.
> remote: Total 39 (delta 11), reused 39 (delta 11), pack-reused 0
> Unpacking objects: 100% (39/39), done.
>
> $ ls
> PairTrading
>
> $ R CMD INSTALL PairTrading/
> * installing to library ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library’
> * installing *source* package ‘PairTrading’ ...
>
> (…)
>
> ** building package indices
> ** testing if installed package can be loaded from temporary location
> ** testing if installed package can be loaded from final location
> ** testing if installed package keeps a record of temporary installation path
> * DONE (PairTrading)
>
>
> Best regards,
> Daniel
>
>
>> On 21 Sep 2020, at 22:38, Alec Schmidt <[hidden email]> wrote:
>>
>> I used to have R version 3.6.0 and tried to install PairTrading but got a message that the package is not available for that version.
>> Now I've updated R to 4.0.2. but still have the message:
>>
>> package �PairTrading� is not available (for R version 4.0.2)
>>
>> I wonder if anything can be done about it or there may be other packages with similar functionality.
>>
>> Thanks! Alec
>>

[...]


--
Enrico Schumann
Lucerne, Switzerland
https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fenricoschumann.net%2F&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Cc6a153bd55104ef7910208d85eb87a52%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363493716383730&amp;sdata=sEkds70iDxr5HsNMuTycpt5bLSVl%2BA35S7GNbheGi7Y%3D&amp;reserved=0

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: PairTrading package

Alec Schmidt-2
In reply to this post by Enrico Schumann-2
I'm trying to reproduce the results described in
 https://r-forge.r-project.org/scm/viewvc.php/*checkout*/pkg/inst/doc/IntroductionToPairTrading.pdf?revision=6&root=pairtrading&pathrev=6 [1]
(and in  https://www.r-bloggers.com/2011/10/pair-trading-strategy-how-to-use-pairtrading-package/)

I have some minor differences, eg
in [1]:
adf.test(as.numeric(reg$spread)) Augmented Dickey-Fuller Test data: as.numeric(reg$spread) Dickey-Fuller = -3.6462, Lag order = 8, p-value = 0.02825
and I have: Dickey-Fuller = -3.4033, Lag order = 8, p-value = 0.05288

Also, the commands
barplot(signal,col="blue",space = 0, border = "blue",xaxt="n",yaxt="n",xlab="",ylab="")
par(new=TRUE)
plot(params$spread, type="l", col = "red",lwd = 3, main = "Spread & Signal")

create in my environment two separate plots (bar and line) rather than one with superimposed bar and line charts like in [1].

Most importantly, I have all returns 'NA':
 > return.pairtrading
           [,1]
2009-01-04   NA
2009-01-05   NA
2009-01-06   NA
2009-01-07   NA

even though the parameters for the function
return.pairtrading <- Return(price.pair, lag(signal), lag(params$hedge.ratio))
look reasonable.

Thanks! Alec
[https://www.r-bloggers.com/wp-content/uploads/2016/08/R_single_01-200-1.png]<https://www.r-bloggers.com/2011/10/pair-trading-strategy-how-to-use-pairtrading-package/>
Pair trading strategy : how to use 'PairTrading' package<https://www.r-bloggers.com/2011/10/pair-trading-strategy-how-to-use-pairtrading-package/>
Mr.Ishikawa(my old friend) and I developed "PairTrading" package, and uploaded it on CRAN.This article shows you how you can use it.The pair trading is a market neutral trading strategy and gives traders a chance to profit regardless of market conditions. The idea of this strategy is ...
www.r-bloggers.com

Introduction to Pair Trading -Based on Cointegration-<https://r-forge.r-project.org/scm/viewvc.php/*checkout*/pkg/inst/doc/IntroductionToPairTrading.pdf?revision=6&root=pairtrading&pathrev=6>
•Gerry Bamberger and Nunzio Tartaglia •Quantitative group at Morgan Stanley •Around 1980s •D.E. Shaw & Co. is famous for this strategy Pair trading was pioneered by … 4
r-forge.r-project.org

________________________________
From: Enrico Schumann <[hidden email]>
Sent: Tuesday, September 22, 2020 1:29 AM
To: Alec Schmidt <[hidden email]>
Cc: Daniel Cegiełka <[hidden email]>; [hidden email] <[hidden email]>
Subject: Re: [R-SIG-Finance] PairTrading package

On Mon, 21 Sep 2020, Alec Schmidt writes:

> Daniel and Jasen, thanks much!
>
> Daniel, sorry, I'm on Windows PC, so I used dir instead of ls and I do see PairTrading but your command
> R CMD INSTALL PairTrading/ doesn't work:
>
> 'R' is not recognized as an internal or external command,operable program or batch file.
>
> Thanks again, Alec

It's more of a question for R-help, but:

If you follow the URL https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcran.r-project.org%2Fpackage%3DPairTrading&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Cc6a153bd55104ef7910208d85eb87a52%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363493716373736&amp;sdata=bsmkUniLSnS6ofp73MaG0DYToVii7VEZbzIgzZcAjR8%3D&amp;reserved=0 ,
you'll find that the package has been archived. You can still follow the
archive link, download the package and then install (on Windows) from
the R-GUI by selecting "install from local files" in the packages menu.
(Keep in mind that packages are archived for a reason, usually.)

If you want to try Daniel's suggestion: That 'R' is not found tells you
it's not in the search path.  You can set it on Windows in the
system/environment variable settings; just add 'C:/Program Files/R
<....>/bin ' (or whereever R is installed on your system) to the path.


kind regards
    Enrico

> ________________________________
> From: Daniel Cegiełka <[hidden email]>
> Sent: Monday, September 21, 2020 5:01 PM
> To: Alec Schmidt <[hidden email]>
> Cc: [hidden email] <[hidden email]>
> Subject: Re: [R-SIG-Finance] PairTrading package
>
> Hi Alec,
>
> $ R --version
> R version 4.0.0 (2020-04-24) -- "Arbor Day"
> Copyright (C) 2020 The R Foundation for Statistical Computing
> Platform: x86_64-apple-darwin17.0 (64-bit)
>
> R is free software and comes with ABSOLUTELY NO WARRANTY.
> You are welcome to redistribute it under the terms of the
> GNU General Public License versions 2 or 3.
> For more information about these matters see
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.gnu.org%2Flicenses%2F&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Cc6a153bd55104ef7910208d85eb87a52%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363493716383730&amp;sdata=55EEnTvuJH429ccclZvkkx1%2BWWejUD5Qy2b17UzGAis%3D&amp;reserved=0.
>
> $ git clone
> https://nam02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fcran%2FPairTrading&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Cc6a153bd55104ef7910208d85eb87a52%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363493716383730&amp;sdata=erwDXdwA44XFD93I1vBXTtmm819Mb9PUQvGvo0kdm%2FM%3D&amp;reserved=0
> Cloning into 'PairTrading'...
> remote: Enumerating objects: 39, done.
> remote: Counting objects: 100% (39/39), done.
> remote: Compressing objects: 100% (24/24), done.
> remote: Total 39 (delta 11), reused 39 (delta 11), pack-reused 0
> Unpacking objects: 100% (39/39), done.
>
> $ ls
> PairTrading
>
> $ R CMD INSTALL PairTrading/
> * installing to library ‘/Library/Frameworks/R.framework/Versions/4.0/Resources/library’
> * installing *source* package ‘PairTrading’ ...
>
> (…)
>
> ** building package indices
> ** testing if installed package can be loaded from temporary location
> ** testing if installed package can be loaded from final location
> ** testing if installed package keeps a record of temporary installation path
> * DONE (PairTrading)
>
>
> Best regards,
> Daniel
>
>
>> On 21 Sep 2020, at 22:38, Alec Schmidt <[hidden email]> wrote:
>>
>> I used to have R version 3.6.0 and tried to install PairTrading but got a message that the package is not available for that version.
>> Now I've updated R to 4.0.2. but still have the message:
>>
>> package �PairTrading� is not available (for R version 4.0.2)
>>
>> I wonder if anything can be done about it or there may be other packages with similar functionality.
>>
>> Thanks! Alec
>>

[...]


--
Enrico Schumann
Lucerne, Switzerland
https://nam02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fenricoschumann.net%2F&amp;data=02%7C01%7Caschmid1%40stevens.edu%7Cc6a153bd55104ef7910208d85eb87a52%7C8d1a69ec03b54345ae21dad112f5fb4f%7C0%7C0%7C637363493716383730&amp;sdata=sEkds70iDxr5HsNMuTycpt5bLSVl%2BA35S7GNbheGi7Y%3D&amp;reserved=0

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Web Scraping of SPY Stocks

Rmetrics mailing list
Dear All,
Please I need help. I ran the below function and got the highlighted error message. How can I correct it?
library(rvest)# Web-scrape SP500 stock listsp_500 <- read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%html_node("table.wikitable") %>%html_table() %>%select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%as_tibble()# Format namesnames(sp_500) <- sp_500 %>%names() %>%str_to_lower() %>%make.names()# Show resultssp_500
Error Message:
Error: Can't subset columns that don't exist.x Column `Ticker symbol` doesn't exist.Run `rlang::last_error()` to see where the error occurred.

The second function and the error is below:
get_stock_prices <- function(ticker), return_format = "tibble", ...) {  # Get stock prices  stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)  # Rename  names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")  # Return in xts format if tibble is not specified  if (return_format == "tibble") {    stock_prices <- stock_prices_xts %>%      as_tibble() %>%      rownames_to_column(var = "Date") %>%      mutate(Date = mdy(Date))  } else {    stock_prices <- stock_prices_xts  }  stock_prices}
"MA" %>%  get_stock_prices(return_format = 'tibble')%>%head()
ERROR MESSAGE:
Warning message:All formats failed to parse. No formats found. 


Thank You and Best Regards, Emeka I. AtumaIntegrity - Walk Your Talk Don't Talk Your Work

   
 
        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Web Scraping of SPY Stocks

Joshua Ulrich
Please repost your message in plain text (as the posting guide
requests).  As you can see below, your HTML message was mangled when
the list server converted it to plain text.

On Fri, Sep 25, 2020 at 3:12 PM AIE ATUMA via R-SIG-Finance
<[hidden email]> wrote:

>
> Dear All,
> Please I need help. I ran the below function and got the highlighted error message. How can I correct it?
> library(rvest)# Web-scrape SP500 stock listsp_500 <- read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%html_node("table.wikitable") %>%html_table() %>%select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%as_tibble()# Format namesnames(sp_500) <- sp_500 %>%names() %>%str_to_lower() %>%make.names()# Show resultssp_500
> Error Message:
> Error: Can't subset columns that don't exist.x Column `Ticker symbol` doesn't exist.Run `rlang::last_error()` to see where the error occurred.
>
> The second function and the error is below:
> get_stock_prices <- function(ticker), return_format = "tibble", ...) {  # Get stock prices  stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)  # Rename  names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")  # Return in xts format if tibble is not specified  if (return_format == "tibble") {    stock_prices <- stock_prices_xts %>%      as_tibble() %>%      rownames_to_column(var = "Date") %>%      mutate(Date = mdy(Date))  } else {    stock_prices <- stock_prices_xts  }  stock_prices}
> "MA" %>%  get_stock_prices(return_format = 'tibble')%>%head()
> ERROR MESSAGE:
> Warning message:All formats failed to parse. No formats found.
>
>
> Thank You and Best Regards, Emeka I. AtumaIntegrity - Walk Your Talk Don't Talk Your Work
>
>
>
>         [[alternative HTML version deleted]]
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.



--
Joshua Ulrich  |  about.me/joshuaulrich
FOSS Trading  |  www.fosstrading.com

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Web Scraping of SPY Stocks

Ezra Tucker-2
Hey there,

The error message has it right, there is no column "Ticker Symbol" in that table. When you look at the wikipedia page, the column's name is simply "Symbol".

For general debugging when using magrittr pipes I find it useful to copy/paste each expression, adding one additional pipe at a time to see at which stage the error message. Also, please make sure to include all of the libraries you were using- in this case, appears to be rvest, magrittr, and stringr.

With regards to that function, agreed with Joshua- if the above doesn't resolve all your issues, please re-post without html formatting as it's tough to see what's going on.

Hope that helps!

-Ezra

--
[hidden email]
m: 818-203-0269
LinkedIn: linkedin.com/in/ezztucker
Github: github.com/minimenchmuncher

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Saturday, September 26, 2020 8:34 AM, Joshua Ulrich <[hidden email]> wrote:

> Please repost your message in plain text (as the posting guide
> requests). As you can see below, your HTML message was mangled when
> the list server converted it to plain text.
>
> On Fri, Sep 25, 2020 at 3:12 PM AIE ATUMA via R-SIG-Finance
> [hidden email] wrote:
>
> > Dear All,
> > Please I need help. I ran the below function and got the highlighted error message. How can I correct it?
> > library(rvest)# Web-scrape SP500 stock listsp_500 <- read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%html_node("table.wikitable") %>%html_table() %>%select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%as_tibble()# Format namesnames(sp_500) <- sp_500 %>%names() %>%str_to_lower() %>%make.names()# Show resultssp_500
> > Error Message:
> > Error: Can't subset columns that don't exist.x Column `Ticker symbol` doesn't exist.Run `rlang::last_error()` to see where the error occurred.
> > The second function and the error is below:
> > get_stock_prices <- function(ticker), return_format = "tibble", ...) { # Get stock prices stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...) # Rename names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted") # Return in xts format if tibble is not specified if (return_format == "tibble") { stock_prices <- stock_prices_xts %>% as_tibble() %>% rownames_to_column(var = "Date") %>% mutate(Date = mdy(Date)) } else { stock_prices <- stock_prices_xts } stock_prices}
> > "MA" %>% get_stock_prices(return_format = 'tibble')%>%head()
> > ERROR MESSAGE:
> > Warning message:All formats failed to parse. No formats found.
> > Thank You and Best Regards, Emeka I. AtumaIntegrity - Walk Your Talk Don't Talk Your Work
> >
> >         [[alternative HTML version deleted]]
> >
> >
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R questions should go.
>
> --
>
> Joshua Ulrich | about.me/joshuaulrich
> FOSS Trading | www.fosstrading.com
>
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Web Scraping of SPY Stocks

Rmetrics mailing list
In reply to this post by Rmetrics mailing list
Dear All,

Please I need help. I ran the below function and got the highlighted error message. How can I correct it?

library(rvest)
library(rvest)
library(pbapply)
library(TTR)
library(dygraphs)
library(lubridate)
library(tidyquant)
library(timetk)
pacman::p_load(dygraphs,DT,quantmod)


# Web-scrape SP500 stock list
sp_500 <- read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
html_node("table.wikitable") %>%
html_table() %>%
select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
as_tibble()
# Format names
names(sp_500) <- sp_500 %>%
names() %>%
str_to_lower() %>%
make.names()
# Show results
sp_500

Error Message:

Error: Can't subset columns that don't exist.
x Column `Ticker symbol` doesn't exist.
Run `rlang::last_error()` to see where the error occurred.


The second function and the error is below:

get_stock_prices <- function(ticker), return_format = "tibble", ...) {
  # Get stock prices
  stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)
  # Rename
  names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")
  # Return in xts format if tibble is not specified
  if (return_format == "tibble") {
    stock_prices <- stock_prices_xts %>%
      as_tibble() %>%
      rownames_to_column(var = "Date") %>%
      mutate(Date = mdy(Date))
  } else {
    stock_prices <- stock_prices_xts
  }
  stock_prices
}

"MA" %>%
  get_stock_prices(return_format = 'tibble')%>%head()

ERROR MESSAGE:

Warning message:
All formats failed to parse. No formats found. 



Thank You and Best Regards, 
Emeka I. Atuma
Integrity - Walk Your Talk Don't Talk Your Work






On Friday, 25 September 2020, 21:12:24 GMT+1, AIE ATUMA <[hidden email]> wrote:





Dear All,

Please I need help. I ran the below function and got the highlighted error message. How can I correct it?

library(rvest)
# Web-scrape SP500 stock list
sp_500 <- read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
html_node("table.wikitable") %>%
html_table() %>%
select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
as_tibble()
# Format names
names(sp_500) <- sp_500 %>%
names() %>%
str_to_lower() %>%
make.names()
# Show results
sp_500

Error Message:

Error: Can't subset columns that don't exist.
x Column `Ticker symbol` doesn't exist.
Run `rlang::last_error()` to see where the error occurred.


The second function and the error is below:

get_stock_prices <- function(ticker), return_format = "tibble", ...) {
  # Get stock prices
  stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)
  # Rename
  names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")
  # Return in xts format if tibble is not specified
  if (return_format == "tibble") {
    stock_prices <- stock_prices_xts %>%
      as_tibble() %>%
      rownames_to_column(var = "Date") %>%
      mutate(Date = mdy(Date))
  } else {
    stock_prices <- stock_prices_xts
  }
  stock_prices
}

"MA" %>%
  get_stock_prices(return_format = 'tibble')%>%head()

ERROR MESSAGE:

Warning message:
All formats failed to parse. No formats found. 



Thank You and Best Regards, 
Emeka I. Atuma
Integrity - Walk Your Talk Don't Talk Your Work

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Web Scraping of SPY Stocks

Rmetrics mailing list
Update:

The second function and the error is below:

get_stock_prices <- function(ticker), return_format = "tibble", ...) {
  # Get stock prices
  stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)
  # Rename
  names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")
  # Return in xts format if tibble is not specified
  if (return_format == "tibble") {
    stock_prices <- stock_prices_xts %>%
      as_tibble() %>%
      rownames_to_column(var = "Date") %>%
      mutate(Date = mdy(Date))
  } else {
    stock_prices <- stock_prices_xts
  }
  stock_prices
}

"MA" %>%
  get_stock_prices(return_format = 'tibble')%>%head()

ERROR MESSAGE:

# A tibble: 6 x 7
  Date        Open  High   Low Close   Volume Adjusted
  <date>     <dbl> <dbl> <dbl> <dbl>    <dbl>    <dbl>
1 NA          9.96  9.97  9.56  9.64 26289000     8.98
2 NA          9.69 10.2   9.53 10.1  27024000     9.42
3 NA         10.1  10.2   9.9  10.1  29632000     9.41
4 NA          9.90 10.2   9.9  10.1  16006000     9.41
5 NA         10.1  10.6  10.1  10.6  36952000     9.85
6 NA         10.6  10.6  10.3  10.5  35099000     9.76
Warning message:
All formats failed to parse. No formats found.



Thank You and Best Regards, 
Emeka I. Atuma
Integrity - Walk Your Talk Don't Talk Your Work






On Saturday, 26 September 2020, 18:09:51 GMT+1, AIE ATUMA via R-SIG-Finance <[hidden email]> wrote:





Dear All,

Please I need help. I ran the below function and got the highlighted error message. How can I correct it?

library(rvest)
library(rvest)
library(pbapply)
library(TTR)
library(dygraphs)
library(lubridate)
library(tidyquant)
library(timetk)
pacman::p_load(dygraphs,DT,quantmod)


# Web-scrape SP500 stock list
sp_500 <- read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
html_node("table.wikitable") %>%
html_table() %>%
select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
as_tibble()
# Format names
names(sp_500) <- sp_500 %>%
names() %>%
str_to_lower() %>%
make.names()
# Show results
sp_500

Error Message:

Error: Can't subset columns that don't exist.
x Column `Ticker symbol` doesn't exist.
Run `rlang::last_error()` to see where the error occurred.


The second function and the error is below:

get_stock_prices <- function(ticker), return_format = "tibble", ...) {
  # Get stock prices
  stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)
  # Rename
  names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")
  # Return in xts format if tibble is not specified
  if (return_format == "tibble") {
    stock_prices <- stock_prices_xts %>%
      as_tibble() %>%
      rownames_to_column(var = "Date") %>%
      mutate(Date = mdy(Date))
  } else {
    stock_prices <- stock_prices_xts
  }
  stock_prices
}

"MA" %>%
  get_stock_prices(return_format = 'tibble')%>%head()

ERROR MESSAGE:

Warning message:
All formats failed to parse. No formats found. 



Thank You and Best Regards, 
Emeka I. Atuma
Integrity - Walk Your Talk Don't Talk Your Work






On Friday, 25 September 2020, 21:12:24 GMT+1, AIE ATUMA <[hidden email]> wrote:





Dear All,

Please I need help. I ran the below function and got the highlighted error message. How can I correct it?

library(rvest)
# Web-scrape SP500 stock list
sp_500 <- read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
html_node("table.wikitable") %>%
html_table() %>%
select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
as_tibble()
# Format names
names(sp_500) <- sp_500 %>%
names() %>%
str_to_lower() %>%
make.names()
# Show results
sp_500

Error Message:

Error: Can't subset columns that don't exist.
x Column `Ticker symbol` doesn't exist.
Run `rlang::last_error()` to see where the error occurred.


The second function and the error is below:

get_stock_prices <- function(ticker), return_format = "tibble", ...) {
  # Get stock prices
  stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)
  # Rename
  names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")
  # Return in xts format if tibble is not specified
  if (return_format == "tibble") {
    stock_prices <- stock_prices_xts %>%
      as_tibble() %>%
      rownames_to_column(var = "Date") %>%
      mutate(Date = mdy(Date))
  } else {
    stock_prices <- stock_prices_xts
  }
  stock_prices
}

"MA" %>%
  get_stock_prices(return_format = 'tibble')%>%head()

ERROR MESSAGE:

Warning message:
All formats failed to parse. No formats found. 



Thank You and Best Regards, 
Emeka I. Atuma
Integrity - Walk Your Talk Don't Talk Your Work


_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Web Scraping of SPY Stocks

Matt Cleary
Hi Emeka,

When you convert the dataset to a tibble you are losing the date values
stored in the rownames. I'd recommend storing the dates externally in the
function before before making the conversion then adding them back with the
mutate function:

library(quantmod)
library(dplyr)
library(tibble)
library(rvest)
library(lubridate)

# Web-scrape SP500 stock list
sp_500 <- read_html("
https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
  html_node("table.wikitable") %>%
  html_table() %>%
  select(`Symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
  as_tibble()

# Format names
names(sp_500) <- sp_500 %>%
  names() %>%
  str_to_lower() %>%
  make.names()

# Show results
sp_500

get_stock_prices <- function(ticker, return_format = "tibble", ...){

  # Get stock prices
  stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)

  dates <- as.Date(rownames(as.matrix(stock_prices_xts)))

  # Rename
  names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume",
"Adjusted")

  # Return in xts format if tibble is not specified
  if (return_format == "tibble") {
    stock_prices <- stock_prices_xts %>%
      as_tibble() %>%
      mutate(Date = dates)
  } else {
    stock_prices <- stock_prices_xts
  }
  stock_prices
}

"MA" %>%
  get_stock_prices(return_format = 'tibble') %>%
  head()

Best,

Matt

On Sat, Sep 26, 2020 at 1:34 PM AIE ATUMA via R-SIG-Finance <
[hidden email]> wrote:

> Update:
>
> The second function and the error is below:
>
> get_stock_prices <- function(ticker), return_format = "tibble", ...) {
>   # Get stock prices
>   stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE,
> ...)
>   # Rename
>   names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume",
> "Adjusted")
>   # Return in xts format if tibble is not specified
>   if (return_format == "tibble") {
>     stock_prices <- stock_prices_xts %>%
>       as_tibble() %>%
>       rownames_to_column(var = "Date") %>%
>       mutate(Date = mdy(Date))
>   } else {
>     stock_prices <- stock_prices_xts
>   }
>   stock_prices
> }
>
> "MA" %>%
>   get_stock_prices(return_format = 'tibble')%>%head()
>
> ERROR MESSAGE:
>
> # A tibble: 6 x 7
>   Date        Open  High   Low Close   Volume Adjusted
>   <date>     <dbl> <dbl> <dbl> <dbl>    <dbl>    <dbl>
> 1 NA          9.96  9.97  9.56  9.64 26289000     8.98
> 2 NA          9.69 10.2   9.53 10.1  27024000     9.42
> 3 NA         10.1  10.2   9.9  10.1  29632000     9.41
> 4 NA          9.90 10.2   9.9  10.1  16006000     9.41
> 5 NA         10.1  10.6  10.1  10.6  36952000     9.85
> 6 NA         10.6  10.6  10.3  10.5  35099000     9.76
> Warning message:
> All formats failed to parse. No formats found.
>
>
>
> Thank You and Best Regards,
> Emeka I. Atuma
> Integrity - Walk Your Talk Don't Talk Your Work
>
>
>
>
>
>
> On Saturday, 26 September 2020, 18:09:51 GMT+1, AIE ATUMA via
> R-SIG-Finance <[hidden email]> wrote:
>
>
>
>
>
> Dear All,
>
> Please I need help. I ran the below function and got the highlighted error
> message. How can I correct it?
>
> library(rvest)
> library(rvest)
> library(pbapply)
> library(TTR)
> library(dygraphs)
> library(lubridate)
> library(tidyquant)
> library(timetk)
> pacman::p_load(dygraphs,DT,quantmod)
>
>
> # Web-scrape SP500 stock list
> sp_500 <- read_html("
> https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
> html_node("table.wikitable") %>%
> html_table() %>%
> select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
> as_tibble()
> # Format names
> names(sp_500) <- sp_500 %>%
> names() %>%
> str_to_lower() %>%
> make.names()
> # Show results
> sp_500
>
> Error Message:
>
> Error: Can't subset columns that don't exist.
> x Column `Ticker symbol` doesn't exist.
> Run `rlang::last_error()` to see where the error occurred.
>
>
> The second function and the error is below:
>
> get_stock_prices <- function(ticker), return_format = "tibble", ...) {
>   # Get stock prices
>   stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE,
> ...)
>   # Rename
>   names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume",
> "Adjusted")
>   # Return in xts format if tibble is not specified
>   if (return_format == "tibble") {
>     stock_prices <- stock_prices_xts %>%
>       as_tibble() %>%
>       rownames_to_column(var = "Date") %>%
>       mutate(Date = mdy(Date))
>   } else {
>     stock_prices <- stock_prices_xts
>   }
>   stock_prices
> }
>
> "MA" %>%
>   get_stock_prices(return_format = 'tibble')%>%head()
>
> ERROR MESSAGE:
>
> Warning message:
> All formats failed to parse. No formats found.
>
>
>
> Thank You and Best Regards,
> Emeka I. Atuma
> Integrity - Walk Your Talk Don't Talk Your Work
>
>
>
>
>
>
> On Friday, 25 September 2020, 21:12:24 GMT+1, AIE ATUMA <
> [hidden email]> wrote:
>
>
>
>
>
> Dear All,
>
> Please I need help. I ran the below function and got the highlighted error
> message. How can I correct it?
>
> library(rvest)
> # Web-scrape SP500 stock list
> sp_500 <- read_html("
> https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
> html_node("table.wikitable") %>%
> html_table() %>%
> select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
> as_tibble()
> # Format names
> names(sp_500) <- sp_500 %>%
> names() %>%
> str_to_lower() %>%
> make.names()
> # Show results
> sp_500
>
> Error Message:
>
> Error: Can't subset columns that don't exist.
> x Column `Ticker symbol` doesn't exist.
> Run `rlang::last_error()` to see where the error occurred.
>
>
> The second function and the error is below:
>
> get_stock_prices <- function(ticker), return_format = "tibble", ...) {
>   # Get stock prices
>   stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE,
> ...)
>   # Rename
>   names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume",
> "Adjusted")
>   # Return in xts format if tibble is not specified
>   if (return_format == "tibble") {
>     stock_prices <- stock_prices_xts %>%
>       as_tibble() %>%
>       rownames_to_column(var = "Date") %>%
>       mutate(Date = mdy(Date))
>   } else {
>     stock_prices <- stock_prices_xts
>   }
>   stock_prices
> }
>
> "MA" %>%
>   get_stock_prices(return_format = 'tibble')%>%head()
>
> ERROR MESSAGE:
>
> Warning message:
> All formats failed to parse. No formats found.
>
>
>
> Thank You and Best Regards,
> Emeka I. Atuma
> Integrity - Walk Your Talk Don't Talk Your Work
>
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions
> should go.
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Web Scraping of SPY Stocks

Rmetrics mailing list
Dear Matt,

This worked but the Date column is presented as the last column as against the second column.

Thank You and Best Regards, 
Emeka I. Atuma
Integrity - Walk Your Talk Don't Talk Your Work






On Saturday, 26 September 2020, 19:19:28 GMT+1, Matt Cleary <[hidden email]> wrote:





Hi Emeka,

When you convert the dataset to a tibble you are losing the date values stored in the rownames. I'd recommend storing the dates externally in the function before before making the conversion then adding them back with the mutate function:

library(quantmod)
library(dplyr)
library(tibble)
library(rvest)
library(lubridate)

# Web-scrape SP500 stock list
sp_500 <- read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
  html_node("table.wikitable") %>%
  html_table() %>%
  select(`Symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
  as_tibble()

# Format names
names(sp_500) <- sp_500 %>%
  names() %>%
  str_to_lower() %>%
  make.names()

# Show results
sp_500

get_stock_prices <- function(ticker, return_format = "tibble", ...){
 
  # Get stock prices
  stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)
 
  dates <- as.Date(rownames(as.matrix(stock_prices_xts)))
 
  # Rename
  names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")
 
  # Return in xts format if tibble is not specified
  if (return_format == "tibble") {
    stock_prices <- stock_prices_xts %>%
      as_tibble() %>%
      mutate(Date = dates)
  } else {
    stock_prices <- stock_prices_xts
  }
  stock_prices
}

"MA" %>%
  get_stock_prices(return_format = 'tibble') %>%
  head()

Best, 

Matt

On Sat, Sep 26, 2020 at 1:34 PM AIE ATUMA via R-SIG-Finance <[hidden email]> wrote:

> Update:
>
> The second function and the error is below:
>
> get_stock_prices <- function(ticker), return_format = "tibble", ...) {
>   # Get stock prices
>   stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)
>   # Rename
>   names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")
>   # Return in xts format if tibble is not specified
>   if (return_format == "tibble") {
>     stock_prices <- stock_prices_xts %>%
>       as_tibble() %>%
>       rownames_to_column(var = "Date") %>%
>       mutate(Date = mdy(Date))
>   } else {
>     stock_prices <- stock_prices_xts
>   }
>   stock_prices
> }
>
> "MA" %>%
>   get_stock_prices(return_format = 'tibble')%>%head()
>
> ERROR MESSAGE:
>
> # A tibble: 6 x 7
>   Date        Open  High   Low Close   Volume Adjusted
>   <date>     <dbl> <dbl> <dbl> <dbl>    <dbl>    <dbl>
> 1 NA          9.96  9.97  9.56  9.64 26289000     8.98
> 2 NA          9.69 10.2   9.53 10.1  27024000     9.42
> 3 NA         10.1  10.2   9.9  10.1  29632000     9.41
> 4 NA          9.90 10.2   9.9  10.1  16006000     9.41
> 5 NA         10.1  10.6  10.1  10.6  36952000     9.85
> 6 NA         10.6  10.6  10.3  10.5  35099000     9.76
> Warning message:
> All formats failed to parse. No formats found.
>
>
>
> Thank You and Best Regards, 
> Emeka I. Atuma
> Integrity - Walk Your Talk Don't Talk Your Work
>
>
>
>
>
>
> On Saturday, 26 September 2020, 18:09:51 GMT+1, AIE ATUMA via R-SIG-Finance <[hidden email]> wrote:
>
>
>
>
>
> Dear All,
>
> Please I need help. I ran the below function and got the highlighted error message. How can I correct it?
>
> library(rvest)
> library(rvest)
> library(pbapply)
> library(TTR)
> library(dygraphs)
> library(lubridate)
> library(tidyquant)
> library(timetk)
> pacman::p_load(dygraphs,DT,quantmod)
>
>
> # Web-scrape SP500 stock list
> sp_500 <- read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
> html_node("table.wikitable") %>%
> html_table() %>%
> select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
> as_tibble()
> # Format names
> names(sp_500) <- sp_500 %>%
> names() %>%
> str_to_lower() %>%
> make.names()
> # Show results
> sp_500
>
> Error Message:
>
> Error: Can't subset columns that don't exist.
> x Column `Ticker symbol` doesn't exist.
> Run `rlang::last_error()` to see where the error occurred.
>
>
> The second function and the error is below:
>
> get_stock_prices <- function(ticker), return_format = "tibble", ...) {
>   # Get stock prices
>   stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)
>   # Rename
>   names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")
>   # Return in xts format if tibble is not specified
>   if (return_format == "tibble") {
>     stock_prices <- stock_prices_xts %>%
>       as_tibble() %>%
>       rownames_to_column(var = "Date") %>%
>       mutate(Date = mdy(Date))
>   } else {
>     stock_prices <- stock_prices_xts
>   }
>   stock_prices
> }
>
> "MA" %>%
>   get_stock_prices(return_format = 'tibble')%>%head()
>
> ERROR MESSAGE:
>
> Warning message:
> All formats failed to parse. No formats found. 
>
>
>
> Thank You and Best Regards, 
> Emeka I. Atuma
> Integrity - Walk Your Talk Don't Talk Your Work
>
>
>
>
>
>
> On Friday, 25 September 2020, 21:12:24 GMT+1, AIE ATUMA <[hidden email]> wrote:
>
>
>
>
>
> Dear All,
>
> Please I need help. I ran the below function and got the highlighted error message. How can I correct it?
>
> library(rvest)
> # Web-scrape SP500 stock list
> sp_500 <- read_html("https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
> html_node("table.wikitable") %>%
> html_table() %>%
> select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
> as_tibble()
> # Format names
> names(sp_500) <- sp_500 %>%
> names() %>%
> str_to_lower() %>%
> make.names()
> # Show results
> sp_500
>
> Error Message:
>
> Error: Can't subset columns that don't exist.
> x Column `Ticker symbol` doesn't exist.
> Run `rlang::last_error()` to see where the error occurred.
>
>
> The second function and the error is below:
>
> get_stock_prices <- function(ticker), return_format = "tibble", ...) {
>   # Get stock prices
>   stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE, ...)
>   # Rename
>   names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume", "Adjusted")
>   # Return in xts format if tibble is not specified
>   if (return_format == "tibble") {
>     stock_prices <- stock_prices_xts %>%
>       as_tibble() %>%
>       rownames_to_column(var = "Date") %>%
>       mutate(Date = mdy(Date))
>   } else {
>     stock_prices <- stock_prices_xts
>   }
>   stock_prices
> }
>
> "MA" %>%
>   get_stock_prices(return_format = 'tibble')%>%head()
>
> ERROR MESSAGE:
>
> Warning message:
> All formats failed to parse. No formats found. 
>
>
>
> Thank You and Best Regards, 
> Emeka I. Atuma
> Integrity - Walk Your Talk Don't Talk Your Work
>
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.
>
> _______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> -- Subscriber-posting only. If you want to post, subscribe first.
> -- Also note that this is not the r-help list where general R questions should go.
>

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.
Reply | Threaded
Open this post in threaded view
|

Re: Web Scraping of SPY Stocks

Matt Cleary
You can use a select() function after the mutate function to rearrange the
columns if need be.

On Sat, Sep 26, 2020, 5:11 PM AIE ATUMA <[hidden email]> wrote:

> Dear Matt,
>
> This worked but the Date column is presented as the last column as against
> the second column.
>
> Thank You and Best Regards,
> Emeka I. Atuma
> Integrity - Walk Your Talk Don't Talk Your Work
>
>
>
>
>
>
> On Saturday, 26 September 2020, 19:19:28 GMT+1, Matt Cleary <
> [hidden email]> wrote:
>
>
>
>
>
> Hi Emeka,
>
> When you convert the dataset to a tibble you are losing the date values
> stored in the rownames. I'd recommend storing the dates externally in the
> function before before making the conversion then adding them back with the
> mutate function:
>
> library(quantmod)
> library(dplyr)
> library(tibble)
> library(rvest)
> library(lubridate)
>
> # Web-scrape SP500 stock list
> sp_500 <- read_html("
> https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
>   html_node("table.wikitable") %>%
>   html_table() %>%
>   select(`Symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
>   as_tibble()
>
> # Format names
> names(sp_500) <- sp_500 %>%
>   names() %>%
>   str_to_lower() %>%
>   make.names()
>
> # Show results
> sp_500
>
> get_stock_prices <- function(ticker, return_format = "tibble", ...){
>
>   # Get stock prices
>   stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE,
> ...)
>
>   dates <- as.Date(rownames(as.matrix(stock_prices_xts)))
>
>   # Rename
>   names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume",
> "Adjusted")
>
>   # Return in xts format if tibble is not specified
>   if (return_format == "tibble") {
>     stock_prices <- stock_prices_xts %>%
>       as_tibble() %>%
>       mutate(Date = dates)
>   } else {
>     stock_prices <- stock_prices_xts
>   }
>   stock_prices
> }
>
> "MA" %>%
>   get_stock_prices(return_format = 'tibble') %>%
>   head()
>
> Best,
>
> Matt
>
> On Sat, Sep 26, 2020 at 1:34 PM AIE ATUMA via R-SIG-Finance <
> [hidden email]> wrote:
> > Update:
> >
> > The second function and the error is below:
> >
> > get_stock_prices <- function(ticker), return_format = "tibble", ...) {
> >   # Get stock prices
> >   stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE,
> ...)
> >   # Rename
> >   names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume",
> "Adjusted")
> >   # Return in xts format if tibble is not specified
> >   if (return_format == "tibble") {
> >     stock_prices <- stock_prices_xts %>%
> >       as_tibble() %>%
> >       rownames_to_column(var = "Date") %>%
> >       mutate(Date = mdy(Date))
> >   } else {
> >     stock_prices <- stock_prices_xts
> >   }
> >   stock_prices
> > }
> >
> > "MA" %>%
> >   get_stock_prices(return_format = 'tibble')%>%head()
> >
> > ERROR MESSAGE:
> >
> > # A tibble: 6 x 7
> >   Date        Open  High   Low Close   Volume Adjusted
> >   <date>     <dbl> <dbl> <dbl> <dbl>    <dbl>    <dbl>
> > 1 NA          9.96  9.97  9.56  9.64 26289000     8.98
> > 2 NA          9.69 10.2   9.53 10.1  27024000     9.42
> > 3 NA         10.1  10.2   9.9  10.1  29632000     9.41
> > 4 NA          9.90 10.2   9.9  10.1  16006000     9.41
> > 5 NA         10.1  10.6  10.1  10.6  36952000     9.85
> > 6 NA         10.6  10.6  10.3  10.5  35099000     9.76
> > Warning message:
> > All formats failed to parse. No formats found.
> >
> >
> >
> > Thank You and Best Regards,
> > Emeka I. Atuma
> > Integrity - Walk Your Talk Don't Talk Your Work
> >
> >
> >
> >
> >
> >
> > On Saturday, 26 September 2020, 18:09:51 GMT+1, AIE ATUMA via
> R-SIG-Finance <[hidden email]> wrote:
> >
> >
> >
> >
> >
> > Dear All,
> >
> > Please I need help. I ran the below function and got the highlighted
> error message. How can I correct it?
> >
> > library(rvest)
> > library(rvest)
> > library(pbapply)
> > library(TTR)
> > library(dygraphs)
> > library(lubridate)
> > library(tidyquant)
> > library(timetk)
> > pacman::p_load(dygraphs,DT,quantmod)
> >
> >
> > # Web-scrape SP500 stock list
> > sp_500 <- read_html("
> https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
> > html_node("table.wikitable") %>%
> > html_table() %>%
> > select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
> > as_tibble()
> > # Format names
> > names(sp_500) <- sp_500 %>%
> > names() %>%
> > str_to_lower() %>%
> > make.names()
> > # Show results
> > sp_500
> >
> > Error Message:
> >
> > Error: Can't subset columns that don't exist.
> > x Column `Ticker symbol` doesn't exist.
> > Run `rlang::last_error()` to see where the error occurred.
> >
> >
> > The second function and the error is below:
> >
> > get_stock_prices <- function(ticker), return_format = "tibble", ...) {
> >   # Get stock prices
> >   stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE,
> ...)
> >   # Rename
> >   names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume",
> "Adjusted")
> >   # Return in xts format if tibble is not specified
> >   if (return_format == "tibble") {
> >     stock_prices <- stock_prices_xts %>%
> >       as_tibble() %>%
> >       rownames_to_column(var = "Date") %>%
> >       mutate(Date = mdy(Date))
> >   } else {
> >     stock_prices <- stock_prices_xts
> >   }
> >   stock_prices
> > }
> >
> > "MA" %>%
> >   get_stock_prices(return_format = 'tibble')%>%head()
> >
> > ERROR MESSAGE:
> >
> > Warning message:
> > All formats failed to parse. No formats found.
> >
> >
> >
> > Thank You and Best Regards,
> > Emeka I. Atuma
> > Integrity - Walk Your Talk Don't Talk Your Work
> >
> >
> >
> >
> >
> >
> > On Friday, 25 September 2020, 21:12:24 GMT+1, AIE ATUMA <
> [hidden email]> wrote:
> >
> >
> >
> >
> >
> > Dear All,
> >
> > Please I need help. I ran the below function and got the highlighted
> error message. How can I correct it?
> >
> > library(rvest)
> > # Web-scrape SP500 stock list
> > sp_500 <- read_html("
> https://en.wikipedia.org/wiki/List_of_S%26P_500_companies") %>%
> > html_node("table.wikitable") %>%
> > html_table() %>%
> > select(`Ticker symbol`, Security, `GICS Sector`, `GICS Sub Industry`) %>%
> > as_tibble()
> > # Format names
> > names(sp_500) <- sp_500 %>%
> > names() %>%
> > str_to_lower() %>%
> > make.names()
> > # Show results
> > sp_500
> >
> > Error Message:
> >
> > Error: Can't subset columns that don't exist.
> > x Column `Ticker symbol` doesn't exist.
> > Run `rlang::last_error()` to see where the error occurred.
> >
> >
> > The second function and the error is below:
> >
> > get_stock_prices <- function(ticker), return_format = "tibble", ...) {
> >   # Get stock prices
> >   stock_prices_xts <- getSymbols(Symbols = ticker, auto.assign = FALSE,
> ...)
> >   # Rename
> >   names(stock_prices_xts) <- c("Open", "High", "Low", "Close", "Volume",
> "Adjusted")
> >   # Return in xts format if tibble is not specified
> >   if (return_format == "tibble") {
> >     stock_prices <- stock_prices_xts %>%
> >       as_tibble() %>%
> >       rownames_to_column(var = "Date") %>%
> >       mutate(Date = mdy(Date))
> >   } else {
> >     stock_prices <- stock_prices_xts
> >   }
> >   stock_prices
> > }
> >
> > "MA" %>%
> >   get_stock_prices(return_format = 'tibble')%>%head()
> >
> > ERROR MESSAGE:
> >
> > Warning message:
> > All formats failed to parse. No formats found.
> >
> >
> >
> > Thank You and Best Regards,
> > Emeka I. Atuma
> > Integrity - Walk Your Talk Don't Talk Your Work
> >
> >
> > _______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R questions
> should go.
> >
> > _______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-sig-finance
> > -- Subscriber-posting only. If you want to post, subscribe first.
> > -- Also note that this is not the r-help list where general R questions
> should go.
> >
>

        [[alternative HTML version deleted]]

_______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-sig-finance
-- Subscriber-posting only. If you want to post, subscribe first.
-- Also note that this is not the r-help list where general R questions should go.