Fwd: Re: [External] Re: read.csv fails in R console in Ubuntu terminal but works in RStudio after R 3.6.3 upgrade to R 4.0.2?

classic Classic list List threaded Threaded
1 message Options
Reply | Threaded
Open this post in threaded view
|

Fwd: Re: [External] Re: read.csv fails in R console in Ubuntu terminal but works in RStudio after R 3.6.3 upgrade to R 4.0.2?

Rui Barradas
Hello,

Sorry, once again, this time I forgot to cc the list.
See the answer to Martin Maechler's mail below.

Rui Barradas


-------- Mensagem reencaminhada --------
Assunto: Re: [R] [External] Re: read.csv fails in R console in Ubuntu
terminal but works in RStudio after R 3.6.3 upgrade to R 4.0.2?
Data: Fri, 17 Jul 2020 14:27:37 +0100
De: Rui Barradas <[hidden email]>
Para: Martin Maechler <[hidden email]>



Hello,

My apologies to the list.

I seldom use the cell phone from where I sent that e-mail and forgot to
check the settings.
The original text, updated since the OP already solved the problem was:

Yes, I thought it's a site policy issue too.
But the file can be accessed and read/downloaded from RStudio and
Firefox so apparently there's no reason why R console shouldn't.
Anyway, in the mean time the OP has solved the problem but not using
read.csv.

Rui Barradas

Às 10:21 de 17/07/2020, Martin Maechler escreveu:

> Dear Rui,
>
> I think you should try re-sending this, *not* using html ...
>
> Best,
>
> Martin
>
>
>
>>>>>> ruipbarradas
>>>>>> on Fri, 17 Jul 2020 09:30:35 +0100 writes:
> > Hello,Yes, I thought it's a site policy issue too. But the
> > file can be accessed and read/downloaded from RStudio and
> > Firefox so apparently there's no reason why R console
> > shouldn't .Anyway, I believe it's time for the OP to say
> > someyhing, maybe he has solved it and there's no point in
> > continuing.Rui Barradas Enviado a partir do meu smartphone
> > Samsung Galaxy.-------- Mensagem original --------De:
> > [hidden email] Data: 17/07/2020 02:59 (GMT+00:00)
> > Para: Ista Zahn <[hidden email]> Cc: Rui Barradas
> > <[hidden email]>, [hidden email], Sam H
> > <[hidden email]> Assunto: Re: [External] Re: [R]
> > read.csv fails in R console in Ubuntu terminal but works
> > in RStudio after R 3.6.3 upgrade to R 4.0.2? On my Ubuntu
> > system the download with read.csv succeeds in an Rconsole
> > if I set the HTTPUserAgent and download.file.method
> > options tomatch the ones used by RStudio.Given how picky
> > the server is being I would worry about whether thisuse is
> > in line with the site's terms of service.Best,lukeOn Thu,
> > 16 Jul 2020, Ista Zahn wrote:> On Thu, Jul 16, 2020 at
> > 5:15 PM Ista Zahn <[hidden email]> wrote:>>>> On Thu,
> > Jul 16, 2020 at 8:18 AM Rui Barradas
> > <[hidden email]> wrote:>>>>>> Hello,>>>>>> Thanks,
> > but no, download.file still gives 403 Forbidden with both
> > method>>> = "libcurl" and method = "wget".>>>> I think
> > that makes it "not an R question". Ask on>>
> > https://unix.stackexchange.com/ maybe?>> Oh, sorry I
> > misread your message. Nevertheless:>> $ curl
> >
> "https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download">
> > <HTML><HEAD>> <TITLE>Access Denied</TITLE>> </HEAD><BODY>>
> > <H1>Access Denied</H1>>> You don't have permission to
> > access>
> >
> "http&#58;&#47;&#47;old&#46;nasdaq&#46;com&#47;screening&#47;companies&#45;by&#45;name&#46;aspx&#63;">
> > on this server.<P>>
> > Reference&#32;&#35;18&#46;5506d217&#46;1594934303&#46;938edcb>
> > </BODY>> </HTML>>> $ wget
> >
> "https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download">
> > --2020-07-16 17:19:12-->
> >
> https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download>
> > Loaded CA certificate
> > '/etc/ssl/certs/ca-certificates.crt'> Resolving
> > old.nasdaq.com
> > (old.nasdaq.com)... 2600:1400:9000:28f::1b46,>
> > 2600:1400:9000:29b::1b46, 23.78.161.120> Connecting to
> > old.nasdaq.com>
> > (old.nasdaq.com)|2600:1400:9000:28f::1b46|:443... connected.>
> > HTTP request sent, awaiting response... 403 Forbidden>
> > 2020-07-16 17:19:12 ERROR 403: Forbidden.>> I don't think
> > this is an R problem.>> Best,> Ista>>>>> Best,>>
> > Ista>>>>>>>> Rui Barradas>>>>>> Às 05:31 de 16/07/20, Jeff
> > Newmiller escreveu:>>>> Perhaps read FAQ 7.43? [1]>>>>>>>>
> > [1]
> >
> https://cran.r-project.org/doc/FAQ/R-FAQ.html#How-can-I-enable-secure-https-downloads-in-R_003f>>>>>>>>
> > On July 15, 2020 4:02:27 PM PDT, Rui Barradas
> > <[hidden email]> wrote:>>>>> Hello,>>>>>>>>>> R
> > 4.0.2 on Ubuntu 20.04 LTS, sessionInfo below.>>>>>>>>>>
> > I'm also unable to read the file with Rscript from the
> > Ubuntu terminal>>>>> but the error is not the same as the
> > OP's.>>>>>>>>>>>>>>> The first try was a file test1.R with
> > the following commands.>>>>>>>>>>
> >
> x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download">>>>>
> > read.csv(x, as.is=TRUE, na="n/a")>>>>>>>>>>>>>>> And run
> > with Rscript>>>>>>>>>> rui@rui:~$ Rscript --vanilla
> > test1.R>>>>> Error in file(file, "rt") :>>>>>    cannot
> > open the connection to>>>>>
> >
> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download'>>>>>
> > Calls: read.csv -> read.table -> file>>>>> In addition:
> > Warning message:>>>>> In file(file, "rt") :>>>>>    cannot
> > open URL>>>>>
> >
> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download':>>>>>>>>>>
> > HTTP status was '403 Forbidden'>>>>> Execution
> > halted>>>>>>>>>>>>>>>>>>>> The second try was
> > download.file() and then read it.>>>>> File test2.R
> > is:>>>>>>>>>>
> >
> x<-"https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download">>>>>
> > download.file(x, "companylist.csv")>>>>>
> > read.csv("companylist.csv", as.is=TRUE,
> > na="n/a")>>>>>>>>>>>>>>> But this too failed with error
> > 403 Forbiden.>>>>>>>>>> rui@rui:~$ Rscript --vanilla
> > test2.R>>>>> trying URL>>>>>
> >
> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download'>>>>>
> > Error in download.file(x, "companylist.csv") :>>>>>
> > cannot open URL>>>>>
> >
> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download'>>>>>
> > In addition: Warning message:>>>>> In download.file(x,
> > "companylist.csv") :>>>>>    cannot open URL>>>>>
> >
> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download':>>>>>>>>>>
> > HTTP status was '403 Forbidden'>>>>> Execution
> > halted>>>>>>>>>>>>>>> This is my session info.>>>>>>>>>>
> > rui@rui:~$ Rscript --vanilla -e 'sessionInfo()'>>>>> R
> > version 4.0.2 (2020-06-22)>>>>> Platform:
> > x86_64-pc-linux-gnu (64-bit)>>>>> Running under: Ubuntu
> > 20.04 LTS>>>>>>>>>> Matrix products: default>>>>> BLAS:
> > /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0>>>>>
> > LAPACK:
> > /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0>>>>>>>>>>
> > locale:>>>>>   [1] LC_CTYPE=pt_PT.UTF-8
> > LC_NUMERIC=C>>>>>   [3] LC_TIME=pt_PT.UTF-8
> > LC_COLLATE=pt_PT.UTF-8>>>>>   [5]
> > LC_MONETARY=pt_PT.UTF-8 LC_MESSAGES=pt_PT.UTF-8>>>>>
> > [7] LC_PAPER=pt_PT.UTF-8 LC_NAME=C>>>>>   [9]
> > LC_ADDRESS=C LC_TELEPHONE=C>>>>> [11]
> > LC_MEASUREMENT=pt_PT.UTF-8 LC_IDENTIFICATION=C>>>>>>>>>>
> > attached base packages:>>>>> [1] stats graphics
> > grDevices utils     datasets  methods base>>>>>>>>>>
> > loaded via a namespace (and not attached):>>>>> [1]
> > compiler_4.0.2>>>>>>>>>>>>>>>>>>>> Às 08:45 de 15/07/20,
> > Sam H escreveu:>>>>>> Hi,>>>>>>>>>>>> I am trying to
> > download some data using read.csv and it works>>>>>
> > perfectly in>>>>>> RStudio and fails in the R console in
> > the terminal in Ubuntu 18.04>>>>> after>>>>>> upgrading
> > from R 3.6.3 to 4.0.2. Before upgrading this worked in
> > the>>>>> R>>>>>> console in the terminal also without any
> > issues.>>>>>>>>>>>> Why would that be? How to fix
> > this?>>>>>>>>>>>> Below please find R code output and
> > sessionInfo().>>>>>>>>>>>> *Works in
> > RStudio*>>>>>>>>>>>>>>>>>>
> >
> read.csv("https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download",>>>>>
> > header=TRUE, as.is=TRUE, na="n/a")>>>>>>
> > Symbol
> > Name>>>>>> LastSale MarketCap IPOyear1      TXG>>>>>> 10x
> > Genomics, Inc.  87.4400     $8.6B    20192
> > YI>>>>>> 111, Inc.   6.4800
> > $533.69M    20183>>>>>> PIH 1347 Property
> > Insurance Holdings, Inc.   4.5350>>>>>> $27.52M
> > 2014>>>>>> sessionInfo()>>>>>> R version 4.0.2
> > (2020-06-22)>>>>>> Platform: x86_64-pc-linux-gnu
> > (64-bit)>>>>>> Running under: Ubuntu 18.04.4
> > LTS>>>>>>>>>>>> Matrix products: default>>>>>> BLAS:
> > /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3>>>>>>
> > LAPACK:
> > /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so>>>>>>>>>>>>
> > locale:>>>>>>    [1] LC_CTYPE=en_US.UTF-8
> > LC_NUMERIC=C>>>>>> LC_TIME=en_US.UTF-8
> > LC_COLLATE=en_US.UTF-8>>>>>>    [5]
> > LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8>>>>>>
> > LC_PAPER=en_US.UTF-8 LC_NAME=C>>>>>>    [9]
> > LC_ADDRESS=C LC_TELEPHONE=C>>>>>>
> > LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C>>>>>>>>>>>>
> > attached base packages:[1] stats     graphics  grDevices
> > utils>>>>>> datasets  methods base>>>>>>>>>>>> loaded
> > via a namespace (and not attached):[1] compiler_4.0.2>>>>>
> > tools_4.0.2>>>>>>>>>>>> *Fails in R console in
> > terminal*>>>>>>>>>>>> >>>>>>
> >
> read.csv("https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download",>>>>>>
> > header=TRUE, as.is=TRUE, na="n/a")>>>>>> Error in
> > file(file, "rt") :>>>>>>     cannot open the connection
> > to>>>>>>>>>>>
> >
> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download'>>>>>>
> > In addition: Warning message:>>>>>> In file(file, "rt")
> > :>>>>>>     URL>>>>>
> >
> 'https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download':>>>>>>
> > status was 'Failure when receiving data from the peer'>
> > traceback()3:>>>>>> file(file, "rt")2: read.table(file =
> > file, header = header, sep =>>>>> sep,>>>>>> quote =
> > quote,>>>>>>          dec = dec, fill = fill, comment.char
> > = comment.char, ...)1:>>>>>>>>>>>
> >
> read.csv("https://old.nasdaq.com/screening/companies-by-name.aspx?letter=0&exchange=1&render=download",>>>>>>
> > header = TRUE, as.is = TRUE, na = "n/a")>
> > sessionInfo()>>>>>> R version 4.0.2 (2020-06-22)>>>>>>
> > Platform: x86_64-pc-linux-gnu (64-bit)>>>>>> Running
> > under: Ubuntu 18.04.4 LTS>>>>>>>>>>>> Matrix products:
> > default>>>>>> BLAS:
> > /usr/lib/x86_64-linux-gnu/openblas/libblas.so.3>>>>>>
> > LAPACK:
> > /usr/lib/x86_64-linux-gnu/libopenblasp-r0.2.20.so>>>>>>>>>>>>
> > locale:>>>>>>    [1] LC_CTYPE=en_US.UTF-8
> > LC_NUMERIC=C>>>>>>    [3] LC_TIME=en_US.UTF-8
> > LC_COLLATE=en_US.UTF-8>>>>>>    [5]
> > LC_MONETARY=en_US.UTF-8
> > LC_MESSAGES=en_US.UTF-8>>>>>>    [7]
> > LC_PAPER=en_US.UTF-8 LC_NAME=C>>>>>>    [9]
> > LC_ADDRESS=C               LC_TELEPHONE=C
> > [11]>>>>>> LC_MEASUREMENT=en_US.UTF-8
> > LC_IDENTIFICATION=C>>>>>>>>>>>> attached base packages:[1]
> > stats     graphics  grDevices utils>>>>>> datasets
> > methods base>>>>>>>>>>>> loaded via a namespace (and not
> > attached):[1] compiler_4.0.2>>>>>>>>>>>>> I also asked
> > this question here>>>>>>>>>>>
> >
> https://stackoverflow.com/questions/62898008/why-read-csv-fails-in-r-console-in-ubuntu-terminal-but-works-in-rstudio-after-r>>>>>>
> > . Since there was no answer on stackoverflow I sent this
> > question>>>>> also to>>>>>> this list.>>>>>>>>>>>> Best
> > regards,>>>>>> Sam>>>>>>>>>>>> [[alternative HTML
> > version deleted]]>>>>>>>>>>>>
> > ______________________________________________>>>>>>
> > [hidden email] mailing list -- To UNSUBSCRIBE and
> > more, see>>>>>>
> > https://stat.ethz.ch/mailman/listinfo/r-help>>>>>> PLEASE
> > do read the posting guide>>>>>
> > http://www.R-project.org/posting-guide.html>>>>>> and
> > provide commented, minimal, self-contained, reproducible
> > code.>>>>>>>>>>>>>>>>
> > ______________________________________________>>>>>
> > [hidden email] mailing list -- To UNSUBSCRIBE and
> > more, see>>>>>
> > https://stat.ethz.ch/mailman/listinfo/r-help>>>>> PLEASE
> > do read the posting guide>>>>>
> > http://www.R-project.org/posting-guide.html>>>>> and
> > provide commented, minimal, self-contained, reproducible
> > code.>>>>>>>>>>
> > ______________________________________________>>>
> > [hidden email] mailing list -- To UNSUBSCRIBE and
> > more, see>>>
> > https://stat.ethz.ch/mailman/listinfo/r-help>>> PLEASE do
> > read the posting guide
> > http://www.R-project.org/posting-guide.html>>> and provide
> > commented, minimal, self-contained, reproducible code.>>
> > ______________________________________________>
> > [hidden email] mailing list -- To UNSUBSCRIBE and
> > more, see> https://stat.ethz.ch/mailman/listinfo/r-help>
> > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html> and provide
> > commented, minimal, self-contained, reproducible code.>--
> > Luke TierneyRalph E. Wareham Professor of Mathematical
> > SciencesUniversity of Iowa
> > Phone:             319-335-3386Department of Statistics
> > and        Fax:               319-335-3017    Actuarial
> > Science241 Schaeffer Hall                  email:
> > [hidden email] City, IA 52242
> > WWW:  http://www.stat.uiowa.edu [[alternative HTML version
> > deleted]]
>
> > ______________________________________________
> > [hidden email] mailing list -- To UNSUBSCRIBE and
> > more, see https://stat.ethz.ch/mailman/listinfo/r-help
> > PLEASE do read the posting guide
> > http://www.R-project.org/posting-guide.html and provide
> > commented, minimal, self-contained, reproducible code.


--
Este e-mail foi verificado em termos de vírus pelo software antivírus Avast.
https://www.avast.com/antivirus

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.