Downloading a .csv through a .jsp url with variable parameters - R code or package?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Downloading a .csv through a .jsp url with variable parameters - R code or package?

Arsenio Staer
Dear R experts,

I'm trying to download a .csv file with daily data on the companies through a javascript url ending in .jsp of the following form:
http://....... .com/.../.../filename.jsp?y=2004&m=3&d=9
where y = year, m = month and d=day. Accessing the link gives you the .csv for that date. I want to download the whole time series day by day say from y = 2000 to 2010 into an R dataset. Any idea how to do that through R code?

Thanks a lot in advance!

Arsenio
Reply | Threaded
Open this post in threaded view
|

Re: Downloading a .csv through a .jsp url with variable parameters - R code or package?

Arsenio Staer
So i realized putting this
raw.untr <- as.data.frame(read.csv(file='.......jsp?y=2008&m=09&d=30', header = TRUE, as.is = TRUE))

reads the file prefectly, so i was wondering how can i create a set of dataframes each with observations for one day (say for 3 years with 250 days of data each year = 750 datasets) ? And in each loop of a cycle the text string in the url would change to reflect the new date, say from y=2008&m=09&d=30 to y=2008&m=09&d=31?

Any help would be great, i somewhat expected more of a reaction given how vibrant R community is,

Arsenio
Reply | Threaded
Open this post in threaded view
|

Re: Downloading a .csv through a .jsp url with variable parameters - R code or package?

jholtman
Something like this should work, only Sept has 30 days, not 31:

> files <- seq(as.Date('2008-09-30'), by = '1 day', length = 250)
> input <- paste(".....jsp?y="
+             , format(files, "%Y")
+             , "&m="
+             , format(files, "%m")
+             , "&d="
+             , format(files, "%d")
+             , sep = ""
+             )
>
> head(input)
[1] ".....jsp?y=2008&m=09&d=30" ".....jsp?y=2008&m=10&d=01"
[3] ".....jsp?y=2008&m=10&d=02" ".....jsp?y=2008&m=10&d=03"
[5] ".....jsp?y=2008&m=10&d=04" ".....jsp?y=2008&m=10&d=05"
>

On Mon, Dec 6, 2010 at 8:18 PM, Arsenio Staer <[hidden email]> wrote:

>
> So i realized putting this
> raw.untr <- as.data.frame(read.csv(file='.......jsp?y=2008&m=09&d=30',
> header = TRUE, as.is = TRUE))
>
> reads the file prefectly, so i was wondering how can i create a set of
> dataframes each with observations for one day (say for 3 years with 250 days
> of data each year = 750 datasets) ? And in each loop of a cycle the text
> string in the url would change to reflect the new date, say from
> y=2008&m=09&d=30 to y=2008&m=09&d=31?
>
> Any help would be great, i somewhat expected more of a reaction given how
> vibrant R community is,
>
> Arsenio
>
> --
> View this message in context: http://r.789695.n4.nabble.com/Downloading-a-csv-through-a-jsp-url-with-variable-parameters-R-code-or-package-tp3070541p3075644.html
> Sent from the R help mailing list archive at Nabble.com.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.
>



--
Jim Holtman
Data Munger Guru

What is the problem that you are trying to solve?

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: Downloading a .csv through a .jsp url with variable parameters - R code or package?

Arsenio Staer
In reply to this post by Arsenio Staer
Jim,

Thanks a lot! works perfectly! now i just use a for loop and rbind to create one dataset with all daily observations.

Arsenio