Suggestion: Make CRAN source URLs immutable

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Suggestion: Make CRAN source URLs immutable

Kurt Wheeler
Hello, I hope the is the right list to send this suggestion to.

I was wondering if it might be possible to have CRAN store the most current
version of a package's source tarball at a location that does not change.
As an example, the source tarball for `httr` is stored at
https://cran.r-project.org/src/contrib/httr_1.3.1.tar.gz. However, once the
next version of `httr` is released, the URL for version 1.3.1 of `httr`
will change to
https://cran.r-project.org/src/contrib/Archive/httr_1.3.1.tar.gz.

The implications of this are that if I want to write a script which
installs version 1.3.1 of `httr`, I need to do so like this:

```
httr_url <- "https://cran.r-project.org/src/contrib/httr_1.3.1.tar.gz"

curl_result <- system(paste0("curl --head ", httr_url), intern=TRUE)
if (grepl("404", curl_result[1])) {
  httr_url <- "
https://cran.r-project.org/src/contrib/Archive/httr_1.3.1.tar.gz"
}

install.packages(httr_url)
```

I think there are two obvious ways of remedying this issue. The first would
be to simply archive the most recent version along with all the old
versions. Then to install a specific version of a package, it would just be:

```
install.packages("
https://cran.r-project.org/src/contrib/Archive/httr_1.3.1.tar.gz")
```

even if 1.3.1 is the most recent version of `httr`.


The other way to remedy this issue would be to simply not use the
`/Archive/` sub-directory. So if I wanted to install an old version of
`httr` it would be:

```
install.packages("https://cran.r-project.org/src/contrib/httr_1.3.0.tar.gz")
```

despite the fact that version 1.3.0 is not the most recent version.


I believe the first way would be a better choice because then any existing
scripts which reference URLs which include `/Archive/` would still work.
However I do not have strong opinions on how this should be implemented, I
would just like a URL which points to a certain version of a package that
will never change.

Thank you,

- Kurt

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Suggestion: Make CRAN source URLs immutable

Kurt Hornik-5
>>>>> Kurt Wheeler writes:

Try e.g.

https://cran.r-project.org/package=httr&version=1.3.1
https://cran.r-project.org/package=httr&version=1.3.0

-k

> Hello, I hope the is the right list to send this suggestion to.
> I was wondering if it might be possible to have CRAN store the most current
> version of a package's source tarball at a location that does not change.
> As an example, the source tarball for `httr` is stored at
> https://cran.r-project.org/src/contrib/httr_1.3.1.tar.gz. However, once the
> next version of `httr` is released, the URL for version 1.3.1 of `httr`
> will change to
> https://cran.r-project.org/src/contrib/Archive/httr_1.3.1.tar.gz.

> The implications of this are that if I want to write a script which
> installs version 1.3.1 of `httr`, I need to do so like this:

> ```
> httr_url <- "https://cran.r-project.org/src/contrib/httr_1.3.1.tar.gz"

> curl_result <- system(paste0("curl --head ", httr_url), intern=TRUE)
> if (grepl("404", curl_result[1])) {
>   httr_url <- "
> https://cran.r-project.org/src/contrib/Archive/httr_1.3.1.tar.gz"
> }

> install.packages(httr_url)
> ```

> I think there are two obvious ways of remedying this issue. The first would
> be to simply archive the most recent version along with all the old
> versions. Then to install a specific version of a package, it would just be:

> ```
> install.packages("
> https://cran.r-project.org/src/contrib/Archive/httr_1.3.1.tar.gz")
> ```

> even if 1.3.1 is the most recent version of `httr`.


> The other way to remedy this issue would be to simply not use the
> `/Archive/` sub-directory. So if I wanted to install an old version of
> `httr` it would be:

> ```
> install.packages("https://cran.r-project.org/src/contrib/httr_1.3.0.tar.gz")
> ```

> despite the fact that version 1.3.0 is not the most recent version.


> I believe the first way would be a better choice because then any existing
> scripts which reference URLs which include `/Archive/` would still work.
> However I do not have strong opinions on how this should be implemented, I
> would just like a URL which points to a certain version of a package that
> will never change.

> Thank you,

> - Kurt

> [[alternative HTML version deleted]]

> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Suggestion: Make CRAN source URLs immutable

Iñaki Ucar
On Wed, 24 Oct 2018 at 11:40, Kurt Hornik <[hidden email]> wrote:
>
> >>>>> Kurt Wheeler writes:
>
> Try e.g.
>
> https://cran.r-project.org/package=httr&version=1.3.1
> https://cran.r-project.org/package=httr&version=1.3.0

This is a nice feature that I didn't know. I recently proposed
enforcing this scheme in Fedora's packaging guidelines, because in
this way, a SPEC would build correctly even if the package was updated
and the old version was archived (is this guaranteed to continue to
work in future? I assumed so...).

There is an odd thing about this format though, and that is the
absence of a file extension. This is a redirection, yes, but the
spectool can't trust the filename that is sent by the remote server,
and uses only the filename extracted from the URL.

Without extension, RPM doesn't know how to unpack the sources. So we
have to do the following similarly odd trick (note the "#"):

Source0: <a href="https://cran.r-project.org/package=%">https://cran.r-project.org/package=%{packname}&version=%{version}#/%{packname}_%{version}.tar.gz

Did you consider this problem? Is there any alternate immutable URL
*with* extension? If not, is there any potential issue with the trick
above?

Regards,
--
Iñaki Ucar

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Suggestion: Make CRAN source URLs immutable

Joris FA Meys
FWIW, you can get the URL and extract the link with extension from there.
Archived packages are always tarballs, so that makes the following possible:

url <- "https://cran.r-project.org/package=httr&version=1.3.0"

library(RCurl)

pkgurl <- gsub(".*(https://cran.+\\.tar.gz).*",
               "\\1",
               getURL(url))

install.packages(pkgurl, type = "source", repos = NULL)

Cheers
Joris


On Fri, Nov 2, 2018 at 5:42 PM Iñaki Ucar <[hidden email]> wrote:

> On Wed, 24 Oct 2018 at 11:40, Kurt Hornik <[hidden email]> wrote:
> >
> > >>>>> Kurt Wheeler writes:
> >
> > Try e.g.
> >
> > https://cran.r-project.org/package=httr&version=1.3.1
> > https://cran.r-project.org/package=httr&version=1.3.0
>
> This is a nice feature that I didn't know. I recently proposed
> enforcing this scheme in Fedora's packaging guidelines, because in
> this way, a SPEC would build correctly even if the package was updated
> and the old version was archived (is this guaranteed to continue to
> work in future? I assumed so...).
>
> There is an odd thing about this format though, and that is the
> absence of a file extension. This is a redirection, yes, but the
> spectool can't trust the filename that is sent by the remote server,
> and uses only the filename extracted from the URL.
>
> Without extension, RPM doesn't know how to unpack the sources. So we
> have to do the following similarly odd trick (note the "#"):
>
> Source0:
> <a href="https://cran.r-project.org/package=%">https://cran.r-project.org/package=%{packname}&version=%{version}#/%{packname}_%{version}.tar.gz
>
> Did you consider this problem? Is there any alternate immutable URL
> *with* extension? If not, is there any potential issue with the trick
> above?
>
> Regards,
> --
> Iñaki Ucar
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>


--
Joris Meys
Statistical consultant

Department of Data Analysis and Mathematical Modelling
Ghent University
Coupure Links 653, B-9000 Gent (Belgium)
<https://maps.google.com/?q=Coupure+links+653,%C2%A0B-9000+Gent,%C2%A0Belgium&entry=gmail&source=g>

-----------
Biowiskundedagen 2017-2018
http://www.biowiskundedagen.ugent.be/

-------------------------------
Disclaimer : http://helpdesk.ugent.be/e-maildisclaimer.php

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Suggestion: Make CRAN source URLs immutable

Iñaki Ucar
On Sat, 3 Nov 2018 at 11:54, Joris Meys <[hidden email]> wrote:

>
> FWIW, you can get the URL and extract the link with extension from there. Archived packages are always tarballs, so that makes the following possible:
>
> url <- "https://cran.r-project.org/package=httr&version=1.3.0"
>
> library(RCurl)
>
> pkgurl <- gsub(".*(https://cran.+\\.tar.gz).*",
>                "\\1",
>                getURL(url))
>
> install.packages(pkgurl, type = "source", repos = NULL)

The proper way to do this would be to fetch just the headers and
extract the location field:

$ URL="https://cran.r-project.org/package=httr&version=1.3.0"
$ curl -sI $URL | awk '/^Location:/{print $2}'
https://cran.r-project.org/src/contrib/Archive/httr/httr_1.3.0.tar.gz

But that's not the point. I'm talking about RPM packaging, and the
point is, as I said, that the tool that expands and downloads sources
from specfiles simply doesn't do that, because it can't trust an
extension sent from a remote server. We need to provide it explicitly
in the URL.

--
Iñaki Úcar

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Suggestion: Make CRAN source URLs immutable

Juan Telleria Ruiz de Aguirre
It would be nice to see such immutable package version links:

E.g.:

https://cran.r-project.org/package=httr&version=1.3.1
https://cran.r-project.org/package=httr&version=1.3.0


In the CRAN package web pages themselves, and specifically in the "Old
Sources: httr Section":

https://cran.r-project.org/web/packages/httr/index.html

In order that package users become aware of it.

But just as actually is also perfectly fine!

Juan

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel