|
Many R packages depend on some unix libraries that are not part of most default installations. I often spend a significant amount of time figuring out where to get the appropriate libraries for compiling these packages, after they give some vague error of something missing. I was wondering why there is no formal system of specifying non-R dependencies in the DESCRIPTION file. If this would be the case, then during the installation of an R package, the user could be prompted to install required system packages (if they have appropriate privileges).
So for example: Package: XML Version: 3.2-0 Depends: R (>= 1.2.0), methods, utils Depends-debian: libxml2-dev Depends-ubuntu: libxml2-dev Depends-redhat: libxml2-devel Depends-suse: libxml2-devel etc. This might make life for many people just a little easier. If they are root and the package is in their system repositories, than it will install automatically. If not, at least they know for which package to look, or request their sys admin to install. |
|
Jeroen,
On Feb 3, 2011, at 9:31 PM, Jeroen Ooms wrote: > > Many R packages depend on some unix libraries that are not part of most > default installations. I often spend a significant amount of time figuring > out where to get the appropriate libraries for compiling these packages, > after they give some vague error of something missing. I was wondering why > there is no formal system of specifying non-R dependencies in the > DESCRIPTION file. If this would be the case, then during the installation of > an R package, the user could be prompted to install required system packages > (if they have appropriate privileges). > > So for example: > > Package: XML > Version: 3.2-0 > Depends: R (>= 1.2.0), methods, utils > Depends-debian: libxml2-dev > Depends-ubuntu: libxml2-dev > Depends-redhat: libxml2-devel > Depends-suse: libxml2-devel > etc. > > This might make life for many people just a little easier. If they are root > and the package is in their system repositories, than it will install > automatically. If not, at least they know for which package to look, or > request their sys admin to install. Well, there is already such system in place and it is the corresponding descriptions in the distributions. Obviously as an author of the package I don't care what any particular Linux distribution uses as a name for the needed dependencies as the corresponding chaos is distribution-specific. The only person who can reasonably determine the dependencies is the maintainer of the distribution and that's what they do and as a user of the above mentioned distributions you should be thankful to them. Fortunately, normal users don't have to worry about it as major distributions already come with a large set of R packages resolving all dependencies. Hence I don't see any reason why this should have anything to do with the DESCRIPTION file. The improvements I could think of would be a parseable entry or a canonical pointer to dependency sources, but that's a whole another story. Cheers, Simon ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-devel |
|
Dear all,
From the writing extensions manual: "Other dependencies (external to the R system) should be listed in the ‘SystemRequirements’ field, possibly amplified in a separate README file." I guess one problem is the user may not realize that the -dev version is needed, and just sees libxml2 installed but the R package installation stopping with the respective error. Giving the package name for specific distributions is of course polite (if the developer knows it). As developer you may also put into the README that the package's mailing list/forum/wiki/... contains information and ask the user to enter the package name on his distro if it is not already there. my 2 ct Claudia Am 04.02.2011 04:48, schrieb Simon Urbanek: > Jeroen, > > On Feb 3, 2011, at 9:31 PM, Jeroen Ooms wrote: > >> >> Many R packages depend on some unix libraries that are not part of most >> default installations. I often spend a significant amount of time figuring >> out where to get the appropriate libraries for compiling these packages, >> after they give some vague error of something missing. I was wondering why >> there is no formal system of specifying non-R dependencies in the >> DESCRIPTION file. If this would be the case, then during the installation of >> an R package, the user could be prompted to install required system packages >> (if they have appropriate privileges). >> >> So for example: >> >> Package: XML >> Version: 3.2-0 >> Depends: R (>= 1.2.0), methods, utils >> Depends-debian: libxml2-dev >> Depends-ubuntu: libxml2-dev >> Depends-redhat: libxml2-devel >> Depends-suse: libxml2-devel >> etc. >> >> This might make life for many people just a little easier. If they are root >> and the package is in their system repositories, than it will install >> automatically. If not, at least they know for which package to look, or >> request their sys admin to install. > > Well, there is already such system in place and it is the corresponding descriptions in the distributions. Obviously as an author of the package I don't care what any particular Linux distribution uses as a name for the needed dependencies as the corresponding chaos is distribution-specific. The only person who can reasonably determine the dependencies is the maintainer of the distribution and that's what they do and as a user of the above mentioned distributions you should be thankful to them. Fortunately, normal users don't have to worry about it as major distributions already come with a large set of R packages resolving all dependencies. Hence I don't see any reason why this should have anything to do with the DESCRIPTION file. The improvements I could think of would be a parseable entry or a canonical pointer to dependency sources, but that's a whole another story. > > Cheers, > Simon > > ______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-devel |
|
Claudia,
thanks for you comments . On Feb 4, 2011, at 3:18 AM, Claudia Beleites wrote: > Dear all, > > From the writing extensions manual: > "Other dependencies (external to the R system) should be listed in the ‘SystemRequirements’ field, possibly amplified in a separate README file." > > I guess one problem is the user may not realize that the -dev version is needed, and just sees libxml2 installed but the R package installation stopping with the respective error. > I'd argue that if a user attempts to install a package from sources instead of using the distribution binaries, he should know what he's doing as there is much more involved (proper tools, usually a different library location etc.). And anyone who knows what he's doing also knows that -dev packages are needed (at the latest when the installation fails you remember ;)). If he doesn't then it should give him a clue that he may want to use something else (and especially Linux users should know better ;)). Clearly, it doesn't prevent users from doing stupid things and I completely agree with you that the README should have the instructions as far as the developer knows. And as a package developer you'll learn soon enough when people start complaining ;). Thanks, Simon > Giving the package name for specific distributions is of course polite (if the developer knows it). As developer you may also put into the README that the package's mailing list/forum/wiki/... contains information and ask the user to enter the package name on his distro if it is not already there. > > > my 2 ct > > Claudia > > > > Am 04.02.2011 04:48, schrieb Simon Urbanek: >> Jeroen, >> >> On Feb 3, 2011, at 9:31 PM, Jeroen Ooms wrote: >> >>> >>> Many R packages depend on some unix libraries that are not part of most >>> default installations. I often spend a significant amount of time figuring >>> out where to get the appropriate libraries for compiling these packages, >>> after they give some vague error of something missing. I was wondering why >>> there is no formal system of specifying non-R dependencies in the >>> DESCRIPTION file. If this would be the case, then during the installation of >>> an R package, the user could be prompted to install required system packages >>> (if they have appropriate privileges). >>> >>> So for example: >>> >>> Package: XML >>> Version: 3.2-0 >>> Depends: R (>= 1.2.0), methods, utils >>> Depends-debian: libxml2-dev >>> Depends-ubuntu: libxml2-dev >>> Depends-redhat: libxml2-devel >>> Depends-suse: libxml2-devel >>> etc. >>> >>> This might make life for many people just a little easier. If they are root >>> and the package is in their system repositories, than it will install >>> automatically. If not, at least they know for which package to look, or >>> request their sys admin to install. >> >> Well, there is already such system in place and it is the corresponding descriptions in the distributions. Obviously as an author of the package I don't care what any particular Linux distribution uses as a name for the needed dependencies as the corresponding chaos is distribution-specific. The only person who can reasonably determine the dependencies is the maintainer of the distribution and that's what they do and as a user of the above mentioned distributions you should be thankful to them. Fortunately, normal users don't have to worry about it as major distributions already come with a large set of R packages resolving all dependencies. Hence I don't see any reason why this should have anything to do with the DESCRIPTION file. The improvements I could think of would be a parseable entry or a canonical pointer to dependency sources, but that's a whole another story. >> >> Cheers, >> Simon >> >> ______________________________________________ >> [hidden email] mailing list >> https://stat.ethz.ch/mailman/listinfo/r-devel > > ______________________________________________ > [hidden email] mailing list > https://stat.ethz.ch/mailman/listinfo/r-devel > > ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-devel |
|
Simon, On 4 February 2011 at 10:01, Simon Urbanek wrote: | Claudia, | | thanks for you comments . | | On Feb 4, 2011, at 3:18 AM, Claudia Beleites wrote: | | > Dear all, | > | > From the writing extensions manual: | > "Other dependencies (external to the R system) should be listed in the ‘SystemRequirements’ field, possibly amplified in a separate README file." | > | > I guess one problem is the user may not realize that the -dev version is needed, and just sees libxml2 installed but the R package installation stopping with the respective error. | > | | I'd argue that if a user attempts to install a package from sources instead | of using the distribution binaries, he should know what he's doing as there | is much more involved (proper tools, usually a different library location | etc.). And anyone who knows what he's doing also knows that -dev packages | are needed (at the latest when the installation fails you remember ;)). If | he doesn't then it should give him a clue that he may want to use something | else (and especially Linux users should know better ;)). | | Clearly, it doesn't prevent users from doing stupid things and I completely | agree with you that the README should have the instructions as far as the | developer knows. And as a package developer you'll learn soon enough when | people start complaining ;). I respectfully disagree. Based on a number of years of supporting users on Linux where people install frequently from source, I can assure you that a rather large number of people fails. Not everybody is fluent with compilers, knows about libraries and their interdependencies, or can even read configure error messages. We all see the r-help messages (or the traffic on the SIG lists). People want to use the wealth of software that is CRAN, and we should help them. I have also been involved in by now two attempts to overcome this in an automated fashion via cran2deb. We had that working somewhat reliably until parts of the infrastructure misteriously self-destructed (a large sqlite table) right when I visited Vienna, and are now in a rewrite which may be never ending (for lack of resources). There was a lot of interest for it when it worked, and there is ongoing interest right now (as a few guys from across Europe just met last weekend to try to use for BioC builds). There were also folks from other distros who tried something similar. What Jeroen suggested is along those lines with the needed meta-data. Whether we make it per-package (as per Jeroen's idea) or 'per-repo-distro-pair' (which is what cran2deb does) is a detail. We need to address this: With 2600+ packages and continued growth, manually wading through README is not good enough. We should do better. Resources (time, money, servers, ...) would help. Maybe one day someone with more time can fold this into a proper sub-project of a larger grant application. It would be worth, and I would try to help, time permitting. Cheers, Dirk | | Thanks, | Simon | | | | > Giving the package name for specific distributions is of course polite (if the developer knows it). As developer you may also put into the README that the package's mailing list/forum/wiki/... contains information and ask the user to enter the package name on his distro if it is not already there. | > | > | > my 2 ct | > | > Claudia | > | > | > | > Am 04.02.2011 04:48, schrieb Simon Urbanek: | >> Jeroen, | >> | >> On Feb 3, 2011, at 9:31 PM, Jeroen Ooms wrote: | >> | >>> | >>> Many R packages depend on some unix libraries that are not part of most | >>> default installations. I often spend a significant amount of time figuring | >>> out where to get the appropriate libraries for compiling these packages, | >>> after they give some vague error of something missing. I was wondering why | >>> there is no formal system of specifying non-R dependencies in the | >>> DESCRIPTION file. If this would be the case, then during the installation of | >>> an R package, the user could be prompted to install required system packages | >>> (if they have appropriate privileges). | >>> | >>> So for example: | >>> | >>> Package: XML | >>> Version: 3.2-0 | >>> Depends: R (>= 1.2.0), methods, utils | >>> Depends-debian: libxml2-dev | >>> Depends-ubuntu: libxml2-dev | >>> Depends-redhat: libxml2-devel | >>> Depends-suse: libxml2-devel | >>> etc. | >>> | >>> This might make life for many people just a little easier. If they are root | >>> and the package is in their system repositories, than it will install | >>> automatically. If not, at least they know for which package to look, or | >>> request their sys admin to install. | >> | >> Well, there is already such system in place and it is the corresponding descriptions in the distributions. Obviously as an author of the package I don't care what any particular Linux distribution uses as a name for the needed dependencies as the corresponding chaos is distribution-specific. The only person who can reasonably determine the dependencies is the maintainer of the distribution and that's what they do and as a user of the above mentioned distributions you should be thankful to them. Fortunately, normal users don't have to worry about it as major distributions already come with a large set of R packages resolving all dependencies. Hence I don't see any reason why this should have anything to do with the DESCRIPTION file. The improvements I could think of would be a parseable entry or a canonical pointer to dependency sources, but that's a whole another story. | >> | >> Cheers, | >> Simon | >> | >> ______________________________________________ | >> [hidden email] mailing list | >> https://stat.ethz.ch/mailman/listinfo/r-devel | > | > ______________________________________________ | > [hidden email] mailing list | > https://stat.ethz.ch/mailman/listinfo/r-devel | > | > | | ______________________________________________ | [hidden email] mailing list | https://stat.ethz.ch/mailman/listinfo/r-devel -- Dirk Eddelbuettel | [hidden email] | http://dirk.eddelbuettel.com ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-devel |
|
In reply to this post by Simon Urbanek
On 2/4/11 9:01 AM, "Simon Urbanek" <[hidden email]> wrote: >I'd argue that if a user attempts to install a package from sources >instead of using the distribution binaries, he should know what he's >doing as there is much more involved (proper tools, usually a different >library location etc.). Most of the time when I build an existing package from source, it's because I'm doing The Open Source Thing, either trying to fix some bug or add some new feature. One should only need to be a competent writer of R code to do that, but often the mechanics of building the package get in the way. -- Ken Williams Senior Research Scientist Thomson Reuters http://labs.thomsonreuters.com ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-devel |
|
In reply to this post by Dirk Eddelbuettel
On Feb 4, 2011, at 1:24 PM, Dirk Eddelbuettel wrote: > > Simon, > > On 4 February 2011 at 10:01, Simon Urbanek wrote: > | Claudia, > | > | thanks for you comments . > | > | On Feb 4, 2011, at 3:18 AM, Claudia Beleites wrote: > | > | > Dear all, > | > > | > From the writing extensions manual: > | > "Other dependencies (external to the R system) should be listed in the ‘SystemRequirements’ field, possibly amplified in a separate README file." > | > > | > I guess one problem is the user may not realize that the -dev version is needed, and just sees libxml2 installed but the R package installation stopping with the respective error. > | > > | > | I'd argue that if a user attempts to install a package from sources instead > | of using the distribution binaries, he should know what he's doing as there > | is much more involved (proper tools, usually a different library location > | etc.). And anyone who knows what he's doing also knows that -dev packages > | are needed (at the latest when the installation fails you remember ;)). If > | he doesn't then it should give him a clue that he may want to use something > | else (and especially Linux users should know better ;)). > | > | Clearly, it doesn't prevent users from doing stupid things and I completely > | agree with you that the README should have the instructions as far as the > | developer knows. And as a package developer you'll learn soon enough when > | people start complaining ;). > > I respectfully disagree. > > Based on a number of years of supporting users on Linux where people install > frequently from source, I can assure you that a rather large number of people > fails. Not everybody is fluent with compilers, knows about libraries and > their interdependencies, or can even read configure error messages. We all > see the r-help messages (or the traffic on the SIG lists). > > People want to use the wealth of software that is CRAN, and we should help > them. I have also been involved in by now two attempts to overcome this in > an automated fashion via cran2deb. We had that working somewhat reliably > until parts of the infrastructure misteriously self-destructed (a large > sqlite table) right when I visited Vienna, and are now in a rewrite which may > be never ending (for lack of resources). There was a lot of interest for it > when it worked, and there is ongoing interest right now (as a few guys from > across Europe just met last weekend to try to use for BioC builds). > > There were also folks from other distros who tried something similar. What > Jeroen suggested is along those lines with the needed meta-data. Whether we > make it per-package (as per Jeroen's idea) or 'per-repo-distro-pair' (which > is what cran2deb does) is a detail. > > We need to address this: With 2600+ packages and continued growth, manually > wading through README is not good enough. We should do better. Resources > (time, money, servers, ...) would help. Maybe one day someone with more time > can fold this into a proper sub-project of a larger grant application. It > would be worth, and I would try to help, time permitting. > Well, as far as I can see you're only agreeing with me :). I said people like you are solving the issues by creating the corresponding distro-specific description and people should be thankful for that. Also I said that the distro-specific way is the only reliable way as package authors cannot know the intricacies of the distros involved. If the distros could share a system that helps to maintain this, it would be perfect - by all mean supported by us if we can, but it's separate from the package authors. And, finally, I also said that it would be nice to have a parseable field that is disto-independent so the distro-maintainer don't need to weed through READMEs to find dependency sources. I'm sorry to hear that cran2deb self-destructed as your Debian handling of packages is immensely helpful, especially for package with intricate dependencies. On a similar note, I also think that it would be useful to have a common support already at the package level -- very few people know how to write good configure scripts and there are many pitfalls that catch the unwary, so we could have a template for the most common case of requiring a few libraries. This would also allow some auto-maigc handling of the extended requirement, possibly with some site support ... (e.g. you could imagine searching debs for the library files that are required and coming up with a suggestion of packages etc.). Cheers, Simon > | > | Thanks, > | Simon > | > | > | > | > Giving the package name for specific distributions is of course polite (if the developer knows it). As developer you may also put into the README that the package's mailing list/forum/wiki/... contains information and ask the user to enter the package name on his distro if it is not already there. > | > > | > > | > my 2 ct > | > > | > Claudia > | > > | > > | > > | > Am 04.02.2011 04:48, schrieb Simon Urbanek: > | >> Jeroen, > | >> > | >> On Feb 3, 2011, at 9:31 PM, Jeroen Ooms wrote: > | >> > | >>> > | >>> Many R packages depend on some unix libraries that are not part of most > | >>> default installations. I often spend a significant amount of time figuring > | >>> out where to get the appropriate libraries for compiling these packages, > | >>> after they give some vague error of something missing. I was wondering why > | >>> there is no formal system of specifying non-R dependencies in the > | >>> DESCRIPTION file. If this would be the case, then during the installation of > | >>> an R package, the user could be prompted to install required system packages > | >>> (if they have appropriate privileges). > | >>> > | >>> So for example: > | >>> > | >>> Package: XML > | >>> Version: 3.2-0 > | >>> Depends: R (>= 1.2.0), methods, utils > | >>> Depends-debian: libxml2-dev > | >>> Depends-ubuntu: libxml2-dev > | >>> Depends-redhat: libxml2-devel > | >>> Depends-suse: libxml2-devel > | >>> etc. > | >>> > | >>> This might make life for many people just a little easier. If they are root > | >>> and the package is in their system repositories, than it will install > | >>> automatically. If not, at least they know for which package to look, or > | >>> request their sys admin to install. > | >> > | >> Well, there is already such system in place and it is the corresponding descriptions in the distributions. Obviously as an author of the package I don't care what any particular Linux distribution uses as a name for the needed dependencies as the corresponding chaos is distribution-specific. The only person who can reasonably determine the dependencies is the maintainer of the distribution and that's what they do and as a user of the above mentioned distributions you should be thankful to them. Fortunately, normal users don't have to worry about it as major distributions already come with a large set of R packages resolving all dependencies. Hence I don't see any reason why this should have anything to do with the DESCRIPTION file. The improvements I could think of would be a parseable entry or a canonical pointer to dependency sources, but that's a whole another story. > | >> > | >> Cheers, > | >> Simon > | >> > | >> ______________________________________________ > | >> [hidden email] mailing list > | >> https://stat.ethz.ch/mailman/listinfo/r-devel > | > > | > ______________________________________________ > | > [hidden email] mailing list > | > https://stat.ethz.ch/mailman/listinfo/r-devel > | > > | > > | > | ______________________________________________ > | [hidden email] mailing list > | https://stat.ethz.ch/mailman/listinfo/r-devel > > -- > Dirk Eddelbuettel | [hidden email] | http://dirk.eddelbuettel.com > > ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-devel |
|
In reply to this post by Ken.Williams-2
From: [hidden email] [mailto:[hidden email]]
On Behalf Of [hidden email] >Sent: February-04-11 1:58 PM >To: [hidden email]; [hidden email] >Cc: [hidden email] >Subject: Re: [Rd] dependencies on system packages >On 2/4/11 9:01 AM, "Simon Urbanek" <[hidden email]> wrote: > >>I'd argue that if a user attempts to install a package from sources >>instead of using the distribution binaries, he should know what he's >>doing as there is much more involved (proper tools, usually a different >>library location etc.). > >Most of the time when I build an existing package from source, it's because new feature. One should only need to be a competent writer of R code to do that, but often the mechanics of >building the package get in the way. Actually, I develop enough software that I don't really have the time to do this (debug a package's source, or extend it), though that may change at any time as I start using it more intensively. Normally, the only time I have been compiling R packages from source is when there were no 64 bit Windows binaries. Using the right RTools, along with R's ability to install from source allowed me to use PostGreSQL and MySQL in 64 bit R on 64 biw Windows 7. This is a counter-example related to Simon's remarks about a user building packages from source. His remark is valid IF the user is making repairs or extensions, as Ken says he does. However, it is problemtic for those users who are faced with a situation in which there IS no binary distribution for their specific platform. I do precisely the same thing with open source software written for some flavour of Unix, even though I don't have a machine with a real unix. On my machine (64 bit Windows 7), I have cygwin installed (and it works fine), and for that unix software I want to use, the usual process of "./configure" followed by make, "make check" or "make test", and "make install" works great. All I have to do is read the documentation for the software or library I want to use, in order to make certain the dependancy requirements are satisfied. NB: I use cygwin ONLY to pass my C++ code through gcc, as a check on the quality of code I have already passed through MSVC++, but that is another story. Yes, I have the skills needed to deal with the issues Simon raises WRT building software n a unix machine, but I know several R users who do not, and I would not want them to be unable to use one package or another because of there being no binary distribution for their specific platform (of course, you know who they ask when they encounter this sort of issue). Thus, I agree with Dirk that more robust and user friendly option is required. For me, I don't care if it is a script that runs from the shell (bash, Windows shell, &c.) or from the R menu, but I know some potential users who would be lost unless it is something simple that can be invoked from R's menu. Cheers, Ted ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-devel |
|
In reply to this post by Simon Urbanek
Simon Urbanek <[hidden email]> writes:
> On Feb 4, 2011, at 1:24 PM, Dirk Eddelbuettel wrote: [...] >> We need to address this: With 2600+ packages and continued growth, >> manually wading through README is not good enough. We should do >> better. Resources (time, money, servers, ...) would help. Maybe one >> day someone with more time can fold this into a proper sub-project of >> a larger grant application. It would be worth, and I would try to >> help, time permitting. I'm one of the people who made stabs at this in the past. I had a fairly complete Gentoo process in place, but then the sickness passed. More recently, I've been peripherally involved with an effort to translate CRAN to redhat-land. When we talked about this most intensely last (waaay back in 2006), I had a detailed submission for an "Extended depends string", a strictly formatted field to accompany or supplant the existing system depends. I also supplied suggested code to translate a string in this format to a simple depstring, which would neatly plug into the existing depstring infrastructure already in place. http://article.gmane.org/gmane.comp.lang.r.devel/9179 So, for an example extended depstring src <- "libgd (>= 1.9.0) (gentoo gd >= 2.0) (debian >= 1.9.2)" extdepstring2depstring(src) yields "libgd (>= 1.9.0)" and extdepstring2depstring(src,systype="debian") yields "libgd (>= 1.9.2)" and extdepstring2depstring(src,systype="unknown") yields the default. I think this would be preferable to a growing taxonomy of Depends-[system]: - Allen S. Rout ______________________________________________ [hidden email] mailing list https://stat.ethz.ch/mailman/listinfo/r-devel |
| Powered by Nabble | Edit this page |
