libPaths displays truncated path?

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

libPaths displays truncated path?

Loris Bennett-2
Hi,

TL;DR
-----

  I define the path

    /cm/shared/apps/R/site-library/3.4.2

  and add it to libPath.  Why does libPath then display it as

    /cm/shared/apps/R/site-library/3.4

  ?

Long version
------------

I run a cluster of diskless nodes for which the OS is loaded
directly into RAM and other software is provided by an NFS server.
However, in the case of R, we use the R version provided by the OS and
just install additional packages on the NFS server.

So that R can find these additional packages, I have the following in
the site-wide Rprofile

  v <- R.Version()
  base_path = "/cm/shared/apps/R/site-library/"
  major_minor_version = paste(v["major"],v["minor"],sep=".")
  cm_shared_lib_path = paste0(base_path,major_minor_version)
  full_cm_shared_lib_path <- c(file.path(chartr("\\", "/", R.home()), "site-library"), cm_shared_lib_path)
  .libPaths( c( .libPaths(), full_cm_shared_lib_path ) )

Thus, when I start R I get this:

  > full_cm_shared_lib_path
  [1] "/usr/lib64/R/site-library"          
  [2] "/cm/shared/apps/R/site-library/3.4.2"

but also this

   > .libPaths()
  [1] "/home/loris/R/x86_64-redhat-linux-gnu-library/3.4"
  [2] "/usr/lib64/R/library"                            
  [3] "/usr/share/R/library"                            
  [4] "/usr/lib64/R/site-library"                        
  [5] "/cm/shared/apps/R/site-library/3.4"      

However, in order to get R to find the packages, I have to add a
symbolic link, thus:

  [/cm/shared/apps/R/site-library] $ ls -l 3.4.2
  lrwxrwxrwx 1 root root 3 Nov 23 09:21 3.4.2 -> 3.4

So, my mistake was to think that "minor" would return "4", whereas it in
fact returns "4.2".  So I actually set the path to ".../3.4.2" and
that's where R looks for packages.

But why does libPaths display the the path I *thought* I had set, but in
fact looks at the path I *really did* set?

Cheers,

Loris

--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin         Email [hidden email]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: libPaths displays truncated path?

Berry, Charles

> On Nov 23, 2017, at 4:34 AM, Loris Bennett <[hidden email]> wrote:
>
> Hi,
>
> TL;DR
> -----
>
>  I define the path
>
>    /cm/shared/apps/R/site-library/3.4.2
>
>  and add it to libPath.  Why does libPath then display it as
>
>    /cm/shared/apps/R/site-library/3.4
>
>  ?
>

Because it is a symbolic link.

?.libPaths says

"For consistency, the paths are always normalized by normalizePath(winslash = "/")."

and ?normalizePath says

"...the Unix-alike platform ... attempts to turn paths into absolute paths in their canonical form (no ./, ../ nor symbolic links)."  

HTH,

Chuck
______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: libPaths displays truncated path?

David Winsemius
In reply to this post by Loris Bennett-2

> On Nov 23, 2017, at 4:34 AM, Loris Bennett <[hidden email]> wrote:
>
> Hi,
>
> TL;DR
> -----
>
>  I define the path
>
>    /cm/shared/apps/R/site-library/3.4.2
>
>  and add it to libPath.  Why does libPath then display it as
>
>    /cm/shared/apps/R/site-library/3.4

Generally one only has a different library for each major version of R. Major versions are consider just the first two numbers in the dot-separated versions system. Apparently libPaths is "smart" enough to make an effort to adhere to that convention.  It appears your definition of "major" differs from the usual convention.

--
David

>
>  ?
>
> Long version
> ------------
>
> I run a cluster of diskless nodes for which the OS is loaded
> directly into RAM and other software is provided by an NFS server.
> However, in the case of R, we use the R version provided by the OS and
> just install additional packages on the NFS server.
>
> So that R can find these additional packages, I have the following in
> the site-wide Rprofile
>
>  v <- R.Version()
>  base_path = "/cm/shared/apps/R/site-library/"
>  major_minor_version = paste(v["major"],v["minor"],sep=".")
>  cm_shared_lib_path = paste0(base_path,major_minor_version)
>  full_cm_shared_lib_path <- c(file.path(chartr("\\", "/", R.home()), "site-library"), cm_shared_lib_path)
>  .libPaths( c( .libPaths(), full_cm_shared_lib_path ) )
>
> Thus, when I start R I get this:
>
>> full_cm_shared_lib_path
>  [1] "/usr/lib64/R/site-library"          
>  [2] "/cm/shared/apps/R/site-library/3.4.2"
>
> but also this
>
>> .libPaths()
>  [1] "/home/loris/R/x86_64-redhat-linux-gnu-library/3.4"
>  [2] "/usr/lib64/R/library"                            
>  [3] "/usr/share/R/library"                            
>  [4] "/usr/lib64/R/site-library"                        
>  [5] "/cm/shared/apps/R/site-library/3.4"      
>
> However, in order to get R to find the packages, I have to add a
> symbolic link, thus:
>
>  [/cm/shared/apps/R/site-library] $ ls -l 3.4.2
>  lrwxrwxrwx 1 root root 3 Nov 23 09:21 3.4.2 -> 3.4
>
> So, my mistake was to think that "minor" would return "4", whereas it in
> fact returns "4.2".  So I actually set the path to ".../3.4.2" and
> that's where R looks for packages.
>
> But why does libPaths display the the path I *thought* I had set, but in
> fact looks at the path I *really did* set?
>
> Cheers,
>
> Loris
>
> --
> Dr. Loris Bennett (Mr.)
> ZEDAT, Freie Universität Berlin         Email [hidden email]
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

David Winsemius
Alameda, CA, USA

'Any technology distinguishable from magic is insufficiently advanced.'   -Gehm's Corollary to Clarke's Third Law

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: libPaths displays truncated path?

Peter Dalgaard-2

> On 23 Nov 2017, at 17:36 , David Winsemius <[hidden email]> wrote:
>
>>
>> On Nov 23, 2017, at 4:34 AM, Loris Bennett <[hidden email]> wrote:
>>
>> Hi,
>>
>> TL;DR
>> -----
>>
>> I define the path
>>
>>   /cm/shared/apps/R/site-library/3.4.2
>>
>> and add it to libPath.  Why does libPath then display it as
>>
>>   /cm/shared/apps/R/site-library/3.4
>
> Generally one only has a different library for each major version of R. Major versions are consider just the first two numbers in the dot-separated versions system. Apparently libPaths is "smart" enough to make an effort to adhere to that convention.  It appears your definition of "major" differs from the usual convention.

Nope. Chuck B. is almost certainly right: The symlink did it, with the normalizePath, in .libPaths(). (And the OP has cause and effect reversed.)

Not even R functions are smart enough to make a decision like you suggest without actually containing code to do it:

> .libPaths
function (new)
{
    if (!missing(new)) {
        new <- Sys.glob(path.expand(new))
        paths <- c(new, .Library.site, .Library)
        paths <- paths[dir.exists(paths)]
        .lib.loc <<- unique(normalizePath(paths, "/"))
    }
    else .lib.loc
}
<bytecode: 0x10219a988>
<environment: 0x1021985f0>

-pd

>
> --
> David
>>
>> ?
>>
>> Long version
>> ------------
>>
>> I run a cluster of diskless nodes for which the OS is loaded
>> directly into RAM and other software is provided by an NFS server.
>> However, in the case of R, we use the R version provided by the OS and
>> just install additional packages on the NFS server.
>>
>> So that R can find these additional packages, I have the following in
>> the site-wide Rprofile
>>
>> v <- R.Version()
>> base_path = "/cm/shared/apps/R/site-library/"
>> major_minor_version = paste(v["major"],v["minor"],sep=".")
>> cm_shared_lib_path = paste0(base_path,major_minor_version)
>> full_cm_shared_lib_path <- c(file.path(chartr("\\", "/", R.home()), "site-library"), cm_shared_lib_path)
>> .libPaths( c( .libPaths(), full_cm_shared_lib_path ) )
>>
>> Thus, when I start R I get this:
>>
>>> full_cm_shared_lib_path
>> [1] "/usr/lib64/R/site-library"          
>> [2] "/cm/shared/apps/R/site-library/3.4.2"
>>
>> but also this
>>
>>> .libPaths()
>> [1] "/home/loris/R/x86_64-redhat-linux-gnu-library/3.4"
>> [2] "/usr/lib64/R/library"                            
>> [3] "/usr/share/R/library"                            
>> [4] "/usr/lib64/R/site-library"                        
>> [5] "/cm/shared/apps/R/site-library/3.4"      
>>
>> However, in order to get R to find the packages, I have to add a
>> symbolic link, thus:
>>
>> [/cm/shared/apps/R/site-library] $ ls -l 3.4.2
>> lrwxrwxrwx 1 root root 3 Nov 23 09:21 3.4.2 -> 3.4
>>
>> So, my mistake was to think that "minor" would return "4", whereas it in
>> fact returns "4.2".  So I actually set the path to ".../3.4.2" and
>> that's where R looks for packages.
>>
>> But why does libPaths display the the path I *thought* I had set, but in
>> fact looks at the path I *really did* set?
>>
>> Cheers,
>>
>> Loris
>>
>> --
>> Dr. Loris Bennett (Mr.)
>> ZEDAT, Freie Universität Berlin         Email [hidden email]
>>
>> ______________________________________________
>> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
>> https://stat.ethz.ch/mailman/listinfo/r-help
>> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
>> and provide commented, minimal, self-contained, reproducible code.
>
> David Winsemius
> Alameda, CA, USA
>
> 'Any technology distinguishable from magic is insufficiently advanced.'   -Gehm's Corollary to Clarke's Third Law
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: libPaths displays truncated path?

Loris Bennett-2
In reply to this post by Berry, Charles
"Berry, Charles" <[hidden email]> writes:

>> On Nov 23, 2017, at 4:34 AM, Loris Bennett <[hidden email]> wrote:
>>
>> Hi,
>>
>> TL;DR
>> -----
>>
>>  I define the path
>>
>>    /cm/shared/apps/R/site-library/3.4.2
>>
>>  and add it to libPath.  Why does libPath then display it as
>>
>>    /cm/shared/apps/R/site-library/3.4
>>
>>  ?
>>
>
> Because it is a symbolic link.
>
> ?.libPaths says
>
> "For consistency, the paths are always normalized by normalizePath(winslash = "/")."
>
> and ?normalizePath says
>
> "...the Unix-alike platform ... attempts to turn paths into absolute paths in their canonical form (no ./, ../ nor symbolic links)."  
>
> HTH,
>
> Chuck

That explains it.  A tad bizarre to my mind, but, as usual with R and as
I should have known, documented.

Thanks,

Loris

--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin         Email [hidden email]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: libPaths displays truncated path?

Loris Bennett-2
In reply to this post by Peter Dalgaard-2
peter dalgaard <[hidden email]> writes:

[snip (40 lines)]

>> I don't quite understand the point about cause and effect.  If
>> normalizePath changes the symbolic link from
>>
>>  /cm/shared/apps/R/site-library/3.4.2
>>
>> to the actual directory
>>
>>  /cm/shared/apps/R/site-library/3.4
>>
>> in libPaths, why does R fail to find the packages *unless* the symbolic
>> link exists?
>
>
> I was unclear/confused about that. I somehow got the impression that
> the symlink was the cause of the problem that it solved, but that is
> not true.
>
> AFAICT there are two scenarios
>
> Ground facts:
>
> library is in  .../3.4
> you're trying to add .../3.4.2 using .libPaths
>
> (A) no symlink:
>
> Actual libpath will have .../3.4.2 which isn't there --> FAIL
>
> (B) symlink in place
>
> Actual libpath normalized to use .../3.4 which is there --> SUCCESS
>
> If normalizePath hadn't been applied, (B) would still succeed, the
> libpath would be to 3.4.2 which exists as a link to 3.4 and (A) would
> still fail because you are fundamentally looking in the wrong place.
>
> -pd

Now I understand.  I had scenario A and FAIL, then added the symbolic
link to get B and SUCCESS, whereby normalizePath means that the actual
directory and not the symbolic link is being used.

It all makes sense now (in an odd sort of way).  Thank you.

[snip (13 lines)]

--
Dr. Loris Bennett (Mr.)
ZEDAT, Freie Universität Berlin         Email [hidden email]

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.