file.exists() on device files

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

file.exists() on device files

Benjamin Tyner
Hi,

On my linux machine (Ubuntu, and also tested on RHEL), I am curious to
know what might be causing file.exists (and also normalizePath) to not
see the final device file here:

    > list.files("/dev/fd", full.names = TRUE)
    [1] "/dev/fd/0" "/dev/fd/1" "/dev/fd/2" "/dev/fd/3"
    > file.exists(list.files("/dev/fd", full.names = TRUE))
    [1]  TRUE  TRUE  TRUE FALSE
    > normalizePath(list.files("/dev/fd", full.names = TRUE))
    [1] "/dev/pts/2" "/dev/pts/2" "/dev/pts/2" "/dev/fd/3"
    Warning message:
    In normalizePath(list.files("/dev/fd", full.names = TRUE)) :
      path[4]="/dev/fd/3": No such file or directory

    > sessionInfo()
    R version 3.2.5 (2016-04-14)
    Platform: x86_64-pc-linux-gnu (64-bit)
    Running under: Ubuntu 16.04.1 LTS

    locale:
     [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C
     [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8
     [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8
     [7] LC_PAPER=en_US.UTF-8       LC_NAME=C
     [9] LC_ADDRESS=C               LC_TELEPHONE=C
    [11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C

    attached base packages:
    [1] stats     graphics  grDevices utils     datasets  methods base

Regards
Ben

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: [FORGED] file.exists() on device files

Rolf Turner
On 11/01/17 23:12, Benjamin Tyner wrote:

> Hi,
>
> On my linux machine (Ubuntu, and also tested on RHEL), I am curious to
> know what might be causing file.exists (and also normalizePath) to not
> see the final device file here:
>
>    > list.files("/dev/fd", full.names = TRUE)
>    [1] "/dev/fd/0" "/dev/fd/1" "/dev/fd/2" "/dev/fd/3"
>    > file.exists(list.files("/dev/fd", full.names = TRUE))
>    [1]  TRUE  TRUE  TRUE FALSE
>    > normalizePath(list.files("/dev/fd", full.names = TRUE))
>    [1] "/dev/pts/2" "/dev/pts/2" "/dev/pts/2" "/dev/fd/3"
>    Warning message:
>    In normalizePath(list.files("/dev/fd", full.names = TRUE)) :
>      path[4]="/dev/fd/3": No such file or directory

(a) Exactly the same thing happens to me (I am also running Ubuntu 16.04).

(b) Things get a bit confused because /dev/fd is actually a symbolic link:

> $ ls -l /dev/fd
> lrwxrwxrwx 1 root root 13 Jan  6 20:16 /dev/fd -> /proc/self/fd/

(c) But then doing

>> file.exists(list.files("/proc/self/fd", full.names = TRUE))

Gives the same result as before:

> [1]  TRUE  TRUE  TRUE FALSE

(d) It turns out that the four "files" in /proc/self/fd are again
symbolic links:

> $ ls -l /proc/self/fd
> total 0
> lrwx------ 1 rolf rolf 64 Jan 12 12:32 0 -> /dev/pts/3
> lrwx------ 1 rolf rolf 64 Jan 12 12:32 1 -> /dev/pts/3
> lrwx------ 1 rolf rolf 64 Jan 12 12:32 2 -> /dev/pts/3
> lr-x------ 1 rolf rolf 64 Jan 12 12:32 3 -> /proc/7150/fd/

(e) But now do it again!!!

> $ ls -l /proc/self/fd
> total 0
> lrwx------ 1 rolf rolf 64 Jan 12 12:32 0 -> /dev/pts/3
> lrwx------ 1 rolf rolf 64 Jan 12 12:32 1 -> /dev/pts/3
> lrwx------ 1 rolf rolf 64 Jan 12 12:32 2 -> /dev/pts/3
> lr-x------ 1 rolf rolf 64 Jan 12 12:32 3 -> /proc/7154/fd/

Different number; 7154 rather than 7150.

(f) The name "/proc" would seem to imply that this has something to do
with processes; the directories "7150", "7154" etc. are being created
and removed on the fly, as a result of some process (presumably the "ls"
process) starting and finishing.

I have no insight into what is being effected here, or what is really
going on "deep down", but the foregoing is some sort of "explanation".
By the time file.exists() is invoked, the ls process called by
list.files() has finished and the associated directory (e.g. "7150",
"7154", ...) has ceased to be.

What you do with this "explanation" is up to you.  My advice would be to
forget about it and go to the pub! :-)

cheers,

Rolf Turner

--
Technical Editor ANZJS
Department of Statistics
University of Auckland
Phone: +64-9-373-7599 ext. 88276

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: [FORGED] file.exists() on device files

Henrik Bengtsson-5
On Wed, Jan 11, 2017 at 3:56 PM, Rolf Turner <[hidden email]> wrote:

> On 11/01/17 23:12, Benjamin Tyner wrote:
>>
>> Hi,
>>
>> On my linux machine (Ubuntu, and also tested on RHEL), I am curious to
>> know what might be causing file.exists (and also normalizePath) to not
>> see the final device file here:
>>
>>    > list.files("/dev/fd", full.names = TRUE)
>>    [1] "/dev/fd/0" "/dev/fd/1" "/dev/fd/2" "/dev/fd/3"
>>    > file.exists(list.files("/dev/fd", full.names = TRUE))
>>    [1]  TRUE  TRUE  TRUE FALSE
>>    > normalizePath(list.files("/dev/fd", full.names = TRUE))
>>    [1] "/dev/pts/2" "/dev/pts/2" "/dev/pts/2" "/dev/fd/3"
>>    Warning message:
>>    In normalizePath(list.files("/dev/fd", full.names = TRUE)) :
>>      path[4]="/dev/fd/3": No such file or directory
>
>
> (a) Exactly the same thing happens to me (I am also running Ubuntu 16.04).
>
> (b) Things get a bit confused because /dev/fd is actually a symbolic link:
>
>> $ ls -l /dev/fd
>> lrwxrwxrwx 1 root root 13 Jan  6 20:16 /dev/fd -> /proc/self/fd/
>
>
> (c) But then doing
>
>>> file.exists(list.files("/proc/self/fd", full.names = TRUE))
>
>
> Gives the same result as before:
>
>> [1]  TRUE  TRUE  TRUE FALSE
>
>
> (d) It turns out that the four "files" in /proc/self/fd are again
> symbolic links:
>
>> $ ls -l /proc/self/fd
>> total 0
>> lrwx------ 1 rolf rolf 64 Jan 12 12:32 0 -> /dev/pts/3
>> lrwx------ 1 rolf rolf 64 Jan 12 12:32 1 -> /dev/pts/3
>> lrwx------ 1 rolf rolf 64 Jan 12 12:32 2 -> /dev/pts/3
>> lr-x------ 1 rolf rolf 64 Jan 12 12:32 3 -> /proc/7150/fd/
>
>
> (e) But now do it again!!!
>
>> $ ls -l /proc/self/fd
>> total 0
>> lrwx------ 1 rolf rolf 64 Jan 12 12:32 0 -> /dev/pts/3
>> lrwx------ 1 rolf rolf 64 Jan 12 12:32 1 -> /dev/pts/3
>> lrwx------ 1 rolf rolf 64 Jan 12 12:32 2 -> /dev/pts/3
>> lr-x------ 1 rolf rolf 64 Jan 12 12:32 3 -> /proc/7154/fd/
>
>
> Different number; 7154 rather than 7150.
>
> (f) The name "/proc" would seem to imply that this has something to do with
> processes; the directories "7150", "7154" etc. are being created and removed
> on the fly, as a result of some process (presumably the "ls"
> process) starting and finishing.

FYI, the /proc is there because Unix has something called the "proc
filesystem (procfs; https://en.wikipedia.org/wiki/Procfs) is a special
filesystem in Unix-like operating systems that presents information
about processes and other system information in a hierarchical
file-like structure".  For instance, you can query the uptime of the
machine by reading from /proc/uptime:

$ cat /proc/uptime
332826.96 661438.10

$ cat /proc/uptime
332871.40 661568.50


You can get all IDs (PIDs) of all processes currently running:

$ ls /proc/ | grep -E '^[0-9]+$'

and for each process you there are multiple attributes mapped as
files, e.g. if I start R as:

$ R --args -e "message('hello there')"

then I can query that process as:

$ pid=$(pidof R)
$ echo $pid
26323

$ cat /proc/26323/cmdline
/usr/lib/R/bin/exec/R--args-emessage('hello there')

Unix is neat

/Henrik

>
> I have no insight into what is being effected here, or what is really going
> on "deep down", but the foregoing is some sort of "explanation".
> By the time file.exists() is invoked, the ls process called by list.files()
> has finished and the associated directory (e.g. "7150", "7154", ...) has
> ceased to be.
>
> What you do with this "explanation" is up to you.  My advice would be to
> forget about it and go to the pub! :-)
>
> cheers,
>
> Rolf Turner
>
> --
> Technical Editor ANZJS
> Department of Statistics
> University of Auckland
> Phone: +64-9-373-7599 ext. 88276
>
>
> ______________________________________________
> [hidden email] mailing list -- To UNSUBSCRIBE and more, see
> https://stat.ethz.ch/mailman/listinfo/r-help
> PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
> and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: [FORGED] file.exists() on device files

Rolf Turner
On 12/01/17 16:33, Henrik Bengtsson wrote:

<SNIP>

> FYI, the /proc is there because Unix has something called the "proc
> filesystem (procfs; https://en.wikipedia.org/wiki/Procfs) is a special
> filesystem in Unix-like operating systems that presents information
> about processes and other system information in a hierarchical
> file-like structure".  For instance, you can query the uptime of the
> machine by reading from /proc/uptime:
>
> $ cat /proc/uptime
> 332826.96 661438.10
>
> $ cat /proc/uptime
> 332871.40 661568.50
>
>
> You can get all IDs (PIDs) of all processes currently running:
>
> $ ls /proc/ | grep -E '^[0-9]+$'
>
> and for each process you there are multiple attributes mapped as
> files, e.g. if I start R as:
>
> $ R --args -e "message('hello there')"
>
> then I can query that process as:
>
> $ pid=$(pidof R)
> $ echo $pid
> 26323
>
> $ cat /proc/26323/cmdline
> /usr/lib/R/bin/exec/R--args-emessage('hello there')
>
> Unix is neat

Indeed.  Couldn't agree more.  Thanks for the insight.

<SNIP>

cheers,

Rolf

--
Technical Editor ANZJS
Department of Statistics
University of Auckland
Phone: +64-9-373-7599 ext. 88276

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: [FORGED] file.exists() on device files

Benjamin Tyner
Thank you for the insights, Rolf and Henrik.

To give another example, this time in non-interactive mode,

    Rscript -e "file.exists(commandArgs(TRUE))" <(echo "Hi")

    [1] TRUE

versus

    Rscript -e "normalizePath(commandArgs(TRUE))" <(echo "Hi")
    [1] "/dev/fd/63"
    Warning message:
    In normalizePath(commandArgs(TRUE)) :
      path[1]="/dev/fd/63": No such file or directory

It almost seems like file.exists and normalizePath use separate criteria
for determining existence?

Regards

Ben

On 01/12/2017 01:42 AM, Rolf Turner wrote:

> On 12/01/17 16:33, Henrik Bengtsson wrote:
>
> <SNIP>
>
>> FYI, the /proc is there because Unix has something called the "proc
>> filesystem (procfs; https://en.wikipedia.org/wiki/Procfs) is a special
>> filesystem in Unix-like operating systems that presents information
>> about processes and other system information in a hierarchical
>> file-like structure".  For instance, you can query the uptime of the
>> machine by reading from /proc/uptime:
>>
>> $ cat /proc/uptime
>> 332826.96 661438.10
>>
>> $ cat /proc/uptime
>> 332871.40 661568.50
>>
>>
>> You can get all IDs (PIDs) of all processes currently running:
>>
>> $ ls /proc/ | grep -E '^[0-9]+$'
>>
>> and for each process you there are multiple attributes mapped as
>> files, e.g. if I start R as:
>>
>> $ R --args -e "message('hello there')"
>>
>> then I can query that process as:
>>
>> $ pid=$(pidof R)
>> $ echo $pid
>> 26323
>>
>> $ cat /proc/26323/cmdline
>> /usr/lib/R/bin/exec/R--args-emessage('hello there')
>>
>> Unix is neat
>
> Indeed.  Couldn't agree more.  Thanks for the insight.
>
> <SNIP>
>
> cheers,
>
> Rolf
>

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.
Reply | Threaded
Open this post in threaded view
|

Re: [FORGED] file.exists() on device files

Jeff Newmiller
Of course they have separate criteria for determining existence... they do different things, and therefore have different requirements for access permissions.
--
Sent from my phone. Please excuse my brevity.

On January 13, 2017 3:54:35 AM PST, Benjamin Tyner <[hidden email]> wrote:

>Thank you for the insights, Rolf and Henrik.
>
>To give another example, this time in non-interactive mode,
>
>    Rscript -e "file.exists(commandArgs(TRUE))" <(echo "Hi")
>
>    [1] TRUE
>
>versus
>
>    Rscript -e "normalizePath(commandArgs(TRUE))" <(echo "Hi")
>    [1] "/dev/fd/63"
>    Warning message:
>    In normalizePath(commandArgs(TRUE)) :
>      path[1]="/dev/fd/63": No such file or directory
>
>It almost seems like file.exists and normalizePath use separate
>criteria
>for determining existence?
>
>Regards
>
>Ben
>
>On 01/12/2017 01:42 AM, Rolf Turner wrote:
>> On 12/01/17 16:33, Henrik Bengtsson wrote:
>>
>> <SNIP>
>>
>>> FYI, the /proc is there because Unix has something called the "proc
>>> filesystem (procfs; https://en.wikipedia.org/wiki/Procfs) is a
>special
>>> filesystem in Unix-like operating systems that presents information
>>> about processes and other system information in a hierarchical
>>> file-like structure".  For instance, you can query the uptime of the
>>> machine by reading from /proc/uptime:
>>>
>>> $ cat /proc/uptime
>>> 332826.96 661438.10
>>>
>>> $ cat /proc/uptime
>>> 332871.40 661568.50
>>>
>>>
>>> You can get all IDs (PIDs) of all processes currently running:
>>>
>>> $ ls /proc/ | grep -E '^[0-9]+$'
>>>
>>> and for each process you there are multiple attributes mapped as
>>> files, e.g. if I start R as:
>>>
>>> $ R --args -e "message('hello there')"
>>>
>>> then I can query that process as:
>>>
>>> $ pid=$(pidof R)
>>> $ echo $pid
>>> 26323
>>>
>>> $ cat /proc/26323/cmdline
>>> /usr/lib/R/bin/exec/R--args-emessage('hello there')
>>>
>>> Unix is neat
>>
>> Indeed.  Couldn't agree more.  Thanks for the insight.
>>
>> <SNIP>
>>
>> cheers,
>>
>> Rolf
>>
>
>______________________________________________
>[hidden email] mailing list -- To UNSUBSCRIBE and more, see
>https://stat.ethz.ch/mailman/listinfo/r-help
>PLEASE do read the posting guide
>http://www.R-project.org/posting-guide.html
>and provide commented, minimal, self-contained, reproducible code.

______________________________________________
[hidden email] mailing list -- To UNSUBSCRIBE and more, see
https://stat.ethz.ch/mailman/listinfo/r-help
PLEASE do read the posting guide http://www.R-project.org/posting-guide.html
and provide commented, minimal, self-contained, reproducible code.