Bug 17432 in readLines with R >= 3.5.0 still a problem

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug 17432 in readLines with R >= 3.5.0 still a problem

Chris Culnane
Bug 17432 (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17432) is still a problem when using pipes for IPC.

The bug is evident when calling R from another process and trying to communicate via StdIn. R will buffer the input and not read lines until the buffer is exceeded or StdIn is closed by the sending process. This prevents interactive communication between a calling process and a child R process.

From a quick look at the source code, it looks like the bug is caused by only disabling buffering when isatty() returns true for a file descriptor (connections.c). This fixes the original bug when the script is run in a terminal, but doesn't help for pipes, which will return false for isatty().

An example R script and python script are provided to demonstrate the problem:

R script (example.r):
================
f <- file("stdin")
open(f)
while(length(line <- readLines(f,n=1)) > 0) {
  write(line, stderr())
}

Python3 script:
============
import sys, os, subprocess
process = subprocess.Popen(['Rscript', 'example.r'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
for line in sys.stdin:
    process.stdin.write((line + '\n').encode('utf-8'))
    process.stdin.flush()


Expected Behaviour:
Run python script, each line entered is echoed back immediately by the R script - which is what happens on 3.4.4

Observed Behaviiour on >=3.5.0 (include devel):
The R script does not process lines as they are sent, it only receives them when StdIn is closed.


Best Regards

Chris

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug 17432 in readLines with R >= 3.5.0 still a problem

Michael Lawrence-3
Thanks, I responded to this on bugzilla.
On Wed, Sep 12, 2018 at 9:04 AM Chris Culnane
<[hidden email]> wrote:

>
> Bug 17432 (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17432) is still a problem when using pipes for IPC.
>
> The bug is evident when calling R from another process and trying to communicate via StdIn. R will buffer the input and not read lines until the buffer is exceeded or StdIn is closed by the sending process. This prevents interactive communication between a calling process and a child R process.
>
> From a quick look at the source code, it looks like the bug is caused by only disabling buffering when isatty() returns true for a file descriptor (connections.c). This fixes the original bug when the script is run in a terminal, but doesn't help for pipes, which will return false for isatty().
>
> An example R script and python script are provided to demonstrate the problem:
>
> R script (example.r):
> ================
> f <- file("stdin")
> open(f)
> while(length(line <- readLines(f,n=1)) > 0) {
>   write(line, stderr())
> }
>
> Python3 script:
> ============
> import sys, os, subprocess
> process = subprocess.Popen(['Rscript', 'example.r'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
> for line in sys.stdin:
>     process.stdin.write((line + '\n').encode('utf-8'))
>     process.stdin.flush()
>
>
> Expected Behaviour:
> Run python script, each line entered is echoed back immediately by the R script - which is what happens on 3.4.4
>
> Observed Behaviiour on >=3.5.0 (include devel):
> The R script does not process lines as they are sent, it only receives them when StdIn is closed.
>
>
> Best Regards
>
> Chris
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug 17432 in readLines with R >= 3.5.0 still a problem

Jennifer Lyon
In reply to this post by Chris Culnane
Michael:

I don't see any comments on Bug 17432 (
https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17432) later than June
1, 2018. Would you please supply a link pointing to the followup to this
discussion on bugzilla?

Thanks.

Jen.

> On Thu Sep 13 14:14:46 CEST 2018 Michael Lawrence wrote:
>
> Thanks, I responded to this on bugzilla.
> On Wed, Sep 12, 2018 at 9:04 AM Chris Culnane
> <christopher.culnane using unimelb.edu.au> wrote:
> >
> > Bug 17432 (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17432)
is still a problem when using pipes for IPC.
> >
> > The bug is evident when calling R from another process and trying to
communicate via StdIn. R will buffer the input and not read lines until the
buffer is exceeded or StdIn is closed by the sending process. This prevents
interactive communication between a calling process and a child R process.
> >
> > From a quick look at the source code, it looks like the bug is caused
by only disabling buffering when isatty() returns true for a file
descriptor (connections.c). This fixes the original bug when the script is
run in a terminal, but doesn't help for pipes, which will return false for
isatty().
> >
> > An example R script and python script are provided to demonstrate the
problem:

> >
> > R script (example.r):
> > ================
> > f <- file("stdin")
> > open(f)
> > while(length(line <- readLines(f,n=1)) > 0) {
> >   write(line, stderr())
> > }
> >
> > Python3 script:
> > ============
> > import sys, os, subprocess
> > process = subprocess.Popen(['Rscript', 'example.r'],
stdin=subprocess.PIPE, stdout=subprocess.PIPE)
> > for line in sys.stdin:
> >     process.stdin.write((line + '\n').encode('utf-8'))
> >     process.stdin.flush()
> >
> >
> > Expected Behaviour:
> > Run python script, each line entered is echoed back immediately by the
R script - which is what happens on 3.4.4
> >
> > Observed Behaviiour on >=3.5.0 (include devel):
> > The R script does not process lines as they are sent, it only receives
them when StdIn is closed.
> >
> >
> > Best Regards
> >
> > Chris

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug 17432 in readLines with R >= 3.5.0 still a problem

Michael Lawrence-3
The actual bug corresponding to this thread is:
https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17470
On Fri, Sep 14, 2018 at 9:22 AM Jennifer Lyon <[hidden email]> wrote:

>
> Michael:
>
> I don't see any comments on Bug 17432 (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17432) later than June 1, 2018. Would you please supply a link pointing to the followup to this discussion on bugzilla?
>
> Thanks.
>
> Jen.
>
> > On Thu Sep 13 14:14:46 CEST 2018 Michael Lawrence wrote:
> >
> > Thanks, I responded to this on bugzilla.
> > On Wed, Sep 12, 2018 at 9:04 AM Chris Culnane
> > <christopher.culnane using unimelb.edu.au> wrote:
> > >
> > > Bug 17432 (https://bugs.r-project.org/bugzilla3/show_bug.cgi?id=17432) is still a problem when using pipes for IPC.
> > >
> > > The bug is evident when calling R from another process and trying to communicate via StdIn. R will buffer the input and not read lines until the buffer is exceeded or StdIn is closed by the sending process. This prevents interactive communication between a calling process and a child R process.
> > >
> > > From a quick look at the source code, it looks like the bug is caused by only disabling buffering when isatty() returns true for a file descriptor (connections.c). This fixes the original bug when the script is run in a terminal, but doesn't help for pipes, which will return false for isatty().
> > >
> > > An example R script and python script are provided to demonstrate the problem:
> > >
> > > R script (example.r):
> > > ================
> > > f <- file("stdin")
> > > open(f)
> > > while(length(line <- readLines(f,n=1)) > 0) {
> > >   write(line, stderr())
> > > }
> > >
> > > Python3 script:
> > > ============
> > > import sys, os, subprocess
> > > process = subprocess.Popen(['Rscript', 'example.r'], stdin=subprocess.PIPE, stdout=subprocess.PIPE)
> > > for line in sys.stdin:
> > >     process.stdin.write((line + '\n').encode('utf-8'))
> > >     process.stdin.flush()
> > >
> > >
> > > Expected Behaviour:
> > > Run python script, each line entered is echoed back immediately by the R script - which is what happens on 3.4.4
> > >
> > > Observed Behaviiour on >=3.5.0 (include devel):
> > > The R script does not process lines as they are sent, it only receives them when StdIn is closed.
> > >
> > >
> > > Best Regards
> > >
> > > Chris
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel