suppressing stderr output from system() calls

[Mark Cowley]

Dear list,
I'm trying to suppress/redirect/squash the output from commands like install.packages, or download.file. The problem is that none of: sink(..., type="message"), sink(..., type="output"), capture.output, suppressMessages are quite doing the trick. Output gets written to the stderr stream, despite any combination of the above suppression commands.
According to ?sink:
Messages sent to ‘stderr()’ (including those from ‘message’, ‘warning’ and ‘stop’) can be diverted by ‘sink(type = "message")’ (see below).

I'm pretty sure it's the system(), or .Internal() calls which are the culprit, which currently write the majority (all?) of their output to the stderr stream.

Simple example:
con <- file("stderr.txt", "w")
sink(con, type="message")
system("ls")
sink(NULL, type="message")
close(con)
# instead of the output going to stderr.txt, it gets printed to the console.

# no good either
capture.output(system("ls"))
character(0)

This is an issue, since i'm writing GenePattern modules to run R code, and if anything is written to stderr, then the job gets hit with a 'job failed' status, when all that might have happened is an R package got installed, or a file got downloaded via FTP.

Any ideas? Can system() and .Internal() output be redirected to stdout?

cheers,
Mark


sessionInfo()
R version 2.13.1 (2011-07-08)
Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)

locale:
[1] en_AU.UTF-8/en_AU.UTF-8/C/C/en_AU.UTF-8/en_AU.UTF-8

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base    
>



-----------------------------------------------------
Mark Cowley, PhD

Pancreatic Cancer Program | Peter Wills Bioinformatics Centre
Garvan Institute of Medical Research, Sydney, Australia
-----------------------------------------------------


        [[alternative HTML version deleted]]


______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Simon Urbanek]

See ?system, in particular ignore.stdout, ignore.stderr and intern.

Cheers,
Simon


On Oct 5, 2011, at 5:36 AM, Mark Cowley wrote:

> Dear list,
> I'm trying to suppress/redirect/squash the output from commands like install.packages, or download.file. The problem is that none of: sink(..., type="message"), sink(..., type="output"), capture.output, suppressMessages are quite doing the trick. Output gets written to the stderr stream, despite any combination of the above suppression commands.
> According to ?sink:
> Messages sent to Œstderr()‚ (including those from Œmessage‚, Œwarning‚ and Œstop‚) can be diverted by Œsink(type = "message")‚ (see below).
>
> I'm pretty sure it's the system(), or .Internal() calls which are the culprit, which currently write the majority (all?) of their output to the stderr stream.
>
> Simple example:
> con <- file("stderr.txt", "w")
> sink(con, type="message")
> system("ls")
> sink(NULL, type="message")
> close(con)
> # instead of the output going to stderr.txt, it gets printed to the console.
>
> # no good either
> capture.output(system("ls"))
> character(0)
>
> This is an issue, since i'm writing GenePattern modules to run R code, and if anything is written to stderr, then the job gets hit with a 'job failed' status, when all that might have happened is an R package got installed, or a file got downloaded via FTP.
>
> Any ideas? Can system() and .Internal() output be redirected to stdout?
>
> cheers,
> Mark
>
>
> sessionInfo()
> R version 2.13.1 (2011-07-08)
> Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)
>
> locale:
> [1] en_AU.UTF-8/en_AU.UTF-8/C/C/en_AU.UTF-8/en_AU.UTF-8
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base    
>>
>
>
>
> -----------------------------------------------------
> Mark Cowley, PhD
>
> Pancreatic Cancer Program | Peter Wills Bioinformatics Centre
> Garvan Institute of Medical Research, Sydney, Australia
> -----------------------------------------------------
>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Prof Brian Ripley]

See ?system2 to answer your subject line.

But you seem confused about external programs and R's own stdout()
connection.  Have you read the R News article on connections?

On Wed, 5 Oct 2011, Mark Cowley wrote:

> Dear list,
> I'm trying to suppress/redirect/squash the output from commands like install.packages, or download.file. The problem is that none of: sink(..., type="message"), sink(..., type="output"), capture.output, suppressMessages are quite doing the trick. Output gets written to the stderr stream, despite any combination of the above suppression commands.
> According to ?sink:
> Messages sent to ?stderr()? (including those from ?message?, ?warning? and ?stop?) can be diverted by ?sink(type = "message")? (see below).
>
> I'm pretty sure it's the system(), or .Internal() calls which are the culprit, which currently write the majority (all?) of their output to the stderr stream.
>
> Simple example:
> con <- file("stderr.txt", "w")
> sink(con, type="message")
> system("ls")
> sink(NULL, type="message")
> close(con)
> # instead of the output going to stderr.txt, it gets printed to the console.
>
> # no good either
> capture.output(system("ls"))
> character(0)
>
> This is an issue, since i'm writing GenePattern modules to run R code, and if anything is written to stderr, then the job gets hit with a 'job failed' status, when all that might have happened is an R package got installed, or a file got downloaded via FTP.
>
> Any ideas? Can system() and .Internal() output be redirected to stdout?
>
> cheers,
> Mark
>
>
> sessionInfo()
> R version 2.13.1 (2011-07-08)
> Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)
>
> locale:
> [1] en_AU.UTF-8/en_AU.UTF-8/C/C/en_AU.UTF-8/en_AU.UTF-8
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>>
>
>
>
> -----------------------------------------------------
> Mark Cowley, PhD
>
> Pancreatic Cancer Program | Peter Wills Bioinformatics Centre
> Garvan Institute of Medical Research, Sydney, Australia
> -----------------------------------------------------
>
>
> [[alternative HTML version deleted]]
>
>

--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  [phone] (self)
1 South Parks Road,                     [phone] (PA)
Oxford OX1 3TG, UK                Fax:  [phone]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Martin Morgan]

On 10/05/2011 02:36 AM, Mark Cowley wrote:
> Dear list,

> I'm trying to suppress/redirect/squash the output from commands like
> install.packages, or download.file. The problem is that none of:
> sink(..., type="message"), sink(..., type="output"), capture.output,
> suppressMessages are quite doing the trick. Output gets written to
> the stderr stream, despite any combination of the above suppression
> commands.

Hi Mark --

For download.file, the argument quiet=TRUE suppresses output messages;
this option can be used in ... for install.packages, too (according to
the documentation). This is good enough to quieten the regular chatter
on a successful operation. To catch errors and suppress warnings also,
maybe a construct like

url = "http://r-project.org/doesnotexist"
tryCatch(suppressWarnings(
     capture.output(download.file(url, tempfile(), quiet=TRUE))),
     error=function(...) {})

Martin

> According to ?sink:
> Messages sent to ‘stderr()’ (including those from ‘message’, ‘warning’ and ‘stop’) can be diverted by ‘sink(type = "message")’ (see below).
>
> I'm pretty sure it's the system(), or .Internal() calls which are the culprit, which currently write the majority (all?) of their output to the stderr stream.
>
> Simple example:
> con<- file("stderr.txt", "w")
> sink(con, type="message")
> system("ls")
> sink(NULL, type="message")
> close(con)
> # instead of the output going to stderr.txt, it gets printed to the console.
>
> # no good either
> capture.output(system("ls"))
> character(0)
>
> This is an issue, since i'm writing GenePattern modules to run R code, and if anything is written to stderr, then the job gets hit with a 'job failed' status, when all that might have happened is an R package got installed, or a file got downloaded via FTP.
>
> Any ideas? Can system() and .Internal() output be redirected to stdout?
>
> cheers,
> Mark
>
>
> sessionInfo()
> R version 2.13.1 (2011-07-08)
> Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)
>
> locale:
> [1] en_AU.UTF-8/en_AU.UTF-8/C/C/en_AU.UTF-8/en_AU.UTF-8
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>>
>
>
>
> -----------------------------------------------------
> Mark Cowley, PhD
>
> Pancreatic Cancer Program | Peter Wills Bioinformatics Centre
> Garvan Institute of Medical Research, Sydney, Australia
> -----------------------------------------------------
>
>
> [[alternative HTML version deleted]]
>
>
>
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

--
Computational Biology
Fred Hutchinson Cancer Research Center
1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109

Location: M1-B861
Telephone: [phone]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

[Mark Cowley]

Thanks Martin, Simon and Brian,
They're all good suggestions for code that I write, however how can I suppress stderr from pre-existing code. For instance, install.packages has 3 system calls.
Perhaps a better worded question would be: how can I redirect the output of this command:

install.packages("MASS", lib="/tmp", type="source")

As I mentioned before, sink doesn't do the trick
con <- file("stderr.txt", "w")
sink(con, type="message")
install.packages("MASS", lib="/tmp", type="source")
sink(NULL, type="message")
close(con)

Martin will be familiar with the Bioconductor package oligo, which likes to download pd info package on the fly and install them - the embedded call to download.file is noisy, as is the resulting installation of said package.

Short of rewriting install.package & all code which has embedded install.packages calls, are there any other ways?

@Brian, thanks for the pointer to the news article on connections - I did learn something there
cheers,

Mark

On 06/10/2011, at 3:59 AM, Martin Morgan wrote:

> On 10/05/2011 02:36 AM, Mark Cowley wrote:
>> Dear list,
>
>> I'm trying to suppress/redirect/squash the output from commands like
>> install.packages, or download.file. The problem is that none of:
>> sink(..., type="message"), sink(..., type="output"), capture.output,
>> suppressMessages are quite doing the trick. Output gets written to
>> the stderr stream, despite any combination of the above suppression
>> commands.
>
> Hi Mark --
>
> For download.file, the argument quiet=TRUE suppresses output messages; this option can be used in ... for install.packages, too (according to the documentation). This is good enough to quieten the regular chatter on a successful operation. To catch errors and suppress warnings also, maybe a construct like
>
> url = "http://r-project.org/doesnotexist"
> tryCatch(suppressWarnings(
>    capture.output(download.file(url, tempfile(), quiet=TRUE))),
>    error=function(...) {})
>
> Martin
>
>> According to ?sink:
>> Messages sent to ‘stderr()’ (including those from ‘message’, ‘warning’ and ‘stop’) can be diverted by ‘sink(type = "message")’ (see below).
>>
>> I'm pretty sure it's the system(), or .Internal() calls which are the culprit, which currently write the majority (all?) of their output to the stderr stream.
>>
>> Simple example:
>> con<- file("stderr.txt", "w")
>> sink(con, type="message")
>> system("ls")
>> sink(NULL, type="message")
>> close(con)
>> # instead of the output going to stderr.txt, it gets printed to the console.
>>
>> # no good either
>> capture.output(system("ls"))
>> character(0)
>>
>> This is an issue, since i'm writing GenePattern modules to run R code, and if anything is written to stderr, then the job gets hit with a 'job failed' status, when all that might have happened is an R package got installed, or a file got downloaded via FTP.
>>
>> Any ideas? Can system() and .Internal() output be redirected to stdout?
>>
>> cheers,
>> Mark
>>
>>
>> sessionInfo()
>> R version 2.13.1 (2011-07-08)
>> Platform: x86_64-apple-darwin9.8.0/x86_64 (64-bit)
>>
>> locale:
>> [1] en_AU.UTF-8/en_AU.UTF-8/C/C/en_AU.UTF-8/en_AU.UTF-8
>>
>> attached base packages:
>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>>
>>
>>
>>
>> -----------------------------------------------------
>> Mark Cowley, PhD
>>
>> Pancreatic Cancer Program | Peter Wills Bioinformatics Centre
>> Garvan Institute of Medical Research, Sydney, Australia
>> -----------------------------------------------------
>>
>>
>> [[alternative HTML version deleted]]
>>
>>
>>
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
> --
> Computational Biology
> Fred Hutchinson Cancer Research Center
> 1100 Fairview Ave. N. PO Box 19024 Seattle, WA 98109
>
> Location: M1-B861
> Telephone: [phone]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel