R started in terminal shell script or ESS steps on LD_LIBRARY_PATH?

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

R started in terminal shell script or ESS steps on LD_LIBRARY_PATH?

Marc Schwartz (via MN)
Hi all,

Just noted this behavior in the past couple of days, where if R is
started in a shell script such as:

  gnome-terminal [-e][-x] R

or in ESS (version 5.2.12 with Emacs or XEmacs), the LD_LIBRARY_PATH
environment variable is not properly appended to, resulting in the loss
of the pre-start value.

This is using R Version 2.2.1 Patched (2006-02-28 r37448) on FC4.

I noted this when attempting to access an Oracle 10g server using RODBC
(version 1.1-5) and to the best of my recollection, this is a new
problem.

It took me a while to figure this out and was about to send an e-mail to
r-sig-db since I was having trouble connecting using odbcConnect(), when
I noted the problem with LD_LIBRARY_PATH. I noted this quite by accident
as I was writing the e-mail to r-sig-db.

I was on the verge of mental vapor lock with this, so if I missed a
documented change in behavior, my apologies. In looking at the R startup
script, it seems appropriate in terms of appending to the existing
value. So it seems that this is being altered elsewhere presumably.



In the two cases I note above, I get:

> Sys.getenv("LD_LIBRARY_PATH")
                                     LD_LIBRARY_PATH
"/usr/local/lib/R/lib:/usr/local/lib:/usr/X11R6/lib"


whereas the pre-start value is:

  LD_LIBRARY_PATH=/usr/lib/oracle/10.2.0.1/client/lib



Importantly, if I start a gnome-terminal session and then start R from
the command line, I get:

> Sys.getenv("LD_LIBRARY_PATH")

LD_LIBRARY_PATH
"/usr/local/lib/R/lib:/usr/local/lib:/usr/X11R6/lib:/usr/lib/oracle/10.2.0.1/client/lib"


In this case, I can use odbcConnect() to the Oracle server and it works
the first time, every time.

This suggests to me some type of problem in the inheritance of the
pre-session environment in the two former cases, but again, I may be
missing something.


Even if I try:

LD_LIBRARY_PATH <- Sys.getenv("LD_LIBRARY_PATH")

Sys.putenv(LD_LIBRARY_PATH = paste(LD_LIBRARY_PATH,
           "/usr/lib/oracle/10.2.0.1/client/lib", sep = ":"))

and then:

> Sys.getenv("LD_LIBRARY_PATH")

LD_LIBRARY_PATH
"/usr/local/lib/R/lib:/usr/local/lib:/usr/X11R6/lib:/usr/lib/oracle/10.2.0.1/client/lib"


I still cannot get odbcConnect() to work. I still get the error that I
was getting in the two initial R startup situations:

Warning messages:
1: [RODBC] ERROR: state 01000, code 0, message [unixODBC][Driver
Manager]Can't open lib
'/usr/lib/oracle/10.2.0.1/client/lib/libsqora.so.10.1' :
libclntsh.so.10.1: cannot open shared object file: No such file or
directory
2: ODBC connection failed in: odbcDriverConnect(st, case = case,
believeNRows = believeNRows)


The above error is what has been driving me nuts for the past two days,
since clearly these files are present and it works in R from the
gnome-terminal command line and when using the sql*plus instant client
directly.  The LD_LIBRARY_PATH issue is the only one that I have noted
that appears to be different.


Any ideas on this?  Did I miss a change someplace?

Thanks for any insights.

Marc Schwartz

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R started in terminal shell script or ESS steps on LD_LIBRARY_PATH?

Marc Schwartz (via MN)
Hi all,

In follow up to my prior post on this issue, I have found a workaround,
but have not yet clearly identified the etiology of the problem.
Whatever it is, it is presumably unique to my system, though if anyone
can replicate this on another FC4 system...  :-)

The workaround involves booting to init 3 rather than init 5 and
starting X manually from the console. I found this after going through
some of the steps described below regarding my X configuration. In this
way, LD_LIBRARY_PATH is preserved and RODBC works without issue in both
ESS and the gnome-terminal shell start up script.

Dirk was kind enough to send me an offlist e-mail yesterday in reply,
which sparked some thoughts as I was away from the computer for a few
hours yesterday afternoon and evening.

Dirk's e-mail logically queried on any issues with gnome-terminal and/or
the bash shell itself.

Since this problem was new (this had all worked previously), I checked
to see if there had been any recent updates to either gnome-terminal or
bash in the FC repos. There were none, although there have been of
course updates to GNOME, GTK and other libs.

This got me to think about other updates since the last known time this
process worked properly. So I spent several hours last night and this
morning reviewing possibilities.

The last Xorg updates are from last September, so these are not new.

Other changes that I had made in the recent past include:

1. Modifying my xorg.conf to support nVidia TwinView hardware
acceleration functionality. TwinView is like xinerama mode, spanning
both displays to give me a virtual 3200x1200 screen, though supporting
HW acceleration on both displays. Previously I had been using two X
servers (also using the nVidia driver in non-xinerama mode) to support
my dual display configuration.  Reverting back to the old configuration
did not resolve the problem.

2. The last nVidia driver update (8178) was in December and this process
had worked since then.

3. There was an updated Cisco VPN client for Linux (4.8) to support
recent kernels. The VPN client is installed from source. This normally
starts up on boot as a service. Disabling the service, thus removing the
kernel module, did not resolve the problem.

4. I had updated the encryption of several of my partitions on my laptop
to use dm-crypt/LUKS with 256 bit AES from regular dm-crypt to take
advantage of pending LUKS support updates in HAL and other system
functions. Disabling the encryption (so the relevant kernel modules did
not load), logging in as root and running ESS from root's home did not
resolve the problem.

5. Just in case, I also reinstalled kernel version 2.6.15-1.1830
(running 1833 now) to see if there was any change there. No joy. I
cannot locate any of the 2.6.14 FC4 kernel versions, as these have been
removed from the repos, so it leaves open the possibility of something
in the .14 to .15 rebase change.


Other than routine system updates via yum, these are the only
"self-inflicted" changes that I have made recently. If any of the above
should spark some thoughts, let me know.

My plans are to live with this for now. FC5 is targeted for release next
Wednesday, presuming that it stays on schedule. I'll do a clean install
with that and see if anything is resolved, perhaps indicating some other
issue that is as yet unidentified.

Many thanks to Dirk for your assistance.

Best regards,

Marc Schwartz

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R started in terminal shell script or ESS steps on LD_LIBRARY_PATH?

Hin-Tak Leung-2
This sounds like a shell init issue... and you probably want to hunt
down where LD_LIBRARY path is *set*, rather than how it is inherited.

When you log in in run-level three, you get a login shell rather than
a normal interactive shell, and your startx inherits your login-shell's
environment, You get a normal interactive shell(?) inside
gnome-terminal/xterm if you start at run-level 5, and finally, you get
a non-interactive shell if you run a script, most of the time.
The environments in the three cases are all different, and
sometimes security related environment variables are not inherited
by forked sub-shells, such as LD_LIBRARY_PATH; or more likely,
LD_LIBRARY_PATH is set up for the login shell, and other shells
simply don't get it.

HTL

 From the INVOCATION part of bash's man page - assuming that's your
login shell, otherwise, others.
===========
When  bash is invoked as an interactive login shell, or as a non-inter-
active shell with the --login option, it first reads and executes  com-
mands  from  the file /etc/profile, if that file exists.  After reading
that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile,
in  that order, and reads and executes commands from the first one that
exists and is readable.  The --noprofile option may be  used  when  the
shell is started to inhibit this behavior.

When  a  login  shell  exits, bash reads and executes commands from the
file ~/.bash_logout, if it exists.

When an interactive shell that is not a login shell  is  started,  bash
reads  and executes commands from ~/.bashrc, if that file exists.  This
may be inhibited by using the --norc option.  The --rcfile file  option
will  force  bash  to  read  and  execute commands from file instead of
~/.bashrc.

When bash is started non-interactively, to  run  a  shell  script,  for
example, it looks for the variable BASH_ENV in the environment, expands
its value if it appears there, and uses the expanded value as the  name
of  a  file to read and execute.  Bash behaves as if the following com-
mand were executed:
    if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
but the value of the PATH variable is not used to search for  the  file
name.
=========



Marc Schwartz (via MN) wrote:

> Hi all,
>
> In follow up to my prior post on this issue, I have found a workaround,
> but have not yet clearly identified the etiology of the problem.
> Whatever it is, it is presumably unique to my system, though if anyone
> can replicate this on another FC4 system...  :-)
>
> The workaround involves booting to init 3 rather than init 5 and
> starting X manually from the console. I found this after going through
> some of the steps described below regarding my X configuration. In this
> way, LD_LIBRARY_PATH is preserved and RODBC works without issue in both
> ESS and the gnome-terminal shell start up script.
>
> Dirk was kind enough to send me an offlist e-mail yesterday in reply,
> which sparked some thoughts as I was away from the computer for a few
> hours yesterday afternoon and evening.
>
> Dirk's e-mail logically queried on any issues with gnome-terminal and/or
> the bash shell itself.
>
> Since this problem was new (this had all worked previously), I checked
> to see if there had been any recent updates to either gnome-terminal or
> bash in the FC repos. There were none, although there have been of
> course updates to GNOME, GTK and other libs.
>
> This got me to think about other updates since the last known time this
> process worked properly. So I spent several hours last night and this
> morning reviewing possibilities.
>
> The last Xorg updates are from last September, so these are not new.
>
> Other changes that I had made in the recent past include:
>
> 1. Modifying my xorg.conf to support nVidia TwinView hardware
> acceleration functionality. TwinView is like xinerama mode, spanning
> both displays to give me a virtual 3200x1200 screen, though supporting
> HW acceleration on both displays. Previously I had been using two X
> servers (also using the nVidia driver in non-xinerama mode) to support
> my dual display configuration.  Reverting back to the old configuration
> did not resolve the problem.
>
> 2. The last nVidia driver update (8178) was in December and this process
> had worked since then.
>
> 3. There was an updated Cisco VPN client for Linux (4.8) to support
> recent kernels. The VPN client is installed from source. This normally
> starts up on boot as a service. Disabling the service, thus removing the
> kernel module, did not resolve the problem.
>
> 4. I had updated the encryption of several of my partitions on my laptop
> to use dm-crypt/LUKS with 256 bit AES from regular dm-crypt to take
> advantage of pending LUKS support updates in HAL and other system
> functions. Disabling the encryption (so the relevant kernel modules did
> not load), logging in as root and running ESS from root's home did not
> resolve the problem.
>
> 5. Just in case, I also reinstalled kernel version 2.6.15-1.1830
> (running 1833 now) to see if there was any change there. No joy. I
> cannot locate any of the 2.6.14 FC4 kernel versions, as these have been
> removed from the repos, so it leaves open the possibility of something
> in the .14 to .15 rebase change.
>
>
> Other than routine system updates via yum, these are the only
> "self-inflicted" changes that I have made recently. If any of the above
> should spark some thoughts, let me know.
>
> My plans are to live with this for now. FC5 is targeted for release next
> Wednesday, presuming that it stays on schedule. I'll do a clean install
> with that and see if anything is resolved, perhaps indicating some other
> issue that is as yet unidentified.
>
> Many thanks to Dirk for your assistance.
>
> Best regards,
>
> Marc Schwartz
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R started in terminal shell script or ESS steps on LD_LIBRARY_PATH?

Marc Schwartz (via MN)
On Wed, 2006-03-08 at 13:54 -0600, Marc Schwartz wrote:

> On Wed, 2006-03-08 at 18:39 +0000, Hin-Tak Leung wrote:  
> > This sounds like a shell init issue... and you probably want to hunt
> > down where LD_LIBRARY path is *set*, rather than how it is inherited.
> >
> > When you log in in run-level three, you get a login shell rather than
> > a normal interactive shell, and your startx inherits your login-shell's
> > environment, You get a normal interactive shell(?) inside
> > gnome-terminal/xterm if you start at run-level 5, and finally, you get
> > a non-interactive shell if you run a script, most of the time.
> > The environments in the three cases are all different, and
> > sometimes security related environment variables are not inherited
> > by forked sub-shells, such as LD_LIBRARY_PATH; or more likely,
> > LD_LIBRARY_PATH is set up for the login shell, and other shells
> > simply don't get it.
> >
> > HTL
> >
> >  From the INVOCATION part of bash's man page - assuming that's your
> > login shell, otherwise, others.
> > ===========
> > When  bash is invoked as an interactive login shell, or as a non-inter-
> > active shell with the --login option, it first reads and executes  com-
> > mands  from  the file /etc/profile, if that file exists.  After reading
> > that file, it looks for ~/.bash_profile, ~/.bash_login, and ~/.profile,
> > in  that order, and reads and executes commands from the first one that
> > exists and is readable.  The --noprofile option may be  used  when  the
> > shell is started to inhibit this behavior.
> >
> > When  a  login  shell  exits, bash reads and executes commands from the
> > file ~/.bash_logout, if it exists.
> >
> > When an interactive shell that is not a login shell  is  started,  bash
> > reads  and executes commands from ~/.bashrc, if that file exists.  This
> > may be inhibited by using the --norc option.  The --rcfile file  option
> > will  force  bash  to  read  and  execute commands from file instead of
> > ~/.bashrc.
> >
> > When bash is started non-interactively, to  run  a  shell  script,  for
> > example, it looks for the variable BASH_ENV in the environment, expands
> > its value if it appears there, and uses the expanded value as the  name
> > of  a  file to read and execute.  Bash behaves as if the following com-
> > mand were executed:
> >     if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
> > but the value of the PATH variable is not used to search for  the  file
> > name.
> > =========
>

<SNIP>


Many thanks for the reply.  Given the subject matter feel free to
respond offlist with any further replies. I can post back should I
figure this out for the sake of closure on the thread.

LD_LIBRARY_PATH is set in ~/.bashrc and this has worked fine previously,
so I am still unclear as to what has changed. Though I am readily
willing to accept that something has been screwed up somehow. Presuming
that a system wide setting has been compromised in some fashion, the
pending clean install of FC 5 may be helpful.

If not, I may need to consider something in my own user profile
configuration.

I also logged into a KDE session from init 5, to see if perhaps whatever
was going on might have been GNOME specific. Unfortunately, the same
behavior is seen in KDE using ESS.

Two more data points under init 5 in GNOME however:

1. If I open a gnome-terminal console and start R from the CLI, things
work. If I exit R and use 'gnome-terminal -x R' within that same console
to mimic my startup script, it does not work, even though the variable
is clearly set in the console prior to entering the command. However, if
from the same initial gnome-terminal console session, I use 'xterm -e
R', it works.

2. If I use the "Run Application..." dialogue from the GNOME menu, type
in 'R' and check "Run in terminal", it does not work.

There is something subtle going on here, that I am just not seeing.

Thanks again for taking the time to reply.

Best regards,

Marc

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R started in terminal shell script or ESS steps on LD_LIBRARY_PATH?

Kasper Daniel Hansen
I am _not_ an expert on bash. But as far as I know, .bashrc is not  
read when you have a login session, whereas .bash_profile is. I have  
never really understood the deep differences between the two - I only  
have some superficial understanding. But for my purposes I just have a
   source .bashrc
in my .bash_profile script. In that way I set the same variables no  
matter what kind of session I have (clearly, I only use .bashrc).

There are important differences when you want to run a program at  
login, but you do not want to run it every time you start up a shell.  
But simply for setting environment variables, this ought to work.

Perhaps this helps? Or perhaps this is something deeper than the  
simple issue outlined above.

/Kasper


On Mar 8, 2006, at 1:42 PM, Marc Schwartz (via MN) wrote:

> On Wed, 2006-03-08 at 13:54 -0600, Marc Schwartz wrote:
>> On Wed, 2006-03-08 at 18:39 +0000, Hin-Tak Leung wrote:
>>> This sounds like a shell init issue... and you probably want to hunt
>>> down where LD_LIBRARY path is *set*, rather than how it is  
>>> inherited.
>>>
>>> When you log in in run-level three, you get a login shell rather  
>>> than
>>> a normal interactive shell, and your startx inherits your login-
>>> shell's
>>> environment, You get a normal interactive shell(?) inside
>>> gnome-terminal/xterm if you start at run-level 5, and finally,  
>>> you get
>>> a non-interactive shell if you run a script, most of the time.
>>> The environments in the three cases are all different, and
>>> sometimes security related environment variables are not inherited
>>> by forked sub-shells, such as LD_LIBRARY_PATH; or more likely,
>>> LD_LIBRARY_PATH is set up for the login shell, and other shells
>>> simply don't get it.
>>>
>>> HTL
>>>
>>>  From the INVOCATION part of bash's man page - assuming that's your
>>> login shell, otherwise, others.
>>> ===========
>>> When  bash is invoked as an interactive login shell, or as a non-
>>> inter-
>>> active shell with the --login option, it first reads and  
>>> executes  com-
>>> mands  from  the file /etc/profile, if that file exists.  After  
>>> reading
>>> that file, it looks for ~/.bash_profile, ~/.bash_login, and  
>>> ~/.profile,
>>> in  that order, and reads and executes commands from the first  
>>> one that
>>> exists and is readable.  The --noprofile option may be  used  
>>> when  the
>>> shell is started to inhibit this behavior.
>>>
>>> When  a  login  shell  exits, bash reads and executes commands  
>>> from the
>>> file ~/.bash_logout, if it exists.
>>>
>>> When an interactive shell that is not a login shell  is  
>>> started,  bash
>>> reads  and executes commands from ~/.bashrc, if that file  
>>> exists.  This
>>> may be inhibited by using the --norc option.  The --rcfile file  
>>> option
>>> will  force  bash  to  read  and  execute commands from file  
>>> instead of
>>> ~/.bashrc.
>>>
>>> When bash is started non-interactively, to  run  a  shell  
>>> script,  for
>>> example, it looks for the variable BASH_ENV in the environment,  
>>> expands
>>> its value if it appears there, and uses the expanded value as  
>>> the  name
>>> of  a  file to read and execute.  Bash behaves as if the  
>>> following com-
>>> mand were executed:
>>>     if [ -n "$BASH_ENV" ]; then . "$BASH_ENV"; fi
>>> but the value of the PATH variable is not used to search for  
>>> the  file
>>> name.
>>> =========
>>
>
> <SNIP>
>
>
> Many thanks for the reply.  Given the subject matter feel free to
> respond offlist with any further replies. I can post back should I
> figure this out for the sake of closure on the thread.
>
> LD_LIBRARY_PATH is set in ~/.bashrc and this has worked fine  
> previously,
> so I am still unclear as to what has changed. Though I am readily
> willing to accept that something has been screwed up somehow.  
> Presuming
> that a system wide setting has been compromised in some fashion, the
> pending clean install of FC 5 may be helpful.
>
> If not, I may need to consider something in my own user profile
> configuration.
>
> I also logged into a KDE session from init 5, to see if perhaps  
> whatever
> was going on might have been GNOME specific. Unfortunately, the same
> behavior is seen in KDE using ESS.
>
> Two more data points under init 5 in GNOME however:
>
> 1. If I open a gnome-terminal console and start R from the CLI, things
> work. If I exit R and use 'gnome-terminal -x R' within that same  
> console
> to mimic my startup script, it does not work, even though the variable
> is clearly set in the console prior to entering the command.  
> However, if
> from the same initial gnome-terminal console session, I use 'xterm -e
> R', it works.
>
> 2. If I use the "Run Application..." dialogue from the GNOME menu,  
> type
> in 'R' and check "Run in terminal", it does not work.
>
> There is something subtle going on here, that I am just not seeing.
>
> Thanks again for taking the time to reply.
>
> Best regards,
>
> Marc
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: R started in terminal shell script or ESS steps on LD_LIBRARY_PATH?

Marc Schwartz (via MN)
On Wed, 2006-03-08 at 14:48 -0800, Kasper Daniel Hansen wrote:

> I am _not_ an expert on bash. But as far as I know, .bashrc is not  
> read when you have a login session, whereas .bash_profile is. I have  
> never really understood the deep differences between the two - I only  
> have some superficial understanding. But for my purposes I just have a
>    source .bashrc
> in my .bash_profile script. In that way I set the same variables no  
> matter what kind of session I have (clearly, I only use .bashrc).
>
> There are important differences when you want to run a program at  
> login, but you do not want to run it every time you start up a shell.  
> But simply for setting environment variables, this ought to work.
>
> Perhaps this helps? Or perhaps this is something deeper than the  
> simple issue outlined above.
>
> /Kasper

<snip>

Thanks for the reply Kasper.

Two brief notes, so as not to consume further time and bandwidth here:

1. Dirk has just suggested that I edit /etc/ld.so.conf (then run
ldconfig). Placing the requisite path into that file works and I'm back
to using init 5. RODBC works now.  Thanks Dirk!

2. As mentioned this all had worked just recently. My preference is to
set up my user profile specifically with any customizations, since this
is a single user laptop system. Thus, root's login is clean, when
required for other purposes.  The default FC4 ~/.bash_profile does call
~/.bashrc, which in turn calls /etc/bashrc. So, presuming that the chain
is intact, this should all work, which it had been.

I'm still confused as to the nature of the underlying change, but
perhaps that will be resolved with a clean install soon.

I do sincerely appreciate everyone's time and feedback on this. Sorry
for consuming bandwidth here, for what honestly appeared to be an R
specific issue.

Best regards,

Marc

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel