Bug: Issues on Windows with SFN disabled

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Bug: Issues on Windows with SFN disabled

Zach Bjornson
Hello,

R appears to assume that Windows drives have short file names (SFN, 8.3)
enabled; for example, that "C:/Program Files/..." is addressable as
"C:/Progra~1/...". Newer versions of Windows have SFN disabled on non-OS
drives, however.

This means that if you install R on a non-OS drive, you
- can't start R.exe from the command line.
- consequently, anything that attempts to spawn a new R process also fails.
This includes a lot of the commands from the popular devtools package. More
discussion and background: https://github.com/hadley/devtools/issues/1514

I don't have access to bugzilla to file this there.

Thanks and best,
Zach

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug: Issues on Windows with SFN disabled

Tomas Kalibera
Hi Zach,

thanks for the report, I can reproduce the problem and confirm it is a
bug in R and will be fixed.

Hopefully it only impacts few users now. The workaround is to create the
short name for the directory where R is installed, using "fsutil file
setshortname" (for all elements of the path that contain space in their
name). One can revert this by setting the shortname to an empty string
(""). At least for the latter one may need to boot in safe mode.

Best
Tomas


On 09/17/2017 08:23 PM, Zach Bjornson wrote:

> Hello,
>
> R appears to assume that Windows drives have short file names (SFN, 8.3)
> enabled; for example, that "C:/Program Files/..." is addressable as
> "C:/Progra~1/...". Newer versions of Windows have SFN disabled on non-OS
> drives, however.
>
> This means that if you install R on a non-OS drive, you
> - can't start R.exe from the command line.
> - consequently, anything that attempts to spawn a new R process also fails.
> This includes a lot of the commands from the popular devtools package. More
> discussion and background: https://github.com/hadley/devtools/issues/1514
>
> I don't have access to bugzilla to file this there.
>
> Thanks and best,
> Zach
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug: Issues on Windows with SFN disabled

Tomas Kalibera

This has now been mostly fixed in R-devel. What remains to be resolved
is that some packages with custom make files cannot be installed from
source (when R is installed into a directory with space in its name and
short file names are not available)

Tomas



On 10/17/2017 10:37 AM, Tomas Kalibera wrote:

> Hi Zach,
>
> thanks for the report, I can reproduce the problem and confirm it is a
> bug in R and will be fixed.
>
> Hopefully it only impacts few users now. The workaround is to create
> the short name for the directory where R is installed, using "fsutil
> file setshortname" (for all elements of the path that contain space in
> their name). One can revert this by setting the shortname to an empty
> string (""). At least for the latter one may need to boot in safe mode.
>
> Best
> Tomas
>
>
> On 09/17/2017 08:23 PM, Zach Bjornson wrote:
>> Hello,
>>
>> R appears to assume that Windows drives have short file names (SFN, 8.3)
>> enabled; for example, that "C:/Program Files/..." is addressable as
>> "C:/Progra~1/...". Newer versions of Windows have SFN disabled on non-OS
>> drives, however.
>>
>> This means that if you install R on a non-OS drive, you
>> - can't start R.exe from the command line.
>> - consequently, anything that attempts to spawn a new R process also
>> fails.
>> This includes a lot of the commands from the popular devtools
>> package. More
>> discussion and background:
>> https://github.com/hadley/devtools/issues/1514
>>
>> I don't have access to bugzilla to file this there.
>>
>> Thanks and best,
>> Zach
>>
>>     [[alternative HTML version deleted]]
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug: Issues on Windows with SFN disabled

Tomas Kalibera

An update on this. Writing R Extensions does not recommend to have a
space character in R_HOME. This means that on Windows one either should
have SFN enabled (which is still the common case), or install into a
directory that does not have a space in its name (so specifically not
into "Program Files"). This recommendation unfortunately needs to stay
for now.

WRE recommends that Makefiles are written to be robust against space
characters inside R_HOME. All path names passed from a Makefile to the
shell should be quoted at least if they include R_HOME. Make "include"
directives should not be used on path names that are derived from
R_HOME, but one should instead use the "-f" option multiple times when
recursively invoking make. Maintainers of packages that use "include"
with R_HOME have been notified. Unfortunately, the number of packages
that do not quote pathnames with R_HOME in Makefiles is rather large, so
fixing will take some time.

Currently, R-devel should build fine on Windows with R_HOME including
space, including all base and recommended packages, and tests for these
packages should pass even though this is not regularly tested. If you
find a case when this does not work, please submit a bug report.

Thanks
Tomas


On 10/20/2017 04:29 PM, Tomas Kalibera wrote:

>
> This has now been mostly fixed in R-devel. What remains to be resolved
> is that some packages with custom make files cannot be installed from
> source (when R is installed into a directory with space in its name
> and short file names are not available)
>
> Tomas
>
>
>
> On 10/17/2017 10:37 AM, Tomas Kalibera wrote:
>> Hi Zach,
>>
>> thanks for the report, I can reproduce the problem and confirm it is
>> a bug in R and will be fixed.
>>
>> Hopefully it only impacts few users now. The workaround is to create
>> the short name for the directory where R is installed, using "fsutil
>> file setshortname" (for all elements of the path that contain space
>> in their name). One can revert this by setting the shortname to an
>> empty string (""). At least for the latter one may need to boot in
>> safe mode.
>>
>> Best
>> Tomas
>>
>>
>> On 09/17/2017 08:23 PM, Zach Bjornson wrote:
>>> Hello,
>>>
>>> R appears to assume that Windows drives have short file names (SFN,
>>> 8.3)
>>> enabled; for example, that "C:/Program Files/..." is addressable as
>>> "C:/Progra~1/...". Newer versions of Windows have SFN disabled on
>>> non-OS
>>> drives, however.
>>>
>>> This means that if you install R on a non-OS drive, you
>>> - can't start R.exe from the command line.
>>> - consequently, anything that attempts to spawn a new R process also
>>> fails.
>>> This includes a lot of the commands from the popular devtools
>>> package. More
>>> discussion and background:
>>> https://github.com/hadley/devtools/issues/1514
>>>
>>> I don't have access to bugzilla to file this there.
>>>
>>> Thanks and best,
>>> Zach
>>>
>>>     [[alternative HTML version deleted]]
>>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug: Issues on Windows with SFN disabled

Dirk Eddelbuettel

On 7 December 2017 at 17:56, Tomas Kalibera wrote:
|
| An update on this. Writing R Extensions does not recommend to have a
| space character in R_HOME. This means that on Windows one either should
| have SFN enabled (which is still the common case), or install into a
| directory that does not have a space in its name (so specifically not
| into "Program Files"). This recommendation unfortunately needs to stay
| for now.
|
| WRE recommends that Makefiles are written to be robust against space
| characters inside R_HOME. All path names passed from a Makefile to the
| shell should be quoted at least if they include R_HOME. Make "include"
| directives should not be used on path names that are derived from
| R_HOME, but one should instead use the "-f" option multiple times when
| recursively invoking make. Maintainers of packages that use "include"
| with R_HOME have been notified. Unfortunately, the number of packages
| that do not quote pathnames with R_HOME in Makefiles is rather large, so
| fixing will take some time.
|
| Currently, R-devel should build fine on Windows with R_HOME including
| space, including all base and recommended packages, and tests for these
| packages should pass even though this is not regularly tested. If you
| find a case when this does not work, please submit a bug report.

Why does the Windows installer default to using a directory with spaces?

Related (but moderately more advanced), why does R still install "everything"
under one (versioned) directory so that uninformed users on upgrade "miss"
all previously installed packages?

Why not (with space for exposition here, imagine s/ // everywhere below)

    $SOMEROOTDIR / R /
                      R-a.b.c/          # before
                      R-a.b.d/          # d > c, here
                      site-library/     # with .libPaths having this preset?

I don't really care as I manage to work mostly / entirely on another OS, but
I just don't understand why we do not put two and two together. But I am
likely unaware of some salient issues.

In any event, I appreciate the thankless work of those taking care of Windoze
(ie Tomas, Jeroen, Duncan (now ex-officio), Brian, ...)

Dirk

--
http://dirk.eddelbuettel.com | @eddelbuettel | [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug: Issues on Windows with SFN disabled

Tomas Kalibera
On 12/07/2017 06:15 PM, Dirk Eddelbuettel wrote:

> On 7 December 2017 at 17:56, Tomas Kalibera wrote:
> |
> | An update on this. Writing R Extensions does not recommend to have a
> | space character in R_HOME. This means that on Windows one either should
> | have SFN enabled (which is still the common case), or install into a
> | directory that does not have a space in its name (so specifically not
> | into "Program Files"). This recommendation unfortunately needs to stay
> | for now.
> |
> | WRE recommends that Makefiles are written to be robust against space
> | characters inside R_HOME. All path names passed from a Makefile to the
> | shell should be quoted at least if they include R_HOME. Make "include"
> | directives should not be used on path names that are derived from
> | R_HOME, but one should instead use the "-f" option multiple times when
> | recursively invoking make. Maintainers of packages that use "include"
> | with R_HOME have been notified. Unfortunately, the number of packages
> | that do not quote pathnames with R_HOME in Makefiles is rather large, so
> | fixing will take some time.
> |
> | Currently, R-devel should build fine on Windows with R_HOME including
> | space, including all base and recommended packages, and tests for these
> | packages should pass even though this is not regularly tested. If you
> | find a case when this does not work, please submit a bug report.
>
> Why does the Windows installer default to using a directory with spaces?
It's a convention on Windows and I guess there may be problems with
permissions on other directories. My hope is we can make R work reliably
without SFN just in time before SFN become disabled by default, after
all, quoting pathnames in Makefiles (or shell scripts for that matter)
is a good practice anyway and avoiding "include" is not a big problem as
very few packages are affected.

But thanks for opening this and I am happy for insights from any Windows
experts on the issue. I would not want to violate the convention for all
users when just few of them have SFN disabled, and as I hope this will
be fixed on R/packages side, but maybe the installer could at least
detect the problem (when "Program Files" or another specified target
directory did not have a short name). Or perhaps also suggest a
different default. Certainly R could print a warning when it starts.

Tomas

> Related (but moderately more advanced), why does R still install "everything"
> under one (versioned) directory so that uninformed users on upgrade "miss"
> all previously installed packages?
>
> Why not (with space for exposition here, imagine s/ // everywhere below)
>
>      $SOMEROOTDIR / R /
>                        R-a.b.c/          # before
>                        R-a.b.d/          # d > c, here
>                        site-library/     # with .libPaths having this preset?
>
> I don't really care as I manage to work mostly / entirely on another OS, but
> I just don't understand why we do not put two and two together. But I am
> likely unaware of some salient issues.

>
> In any event, I appreciate the thankless work of those taking care of Windoze
> (ie Tomas, Jeroen, Duncan (now ex-officio), Brian, ...)
>
> Dirk
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug: Issues on Windows with SFN disabled

Kevin Ushey
For what it's worth, the Windows installers for other programming
language runtimes often install outside of Program Files, so at least
there is 'prior art' to motivate having R install directly into the
root of the home drive:

    - ActiveState Perl installs directly C:/Perl;
    - Python installs (when installing for all users) into C:/Python$VERSION;
    - The Ruby installers at https://rubyinstaller.org/ default to the
root home drive.

So I (as Dirk said earlier) would also be in favor of having R install
directly to the root of the home drive, with e.g. C:/R/R-x.y.z being
the default install location.

Best,
Kevin


On Fri, Dec 8, 2017 at 4:14 AM, Tomas Kalibera <[hidden email]> wrote:

>
> On 12/07/2017 06:15 PM, Dirk Eddelbuettel wrote:
>>
>> On 7 December 2017 at 17:56, Tomas Kalibera wrote:
>> |
>> | An update on this. Writing R Extensions does not recommend to have a
>> | space character in R_HOME. This means that on Windows one either should
>> | have SFN enabled (which is still the common case), or install into a
>> | directory that does not have a space in its name (so specifically not
>> | into "Program Files"). This recommendation unfortunately needs to stay
>> | for now.
>> |
>> | WRE recommends that Makefiles are written to be robust against space
>> | characters inside R_HOME. All path names passed from a Makefile to the
>> | shell should be quoted at least if they include R_HOME. Make "include"
>> | directives should not be used on path names that are derived from
>> | R_HOME, but one should instead use the "-f" option multiple times when
>> | recursively invoking make. Maintainers of packages that use "include"
>> | with R_HOME have been notified. Unfortunately, the number of packages
>> | that do not quote pathnames with R_HOME in Makefiles is rather large, so
>> | fixing will take some time.
>> |
>> | Currently, R-devel should build fine on Windows with R_HOME including
>> | space, including all base and recommended packages, and tests for these
>> | packages should pass even though this is not regularly tested. If you
>> | find a case when this does not work, please submit a bug report.
>>
>> Why does the Windows installer default to using a directory with spaces?
>
> It's a convention on Windows and I guess there may be problems with permissions on other directories. My hope is we can make R work reliably without SFN just in time before SFN become disabled by default, after all, quoting pathnames in Makefiles (or shell scripts for that matter) is a good practice anyway and avoiding "include" is not a big problem as very few packages are affected.
>
> But thanks for opening this and I am happy for insights from any Windows experts on the issue. I would not want to violate the convention for all users when just few of them have SFN disabled, and as I hope this will be fixed on R/packages side, but maybe the installer could at least detect the problem (when "Program Files" or another specified target directory did not have a short name). Or perhaps also suggest a different default. Certainly R could print a warning when it starts.
>
> Tomas
>>
>> Related (but moderately more advanced), why does R still install "everything"
>> under one (versioned) directory so that uninformed users on upgrade "miss"
>> all previously installed packages?
>>
>> Why not (with space for exposition here, imagine s/ // everywhere below)
>>
>>      $SOMEROOTDIR / R /
>>                        R-a.b.c/          # before
>>                        R-a.b.d/          # d > c, here
>>                        site-library/     # with .libPaths having this preset?
>>
>> I don't really care as I manage to work mostly / entirely on another OS, but
>> I just don't understand why we do not put two and two together. But I am
>> likely unaware of some salient issues.
>
>
>>
>> In any event, I appreciate the thankless work of those taking care of Windoze
>> (ie Tomas, Jeroen, Duncan (now ex-officio), Brian, ...)
>>
>> Dirk
>>
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug: Issues on Windows with SFN disabled

Dirk Eddelbuettel

On 8 December 2017 at 08:52, Kevin Ushey wrote:
| For what it's worth, the Windows installers for other programming
| language runtimes often install outside of Program Files, so at least
| there is 'prior art' to motivate having R install directly into the
| root of the home drive:
|
|     - ActiveState Perl installs directly C:/Perl;
|     - Python installs (when installing for all users) into C:/Python$VERSION;
|     - The Ruby installers at https://rubyinstaller.org/ default to the
| root home drive.
|
| So I (as Dirk said earlier) would also be in favor of having R install
| directly to the root of the home drive, with e.g. C:/R/R-x.y.z being
| the default install location.

Yes! And also keep user-installed packages in C:/R/site-library/ by default.

Dirk


PS Thanks to an anonymous benefactor who corrected my inaccurate phrasing.

--
http://dirk.eddelbuettel.com | @eddelbuettel | [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug: Issues on Windows with SFN disabled

Zach Bjornson
In reply to this post by Kevin Ushey
Thanks for fixing this!

For what it's worth, the Windows installers for other programming
language runtimes often install outside of Program Files, so at least
there is 'prior art' to motivate having R install directly into the
root of the home drive:


This is actually a pretty big peeve of mine. "Program Files" is where
programs are *supposed* to reside on Windows.

>> Related (but moderately more advanced), why does R still install
"everything"
>> under one (versioned) directory so that uninformed users on upgrade
"miss"
>> all previously installed packages?


Will this work for native (C/C++) packages? I thought they need to have
matching ABIs with the R executable.

Zach

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug: Issues on Windows with SFN disabled

Zach Bjornson
>
> This is actually a pretty big peeve of mine. "Program Files" is where
> programs are *supposed* to reside on Windows.
>

PS -- Microsoft's reasoning is explained in "Why do program files go into
the Program Files directory?"

https://blogs.msdn.microsoft.com/oldnewthing/20120307-00/?p=8153

There are a few useful comments below it as well, including some folks who
got bit by SFN. The mentioned logo/cert requirements are listed in

https://msdn.microsoft.com/en-us/library/windows/desktop/aa372825(v=vs.85).aspx

-Z

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Bug: Issues on Windows with SFN disabled

Paul McQuesten
In reply to this post by Zach Bjornson
Does Windows allow creating a symlink to "C:/Program Files"?
If so, R could install into the conventional Windows location, but use the
symlink for all its access, yes?

Windows 10 command = mklink

On Fri, Dec 8, 2017 at 11:14 AM, Zach Bjornson <[hidden email]> wrote:

> Thanks for fixing this!
>
> For what it's worth, the Windows installers for other programming
> language runtimes often install outside of Program Files, so at least
> there is 'prior art' to motivate having R install directly into the
> root of the home drive:
>
>
> This is actually a pretty big peeve of mine. "Program Files" is where
> programs are *supposed* to reside on Windows.
>
> >> Related (but moderately more advanced), why does R still install
> "everything"
> >> under one (versioned) directory so that uninformed users on upgrade
> "miss"
> >> all previously installed packages?
>
>
> Will this work for native (C/C++) packages? I thought they need to have
> matching ABIs with the R executable.
>
> Zach
>
>         [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel