linking to package directories broken in R >= 2.10 beta

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

linking to package directories broken in R >= 2.10 beta

Thomas Petzoldt
Dear R developers,

some of our packages come with additional programming examples in a
directory called "/examples" which is created from "/inst/examples".

This directory is linked from the docs (e.g. in inst/doc/index.html):

<dl>
<dt><a href="../examples/">examples</a>:
<dd>Source code of examples
</dl>

Given, that we have a package "foo" this is resolved to:

file:///C:/Programme/R/R-2.9.2/library/foo/examples/

with R <= 2.9.2. With R 2.10 beta (R-beta_2009-10-16_r50118.tar.gz) and
R-devel (svn rev. 50118) we get:

http://127.0.0.1:26383/library/foo/examples/

This is fine, but in contrast to older versions (<= 2.9.2) no automatic
index is created for the linked directory, so we now get:

"URL /library/foo/examples/ was not found"

bu linking to *individual files* (e.g. examples/example.R) works as
expected. We can, of course, add manually maintained index files but I
would much prefer if a default index would be created for the directory
if no index.html is found.

I very much enjoy the new help system and would be even more happy if
that issue could be fixed.

Thomas Petzoldt


PS: A minimal reproducible example (foo_1.0.tar.gz) can be provided by
mail if required.

--
Thomas Petzoldt
Technische Universitaet Dresden
Institut fuer Hydrobiologie        [hidden email]
01062 Dresden                      http://tu-dresden.de/hydrobiologie/
GERMANY

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: linking to package directories broken in R >= 2.10 beta

Duncan Murdoch
Thomas Petzoldt wrote:

> Dear R developers,
>
> some of our packages come with additional programming examples in a
> directory called "/examples" which is created from "/inst/examples".
>
> This directory is linked from the docs (e.g. in inst/doc/index.html):
>
> <dl>
> <dt><a href="../examples/">examples</a>:
> <dd>Source code of examples
> </dl>
>
> Given, that we have a package "foo" this is resolved to:
>
> file:///C:/Programme/R/R-2.9.2/library/foo/examples/
>
> with R <= 2.9.2. With R 2.10 beta (R-beta_2009-10-16_r50118.tar.gz) and
> R-devel (svn rev. 50118) we get:
>
> http://127.0.0.1:26383/library/foo/examples/
>
> This is fine, but in contrast to older versions (<= 2.9.2) no automatic
> index is created for the linked directory, so we now get:
>
> "URL /library/foo/examples/ was not found"
>
> bu linking to *individual files* (e.g. examples/example.R) works as
> expected. We can, of course, add manually maintained index files but I
> would much prefer if a default index would be created for the directory
> if no index.html is found.
>  

By "index" in R <= 2.9.2, you mean the default directory listing
produced by the web server, rather than something produced by R, right?  
The R server does that now if the directory is named "doc", but not for
an arbitrary path.  We are concerned about security:  any user on your
system who can guess your port number can access your help system, so we
want to be sure that such users can't access private files.

Duncan Murdoch

> I very much enjoy the new help system and would be even more happy if
> that issue could be fixed.
>
> Thomas Petzoldt
>
>
> PS: A minimal reproducible example (foo_1.0.tar.gz) can be provided by
> mail if required.
>
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: linking to package directories broken in R >= 2.10 beta

Thomas Petzoldt
Duncan Murdoch wrote:
> Thomas Petzoldt wrote:

[...]

>> This is fine, but in contrast to older versions (<= 2.9.2) no
>> automatic index is created for the linked directory, so we now get:
>>
>>
>> "URL /library/foo/examples/ was not found"
>>
>> but linking to *individual files* (e.g. examples/example.R) works as
>> expected. We can, of course, add manually maintained index files
>> but I would much prefer if a default index would be created for the
>> directory if no index.html is found.
>>
>
> By "index" in R <= 2.9.2, you mean the default directory listing
> produced by the web server, rather than something produced by R,
> right?

Yes, I mean the default directory listing produced by (most) web servers.

> The R server does that now if the directory is named "doc", but not
> for an arbitrary path. We are concerned about security: any user on
> your system who can guess your port number can access your help
> system, so we want to be sure that such users can't access private
> files.


Hmm, I see and have some tendency to understand that this may be an
issue for certain multi-user systems. Looking into the svn log (and
compiling R) it appears that the remaining possibilities where also
regarded as security issue and are now locked down too.

Well, I'm not yet completely convinced that this was a good idea.

1) It does not completely solve security issues; what is so different
between the library/foo/doc and library/foo/examples ???

2) The change will introduce additional work for package authors
that used internal links within their packages. I can, of course,
reorganize everything below doc, e.g. /library/foo/doc/examples ... but
this means that these things are even more hidden.

3) However, according to the changed R-Exts, it was obviously decided
that this was necessary, so *I* will do the required reorganization.

I hope that other package authors accept this change of the rules too.

Nevertheless, thank you very much for the new help system.

Thomas P.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: linking to package directories is NOT broken in R >= 2.10 beta

Prof Brian Ripley
In reply to this post by Thomas Petzoldt
When you linked to ../examples/ R was not involved, and what you are
seeing is what your browser did with a file:// url.  Most browsers
will support a wide range of file types, and list directories: but
that is not something that was ever (AFAICS) documented to work.

The 'issue' is your expectations when creating your own
inst/doc/index.html.  The only relative links that are supported by
the help system are to R package help topics and files, to
documentation under R.home("doc") and a limited set of files in a
package's 'doc' directory to support its use for vignettes, including
the ability to list 'doc' itself (if requested in a particular way).

If links to files under <pkg>/example worked, it was a bug. Because of
security concerns over traffic snooping, what you can see through the
dynamic help system is intentionally very limited.  In fact I suspect
they worked for you only because

(i) you installed into .Library
(ii) you had a file for which text/plain worked (and that is because
files that might be in a vignette directory have been checked)..
(iii) you fell into a code branch marked '# should not get here' in
pre-2.10.0 (but absent in R-devel).

The good news is that if you refer to files under the installed 'doc'
directory this should work -- subdirectory listings work now in
R-devel and will probably be ported to 2.10.0 before release.


On Sat, 17 Oct 2009, Thomas Petzoldt wrote:

> Dear R developers,
>
> some of our packages come with additional programming examples in a directory
> called "/examples" which is created from "/inst/examples".
>
> This directory is linked from the docs (e.g. in inst/doc/index.html):
>
> <dl>
> <dt><a href="../examples/">examples</a>:
> <dd>Source code of examples
> </dl>
>
> Given, that we have a package "foo" this is resolved to:
>
> file:///C:/Programme/R/R-2.9.2/library/foo/examples/
>
> with R <= 2.9.2. With R 2.10 beta (R-beta_2009-10-16_r50118.tar.gz) and
> R-devel (svn rev. 50118) we get:
>
> http://127.0.0.1:26383/library/foo/examples/
>
> This is fine, but in contrast to older versions (<= 2.9.2) no automatic index
> is created for the linked directory, so we now get:
>
> "URL /library/foo/examples/ was not found"
>
> bu linking to *individual files* (e.g. examples/example.R) works as expected.
> We can, of course, add manually maintained index files but I would much
> prefer if a default index would be created for the directory if no index.html
> is found.
>
> I very much enjoy the new help system and would be even more happy if that
> issue could be fixed.
>
> Thomas Petzoldt
>
>
> PS: A minimal reproducible example (foo_1.0.tar.gz) can be provided by mail
> if required.
>
> --
> Thomas Petzoldt
> Technische Universitaet Dresden
> Institut fuer Hydrobiologie        [hidden email]
> 01062 Dresden                      http://tu-dresden.de/hydrobiologie/
> GERMANY
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

--
Brian D. Ripley,                  [hidden email]
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: linking to package directories broken in R >= 2.10 beta

Duncan Murdoch
In reply to this post by Thomas Petzoldt
On 17/10/2009 1:57 PM, Thomas Petzoldt wrote:

> Duncan Murdoch wrote:
>> Thomas Petzoldt wrote:
>
> [...]
>
>>> This is fine, but in contrast to older versions (<= 2.9.2) no
>>> automatic index is created for the linked directory, so we now get:
>>>
>>>
>>> "URL /library/foo/examples/ was not found"
>>>
>>> but linking to *individual files* (e.g. examples/example.R) works as
>>> expected. We can, of course, add manually maintained index files
>>> but I would much prefer if a default index would be created for the
>>> directory if no index.html is found.
>>>
>> By "index" in R <= 2.9.2, you mean the default directory listing
>> produced by the web server, rather than something produced by R,
>> right?
>
> Yes, I mean the default directory listing produced by (most) web servers.
>
>> The R server does that now if the directory is named "doc", but not
>> for an arbitrary path. We are concerned about security: any user on
>> your system who can guess your port number can access your help
>> system, so we want to be sure that such users can't access private
>> files.
>
>
> Hmm, I see and have some tendency to understand that this may be an
> issue for certain multi-user systems. Looking into the svn log (and
> compiling R) it appears that the remaining possibilities where also
> regarded as security issue and are now locked down too.
>
> Well, I'm not yet completely convinced that this was a good idea.
>
> 1) It does not completely solve security issues; what is so different
> between the library/foo/doc and library/foo/examples ???

The doc directory is known to be visible.  It might surprise someone if
arbitrary directories were visible, and readable by any user.

> 2) The change will introduce additional work for package authors
> that used internal links within their packages. I can, of course,
> reorganize everything below doc, e.g. /library/foo/doc/examples ... but
> this means that these things are even more hidden.

Why would someone know to look in .../examples?  Just update whatever
hint you gave them to look there, and tell them to look in
.../doc/examples instead.  I don't think it's likely that most people
would discover either directory without a hint somewhere.  If they were
looking for examples, they'd look in the documented places, the Examples
section of man pages, or in the vignettes.

> 3) However, according to the changed R-Exts, it was obviously decided
> that this was necessary, so *I* will do the required reorganization.

I think it was not so much a decision that this was necessary, as that
it was prudent.

Duncan Murdoch

>
> I hope that other package authors accept this change of the rules too.
>
> Nevertheless, thank you very much for the new help system.
>
> Thomas P.
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: linking to package directories broken in R >= 2.10 beta

Thomas Petzoldt
Duncan Murdoch wrote:

[...]

> The doc directory is known to be visible.  It might surprise someone if
> arbitrary directories were visible, and readable by any user.
>
>> 2) The change will introduce additional work for package authors
>> that used internal links within their packages. I can, of course,
>> reorganize everything below doc, e.g. /library/foo/doc/examples ... but
>> this means that these things are even more hidden.
>
> Why would someone know to look in .../examples?  Just update whatever
> hint you gave them to look there, and tell them to look in
> .../doc/examples instead.  I don't think it's likely that most people
> would discover either directory without a hint somewhere.  If they were
> looking for examples, they'd look in the documented places, the Examples
> section of man pages, or in the vignettes.
>
>> 3) However, according to the changed R-Exts, it was obviously decided
>> that this was necessary, so *I* will do the required reorganization.
>
> I think it was not so much a decision that this was necessary, as that
> it was prudent.
>
> Duncan Murdoch

[...]

ok, I will agree, but let me add one final thought: What is with the
/demo directory?

Thomas P.

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: linking to package directories broken in R >= 2.10 beta

Duncan Murdoch
On 18/10/2009 4:45 PM, Thomas Petzoldt wrote:

> Duncan Murdoch wrote:
>
> [...]
>
>> The doc directory is known to be visible.  It might surprise someone if
>> arbitrary directories were visible, and readable by any user.
>>
>>> 2) The change will introduce additional work for package authors
>>> that used internal links within their packages. I can, of course,
>>> reorganize everything below doc, e.g. /library/foo/doc/examples ... but
>>> this means that these things are even more hidden.
>> Why would someone know to look in .../examples?  Just update whatever
>> hint you gave them to look there, and tell them to look in
>> .../doc/examples instead.  I don't think it's likely that most people
>> would discover either directory without a hint somewhere.  If they were
>> looking for examples, they'd look in the documented places, the Examples
>> section of man pages, or in the vignettes.
>>
>>> 3) However, according to the changed R-Exts, it was obviously decided
>>> that this was necessary, so *I* will do the required reorganization.
>> I think it was not so much a decision that this was necessary, as that
>> it was prudent.
>>
>> Duncan Murdoch
>
> [...]
>
> ok, I will agree, but let me add one final thought: What is with the
> /demo directory?

It's not displayed by the help system, it is displayed by the demo()
function.  There wasn't a way to get to it through the help system in
previous versions, was there?  (If you were running a browser you could
browse files on the local file system, and you could construct a \url{}
in a man page, but I don't think there were any links to the demo
directory generated by R, were there?)

Duncan Murdoch

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: linking to package directories is NOT broken in R >= 2.10 beta

Thomas Petzoldt
In reply to this post by Prof Brian Ripley
Prof Brian Ripley wrote:

> When you linked to ../examples/ R was not involved, and what you are
> seeing is what your browser did with a file:// url.  Most browsers will
> support a wide range of file types, and list directories: but that is
> not something that was ever (AFAICS) documented to work.
>
> The 'issue' is your expectations when creating your own
> inst/doc/index.html.  The only relative links that are supported by the
> help system are to R package help topics and files, to documentation
> under R.home("doc") and a limited set of files in a package's 'doc'
> directory to support its use for vignettes, including the ability to
> list 'doc' itself (if requested in a particular way).
>
> If links to files under <pkg>/example worked, it was a bug. Because of
> security concerns over traffic snooping, what you can see through the
> dynamic help system is intentionally very limited.  In fact I suspect
> they worked for you only because
>
> (i) you installed into .Library
> (ii) you had a file for which text/plain worked (and that is because
> files that might be in a vignette directory have been checked)..
> (iii) you fell into a code branch marked '# should not get here' in
> pre-2.10.0 (but absent in R-devel).
>
> The good news is that if you refer to files under the installed 'doc'
> directory this should work -- subdirectory listings work now in R-devel
> and will probably be ported to 2.10.0 before release.
>

Many thanks for clarification and the good news, i.e. for allowing html
links to /doc (and also to DESCRIPTION).

Let me add one additional suggestion: Yes, I know that there are certain
related functions available (with different semantics), but what about
allowing html links to "/demo" and to some other special files like NEWS
and LICENSE (as found in MASS) or THANKS (like in Hmisc)?


Thanks for consideration.

Thomas Petzoldt

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel