Dialect for shell scripts

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Dialect for shell scripts

Rodrigo Tobar
Dear all,

During a recent package submission, we were highlighted that some lines
in our configure script didn't follow the correct syntax. The lines
looked like this:

x=$(($y/10))

We were indicated at the time that this is because the statement does
not use Bourne shell syntax, which is absolutely true, and also that the
manual warns about this, which is true again. So far everything is clear.

However, what confuses me is that even when the manual says that "you
can include an executable (Bourne) shell script configure in your
package" [1], the associated footnote says something slightly different:
"The script should only assume a POSIX-compliant /bin/sh" [2]. The
footnote goes even further, and links to the POSIX specification of the
Shell Command Language [3] (as published by The Open Group), which
explicitly includes arithmetic expressions like the one above in its
syntax [4].

My question then is: what exact dialect should be considered? Given that
the statement above does not work in the Bourne shell, I conclude that
the Bourne shell is not POSIX-compliant. That in turn would make the
manual ambiguous as to the precise dialect that should be used by our
configure scripts, and either the shells used by R should be changed to
be POSIX-compliants, or the manual edited to be more precise regarding .

Many thanks.

Rodrigo

[1]
https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Configure-and-cleanup
[2] https://cran.r-project.org/doc/manuals/r-release/R-exts.html#FOOT25
[3] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
[4]
http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Dialect for shell scripts

Iñaki Úcar
For what it's worth, Autoconf does not assume that arithmetic
expansion will be available. Instead, it emits the following shell
code:

if ( eval 'test $(( 1 + 1 )) = 2' ) 2>/dev/null; then
  eval 'func_arith ()
  {
    func_arith_result=$(( $* ))
  }'
else
  func_arith ()
  {
    func_arith_result=`expr "$@"`
  }
fi

2017-12-17 23:55 GMT+01:00 Rodrigo Tobar <[hidden email]>:

> Dear all,
>
> During a recent package submission, we were highlighted that some lines in
> our configure script didn't follow the correct syntax. The lines looked like
> this:
>
> x=$(($y/10))
>
> We were indicated at the time that this is because the statement does not
> use Bourne shell syntax, which is absolutely true, and also that the manual
> warns about this, which is true again. So far everything is clear.
>
> However, what confuses me is that even when the manual says that "you can
> include an executable (Bourne) shell script configure in your package" [1],
> the associated footnote says something slightly different: "The script
> should only assume a POSIX-compliant /bin/sh" [2]. The footnote goes even
> further, and links to the POSIX specification of the Shell Command Language
> [3] (as published by The Open Group), which explicitly includes arithmetic
> expressions like the one above in its syntax [4].
>
> My question then is: what exact dialect should be considered? Given that the
> statement above does not work in the Bourne shell, I conclude that the
> Bourne shell is not POSIX-compliant. That in turn would make the manual
> ambiguous as to the precise dialect that should be used by our configure
> scripts, and either the shells used by R should be changed to be
> POSIX-compliants, or the manual edited to be more precise regarding .
>
> Many thanks.
>
> Rodrigo
>
> [1]
> https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Configure-and-cleanup
> [2] https://cran.r-project.org/doc/manuals/r-release/R-exts.html#FOOT25
> [3] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
> [4]
> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel



--
Iñaki Úcar
http://www.enchufa2.es
@Enchufa2

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Dialect for shell scripts

Kurt Hornik-5
>>>>> Iñaki Úcar writes:

Same from here: in addition to what the standards say, it always pays to
be defensive and check "Portable Shell Programming" in the Autoconf
manual.  Among other things, this says

'$((EXPRESSION))'
     Arithmetic expansion is not portable as some shells (most notably
     Solaris 10 '/bin/sh') don't support it.

motivating the code shown below.  Perhaps simplest to always use expr.

-k


> For what it's worth, Autoconf does not assume that arithmetic
> expansion will be available. Instead, it emits the following shell
> code:

> if ( eval 'test $(( 1 + 1 )) = 2' ) 2>/dev/null; then
>   eval 'func_arith ()
>   {
>     func_arith_result=$(( $* ))
>   }'
> else
>   func_arith ()
>   {
>     func_arith_result=`expr "$@"`
>   }
> fi

> 2017-12-17 23:55 GMT+01:00 Rodrigo Tobar <[hidden email]>:
>> Dear all,
>>
>> During a recent package submission, we were highlighted that some lines in
>> our configure script didn't follow the correct syntax. The lines looked like
>> this:
>>
>> x=$(($y/10))
>>
>> We were indicated at the time that this is because the statement does not
>> use Bourne shell syntax, which is absolutely true, and also that the manual
>> warns about this, which is true again. So far everything is clear.
>>
>> However, what confuses me is that even when the manual says that "you can
>> include an executable (Bourne) shell script configure in your package" [1],
>> the associated footnote says something slightly different: "The script
>> should only assume a POSIX-compliant /bin/sh" [2]. The footnote goes even
>> further, and links to the POSIX specification of the Shell Command Language
>> [3] (as published by The Open Group), which explicitly includes arithmetic
>> expressions like the one above in its syntax [4].
>>
>> My question then is: what exact dialect should be considered? Given that the
>> statement above does not work in the Bourne shell, I conclude that the
>> Bourne shell is not POSIX-compliant. That in turn would make the manual
>> ambiguous as to the precise dialect that should be used by our configure
>> scripts, and either the shells used by R should be changed to be
>> POSIX-compliants, or the manual edited to be more precise regarding .
>>
>> Many thanks.
>>
>> Rodrigo
>>
>> [1]
>> https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Configure-and-cleanup
>> [2] https://cran.r-project.org/doc/manuals/r-release/R-exts.html#FOOT25
>> [3] http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
>> [4]
>> http://pubs.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html#tag_18_06_04
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel



> --
> Iñaki Úcar
> http://www.enchufa2.es
> @Enchufa2

> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Dialect for shell scripts

Paul McQuesten
I do not have a dog in this fight, but I have to ask:
    How much person time is worthwhile to invest in supporting Solaris 10?

It has been closed-source (Post-Oracle)
<https://en.wikipedia.org/wiki/Solaris_(operating_system)#Post-Oracle_closed_source_(Solaris_10_after_March_2010,_and_Solaris_11_(2011_and_later))>
since
March 2010.

On Mon, Dec 18, 2017 at 1:23 PM, Kurt Hornik <[hidden email]> wrote:

> >>>>> Iñaki Úcar writes:
>
> Same from here: in addition to what the standards say, it always pays to
> be defensive and check "Portable Shell Programming" in the Autoconf
> manual.  Among other things, this says
>
> '$((EXPRESSION))'
>      Arithmetic expansion is not portable as some shells (most notably
>      Solaris 10 '/bin/sh') don't support it.
>
> motivating the code shown below.  Perhaps simplest to always use expr.
>
> -k
>
>
> > For what it's worth, Autoconf does not assume that arithmetic
> > expansion will be available. Instead, it emits the following shell
> > code:
>
> > if ( eval 'test $(( 1 + 1 )) = 2' ) 2>/dev/null; then
> >   eval 'func_arith ()
> >   {
> >     func_arith_result=$(( $* ))
> >   }'
> > else
> >   func_arith ()
> >   {
> >     func_arith_result=`expr "$@"`
> >   }
> > fi
>
> > 2017-12-17 23:55 GMT+01:00 Rodrigo Tobar <[hidden email]>:
> >> Dear all,
> >>
> >> During a recent package submission, we were highlighted that some lines
> in
> >> our configure script didn't follow the correct syntax. The lines looked
> like
> >> this:
> >>
> >> x=$(($y/10))
> >>
> >> We were indicated at the time that this is because the statement does
> not
> >> use Bourne shell syntax, which is absolutely true, and also that the
> manual
> >> warns about this, which is true again. So far everything is clear.
> >>
> >> However, what confuses me is that even when the manual says that "you
> can
> >> include an executable (Bourne) shell script configure in your package"
> [1],
> >> the associated footnote says something slightly different: "The script
> >> should only assume a POSIX-compliant /bin/sh" [2]. The footnote goes
> even
> >> further, and links to the POSIX specification of the Shell Command
> Language
> >> [3] (as published by The Open Group), which explicitly includes
> arithmetic
> >> expressions like the one above in its syntax [4].
> >>
> >> My question then is: what exact dialect should be considered? Given
> that the
> >> statement above does not work in the Bourne shell, I conclude that the
> >> Bourne shell is not POSIX-compliant. That in turn would make the manual
> >> ambiguous as to the precise dialect that should be used by our configure
> >> scripts, and either the shells used by R should be changed to be
> >> POSIX-compliants, or the manual edited to be more precise regarding .
> >>
> >> Many thanks.
> >>
> >> Rodrigo
> >>
> >> [1]
> >> https://cran.r-project.org/doc/manuals/r-release/R-exts.
> html#Configure-and-cleanup
> >> [2] https://cran.r-project.org/doc/manuals/r-release/R-exts.html#FOOT25
> >> [3] http://pubs.opengroup.org/onlinepubs/9699919799/
> utilities/V3_chap02.html
> >> [4]
> >> http://pubs.opengroup.org/onlinepubs/9699919799/
> utilities/V3_chap02.html#tag_18_06_04
> >>
> >> ______________________________________________
> >> [hidden email] mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
>
>
>
> > --
> > Iñaki Úcar
> > http://www.enchufa2.es
> > @Enchufa2
>
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Dialect for shell scripts

Peter Dalgaard-2
Solaris is pretty much dead at this point (closed source or not), but it is not the only oddball OS around.

The need to support a wide palette of (Unix) OS variations has been rapidly declining in later years. Fifteen years ago every supercomputer seemed to have its own set of OS quirks and we wanted to be able to support the cutting edge. Now things seem to converge on a few variations on the big three: Windows, MacOS, Linux, possibly on some parallel/cloud infrastructure.

However, it is probably worth maintaining the conservative defensive policies at least for a while yet.

(I suspect that the text in WRE is actually older than the current POSIX standard, which is what causes confusion as to what constitutes a Bourne shell. It may be worth editing it to say that we are in fact more restrictive than current POSIX.)

-pd  

> On 18 Dec 2017, at 23:36 , Paul McQuesten <[hidden email]> wrote:
>
> I do not have a dog in this fight, but I have to ask:
>    How much person time is worthwhile to invest in supporting Solaris 10?
>
> It has been closed-source (Post-Oracle)
> <https://en.wikipedia.org/wiki/Solaris_(operating_system)#Post-Oracle_closed_source_(Solaris_10_after_March_2010,_and_Solaris_11_(2011_and_later))>
> since
> March 2010.
>
> On Mon, Dec 18, 2017 at 1:23 PM, Kurt Hornik <[hidden email]> wrote:
>
>>>>>>> Iñaki Úcar writes:
>>
>> Same from here: in addition to what the standards say, it always pays to
>> be defensive and check "Portable Shell Programming" in the Autoconf
>> manual.  Among other things, this says
>>
>> '$((EXPRESSION))'
>>     Arithmetic expansion is not portable as some shells (most notably
>>     Solaris 10 '/bin/sh') don't support it.
>>
>> motivating the code shown below.  Perhaps simplest to always use expr.
>>
>> -k
>>
>>
>>> For what it's worth, Autoconf does not assume that arithmetic
>>> expansion will be available. Instead, it emits the following shell
>>> code:
>>
>>> if ( eval 'test $(( 1 + 1 )) = 2' ) 2>/dev/null; then
>>>  eval 'func_arith ()
>>>  {
>>>    func_arith_result=$(( $* ))
>>>  }'
>>> else
>>>  func_arith ()
>>>  {
>>>    func_arith_result=`expr "$@"`
>>>  }
>>> fi
>>
>>> 2017-12-17 23:55 GMT+01:00 Rodrigo Tobar <[hidden email]>:
>>>> Dear all,
>>>>
>>>> During a recent package submission, we were highlighted that some lines
>> in
>>>> our configure script didn't follow the correct syntax. The lines looked
>> like
>>>> this:
>>>>
>>>> x=$(($y/10))
>>>>
>>>> We were indicated at the time that this is because the statement does
>> not
>>>> use Bourne shell syntax, which is absolutely true, and also that the
>> manual
>>>> warns about this, which is true again. So far everything is clear.
>>>>
>>>> However, what confuses me is that even when the manual says that "you
>> can
>>>> include an executable (Bourne) shell script configure in your package"
>> [1],
>>>> the associated footnote says something slightly different: "The script
>>>> should only assume a POSIX-compliant /bin/sh" [2]. The footnote goes
>> even
>>>> further, and links to the POSIX specification of the Shell Command
>> Language
>>>> [3] (as published by The Open Group), which explicitly includes
>> arithmetic
>>>> expressions like the one above in its syntax [4].
>>>>
>>>> My question then is: what exact dialect should be considered? Given
>> that the
>>>> statement above does not work in the Bourne shell, I conclude that the
>>>> Bourne shell is not POSIX-compliant. That in turn would make the manual
>>>> ambiguous as to the precise dialect that should be used by our configure
>>>> scripts, and either the shells used by R should be changed to be
>>>> POSIX-compliants, or the manual edited to be more precise regarding .
>>>>
>>>> Many thanks.
>>>>
>>>> Rodrigo
>>>>
>>>> [1]
>>>> https://cran.r-project.org/doc/manuals/r-release/R-exts.
>> html#Configure-and-cleanup
>>>> [2] https://cran.r-project.org/doc/manuals/r-release/R-exts.html#FOOT25
>>>> [3] http://pubs.opengroup.org/onlinepubs/9699919799/
>> utilities/V3_chap02.html
>>>> [4]
>>>> http://pubs.opengroup.org/onlinepubs/9699919799/
>> utilities/V3_chap02.html#tag_18_06_04
>>>>
>>>> ______________________________________________
>>>> [hidden email] mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>>
>>
>>> --
>>> Iñaki Úcar
>>> http://www.enchufa2.es
>>> @Enchufa2
>>
>>> ______________________________________________
>>> [hidden email] mailing list
>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>
>
> [[alternative HTML version deleted]]
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

--
Peter Dalgaard, Professor,
Center for Statistics, Copenhagen Business School
Solbjerg Plads 3, 2000 Frederiksberg, Denmark
Phone: (+45)38153501
Office: A 4.23
Email: [hidden email]  Priv: [hidden email]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Dialect for shell scripts

Rodrigo Tobar
Hi all,

Thanks to all for your responses. As pointed out by a few of you, using
expr is indeed safer, and is what we ended up using in our script to get
it accepted in CRAN.

On 19/12/17 07:26, peter dalgaard wrote:
> (I suspect that the text in WRE is actually older than the current POSIX standard, which is what causes confusion as to what constitutes a Bourne shell. It may be worth editing it to say that we are in fact more restrictive than current POSIX.)

This was exactly my point, and I think it would be great to have this
clarified in the manual to avoid any unsuspected surprises during
package submission. Since R expectations are naturally aligned with
autoconf's it would be worth noting that, and maybe also pointing to the
resource pointed at by Iñaki.

I think the discussion about supported OSs is also an interesting one,
but is greater than just about this particular subject.

Regards,

Rodrigo

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel