Unicode characters in ISO8859-15 locale

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

Unicode characters in ISO8859-15 locale

Jeroen Ooms-2
The hunspell package uses the code below to replace curly quotes (aka
fancyquotes) with a regular ascii quotes that are needed for check
spelling:

    chartr("\u2019", "'", input)

As of last week this stopped working on CRAN in the Linux server that
runs in ISO8859-15 locale. From the error message, it seems that R no
longer parses the escaped unique string, which gets turned into
"<U+2019>".

Is this expected?

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Unicode characters in ISO8859-15 locale

Prof Brian Ripley
On 17/12/2020 12:28, Jeroen Ooms wrote:
> The hunspell package uses the code below to replace curly quotes (aka

officially these are directional (or right/left) quotes.

> fancyquotes) with a regular ascii quotes that are needed for check
> spelling:
>
>      chartr("\u2019", "'", input)
>
> As of last week this stopped working on CRAN in the Linux server that
> runs in ISO8859-15 locale. From the error message, it seems that R no
> longer parses the escaped unique string, which gets turned into
> "<U+2019>".

You need to distinguish the string and what gets printed.  I see (for
the CRAN log from a 2-day-old R)

  Error in chartr("<U+2019>", "'", as.character(add_words)) :

which is what I would expect to be printed in that locale:

> x <- "\u2019"
> Encoding(x)
[1] "UTF-8"
> x
[1] "<U+2019>"

> Is this expected?

Changes are expected, as this is an area being worked on (to try to
remove some system-dependent behaviour).  However, I cannot reproduce
this easily:

> chartr("\u2019", "'", "abc\u2019")
[1] "abc'"

As I say, work in progress.

--
Brian D. Ripley,                  [hidden email]
Emeritus Professor of Applied Statistics, University of Oxford

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel