issue with print()ing multibyte characters on R 4.0.4

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

issue with print()ing multibyte characters on R 4.0.4

Hiroaki Yutani
Hi all,

I saw several people on Japanese locale claim that, on R 4.0.4,
print() doesn't display
Japanese characters correctly. This seems to happen only on Windows
and on macOS (I
usually use Linux and I don't see this problem).

For example, in the result below, "鬼" and "外" are displayed in
"\uXXXX" format. What's
curious here is that "は" is displayed as it is, by the way.

> "鬼は外"
[1] "\u9b3cは\u5916"

But, if I use such functions as message() or cat(), the string is
displayed as it is.

> message("鬼は外")
鬼は外

Considering the fact that it seems only Windows and macOS are
affected, I suspect this
is somehow related to this change described in the release note,
(though I have no idea
what change this is):

    The internal table for iswprint (used on Windows, macOS and AIX) has been
    updated to include many recent Unicode characters.
    (https://cran.r-project.org/doc/manuals/r-release/NEWS.html)

Before I'm going to file this issue on Bugzilla, I'd like to confirm
if this is not the intended
change, and, if this is actually intended, I want to discuss how to
improve this behaviour.

Best,
Hiroaki Yutani

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: issue with print()ing multibyte characters on R 4.0.4

Tomas Kalibera
Thanks for the report, I can reproduce this on Windows, we'll investigate.

Best
Tomas

On 2/17/21 5:58 AM, Hiroaki Yutani wrote:

> Hi all,
>
> I saw several people on Japanese locale claim that, on R 4.0.4,
> print() doesn't display
> Japanese characters correctly. This seems to happen only on Windows
> and on macOS (I
> usually use Linux and I don't see this problem).
>
> For example, in the result below, "鬼" and "外" are displayed in
> "\uXXXX" format. What's
> curious here is that "は" is displayed as it is, by the way.
>
>> "鬼は外"
> [1] "\u9b3cは\u5916"
>
> But, if I use such functions as message() or cat(), the string is
> displayed as it is.
>
>> message("鬼は外")
> 鬼は外
>
> Considering the fact that it seems only Windows and macOS are
> affected, I suspect this
> is somehow related to this change described in the release note,
> (though I have no idea
> what change this is):
>
>      The internal table for iswprint (used on Windows, macOS and AIX) has been
>      updated to include many recent Unicode characters.
>      (https://cran.r-project.org/doc/manuals/r-release/NEWS.html)
>
> Before I'm going to file this issue on Bugzilla, I'd like to confirm
> if this is not the intended
> change, and, if this is actually intended, I want to discuss how to
> improve this behaviour.
>
> Best,
> Hiroaki Yutani
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: issue with print()ing multibyte characters on R 4.0.4

Prof Brian Ripley
In reply to this post by Hiroaki Yutani
On 17/02/2021 04:58, Hiroaki Yutani wrote:

> Hi all,
>
> I saw several people on Japanese locale claim that, on R 4.0.4,
> print() doesn't display
> Japanese characters correctly. This seems to happen only on Windows
> and on macOS (I
> usually use Linux and I don't see this problem).
>
> For example, in the result below, "鬼" and "外" are displayed in
> "\uXXXX" format. What's
> curious here is that "は" is displayed as it is, by the way.
>
>> "鬼は外"
> [1] "\u9b3cは\u5916"
>
> But, if I use such functions as message() or cat(), the string is
> displayed as it is.
>
>> message("鬼は外")
> 鬼は外

that does not escape non-printable characters, so as expected.

>
> Considering the fact that it seems only Windows and macOS are
> affected, I suspect this
> is somehow related to this change described in the release note,
> (though I have no idea
> what change this is):
>
>      The internal table for iswprint (used on Windows, macOS and AIX) has been
>      updated to include many recent Unicode characters.
>      (https://cran.r-project.org/doc/manuals/r-release/NEWS.html)
>
> Before I'm going to file this issue on Bugzilla, I'd like to confirm
> if this is not the intended
> change, and, if this is actually intended, I want to discuss how to
> improve this behaviour.

I am sorry: this was not intended but it was no one reported in the run
up to 4.0.4.  It seems to be working in R-devel so I suggest you check
that or go back to 4.0.3.

It looks like a line in the iswprint table got deleted in the merge from
R-devel.  I will try to set up some automated checks to see if I can
find any other problems, but that will take a few days.

--
Brian D. Ripley,                  [hidden email]
Emeritus Professor of Applied Statistics, University of Oxford

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: issue with print()ing multibyte characters on R 4.0.4

Hiroaki Yutani
Thanks for confirming and investigating.

> but it was no one reported in the run up to 4.0.4.

Yes, it was unfortunate that no one had reported it to the right place
before the release...

2021年2月17日(水) 19:20 Prof Brian Ripley <[hidden email]>:

>
> On 17/02/2021 04:58, Hiroaki Yutani wrote:
> > Hi all,
> >
> > I saw several people on Japanese locale claim that, on R 4.0.4,
> > print() doesn't display
> > Japanese characters correctly. This seems to happen only on Windows
> > and on macOS (I
> > usually use Linux and I don't see this problem).
> >
> > For example, in the result below, "鬼" and "外" are displayed in
> > "\uXXXX" format. What's
> > curious here is that "は" is displayed as it is, by the way.
> >
> >> "鬼は外"
> > [1] "\u9b3cは\u5916"
> >
> > But, if I use such functions as message() or cat(), the string is
> > displayed as it is.
> >
> >> message("鬼は外")
> > 鬼は外
>
> that does not escape non-printable characters, so as expected.
> >
> > Considering the fact that it seems only Windows and macOS are
> > affected, I suspect this
> > is somehow related to this change described in the release note,
> > (though I have no idea
> > what change this is):
> >
> >      The internal table for iswprint (used on Windows, macOS and AIX) has been
> >      updated to include many recent Unicode characters.
> >      (https://cran.r-project.org/doc/manuals/r-release/NEWS.html)
> >
> > Before I'm going to file this issue on Bugzilla, I'd like to confirm
> > if this is not the intended
> > change, and, if this is actually intended, I want to discuss how to
> > improve this behaviour.
>
> I am sorry: this was not intended but it was no one reported in the run
> up to 4.0.4.  It seems to be working in R-devel so I suggest you check
> that or go back to 4.0.3.
>
> It looks like a line in the iswprint table got deleted in the merge from
> R-devel.  I will try to set up some automated checks to see if I can
> find any other problems, but that will take a few days.
>
> --
> Brian D. Ripley,                  [hidden email]
> Emeritus Professor of Applied Statistics, University of Oxford

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: issue with print()ing multibyte characters on R 4.0.4

Hiroaki Yutani
I filed an issue for this on R's Bugzilla as well, in case this makes
it easier to track (This is my first time to submit a bug report,
please excuse me if I'm failing to follow the appropriate steps to do
this).

https://bugs.r-project.org/bugzilla/show_bug.cgi?id=18059

2021年2月17日(水) 22:47 Hiroaki Yutani <[hidden email]>:

>
> Thanks for confirming and investigating.
>
> > but it was no one reported in the run up to 4.0.4.
>
> Yes, it was unfortunate that no one had reported it to the right place
> before the release...
>
> 2021年2月17日(水) 19:20 Prof Brian Ripley <[hidden email]>:
>
> >
> > On 17/02/2021 04:58, Hiroaki Yutani wrote:
> > > Hi all,
> > >
> > > I saw several people on Japanese locale claim that, on R 4.0.4,
> > > print() doesn't display
> > > Japanese characters correctly. This seems to happen only on Windows
> > > and on macOS (I
> > > usually use Linux and I don't see this problem).
> > >
> > > For example, in the result below, "鬼" and "外" are displayed in
> > > "\uXXXX" format. What's
> > > curious here is that "は" is displayed as it is, by the way.
> > >
> > >> "鬼は外"
> > > [1] "\u9b3cは\u5916"
> > >
> > > But, if I use such functions as message() or cat(), the string is
> > > displayed as it is.
> > >
> > >> message("鬼は外")
> > > 鬼は外
> >
> > that does not escape non-printable characters, so as expected.
> > >
> > > Considering the fact that it seems only Windows and macOS are
> > > affected, I suspect this
> > > is somehow related to this change described in the release note,
> > > (though I have no idea
> > > what change this is):
> > >
> > >      The internal table for iswprint (used on Windows, macOS and AIX) has been
> > >      updated to include many recent Unicode characters.
> > >      (https://cran.r-project.org/doc/manuals/r-release/NEWS.html)
> > >
> > > Before I'm going to file this issue on Bugzilla, I'd like to confirm
> > > if this is not the intended
> > > change, and, if this is actually intended, I want to discuss how to
> > > improve this behaviour.
> >
> > I am sorry: this was not intended but it was no one reported in the run
> > up to 4.0.4.  It seems to be working in R-devel so I suggest you check
> > that or go back to 4.0.3.
> >
> > It looks like a line in the iswprint table got deleted in the merge from
> > R-devel.  I will try to set up some automated checks to see if I can
> > find any other problems, but that will take a few days.
> >
> > --
> > Brian D. Ripley,                  [hidden email]
> > Emeritus Professor of Applied Statistics, University of Oxford

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel