Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

classic Classic list List threaded Threaded
38 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Gavin Simpson-3
Dear list

On Fedora 31 the pango library has recently updated to version >= 1.44
and in doing so has switched to using the HarfBuzz library (from
FreeType) and dropped Adobe Type 1 font support. This causes problems
with plotmath as all bar one of the glyphs doesn't render (see
attached PNG image if it makes it through the list filters - if not I
have shared a copy via my google drive:
https://drive.google.com/file/d/1llFqKHD7LFKzQbVuq6sibY1UizRn7xxS/view?usp=sharing
)

I'm not the only person who has come across this, e.g.
https://stackoverflow.com/q/60656445/429846 and the resulting reported
bug on the RedHat Bugzilla:
https://bugzilla.redhat.com/show_bug.cgi?id=1815128

Beyond switching to  `type = 'Xlib'`, has anyone worked around this
issue on a Fedora 31 or later system?

Thanks in advance

Gavin

--
Gavin Simpson, PhD

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel

plotmath-issue.png (29K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Iñaki Ucar
On Wed, 25 Mar 2020 at 01:14, Gavin Simpson <[hidden email]> wrote:

>
> Dear list
>
> On Fedora 31 the pango library has recently updated to version >= 1.44
> and in doing so has switched to using the HarfBuzz library (from
> FreeType) and dropped Adobe Type 1 font support. This causes problems
> with plotmath as all bar one of the glyphs doesn't render (see
> attached PNG image if it makes it through the list filters - if not I
> have shared a copy via my google drive:
> https://drive.google.com/file/d/1llFqKHD7LFKzQbVuq6sibY1UizRn7xxS/view?usp=sharing
> )
>
> I'm not the only person who has come across this, e.g.
> https://stackoverflow.com/q/60656445/429846 and the resulting reported
> bug on the RedHat Bugzilla:
> https://bugzilla.redhat.com/show_bug.cgi?id=1815128
>
> Beyond switching to  `type = 'Xlib'`, has anyone worked around this
> issue on a Fedora 31 or later system?

Adding [hidden email] to CC. A workaround is to avoid using PS fonts
for symbols. If you run the following, you'll see

$ fc-match Symbol
StandardSymbolsPS.t1: "Standard Symbols PS" "Regular"

So let's change this. Install a TTF symbol font, such as Symbola:

$ sudo dnf install gdouros-symbola-fonts

Then add the following to /etc/fonts/local.conf (system-wide) or
~/.fonts.conf (just for your user):

<fontconfig>
<match target="pattern">
 <test name="family"><string>Symbol</string></test>
 <edit name="family" mode="prepend" binding="same">
   <string>Symbola</string>
 </edit>
</match>
</fontconfig>

Now you should see this:

$ fc-match Symbol
Symbola.ttf: "Symbola" "Regular"

and symbols should render correctly.

Iñaki

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

R devel mailing list
Le mercredi 25 mars 2020 à 11:28 +0100, Iñaki Ucar a écrit :
> On Wed, 25 Mar 2020 at 01:14, Gavin Simpson <[hidden email]>
> wrote:

Hi,

> Adding [hidden email] to CC. A workaround is to avoid using PS
> fonts for symbols.

PS fonts are dead mid-term everywhere, and already forbidden in new
Fedora font packages (because we are somewhat leading edge, but not as
much as people think)
https://docs.fedoraproject.org/en-US//packaging-guidelines/FontsPolicy/#_font_file_formats

PS font users need to switch to OpenType fonts or work with their
prefered font upstream to convert in modern well supported formats
(font format wars have endend last millenium, even before the browser
wars ended, it’s long past time to deprecate the losers).

That’s normal IT format obsolescence.


That being said, that’s not what is happening here.

R brought this all on itself by hardcoding a Windows-only “Symbol” font
family name in its default conf. Linux systems are UTF-8 by default for
~20 years now, they don’t need the forcing of magic font families to
handle symbols not present in the 8-bit legacy Windows encodings.

The actual effect of this conf is not the selection of font files with
special and unusual symbols. It is to priorize fonts that match the
"Symbol" magic name. And those fonts are few and crumbling on Linux
systems, because no one has needed to bother with them since Linux
switched to UTF-8 last millenium.

Just stop using “Symbol” in R and things will work a lot better.
Alternatively, prepare to maintain the “Symbol” aliasing stack in
fontconfig (and fight with wine for it), because *no* *one* *else*
*cares* about this legacy Windows-specific stuff.


Fontconfig upstream already told this to R users in its own issue
tracker.


Regards,

--
Nicolas Mailhot

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Gavin Simpson-3
In reply to this post by Iñaki Ucar
Thanks Iñaki, that worked a treat.

Gavin

On Wed, 25 Mar 2020 at 04:28, Iñaki Ucar <[hidden email]> wrote:

>
> On Wed, 25 Mar 2020 at 01:14, Gavin Simpson <[hidden email]> wrote:
> >
> > Dear list
> >
> > On Fedora 31 the pango library has recently updated to version >= 1.44
> > and in doing so has switched to using the HarfBuzz library (from
> > FreeType) and dropped Adobe Type 1 font support. This causes problems
> > with plotmath as all bar one of the glyphs doesn't render (see
> > attached PNG image if it makes it through the list filters - if not I
> > have shared a copy via my google drive:
> > https://drive.google.com/file/d/1llFqKHD7LFKzQbVuq6sibY1UizRn7xxS/view?usp=sharing
> > )
> >
> > I'm not the only person who has come across this, e.g.
> > https://stackoverflow.com/q/60656445/429846 and the resulting reported
> > bug on the RedHat Bugzilla:
> > https://bugzilla.redhat.com/show_bug.cgi?id=1815128
> >
> > Beyond switching to  `type = 'Xlib'`, has anyone worked around this
> > issue on a Fedora 31 or later system?
>
> Adding [hidden email] to CC. A workaround is to avoid using PS fonts
> for symbols. If you run the following, you'll see
>
> $ fc-match Symbol
> StandardSymbolsPS.t1: "Standard Symbols PS" "Regular"
>
> So let's change this. Install a TTF symbol font, such as Symbola:
>
> $ sudo dnf install gdouros-symbola-fonts
>
> Then add the following to /etc/fonts/local.conf (system-wide) or
> ~/.fonts.conf (just for your user):
>
> <fontconfig>
> <match target="pattern">
>  <test name="family"><string>Symbol</string></test>
>  <edit name="family" mode="prepend" binding="same">
>    <string>Symbola</string>
>  </edit>
> </match>
> </fontconfig>
>
> Now you should see this:
>
> $ fc-match Symbol
> Symbola.ttf: "Symbola" "Regular"
>
> and symbols should render correctly.
>
> Iñaki



--
Gavin Simpson, PhD

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Iñaki Ucar
In reply to this post by R devel mailing list
On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
<[hidden email]> wrote:

>
> <snip>
>
> R brought this all on itself by hardcoding a Windows-only “Symbol” font
> family name in its default conf. Linux systems are UTF-8 by default for
> ~20 years now, they don’t need the forcing of magic font families to
> handle symbols not present in the 8-bit legacy Windows encodings.
>
> The actual effect of this conf is not the selection of font files with
> special and unusual symbols. It is to priorize fonts that match the
> "Symbol" magic name. And those fonts are few and crumbling on Linux
> systems, because no one has needed to bother with them since Linux
> switched to UTF-8 last millenium.
>
> Just stop using “Symbol” in R and things will work a lot better.
> Alternatively, prepare to maintain the “Symbol” aliasing stack in
> fontconfig (and fight with wine for it), because *no* *one* *else*
> *cares* about this legacy Windows-specific stuff.

So, in the light of Nicolas' input (thanks!), I think that font
selection should be fixed upstream in R. I'd be happy to put all this
together in R's bugzilla, but I don't have an account. Could someone
please invite me?

Iñaki

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Paul Murrell-2
Hi

Thanks for your input on this Iñaki and Nicolas.

I am starting testing an R fix for this problem today.

As suggested, the plan is to allow the R user to specify a font family
other than "symbol" for plotmath output (or, more generally, in R
parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.

Paul


On 27/03/20 11:30 pm, Iñaki Ucar wrote:

> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> <[hidden email]> wrote:
>>
>> <snip>
>>
>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
>> family name in its default conf. Linux systems are UTF-8 by default for
>> ~20 years now, they don’t need the forcing of magic font families to
>> handle symbols not present in the 8-bit legacy Windows encodings.
>>
>> The actual effect of this conf is not the selection of font files with
>> special and unusual symbols. It is to priorize fonts that match the
>> "Symbol" magic name. And those fonts are few and crumbling on Linux
>> systems, because no one has needed to bother with them since Linux
>> switched to UTF-8 last millenium.
>>
>> Just stop using “Symbol” in R and things will work a lot better.
>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
>> fontconfig (and fight with wine for it), because *no* *one* *else*
>> *cares* about this legacy Windows-specific stuff.
>
> So, in the light of Nicolas' input (thanks!), I think that font
> selection should be fixed upstream in R. I'd be happy to put all this
> together in R's bugzilla, but I don't have an account. Could someone
> please invite me?
>
> Iñaki
>
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Iñaki Ucar
Thanks, Paul. I've created a bug report to keep track of this
(https://bugs.r-project.org/bugzilla/show_bug.cgi?id=17748), and taken
the liberty of adding you to CC. We'll need to cherry-pick the fix for
R 3.6.3 in Fedora 31.

Iñaki

On Sun, 29 Mar 2020 at 21:15, Paul Murrell <[hidden email]> wrote:

>
> Hi
>
> Thanks for your input on this Iñaki and Nicolas.
>
> I am starting testing an R fix for this problem today.
>
> As suggested, the plan is to allow the R user to specify a font family
> other than "symbol" for plotmath output (or, more generally, in R
> parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
>
> Paul
>
>
> On 27/03/20 11:30 pm, Iñaki Ucar wrote:
> > On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> > <[hidden email]> wrote:
> >>
> >> <snip>
> >>
> >> R brought this all on itself by hardcoding a Windows-only “Symbol” font
> >> family name in its default conf. Linux systems are UTF-8 by default for
> >> ~20 years now, they don’t need the forcing of magic font families to
> >> handle symbols not present in the 8-bit legacy Windows encodings.
> >>
> >> The actual effect of this conf is not the selection of font files with
> >> special and unusual symbols. It is to priorize fonts that match the
> >> "Symbol" magic name. And those fonts are few and crumbling on Linux
> >> systems, because no one has needed to bother with them since Linux
> >> switched to UTF-8 last millenium.
> >>
> >> Just stop using “Symbol” in R and things will work a lot better.
> >> Alternatively, prepare to maintain the “Symbol” aliasing stack in
> >> fontconfig (and fight with wine for it), because *no* *one* *else*
> >> *cares* about this legacy Windows-specific stuff.
> >
> > So, in the light of Nicolas' input (thanks!), I think that font
> > selection should be fixed upstream in R. I'd be happy to put all this
> > together in R's bugzilla, but I don't have an account. Could someone
> > please invite me?
> >
> > Iñaki
> >
> > ______________________________________________
> > [hidden email] mailing list
> > https://stat.ethz.ch/mailman/listinfo/r-devel
> >



--
Iñaki Úcar

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Paul Murrell-2
In reply to this post by Paul Murrell-2
Hi

I have created an R branch that contains a potential fix ...

https://svn.r-project.org/R/branches/R-symfam/

This allows, for example, ...

cairo_pdf(symbolfamily="OpenSymbol")

... to specify that the OpenSymbol family should be used as the "symbol"
font (e.g., for "plotmath") in R.

This is just a separate branch for now because, while I have tested it
under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
(right now) or Mac (ever) and I do not want to drop a bomb on R-devel at
this stage of the release process for R 4.0.0.

The attached file contains at least an outline of steps required to do a
minimal test if anyone wants to try the fix on Linux.

cc'ing Simon and Jeroen in case they are able to help with checking that
this builds and works on Mac and/or Windows.

NOTEs:
- 'symbolfamily' can only be specified when a graphics device is opened,
and it is then fixed for that device.
- on Windows, for cairo-based devices, the "symbol" font is still
hard-coded as "Standard Symbols L"

Paul

On 30/03/20 8:15 am, Paul Murrell wrote:

> Hi
>
> Thanks for your input on this Iñaki and Nicolas.
>
> I am starting testing an R fix for this problem today.
>
> As suggested, the plan is to allow the R user to specify a font family
> other than "symbol" for plotmath output (or, more generally, in R
> parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
>
> Paul
>
>
> On 27/03/20 11:30 pm, Iñaki Ucar wrote:
>> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
>> <[hidden email]> wrote:
>>>
>>> <snip>
>>>
>>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
>>> family name in its default conf. Linux systems are UTF-8 by default for
>>> ~20 years now, they don’t need the forcing of magic font families to
>>> handle symbols not present in the 8-bit legacy Windows encodings.
>>>
>>> The actual effect of this conf is not the selection of font files with
>>> special and unusual symbols. It is to priorize fonts that match the
>>> "Symbol" magic name. And those fonts are few and crumbling on Linux
>>> systems, because no one has needed to bother with them since Linux
>>> switched to UTF-8 last millenium.
>>>
>>> Just stop using “Symbol” in R and things will work a lot better.
>>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
>>> fontconfig (and fight with wine for it), because *no* *one* *else*
>>> *cares* about this legacy Windows-specific stuff.
>>
>> So, in the light of Nicolas' input (thanks!), I think that font
>> selection should be fixed upstream in R. I'd be happy to put all this
>> together in R's bugzilla, but I don't have an account. Could someone
>> please invite me?
>>
>> Iñaki
>>
>> ______________________________________________
>> [hidden email] mailing list
>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>

--
Dr Paul Murrell
Department of Statistics
The University of Auckland
Private Bag 92019
Auckland
New Zealand
64 9 3737599 x85392
[hidden email]
http://www.stat.auckland.ac.nz/~paul/
______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Gabriel Becker-2
I do my devel/patch work on Mac so I can take a shot at testing your branch
in the next couple days.

~G

On Sun, Mar 29, 2020 at 7:24 PM Paul Murrell <[hidden email]>
wrote:

> Hi
>
> I have created an R branch that contains a potential fix ...
>
> https://svn.r-project.org/R/branches/R-symfam/
>
> This allows, for example, ...
>
> cairo_pdf(symbolfamily="OpenSymbol")
>
> ... to specify that the OpenSymbol family should be used as the "symbol"
> font (e.g., for "plotmath") in R.
>
> This is just a separate branch for now because, while I have tested it
> under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
> (right now) or Mac (ever) and I do not want to drop a bomb on R-devel at
> this stage of the release process for R 4.0.0.
>
> The attached file contains at least an outline of steps required to do a
> minimal test if anyone wants to try the fix on Linux.
>
> cc'ing Simon and Jeroen in case they are able to help with checking that
> this builds and works on Mac and/or Windows.
>
> NOTEs:
> - 'symbolfamily' can only be specified when a graphics device is opened,
> and it is then fixed for that device.
> - on Windows, for cairo-based devices, the "symbol" font is still
> hard-coded as "Standard Symbols L"
>
> Paul
>
> On 30/03/20 8:15 am, Paul Murrell wrote:
> > Hi
> >
> > Thanks for your input on this Iñaki and Nicolas.
> >
> > I am starting testing an R fix for this problem today.
> >
> > As suggested, the plan is to allow the R user to specify a font family
> > other than "symbol" for plotmath output (or, more generally, in R
> > parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
> >
> > Paul
> >
> >
> > On 27/03/20 11:30 pm, Iñaki Ucar wrote:
> >> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> >> <[hidden email]> wrote:
> >>>
> >>> <snip>
> >>>
> >>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
> >>> family name in its default conf. Linux systems are UTF-8 by default for
> >>> ~20 years now, they don’t need the forcing of magic font families to
> >>> handle symbols not present in the 8-bit legacy Windows encodings.
> >>>
> >>> The actual effect of this conf is not the selection of font files with
> >>> special and unusual symbols. It is to priorize fonts that match the
> >>> "Symbol" magic name. And those fonts are few and crumbling on Linux
> >>> systems, because no one has needed to bother with them since Linux
> >>> switched to UTF-8 last millenium.
> >>>
> >>> Just stop using “Symbol” in R and things will work a lot better.
> >>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
> >>> fontconfig (and fight with wine for it), because *no* *one* *else*
> >>> *cares* about this legacy Windows-specific stuff.
> >>
> >> So, in the light of Nicolas' input (thanks!), I think that font
> >> selection should be fixed upstream in R. I'd be happy to put all this
> >> together in R's bugzilla, but I don't have an account. Could someone
> >> please invite me?
> >>
> >> Iñaki
> >>
> >> ______________________________________________
> >> [hidden email] mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>
>
> --
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> [hidden email]
> http://www.stat.auckland.ac.nz/~paul/
> ______________________________________________
> [hidden email] mailing list
> https://stat.ethz.ch/mailman/listinfo/r-devel
>

        [[alternative HTML version deleted]]

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Iñaki Ucar
In reply to this post by Paul Murrell-2
On Mon, 30 Mar 2020 at 04:24, Paul Murrell <[hidden email]> wrote:

>
> Hi
>
> I have created an R branch that contains a potential fix ...
>
> https://svn.r-project.org/R/branches/R-symfam/
>
> This allows, for example, ...
>
> cairo_pdf(symbolfamily="OpenSymbol")
>
> ... to specify that the OpenSymbol family should be used as the "symbol"
> font (e.g., for "plotmath") in R.

Will this be a default on Linux? Or are you planning any mechanism
(env variable, option...) to make it the default? Because, otherwise,
as pango is updated across distributions, R graphics will be "broken"
by default unless the user explicitly calls the graphics device in
that way to set that option, which I would say is uncommon.

Iñaki

> This is just a separate branch for now because, while I have tested it
> under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
> (right now) or Mac (ever) and I do not want to drop a bomb on R-devel at
> this stage of the release process for R 4.0.0.
>
> The attached file contains at least an outline of steps required to do a
> minimal test if anyone wants to try the fix on Linux.
>
> cc'ing Simon and Jeroen in case they are able to help with checking that
> this builds and works on Mac and/or Windows.
>
> NOTEs:
> - 'symbolfamily' can only be specified when a graphics device is opened,
> and it is then fixed for that device.
> - on Windows, for cairo-based devices, the "symbol" font is still
> hard-coded as "Standard Symbols L"
>
>
> Paul
>
> On 30/03/20 8:15 am, Paul Murrell wrote:
> > Hi
> >
> > Thanks for your input on this Iñaki and Nicolas.
> >
> > I am starting testing an R fix for this problem today.
> >
> > As suggested, the plan is to allow the R user to specify a font family
> > other than "symbol" for plotmath output (or, more generally, in R
> > parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
> >
> > Paul
> >
> >
> > On 27/03/20 11:30 pm, Iñaki Ucar wrote:
> >> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> >> <[hidden email]> wrote:
> >>>
> >>> <snip>
> >>>
> >>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
> >>> family name in its default conf. Linux systems are UTF-8 by default for
> >>> ~20 years now, they don’t need the forcing of magic font families to
> >>> handle symbols not present in the 8-bit legacy Windows encodings.
> >>>
> >>> The actual effect of this conf is not the selection of font files with
> >>> special and unusual symbols. It is to priorize fonts that match the
> >>> "Symbol" magic name. And those fonts are few and crumbling on Linux
> >>> systems, because no one has needed to bother with them since Linux
> >>> switched to UTF-8 last millenium.
> >>>
> >>> Just stop using “Symbol” in R and things will work a lot better.
> >>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
> >>> fontconfig (and fight with wine for it), because *no* *one* *else*
> >>> *cares* about this legacy Windows-specific stuff.
> >>
> >> So, in the light of Nicolas' input (thanks!), I think that font
> >> selection should be fixed upstream in R. I'd be happy to put all this
> >> together in R's bugzilla, but I don't have an account. Could someone
> >> please invite me?
> >>
> >> Iñaki
> >>
> >> ______________________________________________
> >> [hidden email] mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>
>
> --
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> [hidden email]
> http://www.stat.auckland.ac.nz/~paul/



--
Iñaki Úcar

On Mon, 30 Mar 2020 at 04:24, Paul Murrell <[hidden email]> wrote:

>
> Hi
>
> I have created an R branch that contains a potential fix ...
>
> https://svn.r-project.org/R/branches/R-symfam/
>
> This allows, for example, ...
>
> cairo_pdf(symbolfamily="OpenSymbol")
>
> ... to specify that the OpenSymbol family should be used as the "symbol"
> font (e.g., for "plotmath") in R.
>
> This is just a separate branch for now because, while I have tested it
> under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
> (right now) or Mac (ever) and I do not want to drop a bomb on R-devel at
> this stage of the release process for R 4.0.0.
>
> The attached file contains at least an outline of steps required to do a
> minimal test if anyone wants to try the fix on Linux.
>
> cc'ing Simon and Jeroen in case they are able to help with checking that
> this builds and works on Mac and/or Windows.
>
> NOTEs:
> - 'symbolfamily' can only be specified when a graphics device is opened,
> and it is then fixed for that device.
> - on Windows, for cairo-based devices, the "symbol" font is still
> hard-coded as "Standard Symbols L"
>
> Paul
>
> On 30/03/20 8:15 am, Paul Murrell wrote:
> > Hi
> >
> > Thanks for your input on this Iñaki and Nicolas.
> >
> > I am starting testing an R fix for this problem today.
> >
> > As suggested, the plan is to allow the R user to specify a font family
> > other than "symbol" for plotmath output (or, more generally, in R
> > parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
> >
> > Paul
> >
> >
> > On 27/03/20 11:30 pm, Iñaki Ucar wrote:
> >> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
> >> <[hidden email]> wrote:
> >>>
> >>> <snip>
> >>>
> >>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
> >>> family name in its default conf. Linux systems are UTF-8 by default for
> >>> ~20 years now, they don’t need the forcing of magic font families to
> >>> handle symbols not present in the 8-bit legacy Windows encodings.
> >>>
> >>> The actual effect of this conf is not the selection of font files with
> >>> special and unusual symbols. It is to priorize fonts that match the
> >>> "Symbol" magic name. And those fonts are few and crumbling on Linux
> >>> systems, because no one has needed to bother with them since Linux
> >>> switched to UTF-8 last millenium.
> >>>
> >>> Just stop using “Symbol” in R and things will work a lot better.
> >>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
> >>> fontconfig (and fight with wine for it), because *no* *one* *else*
> >>> *cares* about this legacy Windows-specific stuff.
> >>
> >> So, in the light of Nicolas' input (thanks!), I think that font
> >> selection should be fixed upstream in R. I'd be happy to put all this
> >> together in R's bugzilla, but I don't have an account. Could someone
> >> please invite me?
> >>
> >> Iñaki
> >>
> >> ______________________________________________
> >> [hidden email] mailing list
> >> https://stat.ethz.ch/mailman/listinfo/r-devel
> >>
>
> --
> Dr Paul Murrell
> Department of Statistics
> The University of Auckland
> Private Bag 92019
> Auckland
> New Zealand
> 64 9 3737599 x85392
> [hidden email]
> http://www.stat.auckland.ac.nz/~paul/



--
Iñaki Úcar

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

R devel mailing list
In reply to this post by Paul Murrell-2
Le lundi 30 mars 2020 à 15:24 +1300, Paul Murrell a écrit :

> Hi
>
> I have created an R branch that contains a potential fix ...
>
> https://svn.r-project.org/R/branches/R-symfam/
>
> This allows, for example, ...
>
> cairo_pdf(symbolfamily="OpenSymbol")
>
> ... to specify that the OpenSymbol family should be used as the
> "symbol" font (e.g., for "plotmath") in R.

Thanks for looking at it!

But, really, there is no such thing as a Symbol font on Linux anymore.
Symbol is pre-unicode thinking. Most modern general-purpose unicode
fonts will include every codepoint Symbol ever shipped, and fontconfig
will fallback gracefully when that’s not the case (unless your
fontconfig integration is broken).

Just use the sans-serif or monospace fontconfig defaults. You don’t
need Symbol, or OpenSymbol, or any special font setup.

Symbol’s codepoint coverage is laughable by 2020’s UTF-8 standards.

Symbol << Normal Unicode font (DejaVu*) << Special math fonts (STIX2)

I you do advanced math stuff, you may need a special math font like
STIX, but that’s lights years more advanced than Symbol, and a general
purpose font like DejaVu has been shipping a MATH block for several
years now

Regards,

--
Nicolas Mailhot

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Paul Murrell-2
In reply to this post by Gabriel Becker-2

Thanks Gabriel.  Sounds like both you and Brian can build the branch on
Mac.  Just need to check that Windows builds before I commit to r-devel.

Paul

On 30/03/20 7:43 pm, Gabriel Becker wrote:

> I do my devel/patch work on Mac so I can take a shot at testing your
> branch in the next couple days.
>
> ~G
>
> On Sun, Mar 29, 2020 at 7:24 PM Paul Murrell <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Hi
>
>     I have created an R branch that contains a potential fix ...
>
>     https://svn.r-project.org/R/branches/R-symfam/
>
>     This allows, for example, ...
>
>     cairo_pdf(symbolfamily="OpenSymbol")
>
>     ... to specify that the OpenSymbol family should be used as the
>     "symbol"
>     font (e.g., for "plotmath") in R.
>
>     This is just a separate branch for now because, while I have tested it
>     under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
>     (right now) or Mac (ever) and I do not want to drop a bomb on
>     R-devel at
>     this stage of the release process for R 4.0.0.
>
>     The attached file contains at least an outline of steps required to
>     do a
>     minimal test if anyone wants to try the fix on Linux.
>
>     cc'ing Simon and Jeroen in case they are able to help with checking
>     that
>     this builds and works on Mac and/or Windows.
>
>     NOTEs:
>     - 'symbolfamily' can only be specified when a graphics device is
>     opened,
>     and it is then fixed for that device.
>     - on Windows, for cairo-based devices, the "symbol" font is still
>     hard-coded as "Standard Symbols L"
>
>     Paul
>
>     On 30/03/20 8:15 am, Paul Murrell wrote:
>      > Hi
>      >
>      > Thanks for your input on this Iñaki and Nicolas.
>      >
>      > I am starting testing an R fix for this problem today.
>      >
>      > As suggested, the plan is to allow the R user to specify a font
>     family
>      > other than "symbol" for plotmath output (or, more generally, in R
>      > parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics
>     device.
>      >
>      > Paul
>      >
>      >
>      > On 27/03/20 11:30 pm, Iñaki Ucar wrote:
>      >> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
>      >> <[hidden email]
>     <mailto:[hidden email]>> wrote:
>      >>>
>      >>> <snip>
>      >>>
>      >>> R brought this all on itself by hardcoding a Windows-only
>     “Symbol” font
>      >>> family name in its default conf. Linux systems are UTF-8 by
>     default for
>      >>> ~20 years now, they don’t need the forcing of magic font
>     families to
>      >>> handle symbols not present in the 8-bit legacy Windows encodings.
>      >>>
>      >>> The actual effect of this conf is not the selection of font
>     files with
>      >>> special and unusual symbols. It is to priorize fonts that match the
>      >>> "Symbol" magic name. And those fonts are few and crumbling on Linux
>      >>> systems, because no one has needed to bother with them since Linux
>      >>> switched to UTF-8 last millenium.
>      >>>
>      >>> Just stop using “Symbol” in R and things will work a lot better.
>      >>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
>      >>> fontconfig (and fight with wine for it), because *no* *one* *else*
>      >>> *cares* about this legacy Windows-specific stuff.
>      >>
>      >> So, in the light of Nicolas' input (thanks!), I think that font
>      >> selection should be fixed upstream in R. I'd be happy to put all
>     this
>      >> together in R's bugzilla, but I don't have an account. Could someone
>      >> please invite me?
>      >>
>      >> Iñaki
>      >>
>      >> ______________________________________________
>      >> [hidden email] <mailto:[hidden email]> mailing list
>      >> https://stat.ethz.ch/mailman/listinfo/r-devel
>      >>
>
>     --
>     Dr Paul Murrell
>     Department of Statistics
>     The University of Auckland
>     Private Bag 92019
>     Auckland
>     New Zealand
>     64 9 3737599 x85392
>     [hidden email] <mailto:[hidden email]>
>     http://www.stat.auckland.ac.nz/~paul/
>     ______________________________________________
>     [hidden email] <mailto:[hidden email]> mailing list
>     https://stat.ethz.ch/mailman/listinfo/r-devel
>

--
Dr Paul Murrell
Department of Statistics
The University of Auckland
Private Bag 92019
Auckland
New Zealand
64 9 3737599 x85392
[hidden email]
http://www.stat.auckland.ac.nz/~paul/

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Paul Murrell-2
In reply to this post by Iñaki Ucar
Hi

On 30/03/20 10:43 pm, Iñaki Ucar wrote:

> On Mon, 30 Mar 2020 at 04:24, Paul Murrell <[hidden email]> wrote:
>>
>> Hi
>>
>> I have created an R branch that contains a potential fix ...
>>
>> https://svn.r-project.org/R/branches/R-symfam/
>>
>> This allows, for example, ...
>>
>> cairo_pdf(symbolfamily="OpenSymbol")
>>
>> ... to specify that the OpenSymbol family should be used as the "symbol"
>> font (e.g., for "plotmath") in R.
>
> Will this be a default on Linux? Or are you planning any mechanism
> (env variable, option...) to make it the default? Because, otherwise,
> as pango is updated across distributions, R graphics will be "broken"
> by default unless the user explicitly calls the graphics device in
> that way to set that option, which I would say is uncommon.

Good question.  Currently, for x11() (and png() etc) the default is
taken from X11.options().  So it is possible to set this default for a
session, or even for an installation via one of the ?Startup mechanisms
(e.g., an R_HOME/etc/Rprofile.site file).

For svg(), cairo_pdf(), and cairo_ps(), the default is hard-coded in the
function arguments, but I *think* they are used less as default graphics
devices.

Another option would be to try to detect Fedora and set the default
X11.options() differently there.  Two problems:  I am not sure there is
a reliable R code chunk for detecting Fedora (sessionInfo()$running?)
let alone Fedora >= 30;   what to set the default to?  (just has to be a
font with a good Unicode coverage that is pretty much guaranteed to be
in a default Fedora install).

Paul

> Iñaki
>
>> This is just a separate branch for now because, while I have tested it
>> under Unbuntu 18.04 and Fedora 31, I cannot even build R for Windows
>> (right now) or Mac (ever) and I do not want to drop a bomb on R-devel at
>> this stage of the release process for R 4.0.0.
>>
>> The attached file contains at least an outline of steps required to do a
>> minimal test if anyone wants to try the fix on Linux.
>>
>> cc'ing Simon and Jeroen in case they are able to help with checking that
>> this builds and works on Mac and/or Windows.
>>
>> NOTEs:
>> - 'symbolfamily' can only be specified when a graphics device is opened,
>> and it is then fixed for that device.
>> - on Windows, for cairo-based devices, the "symbol" font is still
>> hard-coded as "Standard Symbols L"
>>
>>
>> Paul
>>
>> On 30/03/20 8:15 am, Paul Murrell wrote:
>>> Hi
>>>
>>> Thanks for your input on this Iñaki and Nicolas.
>>>
>>> I am starting testing an R fix for this problem today.
>>>
>>> As suggested, the plan is to allow the R user to specify a font family
>>> other than "symbol" for plotmath output (or, more generally, in R
>>> parlance, for 'font=5' or 'fontface=5') on a Cairo-based graphics device.
>>>
>>> Paul
>>>
>>>
>>> On 27/03/20 11:30 pm, Iñaki Ucar wrote:
>>>> On Wed, 25 Mar 2020 at 12:25, Nicolas Mailhot
>>>> <[hidden email]> wrote:
>>>>>
>>>>> <snip>
>>>>>
>>>>> R brought this all on itself by hardcoding a Windows-only “Symbol” font
>>>>> family name in its default conf. Linux systems are UTF-8 by default for
>>>>> ~20 years now, they don’t need the forcing of magic font families to
>>>>> handle symbols not present in the 8-bit legacy Windows encodings.
>>>>>
>>>>> The actual effect of this conf is not the selection of font files with
>>>>> special and unusual symbols. It is to priorize fonts that match the
>>>>> "Symbol" magic name. And those fonts are few and crumbling on Linux
>>>>> systems, because no one has needed to bother with them since Linux
>>>>> switched to UTF-8 last millenium.
>>>>>
>>>>> Just stop using “Symbol” in R and things will work a lot better.
>>>>> Alternatively, prepare to maintain the “Symbol” aliasing stack in
>>>>> fontconfig (and fight with wine for it), because *no* *one* *else*
>>>>> *cares* about this legacy Windows-specific stuff.
>>>>
>>>> So, in the light of Nicolas' input (thanks!), I think that font
>>>> selection should be fixed upstream in R. I'd be happy to put all this
>>>> together in R's bugzilla, but I don't have an account. Could someone
>>>> please invite me?
>>>>
>>>> Iñaki
>>>>
>>>> ______________________________________________
>>>> [hidden email] mailing list
>>>> https://stat.ethz.ch/mailman/listinfo/r-devel
>>>>
>>
>> --
>> Dr Paul Murrell
>> Department of Statistics
>> The University of Auckland
>> Private Bag 92019
>> Auckland
>> New Zealand
>> 64 9 3737599 x85392
>> [hidden email]
>> http://www.stat.auckland.ac.nz/~paul/
>
>
>

--
Dr Paul Murrell
Department of Statistics
The University of Auckland
Private Bag 92019
Auckland
New Zealand
64 9 3737599 x85392
[hidden email]
http://www.stat.auckland.ac.nz/~paul/

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Iñaki Ucar
On Mon, 30 Mar 2020 at 22:41, Paul Murrell <[hidden email]> wrote:

>
> Hi
>
> On 30/03/20 10:43 pm, Iñaki Ucar wrote:
> > On Mon, 30 Mar 2020 at 04:24, Paul Murrell <[hidden email]> wrote:
> >>
> >> Hi
> >>
> >> I have created an R branch that contains a potential fix ...
> >>
> >> https://svn.r-project.org/R/branches/R-symfam/
> >>
> >> This allows, for example, ...
> >>
> >> cairo_pdf(symbolfamily="OpenSymbol")
> >>
> >> ... to specify that the OpenSymbol family should be used as the "symbol"
> >> font (e.g., for "plotmath") in R.
> >
> > Will this be a default on Linux? Or are you planning any mechanism
> > (env variable, option...) to make it the default? Because, otherwise,
> > as pango is updated across distributions, R graphics will be "broken"
> > by default unless the user explicitly calls the graphics device in
> > that way to set that option, which I would say is uncommon.
>
> Good question.  Currently, for x11() (and png() etc) the default is
> taken from X11.options().  So it is possible to set this default for a
> session, or even for an installation via one of the ?Startup mechanisms
> (e.g., an R_HOME/etc/Rprofile.site file).
>
> For svg(), cairo_pdf(), and cairo_ps(), the default is hard-coded in the
> function arguments, but I *think* they are used less as default graphics
> devices.
>
> Another option would be to try to detect Fedora and set the default
> X11.options() differently there.  Two problems:  I am not sure there is
> a reliable R code chunk for detecting Fedora (sessionInfo()$running?)
> let alone Fedora >= 30;   what to set the default to?  (just has to be a
> font with a good Unicode coverage that is pretty much guaranteed to be
> in a default Fedora install).

As per Nicolas' comment (I failed to include him in CC in my last
email, and he's not in this list, sorry for that) any font installed
by default would have good symbol coverage, so there's really no need
to set a different font for symbols. According again to Nicolas (he's
one of the font experts in Fedora), the "sans-serif" or "monospace"
fontconfig defaults would work out of the box, and if a symbol is not
available, fontconfig should fallback gracefully to another font.

So maybe instead of a new "symbolfamily" argument, maybe it's better
to just use the "family" for all characters, including symbols, on
Linux, and fontconfig should take care of everything (if I understood
correctly your explanation, Nicolas; please correct me if I'm wrong).

--
Iñaki Úcar

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Paul Murrell-2
In reply to this post by R devel mailing list
Hi

On 30/03/20 11:12 pm, Nicolas Mailhot wrote:

> Le lundi 30 mars 2020 à 15:24 +1300, Paul Murrell a écrit :
>> Hi
>>
>> I have created an R branch that contains a potential fix ...
>>
>> https://svn.r-project.org/R/branches/R-symfam/
>>
>> This allows, for example, ...
>>
>> cairo_pdf(symbolfamily="OpenSymbol")
>>
>> ... to specify that the OpenSymbol family should be used as the
>> "symbol" font (e.g., for "plotmath") in R.
>
> Thanks for looking at it!
>
> But, really, there is no such thing as a Symbol font on Linux anymore.
> Symbol is pre-unicode thinking. Most modern general-purpose unicode
> fonts will include every codepoint Symbol ever shipped, and fontconfig
> will fallback gracefully when that’s not the case (unless your
> fontconfig integration is broken).

Yep, the "symbol" font is an (outdated) R "plotmath" concept, but one
that would take a fair bit of surgery to remove.  R plotmath converts
certain R expressions (in certain contexts) to code points in the Adobe
Symbol Encoding (ASM), but for cairo-based devices, those are converted
to UTF8 code points.

> Just use the sans-serif or monospace fontconfig defaults. You don’t
> need Symbol, or OpenSymbol, or any special font setup.

Agreed.  I got reasonable coverage from DejaVu Sans and FreeSerif.
There are still a number of ASM code points that are not covered though,
for example, ...

F8EB E6 # LEFT PAREN TOP # parenlefttp (CUS)
F8EC E7 # LEFT PAREN EXTENDER # parenleftex (CUS)
F8ED E8 # LEFT PAREN BOTTOM # parenleftbt (CUS)

Even OpenSymbol is missing a few (though perhaps not very common ones) ...

F8E5 60 # RADICAL EXTENDER # radicalex (CUS)
F8E6 BD # VERTICAL ARROW EXTENDER # arrowvertex (CUS)
F8E7 BE # HORIZONTAL ARROW EXTENDER # arrowhorizex (CUS)
F6DA D2 # REGISTERED SIGN SERIF # registerserif (CUS)
F6D9 D3 # COPYRIGHT SIGN SERIF # copyrightserif (CUS)
F6DB D4 # TRADE MARK SIGN SERIF # trademarkserif (CUS)
F8E8 E2 # REGISTERED SIGN SANS SERIF # registersans (CUS)
F8E9 E3 # COPYRIGHT SIGN SANS SERIF # copyrightsans (CUS)
F8EA E4 # TRADE MARK SIGN SANS SERIF # trademarksans (CUS)

> Symbol’s codepoint coverage is laughable by 2020’s UTF-8 standards.

Right, but there are still code points that are apparently not common in
fonts with a very broad Unicode coverage.

I did find a TrueType font that I could add a Unicode cmap to that gave
complete coverage pretty much all by itself, but it is not distributable.

> Symbol << Normal Unicode font (DejaVu*) << Special math fonts (STIX2)
>
> I you do advanced math stuff, you may need a special math font like
> STIX, but that’s lights years more advanced than Symbol, and a general
> purpose font like DejaVu has been shipping a MATH block for several
> years now

Sure.  One way to look at the new 'symbolfamily' argument is that it
allows us to tell R that the "symbol" font is just a normal font (for
cairo-based graphics devices).

Paul
--
Dr Paul Murrell
Department of Statistics
The University of Auckland
Private Bag 92019
Auckland
New Zealand
64 9 3737599 x85392
[hidden email]
http://www.stat.auckland.ac.nz/~paul/

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

R devel mailing list
Le mardi 31 mars 2020 à 10:14 +1300, Paul Murrell a écrit :

> Hi
>
> On 30/03/20 11:12 pm, Nicolas Mailhot wrote:
> > Le lundi 30 mars 2020 à 15:24 +1300, Paul Murrell a écrit :
> > > Hi
> > >
> > > I have created an R branch that contains a potential fix ...
> > >
> > > https://svn.r-project.org/R/branches/R-symfam/
> > >
> > > This allows, for example, ...
> > >
> > > cairo_pdf(symbolfamily="OpenSymbol")
> > >
> > > ... to specify that the OpenSymbol family should be used as the
> > > "symbol" font (e.g., for "plotmath") in R.
> >
> > Thanks for looking at it!
> >
> > But, really, there is no such thing as a Symbol font on Linux
> > anymore.
> > Symbol is pre-unicode thinking. Most modern general-purpose unicode
> > fonts will include every codepoint Symbol ever shipped, and
> > fontconfig
> > will fallback gracefully when that’s not the case (unless your
> > fontconfig integration is broken).
>
> Yep, the "symbol" font is an (outdated) R "plotmath" concept, but
> one
> that would take a fair bit of surgery to remove.  R plotmath
> converts
> certain R expressions (in certain contexts) to code points in the
> Adobe
> Symbol Encoding (ASM), but for cairo-based devices, those are
> converted
> to UTF8 code points.
>
> > Just use the sans-serif or monospace fontconfig defaults. You don’t
> > need Symbol, or OpenSymbol, or any special font setup.
>
> Agreed.  I got reasonable coverage from DejaVu Sans and FreeSerif.
> There are still a number of ASM code points that are not covered
> though,
> for example, ...
>
> F8EB E6 # LEFT PAREN TOP # parenlefttp (CUS)
> F8EC E7 # LEFT PAREN EXTENDER # parenleftex (CUS)
> F8ED E8 # LEFT PAREN BOTTOM # parenleftbt (CUS)
>
> Even OpenSymbol is missing a few (though perhaps not very common
> ones) ...

All the F8* codepoints are in the private use area. That means you
can’t rely on them existing in standard unicode fonts

You need to use correct Unicode values for things to work:
Ux239… for parenthesis, brackets

https://www.unicode.org/charts/PDF/U2300.pdf

> F8E6    BD      # VERTICAL ARROW EXTENDER       # arrowvertex (CUS)
> F8E7    BE      # HORIZONTAL ARROW EXTENDER     # arrowhorizex (CUS)

and 23AF/23D0 for arrow extensions (though arrow font support seems
messy, probably because it sees little use; it’s a pity R comes so late
to the party, those are just lines, it would have been trivial to get
them into DejaVu before the project gone dormant). GFS NeoHellenic
(Math block) seems complete but it’s not a common font family.

> F6DA    D2      # REGISTERED SIGN SERIF # registerserif (CUS)
> F6D9    D3      # COPYRIGHT SIGN SERIF  # copyrightserif (CUS)
> F6DB    D4      # TRADE MARK SIGN SERIF # trademarkserif (CUS)
> F8E8    E2      # REGISTERED SIGN SANS SERIF    # registersans (CUS)
> F8E9    E3      # COPYRIGHT SIGN SANS SERIF     # copyrightsans (CUS)
> F8EA    E4      # TRADE MARK SIGN SANS SERIF    # trademarksans (CUS)

Those are useless nowadays, just use normal
registered/copyright/trademark codepoints, and a font in the wished
style (serif sans serif, whatever looks nice to you)

Regards

--
Nicolas Mailhot

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Paul Murrell-2
In reply to this post by Iñaki Ucar


On 31/03/20 10:12 am, Iñaki Ucar wrote:

> On Mon, 30 Mar 2020 at 22:41, Paul Murrell <[hidden email]> wrote:
>>
>> Hi
>>
>> On 30/03/20 10:43 pm, Iñaki Ucar wrote:
>>> On Mon, 30 Mar 2020 at 04:24, Paul Murrell <[hidden email]> wrote:
>>>>
>>>> Hi
>>>>
>>>> I have created an R branch that contains a potential fix ...
>>>>
>>>> https://svn.r-project.org/R/branches/R-symfam/
>>>>
>>>> This allows, for example, ...
>>>>
>>>> cairo_pdf(symbolfamily="OpenSymbol")
>>>>
>>>> ... to specify that the OpenSymbol family should be used as the "symbol"
>>>> font (e.g., for "plotmath") in R.
>>>
>>> Will this be a default on Linux? Or are you planning any mechanism
>>> (env variable, option...) to make it the default? Because, otherwise,
>>> as pango is updated across distributions, R graphics will be "broken"
>>> by default unless the user explicitly calls the graphics device in
>>> that way to set that option, which I would say is uncommon.
>>
>> Good question.  Currently, for x11() (and png() etc) the default is
>> taken from X11.options().  So it is possible to set this default for a
>> session, or even for an installation via one of the ?Startup mechanisms
>> (e.g., an R_HOME/etc/Rprofile.site file).
>>
>> For svg(), cairo_pdf(), and cairo_ps(), the default is hard-coded in the
>> function arguments, but I *think* they are used less as default graphics
>> devices.
>>
>> Another option would be to try to detect Fedora and set the default
>> X11.options() differently there.  Two problems:  I am not sure there is
>> a reliable R code chunk for detecting Fedora (sessionInfo()$running?)
>> let alone Fedora >= 30;   what to set the default to?  (just has to be a
>> font with a good Unicode coverage that is pretty much guaranteed to be
>> in a default Fedora install).
>
> As per Nicolas' comment (I failed to include him in CC in my last
> email, and he's not in this list, sorry for that) any font installed
> by default would have good symbol coverage, so there's really no need
> to set a different font for symbols. According again to Nicolas (he's
> one of the font experts in Fedora), the "sans-serif" or "monospace"
> fontconfig defaults would work out of the box, and if a symbol is not
> available, fontconfig should fallback gracefully to another font.
>
> So maybe instead of a new "symbolfamily" argument, maybe it's better
> to just use the "family" for all characters, including symbols, on
> Linux, and fontconfig should take care of everything (if I understood
> correctly your explanation, Nicolas; please correct me if I'm wrong).

I think R will retain the idea of a separate symbol font in at least the
short term because of backward compatibility and cross-platform support
and support for a range of graphics devices.  So this fix is just for
cairo-based devices on Linux at most (probably only Fedora).

So this becomes just a decision about user interface and default settings.

I did consider the option of allowing the existing "family" parameter to
be length-two (with the second one being an optional symbol font
specification), but because of the overlaps of X11/cairo and different
cairo-based device interfaces, this became awkward.  Hence the separate
"symbolfamily" interface.  And in any case, this still means a separate
"symbol" font specification (for the reasons above).

Regarding changing to a default symbolfamily=family on Linux generally
(rather than just on Fedora), I have at least one counter-example (my
Ubuntu 18.04) that shows that this would degrade output significantly.
For one, the symbols are a LOT uglier, plus there are some incorrect
glyphs.  So I think we have to stay with treating Fedora as a special
case for now.

Thanks for your point about just using symbolfamily=family as the Fedora
default.  That seems reasonable (and definitely better than it just
being completely broken!).

That does still leave the problem of how to set the default value for
"symbolfamily" JUST on Fedora.   I am not convinced we can use R code to
detect Fedora >= 30 reliably (but happy to learn otherwise).  Is it a
possibility for the Fedora distribution to include a .Rprofile.site file
that sets the X11.options() ?

Paul
--
Dr Paul Murrell
Department of Statistics
The University of Auckland
Private Bag 92019
Auckland
New Zealand
64 9 3737599 x85392
[hidden email]
http://www.stat.auckland.ac.nz/~paul/

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: [FORGED] Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Paul Murrell-2
In reply to this post by R devel mailing list


On 31/03/20 11:50 am, Nicolas Mailhot wrote:

> Le mardi 31 mars 2020 à 10:14 +1300, Paul Murrell a écrit :
>> Hi
>>
>> On 30/03/20 11:12 pm, Nicolas Mailhot wrote:
>>> Le lundi 30 mars 2020 à 15:24 +1300, Paul Murrell a écrit :
>>>> Hi
>>>>
>>>> I have created an R branch that contains a potential fix ...
>>>>
>>>> https://svn.r-project.org/R/branches/R-symfam/
>>>>
>>>> This allows, for example, ...
>>>>
>>>> cairo_pdf(symbolfamily="OpenSymbol")
>>>>
>>>> ... to specify that the OpenSymbol family should be used as the
>>>> "symbol" font (e.g., for "plotmath") in R.
>>>
>>> Thanks for looking at it!
>>>
>>> But, really, there is no such thing as a Symbol font on Linux
>>> anymore.
>>> Symbol is pre-unicode thinking. Most modern general-purpose unicode
>>> fonts will include every codepoint Symbol ever shipped, and
>>> fontconfig
>>> will fallback gracefully when that’s not the case (unless your
>>> fontconfig integration is broken).
>>
>> Yep, the "symbol" font is an (outdated) R "plotmath" concept, but
>> one
>> that would take a fair bit of surgery to remove.  R plotmath
>> converts
>> certain R expressions (in certain contexts) to code points in the
>> Adobe
>> Symbol Encoding (ASM), but for cairo-based devices, those are
>> converted
>> to UTF8 code points.
>>
>>> Just use the sans-serif or monospace fontconfig defaults. You don’t
>>> need Symbol, or OpenSymbol, or any special font setup.
>>
>> Agreed.  I got reasonable coverage from DejaVu Sans and FreeSerif.
>> There are still a number of ASM code points that are not covered
>> though,
>> for example, ...
>>
>> F8EB E6 # LEFT PAREN TOP # parenlefttp (CUS)
>> F8EC E7 # LEFT PAREN EXTENDER # parenleftex (CUS)
>> F8ED E8 # LEFT PAREN BOTTOM # parenleftbt (CUS)
>>
>> Even OpenSymbol is missing a few (though perhaps not very common
>> ones) ...
>
> All the F8* codepoints are in the private use area. That means you
> can’t rely on them existing in standard unicode fonts
>
> You need to use correct Unicode values for things to work:
> Ux239… for parenthesis, brackets
>
> https://www.unicode.org/charts/PDF/U2300.pdf
>
>> F8E6    BD      # VERTICAL ARROW EXTENDER       # arrowvertex (CUS)
>> F8E7    BE      # HORIZONTAL ARROW EXTENDER     # arrowhorizex (CUS)
>
> and 23AF/23D0 for arrow extensions (though arrow font support seems
> messy, probably because it sees little use; it’s a pity R comes so late
> to the party, those are just lines, it would have been trivial to get
> them into DejaVu before the project gone dormant). GFS NeoHellenic
> (Math block) seems complete but it’s not a common font family.
>
>> F6DA    D2      # REGISTERED SIGN SERIF # registerserif (CUS)
>> F6D9    D3      # COPYRIGHT SIGN SERIF  # copyrightserif (CUS)
>> F6DB    D4      # TRADE MARK SIGN SERIF # trademarkserif (CUS)
>> F8E8    E2      # REGISTERED SIGN SANS SERIF    # registersans (CUS)
>> F8E9    E3      # COPYRIGHT SIGN SANS SERIF     # copyrightsans (CUS)
>> F8EA    E4      # TRADE MARK SIGN SANS SERIF    # trademarksans (CUS)
>
> Those are useless nowadays, just use normal
> registered/copyright/trademark codepoints, and a font in the wished
> style (serif sans serif, whatever looks nice to you)
>
> Regards

Thanks, that's useful.  For my own memory, this is the parenthesis block
that might be useful ...

U+239b Sm LEFT PARENTHESIS UPPER HOOK ⎛
U+239c Sm LEFT PARENTHESIS EXTENSION ⎜
U+239d Sm LEFT PARENTHESIS LOWER HOOK ⎝
U+239e Sm RIGHT PARENTHESIS UPPER HOOK ⎞
U+239f Sm RIGHT PARENTHESIS EXTENSION ⎟
U+23a0 Sm RIGHT PARENTHESIS LOWER HOOK ⎠
U+23a1 Sm LEFT SQUARE BRACKET UPPER CORNER ⎡
U+23a2 Sm LEFT SQUARE BRACKET EXTENSION ⎢
U+23a3 Sm LEFT SQUARE BRACKET LOWER CORNER ⎣
U+23a4 Sm RIGHT SQUARE BRACKET UPPER CORNER ⎤
U+23a5 Sm RIGHT SQUARE BRACKET EXTENSION ⎥
U+23a6 Sm RIGHT SQUARE BRACKET LOWER CORNER ⎦
U+23a7 Sm LEFT CURLY BRACKET UPPER HOOK ⎧

However, the situation is still not completely straightforward.  The
style of the symbols is also an issue and the DejaVu symbols are not as
elegant as, say, the OpenSymbol symbols.  What makes things tricky is
that, AFAICS, DejaVu has (TTX Unicode cmap output) ...

<map code="0x239b" name="uni239B"/><!-- LEFT PARENTHESIS UPPER HOOK -->

... while OpenSymbol has ...

<map code="0xf8eb" name="parenlefttp"/><!-- ???? -->

... but neither has the other.  So we could not simply switch to
standard Unicode code points because, while that would work with the
"ugly" DejaVu glyphs, that would mean that we could not access the
"pretty" OpenSymbol glyphs.

Paul
--
Dr Paul Murrell
Department of Statistics
The University of Auckland
Private Bag 92019
Auckland
New Zealand
64 9 3737599 x85392
[hidden email]
http://www.stat.auckland.ac.nz/~paul/

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: [FORGED] Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

R devel mailing list
Le mardi 31 mars 2020 à 15:07 +1300, Paul Murrell a écrit :

>

> Thanks, that's useful.  For my own memory, this is the parenthesis
> block
> that might be useful ...
>
> U+239b Sm LEFT PARENTHESIS UPPER HOOK ⎛
> U+239c Sm LEFT PARENTHESIS EXTENSION ⎜
> U+239d Sm LEFT PARENTHESIS LOWER HOOK ⎝
> U+239e Sm RIGHT PARENTHESIS UPPER HOOK ⎞
> U+239f Sm RIGHT PARENTHESIS EXTENSION ⎟
> U+23a0 Sm RIGHT PARENTHESIS LOWER HOOK ⎠
> U+23a1 Sm LEFT SQUARE BRACKET UPPER CORNER ⎡
> U+23a2 Sm LEFT SQUARE BRACKET EXTENSION ⎢
> U+23a3 Sm LEFT SQUARE BRACKET LOWER CORNER ⎣
> U+23a4 Sm RIGHT SQUARE BRACKET UPPER CORNER ⎤
> U+23a5 Sm RIGHT SQUARE BRACKET EXTENSION ⎥
> U+23a6 Sm RIGHT SQUARE BRACKET LOWER CORNER ⎦
> U+23a7 Sm LEFT CURLY BRACKET UPPER HOOK ⎧
>
> However, the situation is still not completely straightforward.  The
> style of the symbols is also an issue and the DejaVu symbols are not
> as elegant as, say, the OpenSymbol symbols.  What makes things tricky
> is that, AFAICS, DejaVu has (TTX Unicode cmap output) ...

Ah, the endless design discussions… Myself I prefer a consistent design
like Dejavu, over cobbling symbols of different designs, because they
used to be in separate fonts. Anyway:

> <map code="0x239b" name="uni239B"/><!-- LEFT PARENTHESIS UPPER HOOK
> -->
>
> ... while OpenSymbol has ...
>
> <map code="0xf8eb" name="parenlefttp"/><!-- ???? -->
>
> ... but neither has the other.  

OpenSymbol is incorrect (it suffers from the same pre-unicode bias as
R). However, it is, to my knowledge, actively maintained. You can ask
its upstream (LibreOffice) for Unicode conformance fixes if you find
problems. Especially when it’s just fixing the map of an existing
glyph, that should not be hard for them to fix. Anything PUA-related
won’t interoperate well in an unicode world.

(you can ask DejaVu too, maybe a request from a project like R will
wake up its maintainers. But, that’s a long shot. DejaVu suffers from
an almost done state without enough remaining work to interest
designers).

Regards,

--
Nicolas Mailhot

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
Reply | Threaded
Open this post in threaded view
|

Re: Plotmath on Fedora 31 broken with with pango >= 1.44 - workarounds?

Iñaki Ucar
In reply to this post by Paul Murrell-2
On Tue, 31 Mar 2020 at 03:32, Paul Murrell <[hidden email]> wrote:

>
> I think R will retain the idea of a separate symbol font in at least the
> short term because of backward compatibility and cross-platform support
> and support for a range of graphics devices.  So this fix is just for
> cairo-based devices on Linux at most (probably only Fedora).
>
> So this becomes just a decision about user interface and default settings.
>
> I did consider the option of allowing the existing "family" parameter to
> be length-two (with the second one being an optional symbol font
> specification), but because of the overlaps of X11/cairo and different
> cairo-based device interfaces, this became awkward.  Hence the separate
> "symbolfamily" interface.  And in any case, this still means a separate
> "symbol" font specification (for the reasons above).
>
> Regarding changing to a default symbolfamily=family on Linux generally
> (rather than just on Fedora), I have at least one counter-example (my
> Ubuntu 18.04) that shows that this would degrade output significantly.
> For one, the symbols are a LOT uglier, plus there are some incorrect
> glyphs.  So I think we have to stay with treating Fedora as a special
> case for now.

You can try Noto Sans Symbols (google-noto-sans-symbols-fonts) or
Symbola (gdouros-symbola-fonts). We could make the R package depend on
any of these fonts included in Fedora.

> Thanks for your point about just using symbolfamily=family as the Fedora
> default.  That seems reasonable (and definitely better than it just
> being completely broken!).
>
> That does still leave the problem of how to set the default value for
> "symbolfamily" JUST on Fedora.   I am not convinced we can use R code to
> detect Fedora >= 30 reliably (but happy to learn otherwise).  Is it a
> possibility for the Fedora distribution to include a .Rprofile.site file
> that sets the X11.options() ?

1. I don't think you need to detect you are in Fedora at all, just to
detect the version of pango, and apply this configuration if it's >=
1.44 (e.g., by executing pango-view --version; or better yet, at
building time https://developer.gnome.org/pango/stable/pango-Version-Checking.html).

2. Yes, we can include any custom configuration files or patches. In
fact we will need to patch R 3.6.3 for Fedora 31 at least, because
Fedora 32 is about to be released, and thus R 4.0.0 won't be included
in Fedora 31. The problem with the .Rprofile.site is that any
user-specific .Rprofile will prevent the default from being loaded,
right? And I'd say ~/.Rprofile is pretty common out there, and even
project-specific .Rprofile.

--
Iñaki Úcar

______________________________________________
[hidden email] mailing list
https://stat.ethz.ch/mailman/listinfo/r-devel
12