Support @font-face on Linux #28

SimonSapin · 2013-01-08T19:59:52Z

Today on IRC:

SimonSapin: Hi behdad. Does Pango have APIs to load a specific font file? (This is to implement web fonts in WeasyPrint.)
behdad: SimonSapin: no.
behdad: SimonSapin: however, you can subclass the FcFontMap and do it yourself.
behdad: that's what Firefox does.
behdad: don't ask me about the details... I don't quite know that myself either.
SimonSapin: behdad: I’ll look into it, thanks
behdad: well. that's not correct. I know. you can subclass PangoCairoFcFontMap
behdad: and override the create_font(), and figure it out from there
behdad: humm. something like that. don't remember the details.
behdad: the thing is, you need to work around the fontconfig layer

Now we only need to figure out how subclassing works in gobject. And all of fontconfig.

joeimer · 2016-09-12T15:26:53Z

Hi.
Do you have any updates or more information for this task?

Johannes

liZe · 2016-09-13T13:38:42Z

@joeimer Wow. I've tried for the lulz to implement this solution on Stack Overflow, and… it works. As I've added almost all the features from fonts level 3, @font-face would be the icing on the cake.

Of course, this solution will only work on fontconfig-based backends (only Linux I think), but could be adapted for Uniscribe (Windows) and CoreText (OSX) by anyone intersted.

I'll push a new branch soon for testing.

joeimer · 2016-09-13T14:28:29Z

@liZe that was a fast answer. Thank you very much!
I think for the first part it is ok too only support it on linux.

liZe · 2016-09-13T14:29:43Z

@joeimer I'm closing Kozea/tinycss#6 before creating the new branch 😉.

SimonSapin · 2016-09-14T08:16:53Z

Does that require writing the downloaded font to a temporary file so that fontconfig can find it? (Maybe that can be skipped for file: URLs?)

liZe · 2016-09-14T08:27:05Z

Does that require writing the downloaded font to a temporary file so that fontconfig can find it?

Currently yes, giving an URL to FontConfig doesn't work (unsurprisingly).

SimonSapin · 2016-09-14T08:51:56Z

Yeah, to avoid temporary files I was thinking not of giving URLs to fontconfig but give it an in-memory buffer.

joeimer · 2016-09-14T09:01:37Z

In-memory would be perfect. In xhtml2pdf there was a problem with temporary files (permissions).

liZe · 2016-09-14T16:31:32Z

Kozea/tinycss#6 is fixed, I have the @font-face rules in WeasyPrint ✌️.

It's really easy to add an external font into the list of available fonts. I'm now looking for a way to make pango find them with the name given in the @font-face rule.

In-memory would be perfect. In xhtml2pdf there was a problem with temporary files (permissions).

Good to know, thanks.

liZe · 2016-09-14T17:26:01Z

Did you know that @font-face content is a list of descriptors, and that descriptors are not properties, and that their validators are almost the same, but not exactly the same? 😭.

rokob · 2016-09-26T08:12:31Z

Since tinycss 0.4 has @font-face support, do you have a sense of a timeframe for a release of WeasyPrint with @font-face support? I've got something that needs fonts and I am looking at horrible workarounds at the moment.

liZe · 2016-09-26T11:12:38Z

Since tinycss 0.4 has @font-face support, do you have a sense of a timeframe for a release of WeasyPrint with @font-face support?

No, I don't.

I've pushed to the font-face branch what I've done this week-end. Here is the current status:

@font-face rules are parsed,
if the src descriptor contains one URL, the font is downloaded as a temp file and added to the list of available fonts,
the font can then be used in the document using its real font family and properties (and not the ones given in the descriptors).

You can test the current version with a document like this one:

<html>
  <head>
    <style>
@font-face {
  font-family: 'Baloo Tamma';
  src: url(https://fonts.gstatic.com/s/balootamma/v1/M9jLCKQXJLpx_E5bTHjE-J0EAVxt0G0biEntp43Qt6E.ttf);
}

body {
  font-family: 'Baloo Tamma'
}
    </style>
  </head>
  <body>
    <p>Hello World!<p>
  </body>
<html>

What's missing:

managing multiple URLs in src (should be quite easy),
managing local fonts given by the local "function" (not that hard and not mandatory),
changing the family and the properties of the font in Fontconfig (see below),
cleaning the added font after use (quite hard but not mandatory),
masking local fonts (quite easy when point 2 is fixed and not mandatory),
adding the same feature for OSes using a backend different from Fontconfig, ie Windows and OSX (I won't do that myself).

What's blocking me is the Fontconfig part. I've already spent hours (days?) trying to find something more than the API documentation… If you have a real-life example explaining how to change the matching rules of a font, I'm interested. I've started to read the Firefox source code and I don't understand (yet) how it's working. If it's too hard for me, I'll ask The Great @behdad.

If anyone is interested, the current idea is to edit the added font's matching pattern. That's what's currently done in the code, but it doesn't work. I don't know if the edited pattern is just a copy, or if we need to rebuild the database, or… See the add_font_face function.

SimonSapin · 2016-09-26T11:37:27Z

I went over and asked Bedhad (we’re attending the same event for the next couple days) and he pointed me to:

https://mces.blogspot.com.es/2015/05/how-to-use-custom-application-fonts
Alternatively, subclass PangoFcFontMap to override the behavior of font selection.

liZe · 2016-09-26T11:47:12Z

The first post looks wonderful, with links to font configuration modifications, in-memory fonts, synthetic bold and italic, and more. Thanks @SimonSapin and @behdad, have fun!

liZe · 2016-09-26T18:17:59Z

Changing the family and properties of the font is done in fc5eb6c. This blog entry is really, really useful. I'll fix the multiple src, the masking of the local fonts and maybe the local function really soon.

@rokob You can test the branch if you want!

behdad · 2016-09-27T10:45:25Z

Thanks @SimonSapin and @liZe. Note that subclassing PangoFcFontMap is what Firefox 2 or 3 did to support webfonts.

liZe · 2016-10-01T19:22:33Z

I've added the support of multiple values for src and of the local function. It's been tested with multiple different configurations (including Google Fonts), and it seems to work very well.

Font selection is complete for now, the only descriptor not supported is unicode-range and it's not something I want to support now 😄. I'd like to add the @font-face font features before merging.

I'll add unit tests, for @font-face of course but also for different font features that couldn't be tested before (including the test of condensed fonts that was only working for lucky people). Having this feature is really good news for our the tests.

Some problems that I won't fix:

the unicode-range descriptor is not supported,
the font matching algorithm for @font-face is pretty bad,
the generated fontconfig configurations, the local fonts and the downloaded fonts are stored as temporary files (see the fontconfig bugs #78450 and #78452),
the local and dowloaded fonts are not cleaned (should they be removed when the Document object is deleted?).

liZe · 2016-10-01T19:23:34Z

Oh, and we should create a fontconfig configuration for each different document.

whitelynx · 2016-10-07T08:32:08Z

@liZe: Does your branch include support for specifying font files using data URIs? I tried checking out your branch, but it didn't work with the data URIs I'm using. (I think they're working in Chrome on the same page for me, but I probably need to verify that by uninstalling my local copy of the font first)

liZe · 2016-10-07T09:26:25Z

Does your branch include support for specifying font files using data URIs?

Not yet, but that should be easy to fix!

whitelynx · 2016-10-07T09:53:46Z

@liZe: Sorry, my memory failed me. The actual problem I ran into when using data URIs was that the process crashes with a SIGABRT:

python: fcmatch.c:779: IA__FcFontMatch: Assertion `result != NULL' failed.
fish: Job 2, “font-face-env/bin/python tools/…” terminated by signal SIGABRT (Abort)

Sadly, I haven't figured out how to actually make gdb catch the crash, so I don't really have any more debugging info at the moment.

whitelynx · 2016-10-07T10:22:09Z

I have a minimal test case for the crash: https://gist.github.com/whitelynx/39938eb6f4a997094d305075d41b23ac

liZe · 2016-10-07T12:03:48Z

@whitelynx Thanks a lot for reporting, that wasn't supported at all. I've used the real URL fetcher instead of Python's urlopen, it's much better.

I've also added the support of the font-variant and font-feature-settings descriptors.

The tests I've done seem to work. I somtimes get broken fonts in PDFs, don't know why and don't know how to reproduce.

rokob · 2016-10-10T15:23:28Z

This works locally for me (on a Mac) but fails on my server (running Ubuntu). At the moment, I am assuming this is some server configuration error on my side rather than an issue with the code.

liZe · 2016-10-10T15:28:48Z

This works locally for me (on a Mac)

On a Mac? Are you sure?

but fails on my server (running Ubuntu)

You can post logs here 😄.

rokob · 2016-10-13T09:27:42Z

Maybe I should say, pdfs are generated in both cases. On the Mac I get the correct fonts in the output, on Ubuntu I see the fallback fonts. I will turn the logger back on and see what the errors are on both.

liZe · 2016-10-26T17:27:50Z

Oh, and we should create a fontconfig configuration for each different document.

It's now done and tested, that was actually needed to use the same font family with different attributes set in different tests. I've blindly followed @behdad's post, I'm not sure to always understand everything 😉.

To make the tests reliable, I've created a font based on ahem with OpenType features.

The current implementation works well, but it relies on a global state (using pango_cairo_font_map_set_default). Fixing that is not hard, but it relies to have one more argument through the whole layout. @SimonSapin do you think it's time to put everything related to the document/html/css that is needed for the layout in one class? I think that we have at least style_for, get_image_from_uri, enable_hinting (currently set to True in some functions where it should be available) and now font_config.

SimonSapin · 2016-10-26T17:39:38Z

There’s already a LayoutContext class passed around. Would that work?

liZe · 2016-10-27T16:09:59Z

There’s already a LayoutContext class passed around. Would that work?

Yes. I've cleaned a couple of functions to use that too instead of multiple arguments.

I've open a PR (#374), I'd loooooove to have nice testers!

liZe · 2016-11-03T23:43:46Z

I've open #376 and #377 if anyone wants to support other OSes.

@font-face

Version 0.34 ------------ Released on 2016-12-21. Bug fixes: * `#398 <https://github.com/Kozea/WeasyPrint/issues/398>`_: Honor the presentational_hints option for PDFs. * `#399 <https://github.com/Kozea/WeasyPrint/pull/399>`_: Avoid CairoSVG-2.0.0rc* on Python 2. * `#396 <https://github.com/Kozea/WeasyPrint/issues/396>`_: Correctly close files open by mkstemp. * `#403 <https://github.com/Kozea/WeasyPrint/issues/403>`_: Cast the number of columns into int. * Fix multi-page multi-columns and add related tests. Version 0.33 ------------ Released on 2016-11-28. New features: * `#393 <https://github.com/Kozea/WeasyPrint/issues/393>`_: Add tests on MacOS. * `#370 <https://github.com/Kozea/WeasyPrint/issues/370>`_: Enable @font-face on MacOS. Bug fixes: * `#389 <https://github.com/Kozea/WeasyPrint/issues/389>`_: Always update resume_at when splitting lines. * `#394 <https://github.com/Kozea/WeasyPrint/issues/394>`_: Don't build universal wheels. * `#388 <https://github.com/Kozea/WeasyPrint/issues/388>`_: Fix logic when finishing block formatting context. Version 0.32 ------------ Released on 2016-11-17. New features: * `#28 <https://github.com/Kozea/WeasyPrint/issues/28>`_: Support @font-face on Linux. * Support CSS fonts level 3 almost entirely, including OpenType features. * `#253 <https://github.com/Kozea/WeasyPrint/issues/253>`_: Support presentational hints (optional). * Support break-after, break-before and break-inside for pages and columns. * `#384 <https://github.com/Kozea/WeasyPrint/issues/384>`_: Major performance boost. Bux fixes: * `#368 <https://github.com/Kozea/WeasyPrint/issues/368>`_: Respect white-space for shrink-to-fit. * `#382 <https://github.com/Kozea/WeasyPrint/issues/382>`_: Fix the preferred width for column groups. * Handle relative boxes in column-layout boxes. Documentation: * Add more and more documentation about Windows installation. * `#355 <https://github.com/Kozea/WeasyPrint/issues/355>`_: Add fonts requirements for tests. Version 0.31 ------------ Released on 2016-08-28. New features: * `#124 <https://github.com/Kozea/WeasyPrint/issues/124>`_: Add MIME sniffing for images. * `#60 <https://github.com/Kozea/WeasyPrint/issues/60>`_: CSS Multi-column Layout. * `#197 <https://github.com/Kozea/WeasyPrint/pull/197>`_: Add hyphens at line breaks activated by a soft hyphen. Bux fixes: * `#132 <https://github.com/Kozea/WeasyPrint/pull/132>`_: Fix Python 3 compatibility on Windows. Documentation: * `#329 <https://github.com/Kozea/WeasyPrint/issues/329>`_: Add documentation about installation on Windows. Version 0.30 ------------ Released on 2016-07-18. WeasyPrint now depends on html5lib-0.999999999. Bux fixes: * Fix Acid2 * `#325 <https://github.com/Kozea/WeasyPrint/issues/325>`_: Cutting lines is broken in page margin boxes. * `#334 <https://github.com/Kozea/WeasyPrint/issues/334>`_: Newest html5lib 0.999999999 breaks rendering. Version 0.29 ------------ Released on 2016-06-17. Bug fixes: * `#263 <https://github.com/Kozea/WeasyPrint/pull/263>`_: Don't crash with floats with percents in positions. * `#323 <https://github.com/Kozea/WeasyPrint/pull/323>`_: Fix CairoSVG 2.0 pre-release dependency in Python 2.x. Version 0.28 ------------ Released on 2016-05-16. Bug fixes: * `#189 <https://github.com/Kozea/WeasyPrint/issues/189>`_: ``white-space: nowrap`` still wraps on hyphens * `#305 <https://github.com/Kozea/WeasyPrint/issues/305>`_: Fix crashes on some tables * Don't crash when transform matrix isn't invertible * Don't crash when rendering ratio-only SVG images * Fix margins and borders on some tables Version 0.27 ------------ Released on 2016-04-08. New features: * `#295 <https://github.com/Kozea/WeasyPrint/pull/295>`_: Support the 'rem' unit. * `#299 <https://github.com/Kozea/WeasyPrint/pull/299>`_: Enhance the support of SVG images. Bug fixes: * `#307 <https://github.com/Kozea/WeasyPrint/issues/307>`_: Fix the layout of cells larger than their tables. Documentation: * The website is now on GitHub Pages, the documentation is on Read the Docs. * `#297 <https://github.com/Kozea/WeasyPrint/issues/297>`_: Rewrite the CSS chapter of the documentation.

cinatic · 2017-07-16T16:46:44Z

It's strange on Archlinux x64, the fonts can not be loaded, i think "local" is not yet working right?

@font-face {
font-family: 'OpenSans';
font-style: normal;
font-weight: 400;
src: local('OpenSans'), local('OpenSans-Regular'), url("fonts/openSans.woff2") format('woff2');
unicode-range: U+0000-00FF, U+0131, U+0152-0153, U+02C6, U+02DA, U+02DC, U+2000-206F, U+2074, U+20AC, U+2212, U+2215, U+E0FF, U+EFFD, U+F000;
}

127.0.0.1 - - [16/Jul/2017:18:40:16] "GET /styles/fonts/openSans.woff2 HTTP/1.1" 200 15530 "" "WeasyPrint 0.36 (http://weasyprint.org/)"
WARNING: Failed to load font at "http://shop.de:4444/styles/fonts/openSans.woff2"
WARNING: Font-face "OpenSans" cannot be loaded

liZe · 2017-07-16T16:54:13Z

i think "local" is not yet working right?

local works, as long as the font is installed on your system and can be found by Pango with this name. It works for me (OpenSans is installed on my system). If it's not found, you should get these warnings:

WARNING: Ignored `unicode-range: U+0-FF, U+131, U+152-153, U+2C6, U+2DA, U+2DC, U+2000-206F, U+2074, U+20AC, U+2212, U+2215, U+E0FF, U+EFFD, U+F000` at 7:1, descriptor not supported.
WARNING: Failed to load local font "OpenSans"
WARNING: Failed to load local font "OpenSans-Regular"
WARNING: Failed to load font at "file:///tmp/fonts/openSans.woff2" (URLError: <urlopen error [Errno 2] No such file or directory: '/tmp/fonts/openSans.woff2'>)
WARNING: Font-face "OpenSans" cannot be loaded

unicode-range is not supported (see #378). And I can't reach http://shop.de:4444/styles/fonts/openSans.woff2, that's maybe why you get the error about it.

cinatic · 2017-07-16T16:59:46Z

thx for the information, shop.de is only bound to localhost right now.

you can see it loads the woff file successfully:
127.0.0.1 - - [16/Jul/2017:18:40:16] "GET /styles/fonts/openSans.woff2 HTTP/1.1" 200 15530 ""

i have installed a google font package (e.g. i can select it in libreoffice) i guess they are installed

I replaced the woff file with ttf, now the font works

liZe · 2017-07-16T17:22:54Z

i have installed a google font package (e.g. i can select it in libreoffice) i guess they are installed

If you get the local warnings (and I assume you do, because you wouldn't get the warning about the woff2 file otherwise), it means that the font can't be found on your system.

A common mistake is to install a font only for an user (the one you use for LibreOffice), but not for the user launching WeasyPrint. If you use WeasyPrint in a web app (probably launched by a dedicated user), I bet that it's your problem. If you want to be sure, you can install this AUR, it installs the font for the whole system, not just an user.

I replaced the woff file with ttf, now the font works

FreeType probably doesn't support the woff2 format yet. It's hard to find anything but this short thread about that.

cinatic · 2017-07-16T17:40:43Z

okay, i replaced the local with url location and pointed it to the ttf and it works perfactly fine, didn't know why i even put both the locale thing here. I also think that woff does not work. Thanks for the additional information =)

iosifnicolae2 · 2018-12-24T14:03:54Z

If anyone uses Alpine images you should install ttf-ubuntu-font-family and cairo==1.16 packages.

RUN apk update && \
 apk --update --upgrade add ttf-ubuntu-font-family jpeg-dev zlib-dev libffi-dev  gdk-pixbuf pango-dev && \
 apk add cairo==1.16.0-r0  --update-cache --repository http://dl-3.alpinelinux.org/alpine/edge/main/ --allow-untrusted

SimonSapin mentioned this issue Nov 29, 2014

Can't use font-family. #231

Closed

liZe self-assigned this Oct 27, 2016

liZe added this to the v0.32 milestone Oct 27, 2016

liZe closed this as completed Nov 3, 2016

liZe changed the title ~~@font-face~~ @font-face on Linux Nov 3, 2016

liZe changed the title ~~@font-face on Linux~~ Support @font-face on Linux Nov 3, 2016

Support @font-face on Linux #28

Support @font-face on Linux #28

Comments

SimonSapin commented Jan 8, 2013

joeimer commented Sep 12, 2016

liZe commented Sep 13, 2016

joeimer commented Sep 13, 2016

liZe commented Sep 13, 2016

SimonSapin commented Sep 14, 2016

liZe commented Sep 14, 2016

SimonSapin commented Sep 14, 2016

joeimer commented Sep 14, 2016

liZe commented Sep 14, 2016

liZe commented Sep 14, 2016

rokob commented Sep 26, 2016

liZe commented Sep 26, 2016 • edited Loading

SimonSapin commented Sep 26, 2016

liZe commented Sep 26, 2016

liZe commented Sep 26, 2016

behdad commented Sep 27, 2016

liZe commented Oct 1, 2016

liZe commented Oct 1, 2016

whitelynx commented Oct 7, 2016

liZe commented Oct 7, 2016

whitelynx commented Oct 7, 2016

whitelynx commented Oct 7, 2016

liZe commented Oct 7, 2016

rokob commented Oct 10, 2016

liZe commented Oct 10, 2016

rokob commented Oct 13, 2016

liZe commented Oct 26, 2016

SimonSapin commented Oct 26, 2016

liZe commented Oct 27, 2016

liZe commented Nov 3, 2016

cinatic commented Jul 16, 2017

liZe commented Jul 16, 2017

cinatic commented Jul 16, 2017

liZe commented Jul 16, 2017 • edited Loading

cinatic commented Jul 16, 2017

iosifnicolae2 commented Dec 24, 2018

liZe commented Sep 26, 2016 •

edited

Loading

liZe commented Jul 16, 2017 •

edited

Loading