Consider (optionally) using tx instead of compreffor

[khaledhosny]

It would be nice if it made possible to usetx for subroutinization instead of compreffor as the later seems to be much much slower:

$ time compreffor LibertinusMath-Regular.hint.otf o.otf
   real	0m23.915s
   user	1m32.836s
   sys	0m0.203s

$ time tx -cff +S +b LibertinusMath-Regular.hint.otf o.cff
   real	0m0.383s
   user	0m0.355s
   sys	0m0.022s

Also tx is making a slightly smaller CFF table (436882 bytes vs 479646 from compreffor).

Although tx is command line tool and not a Python module, I think the gain would be worse the effort to wrap it.

[anthrotype]

Yes, i'd like to do that.
I started this some time ago:
https://github.com/anthrotype/afdko-tx-python

Though it uses subprocess, it does the job.
I couldn't figure out how to extract the subroutinizing code for tx to make it a proper C extension module. But using subprocess has some advantages like having a single wheel for all python versions.

If Khaled fancies making a proper wrapper, then great. Otherwise we can just use that one for now.

[khaledhosny]

If past experiences with other AFDKO C code is an indication, I’d not fancy doing this at all. The code might not be prepared for being used as a library (global state, etc.) and it would break in hard to debug ways if used as such.

[anthrotype]

@cjchapman @miguelsousa @josh-hadley Would you be interested in hosting this separate repository https://github.com/anthrotype/afdko-tx-python for packaging up tx as a python subprocess in the adobe-type-tools github organization?
Alternatively I can put it in fonttools or googlefonts org.

I woudn't want fontmake to depend on the entire afdko package, and search within the user's $PATH for a tx executable. The afdko package comes with dozens of other binary scripts which we don't need for fontmake, so I prefer to package tx as a standalone python module and only require that. In fact this afdko_tx package doesn't even install tx in the bin folder, but simply embeds it in the lib and calls it from there.

The name afdko_tx is not ideal, I know, but tx is already taken, and afdko of course is something else and I can't overwrite that (if it were a namespace package, maybe.. but that's too late now to change).

WDYT?

[josh-hadley]

I discussed this a little bit with @cjchapman, who has previously isolated and ported the tx subroutinizer code into makeotf. He thinks (and I agree) that it is feasible to set up a project under the adobe-type-tools org that just does the subroutinization part (basically the guts of subr.c in makeotf). @khaledhosny's concerns about global state, etc. are duly noted and should not be dismissed, but I think it can be done without too much trouble. So we'd have a standalone subroutinizer (as either a C executable or as a Python C Extension). Tool name could be cffsubr? pssubr? Open to suggestions...

[miguelsousa]

+1 for cffsubr (or subrcff).

[anthrotype]

Thank you guys. It would be nice to have that standalone subroutinizer (I like cffsubr).
But it will basically look very much like the one I made, only that instead of embedding tx binary it will embed some other C executable called differently, that only does subroutinizing.

What we need is essnentially this subroutinize function that takes some bytes (the input CFF1/2 table) and returns some other bytes (the subroutinized one), and has an option to select the output format (1 or 2):
https://github.com/anthrotype/afdko-tx-python/blob/b6e59a1917a0be6b46a139082db546df0f85d6ce/src/afdko_tx/__init__.py#L41-L64

May I suggest we do this:

1. move that afdko_tx project under adobe org
2. rename it to cffsubr, and initially keep the fact that it embeds tx executable as an implementation detail. It is not installed in bin/Scripts, it is only embedded, as if it were a library.
3. later, we can work on extracting the subroutinizing code from tx and only build/embed that for this cffsubr python package.

This way we could have this tool ready immediately, by using tx initially, and refine later.

[anthrotype]

@cjchapman @josh-hadley @miguelsousa so WDYT of my proposal above?

[josh-hadley]

@anthrotype : we have set up https://github.com/adobe-type-tools/cffsubr, ready to populate with your current work and proceed.

[anthrotype]

thanks @josh-hadley! i'll do that soon

[anthrotype]

Ok, I populated the repo with the content from the old afdko_tx and renamed the module cffsubr. It only exports a single subroutinize function, and no longer exposes tx command line interface, since we want the reliance on tx executable to be an implementation detail.
I just need somebody from adobe-type-tools GH organization to help encrypting the PyPI token to publish the wheels (adobe-type-tools/cffsubr#1).

[anthrotype]

initial version of cffsubr is now on PyPI, it should be enough to start experimenting with integrating it with ufo2ft.

[anthrotype]

In case somebody is interested in testing out the standalone cffsubr, this now has a functional CLI tool python -m cffsubr, which takes an OTF, runs the tx subroutinizer on the CFF or CFF2 table, and writes the modified OTF (unlike the tx tool that outputs only the single CFF/CFF2 table data, not the whole OTF). It can also convert between CFF and CFF2.
The python API has a cffsubr.subroutinize function that takes a fontTools TTFont object and applies the subroutinization on it (can optionally modify and return a copy with inplace=False).

[khaledhosny]

Thanks @anthrotype, works like a charm!