Embed System Fonts on Android

[uazo]

Description

the idea is to exploit skia's logic concerning fonts for use in android.
skia must be advised of what fonts are available and the order of fallback to be attempted if a character is not available in the selected font.
Since including the entire set of fonts would increase the size of the apk and the resources required, the automatic downloading of chromium components, reactivated via the specific patch, is being exploited.
once the fonts and the xml describing their use are available, the patch only makes the custom fonts available in skia.
currently only active in android, it remains to be seen whether the same logic can also be used in desktop platforms.

All submissions

• there are no other open Pull Requests for the same update/change
• Bromite can be built with these changes
• I have tested that the new change works as intended (AVD or physical device will do)

Format

• patch subject and filename match (e.g. Subject: Alternative cache (NIK-based) -> Alternative-cache-NIK-based.patch)
• patch description contains explanation of changes
• no unnecessary whitespace or unrelated changes

[uazo]

for those who want to help me with the test, this is the one to do:

• install the https://github.com/uazo/cromite/releases/tag/test_pull_981 version. it is the same as the production version, plus the patch. a rollback to the previous version is feasible (if your android allows it).
• go to chrome://components and wait (or force) the download of the component. the automatic download should start 1 minute after start-up.
• go to chrome://flags and disable (remember to reactivate it when finished)
  • chrome://flags/#fingerprinting-canvas-measuretext-noise
  • chrome://flags/#fingerprinting-client-rects-noise
  • chrome://flags/#fingerprinting-canvas-image-data-noise
• go to https://browserleaks.com, disable Viewport Protection
• go to https://browserleaks.com/fonts and write down Unicode Glyphs Fingerprint and the result of the textarea

via browserstack I have already noticed differences that should not be there:

Details

[uazo]

incredible, then, by flattening out the differences in fonts between different devices, the differences in dpi between different devices emerge...

[uazo]

therefore, increasingly complex.
then, the problem actually seems to be how blink calculates the coordinates, or rather, how it transforms them from local to absolute. it was something I had already seen with the viewport protection patch.

Now, the idea is to proceed in this way:

• make a list of the devices on browserstack (to get the real ones) and understand what dpi they use.
• check whether it is possible to choose a dpi to be fixed in cromite a priori, perhaps based on the resolution of the screen, I don't know.

certainly that way, the apparent 'size' of the screen will change for some users. i don't know if it will bother users.

@PF4Public sorry to bother you, I would need to discuss this idea with someone.

[PF4Public]

Did you investigate how those font fingerprinting tests actually work (I didn't :) )? Do they merely measure the glyph rectangle? In that case it is sad that dpi does make difference. Fixing dpi might do more damage than good. It is the same as with viewport protection: fixing a dpi could blur even more text if actual dpi differs. Maybe randomising those rects instead could help?

@PF4Public sorry to bother you

No problem, feel free to ping me if you want to.

[uazo]

Did you investigate how those font fingerprinting tests actually work (I didn't :) )?

theoretically they work, theoretically you can understand the commit of your device.
the best is browserleaks although it is wrong because it should consider rectangles as float and not as int. Let's hope no one thinks of it :)

Do they merely measure the glyph rectangle?

and that is absolutely correct and sufficient. the dimensions are specific and related to the font file, although 'dirty' by the dpi factor of the screen, which introduces very small differences (of the order of 0.001) but extremely important as information. I don't think anyone has ever got there, I don't think anyone has ever compared the result of different devices but with the absolutely same set of fonts.
That is, I think that if before one could only imagine it working, now I am sure, it works, at least with skia+blink.

Fixing dpi might do more damage than good

I don't think so personally, but I think an attempt should be made.
only that, if it works, it changes the way the user is used to viewing it with the browser (whatever that is) and their device.
Let me explain:

Samsung Galaxy S21 v11.0:
Resolution: 1080 x 2400 px
Viewport: 360 x 670 px
DPI factor: 3 with a screen size of 6.2 in
chrome: F90EDB57
cromite+fonts: 9DA50786

Oppo A96 v12.0:
Resolution:  1080 x 2412 px
Viewport: 360 x 665 px
DPI factor: 3 with a screen size of 6.59 in
chrome: 86782DCA
cromite+fonts: 9DA50786

Google Pixel 4 v11:
Resolution:  1080 x 2280 px
Viewport: 393 x 727 px
DPI factor: 2.75 with a screen size of 6.59 in
chrome: 9A1BF10D
cromite+fonts: 9A1BF10D

Google Pixel 3a:
Resolution:  1080 x 2280 px
Viewport: 393 x 680 px
DPI factor: 2.75 with a screen size of 5.6in 
chrome: E0E22B15
cromite+fonts: 9A1BF10D

I would like to set the dpi by resolution, considering only the width, but this would change the 'apparent' screen size ('apparent' for the user)

Maybe randomising those rects instead could help?

It is currently already so in cromite, I wanted to try to improve it.

[PF4Public]

I would like to set the dpi by resolution, considering only the width, but this would change the 'apparent' screen size ('apparent' for the user)

Could this visually break websites, which build UI via JavaScript and rely on those measurements?

[uazo]

Could this visually break websites, which build UI via JavaScript and rely on those measurements?

no, html is by nature 'responsive', if a site does not adapt it is a limitation of the website, which in any case can force the display with the appropriate meta tags.

[uazo]

it is only the user who may be annoyed by the difference between how it appears in cromite and how it appears in other browsers

[PF4Public]

Could this be used to identify Cromite users and behave differently (as ads do when something blocks them)?

[uazo]

Could this be used to identify Cromite users and behave differently

unfortunately, cromite is already recognisable from so much else: so much so that I am considering whether to modify the useragent by declaring it.
The aim would be to flatten all cromite regardless of the device, especially those with strange floating-point DPIs, which I have only now noticed in some devices with lineage, perhaps due to some strange modification not noticed by the creator of the release.

[PF4Public]

which introduces very small differences (of the order of 0.001)

Could you forcefully round values so that this information wouldn't leave the device?

[uazo]

Could you forcefully round values so that this information wouldn't leave the device?

I had also thought of that solution, but there is no precise point in the code (without agreeing with the chromium developers) on which to intervene. I should start to figure out how to request a change to the w3c standards, but who has the time!

on the contrary, I think that acting on the dpi (if it works) would be the ultimate solution that would not force me to chase the developers' changes. but I do not know in advance the effects on the devices of the users who will then suffer the change (despite themselves), which is the thing that worries me.

[PF4Public]

I do not know in advance the effects on the devices of the users who will then suffer the change (despite themselves), which is the thing that worries me.

It is exactly what I think. Perhaps the only way to know is to try yourself and invite others to testing :)

[uazo]

note (for me): there are performance problems to be fixed. starting a new render process is too slow.

[uazo]

so this is the approach I would like to follow.

• test the use of https://www.browserstack.com/app-automate
• define by device resolution a static DPI factor to be applied. so devices will be grouped by resolution, I don't think it is possible to force a user to use a low resolution with a large screen or vice versa.
• concerning start-up performance, wire into the code the sequence used by skia to initialise fonts.

I think that if it works, I can remove the viewport protection patch in android and perhaps also the canvas patch.
I think all devices with the same resolution will get the same fingerprinting values.

[PF4Public]

I can remove the viewport protection patch in android

Would you attempt the same approach on a desktop?

[uazo]

define by device resolution a static DPI factor to be applied. so devices will be grouped by resolution

I was thinking about it. it doesn't make sense. I try to explain myself.
Assuming for the sake of argument that indeed with the same DPI, resolution and font, the values that can be extracted for fingerprinting are equal, the conclusion is that I would allow the creation of device clusters.

now, what would be the difference compared to now?
I might even conclude that I make life easier for fingerprinting scripts, because I flatten the differences and allow for values that definitely define a device.

it is not the right path.

Would you attempt the same approach on a desktop?

there is no reason to believe that desktop platforms behave differently, so the same approach is also exploitable for those platforms.

the question is, does it make sense?

[uazo]

update. I saw how browserstack's app-automate works, really great. i will certainly exploit it to get the fingerpriting values of all the devices in browserstack, for comparison. too bad it can only be useful for android and not for desktop platforms, but, let's get started.

now i'm developing a patch to enable privileged access to some specific javascript api as test support (@jamchowder it can serve as a basis for #1020 (comment)), specifically for the cromite.org site, while fully respecting users‘ privacy. i will tell you about it as soon as a wip patch is ready, so you can tell me if my idea might offend users’ sensitivities.

[PF4Public]

my idea might offend users’ sensitivities

What do you mean specifically?

[uazo]

certainly, I try to explain myself.

browserstack provides the ability to remotely control their physical devices via various frameworks. My choice is to use Appium, if only because it is the one used by privacytests.org, which has been using it for years, and so I think it is a tried and tested platform, and would allow me to possibly propose something to them in the future.

Appium basically allows, via the UIAutomator2 driver, to automate the ui, replacing the user: for example, I can open internet pages and, I think, access the settings to modify them. there is also a specific driver for chromium (https://w3c.github.io/webdriver) which probably allows something else, but I still have to check the differences.

For what Appium does not allow me to do, I need to evaluate alternatives.

The premise is that the browserstack environment does not allow me to access the chromium profile folder, as the devices are not rooted. But I need to enable and disable flags, change content settings, clear caches and maybe more.
I could produce two apks, one for testing and another for users, but it would complicate my life, both in the build/release action and in patch management, so I would like to produce just one apk.

It is a question of what I can do.
So far I have seen one possibility, which is to activate special javascript APIs for the use and consumption of the test battery. I have to do this, however, without introducing possible privacy breaches.

So the idea is this:

• Activate a new setting in ui that will enable the javascript api that will only be usable in cromite.org
  if active, make a banner appear that warns the user. possibly, the new setting should automatically deactivate at each restart.
  notice that cromite.org does not track anything, no log is active (I know, it's a bit dangerous)
• The javascript API must not communicate anything directly, i.e. they must not fetch. The idea is to write in the device log, considering that browserstack provides me with API to fetch it (https://www.browserstack.com/docs/app-automate/appium/debug-failed-tests/device-logs)

For the test battery, the idea for now is to extract all creepjs fingerprints from all browserstack devices, both chrome and cromite, and compare them to see how to move forward.
Not only that, internally chromium has a framework that can extract additional information about fingerprint values, which the chromium team is using to acquire information directly from users for the privacy sandbox.
I want to understand what they are acquiring.

[uazo]

ahh, I finally managed to use browserstack remotely!

https://app-automate.browserstack.com/builds/b01015e85959e0bf0b0804ca9badd1d767183618/sessions/c032efedac86bf448c80e5356c5cfd1041ea9db7?auth_token=0786d20afdc5de180d17e46585219d53e0bd61c75ff38301b859cdb72fc711f7

it was hard, endless problems, poor documentation, but thank goodness that all the tools are opensource and so there are source codes.
there is no need to make any APIs for cromite.org, but I can put them in a chrome:// page on purpose.
I will open an issue for that purpose.

[PF4Public]

So do I understand that correctly: now you can start a device on browserstack and analyze the fingerprint it gets via automation? Are those devices a real ones or a VM? You could create our own VM for this analysis perhaps.

I just noticed that even on desktop with fonts fingerprint mitigation the browser only uses system fonts, even when webfonts are downloaded. Is it how it is supposed to be?

[uazo]

So do I understand that correctly: now you can start a device on browserstack and analyze the fingerprint it gets via automation?

yes, that is exactly what I am doing in #1271.
I am retrieving the fingerprints of creepjs in both cromite and chromium vanilla.

i already have the first 3 runs of chromium, that's a lot of information that i don't know how to handle at the moment. when the 3 runs of cromite (android) also finish i will publish the data, so you can help me find a tool to read it, for now i'm using PowerBI but it's not the right tool.
taking a look at the values, fingerprint values actually define the device in chromium.
I'm trying to get organised to see what happens if I flatten the font list, so we can see if it makes sense to introduce this patch.

[PF4Public]

you can help me find a tool to read it

I'd be happy to help!

You didn't answer my second question :(

[uazo]

You didn't answer my second question :(

because I had not tried.
look here:

Details

it works.

[PF4Public]

it works.

Hmm, indeed it works for me too. Sorry for bothering you with this. :(

[uazo]

I noticed that the blink process in the sandbox has read access to the .so folder.
I need to check whether I can exploit it to insert a fonts-only resource library.
would increase the size of the apk by 60mb, which is a lot, so it has to be understood whether, in android, the apk is really kept all in memory.