EXT_texture_avif implementation

[leon]

My Christmas contribution is trying to get AVIF support in GLTF, Babylon and Three.

I Did a PR for the GLTF spec which you can find here
KhronosGroup/glTF#2235

I have also done a PR for three here
mrdoob/three.js#25171

My thoughts are the following.
Chrome recently gave up on jpeg xl which I personally thought was the superior replacement for gif, png, jpeg, webp.

The next best format after jpeg xl is avif, and since jpeg xl isn't happening I thought I could do a drive trying to improve the file size of gltf files by getting avif support.

Solution
Add the not yet merged KhronosGroup/glTF#2235 EXT_texture_avif as a supported extension.

[sebavan]

Engine Capabilities should have a new flag to specify whether AVIF is supported on the current platform.

The extension should be disabled if AVIF is supported in engine.getCaps()

[sebavan]

Would be great to drop a comment to say Experimental Support as the spec is still evolving.

[sebavan]

A link to the PR would be useful as well :-) cause the provided spec link is not available so far

[bghgary]

Probably should remove the dead link for now. Once the PR is merged, we can update this comment. For now, maybe point to the PR's file for the specification?

https://github.com/KhronosGroup/glTF/blob/5cb7518cf9a1bfb8268320026961b21caf5a4aac/extensions/2.0/Vendor/EXT_texture_avif/README.md

[sebavan]

Thanks a lot for the Contrib @leon I love where this is going

[leon]

@sebavan I added the PR link temporarily.

I modelled the PR after how WebP was implemented in glTF.
I could not find any references to webp in getCaps(), are you sure we need to add it there?

[sebavan]

AVIF might not be as widly supported as webp ???

[leon]

@sebavan the backstory of why I decided to try to also add avif.

I have been keeping my eye on the image formats for a long time,
first it was jpeg 2000,
then flif
then flif was merged into jpeg xl
and parallel to jpeg xl the av1 video encoding got released and based on that the avif format.

jpeg xl looked really promising as the replacement for png, jpeg, gif.

but then google pulled the plug on jpeg xl from chrome 110.
and webp2 seems to only be experimental.
which makes me believe avif is going to win (if jpeg xl doesn't get added back to chrome again)

Recent thought

one of the biggest proponents to having something like this is transfer size.
But as someone pointed out, it's still big in the gpu and using basis universal + ktx2 is probably a better solution.
The problem there is tooling support.
maybe I should try getting ktx2 / basis support in blender instead...

What do you think, is it a good or bad idea to add avif support?

[sebavan]

I like the overall approach :-) but I am super n00b with image formats. Let me add @bghgary to the thread for this part.

Regarding this specific implementation, as long as AVIF is not generalized in all browsers, I am just wondering if we should feature proof it ?

[leon]

In three.js they check for support in the texture loader.

https://github.com/mrdoob/three.js/pull/25173/files

We try loading a img with a base64 encoded 1x1 px
And check if the size is 1px

Same as how they check for webp support.

Maybe we should do something like that?

As for the extension it has fallback support, where you can have both avif and a png.
But when using glb it defeats the purpose of having smaller downloads.

[bghgary]

Looks good from a code perspective besides some minor issues. Adding a test for this would be good also, but maybe don't do it until the spec is close to final?

[bghgary]

Probably should remove the dead link for now. Once the PR is merged, we can update this comment. For now, maybe point to the PR's file for the specification?

https://github.com/KhronosGroup/glTF/blob/5cb7518cf9a1bfb8268320026961b21caf5a4aac/extensions/2.0/Vendor/EXT_texture_avif/README.md

[bghgary]

Probably should update this also?

Babylon.js/packages/dev/serializers/src/glTF/2.0/glTFMaterialExporter.ts

Lines 1089 to 1102 in ad4785d

	// Preserve texture mime type if defined
	const textureMimeType = (babylonTexture as Texture).mimeType;
	if (textureMimeType) {
	switch (textureMimeType) {
	case "image/jpeg":
	case "image/png":
	case "image/webp":
	mimeType = textureMimeType as ImageMimeType;
	break;
	default:
	Tools.Warn("Unsupported media type: ${textureMimeType}");
	break;
	}
	}

[bghgary]

What do you think, is it a good or bad idea to add avif support?

We should have this conversation on the glTF side, which looks like is already happening.

[sebavan]

@leon let me move as a draft until the GLTF group is taking a decision. This is still being discussed actively.

[sebavan]

@bghgary any progress on this one from the group ?

[bghgary]

No, it hasn't been on the agenda recently.

[sebavan]

Still in discussion within the GLTF group so not closing.

[github-actions]

This pull request has been marked as stale because it has been inactive for more than 14 days. Please update to "unstale".

[github-actions]

This pull request has been marked as stale because it has been inactive for more than 14 days. Please update to "unstale".