-
Notifications
You must be signed in to change notification settings - Fork 3.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
KTX2 integration -- Dropping support for KTX #9040
Closed
Closed
Changes from all commits
Commits
Show all changes
19 commits
Select commit
Hold shift + click to select a range
bb98300
KTX2 support added
YVin3D 3fb2094
Added Fix for ETC1
YVin3D 94a8081
Weird then chaining bug
YVin3D d5b3893
Added transcoding testing code
YVin3D f47b6d5
Reverted back to production build for testing GLTF-KTX2 files
YVin3D bf5bc7b
Dropped KTX, Updated sandcastles to support KTX2 instead, updated spe…
YVin3D 04e58bf
Updated specs to remove CRN, also updated LICENSE.md
YVin3D f808f11
Added back missing whiteShapes.png file
YVin3D b03dc27
Incremental changes made based on PR, loadKTX2Spec still fails
YVin3D fda4f2f
Updated Spec models with GLTF 2.0 embedded KTX2 model as per PR
YVin3D 6dda57e
Removed compressedImage3DTiles
YVin3D cf3d392
Updated GltfPipeline
YVin3D b3c83f3
added support for Env map pixel type
YVin3D 008e5ef
Fixed environment map parsing
YVin3D d4a02b7
update transcoder to 78790b5, fix some bugs with KTX2 (ECT1S) mipmaps
likangning93 fdfa914
fix typescript error
likangning93 6358371
move KTX2 parsing to a webworker, un-monkeypatch msc_basis_transcoder.js
likangning93 19e9247
add IE11 support with a Javascript fallback built against KTX-Softwar…
likangning93 a66e5e0
fix parseKTX2 bug with non-square textures
likangning93 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
Binary file not shown.
39 changes: 39 additions & 0 deletions
39
Apps/SampleData/Cesium3DTiles/Tilesets/KTX2Tileset/tileset.json
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,39 @@ | ||
| { | ||
| "asset": { | ||
| "version": "1.0" | ||
| }, | ||
| "geometricError": 70, | ||
| "root": { | ||
| "transform": [ | ||
| -0.8608332815512961, | ||
| -0.026091425783012157, | ||
| -0.5082177671768633, | ||
| 0, | ||
| 0.324224005982592, | ||
| -0.7978680937788927, | ||
| -0.5082177671768631, | ||
| 0, | ||
| -0.39223061496806577, | ||
| -0.6022671686471426, | ||
| 0.6952908760019272, | ||
| 0, | ||
| -2506152.030937531, | ||
| -3848177.6543499078, | ||
| 4412811.205275625, | ||
| 1 | ||
| ], | ||
| "refine": "REPLACE", | ||
| "boundingVolume": { | ||
| "sphere": [ | ||
| 0, | ||
| 0, | ||
| 0, | ||
| 100 | ||
| ] | ||
| }, | ||
| "geometricError": 0, | ||
| "content": { | ||
| "uri": "Balloon.b3dm" | ||
| } | ||
| } | ||
| } |
Binary file not shown.
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,54 @@ | ||
| ## Updating the KTX2 Transcoder Module | ||
|
|
||
| We use a custom build of the `msc_basis_transcoder` tool in Khronos's [KTX-Software](https://github.com/KhronosGroup/KTX-Software/) project to allow for IE11 compatability. | ||
|
|
||
| Updating the transcoder requires building both the web assembly version and Javascript fallback. | ||
|
|
||
| These directions were written against https://github.com/KhronosGroup/KTX-Software/commit/78790b5ea9c9f432a8e877fbc8cfd977617bb775. | ||
|
|
||
| ### Web Assembly Build | ||
|
|
||
| Follow the instructions at https://github.com/KhronosGroup/KTX-Software/blob/master/BUILDING.md#webemscripten. | ||
|
|
||
| You will need a make tool as well as emscripten. | ||
|
|
||
| - Preferably, use Linux or [the Linux Subsystem for Windows 10](https://docs.microsoft.com/en-us/windows/wsl/install-win10) and `make` to build. | ||
| - Optionally, [MSYS2](http://www.msys2.org/). Follow installation instructions and add the path to MSYS2's usr/bin to your `PATH`. For example, if the path is `C:\msys64\usr\bin`, run `PATH=%PATH%;C:\msys64\usr\bin`. | ||
|
|
||
| ### Javascript fallback build | ||
|
|
||
| As of this writing, `KTX-Software` does not provide an official configuration parameter for configuring the build to Javascript/asm.js instead of web assembly. An issue is open here: https://github.com/KhronosGroup/KTX-Software/issues/332 | ||
|
|
||
| You will need to edit the project's `CMakeLists.txt` to include the required configuration flags. | ||
|
|
||
| 1. Make a separate build directory for the Javascript fallback. | ||
| 2. in `KTX-Software/CMakeLists.txt`, locate the variable `KTX_EMC_LINK_FLAGS` | ||
| 3. add emscripten flags `WASM=0`, `LEGACY_VM_SUPPORT=1`, and `--memory-init-file 0`, ie. | ||
|
|
||
| ``` | ||
| set( | ||
| KTX_EMC_LINK_FLAGS | ||
| --bind | ||
| "SHELL:--source-map-base ./" | ||
| "SHELL:-s ALLOW_MEMORY_GROWTH=1" | ||
| "SHELL:-s ASSERTIONS=0" | ||
| "SHELL:-s MALLOC=emmalloc" | ||
| "SHELL:-s MODULARIZE=1" | ||
| "SHELL:-s FULL_ES3=1" | ||
| "SHELL:-s WASM=0" # for asm.js | ||
| "SHELL:-s LEGACY_VM_SUPPORT=1" # for asm.js | ||
| "SHELL: --memory-init-file 0" # for asm.js | ||
| ) | ||
| ``` | ||
|
|
||
| 4. Configure and build in the new directory. Be aware that the directions in the `KTX-Software` assume directory names, make sure to use the new directory, ie. | ||
|
|
||
| ```bash | ||
| cd path/to/new/build/dir && cmake .. -G "Unix Makefiles" -DCMAKE_TOOLCHAIN_FILE="path/to/emsdk/Emscripten.cmake" && cmake --build | ||
| ``` | ||
|
|
||
| ### Adding to Cesium | ||
|
|
||
| Note that both builds will produce a file called `msc_basis_transcoder.js`. Rename the web assembly version (the smaller file with a corresponding `msc_basis_transcoder.wasm`) before placing both in `Source/ThirdParty/Workers`. | ||
|
|
||
| Place `msc_basis_transcoder.wasm` in `Source/ThirdParty`. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Are these instructions out of date/not working?
One problem we had when initially creating this Sandcastle is it was completely opaque what all these coefficients meant, or how to get your own. The user was never supposed to do this process manually. The glTF IBL extension just wasn't ready at the time.
But now it is: https://github.com/KhronosGroup/glTF/tree/master/extensions/2.0/Vendor/EXT_lights_image_based#ext_lights_image_based.
This example can be greatly simplified into essentially one line if we just add support for that extension. I would suggest we do that (not necessarily in this PR, open an issue for it?), and/or just remove this example because it isn't very useful to Cesium users as-is.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yeah, Google's Filament hasn't been updated to support KTX2. I had to rebuild this cubemap by using this website, which gave six PNG images, which I then had adjust the opacity of and then convert to KTX2 using the command line tool from this repo.
We could always open up an issue to build a tool that automates this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
One limitation of using
EXT_lights_image_basedis that it needs to be added to the glTF's individually whereas Cesium users might want to set an environment map for the entire scene.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. It's critical for users to be able to select their own HDR, IBL environments. Can't expect ordinary models to bundle them.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would most users know how to/want to manually specify the 9 coefficients as shown here? That was the part I was thinking we could omit from this sandcastle, and fallback to the procedural.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If you fall back to procedural, the diffuse lighting on the model won't match the specular lighting environment, which is not what a user who cares about lighting environments is going to want to see. That said, I'm no fan of manually specifying spherical harmonics in code, that seems ridiculous and error-prone. But is the alternative to force all users to always accept a default procedural diffuse lighting environment?
By comparison, BabylonJS stores pre-filtered lighting environments in
.envfiles of their own design, which users can create in their Sandbox (instructions). The .env file contains both diffuse and specular lighting pre-filtered, making it ready to render models with. I don't know if the format is standard or open, but I suppose we could ask. For users this is very easy, just load the env and assign it to the scene, and all your models will have the correct lighting and reflections of the environment.In ThreeJS, they can now load
*.hdrfiles directly. But these need some pre-filtering at load time, and they have a class called PMREMGenerator that will do exactly this. This costs some load-time performance, but offers users the convenience of directly using the most common HDR environment format found online.I like the KTX file, but it looks like the file is only storing the pre-filtered specular lighting environment, and dumping out the diffuse as a bunch of SH coefficients for users to deal with separately. Unless maybe KTX2 has some way to include the diffuse environment as well? Ideally there should be a standard environment format that includes both.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I've been working on the VSCode glTF preview, trying to get nice IBLs in all the preview windows, and it would be nice to include Cesium in this too.
In this commit I hacked up a little
sh.txtfile reader, which is the text file that cmgen writes out when it processed an IBL into a KTX. It loads the Spherical Harmonics and builds the array of Cartesians for them. Not sure if this will actually get published, but it seems to work.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Watching a Virtual SIGGRAPH presentation on Filament (Moving Mobile Graphics, ~2:28:00). Romain says they're not using spherical harmonics at all anymore. Instead, they stop the specular IBL mipmaps at 16x16 smallest size with roughness == 1.0 at that size, and they re-use that particular mip level as their diffuse IBL.
I think Cesium should do that too, and ditch the SH coefficients.