Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement OpenXR Foveated rendering support #80881

Merged
merged 1 commit into from
Sep 25, 2023

Conversation

BastiaanOlij
Copy link
Contributor

@BastiaanOlij BastiaanOlij commented Aug 22, 2023

This PR implements the foveated rendering extension that is available in OpenXR. This is a port of the logic from Godot 3.

This extension is currently only available on a limited number of stand alone Android XR devices:
https://github.khronos.org/OpenXR-Inventory/extension_support.html#XR_FB_foveation
The expectation is that other devices will support this extension but also that this will be limited to Android.

The feature is only implemented for the compatibility renderer.

We can't use the approach on the Vulkan renderer because we do not render 3D content to the buffers created by OpenXR. This may change in the future.
For GPUs that support this, our build in VRS solution is a better option however currently VRS is not exposed on Android (as far as I can tell from the documentation, VRS support would be required by this extension as well).

Tested this on a Quest Pro configured to 90Hz refresh rate with a higher poly scene.
Before applying foveated rendering this project runs at 70-75 fps
After applying foveated rendering this project runs at 90 fps

@BastiaanOlij BastiaanOlij added this to the 4.2 milestone Aug 22, 2023
@BastiaanOlij BastiaanOlij self-assigned this Aug 22, 2023
@dsnopek dsnopek requested a review from a team August 22, 2023 11:34
@BastiaanOlij BastiaanOlij marked this pull request as ready for review August 29, 2023 09:26
@BastiaanOlij BastiaanOlij requested review from a team as code owners August 29, 2023 09:26
@BastiaanOlij BastiaanOlij requested review from m4gr3d and dsnopek August 29, 2023 09:51
@decacis
Copy link
Contributor

decacis commented Sep 1, 2023

Great work! I tried it on my main project—even though I was already hitting 90Hz, I increased the render scale to various values to see how much of an impact foveated rendering can make.

I got good results, when pushing it to the limit, I got around 6-10 extra fps on some scenes. My best results came from forcing the foveation_level to be 3, but that should be expected as is the highest possible (with the "worst" quality).

One thing to note is that when you set foveation_dynamic to true changing the foveation_level won't work because the system handles it for you, and you have to set foveation_dynamic to false to force a foveation level.

Another thing worth mentioning—foveated rendering has its limitations. I found out that the distortion is very noticeable when setting the use_point_size to true in a StandardMaterial3D or the equivalent on a shader material. That property is useful to render point clouds, for example, because it only renders vertices not faces.

The points/dots wiggle a lot, specially on level 3 (the highest) and specially on the top side of the screen and sometimes if they are far away from the player, they can flicker. You can see an example of this in the following video, look on the left side:

foveated_rendering_artifact.mp4

Just something to keep in mind because it is a limitation or side effect of using this feature.

@BastiaanOlij
Copy link
Contributor Author

@decacis I think the point size features is one that just can't be used in combination with Foveated rendering, the further out from the centre you get, the more fragment aren't rendered so if your point happens to affect a fragment not being rendered, it will not be rendered. As it moves it will go in and out of fragments that are rendered and you indeed get obvious flickering and such.

I'm actually surprised you're not already having issues with lens distortion resulting in similar issues where fragments that have the points never end up on screen because of the condensing of the outer parts of the render result.

@BastiaanOlij BastiaanOlij force-pushed the openxr_foveation_ext branch 2 times, most recently from 6de9111 to 7179c6a Compare September 19, 2023 07:19
@BastiaanOlij
Copy link
Contributor Author

Discussed this in the XR meeting, everyone agreed this is something we want. Needs to be rebased, David and Fredia to give it a review before merging.

Copy link
Contributor

@dsnopek dsnopek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just tested this on Quest 2, and it worked great! I setup it up so I could press buttons on the controllers to change the foveation level and whether or not it's dynamic, and so I was able to see the effects along the edges of the viewport with different settings.

I only have a handful of notes regarding the defaults and docs.

main/main.cpp Outdated Show resolved Hide resolved
doc/classes/ProjectSettings.xml Outdated Show resolved Hide resolved
modules/openxr/doc_classes/OpenXRInterface.xml Outdated Show resolved Hide resolved
modules/openxr/extensions/openxr_fb_foveation_extension.h Outdated Show resolved Hide resolved
@BastiaanOlij
Copy link
Contributor Author

Thanks @dsnopek , agreed and made the changes, let me know if you have suggestions to the wording

Copy link
Contributor

@dsnopek dsnopek left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great to me!

@akien-mga akien-mga merged commit 9fdf24f into godotengine:master Sep 25, 2023
@akien-mga
Copy link
Member

Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants