Nothing except for world environment renders to XR camera.

[2-3-5-41]

Godot version

v4.0.beta1.mono.official [4ba934b]

System information

Windows 11

Issue description

While trying out the beta, the use_xr bug seems to be fixed, but now, nothing renders to the camera but the environment.

Steps to reproduce

• Setup basic XR scene

Minimal reproduction project

Prototype Testing Project

[Evanaellio]

I was able to make it work by deleting and recreating the XRRig instance in the root scene. With you testing project, it seems like some children of the XRRig (including the XRCamera) were missing. There was a warning next to the XRRig instance due to this.

Also, the hand tracking doesn't seem to work with the skeleton pose, but it works fine with default, aim or grip poses.

[2-3-5-41]

@Evanaellio the xr camera is missing? that's odd, the project on my computer has the camera and shows no error.

Can confirm; deleting and re-creating the XRRig instance resolved this... odd.

[2-3-5-41]

Scratch that...
The use_xr bug has evolved into the reason for there being nothing rendered but the world environment. When you don't run the script for the XRRig to use the xr viewport, it will render the scene(s) just fine, but with the issue of not having an xr viewport.

[2-3-5-41]

This issue persists into Godot version: v4.0.beta2.official [f8745f2]

For me, when use_xr is set to true, godot renders to the headset, but only the world environment; if use_xr is set to false, the full scene renders correctly, but doesn't render to the headset.

I'm testing this on the Valve Index.

[BastiaanOlij]

Tried out the test project, also on a Valve index. Runs for me. The camera does start on the floor so when the headset isn't tracking yet you're "inside" the floor and it won't render. Also the floor has a Y extends of 1 meter so it will be waist high, if you're seated it's possible you might still be inside of the floor when tracking, move it down by a meter and it will be flush with the ground.

@Evanaellio is also correct about using skeleton. The skeleton pose is placed at the base of the hand if hand tracking is used.
For controllers you should use either aim (front of the controller pointering forward) or grip pose (where your hand touches the controller).

[2-3-5-41]

I don't know what happened to the old test project, but it's corrupt or something because of what I'm reading compared to what it was when I uploaded it, I've made a fresh project that's just as basic. Also, I do not track inside the floor, unless tracking space is set to 'local' in the OpenXR reference space.

Here is a video showing what I am experiencing.
Video link

And if you want the current 'new' test project I've just spun up to see if the discrepancies persist.
XR Test.zip

[2-3-5-41]

Is there anything else I could try to provide to help figure out what is causing this?

[BastiaanOlij]

Just tried out your new test project, I was indeed using the original one in the OP.

I had to turn use_xr on but after that it is working fine for me as you can see:

Also using an Index.

That said, you scene does still have the same problem as before.
Your floor mesh has this extends:

But its positioned at the origin:

Your player is also at the origin:

This means that your player is "sunk into the floor" by 1 meter. When tracking hasn't started yet your camera will be inside of the floor. Once tracking starts it depends on how tall you are and whether you are seated.

Remember, out of the box the XR system just tracks, it doesn't adhere to collision shapes or anything like that, it won't magically put your player on the top of the floor.

That all said, it doesn't fully explain your video, especially with those pillars that would be visible even if your camera ends up inside of the floor so I am scratching my head on whats going on.

[2-3-5-41]

I'm willing to brainstorm on how to troubleshoot this, I really want to give godot an honest shot with something I actually like.

[BastiaanOlij]

I'm willing to brainstorm on how to troubleshoot this, I really want to give godot an honest shot with something I actually like.

If you aren't there already the best advise here is to join the Godot Discord server (see https://godotengine.org/community) and come to the XR channel. Loads of people there, and I myself try and spend an hour or so each day there as well to answer questions.

This way it's faster to exchange ideas and see what the issue you are having is. It's really weird seeing I can't seem to reproduce it with your test project even though I'm using the same headset. Maybe others there can give it a try too and see what we're missing.

Have you tried this demo to see if the same thing happens? https://github.com/BastiaanOlij/godot4_openxr_demo

[2-3-5-41]

I just ran your demo and everything is working as expected...

[2-3-5-41]

Current troubleshooting attempts:

These are attempts done in my provided project.
"Same Result" refers to the world not rendering to the xr camera.

• Changing 'Root' node from Node to Node3D...............................Same Result
• Changing from instanced scenes, to all in one scene..................Same result
• Changing from SteamVR beta, to SteamVR stable........................Same result

Logs from most recent attempt:

OpenXR: Running on OpenXR runtime:  SteamVR/OpenXR   0.1.0
Godot Engine v4.0.beta2.official.f8745f2f7 - https://godotengine.org
OpenXR: XrGraphicsRequirementsVulkan2KHR:
 - minApiVersionSupported:  1.0.0
 - maxApiVersionSupported:  1.2.0
Vulkan API 1.2.0 - Using Vulkan Device #0: AMD - AMD Radeon RX 6800 XT
 
OpenXR: Found supported reference space  XR_REFERENCE_SPACE_TYPE_VIEW
OpenXR: Found supported reference space  XR_REFERENCE_SPACE_TYPE_LOCAL
OpenXR: Found supported reference space  XR_REFERENCE_SPACE_TYPE_STAGE
OpenXR: Found supported swapchain format  VK_FORMAT_R8G8B8A8_SRGB
OpenXR: Found supported swapchain format  VK_FORMAT_B8G8R8A8_SRGB
OpenXR: Found supported swapchain format  VK_FORMAT_R32G32B32A32_SFLOAT
OpenXR: Found supported swapchain format  VK_FORMAT_R32G32B32_SFLOAT
OpenXR: Found supported swapchain format  VK_FORMAT_R16G16B16A16_SFLOAT
OpenXR: Found supported swapchain format  VK_FORMAT_D32_SFLOAT
OpenXR: Found supported swapchain format  VK_FORMAT_D16_UNORM
OpenXR: Found supported swapchain format  VK_FORMAT_D24_UNORM_S8_UINT
OpenXR: Found supported swapchain format  VK_FORMAT_D32_SFLOAT_S8_UINT
OpenXR initialised successfully
Using swap chain format: VK_FORMAT_R8G8B8A8_SRGB

Odd behaviour(s)

Just from looking between the two projects (mine and the one provided by Bastiaan), the logs are identical but the project provided by Bastiaan is somehow superior since it's the only godot xr project on my computer that is rendering the world correctly.

Personal conclusion

I'm just gonna wait for beta 3, it seems like some weird and isolated event that my computer is really good at manufacturing.

[2-3-5-41]

Issue persists into beta 3, but with a new quirk.

Screeshot

Showing performance graph, vr view, and godot's remote and local view.

Logs

OpenXR: Running on OpenXR runtime:  SteamVR/OpenXR   0.1.0
Godot Engine v4.0.beta3.mono.official.01ae26d31 - https://godotengine.org
OpenXR: XrGraphicsRequirementsVulkan2KHR:
 - minApiVersionSupported:  1.0.0
 - maxApiVersionSupported:  1.2.0
Vulkan API 1.2.0 - Using Vulkan Device #0: AMD - AMD Radeon RX 6800 XT
 
OpenXR initialised successfully
USER ERROR: Index p_mipmap = 0 is out of bounds (src_texture->mipmaps = 0).
   at: texture_create_shared_from_slice (drivers/vulkan/rendering_device_vulkan.cpp:2290)
USER ERROR: Index p_mipmap = 0 is out of bounds (src_texture->mipmaps = 0).
   at: texture_create_shared_from_slice (drivers/vulkan/rendering_device_vulkan.cpp:2290)

Additional notes

The XR example project provided by Bastiaan still works just fine, but the desktop window has the same artifacts as the one shown in the screenshot.

[2-3-5-41]

Issue persists into v4.0.beta4.mono.official [e675154], including desktop viewport looking corrupted, and with the nasty performance graph. Not enabling the xr viewport renders the world just fine, but with terrible performance.

Logs

No change, looks normal.

OpenXR: Running on OpenXR runtime:  SteamVR/OpenXR   0.1.0
Godot Engine v4.0.beta4.mono.official.e6751549c - https://godotengine.org
OpenXR: XrGraphicsRequirementsVulkan2KHR:
 - minApiVersionSupported:  1.0.0
 - maxApiVersionSupported:  1.2.0
Vulkan API 1.2.0 - Using Vulkan Device #0: AMD - AMD Radeon RX 6800 XT
 
OpenXR initialised successfully

[2-3-5-41]

Thanks to the user 'DETOX#0236' on discord, he asked me to try using the 'clear color' background mode instead of skybox, and that worked!

[linkpy]

Same issue for me on 4b7, on an AMD Radeon RX 6950 XT and using SteamVR

The projects i create, i can't see anything except the sky.
The demo project works, but i have exactly the same setup in my project.
1SMOL's porject doesn't work tho.

I tried following the tutorial on the godot docs website and Bastiaan Olij's youtube tutorial to the letter but the issue persists.

Project I made from scratch that doesn't work : Vr Proj.zip

I'm really confused regarding what's happening or what I did wrong for it to not display anything.

[2-3-5-41]

Here are some updates from the Godot discord:

• Using MSAA (at any level) will work around the issue.

But, this may introduce other graphical anomalies in XR. (From my experience while testing this workaround)

• It's assumed that currently, OpenXR rendering is not setting up depth testing correctly.

Incorrect depth testing is visible while MSAA is disabled, and the background color is set to clear color.

[linkpy]

1SMOL, you are my savior.
Enabling MSAA indeed works. Didn't see any graphical issues for now but I can finally start working on something.

[Cyberrebell]

I did some testing on this issue using the latest beta (16) and the latest master (68299e0).
The issue still occurs with my AMD RX 6600 and latest Adrenalin (AMDVLK) driver on Windows 10. Even enabling MSAA and adding a Clear Color background WorldEnvironment doesn't help.
The issue happens with Valve Index + SteamVR (no matter if I use 80Hz, 90Hz, 120Hz). It also happens with a Quest 2 no matter if I use SteamVR or Oculus as OpenXR server.

If I move my head fast I can see parts of the mesh flicker into the picture so it really seem to be a render order issue.

If I switch from Forward+ to Mobile or Compatibility it works fine. Also the godot4_openxr_demo doesn't render anything visible as soon as I switch it to Forward+.
use_xr = false also renders the mesh fine in a non-VR window using Forward+.

Using the same AMD system on Arch Linux (RADV) rendering works fine with Forward+ even without the MSAA+ClearColor workaround. Another system with an Nvidia GTX 1080, Win10, Quest 2 also works fine without any workaround.

Here is my minimal project (just an XRCamera, a mesh and the workarounds):
vrtest.zip

Here are the errors I get when it's not working using MSAA (the same errors get spammed over and over again while the project is running):

E 0:00:00:0946   _render_pass_create: Invalid framebuffer format attachment(1), in pass (0), if an attachment is marked as multisample, all of them should be multisample and use the same number of samples.
  <C++ Error>    Condition "texture_samples != p_attachments[attachment].samples" is true. Returning: nullptr
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:3738 @ _render_pass_create()
E 0:00:00:0946   _allocate_from_data: Condition "rid.is_null()" is true. Returning: rid
  <C++ Source>   ./servers/rendering/renderer_rd/effects/../storage_rd/../framebuffer_cache_rd.h:170 @ _allocate_from_data()
E 0:00:00:0946   draw_list_begin: Condition "!framebuffer" is true. Returning: INVALID_ID
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:6858 @ draw_list_begin()
E 0:00:00:0946   framebuffer_get_format: Condition "!framebuffer" is true. Returning: INVALID_ID
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:4235 @ framebuffer_get_format()
E 0:00:00:0946   framebuffer_format_get_texture_samples: Condition "!E" is true. Returning: TEXTURE_SAMPLES_1
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:4112 @ framebuffer_format_get_texture_samples()
E 0:00:00:0947   render_pipeline_create: Mismatch fragment shader output mask (1) and framebuffer color output mask (0) when binding both in render pipeline.
  <C++ Error>    Condition "shader->fragment_output_mask != output_mask" is true. Returning: RID()
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:6002 @ render_pipeline_create()
E 0:00:00:0947   _generate_version: Condition "pipeline.is_null()" is true. Returning: RID()
  <C++ Source>   servers/rendering/renderer_rd/pipeline_cache_rd.cpp:61 @ _generate_version()
E 0:00:00:0947   draw_list_bind_render_pipeline: Condition "!dl" is true.
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:7106 @ draw_list_bind_render_pipeline()
E 0:00:00:0947   draw_list_bind_uniform_set: Condition "!dl" is true.
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:7180 @ draw_list_bind_uniform_set()
E 0:00:00:0947   draw_list_bind_uniform_set: Condition "!dl" is true.
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:7180 @ draw_list_bind_uniform_set()
E 0:00:00:0947   draw_list_bind_uniform_set: Condition "!dl" is true.
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:7180 @ draw_list_bind_uniform_set()
E 0:00:00:0947   draw_list_bind_uniform_set: Condition "!dl" is true.
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:7180 @ draw_list_bind_uniform_set()
E 0:00:00:0947   draw_list_bind_index_array: Condition "!dl" is true.
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:7253 @ draw_list_bind_index_array()
E 0:00:00:0947   draw_list_set_push_constant: Condition "!dl" is true.
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:7287 @ draw_list_set_push_constant()
E 0:00:00:0947   draw_list_draw: Condition "!dl" is true.
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:7305 @ draw_list_draw()
E 0:00:00:0947   draw_list_end: Immediate draw list is already inactive.
  <C++ Error>    Condition "!draw_list" is true.
  <C++ Source>   drivers/vulkan/rendering_device_vulkan.cpp:7608 @ draw_list_end()

*edit: The error messages only occur when enabling MSAA
The issue persists with Beta 17.

[2-3-5-41]

Issue persists into v4.0.rc1.official [8843d9a].

XR rendering in 'forward+' is a no go, regardless of all the available 'work arounds'.

XR rendering in 'mobile' or 'compatibility' works if:

• MSAA is enabled, or
• MSAA is disabled, and "submit depth buffer" in the openxr settings is disabled

This was tested on Windows 11 with a Radeon RX 6800 XT.

[Cyberrebell]

for me the AMD Adrenalin 23.2.2 update fixed this issue

[2-3-5-41]

Indeed, the latest stable release of AMD drivers resolves this issue.

[clayjohn]

Thanks for updating! I'll go ahead and close this issue now.

[DKesserich]

I've been able to reproduce this in 4.0.2 with AMD Driver version 23.3.2 when Submit Depth Buffer is enabled, MSAA disabled, and using Sky instead of Clear Color.

[Cyberrebell]

I've been able to reproduce this in 4.0.2 with AMD Driver version 23.3.2 when Submit Depth Buffer is enabled, MSAA disabled, and using Sky instead of Clear Color.

Does it work if you disable "Submit Depth Buffer"? I don't think this option works at the moment.

[benjarmstrong]

It looks like the latest AMD Pro drivers can cause this as well; I reproduced in 4.1.1-stable with the following conditions:

• Windows 10 64 bit
• SteamVR/OpenXR 1.26.7
• AMD RX 6900 XT
• AMD Pro 22.Q4 Driver

Behaviour:

• "Submit depth buffer" enabled:
  • forward+ renderer: black screen only
  • mobile renderer: skybox only
• "Submit depth buffer" disabled:
  • forward+ renderer: skybox only
  • mobile renderer: renders normally

Issue is resolved after installing AMD Driver Adrenilin Edition 23.7.2 (although enabling "submit depth buffer" still exhibits the same behaviour as listed above)

tldr:

1. disable "submit depth buffer"
2. make sure you use the AMD Adrenilin drivers (not the AMD Pro drivers)and make sure they're up to date.