In-Game: Add support for accessing a surface's depth buffer

[iampremo]

Description

Create functions that would allow developers to gain access to the internal depth buffer texture of surfaces so that it could be referenced in deferred shaders. This would allow the already existing depth information to be used in a plethora of standard shader effects, such as screen space ambient occlusion (SSAO), screen space reflections (SSR), depth of field (DOF), shadow mapping, and several others.

Benefit

Currently in GMS2, the only way to effectively create these types of effects is to either use forward rendering and do all the processing on the pixels upfront (which isn't ideal in many situations), or use multiple channels in a custom surface to store the data again, wasting draw calls and GPU time just so the depth information can be accessed. This is a cumbersome, inefficient, non standard approach could be completely rectified by simply allowing access to the built in depth buffer that surfaces have.

This kind of change would probably also require the ability to change surface data types and precision, otherwise specific depth buffer surface functions would be needed.

Justification

I recommend expanding the surface structure to accommodate these changes as this will allow myself and other GMS2 developers unparalleled access to a plethora of common graphics techniques and not have to use hacky, confusing, and difficult work around methods to achieve them.

Other very popular game engines already have this functionality and GMS2 is sorely lacking.

Thank you for your time.

[FoxyOfJungle]

I second this! +1

I would like to suggest that it is possible to get the depth information in the form of a surface, something like surface_get_zbuffer or surface_get_depth_buffer

This will be particularly useful for a lot of things besides the ones mentioned, like creating lighting systems where the lights have depth, fog effects, color grading in specific areas, etc.

[Gamer-XP]

Talking about depth buffer, there are cases where I want to draw on some extra surface, but keep my main buffer for it when drawing. So, I want to be able to share/copy-paste depth buffer around between different surfaces.

Edit: I did deal with it in GMS1 by using Multiple Render Targets shader btw. But it's Windows-only.

[hagand3]

It'd be nice to be able to copy a surface to another along with its depth buffer. Not sure if it's possible to interpolate from surface to surface but copying to a larger buffer while retaining depth buffer info would be ideal.

[Gamer-XP]

While at it, can we have a stencil buffer as well? Is useful for more tricky stuff where you need both z-buffer and some way to mask by already drawn pixels.

[FoxyOfJungle]

Yep, having access to the stencil buffer would be awesome!

[mistletoe]

If we're going to have this, then it'd be nice to have an optional argument for the standard code used to draw Sprites that sets the depth, rather than using the Instance Depth.

[pkraif]

If we're going to have this, then it'd be nice to have an optional argument for the standard code used to draw Sprites that sets the depth, rather than using the Instance Depth.

That is not quite related to this issue. What you want is also already possible with function gpu_set_depth :)

[kraifpatrik]

Implemented and merged for the 2024.6 release.

Changes

Depth buffer functions

• Added new function surface_has_depth(surface), which retrieves whether given surface has a depth buffer (depth buffers can be disabled with existing function surface_depth_disable).
• Added new function surface_get_texture_depth(surface), which retrieves a pointer to a texture of a surface's depth buffer. This pointer can be used for example in functions vertex_submit and texture_set_stage to sample the depth texture in a shader.

Following code can be used to convert depth buffer values to range 0..1 in shaders. Please note that this is not required when using an orthographic projection.

precision highp float; // Might be required on some platforms!

/// @param depth Non-linear depth.
/// @param zparam Equals (zfar / znear).
/// @return Linearized depth, in range 0..1.
float LinearizeDepth(float depth, float zparam)
{
#if !defined(_YY_HLSL11_)
    depth = depth * 2.0 - 1.0;
#endif
    return 1.0 / ((1.0 - zparam) * depth + zparam);
}

• Function surface_set_target now takes an optional "depth_id" parameter (becoming surface_set_target(surface_id, [depth_id])), which can be used to specify a surface whose depth buffer should be used as the current one. When the argument is not specified, it defaults to "surface_id".
• Added new function surface_get_target_depth(), which retrieves the surface whose depth buffer is currently used.

Stencil functions

• Added new constant type Constant.StencilOp and following constants used to configure stencil test operations:
  • stencilop_keep - Keeps the current value in the stencil buffer.
  • stencilop_zero - Sets the stencil buffer value to 0.
  • stencilop_replace - Sets the stencil buffer value to the stencil reference value.
  • stencilop_incr - Increments the stencil buffer value, clamping at the maximum value.
  • stencilop_incr_wrap - Increments the stencil buffer value, wrapping to 0 at the maximum value.
  • stencilop_decr - Decrements the stencil buffer value, clamping at 0.
  • stencilop_decr_wrap - Decrements the stencil buffer value, wrapping to the maximum value at 0.
  • stencilop_invert - Performs a bitwise inversion on the current stencil buffer value.
• Function gpu_get_state was modified to also return following keys:
  • "stencilenable" - true if stencil test is enabled, otherwise false.
  • "stencilfunc" - The current stencil test function (Constant.ZFunction).
  • "stencilref" - The current stencil reference value (Real).
  • "stencilreadmask" - The current stencil read mask (Real).
  • "stencilwritemask" - The current stencil write mask (Real).
  • "stencilfail" - The stencil operation executed when the stencil test fails (Constant.StencilOp).
  • "stencilzfail" - The stencil operation executed when the stencil test passes but the depth test fails (Constant.StencilOp).
  • "stencilpass" - The stencil operation executed when both stencil and depth test pass (Constant.StencilOp).
• Function gpu_set_state now accepts new keys from gpu_get_state.
• Added new function gpu_set_stencil_enable(enable), which can be used to enable/disable stencil test.
• Added new function gpu_get_stencil_enable(), which retrieves whether stencil test is enabled.
• Added new function gpu_set_stencil_func(cmp_func), which sets the stencil test function. Argument cmp_func must be one of the existing Constant.ZFunction constants.
• Added new function gpu_get_stencil_func(), which retrieves the current stencil test function.
• Added new function gpu_set_stencil_ref(ref), which can be used to configure the stencil ref value.
• Added new function gpu_get_stencil_ref(), which retrieves the current stencil ref value.
• Added new function gpu_set_stencil_read_mask(mask), which can be used to configure the stencil read mask.
• Added new function gpu_get_stencil_read_mask(), which retrieves the current stencil read mask.
• Added new function gpu_set_stencil_write_mask(mask), which can be used to configure the stencil write mask.
• Added new function gpu_get_stencil_write_mask(), which retrieves the current stencil write mask.
• Added new function gpu_set_stencil_fail(stencil_op), which can be used to configure the stencil operation taken when the stencil test fails.
• Added new function gpu_get_stencil_fail(), which retrieves the current stencil operation taken when the stencil test fails.
• Added new function gpu_set_stencil_depth_fail(stencil_op), which can be used to configure the stencil operation taken when the stencil test passes but depth test fails.
• Added new function gpu_get_stencil_depth_fail(), which retrieves the current stencil operation taken when the stencil test passes but depth test fails.
• Added new function gpu_set_stencil_pass(stencil_op), which can be used to configure the stencil operation taken when both stencil and depth test pass.
• Added new function gpu_get_stencil_pass(), which retrieves the current stencil operation taken when both stencil and depth test pass.

Depth and stencil clearing functions

• Added new function draw_clear_depth(depth), which can be used to clear just the depth buffer to given value, leaving the color and stencil unaffected.
• Added new function draw_clear_stencil(stencil), which can be used to clear just the stencil buffer to given value, leaving the color and depth unaffected.
• Added new function draw_clear_ext([col], [alpha], [depth], [stencil]), which can be used to clear color, depth or stencil, based on which arguments are provided.

Buffer functions

• Added new function buffer_get_surface_depth(buffer, surface, offset), which can be used to copy values from a surface's depth buffer into a regular buffer (similarly to buffer_get_surface, which copies color).
  • Not supported on platforms running OpenGL ES/WebGL!
  • Not supported with MSAA enabled!
• Added new function buffer_set_surface_depth(buffer, surface, offset), which can be used to copy values from a regular buffer into a surface's depth buffer (similarly to buffer_set_surface, which copies color).
  • Not supported on platforms running OpenGL ES/WebGL!
  • Not supported with MSAA enabled!

Test project

Following is a project for testing whether all functions are working correctly: DepthStencilTest.zip. Since most of the tests are visual, I'm also attaching screenshots of what the results are supposed to look like:

Test 5 on platforms running OpenGL ES/WebGL (GX.games, HTML5, Android, iOS, Switch):

Test 5 on others:

Showcase project

I yet have to clean this one up. I will post it here when it's ready 🙏

[cameron-home]

Verified working as expected in Beta IDE v2024.600.0.562 Runtime v2024.600.0.580. Will move to Verified swim lane once docs are done.

[YYBartT]

Added the depth & stencil buffer documentation to a new page under the "Drawing" section: "The Depth and Stencil Buffer". Updated existing function pages and added reference pages for all new functions, some of them including complete code examples. Various updates and improvements to related pages. Also documented the contents of the DS map used by gpu_get_state() and gpu_set_state(). All changes are currently in the develop.bart branch.