Optimize Occlusion culling jitter

[Flarkk]

This PR ensures the jittering is applied in a uniform manner across the whole viewport.
Previously, pixels closer to the center with farther Z used to get less jitter than those on the edge, as a side effect of perspective division.

This is achieved by jittering the near plane corners instead of the projection matrix, as only those are ultimately passed to embree in the end anyway.

Test project : occlusion_jitter.zip

This improvement supersedes the fix in #99941.

Before	After

Edit : Resolves #53288

[Flarkk]

@lawnjelly this is the improvement we touched base earlier on.

[lawnjelly]

So this is effectively jittering the translation rather than the camera rotation, as discussed in #99941 (comment) ?

I did think of a snag with translation though, so sorry I may have led you on a wild goose chase here ... if my thinking is correct, rotation does have the advantage that it automatically compensates for distance:

The effective gap which will cause the problem described in #53288 does depend on distance, because it's the screen space distance that determines whether two adjacent walls will "merge" in the occlusion buffer. Rotation of the camera means the screen space change of jittered objects will be the same with distance I think which is what we want (the absolute jitter in view space will increase).

Translation will cause the opposite effect if I'm correct (with perspective projection):

• Objects close to the camera will be shifted more than objects far away, rather than less

I suspect we need to make sure the test case works correctly at different distances.

If this is correct, then rotation may be better, even if it produced the undesirable effect that there is more jitter in the centre than the edges, due to perspective. I guess in that case we just calibrate to the centre, and live with slightly less occlusion than ideal at the edges.

Another alternative as I suggested in the original PR is to jitter each ray individually. Even with uniform jitter, you could adjust the magnitude to compensate for the edge effect. Although jittering or adjusting per pixel may be overkill, and an unnecessary expense on CPU for some possible benefit that may not be realised.

UPDATE:
This could be why the box appears to be jittering less than the sphere in your example after the PR. I've just checked and the box is further from the camera than the sphere.

[Flarkk]

So this is the opposite : this PR jitters more like a rotation, while it's a translation right now.

You're right that translating moves closer objects more.
This is precisely why the sphere jitters more on the left : due to the spherical shape and the perspective, pixels on its left side have a closer Z than those on its right side.
The box has instead a uniform jitter because it's flat and laying in the XY plane.

The current implementation (with my last PR merged) translates the whole scene relatively to the rays (equivalently the rays relatively to the scene) in clip space.

What this PR does is actually the alternative you mention above.
It translates only the tips of the rays on the near plane (their origin - the camera - stays fixed).
This is not exactly a rotation but acts more like it, with the additional benefit of allowing pixel perfect jittering.
Of course the implementation doesn't jitter each ray tip one by one, but instead the near plane bottom-left corner. As the code then interpolates from this corner across X and Y across the whole near plane to determine the rays' tips, this effectively jitters each ray.

Will take a look at #53288 and see if this PR solves the problem.

[Flarkk]

Rebased on top of #99941

[lawnjelly]

So this is the opposite : this PR jitters more like a rotation, while it's a translation right now.

If this is the case, this should indeed be an improvement. Yup some decent testing over the distance range will be good.

If I remember right, ideally we should like to see objects jitter by around a pixel (just under in fact) no matter the distance from the camera.

[Flarkk]

I updated the PR to raise the jitter magnitude to 0.99 pixel.

With these settings I can confirm that :

• jitter is 1 pixel wide no matter the occluder's Z position
• no occluder does not jitter (when the amplitude is set too low below 1, it can happen that an occluder's edge is jittered within the same pixel)

occlusion_jitter_2.zip

The more to the right, the farther :

Scene top view	Result

I can also confirm that this seems to greatly improve the situation described in #53288.
The gap causing the red box to be culled is now much smaller and coincides with the moment jitter fills the gap over all 9 frames.

Not culled	Culled

[lawnjelly]

That looks great.

From memory I suspect the ideal value (less than 1) for the current pattern may be 0.66, so this is worth testing too.

The jitter pattern I think is one in the centre, a medium square and a distance square (9 in total). At 1, I suspect it would be mimicking the situation where the whole scene was shifted to the left by a whole pixel, so I'm not absolutely sure we gain as much in terms of resolution.

Effectively by jittering we are aiming to increase the resolution of the occlusion buffer, in this case by 9x (with 3 x the resolution on each axis), therefore the fractions should be 0.0 (centre), 0.333 (mid square), 0.666 (far square). The multiplier we are changing here is for the far square as the mid square is multiplied by 0.5.

Essentially the way I was thinking is that by regularizing this, we are maximizing the chance of a gap occurring in the occlusion buffer for these gaps.

The alternative of a more random pattern is potentially even better, but suffers from difficulty in repeatability for the timeout (of course you could use a fixed random pattern as an alternative).

We can increase the resolution higher with more samples, but that increases the timeout required for stability and makes it more liable to pop in.

A priori I'm not absolutely sure what the best value would be here as we are in uncharted territory - I'm not sure I've seen anyone else using this technique with raster occlusion culling, so it's just a combination of some theory with some empirical testing. We could equally well say that say, 0.5 would deal with even smaller gaps (at the risk of missing some larger ones maybe).

[Flarkk]

With 0.66.
Looks like the gap is even smaller.
Will change the PR right away.

Not culled	Culled

[Flarkk]

Done. You can review now.
Shall we consider this closes #53288 ?
I'm unsure we will achieve anything better with a ray tracing approach.

[lawnjelly]

Sounds good.

I suppose if the occlusion buffer was a certain fixed integer multiple proportion of the screen resolution it should be possible to remove the problem entirely by this super sampling, but the choice of occlusion buffer size is currently pretty complex to do with how many cores. I think we'd need there to be people reporting problems for us to do things like start altering the occlusion buffer sizes.

e.g. at 1920x1080 you would use e.g. 480x270, and use 3 extra temporal jittered samples on each axis.

Might be worth doing on e.g. a console, but probably not for general use at the moment.

[lawnjelly]

I'm not super familiar with the 4.x release process now, but will bump to 4.4 and add to 4.4 merge queue as this should be a good addition.

[Repiteo]

Thanks!