Vulkan: Spotlights parented to XRControllers have a blocky look

[coobenguy]

Godot version

4.0 Alpha 7

System information

Windows 10, gtx 1080, driver 512.59, Vulkan Clustered + Mobile. also tested on Windows 11, rtx 2070 super, driver 512.59

Issue description

while using a spotlight parented to an XRController, the light will have large(depending on the angle of the spotlight) blocky edges.

Steps to reproduce

Parent A spotlight to an XRController

Minimal reproduction project

SpotlightXR.zip

[Calinou]

4.0

4.0 stable isn't released yet. Which alpha are you using (or in the case of a custom build, which commit)?

Also, the minimal reproduction project is using the Vulkan Clustered backend. Can you reproduce this after switching to the Vulkan Mobile backend in the Project Settings?

[coobenguy]

4.0 alpha 7. and yes it happens in both Clustered and Mobile.

[BastiaanOlij]

There is nothing XR does different when it comes to lights. The only thing I can think off is that seeing the light is moving, it may have to do with the spotlights shadow buffer not getting updated because its flagged as static?

[coobenguy]

Im not sure. After more testing it seems that the issue is even more general than I thought before. The spotlight doesnt even need to be parented to anything for this to occur. I also dont think it has to do with a shadow buffer since it happens even with shadow turned off on the light (although I dont know the inner workings of these things so the shadow buffer could still be being used)

[Calinou]

@coobenguy Can you record a video of the problem occurring and link it here?

[coobenguy]

https://streamable.com/9yshra

the left half of the light shows in the left eye and the right half shows in the right eye and the 2 halves overlap

[Calinou]

It's strange that you can reproduce this when using the Vulkan Mobile backend, as this looks a lot like a clustering artifact.

[TOMDM]

https://streamable.com/9yshra

the left half of the light shows in the left eye and the right half shows in the right eye and the 2 halves overlap

I had this exact issue the other day while messing with different environment configurations in alpha 7, however my light was an OmniLight3D that was parented to the root scene.

It stopped at some point though and I haven't been able to recreate it.

[coobenguy]

huh I was never able to do it with an omniLight but it happens no matter what I do with the spotlight

[python273]

having the same problem with not moving decals/lights. I couldn't reproduce it on mobile vulkan, but it has other bugs. Here is discord discussion: https://discord.com/channels/212250894228652034/418572953912082432/993165136107941969

[luzader]

I'm seeing this on any spotlight if XR is enabled and for most sane values of Angle Attenuation.

• The artifact is on the left side in the right eye and right side in the left eye.
• The visibility of the artifact is far worse if Angle Attenuation the effect is low, e.g. 0.01 or less.
• The artifact is invisible if the Angle Attenuation is very high, e.g. 19. The super soft edges likely fade to 0 before this whatever this is blocky edge is hit.

OpenXR: Running on OpenXR runtime: SteamVR/OpenXR 0.1.0 Godot Engine v4.0.alpha14.official.106b68050 - https://godotengine.org OpenXR: XrGraphicsRequirementsVulkan2KHR:

• minApiVersionSupported: 1.0.0
• maxApiVersionSupported: 1.2.0 Vulkan API 1.2.0 - Using Vulkan Device #0: NVIDIA - NVIDIA GeForce RTX 3080

And actually it is possible to see a similar effect even without XR enabled, but only if the Angle Attenuation is set to an absurdly low value, such as 0.0001:

[luzader]

I was hoping this had been indirectly fixed by https://github.com/godotengine/godot/pull/71832 (since the symptoms seemed very similar) but unfortunately, the problem in the first screenshot https://github.com/godotengine/godot/issues/60680#issuecomment-1214224333 is still present in beta 15 and beta 16.

[leandrodreamer]

Still happening with Godot 4.0.2

https://user-images.githubusercontent.com/26260718/233758197-e44c57af-ad04-4f89-91b7-62ed34458d9c.mp4

[DKesserich]

I'm also seeing this. It looks almost like the direct light from spot and omni lights is not being rendered in the area where the views for the left and right eye overlap. If the radius/angle is set low enough it actually becomes impossible to see the lights when they're being looked at straight on.

It becomes more obvious what's going on if you take a shot from the VR View window set to show both eyes than the Godot screen mirror.

(You can also see in this screenshot a red spotlight in the right eye that SHOULD be visible in the left, but isn't)

[luzader]

So the "Forward+"/forward_clustered renderer is way above my level, but I've inferred that the "clustered" part is crudely this: To improve performance, it "plans" which lights (among other things) need to be processed for each square of the display.

To work properly, the planned spotlight footprint (in screen-space) needs to cover at-least all the squares where the scene shader is actually going to draw the light. This is all well and good in monocular mode. In XR/multiview this doesn't work right, because the clustering is unaware of multiview, so it plans the cone as though the camera were right between your eyes. This leads to the right side of the left eye and the left side of the right eye having squares where the spotlight didn't get rendered (because the clustering thought the cone would not be in those clusters).

I don't know how to make the clustering multiview-aware. But those willing to build their own copy of Godot can misuse the following threshold as a workaround:

https://github.com/godotengine/godot/blob/809a98216267f3066b9fec2f02b2042bdc9d3e0d/servers/rendering/renderer_rd/cluster_builder_rd.h#L134

This was intended to deal with really-broad spotlights which are just handled as though they are spheres. If this threshold is changed to 0.0, all spotlights will be treated as spheres. This will cause way more clusters to process each spotlight, but the resulting sphere will generally cover where the spotlight lands no matter which eye you're looking out of. This is an ugly hack since it will make the spotlight processing wasteful (especially if you have many narrow spotlights side-by-side), but at least you won't get artifacts.

If you wanted to be slightly less hacky you could add || is_multiview_enabled in the following conditional (you'd need to determine is_multiview_enabled from somewhere that isn't in scope). This would only be better in that the non-XR performance would be unaffected: https://github.com/godotengine/godot/blob/809a98216267f3066b9fec2f02b2042bdc9d3e0d/servers/rendering/renderer_rd/cluster_builder_rd.h#L310

[BastiaanOlij]

@luzader you are very close, indeed in the forward+ renderer builds a map that determines which lights effect a given tile (square) but does this only from a single perspective. In our stereo rendering case that is indeed from between the eyes, but we do this against a view frustum that encompasses the frustum of both eyes.

As this is now sampled in "screen resolution", as a result when rendering each individual eye, it is checking the wrong tile. #78499 resolves that issue by performing an extra projection on this combined frustum to get the right tile and should resolve the OPs problem.

Note that the issue with setting attenuation really low remains, as you correctly state this also happens in the editor and in mono rendering and is caused by the light volume evaluated by the cluster logic being too small resulting in that blocky outline around the spotlight. Disabling the threshold on multiview isn't the right solution though, we need to look into correcting the light volume. We should probably raise this as a separate issue.