Shadows disappearing on view angle and distance

[VianPatel]

Describe the bug In a large scene (where the largest distance between objects is around 5e6 to 2e7 units) shadows disappear and reappear depending on the view angle and distance to the camera. In medium sized scenes shadows also disappear, except only at a smaller angle and distance. Additionally for medium sized scenes when the view angle is changed the resolution of shadows visibly decreases.

Shadow cascades were able to fix some of the resolution issues (not the shadow disappearing issue), however with large scenes there seems to be an issue where cascades are not transitioning smoothly and jumping between levels.

Additionally on large scenes there is an issue where an uninitialized texture is bound to 'light_shadowMap', causing artifacts to appear on the scene.

If these issues are not the same, I can repost them seperately.

To Reproduce Steps to reproduce the behavior:

1. Download the hammer and spaceship: Resources.zip
2. Place two spaceships, and a hammer at (0, 0, 3), (0, 0, 20000000), (0, 1, -8) respectively.
3. Move the camera to (0, 2, -9) and look down on the hammer

Expected behavior When looking down on the hammer, a shadow should appear on the spaceship.

Screenshots Shadows disappearing: https://user-images.githubusercontent.com/69381882/201444048-4d6a737d-a2a4-4779-ac6c-388bef01a908.mp4

Rough shadow cascade transition: https://user-images.githubusercontent.com/69381882/201444571-bcb5bc34-6b92-4eb7-a7f5-746d79975d0d.mp4

Uninitialized texture:

Logs Uninitialized texture:

Uninitialized texture bound to 'light_shadowMap' in material 'base_lit_opaque' at binding point 0
Uninitialized texture bound to 'light_shadowMap' in material 'Skybox' at binding point 0

Desktop:

• OS: Windows 11 22H2
• GPU: Intel UHD Graphics G1
• Backend: Vulkan
• Filament Version: 1.28.3

Additional context [AR shadows weird · Discussion #5740 · google/filament · GitHub](https://github.com/google/filament/discussions/5740) is likely the same issue, however it fails to provide additional information.

[Camera pan clips shadows? · Issue #3552 · google/filament · GitHub](https://github.com/google/filament/issues/3552) appears to be the same issue, however it was caused by MaterialInstance::setScissor which is not called anywhere in this application.

For my specific use case I am trying to render planetary scale scenes where small objects (trees, spaceships, etc.) and large objects (planets, asteroids, etc.) can cast and receive shadows. I am using a 64 bit fixed-point data type with 3 fractional digits and camera-relative rendering (all entities and lights are transformed each frame) for uniform millimeter precision across the entire scene. However, this issue persists even when this scene is rendered normally without these modifications.

Application specific configuration:

• far plane of 10000
• near plane of 0.1
• using 4 shadow cascades
  • splits are computed with filament::LightManager::ShadowCascades::computeLogSplits
• not using stable shadows
• using a shadow map size of 1024
• using PCF shadows
• camera focal length of 22.0
• aperture of 16.0
• shutter speed of 1/125.0
• sensitivity of 100.0

[pixelflinger]

One comment I have is that near of 0.1 and far of 10000 seems like way too much (if the units were meters, it would be 10cm to 10Km). It is rare that you need such a wide depth range.

[VianPatel]

Without a large far plane, I cannot see shadows over long distances, as shadows appear to stop casting at the far plane. I was under the impression that view::setDynamicLightingOptions sets the max distance for lights to shine and thus the max distance for shadows to cast (as implied in issue #4916), but with experimentation, this does not appear to be true.

Please correct me if I am wrong, but I believe that filament uses an infinite far plane for rendering and a user-specified far plane for geometry and light culling. Because I am rendering planetary scenes, I need distant geometry to appear and shadows to cast over large distances. Ideally, I would want a very large far plane (for distant objects) and a small near plane (~0.001, for objects close to the camera). However, this does not respect the ratio described in [filament/Camera.h at main · google/filament · GitHub](https://github.com/google/filament/blob/main/filament/include/filament/Camera.h#L132) and filament throws an assertion.

[pixelflinger]

Thanks for the precisions.

For the purpose of shadows, I don't think it can work to have such a huge frustum. There might be other problems at play here, but regardless we won't be able to render the shadows with our current shadow culling and such a large frustum.

I think you could try to use 2 cascades and manually adjust the cutoff point so that the separation is at about 10 to 100m.

It's possible we have some bugs when the cascades are used in this way and I will have a look.

[pixelflinger]

Okay I'm reproducing a lot of weirdness with this set-up. Thank you for the report.

[pixelflinger]

@VianPatel could you let me know if #6314 improved things? I don't expect it'll fix all issues, but I think it should fix the shadow flickering with the camera angle (now it might never be there or always be there).

Also for your use case "lispsm" is counter-productive, so I would try to disable it. More changes are needed to really fix the issue.

[VianPatel]

Thank you for working on that. I can confirm that #6314 seems to have fixed the shadow flickering and disappearance with the camera angle. I also no longer see the message Uninitialized texture bound to 'light_shadowMap' in the log. However, this may be due to switching hardware, specifically the GPU (using AMD Radeon Graphics on Ryzen 7 5800HS).

The cascade transition issue still occurs when the camera angle changes, except that the shadow no longer disappears at a low camera angle. I have included a video demonstration of this below that was compiled with filament commit f0b9c5f (Please note that the video quality stays constant, and the low-quality shadows in the video are the direct output of the window).

https://user-images.githubusercontent.com/69381882/204957147-8b7b0fb6-f4f0-4e9f-83b6-95d393088fac.mp4

Additionally, I went ahead and disabled lispsm, though I do not see any immediate improvement.

[VianPatel]

Regarding the cascade issue, the distorted shadows seem to be caused by incorrect shadow maps. When the camera angle is changed, significantly different shadow maps are used.

Shadow map with correct shadows:

Shadow map with incorrect shadows:

I am also noticing that only one shadow pass is done when the shadows are rendered correctly, despite 2 cascades being used. I followed your earlier advice and have a split set at 10m, though it did not make a visible difference. It is possible that I am setting the splits incorrectly, therefore I wanted to clarify that the correct formula for a split is split distance/(near plane * far plane).

[pixelflinger]

With your values of near/far there won't be any satisfying solutions at any angles.

I know how to improve it significantly, but we need a h/w feature (depth_clamp) that's not present on Android, we can emulate it, at a high cost. At the moment I can't make use of it easily.

[pixelflinger]

@VianPatel could you try to replace all instances of TextureFormat::DEPTH16 with TextureFormat::DEPTH32F in ShadowmapManager.cpp and see if it improves your shadows (don't use VSM though). I think there is a chance it might make things a bit better with your scene.

With your scene it is expected that cascades won't help much.

[VianPatel]

I am not seeing any difference in shadow quality with these changes.

With TextureFormat::DEPTH16 https://user-images.githubusercontent.com/69381882/209453554-6a78d19f-d790-4981-8736-820aed90156f.mp4

With TextureFormat::DEPTH32F https://user-images.githubusercontent.com/69381882/209453555-7962a98f-38a6-4017-8269-95cf3f053d36.mp4