Implement clustered lighting as a main lighting system (Root ticket)

[mvaligursky]

This is the issue to track progress for the clustered lighting implementation in the engine. Here are the most important steps:

• [x] Core tech prototype and architecture: https://github.com/playcanvas/engine/pull/2990
• [x] Spot light shadows: https://github.com/playcanvas/engine/pull/3235
• [x] Spot light cookies: https://github.com/playcanvas/engine/pull/3521
• [x] Omni light shadows using the same shadow atlas as spot lights https://github.com/playcanvas/engine/pull/3537
• [x] Omni light cookies using the same cookie atlas as spot lights https://github.com/playcanvas/engine/pull/3555
• [x] Complete lighting model specular: https://github.com/playcanvas/engine/pull/3668
• [x] Add support for area lights https://github.com/playcanvas/engine/pull/3691
• [x] Shadow filtering options on top of default PCF3 https://github.com/playcanvas/engine/pull/3763
• [x] Smarter texture atlas slot assignment https://github.com/playcanvas/engine/pull/3895
• [x] Lightmapper update to use clustered lights https://github.com/playcanvas/engine/pull/3902
• [ ] Remove old lighting system (shader recompiles, uniforms, standard, static lights, chunks ..)

Optional features:

• [x] Debug rendering of the cells and the number of lights in them https://github.com/playcanvas/engine/pull/3873
• [x] Single texture slot solution to store prefiltered data for IBL https://github.com/playcanvas/engine/issues/3681
• [ ] Extract the light evaluation fragment function to standalone chunk to allow custom lighting models (cell shading ..)
• [ ] Optimize static light culling of static meshes to avoid shadow rendering if no change
• [ ] Tighter cone / sphere insertion into the cluster cells to limit number of lights in a cell
• [ ] Allow texture atlases for shadows and cookies to use texture array (light would store viewport and additional index)

Other related PRs

• globally disable shadows or cookies: https://github.com/playcanvas/engine/pull/3567
• API https://github.com/playcanvas/engine/pull/3763

Public release after rounds of beta testing: https://github.com/playcanvas/engine/pull/4586

[Maksims]

One major issue for forward-renderer approach to lights is that light-related shader code is added to materials, and one huge bottleneck with this approach is when enabling/disabling lights - leads to all affected materials shader revalidation and recompilation, which is extremely slow.

I guess with shadow atlas and clustered version, this will not be an issue anymore?

[mvaligursky]

That's correct, adding / removing lights will not rebuild the shaders when fully integrated (it does currently). And also you only pay for the lights that are "nearby" instead of all, for both static and dynamic lights.

[Maksims]

Currently, when creating lights with shadows using atlases, with profiler engine, it does not add/remove VRAM usage from for app.stats.vram.texShadow.

Also, adding new lights, it seems to generate new materials, but might not be removing old ones. app.stats.shaders.materialShaders.

[mvaligursky]

Currently, when creating lights with shadows using atlases, with profiler engine, it does not add/remove VRAM usage from for app.stats.vram.texShadow

I added console.log(app.stats.vram.texShadow); to clustered-spot-shadows example update loop .. and when I change the shadow atlas resolution using the slider, it updates the allocated size. How do you reproduce the issue? Internally, a normal shadow map is allocated, and that already works .. so this works as well.

Note that only a single atlas is used .. there is no per-light allocation taking place. If you set atlas to be 4k x 4k, it allocates it ones and then just subdivides for the visible lights that need shadows during the frame.

Also, adding new lights, it seems to generate new materials, but might not be removing old ones

I think engine keeps all shaders around to avoid compilation should the same shader be needed at a later stage. When clustered lights are fully integrated, it will not create new shaders when lights are added though, this has not been done.