Optimized gaussian splat rendering solution

[mvaligursky]

We need optimized solution, handling large data sets, with culling, LOD and similar. Possibly compute based solution for WebGPU.

References:

• https://www.magnopus.com/blog/how-we-render-extremely-large-point-clouds
• https://twitter.com/garrettkjohnson/status/1729145395820069248

Sorting options:

• https://linebender.org/wiki/gpu/sorting/?s=03

[pjbaron]

If you're looking at changing the architecture, I would like to make a request. Can the new solution avoid storing copies of arraybuffer objects outside of the splat instance?

Justification: I tried to morph one splat object into another. After the morph the target object was always partially transparent and weird looking. I noticed that the splat sorter is cloning the 'centers' list, and that is then duplicated into worker threads.

The problem: movement of splats (such that they will require re-sorting) is impossible in real-time. The setData overhead for sending a full set of new data to the worker threads is too heavy to be real-time feasible. Without feeding the new centers to the sort, all of the splats were in new positions but the sort was ordering them based on their original locations.

I considered and rejected a few possible solutions, and reluctantly gave up on my morph concept as I believe this will be a significant change to the splat renderer architecture. ... But if you're going to rebuild it anyway ...!