Implement LOD / global sorting for gsplat rendering

[hazy-sky]

Implementing something similar to this

[slimbuck]

Hi @hazy-sky ,

Please can you explain your request. Why would you like this paper implemented?

Thanks

[earth-bass]

Mainly for LOD and streaming purposes since we have multiple tiles next to each other and if they aren't merged into a single splat instance bad seams show up due to the fact that the alpha blending isnt happening across tiles, And I don't want to merge the tiles for performance reasons, so the next solution is either depth testing or blue noise dithering and dithering doesn't look great in my use case.

[MAG-AdrianMeredith]

This looks very interesting, especially their claim of improved performance

[mvaligursky]

we're working on global sorting here: https://github.com/playcanvas/engine/pull/7825 which should allow you to dynamically add your tiles as needed (or use LOD we're adding as well)

[earth-bass]

Thanks so much, that's exactly what we want. Not sure how the final API would look like, but it would be great if we could control it such that it won't generate LODs dynamically and there is an option to just load in sogs splats to the manager just for global sorting.

[mvaligursky]

Yep @earth-bass , all this works now (not released yet), so not a problem, give us few more weeks.

[jo-chemla]

Really nice post, pretty catchy to announce the support for 100s of millions of splats!

https://twitter.com/playcanvas/status/1976294072068305080

Since this is beta, is there already a solution to tile a large gsplat scene via eg splat-transform? Is te tiling octree based for scenes with large splat distribution in all 3 dimensions, or quadtree or similar as mentioned in linked issue? Thanks!

[mvaligursky]

@jo-chemla - if you have multiple splat files, you can already load them in a single scene to create a large splat, if that's what you need (not entirely sure). We use a generic tree with any number of children for LOD streaming under the hood, so not a 2D, it's fully 3D.

[jo-chemla]

Thanks, I just saw the PRs do mention octree, great to hear! What I have in mind is we (@iconem) do sometimes produce gsplat scenes with a few tens of millions of splats - via discrete grid tiling of the input dataset spanning a few square kilometers and processing each chunk independently via postshot or similar. If merging all the chunks into a single massive gsplat scene with a few hundreds of million of splats, we would like to compute the HLOD octree structure associated with it. Cesium Ion seem to offer such a service via their Ion offering. We have a pdal-based pipeline almost ready, based only on open-source building blocks, and would love to know if your HLOD tiling processing will become available at some point.

[mvaligursky]

Your case seems a good use for what we have really. This branch contains the generation code: https://github.com/playcanvas/splat-transform/tree/lod-support the command line lets you merge multiple splat files together - so in step one, for each lod level, merge all tiles together, to have lod0, lod1, lod2 splat files or so. And then next run would be to generate streaming lod format from those, and this you can directly load into the engine using lod streaming.

[mvaligursky]

Closing as this has been released already, see https://developer.playcanvas.com/user-manual/gaussian-splatting/building/engine-features/global-sorting/ and https://developer.playcanvas.com/user-manual/gaussian-splatting/building/engine-features/lod-streaming/