f3d-app
Add CPU gaussian sorting
Is your feature request related to a problem? Please describe.
When changing the camera orientation, it is required to sort the gaussians by depth in order to have a correct result.
Sorting is currently executed on the GPU using compute shaders.
However it's not supported on macOS and the GPU sorting can be slow on low-end GPUs.
Let's consider CPU sorting using vtkDepthSortPolyData
Gaussians visualization is done in vtkext\private\module\vtkF3DPointSplatMapper.cxx file, and sorting shader is executed in the method vtkF3DSplatMapperHelper::SortSplats
We probably need to rename the method to SortSplatsGPU and add a new SortSplatsCPU function. The CPU function could be enable on macOS only for now, but it would be useful to have a way to let the user select which backend to use.
I would like to work on this issue. @Meakk can you provide a bit more context in which scenarios this sorting is happening?
Just visualize any splat file with gaussian splatting, eg:
./bin/f3d ../src/testing/data/small.splat --point-sprites --point-sprites-type=gaussian --scalar-coloring --coloring-array=color --coloring-component=-2 --up=-Y
https://github.com/f3d-app/f3d/blob/9994d18147bcad77d351c3ce936d8f1ecb0043cd/vtkext/private/module/vtkF3DPointSplatMapper.cxx#L92
The gaussians are sorted by depth in camera direction in order to have proper alpha blending.
It's done in a compute shader using a bitonic sort for performance reason. Unfortunately it doesn't work on macOS, and OpenGL ES backends (web and android), so we need a fallback cpu method even if it's slower.
@mwestphal @Meakk thanks a lot, I think it should be sufficient, I will start on implementation in near time
Hi @exbluesbreaker
As far as I understand you may not have time to work on this, should I unassign it ? :)
As far as I understand you may not have time to work on this, should I unassign it ? :)
Yes, I think is better to unassign. Apart from me currently being too busy to work on issues, this particular issues definitely required some refinement of OpenGL/computer graphics behavior to understand processing here from my side. So somebody more knowledable is a better candidate to work on it.
No worries on that regard, feel free to comment on it whenever you want to pick it up again :)
@exbluesbreaker are you still interested in this one? I would like to take a look into this issue if you are not interested in this one anymore
@exbluesbreaker are you still interested in this one? I would like to take a look into this issue if you are not interested in this one anymore
You will need VTK 9.5.0 for this one I'm afraid.
@exbluesbreaker are you still interested in this one? I would like to take a look into this issue if you are not interested in this one anymore
You will need VTK 9.5.0 for this one I'm afraid.
interesting topic though. I'll try to find another one then :(
@exbluesbreaker are you still interested in this one? I would like to take a look into this issue if you are not interested in this one anymore
You will need VTK 9.5.0 for this one I'm afraid.
interesting topic though. I'll try to find another one then :(
You can take it over. I am currently too busy to contribute, and this particular issue wasn't the best choice for me anyway
FYI I implemented a version of CPU based splat sorting here: https://github.com/f3d-app/f3d/pull/2593
It works for me, and maybe it can be useful to the anyone else experiencing this issue.
