supersplat
supersplat copied to clipboard
playcanvas
Gaussian Splat labelling workflow
It would be great for (research purposes and others) if this editor could enable users to visualize and edit labelled gaussian point clouds.
Such a feature would require:
- Creation or adoption of a standardized label data format for Gaussian Splatting data. Perhaps using an existing label format developed by segments.ai would be a good approach to avoid standard proliferation.
- CRUD operations on labeled Gaussian Splat data. To enable users to load, edit, update, apply, and save labels for a .ply file within the editor.
- Option to visualize gaussian cloud based on class id. Perhaps by including additional metadata of the class RGB that a point would be rendered as belonging to that class. This would need to be data-visualization friendly by perhaps recommending ordinal data color gradient choices.
I've started the development of such a feature in a fork: supersplat-classifier. Currently, it also includes hot-reloading of .ply files. Ideas and suggestions are welcome :)
Latest work, perhaps someone is interested in helping out!
- Used Segments.ai 3D point cloud Segmentation Label
- Hot reload of splat and label data from python notebook using WebSocket communications with custom message format.
- 'Show Classes' option to color the splat centers based on class color.
- Annotation manager to highlight, select and hide certain annotations (WIP).
https://github.com/user-attachments/assets/c6985075-313f-429d-bd01-fcbee0011820
I wonder if this is creeping features into something complicated that might be usable by very few only. I don't see how hot-loading and python notebooking has to be combined with labeling. My experience when contributing was always that contributions could only be merged if they addressed a single feature and not a combination of multiple ones
@simonbethke while the hot-loading feature is very limited in it's use cases, a wider application would be the implementation of splat streaming to visualize a training process for example. But you are correct, maybe this specific PR should be concerned only with annotation support.