Training with a live streaming

[jinn9403]

Really good research and thanks for sharing the source. Especially the reconstruction using Gaussian splatting in fast time seems to be a step closer to real-time reconstruction.

When doing online training, I noticed that instantNGP has a function to show the training process while receiving streaming via nerfcapture app. You used the dataset_capture_loop function to stream the dataset to a folder and train based on it. Is it possible to apply the def live_streaming_loop function to train in real time?

And the output is supposed to be viewable in open3d. If I want to view it in nerfstudio viewer or any other viewer, please let me know where to modify it.

[Nik-V9]

Thanks!

Yes, updating the code to show live training should be possible. However, we currently don't have this implemented. Furthermore, SplaTAM needs further optimization to enable faster reconstruction.

Regarding other viewers for Gaussian Splats: We support exporting our output to the standard .ply format used by various viewers like SuperSplat & PolyCam (not sure what NeRFStudio uses):

python scripts/export_ply.py configs/...