3DGS editing problems

[SmthFail]

Hi all! Is it a way to edit 3DGS scene? I can load .ply file and splats visualize prettry cool. But i want change some parameters of splats (e.g. color). I see that viser.GaussianSplatHandle has field "buffer" but it seems that i can't assign modified buffer. Is it correct and this field is readonly or Im wrong and it is possible?

[brentyi]

Hi! For updating Gaussians, the best we can do is replace the whole buffer.

Assigning to the .buffer attribute should work, eg splat_handle.buffer = new_buffer, but constructing the buffer is a little bit involved:

https://github.com/nerfstudio-project/viser/blob/0cb94054b29028a012aeeba11b00a5477b09a2fa/src/viser/_scene_api.py#L1342-L1360

An easier approach is to call server.scene.add_gaussian_splats() a second time. If you use the same name, the new splat object should replace the old one. Did you happen to try this?

[SmthFail]

Hi @brentyi! Thanks for answer. I found a way to update like this:

gaussian_splats = server.scene.add_gaussian_splats()
new_buff = gaussian_splats.buffer.copy()
# do some stuff with buffer
gaussian_splats.buffer = new_buff

Also i notice that is similar to point cloud behavior, e.g. you can update field .colors only with new buffer But is it good for update large scene? Or its limitation of viser that i couldn't realize enough?

And another question. Do we need method that will return buffer as more readable format, e.g. SplatFile from example or structed numpy array / dataclass?

[brentyi]

Also i notice that is similar to point cloud behavior, e.g. you can update field .colors only with new buffer But is it good for update large scene? Or its limitation of viser that i couldn't realize enough?

Yeah, unfortunately this is an implementation limitation right now. We could try to support partial updates but it would be tricky and it's not on the roadmap.

And another question. Do we need method that will return buffer as more readable format, e.g. SplatFile from example or structed numpy array / dataclass?

Perhaps we could add a method to GaussianSplatHandle called something like update_buffer(self, rgbs, ...) that takes more human-friendly arrays as input, then sets the .buffer property under the hood? I'm open to a PR for something like this!

[SmthFail]

@brentyi thanks for answer. I think it also be good to have method for get buffer in human-friendly array. Perhaps get_parsed_buffer(). If you want i can try to implement it and open PR in next week