Add prune_scale3d argument to MCMCStrategy
MCMC-based models often suffer from very large splats that reduce the overall quality, as partially discussed in #252 and https://github.com/nerfstudio-project/nerfstudio/pull/3548. This PR adds the same logic as in the default strategy, that is pruning (or in this case relocating) splats that are too large. I used the same naming convention and default values as for the default strategy for consistency.
If it makes things clearer, I can rename prune_scale3d to relocate_scale3d.
Have you ran this and seen if it fixes the issues mentioned on those datasets?
@N-Demir: I only ran it on my datasets (that I can't share) and it improves the situation. Some users on the Discord server tested it and it worked for them as well.
can also confirm this dramatically improves the quality of the exported ply files, this needs to be in nerfstudio asap
Sorry, I tested it with two setup:
ns-train splatfacto-mcmc --pipeline.datamanager.batch-size 3with gsplat 1.4.0ns-train splatfacto-mcmc --pipeline.datamanager.batch-size 3with this PR of gsplat
But the result of 2nd setup is worse. Both quantitative (PSNR, LPIPS, SSIM) and visually qualitative.
So far, Tested it with garden, bicycle, and stump dataset from mip360.
1st setup:
2nd setup:
Dunno why it does not give good result from me. FYI. I have not tested it with native gsplat simple_trainer.py mcmc.
Additional info:
Ubuntu 22.04.3 LTS
Python 3.10.12
Torch 2.1.2+cu118
RTX 4090 on Runpod cloud environment
Nerfstudio commit d5bdd45b5bfd30e928e8b90d6735b2e38ae79a59
python3 examples/simple_trainer.py mcmc --batch-size 3:
1st = Gsplat 1.4.0
2nd = This PR of GSplat
Additional benchmark of MCMC from @MrNeRF in this thread: https://github.com/nerfstudio-project/gsplat/pull/409. It uses batch-size 1.
@ichsan2895 thanks for the feedback! So maybe keep it disabled by default, so that users can enable it for scenes where this improves things? In my case, drone flights for which we look down and not far away (large splats are often located on the border of the scene but "spill" in the focused part).