Why there are many holes and inconsistent regions when exporting the point clouds.

[rockywind]

Hi, The training set has 207 images while the eval set has 23 images. This is the exporting point clouds. The script is below.

ns-export pointcloud --load-config outputs/data_pk2_230_1gpu_ex/nerfacto/2023-02-07_095509/config.yml \
--output-dir exports/pcd_230_10_10_10_6m/ --num-points 6000000 --remove-outliers True --estimate-normals False \
--use-bounding-box True --bounding-box-min -5 -5 -5 --bounding-box-max 5 5 5

This is the rendering image in the eval set. This is the train metrics. This is the eval metrics.

[tancik]

NeRFs will often "cheat" the geomertry to get good quality color outputs. This means that in regions without texture, the nerf might add a hole and fill in the scene further back. It is easier to tell what is going on by exploring the scene in the viewer. In the viewer are the holes present?

[rockywind]

Hi, I can't connect the viewer. Can you help me? This is the script.

ns-train nerfacto  --viewer.websocket-port=7002 \
                --pipeline.model.predict-normals False \
                --experiment-name data_pk2_230_1gpu_ex \
                --data data/own_data/data_pk_230_exhau \
                --load_dir outputs/data_pk2_230_1gpu_ex/nerfacto/2023-02-07_095509/nerfstudio_models \
                --viewer.start-train False

[tancik]

Try adding --viewer.skip-openrelay True to your command. Also try waiting a minute or two to connect, sometime it take a little time.

[rockywind]

Hi, I try adding --viewer.skip-openrelay True, but still con't connect the viewer. This is the script.

ns-train nerfacto  --viewer.websocket-port=7002 \
                --pipeline.model.predict-normals False \
                --experiment-name data_pk2_230_1gpu_ex \
                --data data/own_data/data_pk_230_exhau \
                --load_dir outputs/data_pk2_230_1gpu_ex/nerfacto/2023-02-07_095509/nerfstudio_models \
                --viewer.start-train False \
                --viewer.skip-openrelay True \

[tancik]

This is related to https://github.com/nerfstudio-project/nerfstudio/issues/765

[rockywind]

Hi, @tancik

1. What's the meaning of bounding_box_min and bounding_box_max?
2. What's the meaning of num-points? I increase the number of 'num-points', I found that increase the number of points made the effect of 3D reconstruction worse.
3. Why is the number of point clouds not fixed but dynamic? How to set the optimal number of point clouds？

ns-export pointcloud --load-config outputs/data_pk2_230_1gpu_ex/nerfacto/2023-02-07_095509/config.yml \
--output-dir exports/pcd_230_10_10_10_6m/ --num-points 6000000 --remove-outliers True --estimate-normals False \
--use-bounding-box True --bounding-box-min -5 -5 -5 --bounding-box-max 5 5 5

[rockywind]

Hi, @tancik How can I improve the 3D reconstruction? This my training script.

export CUDA_VISIBLE_DEVICES=0
ns-train nerfacto  --viewer.websocket-port=7009 \
                --machine.num-gpus 1 \
                --experiment-name kitti_00_double_600 \
                --data data/kitti/00_double_200 \
                --vis tensorboard 

This is my 3D reconstruction result. https://user-images.githubusercontent.com/14838781/219057994-ab1a39f6-d214-4907-bf0e-59cf2ac10b54.mp4 This is my export point cloud script.

ns-export pointcloud --load-config outputs/kitti_00_double_600/nerfacto/2023-02-15_100002/config.yml --output-dir exports/pcd_600_kitti_stereo/ --num-points 6000000 --remove-outliers True --estimate-normals False --use-bounding-box True --bounding-box-min -5 -5 -5 --bounding-box-max 5 5 5

This is my render result in validation dataset.

[834810269]

Hi, @rockywind I also want to train nerfacto on kitti-like data, I see that you train on kitti dataset and achieve a good render result. So I want to kown how do you configure nerfacto to train so effectively on kitti, how to set parameters