Train 2D feature extraction and point representation

[yjcaimeow]

Hi @Xharlie and @zexxu ,

I do not understand about "point feature initial" part. Does the model under "init" folder for all dataset? Or for different dataset, I need to train a VGG 2D feature extracting network firstly and seperately? If need, how to train for different dataset.

Can the model under "init" used by other dataset except for dtu? For example, scannet and kitti.

Best, Yingjie

[Xharlie]

hi, the init checkpoint is trained on dtu and can be used for any dataset. It is mentioned in the paper

[yjcaimeow]

Is it ok for outdoor dataset KITTI？

[Xharlie]

i haven't tried it yet.

[yuzewang1998]

@yjcaimeow Hi,have you success to train the 2D feature extracting network from scratch on ScanNet or KITTI?

[SeaBird-Go]

@Xharlie , Hi, I'm still confused about the train 2D feature extraction and point generation part.

From my understanding and the step-by-step debugging, I find you only optimize the FeatureNet and PointAggregator in this part since the option manual_depth_view=0. (BTW, I don't understand the exact meaning of manual_depth_view when setting to different values).

And from your descriptions in your paper, you use the rendering loss to optimize the networks, but why you set the weight of ray_masked_coarse_raycolor=0.0, and the weight of ray_depth_masked_coarse_raycolor=1.0. So you only use the camera_expected_depth to optimize these networks?

Could you help me understand this part?

[Youngju-Na]

@Xharlie , Hi, I'm still confused about the train 2D feature extraction and point generation part.

From my understanding and the step-by-step debugging, I find you only optimize the FeatureNet and PointAggregator in this part since the option manual_depth_view=0. (BTW, I don't understand the exact meaning of manual_depth_view when setting to different values).

And from your descriptions in your paper, you use the rendering loss to optimize the networks, but why you set the weight of ray_masked_coarse_raycolor=0.0, and the weight of ray_depth_masked_coarse_raycolor=1.0. So you only use the camera_expected_depth to optimize these networks?

Could you help me understand this part?

@Xharlie Same question here. Could anyone give an explanation? Thanks!