Problem about Ms and geometric consistance

[JarvisLee0423]

Thank you for your great work! Among reading to your SC-Depthv1 paper, I found that the Self-Discovered Mask is directly computed from the depth difference (D_diff), but the D_diff is used to compute the geometric consistent loss either. Meanwhile, you only remove the invalid pixels defined by auto-mask, and the self-dicovered mask will be a penalized weight for the photometric loss.

In this case, I have the following question about the optimization scheme:

The Geometric Consistent Loss mainly consist of D_diff, and the optimized target is to make the Geometric Consistent Loss become lower. In this case, intuitively, I think the consistent differences of all the valid pixels specified by auto-mask will be decreased.

However, according to your illustration of the auto-mask, it can not remove the invalid pixels caused by the moving object, therefore, the pixels corresponding to the moving object will be seen as valid pixel when compute the Geometric Consistant Loss. Hence, the supervisation scheme will hope the Geometric Consistent Loss get a lower value for the pixel in moving object.

However, when you apply the self-disconverd mask to the photometric loss, the model will hope the penalized weight becomes smaller (1 - D_diff).

Therefore, the Geometric Consistent Loss will give the model a training signal about decrease the value of D_diff for the region in move object. The self-disconvered mask (1 - D_diff) give the model a training signal about increase the value of D_diff for the region in move object (1 - D_diff become lower, then D_diff should be higher).

Is there a conflict when training with this scheme? Or, why the model can finally choose the correct optimization path to only decrease the Geometric Consistent Loss for the truely valid region?