votenet
votenet copied to clipboard
facebookresearch
Large loss and nan loss
**** EPOCH 002 **** Current learning rate: 0.001000 Current BN decay momentum: 0.500000 2023-03-27 11:14:37.992102 ---- batch: 010 ---- mean box_loss: nan mean center_loss: nan mean heading_cls_loss: nan mean heading_reg_loss: nan mean loss: nan mean neg_ratio: 0.254590 mean obj_acc: 0.254375 mean objectness_loss: nan mean pos_ratio: 0.009082 mean sem_cls_loss: nan mean size_cls_loss: nan mean size_reg_loss: nan mean vote_loss: nan ---- batch: 020 ---- mean box_loss: nan mean center_loss: nan mean heading_cls_loss: nan mean heading_reg_loss: nan mean loss: nan mean neg_ratio: 0.000000 mean obj_acc: 0.000000 mean objectness_loss: nan mean pos_ratio: 0.000000 mean sem_cls_loss: nan mean size_cls_loss: nan mean size_reg_loss: nan mean vote_loss: nan
/////////////
---- batch: 080 ---- mean box_loss: 36476394887.889046 mean center_loss: 36476394887.495895 mean heading_cls_loss: 0.000000 mean heading_reg_loss: 0.012006 mean loss: 364819971942.011230 mean neg_ratio: 0.742920 mean obj_acc: 0.906894 mean objectness_loss: 0.160109 mean pos_ratio: 0.053662 mean sem_cls_loss: 2.860292 mean size_cls_loss: 2.870165 mean size_reg_loss: 0.136136 mean vote_loss: 5600754.862075 ---- batch: 090 ---- mean box_loss: 0.804259 mean center_loss: 0.334334 mean heading_cls_loss: 0.000000 mean heading_reg_loss: 0.020176 mean loss: 25.172875 mean neg_ratio: 0.758154 mean obj_acc: 0.868903 mean objectness_loss: 0.155064 mean pos_ratio: 0.048584 mean sem_cls_loss: 2.875596 mean size_cls_loss: 2.889619 mean size_reg_loss: 0.160787 mean vote_loss: 1.347937 ---- batch: 100 ---- mean box_loss: 3856308962287565668352.000000 mean center_loss: 3856308962287565668352.000000 mean heading_cls_loss: 0.000000 mean heading_reg_loss: 0.029361 mean loss: 38563091424330962173952.000000 mean neg_ratio: 0.808154 mean obj_acc: 0.857417 mean objectness_loss: 0.147534 mean pos_ratio: 0.039111 mean sem_cls_loss: 2.843654 mean size_cls_loss: 2.867544 mean size_reg_loss: 0.160195 mean vote_loss: 953150979688.176392 ---- batch: 110 ---- mean box_loss: 0.724312 mean center_loss: 0.253483 mean heading_cls_loss: 0.000000 mean heading_reg_loss: 0.024551 mean loss: 22.057832 mean neg_ratio: 0.727441 mean obj_acc: 0.861654 mean objectness_loss: 0.159908 mean pos_ratio: 0.055566 mean sem_cls_loss: 2.833293 mean size_cls_loss: 2.864183 mean size_reg_loss: 0.159860 mean vote_loss: 1.118188 ---- batch: 120 ---- mean box_loss: 19054326959700664.000000 mean center_loss: 19054326959700664.000000 mean heading_cls_loss: 0.000000 mean heading_reg_loss: 0.029966 mean loss: 190544650858704576.000000 mean neg_ratio: 0.786963 mean obj_acc: 0.841404 mean objectness_loss: 0.150460 mean pos_ratio: 0.043848 mean sem_cls_loss: 2.877827 mean size_cls_loss: 2.862448 mean size_reg_loss: 0.168548 mean vote_loss: 136601700196.556839
are these highly exposed regions in the source images?
Yes, but there are many pictures from different angles corresponding to the same place on the model, some pictures are high exposure, some are not.
- for best 3D reconstruction results (not only texture) images should be acquired at good, but diffuse light, and equal or close to equal exposure (constant light).
- since 1 is a requirement, OpenMVS does not have a good way now to select images without glossiness artifacts, in fact as it is now it is biassed to select such images
- help is needed to implement a better texturing algorithm, pls give a sign if interested
thank you for your reply,
- I marked the high-exposure areas in the image as much as possible, and reduced the scores of these pixels in the view selection stage, but the effect was not good. If I have time, I will continue to study.
- I encountered a problem. There is still a color difference between the texturePatches. I saw that you used Poisson fusion. Why is there such a big difference? Can opencv's seamlessClone replace this function?
good suggestion, a different blending procedure might be better, pls try
I did an experiment. Set the color of the border of the texture block to black as the boundary condition of the Poisson calculation, and the texture map result is shown in the figure. Does this mean that there is a problem with setting the edge color of the texture block in LocalSeamLeveling, causing the Poisson calculation to be wrong.
I'm not sure if my guess is correct.
Interesting experiment. However not clear to me what is the results before and after, and which edge did you set to black exactly. Can you add more screenshots?
Interesting experiment. However not clear to me what is the results before and after, and which edge did you set to black exactly. Can you add more screenshots?
I set the junction of different textures to black, and the original code is set to the average color. By comparison, I suspect that openmvs misses some border pixels, doesn't change their color, causing the poisson calculation to still use the original color, which leads to weird transition areas between different textures.
This is the mask image I saved, and you can see that some edges are not set to gray "border".
In the DrawLine function, "Bresenham's line algorithm" is used to calculate the texture edge pixels. Is there any problem here? Looking forward to your reply.
thanks for the detailed experiment a couple of thoughts:
- mvs-texturing is using mapmap instead of LBP to assign a image to each face, which should be not only faster, but better/fewer patches, and less seams
- recently I discovered a bug in how the camera matrix K is scaled, and I think the same bug affects also other parts of OpenMVS where the image coordinates are scaled, possibly also in texturing, though I can not see one at a quick look
most probably though might be a problem with drawing the line, like you say, or something related; it would be of great help if you could look into this and fix it; you could compare on how this is done in mvs-texturing