Questions on Evaluations

[C-H-Chien]

I have a few questions on your evaluations:

1. For the ABC_NEF dataset, the GT parametric curves have to be shifted and scaled so that they are located at the correct coordinate (see this issue). And since this dataset is a synthetic dataset, the 3D points are unit-less. How do you relate the threshold $\tau$ in millimeters mentioned in your paper to the threshold of the unit-less 3D GT? (In the NEF paper, they set the threshold 0.02, with no unit, as the L2 distance from the reconstructed edge point to the ground truth point.)

2. For generating the 3D GT edges from the DTU dataset, the supp says that the dense point cloud is projected to the images and then cross-comparing these projections with 2D edges observations to mark 3D edges. Are the "cross-comparing" done manually? If not, what strategies/algorithms did you use? In addition, the supp says, (Section B.3) "To ensure accuracy in the ground-truth edge points, we manually set thresholds for each scan and meticulously remove any floating points." What threshold and floating points are you referring to here?

3. For the metrics, what are the definitions of "Accuracy" and "Completeness"? The paper says that it follows the NEF and the LiMAP papers for the metrics but I do not find them from both papers. Maybe it's a typo as I find the definitions in the NEAT paper, Section F of supp. Also, do you have an explicit definition of what an "edge direction consistency" is?

4. In Table 1, are the reported numbers the average of all selected 82 CAD models? Do you use all 50 images per CAD model?