Output intermediate Chamfer distribution score to get F-Score metric.

[gaetan-landreau]

🚀 Feature

NOTE: Allow in the chamfer_distance() function from pytorch3d.loss.chamfer to also output intermediate values of cham_x and cham_y in order to being able for instance to compute F-score metric. https://github.com/facebookresearch/pytorch3d/blob/b8790474f16994d75e6cf64447080f1b52bcc292/pytorch3d/loss/chamfer.py#L217

Motivation

It would be great to have this feature in order to directly infer the F-score from these cham_x and cham_y distributions since F-Score & Chamfer distance are two useful metrics in 3D reconstruction for instance.

Pitch

Adding this feature would allow to easily compute F-score metric, in a similar fashion way as @ThibaultGROUEIX did in one of his repo: https://github.com/ThibaultGROUEIX/ChamferDistancePytorch

import torch

def fscore(dist1, dist2, threshold=0.001):
    """
    Calculates the F-score between two point clouds with the corresponding threshold value.
    :param dist1: Batch, N-Points
    :param dist2: Batch, N-Points
    :param th: float
    :return: fscore, precision, recall
    """
    # NB : In this depo, dist1 and dist2 are squared pointcloud euclidean distances, so you should adapt the threshold accordingly.
    precision_1 = torch.mean((dist1 < threshold).float(), dim=1)
    precision_2 = torch.mean((dist2 < threshold).float(), dim=1)
    fscore = 2 * precision_1 * precision_2 / (precision_1 + precision_2)
    fscore[torch.isnan(fscore)] = 0
    return fscore, precision_1, precision_2

[patricklabatut]

That makes sense.

A simple way to achieve this would be to rename the existing chamfer_distance() function into e.g. chamfer_distance_xy() and change its implementation and return type to be a pair of (cham_x, cham_y) pair and (cham_norm_x, cham_norm_y) pair (or None). And to preserve backward-compatibility, then implement chamfer_distance() in terms of chamfer_distance_xy().

Could you perhaps submit a PR for such a change?

[gaetan-landreau]

Sure ! Going to work on it and submit a PR asap.

[gkioxari]

Why would you want to compute the F-score on the cham_x and cham_y separately? Of course, this depends on what you want to report, but the F-score reported commonly in 3D reconstruction works is from the output of the chamfer distance as is. The definition of the chamfer distance is cham_x + cham_y. Below is example code on how to compute the F-score from Mesh R-CNN

https://github.com/facebookresearch/meshrcnn/blob/master/meshrcnn/utils/metrics.py

[patricklabatut]

Why would you want to compute the F-score on the cham_x and cham_y separately?

My understanding is that @gaetan-landreau would like to get access to both cham_x and cham_y to be able to combine them directly and compute a single F-score for the pair(s) of point clouds. This is matching the implementation seen in other libraries (check Kaolin for instance) as well as the one in Mesh R-CNN.

The definition of the chamfer distance is cham_x + cham_y.

That's the symmetrized Chamfer distance. This is out of scope but we could also consider introducing a sided_distance() and implementing the existing chamfer_distance() with it to decrease code duplication.