generative-evaluation-prdc icon indicating copy to clipboard operation
generative-evaluation-prdc copied to clipboard

Dummy example gives non intuitive result

Open ashual opened this issue 3 years ago • 0 comments

Hello and thank you for this great paper and implementation.

I've run your method with a dummy example:

    fake_features = torch.ones((1024, 4096))
    real_features = torch.ones((1024, 4096))

and would expect 1.0 for both density and coverage but actually got 0.0

There are two changes to the Density metric that might help

  1. Add less or equal (distance_real_fake < np.expand_dims(real_nearest_neighbour_distances, axis=1) => (distance_real_fake <= np.expand_dims(real_nearest_neighbour_distances, axis=1)
  2. Add clamp with self.neareset_k and enforce [0,1] result (distance_real_fake <= real_nearest_neighbour_distances.unsqueeze(1)).sum(dim=0) => (distance_real_fake <= real_nearest_neighbour_distances.unsqueeze(1)).sum(dim=0).clamp(0, self.nearest_k)

Is it make sense or do I miss something?

ashual avatar May 31 '21 11:05 ashual