ignite
ignite copied to clipboard
pytorch
Add Hausdorff distance in Metrics
I think it would be nice to add a standard implementation of the Hausdorff distance in Ignite as a Metric.
@pldelisle could you please add some reference for this distance (and some use-cases for my culture :) ). Thanks
If you would like to contribute, you are welcome !
Hi there, I think I may be able to resolve this Issue. Would you be happy to assign me, and accept my contribution as a Hacktoberfest PR should I manage to put in a PR that closes it?
could you please add some reference for this distance (and some use-cases for my culture :)
Since the original poster has not answered this, I can briefly outline some good resources on the Hausdorff Distance (HD subsequently) that I have found whilst looking into this, as below.
References
HD definition & context:
- This open-access paper An Efficient Algorithm for Calculating the Exact Hausdorff Distance (Taha, 2015) is really clear I think on the many aspects including the definition & theory;
- Use cases are for example object matching applications, e.g. those outlined under 'Applications' on the Wikipedia page for the HD.
Good references for implementation in Python:
-
I found this support forum question posed on the PyTorch Discourse of particular interest, since it points to a module in the project
javiribera/locating-objects-without-bboxeswhere there are functions to calculate both the average and weighted HD (in fact, using PyTorch). -
There is a library
mavillan/py-hausdorffwhich implements the algorithm outlined in the first paper mentioned above to calculate the HD. -
Finally and notably,
scikit-imagehas its own HD metric, as described in their API reference here, with a usage example (and helpful diagram) here. The source code for that would be useful as an analogous implementation.
@sadielbartholomew Thank you for this FR. I'm very curious and interested by such metric for detection and segmentation apps. I think that it could be nice to have a good demonstrative example t show how use this wrt others metrics (iou for instance). What do you think ?
IMO, implementation should be distributed because targeted apps are time consuming. The test should be compared to sklearn implementation. Don't forget the doc with references to resources and literature.
@vfdev-5 ok for you ?
@sadielbartholomew yes, I can assign it to you. Thanks for the link !
should I manage to put in a PR that closes it?
yes
@sdesrozis OK for me.
Thank you ,@sdesrozis and @vfdev-5.
Thank you for this FR. I'm very curious and interested by such metric for detection and segmentation apps.
Just FYI in case you did not notice, I wasn't the one who made the feature request, so I don't personally have a need for this metric. But it seems more than reasonable that others would benefit from having it, given the numerous applications, as well as the original poster. So I will assume the maintainers would like this feature, still, unless I am told otherwise.
I think that it could be nice to have a good demonstrative example t show how use this wrt others metrics (iou for instance). What do you think ?
Sure, that sounds helpful. I can try to create such an example.
IMO, implementation should be distributed because targeted apps are time consuming. The test should be compared to sklearn implementation. Don't forget the doc with references to resources and literature.
Thanks for the advice, I will take all of it on board when I start work on this. If I have questions I will ask them here.
In terms of timescales, I am quite busy until early next week, but then I can start working on it in some of my spare time and in a week or two after that I will hopefully be able to put up a PR. Otherwise, if for some reason I can't make progress I will let you know. Cheers.
But it seems more than reasonable that others would benefit from having it, given the numerous applications, as well as the original poster. So I will assume the maintainers would like this feature, still, unless I am told otherwise.
@sadielbartholomew yes, let's implement this metric :)
In terms of timescales, I am quite busy until early next week, but then I can start working on it in some of my spare time and in a week or two after that I will hopefully be able to put up a PR.
Sounds good !
Hey @vfdev-5, can i take this issue up ? I would like to follow the sklearn implementation and further look into making it a distributed implementation ?
@codeboy5 yes, go ahead, thanks. Please, put here some details on how you would like to implement it and test the implementation.
I was planning to replicate this implementation (https://github.com/scikit-image/scikit-image/blob/main/skimage/metrics/set_metrics.py) in the structure of reset, update and compute which metrics in ignite follow. We can start by this and see further if changes are required. Does this sound good ?
I'm not quite sure about simply porting scikit-image implementation as they use scipy and cKDTree. In our case we would like to keep less external deps. Another point I'd start by an application and make clear on what kind of data this metric would work on: images, vectors, something else ? Another question to answer is: can we compute it in online way ? What structures we need to accumulate and how to compute it on N inputs ? Can it be an average of distances per object or should be a distance between N preds vs N ground truth objects ?
Okay, let me go through the details of the paper and implementation. I will get back to you on these questions and then we can discuss how to proceed further.
Hi @sadielbartholomew any progress on the issue ?
Hi, really sorry I didn't respond again - in the end I got given some urgent work that took up most of my time, so couldn't get this done. I'm glad someone else has offered to do it! Good luck :)