ITK icon indicating copy to clipboard operation
ITK copied to clipboard

Add MatchCardinalityImageToImageMetricv4 for registration of labelmaps

Open dyollb opened this issue 2 years ago • 5 comments

Description

Currently, there is no Match Cardinality metric compatible with the "v4" registration framework.

I can use the mean square metric for binary mask registration, but to register a labelmap with many labels the difference between label 1 and 2 is penalized less than the difference between 1 and 500. This can be managed to some extend via remapping labels and for a few labels, but does not scale.

I could one-hot encode the labelmap and use the mean square metric, since it supports vector images. This is not so bad, but probably increases memory consumption and increases the metric computation time linearly with the number of labels. Currently this workaround is not supported by the Python wrapping.

Expected behavior

It would be nice if there was a MatchCardinalityImageToImageMetricv4, i.e. a match cardinality metric for the v4 registration framework.

Only the v4 registration framework is supported by SimpleITK, so adding this metric would benefit SimpleITK users (see https://github.com/SimpleITK/SimpleITK/issues/1896).

As a workaround (for python users), wrapping MeanSquaresImageToImageMetricv4 for vector real types ({WRAP_ITK_VECTOR_REAL}) would be needed to use one-hot encoded labelmaps. CORRECTION: Looking at VectorImageToImageMetricTraitsv4, it seems the framework must be compiled for a fixed number of components/channels.

Actual behavior

  1. It does not exist
  2. The metric (and vectorimage + mean square metric) workaround is not available in ITK Python and SimpleITK

Additional Information

I had a look into writing this new class based on the MeanSquaresImageToImageMetricv4, but am unsure how the moving image gradient should be handled.

dyollb avatar Feb 23 '23 08:02 dyollb

Adding the explanation why the moving image gradient is needed from image

dyollb avatar Feb 23 '23 09:02 dyollb

Could the desired functionality be achieved by converting the label images to labeled pointsets?

https://insightsoftwareconsortium.github.io/ITKDoxygen/classitk_1_1LabeledPointSetToPointSetMetricv4.html

cookpa avatar Feb 28 '23 16:02 cookpa

@cookpa Maybe, but how can I convert the labelfield to a labeled pointset? If there is such a filter, is it exported to Python (it looks like it is not available in SimpleITK)?

For the actual implementation: Should I subsample the labeled pointset, i.e. convert only a percentage of the labelfield to points (to reduce memory/increase performance)?

dyollb avatar Mar 03 '23 09:03 dyollb

Sorry for being slow @dyollb. I learned about the label point set metric from ANTs, but I'm not super familiar with the implementation. It just sounded conceptually similar to what you're trying to do with matching cardinality. I'm not sure about Python coverage. I looked at SimpleITK and I don't think any point set registration is available there.

In ANTs, labeled points are directly transformed from an index to physical space (code). Optionally, the labels can be downsampled with a LabelContourImageFilter first, so you only have to deal with the boundary points.

cookpa avatar Mar 07 '23 19:03 cookpa

Unfortunately, I still have not tested the suggestion by @cookpa (and it might be good since it would weight the labels by surface area instead of volume. It might also be more robust to bad initialization/poor overlap).

But after looking at the GenericLabelInterpolator module I realized it provides an approach that could be used to implement the MatchCardinality metric using label adaptors (one image adaptor for each label): https://github.com/InsightSoftwareConsortium/ITKGenericLabelInterpolator/blob/master/include/itkLabelImageGenericInterpolateImageFunction.hxx

dyollb avatar Sep 09 '23 07:09 dyollb