Add MatchCardinalityImageToImageMetricv4 for registration of labelmaps
Description
Currently, there is no Match Cardinality metric compatible with the "v4" registration framework.
I can use the mean square metric for binary mask registration, but to register a labelmap with many labels the difference between label 1 and 2 is penalized less than the difference between 1 and 500. This can be managed to some extend via remapping labels and for a few labels, but does not scale.
I could one-hot encode the labelmap and use the mean square metric, since it supports vector images. This is not so bad, but probably increases memory consumption and increases the metric computation time linearly with the number of labels. Currently this workaround is not supported by the Python wrapping.
Expected behavior
It would be nice if there was a MatchCardinalityImageToImageMetricv4, i.e. a match cardinality metric for the v4 registration framework.
Only the v4 registration framework is supported by SimpleITK, so adding this metric would benefit SimpleITK users (see https://github.com/SimpleITK/SimpleITK/issues/1896).
As a workaround (for python users), wrapping MeanSquaresImageToImageMetricv4 for vector real types ({WRAP_ITK_VECTOR_REAL}) would be needed to use one-hot encoded labelmaps.
CORRECTION: Looking at VectorImageToImageMetricTraitsv4, it seems the framework must be compiled for a fixed number of components/channels.
Actual behavior
- It does not exist
- The metric (and vectorimage + mean square metric) workaround is not available in ITK Python and SimpleITK
Additional Information
I had a look into writing this new class based on the MeanSquaresImageToImageMetricv4, but am unsure how the moving image gradient should be handled.
Adding the explanation why the moving image gradient is needed from

Could the desired functionality be achieved by converting the label images to labeled pointsets?
https://insightsoftwareconsortium.github.io/ITKDoxygen/classitk_1_1LabeledPointSetToPointSetMetricv4.html
@cookpa Maybe, but how can I convert the labelfield to a labeled pointset? If there is such a filter, is it exported to Python (it looks like it is not available in SimpleITK)?
For the actual implementation: Should I subsample the labeled pointset, i.e. convert only a percentage of the labelfield to points (to reduce memory/increase performance)?
Sorry for being slow @dyollb. I learned about the label point set metric from ANTs, but I'm not super familiar with the implementation. It just sounded conceptually similar to what you're trying to do with matching cardinality. I'm not sure about Python coverage. I looked at SimpleITK and I don't think any point set registration is available there.
In ANTs, labeled points are directly transformed from an index to physical space (code). Optionally, the labels can be downsampled with a LabelContourImageFilter first, so you only have to deal with the boundary points.
Unfortunately, I still have not tested the suggestion by @cookpa (and it might be good since it would weight the labels by surface area instead of volume. It might also be more robust to bad initialization/poor overlap).
But after looking at the GenericLabelInterpolator module I realized it provides an approach that could be used to implement the MatchCardinality metric using label adaptors (one image adaptor for each label):
https://github.com/InsightSoftwareConsortium/ITKGenericLabelInterpolator/blob/master/include/itkLabelImageGenericInterpolateImageFunction.hxx