Segmentation Maps Border Treatment broken

[thetoby9944]

Summary: Applying augmentations with SegmentationMapOnImage ignores the mode parameter. Any transformations that create a void image area (affine, pad, ...) is executed with constant border treatment only on the labels.

Example:

def segmentation_map_aug(
    self, 
    image: np.array, 
    label: np.array
) -> (np.array, np.array):

    label = SegmentationMapsOnImage(label, shape=image.shape)
    image, label = iaa.Affine(
        rotate=(-180, 180),
        mode="reflect"
    )(
        image=image, 
        segmentation_maps=label
    )
    return image, label.get_arr()

Observed Behavior: The image content gets reflected where the rotation would leave a void area - But the corresponding labels fill the void area black, instead of also reflecting the content. This creates a mismatch in image content and segmentation map content.

Expected Behavior: When the content of an image is reflected during rotation or padding, the corresponding segmentation map should be reflected as well. This also applies to other border treatment modes.

[edumotya]

Well, there is a workaround I've been using for a while, try to override the _mode_segmentation_maps property of your augmenter:

aug._mode_segmentation_maps = aug.mode.value

And in the case you have a composition of augmenters, such as Sequential, you can use the following:

for aug in augmenter.get_all_children():
    try:
        aug._mode_segmentation_maps = aug.mode.value
    except AttributeError:
        pass

Hope it helps.