Compose raising when keypoints are configured but no keypoints are passed

[LucaBonfiglioli]

Describe the bug

Compose raising when keypoints are configured but no keypoints are passed. This is actually a fairly typical situation in self-supervised tasks, where datasets may be partially labeled.

Example

I am currently training a keypoint detector on a dataset that is 50% labeled and 50% not. Some samples, despite belonging to the labeled half still contain 0 keypoints, because no keypoint is actually visible in the image.

Cases:

• Keypoints present -> labeled sample - apply supervision
• Keypoints present but empty list -> labeled sample - apply supervision
• Keypoints missing -> non labeled sample - apply self-supervison

The dataset is small and I am using Albumentations to improve the training a bit. To use keypoints we need to configure them by setting the keypoint_params argument, but if I do so, albumentation will raise an error as soon as a non-labeled sample is passed to the compose object.

To Reproduce

Steps to reproduce the behavior:

import albumentations as A
import numpy as np

cursed = A.Compose(
    keypoint_params=A.KeypointParams(format="xy"),
    transforms=[
        A.RandomCrop(256, 256),
    ],
)

image = (255 * np.random.rand(512, 512, 3)).astype(np.uint8)
cursed(image=image)

Expected behavior

In case keypoints are configured:

• (image + non empty keypoints) -> (image + non empty keypoints)
• (image + empty keypoints) -> (image + empty keypoints)
• (image) -> (image)

Actual behavior

What happens now is the following:

• (image + non empty keypoints) -> (image + non empty keypoints)
• (image + empty keypoints) -> (image + empty keypoints)
• (image) -> KeyError

Traceback (most recent call last):
  File "/home/xxx/z.py", line 12, in <module>
    cursed(image=image)
  File "/home/xxx/lib/python3.10/site-packages/albumentations/core/composition.py", line 219, in __call__
    p.preprocess(data)
  File "/home/xxx/lib/python3.10/site-packages/albumentations/core/utils.py", line 82, in preprocess
    data[data_name] = self.check_and_convert(data[data_name], rows, cols, direction="to")
KeyError: 'keypoints'

[ternaus]

Would this do the job?

if keypoints is None:
    result = cursed(image=image, keypoints=[])
    processed_keypoints = None
else:
    result = cursed(image=image, keypoints=keypoints)
    processed_keypoints = result["keypoints"]
    
processed_image = result["image"]

if processed_keypoints is None:
   => self_supervision
else:
  => supervision

[LucaBonfiglioli]

Yep, that's pretty much how I solved it too. But having to manually check each time a Compose is called should not be client code responsibility in my opinion. Also, keep in mind that bounding boxes must be considered in this check as well, further messing the code up.

Wouldn't it be so much better like this?

keys_to_targets: dict[str, Any] = get_stuff_to_transform(...)
processed = compose(**keys_to_targets)

[ternaus]

It is hard to distinguish if person did not pass boxes or keypoints for a reason, like in your case, or just forgot about it.

But we can add functionality and pass keypoints = None in the case when keypoints are missing, but user still wants transform to work.

I think, in such case:

keys_to_targets: dict[str, Any] = get_stuff_to_transform(...)
processed = compose(**keys_to_targets)

will work

[ternaus]

Addressed in https://github.com/albumentations-team/albumentations/pull/1697