Multiple instances of keypoint labels per image

[kqf]

🐛 Bug

I have an image that contains multiple faces and multiple landmarks. The number of faces and landmark instances is variable. The problem is that alb.KeypointParams allows to pass only a single instance of keypoints.

To Reproduce

image = cv2.cvtColor(
    cv2.imread("couple.jpg"),
    cv2.COLOR_BGR2RGB,
)
# Two instances of bounding boxes for faces
boxes = (
    (332, 128, 542, 424),
    (542, 232, 726, 498),
)

# Two instances of bounding boxes for landmarks
keypoints = (
    [
        [410.562, 223.625],
        [482.817, 268.089],
        [436.5, 286.616],
        [364.246, 301.438],
        [443.911, 344.049],
    ],
    [
        [590.205, 329.531],
        [676.795, 337.857],
        [633.5, 381.152],
        [580.214, 417.786],
        [668.469, 429.442],
    ],
)

transform = alb.Compose(
    bbox_params=alb.BboxParams(
        format="pascal_voc",
        label_fields=["category_ids"],
    ),
    keypoint_params=alb.KeypointParams(
        format="xy",
    ),
    p=1,
    transforms=[
        alb.Resize(height=1024, width=1024, p=1),
    ],
)

sample = transform(
    image=image,
    bboxes=boxes,
    category_ids=np.ones(len(boxes)),
    keypoints=keypoints,
)

This script yields the following error:

Traceback (most recent call last):
  File "example.py", line 49, in <module>
    sample = transform(
  File "/python3.10/site-packages/albumentations/core/composition.py", line 207, in __call__
    p.preprocess(data)
  File "/python3.10/site-packages/albumentations/core/utils.py", line 83, in preprocess
    data[data_name] = self.check_and_convert(data[data_name], rows, cols, direction="to")
  File "/python3.10/site-packages/albumentations/core/utils.py", line 91, in check_and_convert
    return self.convert_to_albumentations(data, rows, cols)
  File "/python3.10/site-packages/albumentations/core/keypoints_utils.py", line 140, in convert_to_albumentations
    return convert_keypoints_to_albumentations(
  File "/python3.10/site-packages/albumentations/core/keypoints_utils.py", line 269, in convert_keypoints_to_albumentations
    return [
  File "/python3.10/site-packages/albumentations/core/keypoints_utils.py", line 270, in <listcomp>
    convert_keypoint_to_albumentations(kp, source_format, rows, cols, check_validity, angle_in_degrees)
  File "/python3.10/site-packages/albumentations/core/keypoints_utils.py", line 220, in convert_keypoint_to_albumentations
    check_keypoint(keypoint, rows, cols)
  File "/python3.10/site-packages/albumentations/core/keypoints_utils.py", line 153, in check_keypoint
    if not 0 <= value < size:
TypeError: '<=' not supported between instances of 'int' and 'list'

Expected behavior

This should work, it looks like a "natural" thing to have: to treat keypoints the same way as bounding boxes. Currently I managed to make it work with the following snippet:

sample = transform(
    image=image,
    bboxes=boxes,
    category_ids=np.ones(len(boxes)),
    keypoints=np.asarray(keypoints).reshape(-1, 2),  # Merge keypoints
)
keypoints = np.asarray(sample["keypoints"]).reshape(-1, 5, 2) # Transform/Reshape them back

However this feels like a dirty workaround. Perhaps there should be an easier way to achieve the same, but without bugs

Environment

• Albumentations version (e.g., 0.1.8): 1.3.1
• Python version (e.g., 3.7): 3.10
• OS (e.g., Linux): any
• How you installed albumentations (conda, pip, source): pip
• Any other relevant information: