[FR] Add keypoint labels to COCO dataset in FiftyOne dataset zoo

[santhoshkelathodi]

I cannot find documenation for downloading, using and visualizing the COCO keypoint dataset

URL(s) with the issue:

https://voxel51.com/docs/fiftyone/user_guide/dataset_creation/index.html

Description of proposal (what needs changing):

I have been trying to download the Humanpose estimation dataset using Keypoints from coco dataset. I am not able to find documentation and examples to do the same. It would be nice to have documentation for downloading, visualizing and analysing the key point annotations from coco dataset.

Willingness to contribute

The FiftyOne Community encourages documentation contributions. Would you or another member of your organization be willing to contribute a fix for this documentation issue to the FiftyOne codebase?

• [ ] Yes. I can contribute a documentation fix independently.
• [x] Yes. I would be willing to contribute a document fix with guidance from the FiftyOne community.
• [ ] No. I cannot contribute a documentation fix at this time.

[brimoor]

Hi @santhoshkelathodi 👋 If I understand correctly, you're asking two questions:

1. Does the FiftyOne Dataset Zoo contain COCO keypoints?
2. How do I load any keypoint dataset in COCO-format into FiftyOne?

Regarding question 1, the COCO-2017 dataset in the zoo currently supports loading detections and/or instance segmentations, but it doesn't support loading the keypoints data, which is stored in a different set of files that aren't currently downloaded by our zoo dataset: https://github.com/voxel51/fiftyone/blob/79fe485ae8da86a7aef156fdf466f6f7d81a7045/fiftyone/zoo/datasets/base.py#L973

It would be pretty easy to add support for downloading the keypoints data (follow the code path above), however, so if you're interested in adding that option, that'd be awesome! 🤗

The way it would work from an API standpoint is that a user could do this:

import fiftyone.zoo as foz

# label_types = 'keypoints' would be a new option here
dataset = foz.load_zoo_dataset(
    "coco-2017",
    split="validation",
    label_types=["keypoints"],
)

Regarding question 2, fortunately our COCO importer does already support loading keypoints data, so if you manually download a COCO JSON file that contains keypoints, then you can load the keypoints data into FiftyOne like so:

import fiftyone as fo

dataset = fo.Dataset.from_dir(
    data_path="/path/to/images/folder",
    labels_path="/path/to/coco/keypoints.json",
    dataset_type=fo.types.COCODetectionDataset,
    label_types=["keypoints"],
)

[il-nietos]

Hi, is adding support for downloading keypoint data through the Fiftyone Dataset Zoo something that is still coming?

[swheaton]

It is not coming, but we would appreciate a community contribution to do this if you can!

[MatoFD]

Hi, I wanted to use the same COCO keypoints dataset, and while I didn't find or add an official way to do it with dataset zoo, I did find this medium post with detailed steps on how to use the keypoints once they were downloaded. In case it is useful for anyone else :)