How to create the same format npzfile for my own dataset

[SizheAn]

Hi,

I'm trying to create my own dataset joint point class, following https://github.com/open-mmlab/mmhuman3d/blob/main/docs/customize_keypoints_convention.md I found the correspondences and created my own .py file for the dataset. The next step is to convert my own data to the .npz file that HumanData supports.

From https://github.com/open-mmlab/mmhuman3d/pull/102, I found the example .npz file. I read it in python and found that it is a type of lib.npyio.NpzFile. It has keys of __key_strict__, __temporal_len__, __keypoints_compressed__, keypoints3d, keypoints3d_mask

Now I have my own 3d joints dataset, the numpy array in dimensional of (N, K, 3), where N is the num_frames, K is the num_joints, and 3 for XYZ axis. I'm wondering how I can create those keys names such as key_strict, temporal_len, keypoints_compressed to make it exactly compatible with other datasets.npz file.

Thanks!

[caizhongang]

Hi @SizheAn , some keys are automatically generated by the HumanData class. @LazyBusyYang may share more details with you.

[LazyBusyYang]

Hi @SizheAn , the keys above are attributes of class HumanData. The data structure offers some checks and utility functions. Following the rules, you can easily create a file that shares the same definition with other npz files. Here's a sample code.

import numpy as np
from mmhuman3d.data.data_structures.human_data import HumanData

# If you have any key which is not in
# HumanData.SUPPORTED_KEYS, set key_strict to False
# to avoid error.
human_data = HumanData.new(key_strict=False)
# After new(), __key_strict__ and __keypoints_compressed__
# have been set automatically.

# Assume that N=100, K=144.
my_own_keypoints = np.zeros(shape=(100, 144, 3))
# If the numpy array in dimensional of (N, K, 3) is
# keypoints2d with confidence, use key 'keypoints2d'.
keypoints2d = my_own_keypoints
# We also need a mask to indicate which keypoint is valid
# if some of the keypoints are bad or place-holders, set them to 0.
keypoints2d_mask = np.ones(
    shape=(my_own_keypoints.shape[1]),
    dtype=np.uint8)
human_data['keypoints2d_mask'] = keypoints2d_mask
human_data['keypoints2d'] = keypoints2d

# If the numpy array in dimensional of (N, K, 3) is
# keypoints3d without confidence, we need to concat conf.
keypoints3d = np.concatenate(
    (my_own_keypoints, np.ones_like(my_own_keypoints[..., 0:1])),
    axis=2)
# mask is same as above.
keypoints3d_mask = np.ones(
    shape=(my_own_keypoints.shape[1]),
    dtype=np.uint8)
human_data['keypoints3d_mask'] = keypoints3d_mask
human_data['keypoints3d'] = keypoints3d
# After __setitem__(), __data_len__, or __temporal_len__ in old version,
# has been set according to keypoints2/3d.

human_data.compress_keypoints_by_mask()
# After compress_keypoints_by_mask(),
# invalid keypoints marked by mask have been
# removed to save space, and __keypoints_compressed__
# is True.
# If you have all ones in mask,
# there's no need to compress.

human_data.dump('human_data.npz')

[SizheAn]

Hi @SizheAn , the keys above are attributes of class HumanData. The data structure offers some checks and utility functions. Following the rules, you can easily create a file that shares the same definition with other npz files. Here's a sample code.

import numpy as np
from mmhuman3d.data.data_structures.human_data import HumanData

# If you have any key which is not in
# HumanData.SUPPORTED_KEYS, set key_strict to False
# to avoid error.
human_data = HumanData.new(key_strict=False)
# After new(), __key_strict__ and __keypoints_compressed__
# have been set automatically.

# Assume that N=100, K=144.
my_own_keypoints = np.zeros(shape=(100, 144, 3))
# If the numpy array in dimensional of (N, K, 3) is
# keypoints2d with confidence, use key 'keypoints2d'.
keypoints2d = my_own_keypoints
# We also need a mask to indicate which keypoint is valid
# if some of the keypoints are bad or place-holders, set them to 0.
keypoints2d_mask = np.ones(
    shape=(my_own_keypoints.shape[1]),
    dtype=np.uint8)
human_data['keypoints2d_mask'] = keypoints2d_mask
human_data['keypoints2d'] = keypoints2d

# If the numpy array in dimensional of (N, K, 3) is
# keypoints3d without confidence, we need to concat conf.
keypoints3d = np.concatenate(
    (my_own_keypoints, np.ones_like(my_own_keypoints[..., 0:1])),
    axis=2)
# mask is same as above.
keypoints3d_mask = np.ones(
    shape=(my_own_keypoints.shape[1]),
    dtype=np.uint8)
human_data['keypoints3d_mask'] = keypoints3d_mask
human_data['keypoints3d'] = keypoints3d
# After __setitem__(), __data_len__, or __temporal_len__ in old version,
# has been set according to keypoints2/3d.

human_data.compress_keypoints_by_mask()
# After compress_keypoints_by_mask(),
# invalid keypoints marked by mask have been
# removed to save space, and __keypoints_compressed__
# is True.
# If you have all ones in mask,
# there's no need to compress.

human_data.dump('human_data.npz')

Thanks a lot for the quick response! @caizhongang @LazyBusyYang Will take a look soon and close the issue if it helps.

[caizhongang]

This doc may be helpful also.