Train own custom dataset with PoseC3D

[Yu-0920]

The doc issue

I followed the steps here https://github.com/open-mmlab/mmaction2/blob/main/configs/skeleton/posec3d/custom_dataset_training.md When I run python ntu_pose_extraction.py some_video_from_my_dataset.mp4 some_video_from_my_dataset.pkl I get the following error

Traceback (most recent call last): File "tools\data\skeleton\ntu_pose_extraction.py", line 281, in anno = ntu_pose_extraction(args.video, args.skip_postproc) File "tools\data\skeleton\ntu_pose_extraction.py", line 235, in ntu_pose_extraction det_results, _ = detection_inference( File "e:\desktop\mmaction2-main\mmaction\apis\inference.py", line 137, in detection_inference model = init_detector( File "e:\desktop\mmaction2-main\mmdetection\mmdet\apis\inference.py", line 97, in init_detector metainfo = DATASETS.build(test_dataset_cfg).metainfo File "C:\Users\KAY\anaconda3\envs\openmmlab\lib\site-packages\mmengine\registry\registry.py", line 545, in build return self.build_func(cfg, *args, **kwargs, registry=self) File "C:\Users\KAY\anaconda3\envs\openmmlab\lib\site-packages\mmengine\registry\build_functions.py", line 135, in build_from_cfg raise type(e)( KeyError: "class CocoDataset in mmdet/datasets/coco.py: 'mmdet.LoadImageFromFile is not in the transform registry. Please check whether the value of mmdet.LoadImageFromFile is correct or it was registered as expected. More details can be found at https://mmengine.readthedocs.io/en/latest/advanced_tutorials/config.html#import-the-custom-module'" But I have installed the latest version, how can I solve this error.

Then I want to ask, after fixing this problem, I will get a lot of .pkl, how can I merge them and then can I use them for training after merging?

But I don’t have labeled actions in videos, so how can I learn what actions are in training?

Suggest a potential alternative/fix

No response

[cir7]

Hi, @Yu-0920

1. LoadImageFromFile has been removed in mmdet, you could modify mmdet.LoadImageFromFile to LoadImageFromFile in demo/demo_configs/faster-rcnn_r50-caffe_fpn_ms-1x_coco-person.py.
2. you can load the pickle file with mmengine.load and concatenate them as a list, following the specified format.
3. MMAction2 currently only supports supervised training, which means you need to annotate your dataset

[AngusBB]

Hi, @cir7 After modify mmdet.LoadImageFromFile to LoadImageFromFile in demo/demo_configs/faster-rcnn_r50-caffe_fpn_ms-1x_coco-person.py , I got the following error:

Loads checkpoint by http backend from path: https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco-person/faster_rcnn_r50_fpn_1x_coco-person_20201216_175929-d022e227.pth Performing Human Detection for each frame [ ] 0/589, elapsed: 0s, ETA:05/14 05:46:45 - mmengine - WARNING - "FileClient" will be deprecated in future. Please use io functions in https://mmengine.readthedocs.io/en/latest/api/fileio.html#file-io 05/14 05:46:45 - mmengine - WARNING - "HardDiskBackend" is the alias of "LocalBackend" and the former will be deprecated in future. [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 589/589, 33.7 task/s, elapsed: 17s, ETA: 0s Loads checkpoint by http backend from path: https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w32_coco_256x192-c78dce93_20200708.pth C:\Users\Users\anaconda3\envs\mmaction\lib\site-packages\mmpose\apis\inference.py:118: UserWarning: Can not load dataset_meta from the checkpoint or the model config. Use COCO metainfo by default. warnings.warn('Can not load dataset_meta from the checkpoint or the ' C:\Users\Users\anaconda3\envs\mmaction\lib\site-packages\mmpose\datasets\datasets\utils.py:102: UserWarning: The metainfo config file "configs/base/datasets/coco.py" does not exist. A matched config file "C:\Users\Users\anaconda3\envs\mmaction\lib\site-packages\mmpose.mim\configs_base_\datasets\coco.py" will be used instead. warnings.warn( Performing Human Pose Estimation for each frame [ ] 0/589, elapsed: 0s, ETA:C:\Users\Users\anaconda3\envs\mmaction\lib\site-packages\mmcv\transforms\loading.py:69: DeprecationWarning: "file_client_args" will be deprecated in future. Please use "backend_args" instead warnings.warn( [>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>] 589/589, 21.4 task/s, elapsed: 28s, ETA: 0s Traceback (most recent call last): File "tools/data/skeleton/ntu_pose_extraction.py", line 281, in anno = ntu_pose_extraction(args.video, args.skip_postproc) File "tools/data/skeleton/ntu_pose_extraction.py", line 249, in ntu_pose_extraction anno['keypoint'] = np.stack( File "<array_function internals>", line 200, in stack File "C:\Users\Users\anaconda3\envs\mmaction\lib\site-packages\numpy\core\shape_base.py", line 464, in stack raise ValueError('all input arrays must have the same shape') ValueError: all input arrays must have the same shape C:\Users\Users\anaconda3\envs\mmaction\lib\tempfile.py:818: ResourceWarning: Implicitly cleaning up <TemporaryDirectory 'C:\Users\Users\AppData\Local\Temp\tmpn3xcm73n'> _warnings.warn(warn_message, ResourceWarning)

How can I solve this error, thanks for your reply.

[cir7]

We have fixed the script in #2402 , you could try it, and we will merge it into the main branch soon.

[jiaqihuang01]

hi, @cir7 , I have used posec3d to train my own dataset, but during the training and testing processes, I found it use the mmengine.runner function. I want to use the obtained ckt file to predict the input video or pkl file. Is there a way to achieve it? Thanks for your reply