How do I get the ID and XY coordinate values for each joint of the animal?

[SpreadKnowledge]

Hello, I use mmpose all the time!

I'm currently trying to estimate the posture of an animal using "top_down_video_demo_with_mmdet.py". After moving the model, how can I save the ID (site) of each joint (keypoint) of the animal and its coordinates? I'm having trouble finding an option in the command that outputs the IDs of the joints and their coordinates.

I would like to use mmpose to detect specific poses of animals in videos. To do this, I need to have the IDs and coordinates information of each joint.

If anyone knows how to do this, I would like to know. Thank you in advance.

[Ben-Louis]

Hi, thanks for your support for long. The demos in MMPose 1.0 have supported saving predicted keypoints.

• To use the new version, please follow the installation guideline. Notice that you have to use the branch dev-1.x instead of 1.x in "Step 1. Install MMPose" if you want to try this new feature of demo.
• After installation, you could run demo on animal data following 2d_animal_demo. The argument for saving the outputs is --save-predictions

[SpreadKnowledge]

Thank you for your support.

I have run the mmpose setup again using the URL you gave me. However, when I used the "-save-predictions" option, I got the following error

topdown_demo_with_mmdet.py: error: unrecognized arguments: --save-predictions

The command I used is as follows

python demo/topdown_demo_with_mmdet.py \
    ./demo/mmdetection_cfg/faster_rcnn_r50_fpn_coco.py \
    ./models/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth \
    ./configs/animal_2d_keypoint/topdown_heatmap/animalpose/td-hm_hrnet-w32_8xb64-210e_animalpose-256x256.py \
    ./models/hrnet_w32_animalpose_256x256-1aa7f075_20210426.pth \
    --input ./image_data/cow_001.png \
    --show \
    --output-root ./results/wahcom_cow_001 \
    --save-predictions \
    --bbox-thr 0.3 \
    --kpt-thr 0.4 \
    --det-cat-id 19

[Ben-Louis]

Could you please provide the information about the branch you use via git branch? If you are using branch 1.x now, you could switch to branch dev-1.x and try again.

[SpreadKnowledge]

The branch I used was "1.0 x", so I changed it to "dev-1.x". We were then able to output the coordinates of the keypoints to json.

By the way, those coordinates didn't have the IDs of the joints. mmpose can further record which coordinate is the ID of which joint in the json? I need to know which key points were at which joints in order to ultimately identify what posture the cows are in.

[
	{
		"keypoints": [
			[
				1546.2182006835938,
				917.546875
			],
			[
				1522.4631958007812,
				917.546875
			],
			[
				1546.2182006835938,
				751.2618408203125
			],
			[
				1506.6265258789062,
				798.7718505859375
			],
			[
				1522.4631958007812,
				1044.240234375
			],
			[
				1372.0148315429688,
				830.4451904296875
			],
			[
				801.8947143554688,
				323.6717529296875
			],
			[
				1395.7698364257812,
				545.3851318359375
			],
			[
				1197.8114624023438,
				798.7718505859375
			],
			[
				1229.4848022460938,
				782.9351806640625
			],
			[
				809.8130493164062,
				577.0584716796875
			],
			[
				857.3230590820312,
				577.0584716796875
			],
			[
				1166.1381225585938,
				814.6085205078125
			],
			[
				1253.2398071289062,
				893.7918701171875
			],
			[
				968.1797485351562,
				751.2618408203125
			],
			[
				960.2614135742188,
				751.2618408203125
			],
			[
				1150.3014526367188,
				870.036865234375
			],
			[
				1134.4647827148438,
				862.1185302734375
			],
			[
				984.0164184570312,
				814.6085205078125
			],
			[
				984.0164184570312,
				830.4451904296875
			]
		],
		"keypoint_scores": [
			0.3911924362182617,
			0.5391932129859924,
			0.5101831555366516,
			0.7085485458374023,
			0.8178417682647705,
			0.8313236832618713,
			0.8531842827796936,
			0.5749420523643494,
			0.5585530400276184,
			0.7991195917129517,
			0.2036595344543457,
			0.2527989447116852,
			0.3217814564704895,
			0.830315351486206,
			0.2486996352672577,
			0.6525574922561646,
			0.47543540596961975,
			0.29314643144607544,
			0.40189871191978455,
			0.7818695306777954
		],
		"bbox": [
			[
				740.923583984375,
				268.0306701660156,
				1551.7611083984375,
				1004.8612670898438
			]
		],
		"bbox_score": 1.0
	}
]

[Ben-Louis]

The meaning of keypoints in the output json file is defined in the dataset metainfo config file. For example, since the model you use is trained on animalpose, the content of keypoint_info in configs/_base_/datasets/animalpose.py defines the mapping between keypoints order and its meaning.

We are planning to add the information about keypoints meaning into the saved json file. Thank you very much for your feedback. And if you are interested in supporting this new feature, please let us know. We are glad to help.

[SpreadKnowledge]

Thank you, first I'll refer to "aimalpose.py" to find out which keypint is which joint.

Classification of posture is a very important task, as well as detecting the position of each keypoints. I hope that new features will be added to mmpose in the future to make the system more practical.