test diff

[2050airobert]

hi， 1 The occlusion keypoints (vis = 1) in the coco-wholebody dataset have participated in the training , right ? 2 If I want to exclude the occlusion keypoints (vis = 1) in the training, which code need to be improved? 3 In fact ，when testing the model below， I find it shows poor results or effect regarding to the occlusion keypoints(just like hand keypoints and feet keypoints) on my own video file. How to improve the result with the occlusion keypoints detection? Any usefull idea ？

4 To my surprise，the points on the face shows consistent good results. Although there is a problem in detection when occlusion occurs, the detection points are still around the face, and the deviation is not very large, just as the following picture.

The config file : configs/wholebody_2d_keypoint/topdown_heatmap/coco-wholebody/td-hm_hrnet-w48_dark-8xb32-210e_coco-wholebody-384x288.py The cpkt ； https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_wholebody_384x288_dark-f5726563_20200918.pth

BR

[Ben-Louis]

Hi, thanks for using MMPose. For your questions,

1. Yes

2. You can change this line from

   keypoints_visible = np.minimum(1, _keypoints[..., 2] > 0)
   

   to

   keypoints_visible = np.minimum(1, _keypoints[..., 2] > 1)
   

3. If you only want to improve the visualization results, you could increase the value of --kpt-thr to filter out the occluded joints which generally have low keypoint scores. If you want to improve the model robustness on occluded joints, some data augmentation methods (e.g. random erase) might help. Maybe @jin-s13 could provide more helpful advice.

4. Face is more rigid than the body and hand. So models can learn the face structure and infer the invisible keypoints.

[2050airobert]

@jin-s13 @Ben-Louis 1 Regarding to question 3 above ，is there detailed config or examples of augmentation methods(e.g. random erase) available ? 2 Regarding to question 3 above ，you mean the trick and detailed method for training the points of face , hand and body is the same, right?

[jin-s13]

We support using albumentations, you may try it https://mmpose.readthedocs.io/en/latest/papers/techniques.html#albumentations-information-2020

[2050airobert]

tks hi, @Ben-Louis 2 Regarding to question 4 above ，you mean the trick and detailed method for training the points of face , hand and body is the same, right?

[Ben-Louis]

Yes. In my perspective, it is a characteristic of face keypoints data and is independent of the model. But I am not sure if there is any exception.

[ly015]

Hi, we will close this issue for now. Please feel free to reopen it if there are further questions.