HRNET+RLE+SIMDR not work!

[2050airobert]

Hi， If I want to do train different models with the combination of different technique or module， how could do improve the code or config of mmpose? For example, there are some trial as following: 1 deep pose with DLA + RLE + IPR + DSNT
2 deep pose with resnet + RLE + SIMDR 3 hrnet with resnet + simdr + RLE 4 hrnet with resnet + RLE + IPR + DSNT
etc. Is there any example to show how to complete these trial above? Do I need to change some source code of mmpose? Or all I need to do is that to change the config file for training (eg. https://github.com/open-mmlab/mmpose/tree/1.x/configs/body_2d_keypoint/topdown_heatmap/coco)? Looking forward to your reply ~ BR

[Tau-J]

Hi @2050airobert ，thanks for using MMPose. For IPR/DSNT+RLE, you just need to change the loss function in config, because all targets you need are generated by codec IntegralRegressionLabel. But for SimCC(SimDR)+RLE, we do not provide a soft-argmax operation in SimCC so that you have to implement a new head to conduct such processing. In addition, you can change backbones directly in config.

[2050airobert]

hi @Tau-J 1 What do you mean about "codec IntegralRegressionLabel", is it this some OP in the config file or config python file just like "configs/body_2d_keypoint/topdown_heatmap/coco/td-hm_hrnet-w48_8xb32-210e_coco-256x192.py"? 2 Is it so hard that to add the oft-argmax operation in SimCC , if not so why not? So you mean there is HRNET+simcc ，yet not HRNET+simcc + RLE ,right ? 3 If I want to change the backbones of the deeppose , simplebaseline or hrnet , all I need to do is change the code in the config python file? 4 Is the hrnet + RLE + IPR + DSNT available now？ If so， how to run it easily？ tks

BR

[Tau-J]

@2050airobert

1. MMPose 1.0 introduced a new module Codec to integrate the encoding and decoding together in a modular and user-friendly form. You can refer to this page for more details.
2. Soft-argmax is easy to implement, but it's not a part of original SimCC method. Yes, you can train HRNet+SimCC by setting backbone in config, but you have to implement a new head yourself if you want to use HRNet+SimCC+RLE, due to RLE requires a coordinate-to-coordinate supervision but SimCC is trained in a vector-to-vector manner.
3. Yes
4. Change backbone, codec and loss in config to combine HRNet, IntegralRegressionLabel and RLELoss. Besides, use a proper head

[2050airobert]

Sound nice kit! Is there two config file as following, as it's not a long time to use this kit? 1 Could you help me to show how to change the simcc_vipnas-mbv3_8xb64-210e_coco-256x192.py to support hrnet + simcc/dr , or share the simcc_hrnet-xxx_coco_256x192.py file ? 2 How about hrnet + RLE + IPR + DSNT ? Could you please share the hrnet_rle_ipr_dsnt_coco-256x192.py file ? Very thanks for you !

[Tau-J]

For new users, we prepare a tutorial, therefore, I suggest referring to this tutorial to learn how to use MMPose 1.0. If you find any part of the tutorial not easy to understand, please let me know, which will help us to improve.

[darcula1993]

seems that we need some code hack to do deeppose+IPR+RLE+DSNT

RLE take 3 parameters as input (pred,sigmas,target) and DSNT head do not provide sigmas for the RLE loss.