fine-tune model

[WESLEY-ZHOU3137]

As mentioned in <Active Transfer Learning for Efficient Video-Specific Human Pose Estimation>, the Posetrack21 dataset is used to pre-train SimpleBaseline and FastPose, followed by fine-tuning the two pre-trained models using the JRDB dataset. But I recently used the model of fastpose or simplebaseline which had used dataset of posetrack to pretrain to fine-tune by the dataset of jrdb.The result of all method ,such as THC+WPU+DUW, the APs is near zero in the Round 0. So I want to know if I should only use the same dataset to fine-tune the pretrained model ?

[ImIntheMiddle]

Thanks for sharing! I think that is because Posetrack21 and JRDB have different skeleton definitions. When you fine-tune with a different keypoint structure, directly fine-tuning all weights doesn't work well. You might need to only transfer the backbone weights and initialize the keypoint prediction heads, rather than fine-tuning the entire model.

[ghost]

So I should set the path of the Posetrack21 pretrained model to MODEL.TRY_LOAD in configs/fastpose_jrdb.yaml, run jrdbpose_train.py, and then run run_active_learning.sh—is that correct?

[ImIntheMiddle]

Yes, that's correct, or as in the case of our paper, you can just run jrdbpose_train.py without a pretrained model and then use the trained model in run_active_learning.sh.

[WESLEY-ZHOU3137]

I have tried fine-tune the model using the method, but the [email protected] is only 0.3 in the round 0. While this result is an improvement over my previous attempts, it still falls short of the performance reported in the paper. I used the dataset of posetrack to train 'pretrained_model/posetrack21/fastpose_256x192.pth' with the 'posetrack_train.py', and then I used the 'jrdbpose_train.py' to fine-tune the pretrained model that I got the result.

[ImIntheMiddle]

Thanks, there might be two reasons for this:

1. Please ensure that your training setup follows the procedure described in Section 5.1 (HP Estimator).
2. AP can vary significantly across individual test videos in ATL. Note that our paper reports the average performance over 15 test videos. Your result might be from a particularly challenging video, so please check the average performance across all test videos?

[WESLEY-ZHOU3137]

First, I use the 'posetrack_train.py' to pretrained the fastpose by dataset of posetrack21(pretrained_models/posetrack21/fastpose_256x192.pth) and save to the 'exp/initial-fastpose_posetrack21/model_best.pth' . Second, I use the 'jrdbpose_train.py' to fine-tune the pretrained model( 'exp/initial-fastpose_posetrack21/model_best.pth') and I set the path of model to the param of TRY_LOAD in the configs. Third, using the run_active_learning.sh.

I want to know what is the correct process？Can you tell me ?

[ImIntheMiddle]

Yes, your training process is correct. Could you share the average performance on test videos? It might also be helpful to confirm that the released pre-trained model works correctly ([email protected] is around 0.56 at the round 0) in your setup.

[WESLEY-ZHOU3137]

ATL_shell_log.txt jrdb_training.log fastpose_jrdb22.yaml al_simple_jrdb.yaml fastpose_posetrack21.yaml posetrack_training.log run_active_learning.sh

After I pretrained model by all of datasets, I use 'python scripts/run_active_learning.py --cfg configs/jrdb-pose/al_simple_jrdb.yaml --uncertainty THC+WPU --representativeness None --filter Coreset --video_id 00 --memo WACV_JRDB --seedfix --continual' to start, but the [email protected] is 0.11 in round 0.And I find that after running the jrdbpose_train.py , the [email protected] of this model is only around 0.1(jrdb_training.log) .

[ImIntheMiddle]

Thank you for sending the log files. Normally, as the OKS threshold increases, the evaluation becomes stricter and AP values should decrease, but your results don't. Therefore, there may be an error in the AP calculation.

I'll check the results on my end as well later, but I'm currently approaching a deadline and don't have much time available. So could you share further details if you find anything? Thanks!

[WESLEY-ZHOU3137]

I think that maybe an error in the AP calculation, because the function uses the COCO format for evaluation, while the jrdb-pose dataset has keypoint annotations that are different from the COCO format. And I download the posetrack21 dataset from "https://github.com/andoer/PoseTrack21" in the Posetrack21 paper, but the paper describes the number of keypoints is 15 ,but it's 17.So maybe adjust the jrdb-pose dataset format or change the function to evaluate the model is a good way.

000043_mpii_train.json