Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis

CVPR, 2022
Lixin Yang * . Kailin Li * · Xinyu Zhan · Jun Lv · Wenqiang Xu · Jiefeng Li · Cewu Lu
\star = equal contribution

This repo contains models, train, and test code.

TODO

• [ ] installation guideline
• [x] testing code and pretrained models (uploading)
• [ ] generating CCV-space
• [ ] training pipeline

Installation

Following the Installation Instruction to setup environment, assets, datasets and models.

Evaluation

Heatmap-based model, HO3Dv2

$ python train/submit_reload.py --cfg config_eval/eval_ho3dv2_clasbased_artiboost.yaml --gpu_id 0 --submit_dump

This script yield the [Ours Clas + Arti] results in main paper Table 2. HO3Dv2 codalab submission file will be dumped at common/eval_ho3dv2_clasbased_artiboost_SUBMIT.json

Regression-based model, HO3Dv2

$ python train/submit_reload.py --cfg config_eval/eval_ho3dv2_regbased_artiboost.yaml --gpu_id 0 --submit_dump

This script yield the [Ours Reg + Arti] results in main paper Table 2.

Heatmap-based model, HO3Dv3

$ python train/submit_reload.py --cfg config_eval/eval_ho3dv3_clasbased_artiboost.yaml --gpu_id 0 --submit_dump

This script yield the [Ours Clas + Arti] results in main paper Table 5.

Heatmap-based model, HO3Dv3, Symmetry model

$ python train/submit_reload.py --cfg config_eval/eval_ho3dv3_clasbased_sym_artiboost.yaml --gpu_id 0 --submit_dump

This script yield the [Ours Clas sym + Arti] results in main paper Table 5.

DexYCB ...

Generate CCV

Training Pipeline

MANO Driver

Acknowledge & Citation

@inproceedings{li2021artiboost,
    title={{ArtiBoost}: Boosting Articulated 3D Hand-Object Pose Estimation via Online Exploration and Synthesis},
    author={Li, Kailin and Yang, Lixin and Zhan, Xinyu and Lv, Jun and Xu, Wenqiang and Li, Jiefeng and Lu, Cewu},
    booktitle={arXiv preprint arXiv:2109.05488},
    year={2021}
}