ihoi
ihoi copied to clipboard
What's in your hands? 3D Reconstruction of Generic Objects in Hands
Yufei Ye, Abhinav Gupta, Shubham Tulsiani in CVPR 2022
Our work aims to reconstruct hand-held objects given a single RGB image. In contrast to prior works that typically assume known 3D templates and reduce the problem to 3D pose estimation, our work reconstructs generic hand-held object without knowing their 3D templates.
[Project Page] [Video] [Colab Demo] [Demo Code] [Arxiv]
Installation
See install.md
Quick Start
-
Step by step interactive notebook
-
Or python script
python -m demo.demo_image --filename demo/test.jpg --out output/ -e weights/
-
Step by step interactive notebook
-
Or python script
python -m demo.demo_image --filename demo/test.jpg --out output/ -e weights/mow/
We also provide some other images docs/demo_%02d.jpg
for you to play around.
Train your own model
Preprocess data
preprocess.md
(Coming Soon)
Start training
# obman
python -m models.ihoi --config experiments/obman.yaml --slurm
# finetune
python -m models.ihoi --config experiments/mow.yaml --ckpt PATH_TO_OBMAN_MODEL/obman/checkpoints/last.ckpt --slurm
python -m models.ihoi --config experiments/ho3d.yaml --ckpt PATH_TO_OBMAN_MODEL/obman/checkpoints/last.ckpt --slurm
Citation
If you use find this code helpful, please consider citing:
@article{ye2022hand,
author = {Ye, Yufei
and Gupta, Abhinav
and Tulsiani, Shubham},
title = {What's in your hands? 3D Reconstruction of Generic Objects in Hands},
booktitle = {CVPR},
year={2022}
}