AutoAvatar
AutoAvatar copied to clipboard
AutoAvatar Autoregressive Neural Fields for Dynamic Avatar Modeling
AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling
Ziqian Bai · Timur Bagautdinov · Javier Romero . Michael Zollhöfer · Ping Tan · Shunsuke Saito
ECCV 2022
![Logo](https://github.com/facebookresearch/AutoAvatar/raw/main/assets/teaser.gif)
AutoAvatar is an autoregressive approach for modeling dynamically deforming human bodies directly from raw scans without the need of precise surface registration.
Data Preparation of DFaust
- Create "DFaust" folder under "<workspace_folder>".
cd <workspace_folder>
mkdir DFaust
-
Download SMPL+H parameters of DFaust from AMASS dataset to "<workspace_folder>/DFaust". Unzip to get the "DFaust_67" folder.
-
Download Dfaust scan data from link. Here, we take subject 50002 as an example in the following steps. Unzip data to "<workspace_folder>/DFaust/scans/50002".
-
Download SMPL model from link. Download SMPL meta data from link. Move SMPL related files "basicmodel_m_lbs_10_207_0_v1.0.0.pkl", "basicModel_f_lbs_10_207_0_v1.0.0.pkl", "uv_info.npz", and "smpl_resample_idxs.npz" into "<workspace_folder>/SMPL".
-
Set up AMASS for DFaust data preprocessing. More specifically, download SMPL+H (smplh.tar.xz) and unzip to "<workspace_folder>/SMPL/smplh". Download DMPLs (dmpls.tar.xz) and unzip to "<workspace_folder>/SMPL/dmpls".
-
clone this repo to "<workspace_folder>".
cd <workspace_folder>
git clone https://github.com/facebookresearch/AutoAvatar.git
- Now we should have the following folder structure:
\<workspace_folder\>
├── DFaust
│ ├── DFaust_67
│ │ └── 50002
│ │ └── *.npz
│ └── scans
│ └── 50002
│ └── \<sequences_folders\>
├── SMPL
│ ├── smplh
│ │ ├── female
│ │ ├── male
│ │ └── neutral
│ ├── dmpls
│ │ ├── female
│ │ ├── male
│ │ └── neutral
| └── \<other_SMPL_related_files\>
└── AutoAvatar
Environment Setup
cd AutoAvatar
conda create -n AutoAvatar python=3.8
conda activate AutoAvatar
bash setup.sh
- Create "external" folder and install human_body_prior for DFaust data preprocess.
mkdir external
cd external
git clone https://github.com/nghorbani/human_body_prior.git
cd human_body_prior
python setup.py develop
Data Preprocess
- Run "DFaust_generate.py" to preprocess data. Note that this may take a long time due to the mesh simplification (the open3d API mesh_o3d.simplify_quadric_decimation() in simplify_scans())! Mesh simplification is to speed up data loading during training.
cd AutoAvatar
export PYTHONPATH=<workspace_folder>/AutoAvatar
python data/DFaust_generate.py --ws_dir <workspace_folder>
Train
- Run "implicit_train_dfaust.py" to train the model.
cd AutoAvatar
export PYTHONPATH=<workspace_folder>/AutoAvatar
python exps/PosedDecKNN_dPoses_dHs/implicit_train_dfaust.py --ws_dir <workspace_folder> --configs_path configs/PosedDecKNN_dPoses_dHs/AutoRegr.yaml --configs_path_rollout configs/PosedDecKNN_dPoses_dHs/AutoRegr_Rollout2.yaml
Test
- Run "implicit_eval_dfaust.py" to test the model.
cd AutoAvatar
export PYTHONPATH=<workspace_folder>/AutoAvatar
python exps/PosedDecKNN_dPoses_dHs/implicit_eval_dfaust.py --ws_dir <workspace_folder> --ckpt_dir <checkpoint_folder>
Pretrained Model
- Download pretrained model for DFaust subject 50002 from link.
Publication
If you find our code or paper useful, please consider citing:
@inproceedings{bai2022autoavatar,
title={AutoAvatar: Autoregressive Neural Fields for Dynamic Avatar Modeling},
author={Bai, Ziqian and Bagautdinov, Timur and Romero, Javier and Zollh{\"o}fer, Michael and Tan, Ping and Saito, Shunsuke},
booktitle={European conference on computer vision},
year={2022},
}
License
CC-BY-NC 4.0. See the LICENSE file.