NVlabs
How are the predicted 54 joint points converted into the pose parameters required for SMPL?
When i test GLAMR on a single-person video with dynamic camera by running this command:
python global_recon/run_demo.py --cfg glamr_static \ --video_path assets/static/basketball.mp4 \ --out_dir out/glamr_static/basketball \ --save_video
i found an error on https://github.com/NVlabs/GLAMR/blob/cc752f185c9f7e83b6dbf027f0b534d61791f8e7/global_recon/models/global_recon_model.py#L103:
Exception has occurred: ValueError(note: full exception trace is shown but execution is paused at: _run_module_as_main)
cannot reshape array of size 47700 into shape (69)
and the shape of smpl_pose_wroot is 300x54x3. But the shape 69 looks like 23x3.
i try to select the former 24 joints as the param SMPL need like: new_dict['smpl_pose'] = smpl_pose_wroot[:, 1:24].reshape(-1, 69) but the vis result looks very strange.
so i want kown how are the predicted 54 joint points converted into the pose parameters required for SMPL?
This probably happens because you use the hrnet version of HybrIK which produces 3x3 rotation matrices, not quaternions (i.e. the dimension you get from HybrIK is 216, which correspond to 24 3x3 rotation matrices, not 54 quaternions). It seems to work for me if I do this instead:
from scipy.spatial.transform import Rotation as R
hybrik_poses_rot = pose_dict['smpl_pose_quat_wroot']
batch_size = hybrik_poses_rot.shape[0]
smpl_pose_wroot = R.as_rotvec(R.from_matrix(hybrik_poses_rot.reshape((-1, 3, 3)))).reshape((batch_size, -1, 3))
smpl_pose_wroot = smpl_pose_wroot.astype(np.float32)
new_dict['smpl_pose'] = smpl_pose_wroot[:, 1:].reshape(-1, 69)
