How to interpolate between two 2D pose-frames (each containing 18 joints)

[mmdrahmani]

Hi. I have a question regarding motion in-between-ing.

I have two images (frames) and I extracted YX positions of 18 joints (see the figure). I would like to use these two images as input_motions, one image as starting pose, and the other image as end pose. And then by motion editing I'd like to generate a few pose-frames in between these two frames. I wonder if this is possible, because currently the sample/edit.py, the input_motions has a shape of 1,263,1,196(n_frame). I do not how to transform my 18-joint XY coordinates into this 263 input shape.

Any suggestions are much appreciated. Thanks

[mayank64ce]

I tried with converting a single frame to HumanML3D format (by keeping the next frame as all zeros). But I don't think it worked :( .

[mmdrahmani]

@mayank64ce thanks for the comment. How did you convert 2D skeleton data into HumanML3D format? I'd appreciate it.

[mayank64ce]

I did not convert 2d skeleton. Rather SMPL parameters I got from image using SMPLify-X.

In this file : https://github.com/EricGuo5513/HumanML3D/blob/main/raw_pose_processing.ipynb , in the fucntion amass_to_pose I just padded every attribute like root_orient, pose_body etc. with zeros of same shape. Basically making a motion sequence of only 2 frames with the 2nd frame with dummy values. Also you will need to edit index.csv accordingly.

But again, I don't think it gave correct results as I have no way of verifying it .

[mmdrahmani]

Thanks a lot. I see. BTW, I think it is possible to verify the results by visualizing/rendering the generated motion. There are functions provided for that, and the code by default saves the videos of the generated motions.