can not get good result when training from Scratch

[renrenzsbbb]

Thanks for your great work. I use https://github.com/ubisoft/ubisoft-laforge-animation-dataset to train from sractch. the reading bvh data code is referred by

from fairmotion.data import bvh

motion = bvh.load(BVH_FILENAME)

positions = motion.positions(local=False)  # (frames, joints, 3)
velocities = positions[1:] - positions[:-1]
orientations = motion.rotations(local=False)[..., :, :2].reshape(-1, 22, _6)

I only adjust 22 to 21 because new. dataset has 21 joint. here is my last epoch logger 237m 45s (- 0m 00s) (140 100.0%) | Recon: 1.618e-02 | KL: 2.287e-07 | PP: 0.000e+00; And then, I play it by gym with your code in random work mode. the result is not normal, can you give me some advices.

[renrenzsbbb]

https://user-images.githubusercontent.com/30514838/168982267-bbcdaa5f-4c60-4962-b1a2-c0d118967742.mp4

[edentliang]

Hi, I am also doing the same thing like you. But I am still struggling with the mocap.npz : 0-3 : root delta x, delta y, delta facing. How do u get these data?

[belinghy]

@renrenzsbbb Thanks for taking interest in our work. For the dataset you are using, you will need to do some hyperparameter tuning. The recon loss should be on the order of 1e-3 or 1e-4, and the KL loss should be roughly on the same scale. You could try tuning the beta of the VAE.