CogVLM
CogVLM copied to clipboard
THUDM
[CogVLM-chat-v1.1] LM weights are different with vicuna-7b-v1.5
While CogVLM is trained, LM weights are fronzen.
From my observation however, the LM weights of cogvlm are different with Vicuna
Vicuna: https://huggingface.co/lmsys/vicuna-7b-v1.5/tree/main CogVLM: cogvlm-chat-v1.1 (both from HF or SAT)
Can I ask why or the proper source of the language model?
- CogVLM-Chat-v1.1 (SAT)
- Vicuna-7B-v1.5
I also tried to train MotionGPT on my custom dataset. It seems that the motions should be transformed to HumanML3D form, which takes the first 22 joints in SMPL-form motions. Then they can be converted to motion features with 263 dimensions. Also, all the texts should follow HumanML3D form. What's more, the dataset might be placed at 'dataset/HumanML3D'. I encountered a problem that the metrics, espacially FID, couldn't be correctly calculated when the dataset is not at 'dataset/HumanML3D'.
@OrigamiStationery Thanks for the reply! How did you convert your custom motion into HumanML3D format/SMPL format? Also what format was your motion capture data stored in? I have my data stored as a .trc file and that is kind of limiting me from converting it into HumanML3D or SMPL formats. Any approach to convert the data to HumanML3D format would be really helpful. Thanks!!
@OrigamiStationery Thanks for the reply! How did you convert your custom motion into HumanML3D format/SMPL format? Also what format was your motion capture data stored in? I have my data stored as a .trc file and that is kind of limiting me from converting it into HumanML3D or SMPL formats. Any approach to convert the data to HumanML3D format would be really helpful. Thanks!!
maybe you can resort to this wonderful repo https://github.com/Dou-Yiming/Pose_to_SMPL. It can convert motion datas represented by joints position .npy file with the format of (nframes, joints_num,3), to SMPL format. As for your own dataset, you may need to define your own kinematic chain in order to use the aforementioned repo.