Not reproducible numbers!

[hongsukchoi]

Hi, thanks for sharing your great work.

However, I think it would be nice of you to provide a proper training code for the reproducibility. As you know, the Mosh SMPL parameter data from Human3.6M and NBA data used in HMMR(2019 CVPR) are not available because of the license issues. Therefore, actually none of the numbers in your paper are reproducible due to the unavailable training datasets.

So, why don't you provide reproducible numbers based on available training datasets? It would be a nice guideline for other researchers who want to reproduce your work and many others.

[mkocabas]

Hi @hongsukchoi,

Thanks for your suggestion! Your point about availability of Human3.6M and NBA is correct. Therefore, we could not use NBA for training. We needed to use Human3.6M mosh data to be able to compare with previous works in a fair setting by the time of submission. But as you pointed out, mosh data is not widely available. However, without using Human3.6M during training, results on 3DPW and MPI-INF-3DHP are still reproducible.

I will make sure to release a model which is trained with publicly available datasets only. Also, as a side note, there was a mistake I made for the "VIBE with 3DPW training" setting, we fixed this and an arxiv update will be available in a few days.

[hongsukchoi]

@mkocabas Indeed, I can get similar numbers without H36M during training if I don't use Penn Action dataset. I think Penn Action data is hindering the training.

BTW, I checked the evaluation code and found that you are feeding the model with no overlapped sequences. Once I put the overlap (stride=1) between sequences via the dataloader, I got better performance in Accel Error.

[Seethevoice]

Dear @hongsukchoi,

Could you please give a more detailed description of how to modify the dataloader to make Accel Error better?