How to reproduce the results on the HO-3D dataset

[hxwork]

Hi,

Thanks for making your code publicly available, and this work is really impressive! I have tried to load the pre-trained model and found that its performance on the HO-3D dataset is slightly worse than that reported in the paper. Besides, I am trying to reproduce the results on the HO-3D dataset by re-training the model using this code. However, I have two problems:

1. The training epoch in the config.py is 70, but the pre-trained model is pred80.pth.tar. Does that mean I need to train more epochs than described in the config.py?
2. I also tried to train a new model using the code in this repo without changing anything, but the results of both models trained with 70 and 80 epochs are not consistent with that shown in the paper, as shown in the table below. So I'd like to ask how to reproduce the results.

Best regards, Hao

[heartStrive]

Hello, How long did it take you to train for 80 epochs and what gpu did you use?

[hxwork]

Hi,

It takes ~10 hours to train for 70 epochs and another 1.5h to train for the remaining 10 epochs, and I used 4 Titan V GPUs to train the network.

[namepllet]

We trained our model using 4 2080Ti and it takes ~10 hours.

In our experiments, mean joint error are same at 70 and 80 epochs and other metrics are slightly better at 80 epochs.

We refactored our code after paper submission and there are some difference in results.

[heartStrive]

Hi， I have the same problem with the reproduction accuracy, have you solved it?

[hxwork]

Hi,

Unfortunately, I have no idea to find where the problem is since I have not changed a line of the code (except the directory path of the dataset) and the results are just shown above. Maybe you can compare your results with mine and see if there is any difference between them.

Best, Hao

[heartStrive]

Hi, This is the result of my reproduction.

[hxwork]

Got it. Your results are slightly better than mine, but it looks like there is still a large margin between the results of the re-trained model and those shown in the final version paper. Hope you can find out where the problem is. : )

[heartStrive]

Ok, I will keep you informed if I resolve it. There is another question I'd like to ask. HandOccNet has documented experimental results of the Pose2Mesh model on the HO3D dataset, but I found that the Pose2Mesh paper did not experiment on the HO3D dataset. Do you know the source of this experimental results?

[hxwork]

Thanks. For the question you mentioned above, I am sorry that I don't know. And I notice that you have opened another issue about this. Maybe you should wait for the author's response.

[JT-Sun]

Hi, I trained the model for 70 epoh on the HO3D dataset, but the result sames a far cry from your results. Do anyone know the reason? [email protected] = 0.345 [email protected] = 0.845 best!

[kyxxc]

Hello. Regarding the final [email protected] , [email protected] How is it viewed

[SUOXIAOXI]

Hello, does the HO3D dataset have to be evaluated on the platform when testing, the limited file size on the platform is 300M, but the JSON I got has 500M, what should I do in this situation？