SynergyNet icon indicating copy to clipboard operation
SynergyNet copied to clipboard

Published 20 hours ago • verified publisher icon choyingw

training accuracy 3.41？

Open liqidong opened this issue 2 years ago • 5 comments

I followed the training steps, but my final accuracy was only more than 3.8. How can I reproduce the accuracy of 3.41?

liqidong avatar Mar 24 '22 02:03 liqidong

We chose the best checkpoint among all epochs. You can save more checkpoints during training and evaluate them to find the best one.

choyingw avatar Mar 25 '22 21:03 choyingw

I tried loading the pre-trained model and saving each batch. The highest accuracy is about 3.58, but it cannot reach 3.41. At the same time, I load the best model (3.41) for iterative training, and usually the accuracy will decrease. How to solve this problem?

liqidong avatar Apr 19 '22 02:04 liqidong

Dear all,

I am interested in the SynergeNet. I also encountered the issue and wonder if @liqidong solved the issue.

As far as @choyingw mentioned above, I saved all checkpoints every epochs and also saved the checkpoint corresponding to best performance in terms of NME. However, I could not reproduce the result (i.e. AFLW2000-3D-All-NME: 3.41) unforuntaetly. In my case, the lowest NME is 3.628.

I understand there may be some error between orinals and mine because of random seed, although I set fixed seed. Thus, there is two questions.

  1. Is the released code in this repository orinal code corresponding to the paper, Synergy between 3DMM and 3D Landmarks for Accurate 3D Facial Geometry? In the paper, the experiments are conducted on four NVIDIA GTX 1080Ti, whereas the experiment may be performed with a single NVIDIA GTX 3090 in README.md in this reposiory. Also, unfortunately the performance of the released pretrained model is not qual to that of the paper.

  2. As far as I mentioned question 1, I think the uploaded checkpoints is not the checkpoint corresponding to the official paper. Could I get the official checkpoint which can makes same results as the recorded in the paper?

Best regards, Vujadeyoon

vujadeyoon avatar Oct 27 '22 01:10 vujadeyoon

This is the official repo, and the released checkpoints correspond to the results we reported in the paper.

The code base we test was based on available newer computing arch, and the computing arch shouldn’t affect the results

choyingw avatar Nov 04 '22 05:11 choyingw

In my reproduce procedure, the whole training process is very unstable. And the NME also reaches 3.6 with mobilenet_v2 backbone.

Could you please share the training log file about the 3.41 nme? @choyingw

starhiking avatar Mar 27 '23 06:03 starhiking