QCNet icon indicating copy to clipboard operation
QCNet copied to clipboard

Questions about result reproducibility.

Open SkylerHuang opened this issue 1 year ago • 1 comments

@ZikangZhou Hi, I followed the script provided in your readme file to reproduce the results, but encountered two issues. Firstly, when I set the batch size to 4 on my 3090 GPU, I encountered out of memory problem. So I adjusted the batch size to 2, while keeping the other hyperparameters unchanged. The resulting reproduction shows that minade=0.732 and minfde=1.294, which has a small deviation from the results reported in the paper. However, there is a significant difference from the pre-trained model that you uploaded, where minade=1.25. I would like to ask if the hyperparameters or other training configurations of the pre-trained model in this version have been changed? Thank you very much.

SkylerHuang avatar Jul 29 '23 04:07 SkylerHuang

No, I didn't change the hyperparameters except that I use different GPUs for training. As you have changed the batch size, I feel that such a degree of deviation in performance is acceptable.

ZikangZhou avatar Jul 29 '23 10:07 ZikangZhou