seq_ppi icon indicating copy to clipboard operation
seq_ppi copied to clipboard

What should I do if I want to reproduce the cross-validation corr mentioned in the paper as 0.873

Open zff1116 opened this issue 2 years ago • 1 comments

    Thank you for your work and code. I have some questions about the results of the reproducing work. I want to repeat your result on binding affinity part. I downloaded SKEMPI dataset and used seq_ppi/regression/model/run.sh for cross-validation after change the  ds_file to SKEMPI dataset.I found that the obtained results did not correspond to those in the paper. After reading the code and the paper, it seems that the folder for the regression task, and the entire github, is not the code used in the paper or that I missed something, For example The paper mentions 10-fold cross validation, while the code uses 5-fold . **What should I do if I want to reproduce the cross-validation corr mentioned in the paper as 0.873** ? Can you help me with this problem?

zff1116 avatar Nov 30 '22 07:11 zff1116

I have the same problem. I also checked the updated rcnn.py which uses 10-fold cross validation. In my case, both models with hidden_dim=25 or hidden_dim=50 cannot reproduce the correlation mentioned in the paper. When hidden dim = 25 mae=[0.13313971] mse=[0.02906957] corr=0.4698897625579699

When hidden dim = 50 mae=[0.12357073] mse=[0.02669512] corr=0.4273866044136481

The run.sh in regression/model is used.

zguo235 avatar Feb 15 '23 08:02 zguo235