BioREx icon indicating copy to clipboard operation
BioREx copied to clipboard

Can you proivde a script or an explanation to reproduce scores in your paper?

Open dongheechoi opened this issue 1 month ago • 2 comments

In the paper (https://arxiv.org/abs/2306.11189), you wrote the scores below. Can you kindly provide a way to reproduce this?

image image

For example, with the model you provided in the repo BioREx PubMedBERT model (Original) and BioREx BioLinkBERT model (Preferred), what score can I get? And how can I get the score?

When I run with BioREx PubMedBERT model (Original) using the code you suggest bash scripts/run_test_pred.sh, I got

Overall 966 652 263 314 0.7125683060109289 0.6749482401656315 0.6932482721956407 in the file locaed in "out_result_file" parameter. I think it would be precision, recall, f1 score, but then I am not sure I can get 79.6 in this case(BioRED+8 datasets in your paper).

If I misunderstood something, please let me know. And again, if you provide a specific parameters to reproduce the scores in the paper (including the baseline approaches like TL(Transfer learning) or MTL(Multi-Task Learning), it would be great help for me as well.

dongheechoi avatar May 15 '24 12:05 dongheechoi