ChemLLMBench icon indicating copy to clipboard operation
ChemLLMBench copied to clipboard

question about pipeline code &dataset

Open evergreen-bupt opened this issue 1 year ago • 3 comments

Hello, I am trying to test the performance of my model on chemllmbench, but I meet some issues as following:

  1. For property prediction tasks, there is a lack of code for calculating the AUC-ROC metric.
  2. For Name Prediction task, it seems that the previous dataset download location has become invalid. Could you help me resolve these issues?

evergreen-bupt avatar Apr 18 '24 13:04 evergreen-bupt

  1. For Name Prediction task, the download link is updated and valid.

taichengguo avatar Jul 26 '24 00:07 taichengguo

  1. For property prediction tasks, @KehanGuo2 can you help fix this issue?

taichengguo avatar Jul 26 '24 00:07 taichengguo

With the "generated_p", which is "Yes"/"No", you can easily calculate the AUROC score

KehanGuo2 avatar Jul 26 '24 06:07 KehanGuo2