R2-LLM icon indicating copy to clipboard operation
R2-LLM copied to clipboard

Reproduction Issue – Results Significantly Lower Than Reported

Open LiGuo12 opened this issue 10 months ago • 4 comments

Hi,

I have attempted to reproduce your model using the code and hyperparameters provided in the repository and paper. However, my results for both stage 1 and stage 2 are significantly lower than the values reported in your paper. Despite strictly following the given settings, I have been unable to achieve similar performance.

To ensure the reproducibility of the reported results, would you be willing to release the pretrained weights? This would help verify the reliability of the reported values.

LiGuo12 avatar Feb 03 '25 23:02 LiGuo12

hi @LiGuo12 , could you please provide the link for pre trained weights?

satheeshKOLA532 avatar Feb 04 '25 12:02 satheeshKOLA532

Hi,

I have attempted to reproduce your model using the code and hyperparameters provided in the repository and paper. However, my results for both stage 1 and stage 2 are significantly lower than the values reported in your paper. Despite strictly following the given settings, I have been unable to achieve similar performance.

To ensure the reproducibility of the reported results, would you be willing to release the pretrained weights? This would help verify the reliability of the reported values.

Hello, may I ask if you can provide Public Medical Data? I don't seem to have found it in the manuscript.

LiLiXJTU avatar May 26 '25 11:05 LiLiXJTU

Hi, I have attempted to reproduce your model using the code and hyperparameters provided in the repository and paper. However, my results for both stage 1 and stage 2 are significantly lower than the values reported in your paper. Despite strictly following the given settings, I have been unable to achieve similar performance. To ensure the reproducibility of the reported results, would you be willing to release the pretrained weights? This would help verify the reliability of the reported values.

Hello, may I ask if you can provide Public Medical Data? I don't seem to have found it in the manuscript.

Your are right, the authors did not provide the public medical data. I generated it by myself from Pubmed. My results are much lower than the values in the manuscript, so I have abandoned this approach.

LiGuo12 avatar Jul 25 '25 17:07 LiGuo12

hi @LiGuo12 , could you please provide the link for pre trained weights?

Sorry, I did not save the ckpt.

LiGuo12 avatar Jul 25 '25 17:07 LiGuo12