Questions about reproducibility

[josecannete]

Hello, thank you so much for your work!

I tried to reproduce your results and I'm not able to do it so far. I followed the instructions for QCR on the SLAKE dataset and my results show almost the same overall accuracy for CLIP and PubMedCLIP (using your checkpoints) and even a little lower for PubMedCLIP, both around 79% overall accuracy. Any advice?

Also, I tried to train my own PubMedCLIP using your code (following the instructions on the PubMedCLIP subdirectory) and it seems to be overfitting in the first 5 epochs for every model, was it similar in your experiments? These models also performed similar to CLIP and your checkpoints of PubMedCLIP on QCR-SLAKE.

Thanks