BY571 comments

Repositories
Issues
Comments

Results 33 comments of


                                            BY571

Discrepancy in ChemBench Score for google/gemma-2-9b-it (0.13 vs Expected 0.48)

Hi @n0w0f, Thanks for your message. I was now able to reproduce the results using both the code you provided and my original setup from above. Interestingly, using `formatted_prompt` does...

Discrepancy in ChemBench Score for google/gemma-2-9b-it (0.13 vs Expected 0.48)

Hi @n0w0f, Here are the package versions from the environment, which give the expected benchmark performance (~0.48): ``` python: 3.10.16 transformers: 4.52.4 torch: 2.7.0 accelerate: 1.7 cuda: 12.6 ``` This...

Inconsistent Inference Results After Saving and Loading LoRA Model

@Xiaohan-Tian Hey, did you figure out how to load the lora adapter?