Self-Prompt-Tuning icon indicating copy to clipboard operation
Self-Prompt-Tuning copied to clipboard

Not Able to Reproduce Major Results Reported in this Paper, Waiting for author responses.

Open Noahsark opened this issue 8 months ago • 4 comments

Dear authors,

After extensively searching the hyperparameters, we have been unable to reproduce the major results presented in your paper using the released code. Our reproduced results significantly deviate from those reported, showing a large margin of difference. Specifically, we have observed that the proposed initializations have a negative effect on most recognition tasks.

Additionally, the efficiency and latency of the initializations are considerably worse than the data reported in the paper. The current implementation copies all data to RAM, causing substantial delays that are much much higher than the reported values.

We request that you address these issues and provide any necessary clarifications or corrections. As the paper has been accepted by ICML, we will also bring this issue to the conference's attention to ensure the reproducibility of the results.

Thank you for your attention to these matters.

Noahsark avatar Jun 10 '24 19:06 Noahsark