Benjamin Bossan

Thanks for the interest. We plan to do a mini sprint next week to make progress on this task. We will share any findings where the community could help contributing....

> That would be great. I conducted a comparative study evaluating AdaLoRA, IA3, and LoRA for text generation tasks using a multi-step conversational dataset. The results indicate that IA3 outperforms...

Thanks for sharing your findings @imcoza. Are the code and the results publicly available? We're still working on the method comparison framework, the progress is looking good so far. You...

> I would love to contribute especially in improving the documentation for PEFT. You had mentioned about adding details on the best uses cases, pros and cons. Is it possible...

Good news everyone, we made big progress on this project by merging #2395. This adds a PEFT method comparison suite that should allow us to compare different PEFT methods in...

@cyaaronk The results are not shared yet. We're working on generating them in a consistent fashion. Once that's done, we plan to deploy a Hugging Face space for everyone to...

@cyaaronk We don't want to share the results prematurely in case we notice some errors. But it's safe to say that LoRA is still a good choice in terms of...

Hi everyone, thanks for your patience. We finally ran all PEFT methods through the MetaMathQA experiment 🚀 🎉 The results can be found [here](https://github.com/huggingface/peft/tree/main/method_comparison/MetaMathQA/results). We're still actively working on this,...

Small update, we have deployed a Gradio app that allows everyone to easily inspect the results: https://huggingface.co/spaces/peft-internal-testing/PEFT-method-comparison

Hey @yuhongwang-xd, thanks for the offer. Regarding your proposals: 1. Right now, for most PEFT methods, we only have a single experimental setting, which most of the times corresponds to...