Mahdi Beitollahi

icon location Canada icon location I am a machine learning scientist working on distributed machine learning and privacy-preserving machine learning methods such as Federated Learning.

Results 7 comments of Mahdi Beitollahi

getting spurious "HSIC computation resulted in NANs"

Yeah. I am having the same issue unfortunately. CKA for image networks works flawlessly. However, for the speech dataset that I am trying, HSIC returns NaN values. I tried normalizing...

Learning the combination weights of pre-trained LoRA Modules

Hi @BenjaminBossan , thanks for taking the time to read the paper. Thank you for your great suggestion. We will evaluate other combination methods in the paper. To answer your...

Learning the combination weights of pre-trained LoRA Modules

Hi @BenjaminBossan, I just opened a draft PR using the first method. Let me know what you think. The main concern that I have is this [line](https://github.com/mahdibeit/peft/blob/11282c856ab70ba03f9d7927a6e706403402b996/src/peft/tuners/lora/layer.py#L152) to freeze pre-trained...

supports learning the combination weights of pre-trained LoRA modules

@BenjaminBossan Yes, for sure. The code implements the learned composition in *Does Combining Parameter-efficient Modules Improve Few-shot Transfer Accuracy?* (Asadi et al., 2024). More specifically, it learns the $v$ for...

supports learning the combination weights of pre-trained LoRA modules

> Thanks for working on this PR and for providing an example. I also see now that you're one of the paper authors :) > > I left a couple...

supports learning the combination weights of pre-trained LoRA modules

So right now using this update we can write the following scripts as test and example usage. I added the example usage to the docs. ### Test ```python import torch...

supports learning the combination weights of pre-trained LoRA modules

Hi @BenjaminBossan, sorry I was busy in the last three weeks. I will apply your comments and push it this week.