Benjamin Bossan
Benjamin Bossan
This issue is to track the TODO from [this comment](https://github.com/huggingface/peft/pull/2247#pullrequestreview-2569656574). Once optimum 1.24.0 and transformers 4.49.0 are released, we should enable gptqmodel in the CI (and remove auto-gptq).
Hi, the [AWQ multi-GPU unit test in PEFT started failing](https://github.com/huggingface/peft/actions/runs/14722066835/job/41317564249) recently with the error: > ValueError: Pointer argument (at 0) cannot be accessed from Triton (cpu tensor?) The full test,...
# Feature request In the paper [LoRA vs Full Fine-tuning: An Illusion of Equivalence](https://arxiv.org/abs/2410.21228), the authors describe a method to detect the degree of forgetting caused by LoRA by identifying...
There is still some chaos in our test suite, despite recent efforts to refactor it. This PR tries to improve the situation a bit. Some of the changes: - for...
I also saw that other PEFT repos have a CITATION.cff, which is also picked up by GH. Not sure how useful it is, it looks like a common standard but...
The MetaMathQA benchmark already had support to enable `torch.compile` but it was not very well implemented. The new changes are: - call compile after applying PEFT, not before - compile...
Resolves #2889 ## Description The reported bug is this: When the base model is quantized with 4bit bitsandbytes, the adapter weights would be cast to float32, even if `autocast_adapter_dtype=False` was...
Currently, the way PEFT deals with tied embedding and LM head weights is not always clear. In #2803, a new argument, `ensure_weight_tying`, was introduced to make it easier for users...
Trying to train LoRA with int4 torchao used to raise a RuntimeError. Lately, this error no longer being raised, suggesting that int4 training is unblocked. For this to work, we...