वेदांत

memory usage for both of the merges calculated with a simple test script ``` Memory-Efficient Method • Peak GPU Memory: 300 MB • Peak CPU Memory: 14.4 MB • Execution...

as of now the PR Does not implement adapter merging back into quantized models ,

The PEFT library's QA-LoRA implementation intentionally doesn't support merging for GPTQ , i think we should keep it the same #2878 we could just enhance the error messages

hey forgot about this changes coming soon :face_with_head_bandage:

hey the issue could be completely valid but the traceback provided explicitly points to `torch.optim.adagrad.py` and not `optim.Adadelta` ``` File "/home/yanzhen/miniconda3/envs/accelerate/lib/python3.10/site-packages/torch/optim/adagrad.py", line 169, in step [rank1]: adagrad( [rank1]: File "/home/yanzhen/miniconda3/envs/accelerate/lib/python3.10/site-packages/torch/optim/adagrad.py",...