Kartik Khare

I got this as well today

@bodaay what is the sie of the adapter.bin you are getting? Mine is like in bytes. Btw, I just commented out the `model = model.merge_and_unload()` line and it works. Merging...

Are you able to resume training from the checkpoint with this?

I haven't yet tested the output of the adapters trained after this. There seems to be a debate on this issue in the linked `alpaca-lora` PR. @artidoro do let us...

Did a bit of verification on `adapter_model.bin` file saved using this fix and it does seem to contain only `lora` layers. ``` >>> import torch >>> state_dict = torch.load("output_redpajama3B_test_2/checkpoint-10/adapter_model/adapter_model.bin") >>>...

Strangely the checkpoints in the output directory are also very small (adapater_model.bin is just 400 bytes). However, the optimizer.pth is around 1GB.

> In that case, we can keep both `optimizeDictionary` (apply to both dimensions and metrics) and `optimizeDictionaryForMetrics` (only apply to metrics) to avoid backward incompatible. I don't see a case...

Yes @taishan1994 is correct, we can't use older version of peft since it doesn't have the qlora changes.

@taishan1994 can you try the changes in the PR and see if it works for you

for merge just comment out the merge_and_unload line it is not mandatory to merge to do inference the combined peft model object can be used as i