How to save lora weights only???

[lucasjinreal]

Currently the weights saved are all models weights include optimzier, is HUGE,

how to set save lora weights only?

[ZYHowell]

Current checkpoint from huggingface/deepspeed does not support this functionality. To store only the LoRA weight after the whole training progress, see here. Maybe one can monkey patch the hf trainer/deepspeed's checkpoint function to do the same thing