Would you consider releasing code that supports lora training 40b model?

[Key-lei]

trafficstars

Very excellent work! When using lora to train a 40b model in my task, I found during the loading inference process that lora did not save the weight of the vision tower, so the effect of my task was very poor. Would you consider supporting lora training and loading with official code?

[Efficient-Large-Language-Model]

Lora training is not well supported. I would recommend doing a regular finetuning.