BlinkDL
Update pip package RWKV.model and v2/chat.py to support LoRA
Update pip package RWKV.model and v2/chat.py to support LoRA based on Blealtan/RWKV-LM-LoRA implementation.
Notice that pip package need to be uploaded again with the new changes to work.
Won't this break everything that uses the rwkv PIP package? It's trying to access attributes on lora. Why not have it as a keyword argument that defaults to None, that way existing code can keep working.
Also, what happens if the model is "preconverted"? I'm assuming it won't work correctly if that part has already occurred, so probably would want to add checks for whether it's a preconverted model before doing the lora part.
@KerfuffleV2 you are right about attributes, Need to change then to optional. About the preconverted I don't know, did not perform any tests about it yet.
About the preconverted I don't know, did not perform any tests about it yet.
I'm almost certain it couldn't work. Those tensors may be in a different format like u8. They will also have some additional fields like mx, my, rx, ry (maximums, minimums related to quantization).
So it may be possible to support that but I'm pretty positive it would need special handling. Probably the easiest approach at first is to just raise an exception if Lora is specified and the model file has been preconverted.
IMO the LoRA merging should go before the conversion since there are too many things happening during converting the model (including the xy quantizing, etc.). +1 on disallowing LoRA when pre-converted model is specified.
