How to train a ChatRWKV model on domain-specific data

[Just4Uzi]

Hi! Thank you for your wonderful work! I want to train a QA bot on my own data, and I guess I shuold train a RWKV-LM v4 model and load this model in chat.py(by change the model path). Am I doing right? Do you have any suggestions?

Thank you very much

[BlinkDL]

Hi you can finetune a RWKV-v4 model using v4neo code in RWKV-LM. Read the "large corpus" part for guide.

[BlinkDL]

Now you can also use https://github.com/Blealtan/RWKV-LM-LoRA