[Enhancement]: Add RWKV model support (RWKV is a 100% RNN Language Model - ctxlen 8192 models available, longer ctxlen soon)

[BlinkDL]

What would you like to be added?

RWKV Raven 7B Gradio Demo: https://huggingface.co/spaces/BlinkDL/Raven-RWKV-7B

Use rwkv.cpp for CPU INT4 / INT8: https://github.com/saharNooby/rwkv.cpp

Github project: https://github.com/BlinkDL/ChatRWKV

Sample code using rwkv pip package: https://github.com/BlinkDL/ChatRWKV/blob/main/v2/benchmark_more.py

Please let me know if you have any questions :)

Why is this needed?

No response

Anything else?

No response

[xiaofan-luan]

What would you like to be added?

RWKV Raven 7B Gradio Demo: https://huggingface.co/spaces/BlinkDL/Raven-RWKV-7B

Use rwkv.cpp for CPU INT4 / INT8: https://github.com/saharNooby/rwkv.cpp

Github project: https://github.com/BlinkDL/ChatRWKV

Sample code using rwkv pip package: https://github.com/BlinkDL/ChatRWKV/blob/main/v2/benchmark_more.py

Please let me know if you have any questions :)

Why is this needed?

No response

Anything else?

No response

@BlinkDL want to take the RWKV features? the RWKV has very long context window, which is great

[SimFG]

We are adding the rwkv embedding, see: #363 If you have other ideas, please feel free to give a comment

[SimFG]

If you have other ideas, please open a new issue