CodeTF icon indicating copy to clipboard operation
CodeTF copied to clipboard

Add RWKV models

Open guangyusong opened this issue 1 year ago • 3 comments

Changes:

Added support for RWKV model family.

Related links:

Paper: https://arxiv.org/abs/2305.13048 Github: https://github.com/BlinkDL/RWKV-LM

Screenshots:

Model list: Screenshot 2023-06-05 at 2 10 45 AM

Sample generation: Screenshot 2023-06-05 at 2 09 48 AM

guangyusong avatar Jun 05 '23 06:06 guangyusong

Thanks for the contribution! Before we can merge this, we need @guangyusong to sign the Salesforce Inc. Contributor License Agreement.

salesforce-cla[bot] avatar Jun 05 '23 06:06 salesforce-cla[bot]

thank you. The RWKV model family looks very nice. However, they are not really code learning models, do you have the checkpoints related to coding tasks?

bdqnghi avatar Jun 06 '23 08:06 bdqnghi

The RWKV model family should have a similar data split as GPT-J. We anticipate releasing a model that's more adept at coding tasks in the near future.

guangyusong avatar Jun 06 '23 19:06 guangyusong