inference
inference copied to clipboard
xorbitsai
FEAT: Support RWKV Pile
Resolve #533
Add support for rwkv-4-pile models. Successfully tested the 169m variant on a local Mac setup.
I would like some guidance on setting the model size. Historically, we've rounded up the sizes for models with parameters fewer than 1 billion. However, with rwkv-4-pile, we encounter two smaller models: 169m and 430m.
I am leaning towards simply excluding the 169m variant and setting the smallest size threshold at 430m (for 1b). However, alternative suggestions are welcomed.
It seems that all the tests are actually passing, but somehow there is a segmentation fault from Stable Diffusion test. Not sure if this is just a probabilistic error or do I need to fix something.
Hi, could you please also update the docs? Here's an example: https://github.com/xorbitsai/inference/pull/587
