RWKV-LM
RWKV-LM copied to clipboard
BlinkDL
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, sa...
Hey BlinkDL, Thank you very very much for such a great model.... I have a question on my mind if the RNN part of the network could be much more...
Hi, Is it possible to use something like GPTQ to get a 4-bit quantized version of the latest 7B Instruct model. That's got to be one of the fastest and...
# What to achive - Define a requirements.txt file to clearly specify the necessary packages. - Enable to set up the learning environment using Docker. - Manage learning environments based...
In the [first formula](https://github.com/BlinkDL/RWKV-LM/blob/main/RWKV-formula.png) in README, RWKV is rewritten into recurrent form by letting $W_n=(n-1)w$. Is there a particular reason for using $n-1$ instead of $n$? The latter is more...
I have 5 old (< 3yo) gaming rigs that are fairly serviceable. Can this be run with a distributed load across multiple machines? I can set up an internal network...
I was checking into the new Mojo AI programming language, and the possible significant performance boost advantages it might provide to RWKV-LM. Has anyone considered checking into this for RWKV-LM?...
Target： For me, I am new to huggingface, just want to convert my fine-tuned model(pth) into huggingface, which can be directly downloaded and used. Done: #113 I've seen this issue,...
