PENG Bo
PENG Bo
Yes but it will be very slow. You can try a tiny model.
1. Similar to usual GPT of the same size, because we are using parallelization to increase training speed. However, you can definitely train it like a RNN to save VRAM...
还需要win10最新21h2版，而且A卡至少需要RX6xxx。
> Exporting to ONNX is something that I've been tinkering with and I can report that the 169m RWKV-4 model does run in browser. Here's my code: https://github.com/AXKuhta/RWKV-LM/tree/onnx Great work...
> * @BlinkDL Yes, I got `TypeError: cannot resolve operator 'Max' with opsets: ai.onnx v13` when trying to use the WebGL backend. How would I go about removing `RWKV_HEAD_QK` and...
And the onnx version might work for AMD & Intel gpus. The DirectML backend supports them (on win10). I tried that for RWKV-1.
> Yeah, that's what's happening. RWKV-v4 is bf16 which can't be losslessly converted to fp16, so fp32 is the next best option. The fp32-converted model also compresses really well since...
> Hi, really exciting project! I'm wondering if you've published the model conversion script that you used to create the [js_models](https://github.com/BlinkDL/AI-Writer/tree/main/docs/eng/js_model) files from the `.pth` model file? It would be...
> @AXKuhta Could have sworn I replied here earlier, sorry - apparently I didn't click send. I fixed the demo according to your comment soon after you posted it Add...
> I have been able to force the full model to run on webgl, but it doesn't produce anything coherent, so something's still broken: > > https://github.com/AXKuhta/RWKV-LM/tree/onnx_webgl > > @BlinkDL...