llama.cpp
llama.cpp copied to clipboard
ggml-org
RISC-V support?
By deleting line 155 (#include <immintrin.h>) in ggml.c, it works just fine on RISC-V. Maybe this can be added in Cmake?
Why RISC-V64? There is no board yet that supports ratified vectorization (rvv) 1.0, only if you simulate it in QEMU 7+.
Why RISC-V64? There is no board yet that supports ratified vectorization (rvv) 1.0, only if you simulate it in QEMU 7+.
Yeah, but it works on rv64gc... slowly(17s/token on visionfive2). I mean maybe we can make the code to compile without modification.
Yeah, but it works on rv64gc... slowly(17s/token on visionfive2). I mean maybe we can make the code to compile without modification.
Did you test it on VisionFive2? Are there any tutorials available?
Did you test it on VisionFive2? Are there any tutorials available?
As I mentioned before, just delete that one line and make.
It should work right away.
Why RISC-V64? There is no board yet that supports ratified vectorization (rvv) 1.0, only if you simulate it in QEMU 7+.
Yeah, but it works on rv64gc... slowly(17s/token on visionfive2). I mean maybe we can make the code to compile without modification.
We just tested on TH1520(16GB DDR) and D1(512MB DDR with some trick, not swap), TH1520 about 7s/token, and D1 about 3min/token due to memory.
We just tested on TH1520(16GB DDR) and D1(512MB DDR with some trick, not swap), TH1520 about 7s/token, and D1 about 3min/token due to memory.
Nice test! (Hilarious 😅) https://twitter.com/SipeedIO/status/1637399800101629956
Why RISC-V64? There is no board yet that supports ratified vectorization (rvv) 1.0, only if you simulate it in QEMU 7+.
I plan to add support for V extend acceleration, use vector intrinsics, it is compatible from 0.7.1 to 1.0
