ava
ava copied to clipboard
Support Nvidia, AMD, AVX, ... (CUDA, ROCm, Vulkan and other BLAS) for windows
This will take some time, so this is just rough sketch for later:
- windows build/binary could expect
llama.dll
- could be straightforward, because zig can cross-compile so I can do this locally on my machine
-
zig build
can download/extract a zip file from url likehttps://github.com/ggerganov/llama.cpp/releases/download/{short_rev}/llama-{short_rev}-bin-{blas}.zip
- where
short_rev
is obtained from thellama.cpp
git submodule -
blas
is something likewin-cuda-cu11.7.1-x64
passed as-Dblas=xxx
to azig build
- where
- both
.exe
and.dll
should be marked as artifact
After this is done, we can make a windows pipeline, with matrix for each BLAS, and hopefully, we will get a .zip
file, which people can just download and run. Of course, they still need to have given BLAS installed on their system.