llama-cpp-python icon indicating copy to clipboard operation
llama-cpp-python copied to clipboard

Published 20 hours ago • verified publisher icon abetlen

arm64 builds for CUDA

Open mcvella opened this issue 1 year ago • 2 comments

Many are using this library on Nvidia Jetson/Orin devices, but there are no prebuilt wheels available for CUDA arm architectures.

Could support for automated builds of these wheels be added? Thanks!

mcvella avatar May 10 '24 18:05 mcvella

@mcvella I think so, while I don't want to expand the builds arbitrarily (big issue atm) it would be good to have them for popular devices if we can limit the CUDA / python versions as well. If we can collect some of that info here I'll see how we can expand the builds.

abetlen avatar May 11 '24 17:05 abetlen