llm icon indicating copy to clipboard operation
llm copied to clipboard

Published 20 hours ago • verified publisher icon rustformers

Support for GGML BLAS

Open darxkies opened this issue 1 year ago • 1 comments

It would be great if BLAS could be enabled in GGML by the user. Enabling BLAS does significantly enhance the performance of GGML. I have tested llama.cpp/GGML with cublas, and it really pays off enabling it.

darxkies avatar May 07 '23 15:05 darxkies

cublas-patch.txt

Ugly patch to add support for cuBLAS.

darxkies avatar May 07 '23 19:05 darxkies

Thanks, can CBLAS be enabled as well ?

virajk31 avatar May 25 '23 07:05 virajk31

CLBlast can be enabled for Linux and Windows using the PR #282.

darxkies avatar Jun 10 '23 15:06 darxkies

Is this done now?

philpax avatar Jun 19 '23 15:06 philpax

From my POV, it is.

darxkies avatar Jun 20 '23 10:06 darxkies