LocalAI icon indicating copy to clipboard operation
LocalAI copied to clipboard

Published 20 hours ago • verified publisher icon mudler

feat(llama.cpp): Vulkan, Kompute, SYCL

Open mudler opened this issue 1 year ago • 4 comments

Tracker for: https://github.com/ggerganov/llama.cpp/discussions/5138 and also ROCm

  • [x] Vulkan: https://github.com/mudler/LocalAI/pull/2648 (upstream https://github.com/ggerganov/llama.cpp/pull/2059 )
  • [ ] Kompute: (upstream https://github.com/ggerganov/llama.cpp/pull/4456 )
  • [x] sycl: https://github.com/mudler/LocalAI/pull/1660 ( upstream https://github.com/ggerganov/llama.cpp/pull/2690 )
  • [x] ROCm: #1595 (#1100)

mudler avatar Jan 26 '24 09:01 mudler

The merge requests linked on this issue appears to be merged upstream. Does that mean LocalAI already supports Vulkan or there are any additional tasks to do before that?

RiQuY avatar Apr 08 '24 15:04 RiQuY

The merge requests linked on this issue appears to be merged upstream. Does that mean LocalAI already supports Vulkan or there are any additional tasks to do before that?

Only kompute is missing as for now

mudler avatar Jun 24 '24 18:06 mudler

The merge requests linked on this issue appears to be merged upstream. Does that mean LocalAI already supports Vulkan or there are any additional tasks to do before that?

Only kompute is missing as for now

It looks like kompute is also merged

jim3692 avatar Sep 09 '24 08:09 jim3692