ipex-llm icon indicating copy to clipboard operation
ipex-llm copied to clipboard
verified publisher icon intel-analytics

enable rpc for multi nodes

Open hli25 opened this issue 7 months ago • 2 comments

For llama cpp sycl backend, please help to enable rpc for multiple nodes.

-DGGML_RPC=ON Please refer to : https://github.com/ggml-org/llama.cpp/tree/master/tools/rpc

hli25 avatar May 28 '25 02:05 hli25

The native RPC of llama cpp is supported, but not released. We are developing a new RPC node for hybrid devices to enable one node with two backend type.

ACupofAir avatar May 28 '25 07:05 ACupofAir

cool, look forward to try the new release with RPC, thanks!

hli25 avatar May 29 '25 01:05 hli25