llama-cpp-python icon indicating copy to clipboard operation
llama-cpp-python copied to clipboard

Published 20 hours ago • verified publisher icon abetlen

Distributed Inference

Open lipere123 opened this issue 10 months ago • 1 comments

For LLama.cpp, distributed inference work great. I am using it right now into my program.

Now, if not implemented, it would be cool to add it to llama-cpp-python. Right now, I am full command line.

Thanks in advance. Best regards. Benjamin.

lipere123 avatar Feb 05 '25 07:02 lipere123

Found it. Currently testing.

lipere123 avatar Feb 06 '25 04:02 lipere123