llama.cpp icon indicating copy to clipboard operation
llama.cpp copied to clipboard

ggml : add metal backend registry / device

Open ggerganov opened this issue 1 year ago • 2 comments

target #9707

Adapt the Metal backend to the new registry and device interfaces.

ggerganov avatar Oct 02 '24 10:10 ggerganov

ggml_backend_metal_buffer_type() also needs to be updated to set the device field.

slaren avatar Oct 02 '24 13:10 slaren

There have been a few minor changes to the interfaces:

  • The get_backend_reg function of the device interface has been removed, instead a pointer is stored directly in ggml_backend_device: https://github.com/ggerganov/llama.cpp/pull/9707/commits/d0c4954fa01e90f61f9b5c5849f0d5732df0ea9f
  • Some functions have been renamed: https://github.com/ggerganov/llama.cpp/pull/9707/commits/cfef355611e38db20dedd9faad5f7b29e14a3004

slaren avatar Oct 03 '24 00:10 slaren

This seems to be working now.

slaren avatar Oct 05 '24 22:10 slaren

Should we put a deprecate notice for these API calls?

https://github.com/ggerganov/llama.cpp/blob/6dcb8991704b40d923691f037cdecc5430ff0440/ggml/include/ggml-metal.h#L41-L51

ggerganov avatar Oct 06 '24 10:10 ggerganov

Should we put a deprecate notice for these API calls?

I think it may be too early for that, it's probably better to wait a bit until all the backends and the ggml examples are updated.

slaren avatar Oct 07 '24 14:10 slaren