ggml icon indicating copy to clipboard operation
ggml copied to clipboard

ggml : improve custom operator support

Open ggerganov opened this issue 1 year ago • 0 comments

We have rudimentary support for adding F32 custom ggml operators in user space via:

https://github.com/ggerganov/ggml/blob/a1d0ea7c2abd44f56822ffdfcfe0a0fcf7170885/include/ggml/ggml.h#L1173-L1240

The API has to be improved to support:

  • multi-threading
  • non-F32 types

We also need to provide a few examples of how to define custom operators.

This functionality will significantly improve adoption of ggml in third-party projects since currently, they need to rely that certain operators first get implemented into ggml before being able to use them

ggerganov avatar Jun 25 '23 09:06 ggerganov