llm
llm copied to clipboard
rustformers
Consider removing the `bindgen` dependency
#22 and #24 show that clang is required as a build dependency because bindgen uses it to read in ggml.h and produce bindings. We should consider just writing the C bindings ourselves - it shouldn't be too difficult as long as we account for the idiosyncrasies (like #6)
Would it be possible to use bindgen as a dev dependency and just push the generated code?
Manually maintaining C bindings sounds very tedious :sweat_smile:
Possibly - not sure if there are any concerns with architecture-specific behaviour that would get baked in. Maintaining the bindings ourselves lets us handle that.
Maintaining the C bindings would be tedious, but I can't imagine that ggml's interface will change much over time. We'd have the initial pain, and then we'd have to add or remove a few lines every so often - shouldn't be too bad!
Hmmm, good point. But one thing that concerns me a bit with doing this manually is that if we mistype something in the bindings, or forget to update something, and then call a function with an incorrect signature, that's UB. Isn't it?
It can be - I hope that doesn't happen, but it's definitely a possibility. We don't depend on too much from ggml as it is now anyway, right?