basic unified API for all backends

[suspicious-pineapple]

• the syntax for different backends is similar, but many config properties (that do exactly the same thing) have different names, for example numPredict (llm-rs) vs n_tok_predict (llama.cpp)

• A "universal api" would have the same property names for every backend, so that switching to a different one would be as simple as changing one parameter.

• It would only support the basic features all backends have (prompt, number of tokens, stop sequence(?), temperature) (hence "basic" in the title)

• Backend-specific features (save/loadSession from llm-rs, mmap from llama.cpp) would be unavailable, or throw an error for backends that haven't implemented them yet (bad because throwing errors for obscure reasons?)

• Parameters that are optional for some backends but mandatory for others (like topP?) could be replaced with "sane" default values.

[hlhr202]

probably consider this later. i have not cleared what kind of models and backends are going to support in the near future.

[hlhr202]

Hi, @end-me-please. Would you mind if join our Discord here? I m collecting all the fantastic feature requests and hopefully if can find someone help us improve or even contribute to this project.

Sorry for this spamming.

[hlhr202]

I m considering to unify at a very basic level