mistral.rs icon indicating copy to clipboard operation
mistral.rs copied to clipboard

[feat] running the server from rust

Open louis030195 opened this issue 7 months ago • 3 comments

i'm looking for some production ready LLM API that I can use from rust as a lib in https://github.com/louis030195/screen-pipe

would it be possible to provide some abstraction like

let api = mistral_rs::api::new("phi42").load()
api.serve()
// openai style api ... function calling available if possible

i also use axum

atm implementing the api myself with candle

louis030195 avatar Jul 25 '24 07:07 louis030195