mistral.rs
mistral.rs copied to clipboard
[feat] running the server from rust
i'm looking for some production ready LLM API that I can use from rust as a lib in https://github.com/louis030195/screen-pipe
would it be possible to provide some abstraction like
let api = mistral_rs::api::new("phi42").load()
api.serve()
// openai style api ... function calling available if possible
i also use axum
atm implementing the api myself with candle