BentoML
BentoML copied to clipboard
bentoml
The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!
**Is your feature request related to a problem? Please describe.** Currently the bento server can be configured using a configuration file as described here: https://docs.bentoml.org/en/latest/guides/configuration.html However, the only way to...
The Bento build process requires loading the `bentoml.Service` object for validating the definition and retrieving required models to package. Currently, running `bentoml build` requires all Service dependencies to be installed,...
Support importing models saved in the neuropod format https://github.com/uber/neuropod
Right now it is a pain to maintain certain packages and library dependency from different OS (alpine, debian, amazon) A proposal is to pull library and required components straight from...
## Goals - Instrument input and output of every prediction in both `api_server` and `runners` - Schematized - Agnostic of the wire protocols, e.g. HTTP, gRPC - Transferred in binary...
**Is your feature request related to a problem? Please describe.** I would like to use bentoml to make offline batch predictions where my dataset is stored on S3. Datasets can...
**Is your feature request related to a problem? Please describe.** Add support for SnapML scoring service. **Describe the solution you'd like** Create a model artifact type that can exploit SnapML...
**Describe the bug** In the file docs/source/guides/monitoring.rst the following passage is outdated, links don't work anymore and the api has changed. ------ **NOTE** Users can also create custom metrics for...
**Is your feature request related to a problem? Please describe.** My use case is an audio classification service where the user will be sending audio file to the service for...
