BentoML
BentoML copied to clipboard
bentoml
The easiest way to serve AI apps and models - Build reliable Inference APIs, LLM apps, Multi-model chains, RAG service, and much more!
**Is your feature request related to a problem? Please describe.** Currently there is not too much visibility for the user to visualize or to understand how BentoML's micro-batching works. having...
**Is your feature request related to a problem? Please describe.** Supports for some Jax-based frameworks, such as [Flax](https://github.com/google/flax), [Trax](https://github.com/google/trax), DM-Haiku to BentoML **Describe the solution you'd like** *WIP* and will...
Tracking issue for small cleanup fixes we should make sure we do after 1.0; will add to this list as we think of more things. - [ ] More structured...
## Description - I added `Parameters` in the POST method to use a variable URL.    ## Motivation and...
### Describe the bug When trying to containerize a bento with conda enable, M1 will fail due to linked library while building psutil. ### To reproduce 1. Enable conda from...
Regarding high performance of transformer models with TensorRT, would it be envisageable to adapt bentoML to work with?
A large tracking issue for things that should probably have specific error messages and suggested fixes: - [ ] Passing a non-supported type to / from runners via DataContainer -...
We should have good docs for using BentoML with various testing frameworks.
Provide supports for ONNXMlir as a first-class citizen TODO: - [x] Use provided docker container for compilation - [x] provides envar `ONNX_MLIR_DIR` if users build ONNX_MLIR locally - [x] need...
**Describe the bug** Currently if users pass in space environment variables `BuildConfig` wouldn't be able to process it accordingly ```yaml service: "service.py:svc" env: my space: 1 ENV: hello world ```...