BentoML
BentoML copied to clipboard
bentoml
The easiest way to serve AI apps and models - Build reliable Inference APIs, LLM apps, Multi-model chains, RAG service, and much more!
## What does this PR address? This PR is a follow up on #2808 for tests related manners This includes e2e, unit tests This will also include a fix in...
Health check endpoints, e.g. `/readyz`, `/livez`, on the API server and runner are limited to checking the health of the processes. Ideally, health check to the API server should be...
There are some docs block that are not render also start migrating :code: to double backtick for inline
We will probably should start enable 3.11 rc CI to start support
This PR migrates to [src-layout](https://setuptools.pypa.io/en/latest/userguide/package_discovery.html#src-layout) depends on #3012
Address #2986 This PR address some tests followup as well as some configuration refactor to be consistent with correct exporter field This PR will also introduce tests related to gRPC...
## What does this PR address? This PR adds the `bentoml test` command which enables testing a Bento for making sure it is correctly packaged. In order to use `bentoml...
Some use cases: ### 1. override one value assign a gpu for a runner > ```BENTOML_CONFIG_OPTIONS='runners.pytorch_mnist.resources."nvidia.com/gpu"=1' bentoml serve pytorch_mnist_demo --production``` equal to ``` runners: pytorch_mnist: resources: nvidia.com/gpu: 1 ``` ###...
https://github.com/bentoml/BentoML/blob/28f0bc320637851a158cafaed90ef15bb87fe82a/bentoml/_internal/frameworks/onnx.py#L415 I have a onnx model that have two outputs ['label', 'probabilities'] but when i use RUN only return 'label' values, and i dont see any configuration to return all...
