BentoML
BentoML copied to clipboard
bentoml
The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!
### Describe the bug Try to build a bento on a machine without Nvidia GPU and has cuda toolkit installed. It failed because method NvidiaGpuResource.from_system throws at ``` try: pynvml.nvmlInit()...
## What does this PR address? Fixes #4460 ## Before submitting: - [x] Does the Pull Request follow [Conventional Commits specification](https://www.conventionalcommits.org/en/v1.0.0/#summary) naming? Here are [GitHub's guide](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request) on how to create...
### Describe the bug Hi. I already created the bentoml model and service and all work as expected. but cant containerize the bento due to this error: after run this...
### Describe the bug  i was getting the keyerror while i was running the code bentoml version. bentoml==1.0.25 bentoml==1.0.10 with the latest version also pip install bentoml==1.2.2 here i...
Simpler version of creating a proper monorepo, and probably better supported.
### Describe the bug When running `bentoml containerize` with a custom docker base_image, the command throws an error ``` => => transferring context: 47.64MB 0.3s => ERROR [base-container 2/8] RUN...
### Describe the bug Exception raised when creating `bentoml` directory with home directory containing `@` character ```bash ~/AI/pet_projects/BentoML/examples/quickstart on main took 21s % python train.py Traceback (most recent call last):...
### Feature request With https://github.com/bentoml/BentoML/pull/3028 it introduces `bentoml.client`. Currently what we are doing under `bentoml.testing` are essentially doing what `bentoml.client` is doing. This refactor would make `bentoml.testing` more clean and...
### Describe the bug The implementation of src/bentoml/_internal/ray/__init__.py had dependencies upon internal Ray object which may change across releases leading to incompatibilities. In particular it references ASGIHTTPSender (which was only...
### Feature request Depending on available models, support the possibility to add endpoints to a service. Main Idea (only a draft, does not work like this in python) ``` import...