BentoML
BentoML copied to clipboard
bentoml
The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!
### Feature request I would like if I could set details that are relevant to our organisation in the openapi docs for my models (contact information, summary, git link) without...
### Feature request Bentos are a great way to collaborate, as they are self-contained and can be converted to virtually anything - loaded in Python as used as runners, converted...
### Feature request Build the docker file with podman build or buildah. ### Motivation Docker is a paid product for most commercial uses. podman or buildah are open source. ###...
### Feature request I would it if runner signatures could be called without an `input_data` argument. ### Motivation We want to be able to access the feature names from a...
### Feature request I tested the queue has a performance impact, you can consider using shared memory for Dynamic Batching. python 3.8 supports shared memory. ### Motivation _No response_ ###...
Signed-off-by: Frost Ming ## What does this PR address? ```python import bentoml @bentoml.service class MyService: def __init__(self) -> None: # do something on service startup, per replica ... def on_shutdown(self)...
Signed-off-by: Frost Ming ## What does this PR address? Fixes #(issue) ## Before submitting: - [ ] Does the Pull Request follow [Conventional Commits specification](https://www.conventionalcommits.org/en/v1.0.0/#summary) naming? Here are [GitHub's guide](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/proposing-changes-to-your-work-with-pull-requests/creating-a-pull-request)...