BentoML
BentoML copied to clipboard
bentoml
The easiest way to serve AI/ML models in production - Build Model Inference Service, LLM APIs, Multi-model Inference Graph/Pipelines, LLM/RAG apps, and more!
When batching is enabled, runner’s `/receive` span and any subsequent spans created in the service are always attributed towards the first trace id. The the trace of the first request...
Todo after the e2e testing rework goes in.
### Feature request Currently, we are using protobuf
### Feature request ssl support for gRPC. ### Motivation _No response_ ### Other _No response_
### Feature request PandasDataFrame support for `ndarray` proto field. ### Motivation _No response_ ### Other _No response_
### Feature request IO descriptor should have ability to accept multiple proto fields. ### Motivation _No response_ ### Other _No response_
### Feature request implement `from_proto` and `to_proto` for PandasSeries ### Motivation _No response_ ### Other _No response_
### Feature request Currently pandas IO descriptor doesn't have any validation in `from_http_request` and `to_http_response` It should. ### Motivation _No response_ ### Other _No response_
Currently we have an enum field to determine a file type to support. Under the hood, this is merely a mapping to a coressponding mime type. However this prevents user...
### Describe the bug Currently, content type for grpc are being set via our interceptor to `application/grpc`. While this conform with gRPC standard, this is not really useful for sysadmin....