MLServer
MLServer copied to clipboard
SeldonIO
An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
As an extension from #694, instead of a Python script, we could let the user pass along a Python wheel, containing their runtime as a Python package. ## Caveats -...
At the moment, MLServer allows you to inject a [custom environment](https://mlserver.readthedocs.io/en/latest/examples/conda/README.html). However, this environment is then used for every loaded model. Ideally, it would be great if MLServer also supported...
Pushing the built docker container to a docker registry is not mentioned after the build process in [Serving a custom model](https://mlserver.readthedocs.io/en/latest/examples/custom/README.html). In my case, it was necessary but I wasn't...
I experienced an issue, when all parallel workers crash for foreseeable reasons, mlserver continues to run as if it was healthy but it is no longer able to process any...
Currently there seem to be no version requirements on `fastapi` in the `setup.py`. In clean environment it gave me fastapi in version ``` poetry show fastapi (mlserver) (⎈ kind-ansible|seldon) name...
When writing custom runtimes, MLServer requires the user to know a bit about the V2 request and response structure, as well as codecs. This can add some friction when it...
Hi! My goal is to serve a mlflow model with a signature via mlserver and observed some issue with the signature enforcement and the request generated by `PandasCodec`. I followed...
Fix an issue where the model is loaded twice, causing double memory usage. And, allow the device argument to accept strings.
## Issue When using the KafkaServer to process Inference Requests while using a model that produces results that contain raw bytes (e.g. any recent transformer), the response is unable to...
