MLServer
MLServer copied to clipboard
SeldonIO
An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
MLServer uses FastAPI under the hood, which has some built-in functionality to generate and expose OpenAPI documentation on how to interact with the API. Additionally, the OpenAPI for the standard...
As part of the Seldon Core addition https://github.com/SeldonIO/seldon-core/pull/4028 which moves the TCP ready checks into proper HTTP request ready checks to `v2/health/ready` there is now a lot of noise from...
Add support to expose custom metrics that are exposed in a similar manner to the v1 python wrapper custom metrics - this could be explored together or separate to the...
## Description In the internal tests of MLServer, [there is a fixture](https://github.com/SeldonIO/MLServer/blob/master/tests/rest/conftest.py#L49) to create a `fastapi.testclient.TestClient` instance to a MLServer having all model loaded. It would be quite practical to...
Both `model-settings.json` and `settings.json` should be versioned as they are treated like client side APIs.
Many folks prefer YAML to JSON, Seldon and Kserve are primarily used in the Kubernetes community which is a YAML centric ecosystem. We should consider accepting YAML alongside JSON for...
Add support in MLServer to report metrics, logging and tracing through OpenTelemetry.
Hi, I had a problem with the performance when I served 20 mlflow models. For building a mlflow model I used the example described here: https://mlserver.readthedocs.io/en/latest/examples/mlflow/README.html I changed only one...
Add support for top-level `raw_input_contents` in gRPC spec. This field was initially introduced to work-around some performance issues within gRPC. You can find more details on these issues: - https://github.com/triton-inference-server/server/issues/1821...
Config-driven detectors based onhttps://docs.google.com/presentation/d/1pMgmF5xmMhc_t6SCN2jnGKWc1f7UV3AccOv63rui0TY/edit?usp=sharing
