MLServer
MLServer copied to clipboard
SeldonIO
An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
Currently, models in MLServer can only be fully loaded or unloaded. That is, they either are fully loaded on memory, or MLServer knows nothing about them. In order to save...
The are inference cases e.g. tf models trained with mlflow (`TypeError: Only dict and DataFrame input types are supported`) that dont consume directlynp.ndarray In these cases the alibi runtime will...
Currently seldon core supports the concept of INPUT TRANSFORMER and OUTPUT TRANSFORMER which have different URL paths for the v2 protocol, which means that these are not supported if MLServer...
Currently, the MLServer CLI won't be able to run in Windows. The main blocker we've found for this is that the `add_signal_handler` method is not supported in Windows AsyncIO loops...
Currently, some `MlflowException` errors are getting raised uncaught until the REST layer returns a generic stacktrace back. We should handle these situations better, to ensure that these get re-raised wrapped...
- [ ] Add example on serving explainers through Alibi Explain runtime - [ ] Add docs around extra config required to serve explainers.
So far we have tested in mlserver Anchors and IntegratedGradients explainers. We should at some point start checking if they other (e.g. SHAP and counterfactual) techniques work well in the...
As a followup of #331 we should explore allowing users to use the `openapi()` and `custom_openapi()` functions that FastAPI exposes https://fastapi.tiangolo.com/advanced/extending-openapi/#normal-fastapi to allow for a simple and robust way of...
Currently the implementation of `mp_predict` is going to block the asyncio loop. This will result in only one predict / explain call to be processed, and therefore loosing some of...
Currently Explain endpoint is served as v2 `predict` endpoint, which is synchronous by design (from the client perspective). In some explanations, especially if we are not using the gpu the...