MLServer
MLServer copied to clipboard
SeldonIO
An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
I'm trying to use `tritonclient.grpc` inside an mlserver model, but looks like they don't really go well with one another. `python -c "import mlserver;import tritonclient.grpc"` results in a `Couldn't build...
Bumps [torchmetrics](https://github.com/Lightning-AI/torchmetrics) from 1.3.2 to 1.4.0. Release notes Sourced from torchmetrics's releases. Metrics for segmentation In Torchmetrics v1.4, we are happy to introduce a new domain of metrics to the...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
As per #1461, MLServer currently spawns worker with the same Python version as the main process, so user-provided environment is forced to match the Python version of the server. However,...
Bumps [transformers](https://github.com/huggingface/transformers) from 4.40.1 to 4.40.2. Release notes Sourced from transformers's releases. v4.40.2 Fix torch fx for LLama model Fix for Neuron (#30259) Fix copies for DBRX - neuron fix...
This PR includes streaming support for MLServer by allowing the user to implement in the runtime the `predict_stream` method which expects as input a async generator of request an outputs...
Pandas codec currently breaks mlflow runtime's schema enforcement (#1625). This is because numeric mlflow datatypes are assigned `NumpyCodec` content type [by default](https://github.com/SeldonIO/MLServer/blob/410cf6f838782443267c88bd410cabf95c760c54/runtimes/mlflow/mlserver_mlflow/metadata.py#L24-L28), which decodes `PandasCodec`'s explicit batch size into a...
PandasCodec does not support null values well at all, the method can_encode is completely misleading by just checking if it is a dataframe.
We don't support Python `3.8` anymore. Poetry `1.8.1` matches what's currently in our Poetry lockfile. Building the docs is currently failing [1], and this ideally will fix that. [1] From...
Bumps [torchmetrics](https://github.com/Lightning-AI/torchmetrics) from 1.3.1 to 1.4.0.post0. Release notes Sourced from torchmetrics's releases. Minor dependency correction Full Changelog: https://github.com/Lightning-AI/torchmetrics/compare/v1.4.0...v1.4.0.post0 Metrics for segmentation In Torchmetrics v1.4, we are happy to introduce a...
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.16.1 to 2.16.2. Release notes Sourced from tensorflow's releases. TensorFlow 2.16.2 Release 2.16.2 Bug Fixes and Other Changes Fixed: Incorrect dependency metadata in TensorFlow Python packages causing...