MLServer
MLServer copied to clipboard
SeldonIO
An inference server for your machine learning models, including support for multiple frameworks, multi-model serving and more
As per our [Slack discussion](https://seldondev.slack.com/archives/C03DQFTFXMX/p1667786578108119) with @adriangonz there is a performance overhead on MLServer in terms of received latency compared to pure FastAPI. As discussed in the same thread some...
Currently where server replies with 503 the batch processing component will log into output file ``` {"parameters": {"batch_index": 0}, "error": {"status": "preprocessing error", "msg": "Expecting value: line 1 column 1...
Add an option to the `mlserver infer` command to stop batch processing when failure happened and retries completed in Core and MLServer. When failure happens and can’t recovered after a...
Hello! I've been testing different ways of making requests, and I've noticed that the performance gets noticeably worse when using requests with many inputs. For example, I have a pandas...
When using the simplified interface introduced in #765, the `predict()`'s method signature should already contain all information to automatically generate the model's metadata (at least, input names, content types and...
Following the initial work done in #765 and, as suggested in https://github.com/SeldonIO/MLServer/issues/697#issuecomment-1267245839, it would be useful to introduce support for Unions in simplified `predict()` interfaces. This would allow for use...
Bumps [tensorflow](https://github.com/tensorflow/tensorflow) from 2.14.1 to 2.16.1. Release notes Sourced from tensorflow's releases. TensorFlow 2.16.1 Release 2.16.1 TensorFlow TensorFlow Windows Build: Clang is now the default compiler to build TensorFlow CPU...
![dependabot[bot] avatar](https://avatars.githubusercontent.com/in/29110?v=4 "dependabot[bot] avatar"){kind=avatar}
Bumps [uvicorn](https://github.com/encode/uvicorn) from 0.28.0 to 0.29.0. Release notes Sourced from uvicorn's releases. Version 0.29.0 Added Cooperative signal handling by @maxfischer2781 in encode/uvicorn#1600 Full Changelog: https://github.com/encode/uvicorn/compare/0.28.1...0.29.0 Version 0.28.1 Fixed Revert raise...
Making `MLServer` compatible with `pydantic v1` **AND** `pydantic v2`. Things added in this PR: - removing `pydantic` dependency from runtimes when not required. - redefining classes so that they use...
