mlflow-ray-serve
mlflow-ray-serve copied to clipboard
ray-project
[FR] Compatibility with MLflow 2.0
Proposal Summary
In MLflow 2.0 (scheduled for release on Nov. 14), we will be making small modifications to the MLflow Model Server's RESTful scoring protocol (documented here: https://output.circle-artifacts.com/output/job/bb07270e-1101-421c-901c-01e72bc7b6df/artifacts/0/docs/build/html/models.html#deploy-mlflow-models) and the MLflow Deployment Client predict() API (documented here: https://output.circle-artifacts.com/output/job/bb07270e-1101-421c-901c-01e72bc7b6df/artifacts/0/docs/build/html/python_api/mlflow.deployments.html#mlflow.deployments.BaseDeploymentClient.predict).
For compatibility with MLflow 2.0, the mlflow-ray-serve plugin will need to be updated to conform to the new scoring protocol and Deployment Client interface. The MLflow maintainers are happy to assist with this process, and we apologize for the short notice.
Motivation
- What is the use case for this feature? Provide a richer, more extensible scoring protocol and broaden the deployment client prediction interface beyond dataframe inputs.
- Why is this use case valuable to support for MLflow Ray Serve Deployment plugin users in general? Necessary for compatibility for MLflow 2.0
- Why is it currently difficult to achieve this use case? Without these changes, the
mlflow-ray-serveplugin will break in MLflow 2.0.
@edoakes @architkulkarni @frascuchon Do you have bandwidth on your end to migrate the mlflow-ray-serve plugin to the updated scoring protocol and adjust the Deployment Client predict() API? By my estimates, it should only take a few hours of work at most. Apologies for the short notice.
Hi @dbczumar, thanks for the heads up! The Ray team won't have bandwidth in the short term for the migration, but we welcome outside contributions.
Is there a migration guide, or should we just compare the new docs with the old version of the docs?
Hi @dbczumar, thanks for the heads up! The Ray team won't have bandwidth in the short term for the migration, but we welcome outside contributions.
Is there a migration guide, or should we just compare the new docs with the old version of the docs?
Hi @architkulkarni , thanks for the context! Comparing the new docs with the old ones should be sufficient - the changes are pretty small here. Hoping the community can help here! :)
We don't have a timeline for updating this unfortunately. Contributions are welcome as always!