Lose /invocations path when using envrironment tarball from model-settings.json

[ylambrus]

Hi,

Is it expected that we lose the /invocations path ( mlflow backward compatible inference path) when using a tarball environment per model, eg when using model-settings.json ?

Note that when using global custom env tarball, it's working fine and invocations path is still present.

To reproduce :

Build a dummy iris model as follows ( or any model that can rely on mlflow ) :

import mlflow
import mlflow.sklearn
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
import pandas as pd
from mlflow.models.signature import infer_signature

data = load_iris()
X_train, X_test, y_train, y_test = train_test_split(data.data, data.target, test_size=0.2, random_state=42)

model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)

input_example = pd.DataFrame(X_train[:1], columns=data.feature_names)
signature = infer_signature(X_train, model.predict(X_train))

with mlflow.start_run() as run:
    mlflow.sklearn.log_model(
        model, 
        "iris_model", 
        signature=signature, 
        input_example=input_example
    )
    model_uri = f"runs:/{run.info.run_id}/iris_model"

print(f"Model saved in MLflow at: {model_uri}")

Then build a conda tarball to match dependencies for this model :

conda env create -f conda.yaml
conda activate mlflow-env
conda install -c conda-forge conda-pack -y
conda pack -n mlflow-env -o environment.tar.gz

Then put your tarball inside your model artefact folder as usual and be sure to have your model-settings.json as follows :

{
    "name": "iris-model",
    "implementation": "mlserver_mlflow.MLflowRuntime",
    "parameters": {
        "uri": ".",
        "environment_tarball": "environment.tar.gz",
        "version": "v1"
    }
}

Then run mlserver ( here we tried with 1.5.0 and 1.6.1 as well):

mlserver start ./mlruns

2024-11-14 14:57:47,635 [mlserver] WARNING - Model name 'iris-model' is different than model's folder name 'iris_model'.
2024-11-14 14:57:47,662 [mlserver.parallel] DEBUG - Starting response processing loop...
2024-11-14 14:57:47,663 [mlserver.rest] INFO - HTTP server running on http://0.0.0.0:8080
INFO:     Started server process [34785]
INFO:     Waiting for application startup.
2024-11-14 14:57:47,676 [mlserver.metrics] INFO - Metrics server running on http://0.0.0.0:8082
2024-11-14 14:57:47,676 [mlserver.metrics] INFO - Prometheus scraping endpoint can be accessed on http://0.0.0.0:8082/metrics
INFO:     Started server process [34785]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
2024-11-14 14:57:47,678 [mlserver.grpc] INFO - gRPC server running on http://0.0.0.0:8081
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8080 (Press CTRL+C to quit)
INFO:     Uvicorn running on http://0.0.0.0:8082 (Press CTRL+C to quit)
2024-11-14 14:57:47,777 [mlserver] INFO - Extracting environment tarball from /Users/yannick.lambruschi/repos/MLServer/mlruns/0/720aee73574a49418d4ae70011aee331/artifacts/iris_model/environment_iris.tar.gz...
2024-11-14 14:57:58,639 [mlserver.parallel][iris-model:v1] DEBUG - Starting response processing loop...

2024-11-14 14:58:22,056 [mlserver][iris-model:v1] INFO - Loaded model 'iris-model' successfully.
2024-11-14 14:58:22,057 [mlserver][iris-model:v1] INFO - Loaded model 'iris-model' successfully.

Now when querying /invocations you'll face a 404 :

curl -X POST "http://localhost:8080/invocations" -H "Content-Type: application/json" -d '{"inputs": [[5.1, 3.5, 1.4, 0.2]]}'

{"detail":"Not Found"}%

==> mlserver logs

INFO:     127.0.0.1:59245 - "POST /invocations HTTP/1.1" 404 Not Found