LOCAL_MODEL_FOLDER not being read correctly when using docker compose deploy

[Adam-Goldenberg-Accrete]

LOCAL_MODEL_FOLDER not being read correctly when using docker compose deploy

Following the Docker Installation - Local Models guide.

I export the LOCAL_MODEL_FOLDER with export LOCAL_MODEL_FOLDER=/home/seldon/models but the Makefile reads the variable as blank. Adding the following to the Makefile in ./scheduler produces the following output during make deploy-local:

Makefile:277: "Checking for exported folder"
Makefile:278: 
Makefile:279: 

Showing the error when loading the model using the following example:

seldon model load -f ./samples/models/sklearn-iris-local.yaml
-->
{}

seldon model list
-->
model	state		reason
-----	-----	------
iris	ModelFailed	Failed to sync/copy /mnt/models/iris to /mnt/agent/rclone/3198703648 Failed rclone request to host:rclone-mlserver port:5572 path:/sync/copy

Any advice is welcome. Thanks for any help!

To reproduce

New install following this guide: https://docs.seldon.io/projects/seldon-core/en/v2/contents/getting-started/docker-installation/index.html#local-models

First I installed seldon using the Preparation section and then ran using the Local Models section + the given example model located at /samples/models/sklearn-iris-local.yaml.

Expected behaviour

I get the expected behavior by commenting out lines 271, 273-275 in the Makefile in ./scheduler and hard hardcoding the local file path /scheduler/model-volume.yaml.

services:

  rclone-mlserver:
    volumes:
      - type: bind
        source: /home/seldon/models  # ${LOCAL_MODEL_FOLDER}
        target: /mnt/models
        
  rclone-triton:
    volumes:
      - type: bind
        source: /home/seldon/models  # ${LOCAL_MODEL_FOLDER}
        target: /mnt/models

seldon model load -f ./samples/models/sklearn-iris-local.yaml
-->
{}

seldon model list
-->
model	state		reason
-----	-----	------
iris	ModelAvailable	

seldon model infer iris \
  '{"inputs": [{"name": "predict", "shape": [1, 4], "datatype": "FP32", "data": [[1, 2, 3, 4]]}]}'
-->
{
	"model_name": "iris_1",
	"model_version": "1",
	"id": "983bd95f-4b4d-4ff1-95b2-df9d6d089164",
	"parameters": {},
	"outputs": [
		{
			"name": "predict",
			"shape": [
				1,
				1
			],
			"datatype": "INT64",
			"parameters": {
				"content_type": "np"
			},
			"data": [
				2
			]
		}
	]
}

Environment

• System: Ubuntu 22.04.2 LTS
• Cloud Provider: Local run
• Kubernetes Cluster Version: Using docker compose version v2.18.1
• Deployed Seldon System Images: Not using Kubernetes

Model Details

Using a locally downloaded version of the model at gs://seldon-models/scv2/samples/mlserver_1.3.5/iris-sklearn (used in this example).