MLServer
MLServer copied to clipboard
Redundancy between CLI argument, MLSERVER_MODELS_DIR and models_repository_root
There are three different places where the path to where the models are located (i.e. the model repository) can be specified:
- CLI argument to the
mlserver start <path>command. MLSERVER_MODELS_DIRenvironment variable, used in MLServer's defaultDockerfile. The value of this environment variable gets injected into the CLI argument described in1.. https://github.com/SeldonIO/MLServer/blob/1c9d9091baa5453edcec9f367aef799628fdf503/Dockerfile#L68- The
models_repository_rootfield of MLServer'ssettings.jsonfile (and thus, theMLSERVER_MODELS_REPOSITORY_ROOTenvironment variable).
This can be confusing to users, and can even lead to situations where the value of 1. and 2. override the value of 3., with no clear indication of what's going on:
https://github.com/SeldonIO/MLServer/blob/1c9d9091baa5453edcec9f367aef799628fdf503/mlserver/cli/serve.py#L30-L31
Therefore, to make it less confusing, it would be good to unify both 2. and 3. above, so that there's a single models_repository_root (and / or MLSERVER_MODELS_REPOSITORY_ROOT env var) where users can define the model's repository root. Note that this can still lead to conflicts where 1. defines a different value to what the settings are (e.g. running MLSERVER_MODELS_REPOSITORY_ROOT='foo' mlserver start 'bar'). However, in these cases, it may be enough to just emit a warning (stating that a different model repository was specified through the CLI), and use the former.