MLServer icon indicating copy to clipboard operation
MLServer copied to clipboard

Redundancy between CLI argument, MLSERVER_MODELS_DIR and models_repository_root

Open adriangonz opened this issue 3 years ago • 0 comments
trafficstars

There are three different places where the path to where the models are located (i.e. the model repository) can be specified:

  1. CLI argument to the mlserver start <path> command.
  2. MLSERVER_MODELS_DIR environment variable, used in MLServer's default Dockerfile. The value of this environment variable gets injected into the CLI argument described in 1.. https://github.com/SeldonIO/MLServer/blob/1c9d9091baa5453edcec9f367aef799628fdf503/Dockerfile#L68
  3. The models_repository_root field of MLServer's settings.json file (and thus, the MLSERVER_MODELS_REPOSITORY_ROOT environment variable).

This can be confusing to users, and can even lead to situations where the value of 1. and 2. override the value of 3., with no clear indication of what's going on: https://github.com/SeldonIO/MLServer/blob/1c9d9091baa5453edcec9f367aef799628fdf503/mlserver/cli/serve.py#L30-L31

Therefore, to make it less confusing, it would be good to unify both 2. and 3. above, so that there's a single models_repository_root (and / or MLSERVER_MODELS_REPOSITORY_ROOT env var) where users can define the model's repository root. Note that this can still lead to conflicts where 1. defines a different value to what the settings are (e.g. running MLSERVER_MODELS_REPOSITORY_ROOT='foo' mlserver start 'bar'). However, in these cases, it may be enough to just emit a warning (stating that a different model repository was specified through the CLI), and use the former.

adriangonz avatar Sep 30 '22 14:09 adriangonz