tensorflow serving does't reload model when models.config change

[DachuanZhao]

Bug Report

System information

• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): ubuntu 20.04
• TensorFlow Serving installed from (source or binary): docker
• TensorFlow Serving version: docker

Describe the problem

I run tensorflow serving in docker :

sudo docker run -p 8501:8501 -p 8500:8500 -d --name="tf_serving" \
--mount type=bind,source=/datas/codes/tf_model,target=/models \
-t tensorflow/serving \
--model_config_file_poll_wait_seconds=60 \
--allow_version_labels_for_unavailable_models=true \
--enable_batching=true \
--model_config_file=/models/config/model.config \
--batching_parameters_file=/models/config/batch_size.config

and my /models is :

├── config
│   ├── batch_size.config
│   └── model.config
└── push
    └── lr
        ├── 20201016
        │   ├── assets
        │   ├── saved_model.pb
        │   └── variables
        │       ├── variables.data-00000-of-00001
        │       └── variables.index
        ├── 20201210
        │   ├── assets
        │   ├── saved_model.pb
        │   └── variables
        │       ├── variables.data-00000-of-00001
        │       └── variables.index
        └── 20210118
            ├── assets
            ├── saved_model.pb
            └── variables
                ├── variables.data-00000-of-00001
                └── variables.index

When I change model.config from

model_config_list {
    config {
        name: 'push_lr'
        base_path: '/models/push/lr'
        model_platform: 'tensorflow'
        model_version_policy {
            specific {
                versions: 20201016
                versions: 20201210
            }
        }
        version_labels {
            key: 'stable'
            value: 20201016
        }
        version_labels {
            key: 'abtest1'
            value: 20201210
        }
    }
}

to

model_config_list {
    config {
        name: 'push_lr'
        base_path: '/models/push/lr'
        model_platform: 'tensorflow'
        model_version_policy {
            specific {
                versions: 20201210
                versions: 20210118
            }
        }
        version_labels {
            key: 'stable'
            value: 20210118
        }
        version_labels {
            key: 'abtest1'
            value: 20201210
        }
    }
}

tensorflow serving doesn't reload models .

[luweiagi]

I also meet this problems too, I have set the param of --model_config_file_poll_wait_seconds=10, and I hope that when I add a pd model and modify the model.config, the tfserving can reload automaticlly. But tfserving actually doesn't reload new model.

[yick2232]

我是通过docker运行的tensorflow/serving:1.13.1，设置参数--model_config_file_poll_wait_seconds=1，同样也是发现修改了models.conf没有生效，查看了docker 的日志，发现第一句就是“unknown argument: --model_config_file_poll_wait_seconds=1”，所以，应该是版本还没有支持通过model_config_file 来进行模型热更新。 具体支持通过model_config_file来进行模型热更新的版本，我还没确定，不过最近的版本肯定支持。

[yick2232]

https://github.com/tensorflow/serving/commit/c3eeed4f245e43f6cf92329d251e2b9d6255d6e5?diff=unified 这里详细描述添加了参数--model_config_file_poll_wait_seconds的时间和版本，1.15以下的版本都不支持通过model_config_file来进行模型热更新；

[singhniraj08]

@DachuanZhao,

As mentioned in Model Server Configuration, we can instruct Tensorflow Serving to periodically poll for updated versions of the configuration file at the specifed path by setting the --model_config_file_poll_wait_seconds flag.

If model A was present in the first config file, which is replaced with a file that contains only model B, the server will load model B and unload model A.

Please try with latest TF serving docker image and let us know if you face any issues. Thank you!

[singhniraj08]

Closing this due to inactivity. Please take a look into the answers provided above, feel free to reopen and post your comments(if you still have queries on this). Thank you!