server
server copied to clipboard
triton-inference-server
explicit mode load model failed
/opt/tritonserver/bin/tritonserver --model-repository=/root/gaea-serving/model/car_atti_0815/model/ --trace-file=/tmp/trace.json --trace-level=MAX --trace-rate=100 --model-control-mode=explicit --load-model=* --pinned-memory-pool-byte-size=1073741824 --backend-config=tensorflow,version=2
when i use this command to startup tritonserver, the expectation was to start all models, but the reality is that none of them started, --load-model arg is ineffective, my version is 21.10.
Hi @zbh0323 Is your --model-repository=/root/gaea-serving/model/car_atti_0815/model/ pointing to your root directory for your models? Triton expects this path to be the top level directory for all models to be loaded.
Also I noticed you are using 21.10. That was from almost a year ago. Our newest release is 22.08. Can you try that?
Hi @zbh0323 Is your
--model-repository=/root/gaea-serving/model/car_atti_0815/model/pointing to your root directory for your models? Triton expects this path to be the top level directory for all models to be loaded.
yes, it's the top level directory, u can see this also
/opt/tritonserver/bin/tritonserver --model-repository=/root/gaea-serving/model/model_person_attr_v2/ --trace-file=/tmp/trace.json --trace-level=MAX --trace-rate=100 --model-control-mode=explicit --load-model=* --pinned-memory-pool-byte-size=1073741824 --backend-config=tensorflow,version=2
Also I noticed you are using 21.10. That was from almost a year ago. Our newest release is 22.08. Can you try that?
Our image has a lot of custom parts, and it is more troublesome to upgrade. Now, are you sure that 21.10 has this problem that the model cannot be loaded?
jbkyang-nvi
@jbkyang-nvi
Also I noticed you are using 21.10. That was from almost a year ago. Our newest release is 22.08. Can you try that?
Our image has a lot of custom parts, and it is more troublesome to upgrade. Now, are you sure that 21.10 has this problem that the model cannot be loaded?
@zbh0323 note the feature of --load-models=* was added a few months ago PR here. If you would like to keep using 21.10, you will need to specify all the models you would like to load like
tritonserver --model-repository=/root/gaea-serving/model/model_person_attr_v2/ --model-control-mode=explicit --load-model=classifer --load-model=detect-attr
Closing issue due to lack of activity. Please re-open the issue if you would like to follow up with this issue.
