Invalid argument: ensemble 'ensemble' depends on 'tensorrt_llm' which has no loaded version. Model 'tensorrt_llm' loading failed with error

[nikhilcms]

Hello,

can anyone explain how did you solved below error ?: E0625 08:07:38.858565 1073 model_repository_manager.cc:579] Invalid argument: ensemble 'ensemble' depends on 'tensorrt_llm' which has no loaded version. Model 'tensorrt_llm' loading failed with error: version 1 is at UNAVAILABLE state: Invalid argument: unable to find backend library for backend '${triton_backend}', try specifying runtime on the model configuration.;

using tensorrt_backend=0.10.0 I have generated engine file and trying to deploy model using below command:

tritonserver --model-repository=path-to-tensorrt-engine --model-control-mode=explicit --load-model=preprocessing --load-model=postprocessing --load-model=tensorrt_llm --load-model=tensorrt_llm_bls --load-model=ensemble --log-verbose=2 --log-info=1 --log-warning=1 --log-error=1

[penPenf28]

Same problem

[manickavela29]

Facing the same issue with 0.11.0 of tensorrt_llm_backend

[nikhilcms]

Inside tesorrt_llm dir, open config.pbtxt file and change backend from ${triton_backend} to tensorrtllm

[manickavela29]

this worked, Thanks

[github-actions[bot]]

This issue is stale because it has been open 30 days with no activity. Remove stale label or comment or this will be closed in 15 days."