server
server copied to clipboard
triton-inference-server
Why is my model in ensemble receiving out-of-order input
The above picture is my ensemble logic diagram. The output of my preprocess is (x0,y0), (x1,y1), (x2,y2), and then x is sent to dali, and the dali results are sent to subsequent models. The results of the models were sent to postprocess, and the corresponding y was sent directly to postprocess, but I found that the x and y received by postprocess were not from the same requets. For example, x0_models_result and y1 are placed in the same postprocess request. I expect x0_models_request and y0 to be placed in the same request. What's going on and how can we fix it? (All models in my ensemble have preserve_ording added to dynamic_batching) Here's my ensemble's pbtxt below， I hope you can help me. Thank you
name: "bls"
platform: "ensemble"
max_batch_size: 32
input [
{
name: "INPUT"
data_type: TYPE_STRING
dims: [ -1 ]
}
]
output [
{
name: "OUTPUT"
data_type: TYPE_STRING
dims: [ -1 ]
}
]
ensemble_scheduling {
step [
{
model_name: "preprocess"
model_version: 1
input_map {
key: "preprocess_input"
value: "INPUT"
}
output_map {
key: "preprocess_output_0"
value: "image_224"
}
output_map {
key: "preprocess_output_1"
value: "image_384"
}
output_map {
key: "preprocess_output_2"
value: "meta_info_out"
}
},
{
model_name: "bloody_small"
model_version: 1
input_map {
key: "input_0"
value: "image_384"
}
output_map {
key: "output_0"
value: "bloody_small_out"
}
},
{
model_name: "bloody_family_bls"
model_version: 1
input_map {
key: "input_0"
value: "bloody_small_out"
}
input_map {
key: "input_1"
value: "image_384"
}
output_map {
key: "output_0"
value: "bloody_family_out"
}
},
{
model_name: "people_congress"
model_version: 1
input_map {
key: "input_0"
value: "image_224"
}
output_map {
key: "output_0"
value: "people_congress_out"
}
},
{
model_name: "scene"
model_version: 1
input_map {
key: "input_0"
value: "image_224"
}
output_map {
key: "output_0"
value: "scene_out"
}
},
{
model_name: "bao_luan"
model_version: 1
input_map {
key: "input_0"
value: "image_224"
}
output_map {
key: "output_0"
value: "bao_luan_out"
}
},
{
model_name: "e_ya_sha"
model_version: 1
input_map {
key: "input_0"
value: "image_384"
}
output_map {
key: "output_0"
value: "e_ya_sha_out"
}
},
{
model_name: "meng_shou_pai_dui"
model_version: 1
input_map {
key: "input_0"
value: "image_384"
}
output_map {
key: "output_0"
value: "meng_shou_pai_dui_out"
}
},
{
model_name: "minecraft"
model_version: 1
input_map {
key: "input_0"
value: "image_384"
}
output_map {
key: "output_0"
value: "minecraft_out"
}
},
{
model_name: "postprocess"
model_version: 1
input_map {
key: "bloody_family"
value: "bloody_family_out"
}
input_map {
key: "people_congress"
value: "people_congress_out"
}
input_map {
key: "scene"
value: "scene_out"
}
input_map {
key: "bao_luan"
value: "bao_luan_out"
}
input_map {
key: "e_ya_sha"
value: "e_ya_sha_out"
}
input_map {
key: "meng_shou_pai_dui"
value: "meng_shou_pai_dui_out"
}
input_map {
key: "minecraft"
value: "minecraft_out"
}
input_map {
key: "meta_info"
value: "meta_info_out"
}
output_map {
key: "postprocess_out"
value: "OUTPUT"
}
}
]
}
Hi @Joenhle, thanks for fling the issue. I was wondering which version of Triton do you use? Can you please provide a minimum reproducer for us to investigate? The order should be correct for your use case.
Closing due to lack of activity. Please re-open the issue if you would like to follow up with this issue.