Rafał Banaś

Hi @frankxyy As you assume, adding `device='gpu'` argument to those operators shouldn't change anything, because they receive gpu input and their placement is inferred to be on gpu. Can you...

Hi @wangshaobobetter. I assume you're using Triton to perform inference. In that case to connect the DALI model with TensorRT network you should use medel ensembles. You can see an...

Okay, so, if I understand correctly, you have some custom piece of C++ code and you want to use it instead of DALI for preprocessing? In that case, Triton gives...

Sending binary data to Triton server through HTTP request is described [here](https://github.com/triton-inference-server/server/blob/main/docs/protocol/extension_binary_data.md#:~:text=The%20binary%20tensor%20data%20extension,field%20of%20its%20Server%20Metadata.). You don't need to adjust anything in model itself to utilize it.

@danvass Hi, I tested sending an inference request to ensemble_dali_inception model with curl. I utilized the fact, that for a single input models (+other [constraints](https://github.com/triton-inference-server/server/blob/main/docs/protocol/extension_binary_data.md#raw-binary-request)) Triton accepts raw binary requests....

@danvass The main problem with your request is the input shape. You have to specify it when sending an infer request - so in your case it would be: `"shape":[1,...

@danvass Yes, I use exactly this example. What version of Triton server are you using? The shape in config is in fact -1, but that means the dynamic dimension -...

Also, I see that the name of the model in your case is ensemble_dali, and in config it's "ensemble_dali_inception" Did you change anything else in any of the configs?

Yup. If you set the max_batch_size to 0, you have to specify the batch dimension in shape field. In case of ensemble (and most model types, including DALI), you should...

@danvass Choosing the optimal max batch size is often a matter of fine tuning. The main limitation is the memory consumption (there is a limit to how big batch can...