Dali backend support feature to handle init function

[farzanehnakhaee70]

Is your feature request related to a problem? Please describe. Currently dali_backend just works with post and preprocessing for the functions which do not requires initialization. What if when the init part takes longer time? Is there any solution for this?

[Tabrizian]

cc @szalpal @jantonguirao

[JanuszL]

Hi @farzanehnakhaee70,

Could you tell us more about what do you mean by:

Currently dali_backend just works with post and preprocessing for the functions which do not require initialization

In the case of DALI if you load the processing model all the necessary operators' initialization happens. In most cases, it is just the memory allocation. Do you have any particular operation in mind which may require extended initialization?

[farzanehnakhaee70]

Hi @JanuszL , Thanks a lot for your answer. The problem for my side is I'm wondering if I can use dali backend for transformer like models and import the tokenizer in the initialization part and afterward use the initialized tokenizer during the processing (as it takes time for initializing the tokenizers during each execution). Is it possible?

[JanuszL]

Hi @farzanehnakhaee70,

DALI is initialized once when the model is loaded. Then when the inference request is processed the pipeline is just run, However I'm not sure if DALI is capable of tokenization. You may consider writing a custom plugin for DALI https://github.com/triton-inference-server/dali_backend/blob/main/docs/examples/dali_plugin/README.md.