wenet
wenet copied to clipboard
wenet-e2e
Auto-generated encoder model config
I think I don't get your point.
Tritonserver supports Auto-Generated Model Configuration. So I just remove the output in the template.
Do you mean that we must define output shape explicitly?
I think I don't get your point.
Tritonserver supports Auto-Generated Model Configuration. So I just remove the output in the template.
Do you mean that we must define output shape explicitly?
It's not a must. But it will be easier for users to debug. For example, the wenetspeech offline model exported by onnx has an encoder of shape [B, T, -1], while for aishell2 u2++ model , the output shape is [B, T, 256]. After we generate the onnx model, users may encounter some issues in server part and if the config.pbtxt can provide more info, it will be more helpful.
Why not use HTTP API?
curl localhost:8000/v2/models/<model name>/config
Then we can completely avoid processing the model details.
Why not use HTTP API?
curl localhost:8000/v2/models/<model name>/configThen we can completely avoid processing the model details.
This suggestion is great! We may call other model's configuration instead of reading onnx_config.json in triton. For example, the feature_size in this line can be inferred from encoder_out tensor from encoder model. However, it's still due to the previous issue I mentioned, this feature size might be '-1', and it will fail from this line.
I'm working on updating this example to the latest triton, 22.01. The issue you mentioned will also be fixed. But I will fix this issue by reading the generated onnx model. I still believe a config file with full details will be better.