Ziyuan Jiang

Can ragged input used together with stateful model?

1

I have a PyTorch stateful model which I successfully used in the triton server with sequence_batching direct scheduling strategy. In order to futher optimize the throughput, I want to use...