MLServer icon indicating copy to clipboard operation
MLServer copied to clipboard

Published 20 hours ago • verified publisher icon SeldonIO

Adaptive Batching is enabled but not supported for inference streaming.

Open rachidrebik opened this issue 1 year ago • 9 comments

Hi,

I am receiving the following warning on my custom model:

"WARNING: Adaptive Batching is enabled for model 'models' but not supported for inference streaming. Falling back to non-batched inference streaming."

What is the problem?

rachidrebik avatar Sep 19 '24 12:09 rachidrebik