MLServer icon indicating copy to clipboard operation
MLServer copied to clipboard

`mp_predict` is a blocking call

Open seldondev opened this issue 4 years ago • 0 comments
trafficstars

Currently the implementation of mp_predict is going to block the asyncio loop. This will result in only one predict / explain call to be processed, and therefore loosing some of the benefits of parallel execution.

This is essentially important for long calls (e.g. explain).

seldondev avatar Oct 18 '21 07:10 seldondev