MLServer
MLServer copied to clipboard
`mp_predict` is a blocking call
trafficstars
Currently the implementation of mp_predict is going to block the asyncio loop. This will result in only one predict / explain call to be processed, and therefore loosing some of the benefits of parallel execution.
This is essentially important for long calls (e.g. explain).