MLServer
MLServer copied to clipboard
Stop (optionally) batch processing when failure happens
Add an option to the mlserver infer command to stop batch processing when failure happened and retries completed in Core and MLServer.
When failure happens and can’t recovered after a number of retries, the mlserver infer process could write back the existing output data to file and exit with summary object that contains exit reasons.