MLServer
MLServer copied to clipboard
Adding model warm up option
Similar to the TFServing and Triton Server it would be nice to have the option of warming up a model (It has a huge impact on models' latency, especially in the case of Deep learning models) as a built-in feature in MLServer. Currently, I am using a for loop in my custom MLServer model to do a few inferences on load but since the feature exists in a similar platform I want to open the discussion for adding it to the MLServer.