MLServer icon indicating copy to clipboard operation
MLServer copied to clipboard

Adding model warm up option

Open saeid93 opened this issue 2 years ago • 0 comments

Similar to the TFServing and Triton Server it would be nice to have the option of warming up a model (It has a huge impact on models' latency, especially in the case of Deep learning models) as a built-in feature in MLServer. Currently, I am using a for loop in my custom MLServer model to do a few inferences on load but since the feature exists in a similar platform I want to open the discussion for adding it to the MLServer.

saeid93 avatar Feb 01 '23 21:02 saeid93