server icon indicating copy to clipboard operation
server copied to clipboard

Add model warmup functionality for ensemble models

Open Timen opened this issue 1 year ago • 6 comments

Is your feature request related to a problem? Please describe. Support model warmup for ensemble models.

Describe the solution you'd like The option to specify warmup inputs for ensemble models.

Describe alternatives you've considered Manually warming up the pipeline

Timen avatar Feb 13 '24 10:02 Timen

Hi @Timen, does setting the Model WarmUp on all composing models not helped? The ensemble model only passes inputs and outputs between composing models and does not do any real work.

kthui avatar Feb 15 '24 17:02 kthui

The effort of setting up the model warmup for each individual part of a pipeline vs just one input to the ensemble is much greater. It just is surprising and not documented that ensembles don't support it.

Timen avatar Feb 15 '24 20:02 Timen

@GuanLuo @nnshah1 do you want to file a ticket for adding warmup config into ensemble model config?

kthui avatar Feb 17 '24 00:02 kthui

@kthui @GuanLuo @nnshah1 any updates on this, this is still a very welcome feature.

Timen avatar Apr 04 '24 09:04 Timen