BentoML icon indicating copy to clipboard operation
BentoML copied to clipboard

triton: supports unload/load model on demand

Open aarnphm opened this issue 2 years ago • 0 comments

Feature request

Implement MODEL_CONTROL_MODE to be explicit and allow given model to be loaded on demand.

We should also provide ability to teardown model after a period of time, that can be configured via configuration.

Motivation

No response

Other

No response

aarnphm avatar Jan 17 '23 08:01 aarnphm