SmartSim icon indicating copy to clipboard operation
SmartSim copied to clipboard

Load ML backends with Orchestrator launch

Open mellis13 opened this issue 3 years ago • 0 comments

Description

When RedisAI is used inside of the SmartSim Orchestrator to serve ML model inferences, the backends are loaded lazily by RedisAI. It has been observed that sometimes the redis cluster is unavailable (i.e. CLUSTERDOWN) during the lazy loading of ML backends. As a result, it might be helpful to load the ML backends during the launch of the Orchestrator so users do not encounter CLUSTERDOWN errors during loading.

Justification

ClUSTERDOWN errors have been observed during ML backend loading.

Implementation Strategy

The AI.CONFIG LOADBACKEND <backend> <path> command can be used to force the loading of the backend.

mellis13 avatar Oct 14 '21 20:10 mellis13