SmartSim
SmartSim copied to clipboard
Load ML backends with Orchestrator launch
Description
When RedisAI is used inside of the SmartSim Orchestrator to serve ML model inferences, the backends are loaded lazily by RedisAI. It has been observed that sometimes the redis cluster is unavailable (i.e. CLUSTERDOWN
) during the lazy loading of ML backends. As a result, it might be helpful to load the ML backends during the launch of the Orchestrator so users do not encounter CLUSTERDOWN
errors during loading.
Justification
ClUSTERDOWN
errors have been observed during ML backend loading.
Implementation Strategy
The AI.CONFIG LOADBACKEND <backend> <path>
command can be used to force the loading of the backend.