ClusterODM icon indicating copy to clipboard operation
ClusterODM copied to clipboard

Add support for autoscaling queue

Open pierotofy opened this issue 5 years ago • 2 comments

Optional support for an autoscaling queue could allow users to queue tasks even if they hit account limit restrictions and provide a better flow.

pierotofy avatar Aug 24 '19 17:08 pierotofy

This is exactly what i need. So the feature would do something like this:

  1. Check quotas for the asr provider before autoscaling.
  2. Push the task in a fifo queue if autoscaling will cause the provider account to met its limits.
  3. With each autoscaled node being removed, get next task from the queue and run 1. again.

FJEANNOT avatar Feb 10 '22 12:02 FJEANNOT

Now that i think of it. If task files are externalized by ClusterODM instead of written on disk, and an external message queue is used, this could be easy to setup multiple clusterODMs on multiple asr providers accounts

FJEANNOT avatar Feb 10 '22 12:02 FJEANNOT