digdag
digdag copied to clipboard
[feature request] Add worker agent information to task queue
I am trying to run digdag on kubernetes. Most functions work properly on kubernetes.
However, it is difficult to handle the scale-out and scale-in behavior of the digdag worker agent. If scale out operation exceeds max-task-threads of all worker agents in task queue, scale out must be performed. However, the scale-in operation cannot be performed unless the task queue is empty. Because we don't know which worker agent is processing the task. This crd of kubernetes(https://github.com/TrsNium/digdag-worker-crd) is the above scale-in and scale-out behavior.
Therefore, we want to add information on which worker agent is processing the task in the task queue and make the scale-in and scale-out processing flexible.