digdag
digdag copied to clipboard
Backfill sessions are not sequential after unpause
When you pause/un-pause a workflow the backfill behavior switches to parallel backfill rather than the expected sequential backfill. For example, a workflow with a one minute interval schedule that is paused for 10 minutes will run 10 parallel sessions as soon as it is un-paused. In contrast, a backfill command with the count set to 10 will run those 10 backfill sessions sequentially. I believe the expected behavior is to run things sequentially by giving the un-pause/backfill operation a chance to properly cycle through all the session times that were missed. This seems to be a bug. The impact is that multiple jobs running in parallel fail due to resource dependencies.
Digdag server version: 0.9.31 Database: PostgreSQL Log/archive storage: S3