numaflow icon indicating copy to clipboard operation
numaflow copied to clipboard

Pipeline Status reflects that it "won't pause"

Open juliev0 opened this issue 6 months ago • 1 comments

Summary

In Numaplane (PPND strategy), we pause a Pipeline and wait for it to be in "Paused" phase before safely updating.

Apparently, there are some cases in Numaflow in which the Numaflow Controller will actively make a decision not to pause and not even to pause on timeout. Numaplane needs to know this.

On one hand Numaplane could try to make this determination itself as far as when Numaflow Controller won't pause, but it seems to make more sense for Numaflow to indicate to Numaplane that this is the case through its Status.

Detailed Expectation

For any case in which a Pipeline cannot be paused, Numaflow Controller writes this information to its status so that Numaplane can read it. (The earlier this is written the better.)

Note that the logic for not pausing also needs to be revisited. Currently, it seems to be the case that it does not pause (and won't even timeout) when the daemon Pod isn't running, but that can result in situations like:


Message from the maintainers:

If you wish to see this enhancement implemented please add a 👍 reaction to this issue! We often sort issues this way to know what to prioritize.

juliev0 avatar Aug 22 '24 01:08 juliev0