Pipeline never paused

[juliev0]

Describe the bug The Numaplane e2e test passes most of the time but I want it to pass 100% of the time. This Issue describes a case.

This is the sequence of steps it had executed up until when the Pipeline never got paused:

• create isbsvc, wait for it and StatefulSet to be reconciled
• once it's ready, create pipeline
• wait for it to be "Running"
• pause pipeline
• wait for it to be "Paused"
• update spec, keeping desiredPhase=Paused (in this case it makes a minor spec change to source.generator.rpu)
• wait for it to be reconciled
• run pipeline
• wait for it be "Running"
• repeat steps above starting from "pause pipeline"
• then "pause pipeline" one more time <-- this is where it got stuck

Pipeline spec:

pipelineSpec        = numaflowv1.PipelineSpec{
		InterStepBufferServiceName: isbServiceRolloutName,
		Vertices: []numaflowv1.AbstractVertex{
			{
				Name: "in",
				Source: &numaflowv1.Source{
					Generator: &numaflowv1.GeneratorSource{
						RPU:      &pipelineSpecSourceRPU,
						Duration: &pipelineSpecSourceDuration,
					},
				},
				Scale: numaflowv1.Scale{Min: &numVertices, Max: &numVertices, ZeroReplicaSleepSeconds: &zeroReplicaSleepSec},
			},
			{
				Name: "out",
				Sink: &numaflowv1.Sink{
					AbstractSink: numaflowv1.AbstractSink{
						Log: &numaflowv1.Log{},
					},
				},
				Scale: numaflowv1.Scale{Min: &numVertices, Max: &numVertices, ZeroReplicaSleepSeconds: &zeroReplicaSleepSec},
			},
		},
		Edges: []numaflowv1.Edge{
			{
				From: "in",
				To:   "out",
			},
		},
	}

This zip file contains the following:

• full Numaflow log file, to which I added a line "NUMAPLANE LOG" with corresponding time stamp each time Numaplane performs some action (creating isbsvc, creating pipeline, updating pipeline)
• result of kubectl get pipeline -o yaml at the time it was remaining pausing

This is the full numaplane log.

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We often sort issues this way to know what to prioritize.

For quick help and support, join our slack channel.