Manually scaling source vertex causing inconsistent behavior

[kohlisid]

Describe the bug A clear and concise description of what the bug is.

• We are able to set the replicas above the max scale limit defined. Eg: Max defined was 10 but with a kubectl scale vtx --replicas=14, able to succeffully scale beyond max limit.
• Additionally, on updating the vertex spec we do not reconcile the changes to the pipeline spec, which keeps on showing the older scale and replica values.

Due to this there is a side effect in the rater where we see inconsistent processing number, due to the active pod tracker not accounting for these new pods.

To Reproduce Steps to reproduce the behavior:

1. Create a pipeline with a generator source, and max, min scale defined
2. Manually scale the vertex, edit the vertex spec with edit vtx ___
3. Check the pipeline spec if it shows the older values

Expected behavior A clear and concise description of what you expected to happen.

Screenshots

  name: in
  pipelineName: map-pl
  replicas: 15
  scale:
    max: 14
    min: 14
  source:
    generator:
      duration: 1s
      msgSize: 500
      rpu: 5000
      value: 100

Additional context Add any other context about the problem here.

---

Message from the maintainers:

Impacted by this bug? Give it a 👍. We often sort issues this way to know what to prioritize.

For quick help and support, join our slack channel.