Alter the logic of "updated" pods in WaitForControlledPodsRunning measurement

[tosi3k]

Before https://github.com/kubernetes/perf-tests/pull/1035 was merged, WaitForControlledPodsRunning barrier was simply waiting for all pods specified by a label selector to reach the running state. Such approach doesn't take into account potential rolling updates of pods belonging to certain deployment(s). https://github.com/kubernetes/perf-tests/pull/1035 introduced an additional check that the appropriate pods' specs match the deployment's template spec as a heuristic to ensure that we're finished with the rolling updates.

This approach doesn't work when there's an additional server logic altering the pods' specs, however. There are two possible ideas how to improve this:

1. User explicitly says which field should be used to distinguish old and new version, e.g. if rolling update changes some env variable or image, we can add a new WaitForControllerPodsRunning's parameter next to checkIfPodsAreUpdated like "fieldToCheck: 'spec.containers[0].env[0]'.
2. Implementing more like three way diff:
   • If the given deployment is being created for the first time, all pods inside are "updated"
   • If we do some update to deployment's spec.template.spec, we do "diff" of previous spec.template.spec with the updated state spec.template.spec (with intent to automatically find modified field) and then check if this "diff" matches a pod's spec.

Option 2 seems to be easier to use, but I'm not sure if we can implement this without larger CL's refactoring (e.g. do we store old version of each deployment)?

Option 1 seems to be easier to implement and is more explicit, but requires some manual changes for all WaitForControlledPodsRunning that wants to use "pod updated" feature.

[fejta-bot]

Issues go stale after 90d of inactivity. Mark the issue as fresh with /remove-lifecycle stale. Stale issues rot after an additional 30d of inactivity and eventually close.

If this issue is safe to close now please do so with /close.

Send feedback to sig-testing, kubernetes/test-infra and/or fejta. /lifecycle stale

[tosi3k]

/remove-lifecycle stale

[tosi3k]

/lifecycle frozen