[ws-manager] - Operation cannot be fulfilled on pods "prebuild-<guid>": the object has been modified; please apply your changes to the latest version and try again

[kylos101]

Bug description

It looks like a finalizer is being modified, failing and we trace as an error, and then successfully try again (refer to below picture).

Logs

Message: Operation cannot be fulfilled on pods "prebuild-69cd1a21-2d04-4f27-8528-f605fc580986": the object has been modified; please apply your changes to the latest version and try again

Steps to reproduce

Unknown

Workspace affected

No response

Expected behavior

According to the trace, the operation is tried again, and was successful. Consider doing the trace, but, not as an error, if it successfully retries later.

Example repository

No response

Anything else?

No response

[jenting]

Message: Operation cannot be fulfilled on pods "prebuild-69cd1a21-2d04-4f27-8528-f605fc580986": the object has been modified; please apply your changes to the latest version and try again

It's because multiple workers are updating the same object resource at the same time.

• Worker A gets pod resource
• Worker B gets pod resource
• Worker A updates the pod resource first, and the resourveVersion updated
• Worker B updates the pod resource without re-get the pod resource, and the error message occurs

So, that's the reason why we retry when conflict https://github.com/gitpod-io/gitpod/blob/d0627ab85e5b10fa867fe82b17f2f0b7ac37e569/components/ws-manager/pkg/manager/annotations.go#L184

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.