volcano
volcano copied to clipboard
scheduler update job failed
What happened: A large number of error logs
E0907 03:39:28.604985 1 job_updater.go:105] Failed to update job <video/pgvx20230907113107.701322940-098ad-45e77d6a-f3d6-42a2-9002-c3b9f7c4c506>: Operation cannot be fulfilled on podgroups.scheduling.volcano.sh "pgvx20230907113107.701322940-098ad-45e77d6a-f3d6-42a2-9002-c3b9f7c4c506": StorageError: invalid object, Code: 4, Key: /registry/scheduling.volcano.sh/podgroups/video/pgvx20230907113107.701322940-098ad-45e77d6a-f3d6-42a2-9002-c3b9f7c4c506, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 265232bc-3fe2-47cb-8335-0808e5a569a4, UID in object meta:
E0907 03:39:28.636600 1 job_updater.go:105] Failed to update job <video/gvx20230907113235.385478059-2deeb-c3fdcbe3-6f70-44ed-ae32-3833abbfc01f>: Operation cannot be fulfilled on podgroups.scheduling.volcano.sh "gvx20230907113235.385478059-2deeb-c3fdcbe3-6f70-44ed-ae32-3833abbfc01f": StorageError: invalid object, Code: 4, Key: /registry/scheduling.volcano.sh/podgroups/video/gvx20230907113235.385478059-2deeb-c3fdcbe3-6f70-44ed-ae32-3833abbfc01f, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 673133c8-ba6a-468c-820b-b2c540769575, UID in object meta:
E0907 03:39:28.698327 1 job_updater.go:105] Failed to update job <video/gvx20230907113230.237306638-fcab6-af772ecb-df01-4977-ac5c-d30d7275b365>: Operation cannot be fulfilled on podgroups.scheduling.volcano.sh "gvx20230907113230.237306638-fcab6-af772ecb-df01-4977-ac5c-d30d7275b365": StorageError: invalid object, Code: 4, Key: /registry/scheduling.volcano.sh/podgroups/video/gvx20230907113230.237306638-fcab6-af772ecb-df01-4977-ac5c-d30d7275b365, ResourceVersion: 0, AdditionalErrorMsg: Precondition failed: UID in precondition: 020110aa-deca-4503-a635-87eb75e09ad2, UID in object meta:
What you expected to happen:
How to reproduce it (as minimally and precisely as possible):
Anything else we need to know?:
Environment:
- Volcano Version: v1.7.0
- Kubernetes version (use
kubectl version): v1.20.15 - Cloud provider or hardware configuration:
- OS (e.g. from /etc/os-release):
- Kernel (e.g.
uname -a): - Install tools:
- Others:
/assign @Mufengzhe
@william-wang: GitHub didn't allow me to assign the following users: Mufengzhe.
Note that only volcano-sh members, repo collaborators and people who have commented on this issue/PR can be assigned. Additionally, issues/PRs can only have 10 assignees at the same time. For more information please see the contributor guide
In response to this:
/assign @Mufengzhe
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.
I did not find the problem you described. Can you provide more information to help me reproduce this problem?
I took a look at the code, and there's only one place that could be causing this error.