tidb-operator icon indicating copy to clipboard operation
tidb-operator copied to clipboard

1.5.1 can not upgrade tidb 6.5 to 7.5 due to not support alibaba oss volume

Open imhun opened this issue 1 year ago • 3 comments

Bug Report

What version of Kubernetes are you using?

1.24.6 What version of TiDB Operator are you using?

1.5.1 What storage classes exist in the Kubernetes cluster and what are used for PD/TiKV pods?

What's the status of the TiDB cluster pods?

image

What did you do?

helm upgrade install tidb-operator 1.5.1 edit tidbcluster.yaml to 7.5

pd and tiflash was updated from 6.5 to 7.,5 but tikv was not updated,still 6.5

and tidb-controller-manager error log: 2023-12-12T09:35:29.254709747+08:00 W1212 01:35:29.254643 1 sync_volume_status.go:80] skip to observe volume status for pod test-tidb/test-tikv-1, because failed to get actual volumes: storageclass.storage.k8s.io "oss" not found

2023-12-12T09:35:29.254726266+08:00 W1212 01:35:29.254665 1 sync_volume_status.go:80] skip to observe volume status for pod test-tidb/test-tikv-0, because failed to get actual volumes: storageclass.storage.k8s.io "oss" not found

2023-12-12T09:35:29.254728926+08:00 W1212 01:35:29.254672 1 sync_volume_status.go:80] skip to observe volume status for pod test-tidb/test-tikv-2, because failed to get actual volumes: storageclass.storage.k8s.io "oss" not found What did you expect to see? tikv upgrade to 7.5,or all tidb rollback to 6.5 What did you see instead? pd and tiflash is 7.5,but tikv and tidb is still 6.5

imhun avatar Dec 12 '23 01:12 imhun

The above is just some warn logs, are there more logs?

csuzhangxc avatar Dec 14 '23 03:12 csuzhangxc

no more logs,upgrade is stuck because the pvc check failed it should ignore this error,because the old version tikv pod was already running it should check it when create new pod instead of checking what already exists

imhun avatar Dec 26 '23 08:12 imhun

but the continue wouldn't block reconciliation,

https://github.com/pingcap/tidb-operator/blob/v1.5.1/pkg/manager/volumes/sync_volume_status.go#L79-L82

csuzhangxc avatar Dec 27 '23 02:12 csuzhangxc