hwameistor icon indicating copy to clipboard operation
hwameistor copied to clipboard

[Large scale test]Evicte 100 pods that use storage,There is a lvmigrate stuck in SyncReplica

Open FloatXD opened this issue 1 year ago • 3 comments

Evicte 100 pods that use storage,There is a lvmigrate stuck in SyncReplica

[root@k8s-master ~]# kubectl describe lvmigrate evictor-pvc-77849b03-8306-4f50-8e91-d0075acdd90f
Name:         evictor-pvc-77849b03-8306-4f50-8e91-d0075acdd90f
Namespace:    
Labels:       <none>
Annotations:  <none>
API Version:  hwameistor.io/v1alpha1
Kind:         LocalVolumeMigrate
Metadata:
  Creation Timestamp:  2023-12-11T07:29:49Z
  Finalizers:
    hwameistor.io/eviction-protect
  Generation:        1
  Resource Version:  288902
  UID:               9ed451d2-d2b7-4d30-b88a-08989e252b8e
Spec:
  Abort:             false
  Migrate All Vols:  true
  Source Node:       k8s-node1
  Target Nodes Suggested:
  Volume Name:  pvc-77849b03-8306-4f50-8e91-d0075acdd90f
Status:
  Message:                  waiting for the sync job to complete: evictor-pvc-77849b03-8306-datacopy-local-storage-pvc-lvm30
  Original Replica Number:  1
  State:                    SyncReplica
  Target Node:              k8s-node2
  Volumes:
    pvc-77849b03-8306-4f50-8e91-d0075acdd90f
Events:  <none>
[root@k8s-master ~]# kubectl get pod -n hwameistor
NAME                                                       READY   STATUS             RESTARTS      AGE
hwameistor-admission-controller-d684c7b75-88n5w            1/1     Running            0             22m
hwameistor-admission-controller-d684c7b75-f8sm9            1/1     Terminating        2 (58m ago)   3d5h
hwameistor-admission-controller-d684c7b75-k95rf            1/1     Running            0             22m
hwameistor-admission-controller-d684c7b75-smb4v            1/1     Running            0             18m
hwameistor-apiserver-6f6579b5c-kkrt7                       1/1     Running            1 (61m ago)   3d5h
hwameistor-auditor-6778bb5df7-vssrg                        0/1     CrashLoopBackOff   8 (2m ago)    18m
hwameistor-exporter-75cc84967-bghls                        1/1     Running            1 (61m ago)   3d5h
hwameistor-failover-assistant-6c675d589b-rtr22             1/1     Running            1 (61m ago)   3d5h
hwameistor-local-disk-action-controller-6878df7545-nlj5r   1/1     Running            0             18m
hwameistor-local-disk-csi-controller-6f6fc5875b-jqwr9      2/2     Running            2 (61m ago)   3d5h
hwameistor-local-disk-manager-b2f9j                        2/2     Running            2 (61m ago)   3d5h
hwameistor-local-disk-manager-q4972                        2/2     Running            2 (61m ago)   3d5h
hwameistor-local-disk-manager-tr7dk                        2/2     Running            2 (61m ago)   3d5h
hwameistor-local-storage-c4f5w                             2/2     Running            2 (61m ago)   3d5h
hwameistor-local-storage-csi-controller-69dd5654dc-tj8jp   6/6     Running            6 (61m ago)   3d5h
hwameistor-local-storage-l6bs8                             2/2     Running            3 (59m ago)   3d5h
hwameistor-local-storage-vkf4s                             2/2     Running            2 (61m ago)   3d5h
hwameistor-pvc-autoresizer-648f4d7bbd-lcdc6                1/1     Running            1 (61m ago)   3d5h
hwameistor-scheduler-58cb757788-2gr7t                      1/1     Running            0             21m
hwameistor-scheduler-58cb757788-46r56                      1/1     Running            0             21m
hwameistor-scheduler-58cb757788-7tqpd                      1/1     Terminating        1 (61m ago)   3d5h
hwameistor-scheduler-58cb757788-x2v4n                      1/1     Running            0             18m
hwameistor-ui-78756c7557-9qn9d                             1/1     Running            1 (61m ago)   3d5h
hwameistor-volume-evictor-569cbf4bcd-w8xmh                 1/1     Running            0             18m
hwameistor-volume-evictor-569cbf4bcd-ztm6b                 1/1     Terminating        1 (61m ago)   3d5h
[root@k8s-master ~]# kubectl get lvr |grep node1
pvc-77849b03-8306-4f50-8e91-d0075acdd90f-kkch5t   20971520   k8s-node1   Ready   true     /dev/LocalStorage_PoolHDD/pvc-77849b03-8306-4f50-8e91-d0075acdd90f   52m
[root@k8s-master ~]# kubectl get lv |grep d0075acdd90f
pvc-77849b03-8306-4f50-8e91-d0075acdd90f   LocalStorage_PoolHDD   2          20971520   1134592   Ready               52m
[root@k8s-master ~]# kubectl get pvc |grep pvc-77849b03-8306-4f50-8e91-d0075acdd90f
local-storage-pvc-lvm30   Bound    pvc-77849b03-8306-4f50-8e91-d0075acdd90f   20Mi       RWO            local-storage-hdd-lvm   54m

FloatXD avatar Dec 11 '23 07:12 FloatXD