[BUG] Data is lost after the primary and standby cluster is restarted

[JashBook]

Describe the bug A clear and concise description of what the bug is.

To Reproduce Steps to reproduce the behavior:

1. create cluster

apiVersion: apps.kubeblocks.io/v1alpha1
kind: Cluster
metadata:
  name: redis-nhdkgq
  namespace: default
spec:
  clusterDefinitionRef: redis
  clusterVersionRef: redis-7.0.6
  terminationPolicy: WipeOut
  componentSpecs:
    - name: redis
      componentDef: redis
      replicas: 2
      resources:
        requests:
          cpu: 100m
          memory: 0.5Gi
        limits:
          cpu: 100m
          memory: 0.5Gi
      switchPolicy:
        type: Noop
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 1Gi
    - name: redis-sentinel
      componentDef: redis-sentinel
      replicas: 3
      resources:
        requests:
          cpu: 100m
          memory: 0.5Gi
        limits:
          cpu: 100m
          memory: 0.5Gi
      volumeClaimTemplates:
        - name: data
          spec:
            storageClassName:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 1Gi

      `kbcli cluster list-instances redis-nhdkgq --namespace default `
    
NAME                            NAMESPACE   CLUSTER        COMPONENT        STATUS    ROLE      ACCESSMODE   AZ       CPU(REQUEST/LIMIT)   MEMORY(REQUEST/LIMIT)   STORAGE    NODE                    CREATED-TIME                 
redis-nhdkgq-redis-0            default     redis-nhdkgq   redis            Running   primary   <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:01 UTC+0800   
redis-nhdkgq-redis-1            default     redis-nhdkgq   redis            Running   primary   <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:01 UTC+0800   
redis-nhdkgq-redis-sentinel-0   default     redis-nhdkgq   redis-sentinel   Running   <none>    <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:01 UTC+0800   
redis-nhdkgq-redis-sentinel-1   default     redis-nhdkgq   redis-sentinel   Running   <none>    <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:01 UTC+0800   
redis-nhdkgq-redis-sentinel-2   default     redis-nhdkgq   redis-sentinel   Running   <none>    <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:01 UTC+0800   

set data

      `echo "set mykey 'ndupatzcct'" | kbcli cluster connect redis-nhdkgq --namespace default `
    
Connect to instance redis-nhdkgq-redis-1: out of redis-nhdkgq-redis-1(primary), redis-nhdkgq-redis-0(primary)
Unable to use a TTY - input is not a terminal or the right kind of file
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
OK

test failover

      `kubectl delete pod redis-nhdkgq-redis-0  --namespace default `
    
pod "redis-nhdkgq-redis-0" deleted
failover pod name:redis-nhdkgq-redis-1
failover  Success

stop start ops

      `kbcli cluster stop redis-nhdkgq --auto-approve  --namespace default `
    
OpsRequest redis-nhdkgq-stop-hm79h created successfully, you can view the progress:
	kbcli cluster describe-ops redis-nhdkgq-stop-hm79h -n default
check cluster status

      `kbcli cluster list redis-nhdkgq --show-labels  --namespace default `
    
NAME           NAMESPACE   CLUSTER-DEFINITION   VERSION       TERMINATION-POLICY   STATUS    CREATED-TIME                 LABELS                                                                                                                             
redis-nhdkgq   default     redis                redis-7.0.6   WipeOut              Stopped   Jan 25,2024 15:01 UTC+0800   app.kubernetes.io/instance=redis-nhdkgq,clusterdefinition.kubeblocks.io/name=redis,clusterversion.kubeblocks.io/name=redis-7.0.6   

      `kbcli cluster start redis-nhdkgq --namespace default `
    
OpsRequest redis-nhdkgq-start-cd77h created successfully, you can view the progress:
	kbcli cluster describe-ops redis-nhdkgq-start-cd77h -n default
check cluster status


      `kbcli cluster list-instances redis-nhdkgq --namespace default `
    
NAME                            NAMESPACE   CLUSTER        COMPONENT        STATUS    ROLE      ACCESSMODE   AZ       CPU(REQUEST/LIMIT)   MEMORY(REQUEST/LIMIT)   STORAGE    NODE                    CREATED-TIME                 
redis-nhdkgq-redis-0            default     redis-nhdkgq   redis            Running   primary   <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:04 UTC+0800   
redis-nhdkgq-redis-1            default     redis-nhdkgq   redis            Running   primary   <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:04 UTC+0800   
redis-nhdkgq-redis-sentinel-0   default     redis-nhdkgq   redis-sentinel   Running   <none>    <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:04 UTC+0800   
redis-nhdkgq-redis-sentinel-1   default     redis-nhdkgq   redis-sentinel   Running   <none>    <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:04 UTC+0800   
redis-nhdkgq-redis-sentinel-2   default     redis-nhdkgq   redis-sentinel   Running   <none>    <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:04 UTC+0800   
check pod status done
check cluster connect
check cluster connect done
check ops status

hscale cluster

      `kbcli cluster hscale redis-nhdkgq --auto-approve --components redis --replicas 3 --namespace default `
    
OpsRequest redis-nhdkgq-horizontalscaling-2t465 created successfully, you can view the progress:
	kbcli cluster describe-ops redis-nhdkgq-horizontalscaling-2t465 -n default
check cluster status

    
NAME                            NAMESPACE   CLUSTER        COMPONENT        STATUS    ROLE        ACCESSMODE   AZ       CPU(REQUEST/LIMIT)   MEMORY(REQUEST/LIMIT)   STORAGE    NODE                    CREATED-TIME                 
redis-nhdkgq-redis-0            default     redis-nhdkgq   redis            Running   secondary   <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:04 UTC+0800   
redis-nhdkgq-redis-1            default     redis-nhdkgq   redis            Running   primary     <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:04 UTC+0800   
redis-nhdkgq-redis-2            default     redis-nhdkgq   redis            Running   secondary   <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:06 UTC+0800   
redis-nhdkgq-redis-sentinel-0   default     redis-nhdkgq   redis-sentinel   Running   <none>      <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:04 UTC+0800   
redis-nhdkgq-redis-sentinel-1   default     redis-nhdkgq   redis-sentinel   Running   <none>      <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:04 UTC+0800   
redis-nhdkgq-redis-sentinel-2   default     redis-nhdkgq   redis-sentinel   Running   <none>      <none>       <none>   100m / 100m          512Mi / 512Mi           data:1Gi   minikube/192.168.49.2   Jan 25,2024 15:04 UTC+0800   
check pod status done
check cluster connect
check cluster connect done
check ops status

get data nil

kbcli cluster connect redis-nhdkgq --namespace default 
Connect to instance redis-nhdkgq-redis-1: out of redis-nhdkgq-redis-1(primary), redis-nhdkgq-redis-0(secondary), redis-nhdkgq-redis-2(secondary)
Warning: Using a password with '-a' or '-u' option on the command line interface may not be safe.
127.0.0.1:6379> get mykey
(nil)
127.0.0.1:6379> 

Expected behavior Data no lost after the primary and standby cluster is restarted.

Screenshots If applicable, add screenshots to help explain your problem.

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Additional context Add any other context about the problem here.