[QUESTION]: Since the PV is not provisioned, the PVC becomes Pending, so I would like to resolve it.

[hara1101kazu]

How can the Team help you today?

failed to provision volume with StorageClass "<StorageClass for replication>": rpc error: code = Internal desc = can't find volume '<fictitious volume that doesn't exist>': When requesting a PV in a storage class for replication, an error is output stating that a non-existent fictitious PV cannot be found.

our environment:

• PowerStore 1200T
• OpenShift Container Platform 4.12.44
• CSI Driver for Dell PowerStore 2.7.0
• CSM for Replication 1.5.0

Details: PVC was generated at rook-ceph(Storage Orchestration for Kubernetes) https://github.com/rook/rook The generated PVC is as below

$ oc get pvc rook-ceph-mon-a -o yaml
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  annotations:
    volume.beta.kubernetes.io/storage-provisioner: csi-powerstore.dellemc.com
    volume.kubernetes.io/storage-provisioner: csi-powerstore.dellemc.com
  creationTimestamp: "2024-03-28T04:35:14Z"
  finalizers:
  - kubernetes.io/pvc-protection
  labels:
    app: rook-ceph-mon
    app.kubernetes.io/component: cephclusters.ceph.rook.io
    app.kubernetes.io/created-by: rook-ceph-operator
    app.kubernetes.io/instance: a
    app.kubernetes.io/managed-by: rook-ceph-operator
    app.kubernetes.io/name: ceph-mon
    app.kubernetes.io/part-of: rook-ceph
    ceph-version: 17.2.6-0
    ceph_daemon_id: a
    ceph_daemon_type: mon
    mon: a
    mon_cluster: pf-rook-ceph
    pvc_name: rook-ceph-mon-a
    pvc_size: 10Gi
    rook-version: v1.12.2
    rook.io/operator-namespace: pf-rook-ceph
    rook_cluster: pf-rook-ceph
  name: rook-ceph-mon-a
  namespace: pf-rook-ceph
  ownerReferences:
  - apiVersion: ceph.rook.io/v1
    blockOwnerDeletion: true
    controller: true
    kind: CephCluster
    name: rook-ceph
    uid: 22ac015f-c63d-4d55-97b3-aaf3a6d66d26
  resourceVersion: "54087199"
  uid: 493e342e-a6dc-4bd0-aa03-5c9c8a7050be
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 10Gi
  storageClassName: <StorageClass for replication>
  volumeMode: Filesystem
status:
  phase: Pending

However, the error failed to provision volume with StorageClass "<StorageClass for replication>": rpc error: code = Internal desc = can't find volume '<fictitious volume that doesn't exist>': introduced at the beginning was output. Therefore, rook-ceph itself is in Pending state as PV is not provisioned. It is confirmed from the PowerStore GUI that the PV does not exist.

It worked fine when I implemented it in other environments, so I believe this is a problem specific to this environment. However, I have reinstalled the CSI driver and replication controller, but the situation does not improve. It is unclear whether the PV is not provided because the replication has failed, or whether the replication has failed because the PV is not provided.

that's all If you have any solutions or advice regarding the issue where PV is not provisioned, please let us know.

[coulof]

@hara1101kazu, in your PVC definition you have storageClassName: <StorageClass for replication> ; are you sure you put the correct value ?

[ghost]

Thank you for your reply. Yes, I understand that.

$ oc get sc <StorageClass for replication> -o yaml
allowVolumeExpansion: true
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  creationTimestamp: "2024-04-04T11:16:49Z"
  name: <StorageClass for replication>
  resourceVersion: "29215339"
  uid: 11333a97-6cd4-4730-a9a0-00e5b1a11db3
parameters:
  arrayID: PS82f1db311b16
  csi.storage.k8s.io/fstype: ext4
  replication.storage.dell.com/ignoreNamespaces: "false"
  replication.storage.dell.com/isReplicationEnabled: "true"
  replication.storage.dell.com/remoteClusterID: cluster-1
  replication.storage.dell.com/remoteStorageClassName: <StorageClass for replication>
  replication.storage.dell.com/remoteSystem: Side3
  replication.storage.dell.com/rpo: Five_Minutes
  replication.storage.dell.com/volumeGroupPrefix: csi
provisioner: csi-powerstore.dellemc.com
reclaimPolicy: Delete
volumeBindingMode: Immediate
$

The configured storage class value is as above

[delldubey]

@hara1101kazu, By the description shared by you and going by the error, it seems the API is getting the volume it just created. To debug more, we may need to look through driver logs.

Can you please share, the driver logs, and replication controller logs?

Thanks. Utkarsh

[ghost]

@delldubey

Thank you for your reply.

Currently, the problem is no longer reproducible. PVCs are not Pending and PVs are provisioned as expected.

Since the problem cannot be reproduced, I would like to close this issue. Is this okay? If it happens again, I will share it with you, so I would appreciate it if you could consult with me.

[boyamurthy]

hi @hara1101kazu , thank you for the update and confirmation. If the issue happens again , feel free to create github issue/question and team will look into it. Closing this for now.