cloud-provider-openstack icon indicating copy to clipboard operation
cloud-provider-openstack copied to clipboard

[csi-cinder-plugin] Pods stuck in "ContainerCreating" because a volume could not be formatted and mounted

Open NymanRobin opened this issue 1 year ago • 0 comments
trafficstars

What happened: When creating pods with a cinder volume, they may occasionally get stuck in the 'ContainerCreating' state. Upon closer examination, the disk appears with a size of 0B. This issue likely arises because occasionally hypervisors don't send a new volume size to the kernel when a volume is in use.

However, if we force a rescan of the block device on the node by executing echo "1" > /sys/class/block/[device]/device/rescan, the pod will start

What you expected to happen:

The pod to be started normally

How to reproduce it:

Issue it not consistently reproducible

Would it be possible to add the rescan bit during provisioning, not only on resize? I understand there might be a slight challenge since not all devices support this. However, if the rescan-on-resize is set, could we also set the rescan bit?

Similar to the probeVolume case where "- - -" is set to /scan, we would also set "1" to /rescan as the "- - -" won't cover the case were the size is incorrectly reported

/kind bug

NymanRobin avatar May 06 '24 17:05 NymanRobin