gluster
parallel volume deletion requests fail with device or resource busy for some of the PVs
On a GCS deployment (with brick multiplexing enabled), when PV deletion requests are sent on parallel, some of volume delete requests fail with device or resource busy error which means the lv unmount has failed as the underlying brick might be holding some reference. However the surprising fact was ls on the brick path resulted in 'no such file or directory' error.
Unfortunately I couldn't persist the logs when this issue was seen however this is fairly reproducible. The only thing what we need to do here is scale 100 PVCs (with smart volumes) on a GD2 cluster and start sending parallel volume stop/delete requests. On GCS deployment sending parallel PV delete requests from kube would replicate the same problem.
Taking this out from GCS/1.0 tag considering we're not going to make brick multiplexing a default option in GCS/1.0 release.