piraeusdatastore
Can not use PVC for multi Pods
I want to deploy an application with 2 pods on 1 pvc but it's not working.
- Piraeus deploy by Helm
- piraeus-server:v1.29.2, drbd-reactor:v1.6.0, drbd9-noble:v9.2.12
Please help to advise. Thanks.
RWX is not supported. I think we have a bug somewhere in the CSI driver, because it simply should not allow you to create such volumes in the first place.
thanks, @WanzenBug. I already tried with RWO pvc but it's still the same issues.
Yes. Because RWO means only one node can use the volume at a time. On that node, there can be many Pods using the same volume. But if there is no scheduling constraint with some inter-pod affinity, the two Pods may be moved to different Nodes, and then you get this error.
In general, having two Pods access the same volume is almost always a bad idea. It's a poor API, and it is slow, too.
@WanzenBug, that mean Linstor does not support to share pvc to multi nodes, right?
Hello.
Hi, you can use rwx, for example, using an intermediate nfs server. In some cases, this can be useful. But here you need to consider the specifics of use/performance and the actual use case.
In my case, we use this scheme for nextcloud.
https://kb.linbit.com/how-can-i-achieve-rwx-for-linstor-volumes-which-are-rwo
@3pings The project is called Linstor Gateway. But AFAIK there is no adapter for K8S yet. Basically you have to deploy an NFS server to provide the RWX capabilities on top of the existing Linstor/Piraeus storage system.
I've written a pretty raw approach to that years ago, but it still works. The concept is explained in the documentation: https://github.com/PHOENIX-MEDIA/External-NFS-Server-Provisioner. The Linstor Gateway provides a more generic and declarative approach of deploying an NFS server. But it can get quite complex if you try to deploy more than one server in a cluster as you will also need to reference a new server in the PV or create a StorageClass for every NFS instance.
