Proper PV support - ReadWriteOnce

[andreimc]

trafficstars

Hey,

I tried your PV support with EFS RWX but it wasn't that good so I have a proposed implementation for PV support inside the operator.

eg

# Default values for nats-streaming-cluster
nats:
  replicas: 3
  clusterId: "nats-cluster"

stan:
  replicas: 3
  storage:
    size: 10Gi
    keepOnDelete: true
  clusterId: "nats-streaming-cluster"

where it actually creates the pvs with the pods themselves or mounts them if they exist? The only thing I am not sure about what happens when you delete the operator you could leave the volumes hanging around if you delete your cluster?

What do you guys think?

[wallyqs]

sounds interesting so a PV would be created for each one of the pods? Maybe could use owner references to delete the volumes when pods go away?

[andreimc]

@wallyqs yeah that's what I am thinking, I will get time later this week to have a crack at this.

[Lazyshot]

This sounds absolutely necessary as we need to run NATS Streaming in an HA manner.

[C0en]

What he said ^

Maybe we can have a volume claimtemplate like in statefulsets, so each node gets a seperate volumeclaim with a number and then use a storage class to provision the volumes. Proposed like this:

---

apiVersion: "streaming.nats.io/v1alpha1" kind: "NatsStreamingCluster" metadata: name: "example-stan-pv" spec: size: 3 natsSvc: "example-nats" config: storeDir: "/pv/stan" template: spec: containers: - name: nats-streaming volumeMounts: - mountPath: /pv name: stan-store-dir volumeClaimTemplates:

• metadata: name: stan-store-dir spec: accessModes: - ReadWriteOnce storageClassName: stanstorage resources: requests: storage: 8Gi

[wallyqs]

The volumeClaimTemplates idea to make it provision PV dynamically per requested instance I think sounds good

[pvanderlinden]

Currently you can only have a cluster of servers without persistence or just a single server with persistence? The volumeClaimTemplates would be a good idea to fix this indeed, as it will be consistent with Statefulsets

[rajugupta15]

Is any update on this?

[snayakie]

Hi, I just want to confirm that the nats-streaming-operator doesn't support statefulset stylevolumeClaimTemplates in the NatsStreamingCluster crd , does it?

I just tried the following, and it doesn't seem to work....so just want to confirm that its not supported.

---
apiVersion: "streaming.nats.io/v1alpha1"
kind: "NatsStreamingCluster"
metadata:
  name: "stan-pvc-template"
spec:
  size: 3
  natsSvc: "example-nats"

  config:
    storeDir: "/pv/stan"

  volumeClaimTemplates:
    - metadata:
        name: streaming-pvc
      spec:
        storageClassName: "local-storage"
        accessModes:
          - ReadWriteOnce
        volumeMode: Filesystem
        resources:
          requests:
            storage: 128Mi
      
  # Define mounts in the Pod Spec
  template:
    spec:
      containers:
        - name: nats-streaming
          volumeMounts:
          - mountPath: /pv/stan
            name: streaming-pvc