Example RedisCluster is not working

[sebt3]

What version of redis operator are you using?

kubectl logs -n kydah-dbo redis-operator-854bf96984-v6d8b |grep rennes-dev
{"level":"info","ts":1734178862.9923024,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178863.0176885,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178863.032022,"logger":"controller_redis","msg":"Redis service get action is failed","Request.Service.Namespace":"rennes-dev","Request.Service.Name":"redis-cluster-leader-headless"}
{"level":"info","ts":1734178863.046514,"logger":"controller_redis","msg":"Redis service get action is failed","Request.Service.Namespace":"rennes-dev","Request.Service.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178863.0701113,"logger":"controller_redis","msg":"Redis service get action is failed","Request.Service.Namespace":"rennes-dev","Request.Service.Name":"redis-cluster-leader-additional"}
{"level":"info","ts":1734178863.086634,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178863.0914545,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178863.091487,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":1734178863.0915482,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178863.122763,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178863.1283746,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178863.1284044,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":1734178863.1284575,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178863.1611688,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178863.165395,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178863.1654243,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":1734178863.1654704,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178863.2176614,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178863.22232,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178863.222351,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":1734178863.222404,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178863.3127275,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178863.3219056,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178863.3219402,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":1734178863.3220003,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178863.3975441,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178863.405034,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178863.4050715,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":1734178863.4051323,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178863.440944,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178863.4475963,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178863.447642,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":1734178863.5653358,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178863.6085036,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178863.6148853,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178863.6149182,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":1734178864.2550938,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178864.298574,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178864.305149,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178864.3051844,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":1734178865.5853336,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178865.6274717,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178865.635912,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178865.6359558,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":1734178868.1964955,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178868.2360623,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178868.2423453,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178868.2423747,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}
{"level":"info","ts":1734178873.3634644,"logger":"controllers.RedisCluster","msg":"Reconciling opstree redis Cluster controller","Request.Namespace":"rennes-dev","Request.Name":"redis-cluster"}
{"level":"info","ts":1734178873.4082146,"logger":"controller_redis","msg":"Redis PodDisruptionBudget get action failed","Request.PodDisruptionBudget.Namespace":"rennes-dev","Request.PodDisruptionBudget.Name":"redis-cluster-leader"}
{"level":"info","ts":1734178873.4146829,"logger":"controller_redis","msg":"Redis statefulset get action failed","Request.StatefulSet.Namespace":"rennes-dev","Request.StatefulSet.Name":"redis-cluster-follower"}
{"level":"error","ts":1734178873.4147158,"logger":"controller.rediscluster","msg":"Reconciler error","reconciler group":"redis.redis.opstreelabs.in","reconciler kind":"RedisCluster","name":"redis-cluster","namespace":"rennes-dev","error":"statefulsets.apps \"redis-cluster-follower\" not found","stacktrace":"sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266\nsigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2\n\t/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227"}

redis-operator version: quay.io/opstree/redis-operator:v0.15.1

Does this issue reproduce with the latest release? As far as I know ;)

What operating system and processor architecture are you using (kubectl version)?

kubectl version Output

$ kubectl version
Client Version: v1.30.6
Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
Server Version: v1.30.6

What did you do?

kubectl create RedisCluster command

$ kubectl apply -f - <<ENDF
apiVersion: redis.redis.opstreelabs.in/v1beta2
kind: RedisCluster
metadata:
  name: redis-cluster
#  annotations:
#    rediscluster.opstreelabs.in/skip-reconcile: "true"
spec:
  clusterSize: 3
  clusterVersion: v7
  podSecurityContext:
    runAsUser: 1000
    fsGroup: 1000
  persistenceEnabled: true
  kubernetesConfig:
    image: quay.io/opstree/redis:v7.0.12
    imagePullPolicy: IfNotPresent
  redisExporter:
    enabled: false
    image: quay.io/opstree/redis-exporter:v1.44.0
    imagePullPolicy: IfNotPresent
  storage:
    volumeClaimTemplate:
      spec:
        # storageClassName: standard
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 1Gi
    nodeConfVolumeClaimTemplate:
      spec:
        accessModes: ["ReadWriteOnce"]
        resources:
          requests:
            storage: 1Gi
            # storageClassName: standard
ENDF

What did you expect to see? A working cluster. This is from the examples...

What did you see instead?

kubectl get po command

$ kubectl get po
NAME                     READY   STATUS             RESTARTS       AGE
redis-cluster-leader-0   0/1     CrashLoopBackOff   5 (105s ago)   4m52s

kubectl logs redis-cluster-leader-0 command

$ kubectl logs redis-cluster-leader-0 
mkdir: can't create directory '/node-conf': Permission denied
Redis is running without password which is not recommended
sed: /node-conf/nodes.conf: No such file or directory
Running without TLS mode
ACL_MODE is not true, skipping ACL file modification
Starting redis service in cluster mode.....
7:C 14 Dec 2024 12:27:01.553 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
7:C 14 Dec 2024 12:27:01.553 # Redis version=7.0.12, bits=64, commit=00000000, modified=0, pid=7, just started
7:C 14 Dec 2024 12:27:01.553 # Configuration loaded
7:M 14 Dec 2024 12:27:01.553 * monotonic clock: POSIX clock_gettime
7:M 14 Dec 2024 12:27:01.553 # Can't open /node-conf/nodes.conf in order to acquire a lock: No such file or directory

The RedisCluster is created using the recommanded podSecurityContext yet before even starting we can see that

mkdir: can't create directory '/node-conf': Permission denied

So the cluster will go nowhere. what am I doing wrong ? Using an other storageClass doesnt help

[husnialhamdani]

I believe this issues is caused by unsuccessful of PVC NodeConf