Rolling update drops all data without persistence

[excieve]

If persistence isn't enabled then any change to the DistributedRedisCluster resource (e.g. image or resource allocation update) causes the Redis cluster to loose all the data.

It appears that the operator waits until all the Redis nodes in a shard (StatefulSet) are updated before configuring them all, instead of doing it one by one for each node. This means that new nodes when they're brought up can't replicate from the existing ones. Making this operator impossible to use without persistence enabled, if any maintenance to be ever done.

To address this, one option is that nodes should be joined to the cluster through an init container when updating and health check should ensure replication being complete before declaring the node healthy.

[polefishu]

Only the redis cluster that use persistent storage(pvc) can recover after accidental deletion or rolling update. Even if you do not use persistence(like rdb or aof), you need to set pvc for redis.