Easy Cluster mode on kubernetes

[ecyrbe]

Feature request type

Kubernetes Operator, or easy config file setup for garnet

Is your feature request related to a problem? Please describe

Using cluster mode on kubernetes is impossible without an operator to meet the other nodes in a statefulset. Indeed, garnet uses the same cluster model as redis where stafulset/service (with clusterIP none) dns naming pattern is unknown at the config level.

Describe the solution you'd like

provide an operator that orchestrates cluster setup, recovery, migration based on a custom CRD that describes the cluster topology :

apiVersion: garnet.microsoft/v1
kind: GarnetCluster
metadata:
  name: garnet-cluster
spec:
  masterCount: 3
  replicasPerMaster: 1
  storage:
    volumeClaimTemplate:
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 1Gi

• Changing masterCount should reshard and migrate data to resized masters
• changing replicasPerMaster should replicate data

Describe alternatives you've considered

Allow graceful recovery of the cluster when one or many nodes restart using a dns naming pattern (would be nice) using a predefined cluster DNS names lists. Example : in garnet.conf

{
  "clusterMasterNames": ["garnet-0", "garnet-1", "garnet-3"],
  "clusterReplicaNames" : ["garnet-4", "garnet-5", "garnet-6"]
}

• clusterReplicaNames vector should have a size that is multiple of clustesMasterNames
• each cluster know who he is by getting it's hostname as kubernetes would provision it whensetting a statefulset
• only static setup would be supported (no dynamic resharding, or dynamic replication)

Additional context

No response

[vazois]

I am not an expert on Kubernetes but I see that there are a couple of open-source options for redis operators that should work with Garnet also. Have you tried any of the following?

https://github.com/spotahome/redis-operator https://github.com/OT-CONTAINER-KIT/redis-operator

[ecyrbe]

Thanks for the suggestions,

Unfortunately, Redis operators rely on redis specifics (like binaries, docker images, config files, etc),. not just API surface.

So i'm afraid a dedicated operator is kind of mandatory for garnet.

But maybe it's out of scope of garnet project ?

[badrishc]

It is definitely interesting for the project, and we would be thrilled to see something come from the community.

[jainnitin2411]

A garnet specific Kubernetes Operator is available. Although I am not sure about its stability and production readiness. Please refer the below github repository.

https://github.com/marcusbooyah/garnet-operator