redis-enterprise-k8s-docs
redis-enterprise-k8s-docs copied to clipboard
Published
20 hours ago •
RedisLabs
RedisLabs
Unable to Create CRDB Active-Active database in kubernetes Cluster
I have a Self manage kubernetes cluster(kubeadm in Ec2 instance) Will have to deploy a Redis Enterprise into the cluster. I have successfully Deploy a three-node redis Enterprise Cluster and deploy a test database. It's working well.
NAME NODES VERSION STATE SPEC STATUS LICENSE STATE SHARDS LIMIT LICENSE EXPIRATION DATE AGE
rec 3 6.2.12-82 Running Valid Valid 4 2022-10-29T09:42:03Z 3h13m
Now I have to deploy an Active-Active CRDB ( Conflict Free Replication Database ). I tried following the document but it seems it was not created.
https://docs.redis.com/latest/kubernetes/re-clusters/create-aa-database/
Can any one assist me on this issue.
Hi,
Yes, we can assist you, but we will need details. Please open a Support Ticket with Redis and provide:
- the archive generated by running the log collector script for each participating Redis Enterprise cluster (see https://docs.redis.com/latest/kubernetes/logs/collect-logs/).
- the command you ran to create the CRDB database as well as its console output.
- what is the ingress controller being used on each k8s cluster.
Laurent.
Hi,
Please find the Collector script logs.
2022-09-29 13:13:28,470 - INFO - Started Redis Enterprise k8s log collector
2022-09-29 13:13:28,683 - INFO - Namespace 'None': Collected cluster-info
2022-09-29 13:13:28,772 - INFO - Started collecting from namespace 'redis'
2022-09-29 13:13:28,980 - INFO - Namespace 'None': Collected connectivity check via curl
2022-09-29 13:13:29,102 - INFO - Namespace 'redis': Collected connectivity check via k8s cli
2022-09-29 13:13:29,185 - INFO - Trying to extract debug info from RS pods: {[u'rec-test-0', u'rec-test-1', u'rec-test-2']}
2022-09-29 13:14:17,036 - INFO - Namespace 'redis': debug info created on pod rec-test-0 in path /tmp/debuginfo.20220929-131330.tar.gz
2022-09-29 13:14:17,547 - INFO - Namespace 'redis': Collected Redis Enterprise cluster debug package from pod: rec-test-0
2022-09-29 13:14:19,347 - INFO - Namespace 'redis': Collected rs logs from pod marked as not ready, pod name: rec-test-0
2022-09-29 13:14:19,500 - INFO - Collected rs config from pod marked as not ready, pod name: rec-test-0
2022-09-29 13:14:19,661 - INFO - Namespace 'redis': Collected rs logs from pod marked as not ready, pod name: rec-test-1
2022-09-29 13:14:19,806 - INFO - Collected rs config from pod marked as not ready, pod name: rec-test-1
2022-09-29 13:14:20,519 - INFO - Namespace 'redis': Collected rs logs from pod marked as not ready, pod name: rec-test-2
2022-09-29 13:14:20,666 - INFO - Collected rs config from pod marked as not ready, pod name: rec-test-2
2022-09-29 13:14:20,793 - INFO - Namespace 'redis': Collected resources list
2022-09-29 13:14:20,866 - INFO - Namespace 'redis': Collected events
2022-09-29 13:14:20,944 - INFO - Namespace 'redis': Collecting API resources
2022-09-29 13:14:21,020 - INFO - Namespace 'redis': + Collected RedisEnterpriseCluster
2022-09-29 13:14:21,096 - INFO - Namespace 'redis': + Collected RedisEnterpriseDatabase
2022-09-29 13:14:21,197 - WARNING - Namespace 'redis': Failed to get RedisEnterpriseRemoteCluster resource: error: the server doesn't have a resource type "RedisEnterpriseRemoteCluster".
2022-09-29 13:14:21,488 - WARNING - Namespace 'redis': Failed to get RedisEnterpriseActiveActivePeering resource: error: the server doesn't have a resource type "RedisEnterpriseActiveActivePeering".
2022-09-29 13:14:21,749 - INFO - Namespace 'redis': + Collected StatefulSet
2022-09-29 13:14:21,828 - INFO - Namespace 'redis': + Collected Deployment
2022-09-29 13:14:21,908 - INFO - Namespace 'redis': + Collected Service
2022-09-29 13:14:21,987 - INFO - Namespace 'redis': + Collected ConfigMap
2022-09-29 13:14:22,078 - WARNING - Namespace 'redis': Failed to get Route resource: error: the server doesn't have a resource type "Route".
2022-09-29 13:14:22,345 - INFO - Namespace 'redis': + Collected Ingress
2022-09-29 13:14:22,424 - INFO - Namespace 'redis': + Collected Role
2022-09-29 13:14:22,500 - INFO - Namespace 'redis': + Collected RoleBinding
2022-09-29 13:14:22,577 - INFO - Namespace 'redis': + Collected PersistentVolume
2022-09-29 13:14:22,650 - INFO - Namespace 'redis': + Collected PersistentVolumeClaim
2022-09-29 13:14:22,741 - INFO - Namespace 'redis': + Collected Node
2022-09-29 13:14:22,813 - INFO - Namespace 'redis': + Collected PodDisruptionBudget
2022-09-29 13:14:22,886 - INFO - Namespace 'redis': + Collected ResourceQuota
2022-09-29 13:14:22,963 - INFO - Namespace 'redis': + Collected Endpoints
2022-09-29 13:14:23,060 - INFO - Namespace 'redis': + Collected Pod
2022-09-29 13:14:23,134 - INFO - Namespace 'redis': + Collected NetworkPolicy
2022-09-29 13:14:23,615 - INFO - Namespace 'redis': + Collected CustomResourceDefinition
2022-09-29 13:14:23,684 - INFO - Namespace 'redis': + Collected CertificateSigningRequest
2022-09-29 13:14:23,758 - INFO - Namespace 'redis': + Collected ValidatingWebhookConfiguration
2022-09-29 13:14:23,845 - WARNING - Namespace 'redis': Failed to get NamespacedValidatingType resource: error: the server doesn't have a resource type "NamespacedValidatingType".
2022-09-29 13:14:24,130 - WARNING - Namespace 'redis': Failed to get NamespacedValidatingRule resource: error: the server doesn't have a resource type "NamespacedValidatingRule".
2022-09-29 13:14:24,442 - INFO - Namespace 'redis': + Collected ClusterRole
2022-09-29 13:14:24,540 - INFO - Namespace 'redis': + Collected ClusterRoleBinding
2022-09-29 13:14:24,627 - WARNING - Namespace 'redis': Failed to get ClusterServiceVersion resource: error: the server doesn't have a resource type "ClusterServiceVersion".
2022-09-29 13:14:24,894 - WARNING - Namespace 'redis': Failed to get Subscription resource: error: the server doesn't have a resource type "Subscription".
2022-09-29 13:14:25,168 - WARNING - Namespace 'redis': Failed to get InstallPlan resource: error: the server doesn't have a resource type "InstallPlan".
2022-09-29 13:14:25,451 - WARNING - Namespace 'redis': Failed to get CatalogSource resource: error: the server doesn't have a resource type "CatalogSource".
2022-09-29 13:14:25,721 - WARNING - Namespace 'redis': Failed to get PodSecurityPolicy resource: error: the server doesn't have a resource type "PodSecurityPolicy".
2022-09-29 13:14:25,981 - INFO - Namespace 'redis': + Collected ReplicaSet
2022-09-29 13:14:26,052 - INFO - Namespace 'redis': + Collected StorageClass
2022-09-29 13:14:26,055 - INFO - Namespace 'redis': Collecting API resources description
2022-09-29 13:14:26,136 - INFO - Namespace: 'redis' + Collected RedisEnterpriseCluster
2022-09-29 13:14:26,206 - INFO - Namespace: 'redis' + Collected RedisEnterpriseDatabase
2022-09-29 13:14:26,293 - WARNING - Namespace 'redis': Failed to describe RedisEnterpriseRemoteCluster resource: error: the server doesn't have a resource type "RedisEnterpriseRemoteCluster".
2022-09-29 13:14:26,566 - WARNING - Namespace 'redis': Failed to describe RedisEnterpriseActiveActivePeering resource: error: the server doesn't have a resource type "RedisEnterpriseActiveActivePeering".
2022-09-29 13:14:26,844 - INFO - Namespace: 'redis' + Collected StatefulSet
2022-09-29 13:14:26,941 - INFO - Namespace: 'redis' + Collected Deployment
2022-09-29 13:14:27,051 - INFO - Namespace: 'redis' + Collected Service
2022-09-29 13:14:27,157 - INFO - Namespace: 'redis' + Collected ConfigMap
2022-09-29 13:14:27,247 - WARNING - Namespace 'redis': Failed to describe Route resource: error: the server doesn't have a resource type "Route".
2022-09-29 13:14:27,527 - INFO - Namespace: 'redis' + Collected Ingress
2022-09-29 13:14:27,607 - INFO - Namespace: 'redis' + Collected Role
2022-09-29 13:14:27,685 - INFO - Namespace: 'redis' + Collected RoleBinding
2022-09-29 13:14:27,787 - INFO - Namespace: 'redis' + Collected PersistentVolume
2022-09-29 13:14:27,897 - INFO - Namespace: 'redis' + Collected PersistentVolumeClaim
2022-09-29 13:14:28,081 - INFO - Namespace: 'redis' + Collected Node
2022-09-29 13:14:28,163 - INFO - Namespace: 'redis' + Collected PodDisruptionBudget
2022-09-29 13:14:28,234 - INFO - Namespace: 'redis' + Collected ResourceQuota
2022-09-29 13:14:28,332 - INFO - Namespace: 'redis' + Collected Endpoints
2022-09-29 13:14:28,447 - INFO - Namespace: 'redis' + Collected Pod
2022-09-29 13:14:28,519 - INFO - Namespace: 'redis' + Collected NetworkPolicy
2022-09-29 13:14:29,085 - INFO - Namespace: 'redis' + Collected CustomResourceDefinition
2022-09-29 13:14:29,156 - INFO - Namespace: 'redis' + Collected CertificateSigningRequest
2022-09-29 13:14:29,236 - INFO - Namespace: 'redis' + Collected ValidatingWebhookConfiguration
2022-09-29 13:14:29,325 - WARNING - Namespace 'redis': Failed to describe NamespacedValidatingType resource: error: the server doesn't have a resource type "NamespacedValidatingType".
2022-09-29 13:14:29,598 - WARNING - Namespace 'redis': Failed to describe NamespacedValidatingRule resource: error: the server doesn't have a resource type "NamespacedValidatingRule".
2022-09-29 13:14:30,092 - INFO - Namespace: 'redis' + Collected ClusterRole
2022-09-29 13:14:30,321 - INFO - Namespace: 'redis' + Collected ClusterRoleBinding
2022-09-29 13:14:30,413 - WARNING - Namespace 'redis': Failed to describe ClusterServiceVersion resource: error: the server doesn't have a resource type "ClusterServiceVersion".
2022-09-29 13:14:30,698 - WARNING - Namespace 'redis': Failed to describe Subscription resource: error: the server doesn't have a resource type "Subscription".
2022-09-29 13:14:30,986 - WARNING - Namespace 'redis': Failed to describe InstallPlan resource: error: the server doesn't have a resource type "InstallPlan".
2022-09-29 13:14:31,271 - WARNING - Namespace 'redis': Failed to describe CatalogSource resource: error: the server doesn't have a resource type "CatalogSource".
2022-09-29 13:14:31,552 - WARNING - Namespace 'redis': Failed to describe PodSecurityPolicy resource: error: the server doesn't have a resource type "PodSecurityPolicy".
2022-09-29 13:14:31,832 - INFO - Namespace: 'redis' + Collected ReplicaSet
2022-09-29 13:14:31,907 - INFO - Namespace: 'redis' + Collected StorageClass
2022-09-29 13:14:31,910 - INFO - Namespace 'redis': Collecting pods' logs:
2022-09-29 13:14:46,625 - INFO - Archived files into /home/centos/yamls/redis-en/redis_enterprise_k8s_debug_info_20220929-131328.tar.gz
2022-09-29 13:14:46,702 - INFO - Finished Redis Enterprise log collector
2022-09-29 13:14:46,702 - INFO - --- Run time: 1 minutes ---
We are using the NGINX controller and resources. As kubernetes cluster is self-managed, So we have additionally configure the Nginx Load Balancer and with the help of nodePort IP We are trying it.
I am not able to run the CRDB command just yet because I believe the initial setup is still broken.
Hi,
The log_collector script console output is not what we will need. It does generate a tarball that you will need to attach to the Support Ticket when you open it. Also, a CRDB database involves at least two Redis Enterprise clusters. I don't see any indication that you have created a 2nd Redis Enterprise cluster on a separate k8s cluster.
Kindly open a Support Ticket as we can't use github to help you efficiently.
Laurent.
As we are testing redis Enterprise. Can you please help me supply this values
Database name <db-name>:
• Description: Combined with ingress suffix to create the Active-Active database hostname
REC hostname <rec-hostname>:
• Description: Hostname used to identify your Redis Enterprise cluster in the crdb-cli command. This MUST be different from other participating clusters.
API hostname <api-hostname>:
• Description: Hostname used to access the Redis Enterprise cluster API from outside the K8s cluster
Ingress suffix <ingress-suffix>:
• Description: Combined with database name to create the Active-Active database hostname
In Redis Enterprise Cluster spec
activeActive:
apiIngressUrl: <api-hostname>
dbIngressSuffix: <ingress-suffix>
Here is Our REC Spec YAML
apiVersion: "app.redislabs.com/v1"
kind: "RedisEnterpriseCluster"
metadata:
name: "test-rec"
spec:
nodes: 3
uiServiceType: ClusterIP
username:
persistentSpec:
enabled: true
storageClassName: "local-storage"
volumeSize: "5Gi"
redisEnterpriseNodeResources:
limits:
cpu: 900m
memory: 3Gi
requests:
cpu: 500m
memory: 1Gi
activeActive:
apiIngressUrl: ???????
dbIngressSuffix: ?????
ingressAnnotations:
kubernetes.io/ingress.class: "nginx"
nginx.ingress.kubernetes.io/backend-protocol: "HTTP"
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
method: ingress
Ingress YAML
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: rec-ingress
namespace: redis
annotations:
kubernetes.io/ingress.class: nginx
nginx.ingress.kubernetes.io/ssl-passthrough: "true"
spec:
rules:
- host: rec.k8s.my.redisdemo.com
http:
paths:
- path: /
pathType: ImplementationSpecific
backend:
service:
name: db-test
port:
name: redis
I will be Helpfull for setting up redis enterprise with ingress and then same we implement in our Cluster