KeyDB icon indicating copy to clipboard operation
KeyDB copied to clipboard

Active-active replication is broken in 6.0.18-6.2.1

Open nicknezis opened this issue 3 years ago • 5 comments

Describe the bug

I spin up an Active-active cluster in Kubernetes with default values (based on the enapter/keydb helm chart). I then load a fake dataset with 6 million key value pairs using cat data.txt | keydb-cli --pipe.

When monitoring each pod with kubectl log and watch -d ls -lart /data/ the replication of data never completes and eventually ends in some pods spewing errors to the log. During the load, there are temp.rdb files present alongside the main dump.rdb file in each pod. These temp files never fully go away.

When downgrading the verison of the Keydb docker image to 6.0.16, the buggy behavior is not present. With this version the dataset loads successfully without failure. The cluster reaches a steady state with a single dump.rdb in each pod and no constant resyncing attempts that ultimately fail.

nicknezis avatar Jan 12 '22 06:01 nicknezis

I don't know if it's related, but #378 gave me the inspiration to test with an older version of Keydb. I don't know if it is related in any way, but thought I'd mention it.

nicknezis avatar Jan 12 '22 06:01 nicknezis

@nicknezis Did it work with an older version? which version?

esatterwhite avatar Jan 12 '22 14:01 esatterwhite

@nicknezis Did it work with an older version? which version?

6.0.16 seemed to behave.

nicknezis avatar Jan 12 '22 16:01 nicknezis

I think the link you posted to the helm chart is wrong?

esatterwhite avatar Jan 14 '22 06:01 esatterwhite

I think the link you posted to the helm chart is wrong?

Thanks for catching that. I've updated the link to the proper URL. https://artifacthub.io/packages/helm/enapter/keydb

nicknezis avatar Jan 14 '22 14:01 nicknezis

Is there anyone still experiencing this issue, will be helpful to understand how to prioritize this.

msotheeswaran-sc avatar Mar 10 '23 20:03 msotheeswaran-sc

Closing as there has been no response in 30 days

msotheeswaran-sc avatar Apr 12 '23 20:04 msotheeswaran-sc