riot icon indicating copy to clipboard operation
riot copied to clipboard

riot-redis migrate from elasticache clustered enabled issue

Open SQLConjuror opened this issue 3 years ago β€’ 3 comments

Hi

riot-redis version: riot-redis-2.0.8

In my source elasticache (test) that is clustered enabled, I wanted to migrate 25355 keys.

[root@ip-10-100-1-111 ~]# redis-cli -h test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com:6379> test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com:6379> info keyspace

Keyspace

db0:keys=25355,expires=0,avg_ttl=0

But when I ran the command below, first I get warnings, and then only 10067 was migrated.

[root@ip-10-100-1-111 bin]# ./riot-redis --cluster -r redis://test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com:6379 replicate -r redis://efRK7H [email protected]:16673 --live Replicating from RedisURI [host='test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com', port=6379] to RedisURI [host='redis-16673.c19675.us-west1-1 .gcp.cloud.rlrcp.com', port=16673] ? % β”‚β–ˆWARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by io.lettuce.core.internal.DefaultMethods$MethodHandleLookup (file:/root/riot-redis-2.0.8/lib/lettuce-core-5.3.1.RELEASE.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class) WARNING: Please consider reporting this to the maintainers of io.lettuce.core.internal.DefaultMethods$MethodHandleLookup WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Could not read values: Deque full Replicating from RedisURI [host='test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com', port=6379] to RedisURI [host='redis-16673.c19675.us-west1-1 Replicating from RedisURI [host='test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com', port=6379] to RedisURI [host='redis-16673.c19675.us-west1-1 Replicating from RedisURI [host='test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com', port=6379] to RedisURI [host='redis-16673.c19675.us-west1-1 Replicating from RedisURI [host='test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com', port=6379] to RedisURI [host='redis-16673.c19675.us-west1-1 Replicating from RedisURI [host='test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com', port=6379] to RedisURI [host='redis-16673.c19675.us-west1-1 Replicating from RedisURI [host='test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com', port=6379] to RedisURI [host='redis-16673.c19675.us-west1-1 Replicating from RedisURI [host='test-redis-cluster-migrate.bbssxp.clustercfg.apse1.cache.amazonaws.com', port=6379] to RedisURI [host='redis-16673.c19675.us-west1-1.gcp.cloud.rlrcp.com', port=16673] ? % β”‚ β–ˆ β”‚ 10100/? (0:00:44 / ?) 229.5/s [root@ip-10-100-1-111 bin]#

Below we can see that only 10067 were migrated

redis-16673.c19675.us-west1-1.gcp.cloud.rlrcp.com:16673> info keyspace Keyspace db0:keys=10067,expires=0,avg_ttl=0 redis-16673.c19675.us-west1-1.gcp.cloud.rlrcp.com:16673>

Please advise.

Thank you.

SQLConjuror avatar May 26 '22 11:05 SQLConjuror

I think this is the issue I'm running in to as well.

I am trying to replicate from elasticache (cluster enabled) to sentinel redis running on k8s.

RIOT runs fine, with no issue when I use it in snapshot mode, but when I run it in either live or liveonly mode it starts with

Listening  ? % β”‚β–ˆ                                                               β”‚ 0/? (0:00:00 / ?) ?/s

and then emits the following Warnings:

WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by io.lettuce.core.internal.DefaultMethods$MethodHandleLookup (file:/home/ubuntu/riot-redis-2.18.3/lib/lettuce-core-6.2.0.RELEASE.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class)
WARNING: Please consider reporting this to the maintainers of io.lettuce.core.internal.DefaultMethods$MethodHandleLookup
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release
~/riot-redis-2.18.3$ ./bin/riot-redis --version

      β–€        β–ˆ     β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
 β–ˆ β–ˆβ–ˆ β–ˆ  β–ˆβ–ˆβ–ˆ  β–ˆβ–ˆβ–ˆβ–ˆ   β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
 β–ˆβ–ˆ   β–ˆ β–ˆ   β–ˆ  β–ˆ     β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
 β–ˆ    β–ˆ β–ˆ   β–ˆ  β–ˆ     β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ
 β–ˆ    β–ˆ  β–ˆβ–ˆβ–ˆ    β–ˆβ–ˆ   β–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆβ–ˆ  v2.18.3

It also doesn't seem to be working. If I switch to another window in screen and run a compare I can see that the replication is not being kept up to date.

Any ideas on why this is performing an illegal reflective access or how to stop it?

M-JobPixel avatar Oct 04 '22 15:10 M-JobPixel

Hi M-JobPixel,

Have you tried using the docker method instead?

docker run jruaux/riot-redis

this could be a bug in the old version of RIOT.

SQLConjuror avatar Oct 04 '22 17:10 SQLConjuror

I have not. I will take a look at doing that...

Yes that seems to work ok. No warnings.

I haven't yet seen that it syncs things but I'll report back soon

M-JobPixel avatar Oct 04 '22 17:10 M-JobPixel