[NEW] support redis 7

[raphaelauv]

The problem/use-case that the feature addresses

multiple feature of redis 7 like :

Cluster: Support for hostnames, instead of IP addresses only -> https://github.com/redis/redis/pull/9530

are not yet part of keyDB , is there a roadmap to support redis 7 features ? thanks

[JohnSully]

Yes, we want to do this for early Q3. It takes about a month to complete the merge so it's a large time investment.

[MagicKriss]

@JohnSully, any ETA on when this could be expected?

[MagicKriss]

So... on KeyDB website it states:

KeyDB works to maintain parity with features and updates in the upstream Redis codebase. In general we attempt to perform these upstream merges quarterly.

On April it's going to be a year since Redis 7.0 is out. And still no info on whether this is even planned

@JohnSully what happened to Q3 ?

Yes, we want to do this for early Q3. It takes about a month to complete the merge so it's a large time investment.

[TomHellier]

It looks like @JohnSully has been busy working on some other component for Snap. https://github.com/Snapchat/KeyDB/issues/494#issuecomment-1271655744

I'd be interested in an update about whether this might be coming in the next 6 months?

[cjabrantes]

Hi, Is there any update when this merge is expected? Thanks,

[wacdev]

any update？

[vainkop]

Any updates?

[the-wondersmith]

Any updates?

[iabramov]

Any news?

[javedsha]

@JohnSully just wondering if the Redis 7.0 merge is going to happen this year 2023. We are interested in some of the Redis 7.0 ACL features.

[neonknight]

We are running Debian 12 which brings Redis 7.0 by default. We would like to migrate to KeyDB. However, due to the lack of compatibility we are unable to migrate without losing all data from Redis. Unfortunately starting with a clean DB is not acceptable. Please advise.

Reading dump.rdb or connecting as replica to Redis both lead to crash of KeyDB with error message "Can't handle RDB format version 10 / Fatal error loading the DB: Invalid argument. Exiting." Such restriction is not mentioned in https://docs.keydb.dev/docs/migration

[violuke]

We would love to switch to KeyDB but need option support on the EXPIRE command (NX, XX, GT & LT). Any news on this would be amazing. Thank you.

[Jarod1662]

@JohnSully - any update on this?

[chibisuke]

There have not been any major upgrades for more then 2 years now. @JohnSully can you please confirm (or deny and if whats the timeline) KeyDB is dead?

[vwbusguy]

There's talk now about potentially obsoleting redis for KeyDB in Fedora and EPEL, but unfortunately Fedora already ships redis-7, so doing so could break things for existing Fedora redis users that are using the v7 features. Otherwise, s/redis/keydb would be a much more straightforward proposition for Fedora and EPEL that would certainly gain KeyDB a significant growth in user base and potential contributors.

To that end, what can we (KeyDB users on this issue) do to help with this?

[JohnSully]

KeyDB is used heavily in Snap and is not going anywhere, but the reality is there is limited time for me to support issues not directly affecting Snap. The project would of course welcome outside help and we can certainly name additional maintainers if there is community interest in helping.

But I want to be realistic about what I will be able to do with my current resourcing.

[JohnSully]

Also to an earlier comment there was a release done a few months ago.

[vwbusguy]

The project would of course welcome outside help and we can certainly name additional maintainers if there is community interest in helping.

There's another project that has sprung up recently. It seems like the best of all worlds if redict and keydb could ultimately work together on this: https://codeberg.org/redict/redict

Perhaps @ddevault would be a good candidate for this, given his contributions to redict? Maybe redict could become the new upstream for KeyDB, if an outright merger isn't in the cards?

[cjabrantes]

Hi,

I understood until now that keydb is not going anywhere, but the release of v7(that have nice feature like be able to monitor streams lag of consumer) may not be here any time soon as there is not ETA for it.

i come across with the following article: https://techcrunch.com/2024/03/31/why-aws-google-and-oracle-are-backing-the-valkey-redis-fork/?guccounter=1

"The Linux Foundation last week announced that it will host Valkey, a fork of the Redis in-memory data store. Valkey is backed by Amazon Web Services, Google Cloud, Oracle, Ericsson and Snap."

Is this Snap the same Snap that is now behind/hosting KeyDB? And assuming yes, is this changing the plans for keydb?

[CanRau]

pretty sure it is the same Snap Inc and read somewhere that features & efforts might influence and potentially make it into ValKey 🥳

[raphaelauv]

yes it's snap inc

https://www.linuxfoundation.org/press/linux-foundation-launches-open-source-valkey-community

[cjabrantes]

ok, but in that case what can we expect from KeyDB, i mean, Snap is now backing up 2 key store DBs that are a fork from redis?

And if Valkey is also backed up by big players as AWS or google, what can we expect to happen to keyDB in the middle/long run?

[JohnSully]

There was another ticket open where I mentioned that KeyDB isn't staffed to be the "new Redis" and so we are not trying to compete in that space. This is where Valkey is competing.

However KeyDB has never existed in a vacuum there have always been bigger scoped projects to draw on. We are focussed on a few core features not in either Valkey or the previously open source Redis:

• Better vertical scaling with multithreading
• Support for FLASH and tiered storage

Right now Valkey doesn't have those features and Snap needs those - we have many 500+ node clusters and its simply impractical to x8 them using single threaded instances. FLASH is still not quite prod ready due to its performance but is getting close in the async_flash branch and we are continuing to guide that through. Once we have use cases running well on it we will do a new KeyDB release including this technology.

As for Snap supporting both Valkey and KeyDB, in the early days Redis never allowed us to upstream either of the two technologies because it competed with their commercial business. Valkey will hopefully be more pragmatic and I would love to see some of our efforts merged. There is a lot of work to do before that can happen and KeyDB development will continue in the meantime.

While I expect the code bases of KeyDB and Valkey to get closer together with time I still hope for KeyDB to be the place where experiments and new features can happen and get proven out. The refusal to merge needed features really held Redis back and this is an important relief valve for innovation to happen.

If there is a key feature in Redis 7 you are missing I am happy to merge it so you can get it sooner.

[raphaelauv]

Thanks for the reply @JohnSully

Btw keyDB multithreading is still a x8 performance improvement compared to redis 7 ?

[cjabrantes]

Hi @JohnSully,

Thanks for your time and efforts, on my side:

• Capability to measure consumer lag on a stream, as is we can not monitoring streams, if messages are being lost for example if we have slow clients, this is important to understand, is our stream the right size to accommodate spikes? arae the clients reading fast enough? in redis 7 XINFO GROUPS added a new lag parameter, something that i personally think essential.

• The BUG i opened https://github.com/Snapchat/KeyDB/issues/613, there was no feedback, so i dont know if it was solved in more recent versions, but assuming its a bug and not bad config, it jeopardize HA 2 nodes with replication when 1 goes down.

The answers to this would make me very happy :)

[cjabrantes]

Hi @JohnSully,

Do you have an idea if what i "requested" is possible? more important the first point related with streams monitoring? The second sadly i m not able to reproduce anymore :(

Thanks,