bullmq
bullmq copied to clipboard
taskforcesh
Migrate to official node-redis
Is your feature request related to a problem? Please describe.
ioredis is being non-actively maintained for a while, it seems will be deprecated soon. https://github.com/redis/node-redis/issues/2658#issuecomment-1968104973
Describe the solution you'd like Migrate to https://github.com/redis/node-redis
Describe alternatives you've considered n/a
Additional context n/a
Yeah, this is a quite a sad situation, it almost seems like Redislabs is being run by a chicken without head... most frustrating is the lack of communication with the community, leaving everybody wondering what is going on.
But from BullMQ's point of view, there is not a single issue with IORedis that we are aware of, and would such an issue arise we will fork IORedis and fix it for us. The only feature that would be relevant in terms of performance would be to have support for RESPv3 but neither IORedis nor node-redis supports it.
The APIs of IORedis and node-redis are different enough to make the work of supporting node-redis non-trivial, and there is no guarantee it will be better maintained either, in fact the number of commits seems to be quite low too...
If somebody really wants to use node-redis, one possibility would be to create a wrapper on top of node-redis that implements the IORedis specific APIs that are used by BullMQ... but I do not think this is worthy right now.
Btw, it could be that some security advisory arises in IORedis that forces us to fork it even though there are no other issues, but lets give Redislabs a chance to make things right.
What about moving to keydb or dragonfly? Both are decent drop-in replacements for redis and both open source. in terms of performance, they both claim to be much faster than redis too. Sounds like multiple wins then?
Dragonfly is supported, as it is 100% compatible with Redis (https://bullmq.io/news/101023/dragonfly-compatibility/). KeyDB did not properly work with BullMQ last time I tested and probably does still not work but I am not sure.
In any case this discussion is about the NodeJS client library and less about the change in Redis license. I do not think the new Redis license affects regular users, but instead hosting providers which will now need to get a commercial license agreement from Redis in order to provide the newer versions. If I have to guess, these changes will just encourage new forks and competition to fill the need for a robust in-memory database, so not bad news for current Redis users in the long run.
Describe the solution you'd like Migrate to https://github.com/redis/node-redis
It's not well supported either 😞 Look at the number of issues, at the absence of comments there, hanging PRs, look at the release history.
@ivan-kleshnin yes, thats why I am more confident to continue using just IORedis, as over the years @luin has fixed the issues that affected BullMQ and AFAIK there are no known issues that affect us... just keeping IORedis up-to-date regarding security updates should not be a big effort, but obviously some maintainer must merge the PRs and make the releases otherwise nothing will happen.
But from BullMQ's point of view, there is not a single issue with IORedis that we are aware of, and would such an issue arise we will fork IORedis and fix it for us.
That's true, but there are plenty of bugs in ioredis that don't directly affect BullMQ. If a job uses Redis for something else (like geodist calculations), it likely needs to load two redis clients. That's a waste of developer time, CPU time, and CO2.
I doubt that the official redis client will address any issues related to one of the redis forks (e.g. https://github.com/valkey-io/valkey). And since the usage of redis forks will increase due to the redis license change, I do not think it is a good idea to replace ioredis with node-redis.
FYI @manast , the valkey guys already forked ioredis and call it iovalkey now.
(AFAIK valkey this is now the Redis fork with the biggest budget, since a number of big companies are backing it.)
As long as IORedis works reliably for us, I do not see any benefit in using any other client... it will just introduce more risk and more likely new issues until all edge cases are ironed out.
https://github.com/redis/ioredis/issues/1865
^This one is busted :/, reason enough to switch?
@sirrodgepodge the thing is, how do we know node-redis does not have other issues that would impact us? so if we do all the migration work (which is not completely trivial), and then we discover other break-dealer issues, what then? I think the most sensible thing to do is try to fix the issues in ioredis that affect BullMQ, worst case we could fork the project, but that is not an ideal situation either. It is quite incredible that Redis took over the maintenance of IORedis just to not maintain it. At least when the maintainer was @luin it tried its best to solve the issues...
@sirrodgepodge, seems like the issue you linked is "ElastiCache Serverless" specific, since people using regular redis or dragonfly do not seem to have that issues.
Instead of creating yet another fork, perhaps you could consider adopting and contributing to the iovalkey client (like mentioned above)? My team has had a very positive experience with it. We submitted a PR to the iovalkey repository with stability fixes for clusters (no new features), and it was properly reviewed and merged within a week. In contrast, our original PR on ioredis has been open since March without any comments from the maintainers and will probably never be reviewed and merged. Additionally, the iovalkey client currently doesn't introduce any breaking changes compared to ioredis — just bug fixes.
