node-redis-dump icon indicating copy to clipboard operation
node-redis-dump copied to clipboard

Out of memory

Open Freakachoo opened this issue 5 years ago • 3 comments

On a medium-big DBs it gives an error:

FATAL ERROR: CALL_AND_RETRY_LAST Allocation failed - JavaScript heap out of memory

Freakachoo avatar Aug 06 '19 09:08 Freakachoo

Hi, this is due to how the dump is done: using KEYS *, which redis dump rely on, is an expensive and memory inefficient solution, which works very well on small databases, but is not designed to handle the bigger ones.

I would suggest to look at other options to export/backup your redis data (replication, incremental export etc...)

jeremyfa avatar Aug 06 '19 09:08 jeremyfa

Thank you for suggestions. There are also other dump solutions. I just thought, that maybe would be possible to fix this tool by using SCAN command.

Freakachoo avatar Aug 06 '19 10:08 Freakachoo

Yes, this can probably be improved using smarter commands, but I don't plan to do it myself as I don't need it personally. I may however review and accept any Pull Request to change that, given that it doesn't create regressions in existing features

jeremyfa avatar Aug 06 '19 11:08 jeremyfa