node-rate-limiter-flexible icon indicating copy to clipboard operation
node-rate-limiter-flexible copied to clipboard

Memory limiter: dump and restore functionality

Open sido420 opened this issue 4 years ago • 6 comments

I was thinking if we could add a dump/restore functionality to this module.

Due to the nature of nodejs process, if it goes down we loose everything stored in the in-memory db.

What are your thoughts?

sido420 avatar Sep 28 '19 00:09 sido420

@sido420 This is interesting idea. It would be definitely useful to have such feature in some cases.

animir avatar Sep 28 '19 09:09 animir

It came from the popular LRU cache nodejs module, see https://github.com/isaacs/node-lru-cache.

I think it'd be a huge benefit for single node setups where external storage is unavailable. Or, even multi process setups with sticky sessions etc. or even without sticky sessions it might be of help. Quick app prototypes should also benefit from it :)

sido420 avatar Sep 28 '19 15:09 sido420

@sido420 Yeah, I think so. Community would appreciate it. Let us know if you started to implement it, so nobody else does the same in parallel. Thanks.

animir avatar Sep 29 '19 02:09 animir

No, I haven't started any work on it.

I will comment here if I plan on doing so (not anytime soon).

sido420 avatar Sep 30 '19 03:09 sido420

I stringified _storage in memory storage and got the following:

{"_storage":{"Msg-ByID:1":{"_value":1,"_expiresAt":"2019-10-08T00:59:58.025Z","_timeoutId":{"_called":false,"_idleTimeout":600000,"_idlePrev":null,"_idleNext":null,"_idleStart":229
:/conta│15556,"_repeat":null,"_destroyed":false,"_handle":{}}}}}

Can you tell me what needs to persist so it could be restored at startup?

sido420 avatar Oct 08 '19 00:10 sido420

@sido420 _timeoutId contains data we can't use really. Every key should re-create a timer with setTimeout function depending on _expiresAt and set _timeoutId. So you could remove _timeoutId from dump.

animir avatar Oct 13 '19 03:10 animir