chokidar

chokidar copied to clipboard

Describe the bug

When lots of files are being watched the way throttling is currently implemented causes a performance issue.

Versions (please complete the following information):

• Chokidar version: 3.3.1
• Node version: 12.14.1
• OS version: macOS 10.14

To Reproduce:

Watch a directory with lots of files, the following screenshots are the result of watching a directory with ~2000 files.

Expected behavior

Throttling shouldn't account for a sizable portion of the work being done by chokidar.

Additional context

First of all there's a significant amount of work caused by throttling:

Most of the time is being wasted just setting timeouts:

I'm not sure if throttling is really necessary when emitting the first add events, but if it's needed it shouldn't account for this much work.

Currently setting these timeouts is a O(n) operation, it could kind of be made a O(1) operation by setting an interval instead, which every 50ms or whatever loops through all throttled invocations, and if at least 50ms have passed since they got queued they are removed from the queue.

Once those timeouts elapse there's another big computation chunk that perhaps could be avoided:

Some of that work seems to be spent clearing out timeouts:

It might be useful to note here that when the clear function is called from the timeout it doesn't make sense to clear out the timeout that just got triggered, as nothing would practically change, so perhaps we could save some time there by avoiding clearing out timeouts unnecessarily.

Apparently all those actions being throttled are also getting throttled for 0ms, which becomes 1ms under Node, that's a little weird, maybe on initial adds there shouldn't be a need for throttling at all 🤔 like if we are getting a duplicate filesystem event within 1ms that probably means there's a bug somewhere that can be fixed.

Fabio, thank you for doing the profiling!