timesnap icon indicating copy to clipboard operation
timesnap copied to clipboard

Run-time requires cause increase in open file descriptors

Open Asheboy opened this issue 5 years ago • 2 comments

The requires here can cause a large number of file descriptors to be opened. I experienced this issue when taking over 15,000 screenshots and noticed the following number of files open:

  23828 /node_modules/puppeteer/lib/ExecutionContext.js                                                                         
  23828 /node_modules/puppeteer/lib/Connection.js                                                                               
  23826 /node_modules/timesnap/lib/overwrite-time.js                                                                            
  23826 /node_modules/puppeteer/lib/helper.js                                                                                   
  11913 /node_modules/timesnap/lib/utils.js                                                                                     
  11913 /node_modules/timesnap/index.js                                                                                         
  11913 /node_modules/puppeteer/lib/DOMWorld.js

You can resolve this by moving the requires up out of the exported function, so they are only required once. See here and here.

Asheboy avatar Feb 25 '20 20:02 Asheboy

Can you verify that your changes result in less file descriptors? From a quick, cursory search, I didn't find too much about beginning-of-file requires being more efficient than in-program ones, and in both cases they should only be require'd once per run.

tungs avatar Mar 04 '20 02:03 tungs

@tungs I agree with you, node should cache the requires. When I was investigating this, the requires where the only thing I could think of which could be causing this, and to confirm, moving them up did stop the mass of file descriptors being opened.

Asheboy avatar Mar 04 '20 09:03 Asheboy