asset-rack icon indicating copy to clipboard operation
asset-rack copied to clipboard

EMFILE when using StaticAssets with a lot of files.

Open chakrit opened this issue 12 years ago • 3 comments

I have made a test repository for this: https://github.com/chakrit/rack-emfile-issue It basically includes:

  • A minified distribution of ace.c9.io with all files.
  • A test file that creates a new Rack with a StaticAssets pointing to the ace files.

Basically, StaticAssets cannot seem to handle EMFILE well. Running node test.js result in this:

/private/tmp/test/node_modules/asset-rack/compiled/rack.js:42
          throw error;
                ^
Error: EMFILE, open '/private/tmp/test/public/ace/snippets/velocity.js'

A bit of googling leads me to https://github.com/isaacs/node-graceful-fs but not sure it'd be as simple as replacing all fs with graceful-fs. This is blocking me from working on one of my websites.

Will check up on the codebase and make a PR if I have time but if you can take a look at this I'd greatly appreciate it.


node --version is v0.10.17 and I'm on OS X ML.

chakrit avatar Aug 31 '13 18:08 chakrit

@chakrit I just ran your test, and I didn't get the error. I would try upping the number of open file handles on your system.

graceful-js seems like a good option to me.

In general, asset-rack is not designed for "large" numbers of static assets as it holds everything in memory.

techpines avatar Aug 31 '13 20:08 techpines

@techpines would you accept a graceful-fs patch then?

I think the problem might be that it is opening too many files too quickly during initialization to load things in memory.

chakrit avatar Sep 01 '13 05:09 chakrit

I think so, as long as it didn't break anything.

techpines avatar Sep 01 '13 22:09 techpines