grunt icon indicating copy to clipboard operation
grunt copied to clipboard

grunt.file.expand have sync bottleneck

Open stefanocudini opened this issue 7 years ago • 3 comments

I think that the synchronous code in this line causes a big slow down compared to the reading speed that could be obtained by using the asynchronous method of glob()

https://github.com/gruntjs/grunt/blob/master/lib/grunt/file.js#L105

var matches = processPatterns(patterns, function(pattern) {
    // Find all matching files for this pattern.
    return file.glob.sync(pattern, options);
  });

file.glob.sync() they are executed in sequence instead of in parallel way

stefanocudini avatar Sep 27 '18 15:09 stefanocudini

At the moment all the grunt.file APIs are sync. A large portion of the community expects them to be sync, so I think changing them to be async, is not only a huge task and breaking change, but would be met with a lot of resistance.

Although if any internal parts can become async and remain backwards compatible then I'd be happy to review a PR for it.

shama avatar Sep 27 '18 15:09 shama

I did not mean to say that the API should be asynchronous, even I prefer synchronicity on Grunt it's good.

perhaps the internal loop of processPatterns() in this case can be parallelized but I'm not sure about the performance improvements

stefanocudini avatar Oct 02 '18 14:10 stefanocudini

If you change the internal function to be async, the user facing API also needs to be async. Consider the above example:

var matches = processPatterns(patterns, function(pattern) {
  file.glob(pattern, options, function (err, files) {
    // I have files now but no way to return them unless the user supplies a callback
    // function for me to call but then their API would be async too
  });
});

In order for the user facing APIs to remain sync, the process needs to wait for file.glob.sync to finish. It cannot be parallelized without making it async.

shama avatar Oct 02 '18 15:10 shama