chokidar

chokidar copied to clipboard

I've added dirent support to readdirp v3.

The initial implementation of the approach resides in branch https://github.com/paulmillr/chokidar/tree/dirent.

We need to make it default, when alwaysStat: false

does it make sens to take a look at https://github.com/thecodrr/fdir ?

Just glanced at the project. Seems interesting, but not quite.

After I adjusted its code to use stats instead of Dirent, the performance story became 4x less interesting and in line with readdirp

Running "Synchronous (2079 files, 182 folders)" suite...
  fdir 3.3.0 sync:
    84 ops/s, ±0.39%   | fastest
Running "Asynchronous (2079 files, 182 folders)" suite...
  fdir 3.3.0 async:
    72 ops/s, ±0.57%   | fastest
  readdirp:
    65 ops/s, ±0.51%   | slowest, 9.72% slower

It's definitely interesting for the dirent-only case, but we need to see what's doable with readdirp. In fact, the initial scan doesn't seem to be a big problem for folks. Instead, what seems to matter more is when a single directory has >200k files. Node.js added scandir to resolve this issue recently, but the scandir by itself was slow last time I checked.

FYI I've been using the following function for a while and yesterday I benchmarked it against fdir, after reading this issue, and this simple function is about twice as fast as fdir:

  function readdirp ( folderPath: string ): Promise<string[]> {

    const normalizePath = Env.is.windows ? ( filePath: string ): string => filePath.replace ( /(?:\/|\\)+/g, '\\' ) : ( filePath: string ): string => filePath,
          files: string[] = [];

    const populateFiles = async ( folderPath: string ): Promise<string[]> => {

      const dirents = await fs.promises.readdir ( folderPath, { withFileTypes: true } ).catch ( error => console.error ( error ) ) || [];

      if ( !dirents.length ) return files;

      await Promise.all ( dirents.map ( ( dirent ): Promise<string[]> | undefined => {

        if ( SettingsStatic.junk.nameRe.test ( dirent.name ) ) return;

        const subPath = Path.resolve ( folderPath, dirent.name );

        if ( dirent.isDirectory () ) return populateFiles ( subPath );

        files.push ( normalizePath ( subPath ) );

      }));

      return files;

    };

    return populateFiles ( folderPath );

  }