Watching a Directory does not fire Events

[jayharris]

Occasionally I want to watch an entire directory, such as someDirectory/** and trigger an event on any changes to the directory. This will create not only an fs.watch for the directory but fs.watchFile for each file in the directory. If I change one file in the directory, both will fire (as expected).

However, as we already know, this system slows down with a lot of file handles. Since both fire, it would be helpful to check to see if the pattern is a directory (perhaps using someDirectory/ as a pattern rather than someDirectory/**, and if the source pattern is the directory itself, still fire the changed event.

Currently, if the pattern is the directory itself, nothing happens; no events are fired.

This change would give us the option to lighten the file handle load on many systems, especially in situations such as as grunt watch where js/**/*.js and js/ are often equivalent.

[shama]

What is the expected behavior of watching just a folder? Just knowing when that folder has been renamed or deleted? If you just want to watch a single layer of files within a folder you could do: dir/*.

There is a recursive folder watch that uses native events coming down the tubes aimed for node v0.12 that would be interesting to look into. Although the next version of gaze will use all native events with continued cross-platform support (but still open file descriptors for each file). Check out the v0.5 branch here. So fairly soon we're getting a big performance boost.

[jayharris]

My thought was that someFolder/ as a pattern could be functionally equivalent to someFolder/*.* Changing a file seems to not only emit changed on its watchFile, but also changed on the watch for its directory.

If the callback intends to be fired on all file changes in the directory, someDirectory/ could serve as a shortcut for this purpose without creating the file handles for every file in the directory.

[jayharris]

Thinking on this over the evening, someFolder/ is still a valid pattern. someFolder/**/ is as well, which would return someFolder/ as well as all sub-directories, but not the files.

It would be valuable if changed and renamed events on fs.watch(dir) both emitted a changed event within Gaze.

I'm working on a PR for this right now...

[shama]

Be sure to check out the v0.5 branch. Lots of changes happening there for the next version. Also we should keep the patterns true to what globule provides for compatibility. As many libraries use the glob patterns and expect watching to operate the same.

[jayharris]

Using Gulp or Grunt, if I am watching ./js/*.js with 1,000 javascript files, the open file handles will kill my local machine.

However, since the only thing in my ./js/ folder is javascript files, I could just as easily throw fs.watch on the directory, rather than on every individual file. Since changing, adding, deleting, or renaming a file will fire the event on its containing directory, too, if any one of those files changes, it will still trigger changed on the directory. I can set Grunt to watch ./js/, instead of ./js/*.js, and it would still fire changed and still trigger the Grunt task.

globule supports directory patterns. As I'm sure you are already aware, **/* will return the directories as well as the files, and **/ will return just the directories. So, this should work without causing any compatibility issues or changing any existing operation.

Neither v0.4 nor v0.5 emits changed on a directory, even if the directory is monitored. However, because _watched keys are those containing a watched file, and _watched values are all directories within that key directory, it's not as easy as just emitting changed inside of _watchedDir or its callback, as changed could be fired on unmonitored directories.

I know that this isn't a solution for everything. If I have .coffee and .js files in the directory, watching ./js/ means that the Coffee tasks would trigger even when changing a javascript file. However, in very large file systems, the expense of running the extra grunt tasks would be far, far less than the expense of 900 file handles; overall, there would be a big boost in performance.

I would certainly want Grunt to stay with ./**/*.js by default, but emitting these changed events from Gaze when monitoring a directory would give large-system developers the option of changing their ./**/*.js globs to ./js/ to help with local system performance.

Does this all make sense?