Changes to the main entry with in memory cache are ignored

[ngryman]

Hey,

If I modify the contents of the main bundle entry re-using the same bundler instance, output is not updated accordingly. It seems that the cache is not properly invalidated. However sometimes it does work. Could it be related to mtime resolution not being precise enough? I'm on OS X.

Here is a gist showing the issue: https://gist.github.com/ngryman/52ca69f809d4c38f4cdb4db710de3592

git clone https://gist.github.com/52ca69f809d4c38f4cdb4db710de3592.git
cd 52ca69f809d4c38f4cdb4db710de3592
npm install
node index.js # re-use the same bundler instance, fails
node index.js --force # force re-create a new bundler, succeeds

/ref: https://github.com/ngryman/gulp-bro/issues/4

[ngryman]

@jsdf Do you think you some have time to throw a glance? Thanks!

[jsdf]

It looks like the issue is mtime resolution. In your example, if you move delay(2000) before updateMainEntry, it works correctly.

Unfortunately this is just a limitation of fs.stat on HFS+ partitions. The coarse, 1 second resolution means it's simply not possible to observe that a file has been changed multiple times in one second via fs.stat on the on HFS+ filesystem.

It looks like you're trying to drive browserify-incremental via gulp watch, which to me just seems like a use case which is better served by watchify. If you look at how watchify is implemented, it uses every filesystem event to invalidate the cache, as well as to kick off bundling. browserify-incremental is more intended for the use case where changes to the files have settled before bundling starts. If you want to use browserify-incremental in this way, debouncing the call the bundle by 1000 ms might help.

A more robust fix for the resolution problem might be for browserify-incremental to implement content digest/hash based caching, which isn't a bad idea from a cache-correctness point of view. However such a strategy would be slower to invalidate the cache for npm dependencies with many files (React etc), as they would all have to be read into memory and hashed before each build, so you would probably want a hybrid strategy (eg. hash based caching of app code and mtime based for node_modules) and I probably don't have in the near future time to get all that working.

[ngryman]

Thanks a lot for those clarifications.

If it's related to mtime resolution, then why does it only fail with the main bundle file, and not any other file? I'm trying to understand this.

Thanks!

[ngryman]

@jsdf Do you have an explanation for this? Thanks!