shoebox

shoebox copied to clipboard

Currently only one Store instance can listen to a given directory at a time (although it isn't currently enforced through locking, although it should be).

However, through the Java WatchService it should be able to listen for file modifications caused by other Store instances, possibly running in a different JVM or even on a different server (using a shared filesystem like Amazon's EFS).

Unfortunately this is complicated by the fact that WatchService doesn't work reliably across platforms. There are workarounds like this Mac specific library, so they will need to be used on platforms where WatchService isn't reliable.

Note: Should verify that Amazon's EFS supports realtime file modification notification before we recommend this use-case.

FYI - I asked a question about this on r/java - https://www.reddit.com/r/java/comments/604rfj/is_there_a_reliable_crossplatform_java_library/

Based on what I've learned so-far, I'm concerned that there may not be a sufficiently reliable and efficient way to be notified of file creations, deletions, or updates, that is cross-platform.

To allow Shoebox to scale horizontally, it may therefore be necessary to build on top of something like Hazelcast.

Tape might be an option for logging changes to a file and allowing multiple Shoebox instances to listen for changes.