backup-bash
backup-bash copied to clipboard
Occasionally misses folder creations
If, while running rsync, a new directory is created in the watched tree, it will not be automatically added to the watch (which is the default behavior). This is because inotifywait is only listening between rsync sessions.
My use case is a series of dirs like so: 2015/ = year 01/ = month 15/ = day 01/ = hour 02/ = hour 03/ = hour 03_18_17.file = file
in this case, a file may be written to 2015/01/15/03/03_59_59.file, and while rsync is occurring, a new dir is created (04/) and files written (04_00_03.file) and any subsequent writes to 04/ will not be caught and therefore will not trigger a sync.
There are a few ways to address:
- spawn rsync in a subshell so that the inotifywait do...while can immediately respawn inotifywait
- we can still miss events here, but the worst that can happen is that we miss the directory creation but hopefully catch the next one, and the rsync will sync all changes including those in the "missed" directory
- downside of this is we must enable pid tracking/locking of rsync so that we never run multiple instances
- keep inotifywait running in a subshell with -m option, outputting changes to a FIFO
- anytime the FIFO has text, a sync needs to be done
- again, need to keep track of how many rsyncs are running
- unsure what the effect of having the FIFO blocking the output of inotifywait. Will it continue to hold events until the FIFO is read?
I can't think of doing it any other way but with a subshell in either case.
I understand, this was the best possible way to do it without having to spawn an endless loop of rsyncs, I like the method lsyncd uses
aggregates and combines events for a few seconds and then spawns one (or more) process(es) to synchronize the changes.
may implement this when I have time, PR's are welcome