rsync icon indicating copy to clipboard operation
rsync copied to clipboard

Use case: backup only newer files to intermediary SSD disk: option --newerthan needed

Open reikred opened this issue 3 years ago • 3 comments

Consider use case: backup only newer files to intermediary SSD disk

  1. suppose I have a main drive and some backup drives, plus an SSD

  2. I don't want the backup drives to spin all the time, and the drives may not even be plugged in (hotplug as needed).

  3. Hence it would be nice if I could rsync certain directories from main to SSD, but only the chosen dir and only files that are newer than some time limit (say, hours or days).

  4. the resulting file tree on the ssd could cover multiple partitions of the main drive and could be quite a sparse copy of only newer files.

  5. the ssd backup could then be further rsync'ed to the backup drives at some reduced frequency such as once per day or week.

  6. to this end, it would be nice if rsync accepted an option such as

   --newerthan F:somefile
   --newerthan F:somedir
   --newerthan D:2021-07-01T14:53:00
   --newerthan m:7minutes
   --newerthan h:2hours
   --newerthan d:1days

which restricts files that otherwise would qualify for copying to the backup SSD.

Maybe also support a +/- notation to change the meaning of newer to older. For example

   --newerthan  D:2021-07-01T14:53:00 (newer than)
   --newerthan +D:2021-07-01T14:53:00 (newer than)
   --newerthan -D:2021-07-01T14:53:00 (older than)

There are many users that have asked how to do the above, or essentially equivalent scenarios, that require "newerthan", as can bee seen from a google search or stackexchange search. The solution that keeps coming up is using "find", but would it not be much better if rsync could handle --newerthan internally?

reikred avatar Jul 02 '21 19:07 reikred

It might be sensible to have separate (and simultaneously working) options

--newerthan --olderthan

rather than the aforementioned +/- notation, which is perhaps prone to misunderstanding.

reikred avatar Jul 03 '21 22:07 reikred

I'd suggest that you find the files you want to send using "find" and then send them using a --files-from list.

WayneD avatar Jul 04 '21 19:07 WayneD

I realize there may not be much appetite for implementing --newerthan and --olderthan.

But just for reference, in case someone bites, I realized also that perhaps the better way of implementing time filtering would be to add the functionality as a new form of filter rule that accepts new keywords newerthan and olderthan, that is


--filter olderthan SPEC
--filter newerthan SPEC

where the SPEC uses the syntax I described above (that is, F:somefile , D:datetimestamp, etc)

reikred avatar Jul 05 '21 18:07 reikred