wyng-backup
wyng-backup copied to clipboard
Grouping updates together into single transactions
Grouping multiple updates from send
(for example) into a single transaction may have advantages for both speed and integrity.
Ideas:
- Add metadata .tmp updates to a queue in the ArchiveSet, instead of going through the normal save-and-rename chain.
- Use a global transaction id and append it to the name of each metadata file (instead of using '.tmp' suffix).
- Do all update_delta_digest() in one batch (for incrementals) and then group the zero-change vols together during
send
; then process the resulting metadata update queue. This saves a lot of latency overhead when--send-unchanged
is used. - Possibly use this to simplify recovery from in_process interruption.
- Possibly re-use tar stream process and start/manage it in monitor_send() instead.