jimsalterjrs
Is there a way to tell syncoid to remove datasets on the target which no longer exist in the source?
First of all, thank you so much for both Syncoid and Sanoid, been using them for years and recommended it to everyone ever since I found them
I've been running into a small issue lately on a particular host and was wondering if Syncoid can help. I have a server with docker running in a ZFS filesystem. Docker creates datasets for the containers, and everytime one of its components is updated, the dataset from that container is destroyed and a new one is created.
As I obviously use syncoid to back it up, I've run into a problem, where the datasets created by docker are deleted on source, but since the target doesn't know anything about them, they stack up on the backup host and at some point I have to either delete them manually or destroy everything in that pool and run a "clean" syncoid to copy it all back again.
I was wondering if is there a way to tell syncoid to delete any child dataset in the target which doesn't exist in source? The Readme mentions
--force-delete
Remove target datasets recursively (WARNING: this will also affect child datasets with matching snapshots/bookmarks), if there are no matching snapshots/bookmarks. Also removes conflicting snapshots if the replication would fail because of a snapshot which has the same name between source and target but different contents.
But I understand this is only for datasets that exist both in source and target but don't match snapshots, not datasets which exist in target but don't exist in source?
Thanks again for all the work you've put into these tools!
this might be related to your query by the sound of it: https://github.com/jimsalterjrs/sanoid/issues/822
@ovizii That issue is just referring to snapshots?
I'm also interested in this feature. Like @Biktorgj mentioned, I would have expected --force-delete to do this, but it appears not to.
I think I might be seeing the same issue, using both --force-delete and --delete-target-snapshots:
# syncoid --no-sync-snap --delete-target-snapshots --no-privilege-elevation --force-delete --recursive [email protected]:tank2/vms/lab2 dozer0/lab-odin/tank2/vms/lab2
WARN: lzop not available on target [local machine] - sync will continue without compression.
WARN: mbuffer not available on target [local machine] - sync will continue without target buffering.
WARN: pv not available on local machine - sync will continue without progress bar.
NEWEST SNAPSHOT: autosnap_2023-11-10_17:00:01_hourly
INFO: no snapshots on source newer than autosnap_2023-11-10_17:00:01_hourly on target. Nothing to do, not syncing.
Source (odin)
zfs list -t snapshot tank2/vms/lab2 | grep hourly
tank2/vms/lab2@autosnap_2023-11-09_18:00:03_hourly 904M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-09_19:00:02_hourly 490M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-09_20:00:03_hourly 560M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-09_21:00:05_hourly 2.23G - 5.77T -
tank2/vms/lab2@autosnap_2023-11-09_22:00:03_hourly 961M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-09_23:00:03_hourly 537M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_00:00:04_hourly 2.42M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_01:00:20_hourly 573M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_02:00:00_hourly 498M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_03:00:04_hourly 639M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_04:00:00_hourly 669M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_05:00:04_hourly 486M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_06:00:04_hourly 471M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_07:00:03_hourly 472M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_08:00:04_hourly 614M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_09:00:04_hourly 944M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_10:00:05_hourly 745M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_11:00:00_hourly 559M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_12:00:01_hourly 830M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_13:00:01_hourly 582M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_14:00:00_hourly 563M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_15:00:01_hourly 577M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_16:00:04_hourly 618M - 5.77T -
tank2/vms/lab2@autosnap_2023-11-10_17:00:01_hourly 539M - 5.77T -
Destination
zfs list -t snapshot -r dozer0/lab-odin/tank2/vms/lab2 | grep hourly
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_08:00:02_hourly 1.26G - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_09:00:01_hourly 637M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_10:00:01_hourly 583M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_11:00:01_hourly 493M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_12:00:01_hourly 468M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_13:00:02_hourly 543M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_14:00:01_hourly 1.91G - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_15:00:01_hourly 991M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_16:00:01_hourly 588M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_17:00:01_hourly 612M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_18:00:02_hourly 609M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_19:00:01_hourly 510M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_20:00:01_hourly 487M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_21:00:01_hourly 537M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_22:00:01_hourly 556M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-08_23:00:02_hourly 575M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-09_17:00:03_hourly 827M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-09_18:00:03_hourly 488M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-09_19:00:02_hourly 485M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-09_20:00:03_hourly 555M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-09_21:00:05_hourly 2.23G - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-09_22:00:03_hourly 956M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-09_23:00:03_hourly 533M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_00:00:04_hourly 2.33M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_01:00:20_hourly 568M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_02:00:00_hourly 493M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_03:00:04_hourly 633M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_04:00:00_hourly 664M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_05:00:04_hourly 481M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_06:00:04_hourly 467M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_07:00:03_hourly 468M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_08:00:04_hourly 609M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_09:00:04_hourly 938M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_10:00:05_hourly 739M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_11:00:00_hourly 555M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_12:00:01_hourly 825M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_13:00:01_hourly 577M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_14:00:00_hourly 558M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_15:00:01_hourly 572M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_16:00:04_hourly 613M - 5.76T -
dozer0/lab-odin/tank2/vms/lab2@autosnap_2023-11-10_17:00:01_hourly 0B - 5.76T -
