sanoid icon indicating copy to clipboard operation
sanoid copied to clipboard

Recursively sending datasets via `syncoid` with `--recvoptions="uo canmount=off"` to avoid conflicting mountpoint=values fails to recieve zfs volumes due to `canmount` not being a relevant flag.

Open ipaqmaster opened this issue 1 year ago • 2 comments

Hey fellas,

I've got a machine recently reconfigured to use syncoid for recursively replicating its primary zpool and child datasets to a different local zpool. This includes the host's rootfs dataset, mounted and in use.

The initial issue experienced approaching this machine was that recursively sending the datasets resulted in the received side overmounting their live counterparts eventually. Such as when the host is rebooted - that is to say they're not configured to use mountpoint=legacy which the use of would solve this problem.

At boot time on this particular machine the received datasets overmount their intended live counterparts, most critically the rootfs dataset causing critical systemd services to topple over at startup including networking which prevents remote troubleshooting without an out of band hardware level access solution as even getty@ttyS0 services fail.

To solve this I set canmount=off and -u to avoid immediately mounting them on the receiving end so they won't overmount the rootfs at boot time for this setup. To avoid this catastrophe I'm running syncoid with the equivalent flags --recvoptions="uo canmount=off" so when they're received they don't cause the host to boot into a confused state.

By good design volume's lack a canmount option because they're virtual block devices. There's no check in syncoid for whether an option is applicable for a particular dataset/volume so the recv command fairly fails with cannot receive incremental stream: property 'canmount' does not apply to datasets of this type.


I suppose that's where I'm up to with this one. Recursively sending datasets to the other zpool causes them the overmount the live system (at least at boot time) which fairly confuses systemd. The canmount=no option has taken care of this though not for zvol's. Some short questions regarding this situation:

  • Any thoughts on a more sane way to handle this mounting problem so that --recvoptions="uo canmount=off" isn't required thus allowing zvol's to also be sent without the zfs-recv side erroring? I understand that using mountpoint=legacy is an acceptable answer to this problem however this machine isn't in an immediate position to have that change made to its many child datasets just yet so this isn't my first pick. But if it's the best way to go about things then OK.

  • Is there another in-built way to deal with this from ZFS or Syncoid that I'm forgetting about?

  • Is this something syncoid should even care about upon seeing a receive option for a filesystem/snapshot/volume/bookmark which doesn't support it?

Cheers

ipaqmaster avatar May 06 '23 09:05 ipaqmaster

Unfortunately I haven't been able to find a workaround to send recursive zpool datasets without them mounting on the other side while also sending zvols.

canmount isn't inheritable so while sending datasets with --recvoptions="o canmount=off" does the trick, it causes zvol sends to fail because of cannot receive incremental stream: property 'canmount' does not apply to datasets of this type.

Omitting the send option p successfully inherits a parent "received" dataset's mountpoint=none, but this is destructive as we lose the dataset's original mountpoint data this way. Meaning --sendoptions="pw" is still required for our encrypted datset, with --recvoptions="o canmount=off"

I don't think there's any other way to surpass this problem for recursive sends without adding extra functionality in syncoid to detect a zvol being sent and omit the option for that particular thing (When its turn to send comes up).

ipaqmaster avatar Aug 12 '23 04:08 ipaqmaster

** As in, having Syncoid verify user specified recvoptions for validity before sending each dataset type before attempting the send.

e.g. Checking zfs get ${recvoptionsPropertyToBeSent} xx/yy/zz to see if a property we're about to ask zfs-recv to enforce, is even relevant for a dataset type which is about to be sent. Validating receive options on a dataset to dataset basis.

ipaqmaster avatar Aug 12 '23 04:08 ipaqmaster