Syncoid: resume check on OmniOS broken

[asche77]

Syncoid fails to detect Illumos ZFS resume capability, and errors "WARN: ZFS resume feature not available on source and target machine - sync will continue without resume support."

However, Illumos/OmniOS supports zfs send/receive resume.

Syncoid Check command is correct: "zpool get -o value -H feature@extensible_dataset ssdpool" returns "active".

However the regex grep in syncoid fails to trigger: "grep '\(active\|enabled\)'" does not catch the "active". Without the \\ it should work...?

(Please note all \ above are double \ in the code, but this does not display correctly here)

Should be a simple fix but I am not a coder...

[asche77]

Ran some half-assed tests:

Yes, the current grep regexp in syncoid fails under illumos. It also fails in online regexp checkers.

What works under Illumos/grep 3.4 is:

grep -E (active|enabled)

or

grep -E  ^(active|enabled)

(no escape chars). Without "-E" (extended regexps), it fails.

Using the following permits resumable syncoid in Illumos:

grep -E ^(active|enabled)

Therefore, please could someone prepare a pull request?

<< $avail{'sourceresume'} = system("$sourcessh $resumechkcmd $srcpool 2>/dev/null | grep '\\(active\\|enabled\\)' >/dev/null 2>&1");

>> $avail{'sourceresume'} = system("$sourcessh $resumechkcmd $srcpool 2>/dev/null | grep -E '^(active|enabled)' >/dev/null 2>&1");

<< $avail{'targetresume'} = system("$targetssh $resumechkcmd $dstpool 2>/dev/null | grep '\\(active\\|enabled\\)' >/dev/null 2>&1");

>> $avail{'targetresume'} = system("$targetssh $resumechkcmd $dstpool 2>/dev/null | grep -E '^(active|enabled)' >/dev/null 2>&1");
 

[phreaker0]

@asche77 are you sure? According to the manpage https://illumos.org/man/1/grep it also uses BRE by default. Can you try the following in a bash shell on illumos:

echo "active" | grep "\(active\|enabled\)" && echo match

[asche77]

Hi phreaker0, thank you.

1. I am sure that the check in the current 2.0.3 syncoid is broken because it claims resume is not supported. Implementing my hacked regexp permits syncoid to resume zfs sends on OmniOS r34.

2. Your pipe has no output on OmniOS r34: xxx@omniosce:~$ echo "active" | grep "\(active\|enabled\)" && echo match xxx@omniosce:~$ xxx@omniosce:~$ echo "active" | grep -E "\(active\|enabled\)" && echo match xxx@omniosce:~$ xxx@omniosce:~$ echo "active" | grep "^(active|enabled)" && echo match xxx@omniosce:~$

however with the "-E" switch and the "^(...)" regex:

xxx@omniosce:~$ echo "active" | grep -E "^(active|enabled)" && echo match active match

3. On Debian, however, your pipe works:

xxx@debian-qotom:~$ echo "active" | grep "\(active\|enabled\)" && echo match active match

alternative ^ works only with the -E switch:

xxx@debian-qotom:~$ echo "active" | grep "^(active|enabled)" && echo match xxx@debian-qotom:~$ echo "active" | grep -E "^(active|enabled)" && echo match active match

4. The OmniOS tests were made under ksh (the standard shell), but behaviour under bash is the same, i.e. your pipe does not output anything:

xxx@omniosce:~$ echo $0 bash xxx@omniosce:~$ echo "active" | grep "\(active\|enabled\)" && echo match xxx@omniosce:~$

Thanks for picking this up!

[secabeen]

Is there also an issue with the number of backslashes? I'm getting a failure to see the active feature on FreeBSD with a remote source and local target. When reaching out through ssh, you need a second set of slashes to protect against the far end shell, but when running locally, you don't.

It's probably worth trying to re-write the grep regex to not have backslashes at all.

[mreymann]

Same problem on SmartOS. Well two problems actually:

Standard grep never matches extended regular expressions IMO. Here is an example which should be portable between Linux and SmartOS: $ echo -e "active\nfooo\nenabled" | grep -E "^(active|enabled)$" && echo match

The other problem is that the system call returns "256":

[17:03:18][root@smart01:~]$perl -e 'my $ret = system("zpool get -o value -H feature@extensible_dataset zones 2>/dev/null | grep enabled >/dev/null 2>&1"); print "ret=$ret\n"'
ret=256

while syncoid expects a "0" in order to enable the resume feature. On plain shell this works:

[17:05:23][root@smart01:~]$zpool get -o value -H feature@extensible_dataset zones 2>/dev/null | grep enabled >/dev/null 2>&1
[17:05:37][root@smart01:~]$echo $?
0

Since I'm not a Perl coder I can only supect that the exit code stems from the pipe or the shell redirection in the system call.

[asche77]

@phreaker0 - since this affects some further people and operating systems, pinging you that this issue is still extant (I closed it in error on 4 June).

[hyperbart]

@phreaker0 running Ubuntu 20.04 LTS on the target and the source is running Debian, all recent. Experiencing the same issues, although everything is updated it states that the source side does not support resume.

[neobenedict]

edit: never mind

[mreymann]

Seems this software is abandoned and we won't get a fix - broken on debian too. Anyone have an alternative?

I suggest Tobi Oetiker's https://www.znapzend.org/

[jimsalterjrs]

Of course it's not abandoned.

[redmop]

I have the current release version running on CentOS 7, Debian 11, Ubuntu 18.04, 20.04, and 22.04, as well as Proxmox 7. Both sanoid and syncoid are working perfectly.

[asche77]

It's really only the one change to the "grep" command required to make it run fine on OmniOS (and presumably Illumos and SmartOS, too). Been running it for 2+ years now on OmniOS with a manual patch to the grep command.

On Debian both grep commands work fine - the one that phreaker proposed and the one used by me on OmniOS, i.e. the "-E" switch.