`configure` script misdetects new versions of macOS

[carlocab]

System (please complete the following information):

• OS: macOS 11.6
• Kernel version (if applicable): Darwin Kernel Version 20.6.0: Mon Aug 30 06:12:21 PDT 2021; root:xnu-7195.141.6~3/RELEASE_X86_64
• strongSwan version(s): 5.9.4
• Tested/confirmed with the latest version: yes

Describe the bug

The configure script packaged with the strongswan-5.9.4.tar.bz2 tarball misdetects macOS versions 11 and newer and mistakenly identifies them as an old version of OS X.

This causes configure to erroneously pass the -flat_namespace flag to the linker during the compilation of libvici.0.dylib. This flag is needed for compatibility with (very old) versions of OS X, but can unnecessarily cause issues for users of this shared library on current versions of macOS.

See, for example, Homebrew/homebrew-core#85973, Homebrew/homebrew-core#85974, Homebrew/homebrew-core#87103.

To Reproduce Steps to reproduce the behavior:

1. Use macOS 11 or newer.
2. In the root of the unpacked strongswan-5.9.4.tar.bz2, run ./configure using the flags shown here, and then make.
3. Run otool -hv src/libcharon/plugins/vici/.libs/libvici.0.dylib to inspect the mach-o header flags.
4. Observe that the TWOLEVEL flag is missing.

Expected behavior The TWOLEVEL flag should be present to indicate that libvici.0.dylib was compiled without the -flat_namespace flag. This flag is present if you follow the steps above on macOS 10.15.

For reference, this is the output of otool -hV libvici.0.dylib when built on macOS 11:

libvici.0.dylib:
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64   X86_64        ALL  0x00       DYLIB    15       1536 DYLDLINK NO_REEXPORTED_DYLIBS

This the the same command run when built on macOS 10.15:

libvici.0.dylib:
Mach header
      magic  cputype cpusubtype  caps    filetype ncmds sizeofcmds      flags
MH_MAGIC_64   X86_64        ALL  0x00       DYLIB    15       1584   NOUNDEFS DYLDLINK TWOLEVEL NO_REEXPORTED_DYLIBS

Logs/Backtraces N/A

Additional context At Homebrew, we are using this patch on configure to correct this behaviour. See Homebrew/homebrew-core#87564.

The configure patch is based on this patch to libtool.m4. The patch has not yet been merged since the Libtool maintainers are short on time these days, but it should be included in the next release of libtool.

I'm not sure if you're willing to carry a patch to libtool.m4 in the meantime. If not, it might be helpful to document this behaviour for your macOS users until this is fixed upstream.

[tobiasbrunner]

Thanks for the report. As you noticed, we don't keep a local copy of libtool.m4 (or any other autotools macros) in our repository. And I don't really want to change that. I guess we could try to patch configure in the dist-hook but not sure how reliable that would be with different versions of autoconf/libtool. So I'm currently leaning towards just documenting it. I've added a note to macOS for now and will keep this issue open until the patch isn't required anymore.

By the way, the Linux build fix in https://github.com/Homebrew/homebrew-core/commit/86de7b35b86bc5b82ec354fd09dbcdb2ee12069e is incomplete as it leaves Linux users without a kernel-net plugin. On Linux, --enable-kernel-netlink should be added (provides kernel-ipsec and kernel-net), and --enable-kernel-pfkey should be removed (the PF_KEYv2 interface on Linux is severely limited).

[carlocab]

Thanks for taking a look! I've opened Homebrew/homebrew-core#87998 to make the changes you suggested.