Failures on Darwin PPC

[barracuda156]

There are two problems:

1. On 10.5.8 (ppc/x86) and 10.6 PPC it fails to build due to missing libdispatch.
2. On 10.6.8 Rosetta recent versions (2.0.6–2.0.7) fail because the build system erroneously pulls Intel SSE instructions. Details of both failures: https://trac.macports.org/ticket/64202

Right now I am on Rosetta (away from my PowerMacs), and trying to fix 2.0.7 for PPC for Macports. I tried setting CMAKE_SYSTEM_PROCESSOR to ppc, tried disabling SSE via -DTORRENT_HAS_SSE=0, nothing works. I get multiple errors of this kind:

In file included from /opt/local/lib/gcc12/gcc/powerpc-apple-darwin10/12.2.0/include/immintrin.h:35:
/opt/local/lib/gcc12/gcc/powerpc-apple-darwin10/12.2.0/include/pmmintrin.h:49:2: error: #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this warning."
   49 | #error "Please read comment above.  Use -DNO_WARN_X86_INTRINSICS to disable this warning."
      |  ^~~~~

/opt/local/lib/gcc12/gcc/powerpc-apple-darwin10/12.2.0/include/smmintrin.h: In function '__m128 _mm_round_ps(__m128, int)':
/opt/local/lib/gcc12/gcc/powerpc-apple-darwin10/12.2.0/include/smmintrin.h:209:24: error: '__builtin_vsx_xvrspic' requires the '-mvsx' option
  209 |         __r = vec_rint ((__v4sf) __A);
      |                        ^
/opt/local/lib/gcc12/gcc/powerpc-apple-darwin10/12.2.0/include/smmintrin.h:209:24: note: overloaded builtin '__builtin_vec_rint' is implemented by builtin '__builtin_vsx_xvrspic'
/opt/local/lib/gcc12/gcc/powerpc-apple-darwin10/12.2.0/include/smmintrin.h:232:24: error: '__builtin_vsx_xvrspic' requires the '-mvsx' option
  232 |         __r = vec_rint ((__v4sf) __A);

In file included from /opt/local/libexec/boost/1.76/include/boost/config/detail/suffix.hpp:546,
                 from /opt/local/libexec/boost/1.76/include/boost/config.hpp:61,
                 from /opt/local/var/macports/build/_opt_PPCRosettaPorts_net_libtorrent-rasterbar/libtorrent-rasterbar/work/libtorrent-rasterbar-2.0.7/include/libtorrent/config.hpp:50,
                 from /opt/local/var/macports/build/_opt_PPCRosettaPorts_net_libtorrent-rasterbar/libtorrent-rasterbar/work/libtorrent-rasterbar-2.0.7/src/bt_peer_connection.cpp:41:
/opt/local/libexec/boost/1.76/include/boost/multiprecision/cpp_int/intel_intrinsics.hpp: In function 'unsigned char boost::multiprecision::detail::addcarry_limb(unsigned char, boost::multiprecision::limb_type, boost::multiprecision::limb_type, boost::multiprecision::limb_type*)':
/opt/local/libexec/boost/1.76/include/boost/multiprecision/cpp_int/intel_intrinsics.hpp:103:22: error: '_addcarry_u32' was not declared in this scope
  103 |    return BOOST_JOIN(BOOST_MP_ADDC, 32)(carry, a, b, reinterpret_cast<unsigned int*>(p_result));
      |                      ^~~~~~~~~~~~~
/opt/local/libexec/boost/1.76/include/boost/multiprecision/cpp_int/intel_intrinsics.hpp: In function 'unsigned char boost::multiprecision::detail::subborrow_limb(unsigned char, boost::multiprecision::limb_type, boost::multiprecision::limb_type, boost::multiprecision::limb_type*)':
/opt/local/libexec/boost/1.76/include/boost/multiprecision/cpp_int/intel_intrinsics.hpp:108:22: error: '_subborrow_u32' was not declared in this scope
  108 |    return BOOST_JOIN(BOOST_MP_SUBB, 32)(carry, a, b, reinterpret_cast<unsigned int*>(p_result));
      |                      ^~~~~~~~~~~~~

All these includes should not be used at all for ppc (Big Endian). VSX is not supported on G4/G5, and even if it were, those GCC headers are useful only for ppc64le.

UPD. Strangely, the second failure seems to be specific to gcc12 – it does not occur with gcc10.

[arvidn]

part of the fix is probably to make sure TORRENT_USE_SC_NETWORK_REACHABILITY is disabled for old MacOS versions.

[barracuda156]

@iains We may have a peculiar failure with gcc12, it tries to use Intel intrinsics-related includes aimed for ppc64le with support of VSX (ISA 2.07+) for Rosetta, which obviously cannot work. In a completely identical scenario gcc10 does not do that, and the build does not fail (will update with logs once it succeeds).

I am building libtorrent-rasterbar now from the default portfile, no extra patches or Rosetta-specific settings mentioned above. gcc10 works, gcc12 fails. (To be precise, I use gcc10-bootstrap, which is 10.3.0, and gcc12 12.2.0, both built for ppc on 10.6.8.)

P. S. I thought something broke down with libtorrent-rasterbar post-2.0.5 or otherwise I cannot figure out what I did last time so that it built on Rosetta, but now fails. Turned out, the issue is a different compiler – most likely I used gcc11 on Rosetta before (for sure not gcc12, did not build it yet that time).

[barracuda156]

part of the fix is probably to make sure TORRENT_USE_SC_NETWORK_REACHABILITY is disabled for old MacOS versions.

@arvidn Thank you, I will try it on Leopard and 10.6 PPC once I am back to my PPC hardware (somewhat later this month). On 10.6.8 it will likely work without that, I had 2.0.5 built on Rosetta earlier – looks like this time it is a GCC issue here.

[arvidn]

actually, it looks lit it's really TORRENT_USE_SYSTEMCONFIGURATION

[barracuda156]

@iains @arvidn Logs of the builds: libtorrent-rasterbar_gcc10_ppc32.log libtorrent-rasterbar_gcc12_ppc32_fail.log

[iains]

1. it is not impossible that there could be such an error (there have been a lot of changes in the PPC intrinsics recently).
2. there are all kinds of X86-related error messages in the logs
3. You need to isolate a repeatable reproducer that we can use for a GCC bug report, the intrinsics code is common and therefore it might be necessary for someone else to take a look too.

To do 3, the first step would be to try building the package with a known working 12.2 compiler outside of the macports structure If you are using rosetta on 10.6.8 then you have to be triple-careful that the configuration process does not get confused into thinking it's on X86.

[barracuda156]

@iains It uses Cmake, so normal triple settings do not work. I can say for sure that CMAKE_OS_X_ARCHITECTURES was set to ppc (Macports cmake PG does that) and I tried also to add CMAKE_SYSTEM_PROCESSOR manually which was passed to configure successfully. Other than that, I am not aware of any methods to force a correct arch in such a scenario. (Of course I saw x86-related errors and was pretty sure it is Cmake pulls cpu_arch from somewhere, so spent quite some time to force ppc, and then to disable SSE manually, passing flags and patching sources. Nothing worked, zero effect.) gcc12 is built with no patches added from my side (it is Rosetta, so it builds fine as is). It is working normally, I have built tons of ports with it including latest Cmake, llvm5, folly etc. Settings were strictly identical, where gcc10-bootstrap works and gcc12 fails.

UPD. gcc12 config:

10:~ svacchanda$ /opt/local/bin/gcc-mp-12 -v
Using built-in specs.
COLLECT_GCC=/opt/local/bin/gcc-mp-12
COLLECT_LTO_WRAPPER=/opt/local/libexec/gcc/powerpc-apple-darwin10/12.2.0/lto-wrapper
Target: powerpc-apple-darwin10
Configured with: /opt/local/var/macports/build/_opt_PPCRosettaPorts_lang_gcc12/gcc12/work/gcc-12.2.0/configure --prefix=/opt/local --build=powerpc-apple-darwin10 --enable-languages=c,c++,objc,obj-c++,lto,fortran,jit --libdir=/opt/local/lib/gcc12 --includedir=/opt/local/include/gcc12 --infodir=/opt/local/share/info --mandir=/opt/local/share/man --datarootdir=/opt/local/share/gcc-12 --with-local-prefix=/opt/local --with-system-zlib --disable-nls --program-suffix=-mp-12 --with-gxx-include-dir=/opt/local/include/gcc12/c++/ --with-gmp=/opt/local --with-mpfr=/opt/local --with-mpc=/opt/local --with-isl=/opt/local --with-zstd=/opt/local --enable-checking=release --disable-multilib --enable-lto --enable-libstdcxx-time --without-build-config --with-as=/opt/local/bin/as --with-ld=/opt/local/bin/ld --with-ar=/opt/local/bin/ar --with-bugurl=https://trac.macports.org/newticket --enable-host-shared --with-darwin-extra-rpath=/opt/local/lib/libgcc --with-libiconv-prefix=/opt/local --with-tune-cpu=G4 --disable-tls --with-pkgversion='MacPorts gcc12 12.2.0_0'
Thread model: posix
Supported LTO compression algorithms: zlib zstd
gcc version 12.2.0 (MacPorts gcc12 12.2.0_0)

[iains]

OK - understood - but we will still need a reproducer for the bug outside the macports system, to file as a GCC bug, there is no reasonable way for the GCC devs to debug a compiler problem within an external distribution's build system.

[barracuda156]

@arvidn @iains It looks like all Intel-related errors with gcc12 are coming from intrinsic-related headers. Source code has three instances where those are mentioned:

1. In cpuid.cpp:

#if defined _MSC_VER && TORRENT_HAS_SSE
#include <intrin.h>
#include <nmmintrin.h>
#endif

2. In crc32c.cpp:

#include <boost/crc.hpp>
#if (defined _MSC_VER && _MSC_VER >= 1600 && (defined _M_IX86 || defined _M_X64))
#include <nmmintrin.h>
#endif

On these two gcc12 fails. Further down the road there is one in ffs.cpp:

#if (defined _MSC_VER && _MSC_VER >= 1600 && (defined _M_IX86 || defined _M_X64))
#include <nmmintrin.h>
#endif

Internal config.hpp header has this:

#if (defined _M_AMD64 || defined _M_IX86 || defined _M_X64 \
	|| defined __amd64__ || defined __i386 || defined __i386__ \
	|| defined __x86_64__ || defined __x86_64) \
	&& (defined __GNUC__ || (defined _MSC_VER && _MSC_VER >= 1600))
#define TORRENT_HAS_SSE 1
#else
#define TORRENT_HAS_SSE 0
#endif

I do not know what _MSC_VER define stands for. Passing -DTORRENT_HAS_SSE=0 to configure made no effect.

So on one hand it seems that Cmake does pull x86 define from somewhere, which triggers all these Intel intrinsics-related headers being pulled in. If so, then it can (and perhaps should) be fixed within libtorrent code. On another hand, I do not understand why gcc10 do not behave in a same way and builds the port just fine.

[iains]

Well I have to say, at present, this seems a configuration issue - _MSC_VER relates to Windows. None of the defines mentioned in the third entry should be active for a ppc arch ..

Look for (a) somehow the system thinking it's on Windows .. (b) somehow x86 getting enabled - which would set __i386__ et al.

you could also do:

gcc-12 -arch ppc -dM -E -xc /dev/null | grep i386

( or any of the other defines ) to make sure that the compiler really is not setting them!

edit: I'm not sure what _M_X64 relates to .. perhaps some other header is setting that (it seems to be common between the cases).

[iains]

BTW, I see in the first comment On 10.5.8 (ppc/x86) and 10.6 PPC it fails to build due to missing libdispatch.

If the code is using Apple Blocks, then it will not work with any current GCC .. you'd have to look for each place using blocks and find the equivalent API without blocks, and then compile conditionally on ___BLOCKS___

Often it is possible to find a suitable alternative.

[barracuda156]

Well I have to say, at present, this seems a configuration issue - _MSC_VER relates to Windows. None of the defines mentioned in the third entry should be active for a ppc arch .. Look for (a) somehow the system thinking it's on Windows .. (b) somehow x86 getting enabled - which would set __i386__ et al.

@arvidn Maybe you could suggest some way to ensure none Intel definitions are used on 10.6.x Rosetta (where the physical cpu in Intel, but ppc is emulated natively by the OS itself)?

you could also do:

gcc-12 -arch ppc -dM -E -xc /dev/null | grep i386

( or any of the other defines ) to make sure that the compiler really is not setting them!

@iains No output:

10:~ svacchanda$ /opt/local/bin/gcc-mp-12 -arch ppc -dM -E -xc /dev/null | grep i386
10:~ svacchanda$ /opt/local/bin/gcc-mp-12 -arch ppc -dM -E -xc /dev/null | grep x86

Why though gcc10 does not fail identically (if the error is caused by Cmake – I cannot see any explicit problem with libtorrent definitions)?

[iains]

@iains No output:

10:~ svacchanda$ /opt/local/bin/gcc-mp-12 -arch ppc -dM -E -xc /dev/null | grep i386
10:~ svacchanda$ /opt/local/bin/gcc-mp-12 -arch ppc -dM -E -xc /dev/null | grep x86

OK .. as expected.

Why though gcc10 does not fail identically (if the error is caused by Cmake – I cannot see any explicit problem with libtorrent definitions)?

Of course, it is possible there is a fault in GCC-12, of course (hence the request for a reproducer).

OTOH, checking the definition of "working" (gcc-10) do you mean that it builds? or do you mean that it has been tested as functional?

[barracuda156]

@arvidn @iains Did not get a chance to test functionality yet, however I got another update re GCC:

gcc11 builds libtorrent on 10.6.8 Rosetta normally, without x86 intrinsics errors. So far only gcc12 fails.

libtorrent_gcc11_ppc32.log

[barracuda156]

@iains @arvidn To update on the matter, on PPC natively gcc12 fails absolutely identically. So it is not a fault of physical cpu being x86 in a case of Rosetta.

I do not have a clean standalone installation of gcc12 at the moment to submit a ticket to GCC Bugzilla, but will do that.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[barracuda156]

To keep issue alive, the issue with GCC and pulling in x86 atomics is still there. Possibly, caused by Boost: https://github.com/iains/darwin-toolchains-start-here/discussions/43

[barracuda156]

Closing, has been fixed with Boost: https://github.com/boostorg/multiprecision/issues/525#issuecomment-1402447979