Build failure in VMs: Error: no such instruction: `tpause %ecx'

[mcmillan03]

We are pulling the latest on master branch (but have tried other recent tagged versions) as a submodule to another project. The work is being done in an Ubuntu 18.04 LTS VM in VMWare Workstation on a Windows x86 machine and in VMWare Fusion on an x86 macBook. We are using gcc/g++ 11.1.0.

We get the following error message on both systems (there is no such error when not using a VM):

[100%] Linking CXX shared library ../../../../gnu_11.1_cxx20_64_release/libtbb.so
cd /home/smcmillan/github/gbtl/build_lums_cmake/external/oneTBB/src/tbb && /usr/bin/cmake -E cmake_link_script CMakeFiles/tbb.dir/link.txt --verbose=true
/usr/bin/c++ -fPIC -O3 -DNDEBUG -flto -fno-fat-lto-objects -Wl,--version-script=/home/smcmillan/github/gbtl/external/oneTBB/src/tbb/def/lin64-tbb.def -shared -Wl,-soname,libtbb.so.12 -o ../../../../gnu_11.1_cxx20_64_release/libtbb.so.12.7 CMakeFiles/tbb.dir/address_waiter.cpp.o CMakeFiles/tbb.dir/allocator.cpp.o CMakeFiles/tbb.dir/arena.cpp.o CMakeFiles/tbb.dir/arena_slot.cpp.o CMakeFiles/tbb.dir/concurrent_bounded_queue.cpp.o CMakeFiles/tbb.dir/dynamic_link.cpp.o CMakeFiles/tbb.dir/exception.cpp.o CMakeFiles/tbb.dir/governor.cpp.o CMakeFiles/tbb.dir/global_control.cpp.o CMakeFiles/tbb.dir/itt_notify.cpp.o CMakeFiles/tbb.dir/main.cpp.o CMakeFiles/tbb.dir/market.cpp.o CMakeFiles/tbb.dir/misc.cpp.o CMakeFiles/tbb.dir/misc_ex.cpp.o CMakeFiles/tbb.dir/observer_proxy.cpp.o CMakeFiles/tbb.dir/parallel_pipeline.cpp.o CMakeFiles/tbb.dir/private_server.cpp.o CMakeFiles/tbb.dir/profiling.cpp.o CMakeFiles/tbb.dir/rml_tbb.cpp.o CMakeFiles/tbb.dir/rtm_mutex.cpp.o CMakeFiles/tbb.dir/rtm_rw_mutex.cpp.o CMakeFiles/tbb.dir/semaphore.cpp.o CMakeFiles/tbb.dir/small_object_pool.cpp.o CMakeFiles/tbb.dir/task.cpp.o CMakeFiles/tbb.dir/task_dispatcher.cpp.o CMakeFiles/tbb.dir/task_group_context.cpp.o CMakeFiles/tbb.dir/version.cpp.o CMakeFiles/tbb.dir/queuing_rw_mutex.cpp.o  -ldl -pthread
/tmp/cclIwmtb.s: Assembler messages:
/tmp/cclIwmtb.s:16988: Error: no such instruction: `tpause %ecx'
lto-wrapper: fatal error: /usr/bin/c++ returned 1 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
external/oneTBB/src/tbb/CMakeFiles/tbb.dir/build.make:529: recipe for target 'gnu_11.1_cxx20_64_release/libtbb.so.12.7' failed
make[3]: *** [gnu_11.1_cxx20_64_release/libtbb.so.12.7] Error 1

To workaround this problem, we editted include/oneapi/tbb/detail/_config.h in TBB and changed lines around 332 to look like this

#if 0
#define __TBB_WAITPKG_INTRINSICS_PRESENT ((__INTEL_COMPILER >= 1900 || __TBB_GCC_VERSION >= 110000 || __TBB_CLANG_VERSION >= 120000) \
                                         && (_WIN32 || _WIN64 || __unix__ || __APPLE__) && (__TBB_x86_32 || __TBB_x86_64) && !__ANDROID__)
#else
#define __TBB_WAITPKG_INTRINSICS_PRESENT 0
#endif

Basically, to set __TBB_WAITPKG_INTRINSICS_PRESENT to be false.

I am sure there is a better way, but what is it?

[kboyarinov]

@mcmillan03, thank you for reporting the issue. I will investigate why such failure takes place and return back once some results would be obtained.

[kboyarinov]

@mcmillan03, Unfortunately, currently we don't have the environment that exactly matches with the one you described. Similar configuration on Oracle VirtualBox did not show the issue. But according to the log you provided, the error was occurred during the linkage stage with the enabled LTO. Could you please try to disable the LTO by re-running cmake with -DTBB_ENABLE_IPO=OFF option? Thank you in advance.

[mcmillan03]

I tried this and got the following...tail of compilation:

[ 90%] Building CXX object external/oneTBB/src/tbb/CMakeFiles/tbb.dir/queuing_rw_mutex.cpp.o
cd /home/smcmillan/github/gbtl/build_lums_cmake/external/oneTBB/src/tbb && /usr/bin/c++ -D__TBB_BUILD -D__TBB_USE_ITT_NOTIFY -I/home/smcmillan/github/gbtl/external/oneTBB/src/tbb/../../include -DTBB_ENABLE_IPO=OFF -O3 -DNDEBUG -flto=auto -fno-fat-lto-objects -fPIC -flifetime-dse=1 -Wall -Wextra -Werror -Wfatal-errors -mrtm -mwaitpkg -pthread -std=c++20 -MD -MT external/oneTBB/src/tbb/CMakeFiles/tbb.dir/queuing_rw_mutex.cpp.o -MF CMakeFiles/tbb.dir/queuing_rw_mutex.cpp.o.d -o CMakeFiles/tbb.dir/queuing_rw_mutex.cpp.o -c /home/smcmillan/github/gbtl/external/oneTBB/src/tbb/queuing_rw_mutex.cpp
[ 90%] Linking CXX shared library ../../../../gnu_11.1_cxx20_64_release/libtbb.so
cd /home/smcmillan/github/gbtl/build_lums_cmake/external/oneTBB/src/tbb && /usr/bin/cmake -E cmake_link_script CMakeFiles/tbb.dir/link.txt --verbose=1
/usr/bin/c++ -fPIC  -DTBB_ENABLE_IPO=OFF -O3 -DNDEBUG -flto=auto -fno-fat-lto-objects -Wl,--version-script=/home/smcmillan/github/gbtl/external/oneTBB/src/tbb/def/lin64-tbb.def -shared -Wl,-soname,libtbb.so.12 -o ../../../../gnu_11.1_cxx20_64_release/libtbb.so.12.7 CMakeFiles/tbb.dir/address_waiter.cpp.o CMakeFiles/tbb.dir/allocator.cpp.o CMakeFiles/tbb.dir/arena.cpp.o CMakeFiles/tbb.dir/arena_slot.cpp.o CMakeFiles/tbb.dir/concurrent_bounded_queue.cpp.o CMakeFiles/tbb.dir/dynamic_link.cpp.o CMakeFiles/tbb.dir/exception.cpp.o CMakeFiles/tbb.dir/governor.cpp.o CMakeFiles/tbb.dir/global_control.cpp.o CMakeFiles/tbb.dir/itt_notify.cpp.o CMakeFiles/tbb.dir/main.cpp.o CMakeFiles/tbb.dir/market.cpp.o CMakeFiles/tbb.dir/misc.cpp.o CMakeFiles/tbb.dir/misc_ex.cpp.o CMakeFiles/tbb.dir/observer_proxy.cpp.o CMakeFiles/tbb.dir/parallel_pipeline.cpp.o CMakeFiles/tbb.dir/private_server.cpp.o CMakeFiles/tbb.dir/profiling.cpp.o CMakeFiles/tbb.dir/rml_tbb.cpp.o CMakeFiles/tbb.dir/rtm_mutex.cpp.o CMakeFiles/tbb.dir/rtm_rw_mutex.cpp.o CMakeFiles/tbb.dir/semaphore.cpp.o CMakeFiles/tbb.dir/small_object_pool.cpp.o CMakeFiles/tbb.dir/task.cpp.o CMakeFiles/tbb.dir/task_dispatcher.cpp.o CMakeFiles/tbb.dir/task_group_context.cpp.o CMakeFiles/tbb.dir/version.cpp.o CMakeFiles/tbb.dir/queuing_rw_mutex.cpp.o  -ldl -pthread 
/tmp/ccXIbG4k.s: Assembler messages:
/tmp/ccXIbG4k.s:9166: Error: no such instruction: `tpause %ecx'
/tmp/ccXIbG4k.s:10162: Error: no such instruction: `tpause %ecx'
make[4]: *** [/tmp/cc9ICkYG.ltrans2.ltrans.o] Error 1
make[4]: *** Waiting for unfinished jobs....
/tmp/ccUJHRYl.s: Assembler messages:
/tmp/ccUJHRYl.s:16988: Error: no such instruction: `tpause %ecx'
make[4]: *** [/tmp/cc9ICkYG.ltrans0.ltrans.o] Error 1
lto-wrapper: fatal error: make returned 2 exit status
compilation terminated.
/usr/bin/ld: error: lto-wrapper failed
collect2: error: ld returned 1 exit status
external/oneTBB/src/tbb/CMakeFiles/tbb.dir/build.make:529: recipe for target 'gnu_11.1_cxx20_64_release/libtbb.so.12.7' failed
make[3]: *** [gnu_11.1_cxx20_64_release/libtbb.so.12.7] Error 1

[markdewing]

I also saw this issue building TBB branch v2021.4.0 with gcc 11.1 on Ubuntu 18.04. The binutils version is 2.30. I worked around it by building a newer copy of binutils.

[phprus]

tpause %ecx supported by binutils >= 2.31.1

binutils commit: https://github.com/bminor/binutils-gdb/commit/ae1d38437284b31d5a1c

[mcmillan03]

"ld -v" reports binutils 2.30. Will try building a newer version next week.

[mcmillan03]

Update: I am still trying to figure out how to upgrade binutils beyond version 2.30 on ubuntu 18.04.

[mcmillan03]

The only way I found to upgrade binutils was to upgrade my Ubuntu to 20.04 LTS. The code the compiled without the modification described above.

[isaevil]

@mcmillan03 is upgrading binutils an acceptable solution of your issue?

[mcmillan03]

It is an acceptable workaround for now. Is there a decision not to support binutils <= version 2.30?

[isaevil]

There wasn't such decision but I actually don't really see a straightforward way how to check binutils version in CMake. I would suggest just upgrading binutils version if you are going to use gcc-11.

[isaevil]

@mcmillan03 I would like to advise to stick with proposed workaround (updating binutils). Closing this issue. Feel free to reopen if any questions are left.