Issues building Seastar on CentOS 8

[talawahtech]

I just wanted to quickly outline some of the challenges I had building Seastar on CentOS 8 using the default c++-dialect (gnu++20) and some possible solutions.

gcc

install-dependencies.sh will install gcc-toolset-9 by default, but you will run into a number of challenges with gcc-9 e.g. The CXX_STANDARD property on target "seastar" contained an invalid value: "2a". Using gcc-toolset-11 instead will resolve those initial issues, but then you will likely run into the issues below.

boost

The version of boost that ships with CentOS 8 (1.66) will not build successfully using the gnu++20 dialect, even though it is greater than the minimum version required by configure (1.64). The two best options are:

1. Use --cook to automatically download and build a more recent version of boost (1.77): ./configure.py --mode=release --without-tests --cook Boost

2. Switch to the gnu++17 dialect: ./configure.py --mode=release --c++-dialect gnu++17 --without-tests

[nyh]

Thanks.

If gcc-toolset-11 exists in Centos 8, I think we should change install-dependencies.sh to use it, and not the older (and officially unsupported by Seastar) gcc-toolset-9. Somebody with experience with CentOS 8 (i.e., not me) can send a patch and test it.

Regarding the Boost issue:

1. I would begin by trying (if possible...) to work around the problem: What fails in the older version Boost version? Could we #ifdef around, perhaps working a slightly less efficient or less sightly - but working - alternative on the older Boost?
2. If this is not possible, configure.py could detect this problem (perhaps by compiling a small test program, or as a last resort checking version numbers), and tell the user that to proceed you must pass either --cook Boost or --c++dialect gnu++17.

[talawahtech]

Hi @nyh,

Here is an example of one the many errors generated using gcc-toolset-11 on CentOS 8 with the provided version of boost. Let me know if you need the complete output and I can go back and reproduce it.

[14/149] Building CXX object CMakeFiles/seastar.dir/src/core/reactor_backend.cc.o
FAILED: CMakeFiles/seastar.dir/src/core/reactor_backend.cc.o
/opt/rh/gcc-toolset-11/root/usr/bin/g++ -DFMT_LOCALE -DFMT_SHARED -DSEASTAR_API_LEVEL=6 -DSEASTAR_DEFERRED_ACTION_REQUIRE_NOEXCEPT -DSEASTAR_HAS_MEMBARRIER -DSEASTAR_HAVE_ASAN_FIBER_SUPPORT -DSEASTAR_HAVE_HWLOC -DSEASTAR_HAVE_LZ4_COMPRESS_DEFAULT -DSEASTAR_HAVE_NUMA -DSEASTAR_SCHEDULING_GROUPS_COUNT=16 -I../../include -Igen/include -I../../src -Igen/src -O2 -g -DNDEBUG -std=gnu++20 -U_FORTIFY_SOURCE -Wno-maybe-uninitialized -DSEASTAR_SSTRING -Wno-error=unused-result -fvisibility=hidden -UNDEBUG -Wall -Werror -Wno-array-bounds -Wno-error=deprecated-declarations -gz -std=gnu++2a -MD -MT CMakeFiles/seastar.dir/src/core/reactor_backend.cc.o -MF CMakeFiles/seastar.dir/src/core/reactor_backend.cc.o.d -o CMakeFiles/seastar.dir/src/core/reactor_backend.cc.o -c ../../src/core/reactor_backend.cc
In file included from /usr/include/boost/lockfree/spsc_queue.hpp:28,
                 from ../../src/core/syscall_work_queue.hh:29,
                 from ../../src/core/thread_pool.hh:24,
                 from ../../src/core/reactor_backend.cc:22:
/usr/include/boost/lockfree/detail/parameter.hpp: In instantiation of ‘struct boost::lockfree::detail::extract_allocator<boost::parameter::aux::arg_list<boost::lockfree::capacity<128>, boost::parameter::aux::empty_arg_list>, seastar::syscall_work_queue::work_item*>’:
/usr/include/boost/lockfree/spsc_queue.hpp:650:48:   required from ‘struct boost::lockfree::detail::make_ringbuffer<seastar::syscall_work_queue::work_item*, boost::lockfree::capacity<128> >’
/usr/include/boost/lockfree/spsc_queue.hpp:687:7:   required from ‘class boost::lockfree::spsc_queue<seastar::syscall_work_queue::work_item*, boost::lockfree::capacity<128> >’
../../src/core/syscall_work_queue.hh:38:14:   required from here
/usr/include/boost/lockfree/detail/parameter.hpp:57:63: error: no class template named ‘rebind’ in ‘boost::lockfree::detail::extract_allocator<boost::parameter::aux::arg_list<boost::lockfree::capacity<128>, boost::parameter::aux::empty_arg_list>, seastar::syscall_work_queue::work_item*>::allocator_arg’ {aka ‘class std::allocator<seastar::syscall_work_queue::work_item*>’}
   57 |     typedef typename allocator_arg::template rebind<T>::other type;
      |                                                               ^~~~
/usr/include/boost/lockfree/detail/parameter.hpp: In instantiation of ‘struct boost::lockfree::detail::extract_allocator<boost::parameter::aux::arg_list<boost::lockfree::capacity<128>, boost::parameter::aux::empty_arg_list>, seastar::smp_message_queue::work_item*>’:
/usr/include/boost/lockfree/spsc_queue.hpp:650:48:   required from ‘struct boost::lockfree::detail::make_ringbuffer<seastar::smp_message_queue::work_item*, boost::lockfree::capacity<128> >’
/usr/include/boost/lockfree/spsc_queue.hpp:687:7:   required from ‘class boost::lockfree::spsc_queue<seastar::smp_message_queue::work_item*, boost::lockfree::capacity<128> >’
../../include/seastar/core/smp.hh:181:40:   required from here
/usr/include/boost/lockfree/detail/parameter.hpp:57:63: error: no class template named ‘rebind’ in ‘boost::lockfree::detail::extract_allocator<boost::parameter::aux::arg_list<boost::lockfree::capacity<128>, boost::parameter::aux::empty_arg_list>, seastar::smp_message_queue::work_item*>::allocator_arg’ {aka ‘class std::allocator<seastar::smp_message_queue::work_item*>’}

[nyh]

@talawahtech unfortunately it's hard for me to make sense of these error messages. I'm not very surprised at all that older versions of Boost did not compile on C++20 - both because of changes to the language (despite attempts to make the language backward-compatible...) and errors in the code which were "forgiven" by earlier standards. But what I am surprised about is that a Linux distribution (in this case CentOS 8) which is inconsistent in the sense that it includes both gcc 11 and a Boost version that can't be used on this gcc version. Of course what I'm saying here is more a rant about CentOS and less a solution :-(

From browsing the web (e.g., see https://centos.pkgs.org/8/epel-x86_64/, I don't use CentOS myself to test...) I see suggestions that an optional package "boost169" is available as a replacement for "boost", using version 1.69 instead of 1.66. Can you please try to see maybe that one is new enough to solve the C++20 issues?

[talawahtech]

@nyh I just tried it, and boost169 builds successfully! I had pass in some additional environment variables to get cmake to pick up the correct directories though. This is how I called configure:

BOOST_INCLUDEDIR=/usr/include/boost169 BOOST_LIBRARYDIR=/usr/lib64/boost169 ./configure.py --mode=release --without-tests

[nyh]

@talawahtech thanks for checking this.

So in my view (but others' opinions might differ), the best solution on CentOS 8 is:

1. Have install-dependencies.sh install gcc-toolset-11 and boost169.
2. Have configure.py detect this installations. It's better not start by looking for boost169 (this will not be a good choice a few years from now) - and rather have configure.py try to compile a simple example program that on CentOS fails without boost169, and only if this fails, retry with boost169. Moreover, if the user explicitly asks for c++17, configure.py will not detect the need for boost169 and continue to use the older, default, boost. In any case, I don't want the user to have to pass in BOOST_INCLUDEDIR/BOOST_LIBRARYDIR on their own, that's ugly and can be determined automatically.

I think the other two options you proposed earlier are less optimal: "cook" is always the wrong choice if a distribution provides some package, and forcing c++17 is not a bad option, but allowing also the better c++20 (and defaulting to it), is even better.