foundationdb
foundationdb copied to clipboard
apple
build fails on AAch64, Fedora 33
(base) [jw@cn06 build-it]$ ninja -v -j48 ... -- [download 100% complete] -- verifying file... file='/data/jw/apple/foundationdb/build-it/boost_targetProject-prefix/src/boost_1_72_0.tar.bz2' -- Downloading... done -- extracting... src='/data/jw/apple/foundationdb/build-it/boost_targetProject-prefix/src/boost_1_72_0.tar.bz2' dst='/data/jw/apple/foundationdb/build-it/boost_targetProject-prefix/src/boost_targetProject' -- extracting... [tar xfz] -- extracting... [analysis] -- extracting... [rename] -- extracting... [clean up] -- extracting... done [348/854] /usr/bin/cc -DBOOST_ERROR_CODE_HEADER_ONLY -DBOOST_SYSTEM_NO_DEPRECATED -DNO_INTELLISENSE -DSSD_ROCKSDB_EXPERIMENTAL -I../ -I. -O2 -DNDEBUG -DCMAKE_BUILD -ggdb -fno-omit-frame-pointer -Werror -Wno-pragmas -Wno-attributes -Wno-error=format -Wunused-variable -Wno-deprecated -fvisibility=hidden -Wreturn-type -fPIC -march=armv8.2-a+crc+simd -w -std=gnu11 -MD -MT fdbserver/CMakeFiles/fdb_sqlite.dir/sqlite/sqlite3.amalgamation.c.o -MF fdbserver/CMakeFiles/fdb_sqlite.dir/sqlite/sqlite3.amalgamation.c.o.d -o fdbserver/CMakeFiles/fdb_sqlite.dir/sqlite/sqlite3.amalgamation.c.o -c ../fdbserver/sqlite/sqlite3.amalgamation.c ninja: build stopped: subcommand failed. (base) [jw@cn06 build-it]$
Building with -j48 would almost certainly cause an OOM. Can you try building with lower parallelism? ~3Gb per process should be enough, but you might even need more memory.
It even fails with -j4, about 275 steps are left to build. Is there no option to manage the process by limiting the RAM usage? I worked on hundreds of different softwares, but this is the most extreme case I ever came across. Why does it need so much RAM just for building the software? Thanks!
Why does it need so much RAM just for building the software?
Basically because of heavy use of c++ template metaprogramming. FWIW recent versions of clang seem better about memory usage.
It even fails with -j4
That makes me wonder if it isn't memory usage after all. Does it work with -j1? Is there anything indicating an actual compiler error (i.e. not the compiler process getting killed because of OOM) ? Can you monitor memory usage and see if it's actually running out?
I just built the master on aarch64 (AWS Graviton2) without any issues.
One thing to note is that we're using foundationdb/build docker container which comes with CentOS7 + gcc8.
We have not tested on other OSes or compilers.
I'm having progress, e.g. a library was missing. A ran the test and get 2 errors:
(base) [jw@cn05 build-it]$ ninja -v -j$(nproc) test [0/1] cd /data/jw/apple/foundationdb/build-it && /usr/bin/ctest --force-new-ctest-process Test project /data/jw/apple/foundationdb/build-it Start 1: SimpleExternalTest ... 69/133 Test #69: restarting/from_7.0.0/ConfigureTestRestart-1 ....................***Failed ... sec ... 90/133 Test #90: slow/DDBalanceAndRemoveStatus ...................................***Failed ... sec ...
98% tests passed, 2 tests failed out of 133
Label Time Summary: ...
Total Test time (real) = ... sec
The following tests FAILED: 69 - restarting/from_7.0.0/ConfigureTestRestart-1 (Failed) 90 - slow/DDBalanceAndRemoveStatus (Failed) Errors while running CTest FAILED: CMakeFiles/test.util cd /data/jw/apple/foundationdb/build-it && /usr/bin/ctest --force-new-ctest-process ninja: build stopped: subcommand failed. (base) [jw@cn05 build-it]$
