build error - undefined reference to `facebook::strings::ByteSinkBuffer::flush()'

[donghekang]

trafficstars

Hi all,

Building the dwrf tests throws error because it cannot find the definition of the ByteSinkBuffer::flush function. I don't find the definition in the code base. Does anyone know why this happens and how to fix?

Here is an example error log:

[795/1208] Linking CXX executable velox/dwio/dwrf/test/velox_dwrf_writer_extended_test FAILED: velox/dwio/dwrf/test/velox_dwrf_writer_extended_test : && /home/kang.1002/app/gcc-12.2.0/bin/c++ -I/home/kang.1002/app/re2/usr/local/include -L/home/kang.1002/app/re2/usr/local/lib -lre2 -I/home/kang.1002/app/protobuf-3.21.3/include -L/home/kang.1002/app/protobuf-3.21.3/lib -lprotobuf -lprotoc -mavx2 -mfma -mavx -mf16c -mlzcnt -std=c++17 -mbmi2 -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -Wall -Wextra -Wno-unused -Wno-unused-parameter -Wno-sign-compare -Wno-ignored-qualifiers -Wno-implicit-fallthrough -Wno-empty-body -Wno-class-memaccess -Wno-comment -Wno-int-in-bool-context -Wno-redundant-move -Wno-array-bounds -Wno-maybe-uninitialized -Wno-unused-result -Wno-format-overflow -Wno-strict-aliasing -Wno-type-limits -O3 -DNDEBUG -Wl,-export-dynamic velox/dwio/dwrf/test/CMakeFiles/velox_dwrf_writer_extended_test.dir/WriterExtendedTests.cpp.o -o velox/dwio/dwrf/test/velox_dwrf_writer_extended_test -Wl,-rpath,/home/kang.1002/app/libevent-2.1.12/lib:/home/kang.1002/app/lz4-1.9.4/lib:/home/kang.1002/app/zstd-1.5.2/lib:/home/kang.1002/app/glog/lib:/home/kang.1002/app/boost_1_75_0/lib velox/dwio/dwrf/test/utils/libvelox_dwrf_test_utils.a velox/common/caching/libvelox_caching.a velox/dwio/common/libvelox_dwio_common.a velox/dwio/common/exception/libvelox_dwio_common_exception.a velox/dwio/type/fbhive/libvelox_dwio_type_fbhive.a velox/common/encode/libvelox_encode.a velox/common/base/libvelox_exception.a velox/common/memory/libvelox_memory.a velox/common/process/libvelox_process.a velox/common/serialization/libvelox_serialization.a velox/type/libvelox_type.a velox/vector/libvelox_vector.a _deps/folly-build/libfolly.a /home/kang.1002/app/double_conversion/usr/local/lib/libdouble-conversion.a /home/kang.1002/app/libevent-2.1.12/lib/libevent.so /home/kang.1002/app/snappy-1.1.8/lib/libsnappy.a -ldl /home/kang.1002/app/fmt-8.0.0/lib/libfmt.a /home/kang.1002/app/fmt-8.0.0/lib/libfmt.a _deps/folly-build/libfolly.a /home/kang.1002/app/double_conversion/usr/local/lib/libdouble-conversion.a /home/kang.1002/app/libevent-2.1.12/lib/libevent.so /home/kang.1002/app/snappy-1.1.8/lib/libsnappy.a -ldl /home/kang.1002/app/fmt-8.0.0/lib/libfmt.a /home/kang.1002/app/fmt-8.0.0/lib/libfmt.a /home/kang.1002/app/lz4-1.9.4/lib/liblz4.so /home/kang.1002/app/lzo-2.10/lib/liblzo2.a /home/kang.1002/app/zstd-1.5.2/lib/libzstd.so /usr/lib/x86_64-linux-gnu/libz.so velox/dwio/common/tests/utils/libvelox_dwio_common_test_utils.a velox/dwio/dwrf/proto/libvelox_dwio_dwrf_proto.a velox/dwio/dwrf/reader/libvelox_dwio_dwrf_reader.a velox/dwio/dwrf/writer/libvelox_dwio_dwrf_writer.a lib/libgtest.a lib/libgtest_main.a lib/libgmock.a /home/kang.1002/app/glog/lib/libglog.so -lstdc++fs /home/kang.1002/app/lzo-2.10/lib/liblzo2.a velox/dwio/dwrf/common/libvelox_dwio_dwrf_common.a velox/dwio/dwrf/utils/libvelox_dwio_dwrf_utils.a velox/dwio/dwrf/proto/libvelox_dwio_dwrf_proto.a /home/kang.1002/app/protobuf-3.21.3/lib/libprotobuf.so velox/dwio/common/libvelox_dwio_common.a velox/common/caching/libvelox_caching.a velox/common/file/libvelox_file.a velox/dwio/common/compression/libvelox_dwio_common_compression.a velox/dwio/common/encryption/libvelox_dwio_common_encryption.a velox/dwio/common/exception/libvelox_dwio_common_exception.a velox/vector/libvelox_vector.a velox/type/libvelox_type.a velox/common/encode/libvelox_encode.a velox/common/serialization/libvelox_serialization.a velox/type/tz/libvelox_type_tz.a velox/external/date/libvelox_external_date.a velox/common/time/libvelox_time.a velox/buffer/libvelox_buffer.a velox/common/memory/libvelox_memory.a velox/common/base/libvelox_common_base.a velox/common/base/libvelox_exception.a velox/common/process/libvelox_process.a /home/kang.1002/app/glog/lib/libglog.so /usr/lib/x86_64-linux-gnu/libgflags.so.2.2.1 _deps/folly-build/libfolly.a /usr/lib/x86_64-linux-gnu/libgflags.so.2.2.1 /home/kang.1002/app/fmt-8.0.0/lib/libfmt.a /home/kang.1002/app/glog/lib/libglog.so /usr/lib/x86_64-linux-gnu/libz.so /usr/lib/x86_64-linux-gnu/libssl.so /usr/lib/x86_64-linux-gnu/libcrypto.so /home/kang.1002/app/lz4-1.9.4/lib/liblz4.so /home/kang.1002/app/zstd-1.5.2/lib/libzstd.so /home/kang.1002/app/boost_1_75_0/lib/libboost_program_options.so.1.75.0 /home/kang.1002/app/boost_1_75_0/lib/libboost_context.so.1.75.0 /home/kang.1002/app/boost_1_75_0/lib/libboost_filesystem.so.1.75.0 /home/kang.1002/app/boost_1_75_0/lib/libboost_regex.so.1.75.0 -licudata -licui18n -licuuc /home/kang.1002/app/boost_1_75_0/lib/libboost_thread.so.1.75.0 /home/kang.1002/app/boost_1_75_0/lib/libboost_system.so.1.75.0 /home/kang.1002/app/boost_1_75_0/lib/libboost_date_time.so.1.75.0 /home/kang.1002/app/boost_1_75_0/lib/libboost_atomic.so.1.75.0 /home/kang.1002/app/double_conversion/usr/local/lib/libdouble-conversion.a /home/kang.1002/app/libevent-2.1.12/lib/libevent.so /home/kang.1002/app/snappy-1.1.8/lib/libsnappy.a -ldl /home/kang.1002/app/fmt-8.0.0/lib/libfmt.a lib/libgtest.a -lpthread && : velox/dwio/dwrf/reader/libvelox_dwio_dwrf_reader.a(SelectiveTimestampColumnReader.cpp.o): In function facebook::strings::ByteSinkBuffer::sync()': SelectiveTimestampColumnReader.cpp:(.text._ZN8facebook7strings14ByteSinkBuffer4syncEv[_ZN8facebook7strings14ByteSinkBuffer4syncEv]+0x5): undefined reference to facebook::strings::ByteSinkBuffer::flush()' collect2: error: ld returned 1 exit status

[kagamiori]

I haven't seen this error before. ByteSinkBuffer::flush() should have been defined in velox/common/strings/ByteStream.h.

cc: @kgpai

[mbasmanova]

@donghekang Thank you for reporting this issue. Which platform do you use for building?

[donghekang]

I build it on Ubuntu 18.04.6 LTS (Bionic Beaver) . I don't have the root privilege so that I installed all dependence by myself. Here is how I compile it:

EXTRA_CMAKE_FLAGS="-DCMAKE_PREFIX_PATH="/home/kang.1002/app/boost_1_75_0/;/home/kang.1002/app/glog;/home/kang.1002/app/re2/usr/local/;/home/kang.1002/app/double_conversion/usr/local;/home/kang.1002/app/libevent-2.1.12;/home/kang.1002/app/protobuf-3.21.3;/home/kang.1002/app/fmt-8.0.0;/home/kang.1002/app/lz4-1.9.4;/home/kang.1002/app/lzo-2.10;/home/kang.1002/app/snappy-1.1.8;/home/kang.1002/app/zstd-1.5.2" -DCMAKE_CXX_FLAGS="-I/home/kang.1002/app/re2/usr/local/include -L/home/kang.1002/app/re2/usr/local/lib -lre2 -I/home/kang.1002/app/protobuf-3.21.3/include -L/home/kang.1002/app/protobuf-3.21.3/lib -lprotobuf -lprotoc"" TREAT_WARNINGS_AS_ERRORS=0 make

[dillon-cullinan]

Hello, I ran into a similar issue. I was able to solve the issue by downgrading to GCC 10.

[assignUser]

Closed as stale. Feel free to reopen if the same error persists or open a new issue if there are additional issues.