AIL-framework icon indicating copy to clipboard operation
AIL-framework copied to clipboard

ARDB install error

Open John33000 opened this issue 5 years ago • 10 comments

Hi, I following your step for installing CIRCL. But I have error with the :

./installing_deps.sh

I have this error at the end of installation :

til/status.cc: In static member function ‘static const char* rocksdb::Status::CopyState(const char*)’:
util/status.cc:28:15: error: ‘char* strncpy(char*, const char*, size_t)’ output truncated before terminating nul copying as many bytes from a string as its length [-Werror=stringop-truncation]
   std::strncpy(result, state, cch - 1);
   ~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~
util/status.cc:19:18: note: length computed here
       std::strlen(state) + 1; // +1 for the null terminator
       ~~~~~~~~~~~^~~~~~~
cc1plus: all warnings being treated as errors
make[2]: *** [Makefile:1879: util/status.o] Error 1
make[2] : on quitte le répertoire « /root/AIL-framework/ardb/deps/rocksdb-5.14.2 »
make[1]: *** [Makefile:401: /root/AIL-framework/ardb/src/../deps/rocksdb-5.14.2/librocksdb.a] Error 2
make[1] : on quitte le répertoire « /root/AIL-framework/ardb/src »
make: *** [Makefile:4: all] Error 2
 

Best Regards

John33000 avatar Jul 29 '19 13:07 John33000

Hi @John33000 !

This issue is related to ARDB, which system are you using ?

Terrtia avatar Jul 30 '19 12:07 Terrtia

I have the identical error, running minimal Debian 10

yannw avatar Jul 30 '19 15:07 yannw

Hi, Like yannw, Debian 10

John33000 avatar Jul 30 '19 15:07 John33000

same over here Linux ail 5.0.0-23-generic #24-Ubuntu SMP Mon Jul 29 15:36:44 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux / Ubuntu 19.04

M-ir-sta avatar Aug 06 '19 08:08 M-ir-sta

EDIT: https://github.com/yinqiwen/ardb/issues/455 solve my issue: changing compilation to gcc-7 and g++-7 in the make file from ardb.

yeah not exactly the same

CXX statistics.o

statistics.cpp: In member function ‘virtual void ardb::CostTrack::Dump(void ()(const string&, void), void*)’: statistics.cpp:120:44: error: ‘%s’ directive output may be truncated writing up to 1023 bytes into a region of size 1013 [-Werror=format-truncation=] snprintf(tmp, sizeof(tmp) - 1, "coststat_%s_%s:calls=%" PRIu64 ",costs=%" PRIu64 ",cost_per_call=%" PRIu64 ",percents=%.4f%%", name.c_str(), range, recs[i].count, ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ~~~~~ statistics.cpp:120:44: note: using the range [0, 18446744073709551615] for directive argument statistics.cpp:120:44: note: assuming directive output of 6 bytes In file included from /usr/include/stdio.h:867, from common/common.hpp:165, from statistics.hpp:32, from statistics.cpp:29: /usr/include/x86_64-linux-gnu/bits/stdio2.h:67:35: note: ‘__builtin___snprintf_chk’ output 57 or more bytes (assuming 1083) into a destination of size 1023 return __builtin___snprintf_chk (__s, __n, __USE_FORTIFY_LEVEL - 1, ~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ __bos (__s), __fmt, __va_arg_pack ()); ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ cc1plus: all warnings being treated as errors make[1]: *** [Makefile:177: statistics.o] Error 1 make[1]: Leaving directory '/root/AIL-framework/ardb/src' make: *** [Makefile:4: all] Error 2

M-ir-sta avatar Aug 06 '19 08:08 M-ir-sta

We still have the same problem and are unable to install the rocksdb. Applied this pull request without any luck -> https://github.com/CIRCL/AIL-framework/pull/378/commits/be1bb1074d51e9cf3af3fee921cc83ea5884d0c0

Switched also from the Rocksdb version 5.14.2 to 5.15.10 in ardb/src/Makefile.

We use Debian OS that comes with gcc 9 per default. Also installed gcc 7 and gcc 8. Tried also CXX=g++-7 CC=gcc-7 and CXX=g++-8 CC=gcc-8 Still with the same error. Is there any other workaround to get AIL up and running ?

ITSEC-DACHSER avatar Dec 05 '19 13:12 ITSEC-DACHSER

I have the exact same error at the end of the install : [Makefile:4: all] Error 2. I understand that error comes from ARDB, but how to solve it? Any idea? I read a topic about changing compilation but I don't understand how to do that. Thanks for your help!

DocKali avatar Jan 15 '20 20:01 DocKali

Hello i have the same error, i tried to bum RocksDB version and downgrade gcc version but dont work.

sorrow@sorrow:~/AIL-framework/ardb/src$ uname -a Linux sorrow 5.4.0-kali4-amd64 #1 SMP Debian 5.4.19-1kali1 (2020-02-17) x86_64 GNU/Linux

Error (click here)

Error logs

>>>>> Building ROCKSDB
make[2] : on entre dans le répertoire « /home/sorrow/AIL-framework/ardb/deps/rocksdb-5.15.10 »
  GEN      util/build_version.cc
  GEN      util/build_version.cc
  CC       db/builder.o
In file included from ./db/range_del_aggregator.h:17,
                 from ./db/memtable.h:19,
                 from ./db/memtable_list.h:17,
                 from ./db/column_family.h:17,
                 from ./db/version_set.h:31,
                 from ./db/compaction.h:11,
                 from ./db/compaction_iterator.h:12,
                 from db/builder.cc:16:
./db/version_edit.h: In constructor ‘rocksdb::FdWithKeyRange::FdWithKeyRange(rocksdb::FileDescriptor, rocksdb::Slice, rocksdb::Slice, rocksdb::FileMetaData*)’:
./db/version_edit.h:172:33: error: implicitly-declared ‘constexpr rocksdb::FileDescriptor::FileDescriptor(const rocksdb::FileDescriptor&)’ is deprecated [-Werror=deprecated-copy]
  172 |         largest_key(_largest_key) {}
      |                                 ^
./db/version_edit.h:47:19: note: because ‘rocksdb::FileDescriptor’ has user-provided ‘rocksdb::FileDescriptor& rocksdb::FileDescriptor::operator=(const rocksdb::FileDescriptor&)’
   47 |   FileDescriptor& operator=(const FileDescriptor& fd) {
      |                   ^~~~~~~~
./db/version_edit.h: In instantiation of ‘constexpr std::pair<_T1, _T2>::pair(_U1&&, _U2&&) [with _U1 = int&; _U2 = rocksdb::FileMetaData; typename std::enable_if<(std::_PCC<true, _T1, _T2>::_MoveConstructiblePair<_U1, _U2>() && std::_PCC<true, _T1, _T2>::_ImplicitlyMoveConvertiblePair<_U1, _U2>()), bool>::type <anonymous> = true; _T1 = int; _T2 = rocksdb::FileMetaData]’:
/usr/include/c++/9/ext/new_allocator.h:147:4:   required from ‘void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = std::pair<int, rocksdb::FileMetaData>; _Args = {int&, rocksdb::FileMetaData}; _Tp = std::pair<int, rocksdb::FileMetaData>]’
/usr/include/c++/9/bits/alloc_traits.h:484:4:   required from ‘static void std::allocator_traits<std::allocator<_CharT> >::construct(std::allocator_traits<std::allocator<_CharT> >::allocator_type&, _Up*, _Args&& ...) [with _Up = std::pair<int, rocksdb::FileMetaData>; _Args = {int&, rocksdb::FileMetaData}; _Tp = std::pair<int, rocksdb::FileMetaData>; std::allocator_traits<std::allocator<_CharT> >::allocator_type = std::allocator<std::pair<int, rocksdb::FileMetaData> >]’
/usr/include/c++/9/bits/vector.tcc:115:30:   required from ‘void std::vector<_Tp, _Alloc>::emplace_back(_Args&& ...) [with _Args = {int&, rocksdb::FileMetaData}; _Tp = std::pair<int, rocksdb::FileMetaData>; _Alloc = std::allocator<std::pair<int, rocksdb::FileMetaData> >]’
./db/version_edit.h:242:48:   required from here
./db/version_edit.h:76:8: error: implicitly-declared ‘constexpr rocksdb::FileDescriptor::FileDescriptor(const rocksdb::FileDescriptor&)’ is deprecated [-Werror=deprecated-copy]
   76 | struct FileMetaData {
      |        ^~~~~~~~~~~~
./db/version_edit.h:47:19: note: because ‘rocksdb::FileDescriptor’ has user-provided ‘rocksdb::FileDescriptor& rocksdb::FileDescriptor::operator=(const rocksdb::FileDescriptor&)’
   47 |   FileDescriptor& operator=(const FileDescriptor& fd) {
      |                   ^~~~~~~~
In file included from /usr/include/c++/9/bits/stl_algobase.h:64,
                 from /usr/include/c++/9/bits/char_traits.h:39,
                 from /usr/include/c++/9/string:40,
                 from ./db/builder.h:9,
                 from db/builder.cc:10:
/usr/include/c++/9/bits/stl_pair.h:342:64: note: synthesized method ‘rocksdb::FileMetaData::FileMetaData(rocksdb::FileMetaData&&)’ first required here
  342 |  : first(std::forward<_U1>(__x)), second(std::forward<_U2>(__y)) { }
      |                                                                ^
In file included from ./db/range_del_aggregator.h:17,
                 from ./db/memtable.h:19,
                 from ./db/memtable_list.h:17,
                 from ./db/column_family.h:17,
                 from ./db/version_set.h:31,
                 from ./db/compaction.h:11,
                 from ./db/compaction_iterator.h:12,
                 from db/builder.cc:16:
./db/version_edit.h: In instantiation of ‘constexpr std::pair<_T1, _T2>::pair(_U1&&, const _T2&) [with _U1 = int&; typename std::enable_if<std::_PCC<true, _T1, _T2>::_MoveCopyPair<true, _U1, _T2>(), bool>::type <anonymous> = true; _T1 = int; _T2 = rocksdb::FileMetaData]’:
/usr/include/c++/9/ext/new_allocator.h:147:4:   required from ‘void __gnu_cxx::new_allocator<_Tp>::construct(_Up*, _Args&& ...) [with _Up = std::pair<int, rocksdb::FileMetaData>; _Args = {int&, const rocksdb::FileMetaData&}; _Tp = std::pair<int, rocksdb::FileMetaData>]’
/usr/include/c++/9/bits/alloc_traits.h:484:4:   required from ‘static void std::allocator_traits<std::allocator<_CharT> >::construct(std::allocator_traits<std::allocator<_CharT> >::allocator_type&, _Up*, _Args&& ...) [with _Up = std::pair<int, rocksdb::FileMetaData>; _Args = {int&, const rocksdb::FileMetaData&}; _Tp = std::pair<int, rocksdb::FileMetaData>; std::allocator_traits<std::allocator<_CharT> >::allocator_type = std::allocator<std::pair<int, rocksdb::FileMetaData> >]’
/usr/include/c++/9/bits/vector.tcc:115:30:   required from ‘void std::vector<_Tp, _Alloc>::emplace_back(_Args&& ...) [with _Args = {int&, const rocksdb::FileMetaData&}; _Tp = std::pair<int, rocksdb::FileMetaData>; _Alloc = std::allocator<std::pair<int, rocksdb::FileMetaData> >]’
./db/version_edit.h:247:37:   required from here
./db/version_edit.h:76:8: error: implicitly-declared ‘constexpr rocksdb::FileDescriptor::FileDescriptor(const rocksdb::FileDescriptor&)’ is deprecated [-Werror=deprecated-copy]
   76 | struct FileMetaData {
      |        ^~~~~~~~~~~~
./db/version_edit.h:47:19: note: because ‘rocksdb::FileDescriptor’ has user-provided ‘rocksdb::FileDescriptor& rocksdb::FileDescriptor::operator=(const rocksdb::FileDescriptor&)’
   47 |   FileDescriptor& operator=(const FileDescriptor& fd) {
      |                   ^~~~~~~~
In file included from /usr/include/c++/9/bits/stl_algobase.h:64,
                 from /usr/include/c++/9/bits/char_traits.h:39,
                 from /usr/include/c++/9/string:40,
                 from ./db/builder.h:9,
                 from db/builder.cc:10:
/usr/include/c++/9/bits/stl_pair.h:312:51: note: synthesized method ‘rocksdb::FileMetaData::FileMetaData(const rocksdb::FileMetaData&)’ first required here
  312 |        : first(std::forward<_U1>(__x)), second(__y) { }
      |                                                   ^
cc1plus: all warnings being treated as errors
make[2]: *** [Makefile:1892: db/builder.o] Error 1
make[2] : on quitte le répertoire « /home/sorrow/AIL-framework/ardb/deps/rocksdb-5.15.10 »
make[1]: *** [Makefile:402: /home/sorrow/AIL-framework/ardb/src/../deps/rocksdb-5.15.10/librocksdb.a] Error 2
make[1] : on quitte le répertoire « /home/sorrow/AIL-framework/ardb/src »
make: *** [Makefile:4: all] Error 2

@Terrtia did you have some trick to pass this error ?

Sorrow81 avatar Mar 14 '20 17:03 Sorrow81

@M-ir-sta same problem here

@ITSEC-DACHSER I've tried also to switch to latest rocksdb (6.8.1) because in 5.14.2 there were a lot of compile problems but without success.

@Sorrow81 you have to switch rocksdb version on src/Makefile

AIL (and ARDB) stops on statistics.cpp

statistics.cpp: In member function ‘virtual void ardb::CostTrack::Dump(void (*)(const string&, void*), void*)’:
statistics.cpp:120:44: error: ‘%s’ directive output may be truncated writing up to 1023 bytes into a region of size 1013 [-Werror=format-truncation=]
  120 |             snprintf(tmp, sizeof(tmp) - 1, "coststat_%s_%s:calls=%" PRIu64 ",costs=%" PRIu64 ",cost_per_call=%" PRIu64 ",percents=%.4f%%", name.c_str(), range, recs[i].count,
      |                                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~                ~~~~~
statistics.cpp:120:44: note: using the range [0, 18446744073709551615] for directive argument
statistics.cpp:120:44: note: assuming directive output of 6 bytes
statistics.cpp:120:21: note: ‘snprintf’ output 57 or more bytes (assuming 1083) into a destination of size 1023
  120 |             snprintf(tmp, sizeof(tmp) - 1, "coststat_%s_%s:calls=%" PRIu64 ",costs=%" PRIu64 ",cost_per_call=%" PRIu64 ",percents=%.4f%%", name.c_str(), range, recs[i].count,
      |             ~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  121 |                     recs[i].cost, recs[i].cost / recs[i].count, (double(recs[i].count) / double(recs[0].count)) * 100);
      |                     ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1plus: all warnings being treated as errors
make[1]: *** [Makefile:177: statistics.o] Error 1

EDIT: I solved in this way:

ardb/src/Makefile:

 55 ROCKSDB_VER=6.8.1

120 CXXFLAGS=-Wall -Wno-error=class-memaccess -Wno-error=format-truncation= -g ${OPT}  -fPIC -D__STDC_FORMAT_MACROS -D__STDC_LIMIT_MACROS -DARDB_VERSION='"${ARDB_VERSION}"'
121 CCFLAGS=-Wall -Wno-error=class-memaccess -Wno-error=format-truncation= -std=gnu99 ${OPT} -fPIC -pedantic -g -D__STDC_FORMAT_MACROS -DARDB_VERSION='"${ARDB_VERSION}"'

ardb/deps/sparsehash-sparsehash-2.0.3/Makefile

 205 CFLAGS = -Wno-error=class-memaccess -g -O2
 207 CPPFLAGS = -Wno-error=class-memaccess
 211 CXXFLAGS = -Wno-error=class-memaccess -g -O2

installing_deps.sh after line 89 I've added:

. ./AILENV/bin/activate

I'm sorry if something is redundant....but I'm not a dev!!

lantuin avatar May 06 '20 06:05 lantuin

On my side, I had to use CXXFLAGS='-Wno-error=deprecated-copy -Wno-error=pessimizing-move -Wno-error=class-memaccess' working after in my ansible role and test https://travis-ci.org/github/juju4/ansible-ail-framework/jobs/685295595

juju4 avatar May 10 '20 13:05 juju4

Fixed in AIL v5.0

The database has been migrated from ARDB to Kvrocks.

Terrtia avatar Jun 06 '23 10:06 Terrtia