Fulcrum lots of build questions about dependencies, structure, non-dockerized

lots of build questions about dependencies, structure, non-dockerized

Open nyxnor opened this issue 1 year ago • 15 comments

System

Platform: Debian Bullseye. Network: Bitcoin Checkout: v1.8.1

I will refer to the package names are present in debian.

Dependencies lost in different files

It is hard to find all the necessary dependencies because they are on different files. https://github.com/cculianu/Fulcrum/blob/master/contrib/build/linux/Dockerfile#L14-L18 https://github.com/cculianu/Fulcrum/blob/master/contrib/build/linux/Dockerfile#L26 Jemalloc not mentioned on the docs https://github.com/cculianu/Fulcrum/blob/master/contrib/build/build.sh#L87, libjemalloc-dev on debian.

https://github.com/cculianu/Fulcrum/#requirements here it says about Qt Core and Qt Networking that can be installed with qt6-base-dev (I chose the highest version, not qt5, qt6). This section also mentions about libzmq3-dev, clang and g++. All fine.

https://github.com/cculianu/Fulcrum/#how-to-compile On this section, it mentions about qmake6, libbz2-dev.

https://github.com/cculianu/Fulcrum/#what-to-do-if-compiling-fails Then on this section is mentions about librocksdb.

My point is that the dependencies are on different sections, instead of one section specifying dependency for platform. Some dependencies are mentioned twice or three times on the readme, but others are missing.

Can you organize the dependencies in one single section?

Building without docker on debian

As I am trying to build without docker because the current docker build does not check integrity and authenticity of downloaded files

https://github.com/cculianu/Fulcrum/blob/cfce15f2a3d60ab00321088f80dabe9b542c912c/contrib/build/linux/Dockerfile#L26
https://github.com/cculianu/Fulcrum/blob/master/contrib/build/linux/Dockerfile#L32
https://github.com/cculianu/Fulcrum/blob/master/contrib/build/build.sh#L90 (checkout commit is insufficient)
https://github.com/cculianu/Fulcrum/blob/master/contrib/build/build.sh#L112 (checkout commit is insufficient)
https://github.com/cculianu/Fulcrum/blob/master/contrib/build/linux/_build.sh#L25 (checkout commit is insufficient)

Found dependencies

From reading three build scripts plus the readme, found the following dependencies for debian:

sudo apt install -y git build-essential cmake qmake6 clang pkg-config \
  qt6-base-dev libzmq3-dev libbz2-dev librocksdb-dev libjemalloc-dev \
  zlib1g-dev libssl-dev libnss3-dev libxslt1-dev libxml2-dev libzstd-dev \
  libgssapi-krb5-2

If you notice, I'm installing the debian package for the libraries, not installing from git. Anyway, jemalloc and rocksdb fixed commit seems to be 2years old https://github.com/cculianu/Fulcrum/blob/master/contrib/build/linux/jemalloc-commit-hash https://github.com/cculianu/Fulcrum/blob/master/contrib/build/linux/rocksdb-commit-hash

Here is my software libraries version:

Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                   Version                 Architecture Description
+++-======================-=======================-============-==================================================================
ii  build-essential        12.9                    amd64        Informational list of build-essential packages
ii  clang                  1:11.0-51+nmu5          amd64        C, C++ and Objective-C compiler (LLVM based), clang binary
ii  cmake                  3.18.4-2+deb11u1        amd64        cross-platform, open-source make system
ii  git                    1:2.30.2-1              amd64        fast, scalable, distributed revision control system
ii  libbz2-dev:amd64       1.0.8-4                 amd64        high-quality block-sorting file compressor library - development
ii  libgssapi-krb5-2:amd64 1.18.3-6+deb11u2        amd64        MIT Kerberos runtime libraries - krb5 GSS-API Mechanism
ii  libjemalloc-dev        5.2.1-3                 amd64        development files and documentation for jemalloc
ii  libnss3-dev:amd64      2:3.61-1+deb11u2        amd64        Development files for the Network Security Service libraries
ii  librocksdb-dev         6.11.4-3+deb11u1        amd64        persistent Key-Value Store for Flash and RAM Storage (development)
ii  libssl-dev:amd64       1.1.1n-0+deb11u3        amd64        Secure Sockets Layer toolkit - development files
ii  libxml2-dev:amd64      2.9.10+dfsg-6.7+deb11u2 amd64        Development files for the GNOME XML library
ii  libxslt1-dev:amd64     1.1.34-4+deb11u1        amd64        XSLT 1.0 processing library - development kit
ii  libzmq3-dev:amd64      4.3.4-1                 amd64        lightweight messaging kernel (development files)
ii  libzstd-dev:amd64      1.4.8+dfsg-2.1          amd64        fast lossless compression algorithm -- development files
ii  pkg-config             0.29.2-1                amd64        manage compile and link flags for libraries
ii  qmake6:amd64           6.3.1+dfsg-9~bpo11+2    amd64        Qt 6 qmake Makefile generator tool
ii  qt6-base-dev:amd64     6.3.1+dfsg-9~bpo11+2    amd64        Qt 6 base development files
ii  zlib1g-dev:amd64       1:1.2.11.dfsg-2+deb11u2 amd64        compression library - development

qmake build command

I moved the libraries to tmp, stripped them and linked against them. I don't know if it was necessary to strip them all, because there was no diff from the stripped ones, meaning debian already strips them stripped.

Also, qmake is pointing to qmake6.

$ qmake ../Fulcrum.pro   "CONFIG-=debug"   "CONFIG+=release"   "LIBS+=-L${rocksdb_libdir_target} -lrocksdb"   "LIBS+=-lz -lbz2" "LIBS+=-L${jemalloc_libdir_target} -ljemalloc" "LIBS+=-L${zmq_libdir_target} -lzmq" "LIBS+=-L${zstd_libdir_target} -lzstd" "LIBS+=-L/usr/lib/x86_64-linux-gnu -lgssapi_krb5" "INCLUDEPATH+=/usr/include"

If you notice, the qmake command is different then on https://github.com/cculianu/Fulcrum/blob/master/contrib/build/linux/_build.sh#L86 I added -lgssapi_krb5 and -lzstd because without it, it was giving me problems error adding symbols: DSO missing from command line

Project MESSAGE: CLI overrides: LIBS=-L/tmp/fulcrum-build/lib/rocksdb -lrocksdb -lz -lbz2 -L/tmp/fulcrum-build/lib/jemalloc -ljemalloc -L/tmp/fulcrum-build/lib/zmq -lzmq -L/tmp/fulcrum-build/lib/zstd -lzstd -L/usr/lib/x86_64-linux-gnu -lgssapi_krb5
Project MESSAGE: CLI overrides: INCLUDEPATH=/usr/include
Project MESSAGE: ZMQ: using CLI override
Project MESSAGE: rocksdb: using CLI override
Project MESSAGE: jemalloc: using CLI override
Project MESSAGE: Including embedded secp256k1
Project MESSAGE: Installation dir prefix is /usr/local

Build error log

g++ -Wl,-O1 -o Fulcrum  AbstractConnection.o App.o BTC.o BTC_Address.o BitcoinD.o BitcoinD_RPCInfo.o BlockProc.o CityHash.o Common.o Controller.o Controller_SynchDSPsTask.o CoTask.o DSProof.o Json.o Json_Parser.o tests.o Logger.o main.o Mempool.o Merkle.o Mixins.o Mgr.o Options.o PeerMgr.o RecordFile.o RollingBloomFilter.o RPC.o RPCMsgId.o ServerMisc.o Servers.o SrvMgr.o Storage.o SSLCertMonitor.o SubsMgr.o SubStatus.o ThreadPool.o TXO.o Util.o VarInt.o Version.o WebSocket.o ZmqSubNotifier.o register_MetaTypes.o amount.o base58.o block.o cleanse.o cashaddr.o cashaddrenc.o aes.o chacha20.o ctaes.o hmac_sha256.o hmac_sha512.o ripemd160.o sha1.o sha256.o sha256_sse4.o sha512.o hash.o interpreter.o pubkey.o script.o script_error.o script_standard.o sigencoding.o test.o token.o transaction.o uint256.o utilstrencodings.o utilstring.o secp256k1.o qrc_resources.o moc_AbstractConnection.o moc_App.o moc_BitcoinD.o moc_Controller.o moc_Logger.o moc_Mgr.o moc_PeerMgr.o moc_RPC.o moc_Servers.o moc_SrvMgr.o moc_SSLCertMonitor.o moc_SubsMgr.o moc_ThreadPool.o moc_WebSocket.o moc_ZmqSubNotifier.o   -L/tmp/fulcrum-build/lib/rocksdb -lrocksdb -lz -lbz2 -L/tmp/fulcrum-build/lib/jemalloc -ljemalloc -L/tmp/fulcrum-build/lib/zmq -lzmq -L/tmp/fulcrum-build/lib/zstd -lzstd -L/usr/lib/x86_64-linux-gnu -lgssapi_krb5 -ldl /usr/lib/x86_64-linux-gnu/libQt6Network.so /usr/lib/x86_64-linux-gnu/libQt6Core.so -lpthread   
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): warning: relocation against `NORM_SESSION_INVALID' in read-only section `.text'
/usr/bin/ld: /tmp/fulcrum-build/lib/rocksdb/librocksdb.a(format.o): in function `rocksdb::UncompressBlockContentsForCompressionType(rocksdb::UncompressionInfo const&, char const*, unsigned long, rocksdb::BlockContents*, unsigned int, rocksdb::ImmutableCFOptions const&, rocksdb::MemoryAllocator*)':
(.text+0x39c1): undefined reference to `snappy::GetUncompressedLength(char const*, unsigned long, unsigned long*)'
/usr/bin/ld: (.text+0x3a3a): undefined reference to `snappy::RawUncompress(char const*, unsigned long, char*)'
/usr/bin/ld: /tmp/fulcrum-build/lib/rocksdb/librocksdb.a(format.o): in function `rocksdb::LZ4_Uncompress(rocksdb::UncompressionInfo const&, char const*, unsigned long, int*, unsigned int, rocksdb::MemoryAllocator*)':
(.text._ZN7rocksdb14LZ4_UncompressERKNS_17UncompressionInfoEPKcmPijPNS_15MemoryAllocatorE[_ZN7rocksdb14LZ4_UncompressERKNS_17UncompressionInfoEPKcmPijPNS_15MemoryAllocatorE]+0x81): undefined reference to `LZ4_createStreamDecode'
/usr/bin/ld: (.text._ZN7rocksdb14LZ4_UncompressERKNS_17UncompressionInfoEPKcmPijPNS_15MemoryAllocatorE[_ZN7rocksdb14LZ4_UncompressERKNS_17UncompressionInfoEPKcmPijPNS_15MemoryAllocatorE]+0x9d): undefined reference to `LZ4_setStreamDecode'
/usr/bin/ld: (.text._ZN7rocksdb14LZ4_UncompressERKNS_17UncompressionInfoEPKcmPijPNS_15MemoryAllocatorE[_ZN7rocksdb14LZ4_UncompressERKNS_17UncompressionInfoEPKcmPijPNS_15MemoryAllocatorE]+0xb3): undefined reference to `LZ4_decompress_safe_continue'
/usr/bin/ld: (.text._ZN7rocksdb14LZ4_UncompressERKNS_17UncompressionInfoEPKcmPijPNS_15MemoryAllocatorE[_ZN7rocksdb14LZ4_UncompressERKNS_17UncompressionInfoEPKcmPijPNS_15MemoryAllocatorE]+0xbd): undefined reference to `LZ4_freeStreamDecode'
/usr/bin/ld: /tmp/fulcrum-build/lib/rocksdb/librocksdb.a(block_based_table_builder.o): in function `rocksdb::(anonymous namespace)::CompressBlockInternal(rocksdb::Slice const&, rocksdb::CompressionInfo const&, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)':
(.text+0xe52): undefined reference to `snappy::MaxCompressedLength(unsigned long)'
/usr/bin/ld: (.text+0xe71): undefined reference to `snappy::RawCompress(char const*, unsigned long, char*, unsigned long*)'
/usr/bin/ld: (.text+0x105e): undefined reference to `LZ4_compressBound'
/usr/bin/ld: (.text+0x1079): undefined reference to `LZ4_createStream'
/usr/bin/ld: (.text+0x10b3): undefined reference to `LZ4_compress_fast_continue'
/usr/bin/ld: (.text+0x10c1): undefined reference to `LZ4_freeStream'
/usr/bin/ld: (.text+0x1129): undefined reference to `LZ4_compressBound'
/usr/bin/ld: (.text+0x115e): undefined reference to `LZ4_createStreamHC'
/usr/bin/ld: (.text+0x116f): undefined reference to `LZ4_resetStreamHC'
/usr/bin/ld: (.text+0x118f): undefined reference to `LZ4_loadDictHC'
/usr/bin/ld: (.text+0x11af): undefined reference to `LZ4_compress_HC_continue'
/usr/bin/ld: (.text+0x11b9): undefined reference to `LZ4_freeStreamHC'
/usr/bin/ld: (.text+0x13b3): undefined reference to `LZ4_loadDict'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-ip.o): in function `zmq::initialize_network()':
(.text+0x592): undefined reference to `pgm_init'
/usr/bin/ld: (.text+0x621): undefined reference to `pgm_error_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-ip.o): in function `zmq::shutdown_network()':
(.text+0x695): undefined reference to `pgm_shutdown'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-socket_base.o): in function `zmq::socket_base_t::connect_internal(char const*)':
(.text+0x6eb1): undefined reference to `pgm_freeaddrinfo'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_client.o): in function `zmq::curve_client_t::produce_initiate(zmq::msg_t*)':
(.text+0x722): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text+0x7e5): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_client.o): in function `zmq::curve_client_t::process_ready(unsigned char const*, unsigned long)':
(.text+0xa07): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text+0xacd): undefined reference to `crypto_box_open_afternm'
/usr/bin/ld: (.text+0xb22): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_client.o): in function `zmq::curve_client_tools_t::produce_hello(void*, unsigned char const*, unsigned long, unsigned char const*, unsigned char const*)':
(.text._ZN3zmq20curve_client_tools_t13produce_helloEPvPKhmS3_S3_[_ZN3zmq20curve_client_tools_t13produce_helloEPvPKhmS3_S3_]+0x3e): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t13produce_helloEPvPKhmS3_S3_[_ZN3zmq20curve_client_tools_t13produce_helloEPvPKhmS3_S3_]+0xa2): undefined reference to `crypto_box'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t13produce_helloEPvPKhmS3_S3_[_ZN3zmq20curve_client_tools_t13produce_helloEPvPKhmS3_S3_]+0x148): undefined reference to `sodium_free'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t13produce_helloEPvPKhmS3_S3_[_ZN3zmq20curve_client_tools_t13produce_helloEPvPKhmS3_S3_]+0x1c8): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_client.o): in function `zmq::curve_client_tools_t::process_welcome(unsigned char const*, unsigned long, unsigned char const*, unsigned char const*, unsigned char*, unsigned char*, unsigned char*)':
(.text._ZN3zmq20curve_client_tools_t15process_welcomeEPKhmS2_S2_PhS3_S3_[_ZN3zmq20curve_client_tools_t15process_welcomeEPKhmS2_S2_PhS3_S3_]+0x58): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t15process_welcomeEPKhmS2_S2_PhS3_S3_[_ZN3zmq20curve_client_tools_t15process_welcomeEPKhmS2_S2_PhS3_S3_]+0x14b): undefined reference to `crypto_box_open'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t15process_welcomeEPKhmS2_S2_PhS3_S3_[_ZN3zmq20curve_client_tools_t15process_welcomeEPKhmS2_S2_PhS3_S3_]+0x16b): undefined reference to `sodium_free'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t15process_welcomeEPKhmS2_S2_PhS3_S3_[_ZN3zmq20curve_client_tools_t15process_welcomeEPKhmS2_S2_PhS3_S3_]+0x1fb): undefined reference to `crypto_box_beforenm'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t15process_welcomeEPKhmS2_S2_PhS3_S3_[_ZN3zmq20curve_client_tools_t15process_welcomeEPKhmS2_S2_PhS3_S3_]+0x2be): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_client.o): in function `zmq::curve_client_tools_t::curve_client_tools_t(unsigned char const (&) [32], unsigned char const (&) [32], unsigned char const (&) [32])':
(.text._ZN3zmq20curve_client_tools_tC2ERA32_KhS3_S3_[_ZN3zmq20curve_client_tools_tC5ERA32_KhS3_S3_]+0x5c): undefined reference to `crypto_box_keypair'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_client.o): in function `zmq::curve_client_t::produce_initiate(zmq::msg_t*) [clone .cold]':
(.text.unlikely+0x4d): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_client.o): in function `zmq::curve_client_t::process_ready(unsigned char const*, unsigned long) [clone .cold]':
(.text.unlikely+0x6d): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_client.o): in function `zmq::curve_client_tools_t::produce_initiate(void*, unsigned long, unsigned long, unsigned char const*, unsigned char const*, unsigned char const*, unsigned char const*, unsigned char const*, unsigned char const*, unsigned char const*, unsigned char const*, unsigned long)':
(.text._ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m[_ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m]+0x7d): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m[_ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m]+0xf8): undefined reference to `randombytes'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m[_ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m]+0x119): undefined reference to `crypto_box'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m[_ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m]+0x190): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m[_ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m]+0x286): undefined reference to `crypto_box'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m[_ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m]+0x334): undefined reference to `sodium_free'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m[_ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m]+0x353): undefined reference to `sodium_free'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m[_ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m]+0x562): undefined reference to `sodium_free'
/usr/bin/ld: (.text._ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m[_ZN3zmq20curve_client_tools_t16produce_initiateEPvmmPKhS3_S3_S3_S3_S3_S3_S3_m]+0x574): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_mechanism_base.o): in function `zmq::curve_encoding_t::decode(zmq::msg_t*, int*) [clone .part.0]':
(.text+0x67): undefined reference to `crypto_box_open_easy_afternm'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_mechanism_base.o): in function `zmq::curve_encoding_t::encode(zmq::msg_t*)':
(.text+0x452): undefined reference to `crypto_box_easy_afternm'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_server.o): in function `zmq::curve_server_t::curve_server_t(zmq::session_base_t*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, zmq::options_t const&, bool)':
(.text+0x49b): undefined reference to `crypto_box_keypair'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_server.o): in function `zmq::curve_server_t::curve_server_t(zmq::session_base_t*, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, zmq::options_t const&, bool)':
(.text+0x612): undefined reference to `crypto_box_keypair'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_server.o): in function `zmq::curve_server_t::process_hello(zmq::msg_t*)':
(.text+0x875): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text+0x92e): undefined reference to `crypto_box_open'
/usr/bin/ld: (.text+0x944): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_server.o): in function `zmq::curve_server_t::produce_welcome(zmq::msg_t*)':
(.text+0xa63): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text+0xac0): undefined reference to `randombytes'
/usr/bin/ld: (.text+0xb2d): undefined reference to `randombytes'
/usr/bin/ld: (.text+0xb45): undefined reference to `crypto_secretbox'
/usr/bin/ld: (.text+0xb5c): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text+0xbbe): undefined reference to `randombytes'
/usr/bin/ld: (.text+0xc4e): undefined reference to `crypto_box'
/usr/bin/ld: (.text+0xd17): undefined reference to `sodium_free'
/usr/bin/ld: (.text+0xd1f): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_server.o): in function `zmq::curve_server_t::process_initiate(zmq::msg_t*)':
(.text+0x115d): undefined reference to `crypto_secretbox_open'
/usr/bin/ld: (.text+0x1265): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text+0x1334): undefined reference to `crypto_box_open'
/usr/bin/ld: (.text+0x134b): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text+0x1413): undefined reference to `crypto_box_open'
/usr/bin/ld: (.text+0x1464): undefined reference to `crypto_box_beforenm'
/usr/bin/ld: (.text+0x14e4): undefined reference to `sodium_free'
/usr/bin/ld: (.text+0x1503): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_server.o): in function `zmq::curve_server_t::produce_ready(zmq::msg_t*)':
(.text+0x198d): undefined reference to `sodium_allocarray'
/usr/bin/ld: (.text+0x1a55): undefined reference to `crypto_box_afternm'
/usr/bin/ld: (.text+0x1ac3): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_server.o): in function `zmq::curve_server_t::process_hello(zmq::msg_t*) [clone .cold]':
(.text.unlikely+0x78): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_server.o): in function `zmq::curve_server_t::produce_welcome(zmq::msg_t*) [clone .cold]':
(.text.unlikely+0x8b): undefined reference to `sodium_free'
/usr/bin/ld: (.text.unlikely+0x93): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_server.o): in function `zmq::curve_server_t::process_initiate(zmq::msg_t*) [clone .cold]':
(.text.unlikely+0xa4): undefined reference to `sodium_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-curve_server.o):(.text.unlikely+0xb6): more undefined references to `sodium_free' follow
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-ws_engine.o): in function `zmq::ws_engine_t::server_handshake()':
(.text+0x1a01): undefined reference to `strlcpy'
/usr/bin/ld: (.text+0x1a55): undefined reference to `strlcpy'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-ws_engine.o): in function `zmq::ws_engine_t::client_handshake()':
(.text+0x240c): undefined reference to `strlcpy'
/usr/bin/ld: (.text+0x2433): undefined reference to `strlcpy'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::init(char const*, bool, bool)':
(.text+0x541): undefined reference to `NORM_NODE_ANY'
/usr/bin/ld: (.text+0x663): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: (.text+0x67a): undefined reference to `NormCreateSession'
/usr/bin/ld: (.text+0x681): undefined reference to `NORM_SESSION_INVALID'
/usr/bin/ld: (.text+0x696): undefined reference to `NormIsUnicastAddress'
/usr/bin/ld: (.text+0x6af): undefined reference to `NormSetTTL'
/usr/bin/ld: (.text+0x6c7): undefined reference to `NormSetRxPortReuse'
/usr/bin/ld: (.text+0x6d8): undefined reference to `NormSetLoopback'
/usr/bin/ld: (.text+0x6ec): undefined reference to `NormSetMulticastInterface'
/usr/bin/ld: (.text+0x736): undefined reference to `NormSetDefaultUnicastNack'
/usr/bin/ld: (.text+0x74c): undefined reference to `NormSetDefaultSyncPolicy'
/usr/bin/ld: (.text+0x75d): undefined reference to `NormStartReceiver'
/usr/bin/ld: (.text+0x778): undefined reference to `NormGetRandomSessionId'
/usr/bin/ld: (.text+0x7a3): undefined reference to `NormStartSender'
/usr/bin/ld: (.text+0x7bf): undefined reference to `NormSetCongestionControl'
/usr/bin/ld: (.text+0x7e2): undefined reference to `NormStreamOpen'
/usr/bin/ld: (.text+0x7e9): undefined reference to `NORM_OBJECT_INVALID'
/usr/bin/ld: (.text+0x80d): undefined reference to `NormDestroyInstance'
/usr/bin/ld: (.text+0x814): undefined reference to `NORM_SESSION_INVALID'
/usr/bin/ld: (.text+0x822): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: (.text+0x843): undefined reference to `NormCreateInstance'
/usr/bin/ld: (.text+0x84a): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: (.text+0x890): undefined reference to `NormDestroyInstance'
/usr/bin/ld: (.text+0x897): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::recv_data(void const*)':
(.text+0xd75): undefined reference to `NORM_OBJECT_INVALID'
/usr/bin/ld: (.text+0xd86): undefined reference to `NormObjectGetType'
/usr/bin/ld: (.text+0xd97): undefined reference to `NormObjectGetUserData'
/usr/bin/ld: (.text+0x1004): undefined reference to `NormStreamSeekMsgStart'
/usr/bin/ld: (.text+0x1025): undefined reference to `NormStreamRead'
/usr/bin/ld: (.text+0x106a): undefined reference to `NormStreamRead'
/usr/bin/ld: (.text+0x145c): undefined reference to `NormObjectSetUserData'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::restart_input()':
(.text+0x1608): undefined reference to `NORM_OBJECT_INVALID'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `non-virtual thunk to zmq::norm_engine_t::restart_input()':
(.text+0x1648): undefined reference to `NORM_OBJECT_INVALID'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::shutdown()':
(.text+0x1789): undefined reference to `NORM_SESSION_INVALID'
/usr/bin/ld: (.text+0x1793): undefined reference to `NormDestroySession'
/usr/bin/ld: (.text+0x17a8): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: (.text+0x17b2): undefined reference to `NormStopInstance'
/usr/bin/ld: (.text+0x17be): undefined reference to `NormDestroyInstance'
/usr/bin/ld: (.text+0x17d1): undefined reference to `NormStopReceiver'
/usr/bin/ld: (.text+0x1815): undefined reference to `NormStopSender'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::send_data()':
(.text+0x1a41): undefined reference to `NormStreamWrite'
/usr/bin/ld: (.text+0x1ab2): undefined reference to `NormStreamFlush'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::plug(zmq::io_thread_t*, zmq::session_base_t*)':
(.text+0x1b60): undefined reference to `NormGetDescriptor'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::in_event()':
(.text+0x1bf8): undefined reference to `NormGetNextEvent'
/usr/bin/ld: (.text+0x1cc6): undefined reference to `NormNodeDelete'
/usr/bin/ld: (.text+0x1cd6): undefined reference to `NormObjectGetUserData'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::norm_engine_t(zmq::io_thread_t*, zmq::options_t const&)':
(.text+0x1e02): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: (.text+0x1e1c): undefined reference to `NORM_SESSION_INVALID'
/usr/bin/ld: (.text+0x1e38): undefined reference to `NORM_OBJECT_INVALID'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-random.o): in function `zmq::random_open()':
(.text+0x45): undefined reference to `sodium_init'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-random.o): in function `zmq::random_close()':
(.text+0xb1): undefined reference to `randombytes_close'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-zmq_utils.o): in function `zmq_curve_keypair':
(.text+0x6a7): undefined reference to `crypto_box_keypair'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-zmq_utils.o): in function `zmq_curve_public':
(.text+0x743): undefined reference to `crypto_scalarmult_base'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::init_address(char const*, pgm_addrinfo_t**, unsigned short*)':
(.text+0x58e): undefined reference to `pgm_getaddrinfo'
/usr/bin/ld: (.text+0x67f): undefined reference to `pgm_error_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::~pgm_socket_t()':
(.text+0x6d8): undefined reference to `pgm_close'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::get_receiver_fds(int*, int*)':
(.text+0x9d6): undefined reference to `pgm_getsockopt'
/usr/bin/ld: (.text+0xa09): undefined reference to `pgm_getsockopt'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::get_sender_fds(int*, int*, int*, int*)':
(.text+0xc92): undefined reference to `pgm_getsockopt'
/usr/bin/ld: (.text+0xcc5): undefined reference to `pgm_getsockopt'
/usr/bin/ld: (.text+0xcf8): undefined reference to `pgm_getsockopt'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o):(.text+0xd2b): more undefined references to `pgm_getsockopt' follow
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::send(unsigned char*, unsigned long)':
(.text+0x115f): undefined reference to `pgm_send'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::get_rx_timeout()':
(.text+0x132a): undefined reference to `pgm_getsockopt'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::get_tx_timeout()':
(.text+0x141e): undefined reference to `pgm_getsockopt'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::get_max_tsdu_size()':
(.text+0x1509): undefined reference to `pgm_getsockopt'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::init(bool, char const*)':
(.text+0x16ac): undefined reference to `pgm_socket'
/usr/bin/ld: (.text+0x1752): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1775): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x17ba): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x17ec): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1896): undefined reference to `pgm_setsockopt'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o):(.text+0x18bb): more undefined references to `pgm_setsockopt' follow
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::init(bool, char const*)':
(.text+0x1a0e): undefined reference to `pgm_gsi_create_from_data'
/usr/bin/ld: (.text+0x1a6b): undefined reference to `pgm_bind3'
/usr/bin/ld: (.text+0x1b32): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1b45): undefined reference to `pgm_close'
/usr/bin/ld: (.text+0x1b5b): undefined reference to `pgm_freeaddrinfo'
/usr/bin/ld: (.text+0x1b73): undefined reference to `pgm_error_free'
/usr/bin/ld: (.text+0x1bbe): undefined reference to `pgm_socket'
/usr/bin/ld: (.text+0x1cdc): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1d04): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1d31): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1d54): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1d98): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1daa): undefined reference to `pgm_freeaddrinfo'
/usr/bin/ld: (.text+0x1dd6): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1e03): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1e38): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1e5b): undefined reference to `pgm_setsockopt'
/usr/bin/ld: (.text+0x1e6e): undefined reference to `pgm_connect'
/usr/bin/ld: (.text+0x1f06): undefined reference to `pgm_setsockopt'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::receive(void**, pgm_tsi_t const**)':
(.text+0x2383): undefined reference to `pgm_recvmsgv'
/usr/bin/ld: (.text+0x27cf): undefined reference to `pgm_free'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-pgm_socket.o): in function `zmq::pgm_socket_t::process_upstream()':
(.text+0x291b): undefined reference to `pgm_recvmsgv'
/usr/bin/ld: warning: creating DT_TEXTREL in a PIE
collect2: error: ld returned 1 exit status
make: *** [Makefile:445: Fulcrum] Error 1

Sep 20 '22 09:09 nyxnor

It looks like some of the things that libzmq depends on are not being linked. Things such as -lsodium and others. You may try and see if you can get it to link by manually cutting/pasting the final link command that fails into your terminal and seeing if you can add things to it to get it to work... (things such as -lsodium ). If that succeeds then figure out how to get those extra libs added to the build line.

You are right in that I have not done the best of jobs in describing exactly what the dependencies are for building. The reason for this is that the dependencies expanded over time and I neglected to go back and update readme files.

You did a great job in figuring out what they might be.

Sep 22 '22 04:09 cculianu

If you like, after you are done and have successfully trouble-shot this, you can try composing a guide for people or suggesting what I must change in the readmes, etc?

Note I actually work on macos currently as my dev system, and I use MacPorts, so I don't always do the best of jobs at describing to Linux users what they need to do on their systems. MacPorts package manager sometimes has the same package names as, say, Debian based systems, but sometimes it doesn't.

Anyway I will look at your message here in more detail and see if I can come up with a better build guide for people... especially one specific to e.g. Ubuntu and/or Debian which seems to be the most popular thing these days...

Sep 22 '22 04:09 cculianu

It looks like some of the things that libzmq depends on are not being linked. Things such as -lsodium and others. You may try and see if you can get it to link by manually cutting/pasting the final link command that fails into your terminal and seeing if you can add things to it to get it to work... (things such as -lsodium ). If that succeeds then figure out how to get those extra libs added to the build line.

Thanks for the reply. Ok, seeing the log again, I found these missing dependencies, let me try again:

libpgm-dev
libsodium-dev
libsnappy-dev
liblz4-dev

$ apt list --installed libpgm-dev libsodium-dev libsnappy-dev liblz4-dev
Listing... Done
libpgm-dev/stable,now 5.3.128~dfsg-2 amd64 [installed,automatic]
libsodium-dev/stable,now 1.0.18-1 amd64 [installed,automatic]

so libsodium and libpgm was already installed actually, but they probably need to be linked.

also something wrong with zmq as can be seen on the log, but don't know yet as I actually linked it. Do you know what is the zmq issue?

If you like, after you are done and have successfully trouble-shot this, you can try composing a guide for people or suggesting what I must change in the readmes, etc?

Of course, will do after this works, maybe even an installation script.

Note I actually work on macos currently as my dev system, and I use MacPorts, so I don't always do the best of jobs at describing to Linux users what they need to do on their systems. MacPorts package manager sometimes has the same package names as, say, Debian based systems, but sometimes it doesn't.

I understand it is difficult to find compatibility and requirements for different systems. The process is tiresome, what I did to find the best versions was doing apt search PKG and reading the output, then selecting package and using apt show PKG to read the description to see if that is what is need. But once this is finished, next updates will be easier as few packages will be added I guess.

Sep 22 '22 08:09 nyxnor

Could you please explain better why moving the ssl libraries libcrypto*.so out of the way? And should I specify the -lcrypto or -lssl with qmake?

Sep 22 '22 10:09 nyxnor

Checked out at the las commit as of now cfd484c6f1447a8547eeccbfdd0e99c2a9983f79

$ qmake6 ../Fulcrum.pro "CONFIG-=debug" "CONFIG+=release"  "LIBS+=-L/tmp/fulcrum-build/lib/rocksdb -lrocksdb -lz -lbz2 -L/tmp/fulcrum-build/lib/jemalloc -ljemalloc -L/tmp/fulcrum-build/lib/zmq -lzmq -L/tmp/fulcrum-build/lib/zstd -lzstd -L/usr/lib/x86_64-linux-gnu -lgssapi_krb5 -L/tmp/fulcrum-build/lib/sodium -lsodium -L/tmp/fulcrum-build/lib/lz4 -llz4 -L/tmp/fulcrum-build/lib/pgm -lpgm -L/tmp/fulcrum-build/lib/snappy -lsnappy" "INCLUDEPATH+=/usr/include"
Info: creating stash file /home/user/fulcrum/build/.qmake.stash
Project MESSAGE: CLI overrides: LIBS=-L/tmp/fulcrum-build/lib/rocksdb -lrocksdb -lz -lbz2 -L/tmp/fulcrum-build/lib/jemalloc -ljemalloc -L/tmp/fulcrum-build/lib/zmq -lzmq -L/tmp/fulcrum-build/lib/zstd -lzstd -L/usr/lib/x86_64-linux-gnu -lgssapi_krb5 -L/tmp/fulcrum-build/lib/sodium -lsodium -L/tmp/fulcrum-build/lib/lz4 -llz4 -L/tmp/fulcrum-build/lib/pgm -lpgm -L/tmp/fulcrum-build/lib/snappy -lsnappy
Project MESSAGE: CLI overrides: INCLUDEPATH=/usr/include
Info: creating cache file /home/user/fulcrum/build/.qmake.cache
Checking for builtin_clzl... yes
Checking for builtin_clzll... yes
Checking for endian_big... no
Checking for endian_little... yes
Project MESSAGE: ZMQ: using CLI override
Project MESSAGE: rocksdb: using CLI override
Project MESSAGE: jemalloc: using CLI override
Project MESSAGE: Including embedded secp256k1
Project MESSAGE: Installation dir prefix is /usr/local

Things are getting better, just zmq missing something now.

build error log:

g++ -Wl,-O1 -o Fulcrum  AbstractConnection.o App.o BTC.o BTC_Address.o BitcoinD.o BitcoinD_RPCInfo.o BlockProc.o CityHash.o Common.o Controller.o Controller_SynchDSPsTask.o CoTask.o DSProof.o Json.o Json_Parser.o tests.o Logger.o main.o Mempool.o Merkle.o Mixins.o Mgr.o Options.o PeerMgr.o RecordFile.o RollingBloomFilter.o RPC.o RPCMsgId.o ServerMisc.o Servers.o SrvMgr.o Storage.o SSLCertMonitor.o SubsMgr.o SubStatus.o ThreadPool.o TXO.o Util.o VarInt.o Version.o WebSocket.o ZmqSubNotifier.o register_MetaTypes.o amount.o base58.o block.o cleanse.o cashaddr.o cashaddrenc.o aes.o chacha20.o ctaes.o hmac_sha256.o hmac_sha512.o ripemd160.o sha1.o sha256.o sha256_sse4.o sha512.o hash.o interpreter.o pubkey.o script.o script_error.o script_standard.o sigencoding.o test.o token.o transaction.o uint256.o utilstrencodings.o utilstring.o secp256k1.o qrc_resources.o moc_AbstractConnection.o moc_App.o moc_BitcoinD.o moc_Controller.o moc_Logger.o moc_Mgr.o moc_PeerMgr.o moc_RPC.o moc_Servers.o moc_SrvMgr.o moc_SSLCertMonitor.o moc_SubsMgr.o moc_ThreadPool.o moc_WebSocket.o moc_ZmqSubNotifier.o   -L/tmp/fulcrum-build/lib/rocksdb -lrocksdb -lz -lbz2 -L/tmp/fulcrum-build/lib/jemalloc -ljemalloc -L/tmp/fulcrum-build/lib/zmq -lzmq -L/tmp/fulcrum-build/lib/zstd -lzstd -L/usr/lib/x86_64-linux-gnu -lgssapi_krb5 -L/tmp/fulcrum-build/lib/sodium -lsodium -L/tmp/fulcrum-build/lib/lz4 -llz4 -L/tmp/fulcrum-build/lib/pgm -lpgm -L/tmp/fulcrum-build/lib/snappy -lsnappy -ldl /usr/lib/x86_64-linux-gnu/libQt6Network.so /usr/lib/x86_64-linux-gnu/libQt6Core.so -lpthread   
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): warning: relocation against `NORM_SESSION_INVALID' in read-only section `.text'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-ws_engine.o): in function `zmq::ws_engine_t::server_handshake()':
(.text+0x1a01): undefined reference to `strlcpy'
/usr/bin/ld: (.text+0x1a55): undefined reference to `strlcpy'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-ws_engine.o): in function `zmq::ws_engine_t::client_handshake()':
(.text+0x240c): undefined reference to `strlcpy'
/usr/bin/ld: (.text+0x2433): undefined reference to `strlcpy'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::init(char const*, bool, bool)':
(.text+0x541): undefined reference to `NORM_NODE_ANY'
/usr/bin/ld: (.text+0x663): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: (.text+0x67a): undefined reference to `NormCreateSession'
/usr/bin/ld: (.text+0x681): undefined reference to `NORM_SESSION_INVALID'
/usr/bin/ld: (.text+0x696): undefined reference to `NormIsUnicastAddress'
/usr/bin/ld: (.text+0x6af): undefined reference to `NormSetTTL'
/usr/bin/ld: (.text+0x6c7): undefined reference to `NormSetRxPortReuse'
/usr/bin/ld: (.text+0x6d8): undefined reference to `NormSetLoopback'
/usr/bin/ld: (.text+0x6ec): undefined reference to `NormSetMulticastInterface'
/usr/bin/ld: (.text+0x736): undefined reference to `NormSetDefaultUnicastNack'
/usr/bin/ld: (.text+0x74c): undefined reference to `NormSetDefaultSyncPolicy'
/usr/bin/ld: (.text+0x75d): undefined reference to `NormStartReceiver'
/usr/bin/ld: (.text+0x778): undefined reference to `NormGetRandomSessionId'
/usr/bin/ld: (.text+0x7a3): undefined reference to `NormStartSender'
/usr/bin/ld: (.text+0x7bf): undefined reference to `NormSetCongestionControl'
/usr/bin/ld: (.text+0x7e2): undefined reference to `NormStreamOpen'
/usr/bin/ld: (.text+0x7e9): undefined reference to `NORM_OBJECT_INVALID'
/usr/bin/ld: (.text+0x80d): undefined reference to `NormDestroyInstance'
/usr/bin/ld: (.text+0x814): undefined reference to `NORM_SESSION_INVALID'
/usr/bin/ld: (.text+0x822): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: (.text+0x843): undefined reference to `NormCreateInstance'
/usr/bin/ld: (.text+0x84a): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: (.text+0x890): undefined reference to `NormDestroyInstance'
/usr/bin/ld: (.text+0x897): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::recv_data(void const*)':
(.text+0xd75): undefined reference to `NORM_OBJECT_INVALID'
/usr/bin/ld: (.text+0xd86): undefined reference to `NormObjectGetType'
/usr/bin/ld: (.text+0xd97): undefined reference to `NormObjectGetUserData'
/usr/bin/ld: (.text+0x1004): undefined reference to `NormStreamSeekMsgStart'
/usr/bin/ld: (.text+0x1025): undefined reference to `NormStreamRead'
/usr/bin/ld: (.text+0x106a): undefined reference to `NormStreamRead'
/usr/bin/ld: (.text+0x145c): undefined reference to `NormObjectSetUserData'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::restart_input()':
(.text+0x1608): undefined reference to `NORM_OBJECT_INVALID'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `non-virtual thunk to zmq::norm_engine_t::restart_input()':
(.text+0x1648): undefined reference to `NORM_OBJECT_INVALID'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::shutdown()':
(.text+0x1789): undefined reference to `NORM_SESSION_INVALID'
/usr/bin/ld: (.text+0x1793): undefined reference to `NormDestroySession'
/usr/bin/ld: (.text+0x17a8): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: (.text+0x17b2): undefined reference to `NormStopInstance'
/usr/bin/ld: (.text+0x17be): undefined reference to `NormDestroyInstance'
/usr/bin/ld: (.text+0x17d1): undefined reference to `NormStopReceiver'
/usr/bin/ld: (.text+0x1815): undefined reference to `NormStopSender'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::send_data()':
(.text+0x1a41): undefined reference to `NormStreamWrite'
/usr/bin/ld: (.text+0x1ab2): undefined reference to `NormStreamFlush'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::plug(zmq::io_thread_t*, zmq::session_base_t*)':
(.text+0x1b60): undefined reference to `NormGetDescriptor'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::in_event()':
(.text+0x1bf8): undefined reference to `NormGetNextEvent'
/usr/bin/ld: (.text+0x1cc6): undefined reference to `NormNodeDelete'
/usr/bin/ld: (.text+0x1cd6): undefined reference to `NormObjectGetUserData'
/usr/bin/ld: /tmp/fulcrum-build/lib/zmq/libzmq.a(libzmq_la-norm_engine.o): in function `zmq::norm_engine_t::norm_engine_t(zmq::io_thread_t*, zmq::options_t const&)':
(.text+0x1e02): undefined reference to `NORM_INSTANCE_INVALID'
/usr/bin/ld: (.text+0x1e1c): undefined reference to `NORM_SESSION_INVALID'
/usr/bin/ld: (.text+0x1e38): undefined reference to `NORM_OBJECT_INVALID'
/usr/bin/ld: warning: creating DT_TEXTREL in a PIE
collect2: error: ld returned 1 exit status
make: *** [Makefile:508: Fulcrum] Error 1

Do you know how to fix this?

Maybe something from these answers https://stackoverflow.com/questions/65407610/relocation-against-xxx-in-read-only-section-text-wrong-compiler-or-linux-se but not sure.

Sep 22 '22 11:09 nyxnor

Ok, tried it again, moved zmq as the last library, worked.

qmake6 ../Fulcrum.pro "CONFIG-=debug" "CONFIG+=release"  "LIBS+=-L/tmp/fulcrum-build/lib/rocksdb -lrocksdb -lz -lbz2 -L/tmp/fulcrum-build/lib/jemalloc -ljemalloc -L/tmp/fulcrum-build/lib/zstd -lzstd -L/usr/lib/x86_64-linux-gnu -lgssapi_krb5 -L/tmp/fulcrum-build/lib/sodium -lsodium -L/tmp/fulcrum-build/lib/lz4 -llz4 -L/tmp/fulcrum-build/lib/pgm -lpgm -L/tmp/fulcrum-build/lib/snappy -lsnappy -L/tmp/fulcrum-build/lib/zmq -lzmq" "INCLUDEPATH+=/usr/include"

Happy.

Sep 22 '22 12:09 nyxnor

Could you please explain better why moving the ssl libraries libcrypto*.so out of the way? And should I specify the -lcrypto or -lssl with qmake?

That was in the Dockerfile. It was a trick (that works) to get it to just statically link the .a files.

Sep 22 '22 13:09 cculianu

Install dependencies:

$ export DEBIAN_FRONTENT=noninteractive
$ sudo apt update && sudo apt install \
  git python3 build-essential cmake qmake6 clang pkg-config \
  qt6-base-dev libzmq3-dev libbz2-dev librocksdb-dev libjemalloc-dev \
  zlib1g-dev libssl-dev libnss3-dev libxslt1-dev libxml2-dev libzstd-dev \
  libgssapi-krb5-2 libpgm-dev libsodium-dev libsnappy-dev liblz4-dev

Set program priority:

$ sudo update-alternatives \
  --install /usr/bin/qmake qmake /usr/bin/qmake6 60

Move libraries to temporary directory and strip them:

## do not use trailing "/" slash at the end of directory name
zstd_libdir_orig="$(pkg-config --variable=libdir libzstd)"
rocksdb_libdir_orig="/usr/lib"
pgm_libdir_orig="/usr/lib/x86_64-linux-gnu"
sodium_libdir_orig="$(pkg-config --variable=libdir libsodium)"
snappy_libdir_orig="$(pkg-config --variable=libdir snappy)"
lz4_libdir_orig="$(pkg-config --variable=libdir liblz4)"
jemalloc_libdir_orig="$(pkg-config --variable=libdir jemalloc)"
zmq_libdir_orig="$(pkg-config --variable=libdir libzmq)"

## please match the above variables without '$' to the variable below
all_libdir_orig="zstd_libdir_orig rocksdb_libdir_orig pgm_libdir_orig sodium_libdir_orig snappy_libdir_orig lz4_libdir_orig jemalloc_libdir_orig zmq_libdir_orig"

fulcrum_buildtmp="/tmp/fulcrum-build"

## start with variable empty
all_libdir_target=""
for library in ${all_libdir_orig}; do
  ## get only library name
  lib_name="${library%%_*}"
  ## get libdir path
  tmp_var="$(eval printf '%s' '$'"${lib_name}"_libdir_target)"
  ## assign to _target the library target dir
  eval "${lib_name}"_libdir_target="${fulcrum_buildtmp}/lib/${lib_name}"
  ## save libdir path to all targets to be created and targeted
  all_libdir_target="${all_libdir_target} ${tmp_var}"
done
## unset variable(s)
lib_name=""

mkdir -p ${all_libdir_target}

for dir in ${all_libdir_target}; do
  ## get library name by target path (it does not contain lib prefix)
  lib_name="${dir##*/}"
  ## get library origin directory by evaluation
  eval lib_orig="$(printf '%s' '$'${lib_name}_libdir_orig)"
  ## cp library*.a files to target dir
  cp ${lib_orig}/lib${lib_name}*.a "${dir}"
  ## strip all files present in target dir
  for file in ${dir}/*; do
    strip -g "${file}"
  done
done

Import maintainer's key (also available at https://raw.githubusercontent.com/Electron-Cash/keys-n-hashes/master/pubkeys/calinkey.txt)

$ gpg --recv-keys D465135F97D0047E18E99DC321810A542031C02C

Clone the repo:

$ git clone https://github.com/cculianu/Fulcrum ~/Fulcrum

Enter the repo:

$ cd  ~/Fulcrum

Get the last tag:

$ fulcrum_tag="$(git tag -l | tail -1)"
$ echo "${fulcrum_tag}"

Verify last tag (expect Good signature, else do not proceed):

Fulcrum does not sign tags, but signs the commit that the tag references, this is why we are using verify-commit and not verify-tag.

$ git verify-commit ${fulcrum_tag}^{commit}
## example:
## $ git verify-commit v1.8.1^{commit}

Checkout the verified tag:

$ git checkout ${fulcrum_tag}

From the toplevel of the fulcrum git repository, create the build directory and enter it:

$ mkdir build && cd build

Qmake:

$ qmake ../Fulcrum.pro \
  "CONFIG-=debug" \
  "CONFIG+=release" \
  "LIBS+=-L${rocksdb_libdir_target} -lrocksdb" \
  "LIBS+=-lz -lbz2" \
  "LIBS+=-L${jemalloc_libdir_target} -ljemalloc" \
  "LIBS+=-L${zstd_libdir_target} -lzstd" \
  "LIBS+=-L/usr/lib/x86_64-linux-gnu -lgssapi_krb5" \
  "LIBS+=-L${sodium_libdir_target} -lsodium" \
  "LIBS+=-L${lz4_libdir_target} -llz4" \
  "LIBS+=-L${pgm_libdir_target} -lpgm" \
  "LIBS+=-L${snappy_libdir_target} -lsnappy" \
  "LIBS+=-L${zmq_libdir_target} -lzmq" \
  "INCLUDEPATH+=/usr/include"

Make:

#make -j$(echo "$(nproc)/2" | bc)  ## make with half of the available cores
$ make -j$(nproc) ## make -j#, where "#" is the number of cores

If you notice, gssapi_krb5 was not moved nor stripped, this is because there is no .a file, just .so.

Could you please explain better why moving the ssl libraries libcrypto*.so out of the way? And should I specify the -lcrypto or -lssl with qmake?

That was in the Dockerfile. It was a trick (that works) to get it to just statically link the .a files.

https://github.com/cculianu/Fulcrum/blob/master/contrib/build/linux/_build.sh#L81 https://github.com/cculianu/Fulcrum/blob/master/contrib/build/linux/_build.sh#L101 Sorry, didn't understand, should I use it? Is there a better way than moving the libraries and them returning them to path?

Sep 22 '22 13:09 nyxnor

About the instructions above, the shell code is posix compliant. The dependencies are per platform and I can't solve that. The libraries path are tricky to get, pkg-config does not always works reliably, sometimes it reports a different directory then it should, so what I leave for the reader is to only change the library variables value, it is the only thing to do, the rest the script handles.

Sep 22 '22 13:09 nyxnor

Can this script be added to supported platforms? If so, where it fits best?

Debian builds do not require docker, and the patches and fixed commits are kinda old, so I think it is best to use the package manager as it already has signature verification.

If someone is using fedora or macos, they need to add the dependencies to the script that will detect the platform, and the libraries path it pkg-config fails or if it is not linux-gnu.

Sep 22 '22 13:09 nyxnor

@nyxnor I really appreciate all your work here. I am feeling a bit under the weather today -- let me re-read and digest what you have discovered and recommended a bit later today or tomorrow and get back to you. Potentially, yes, maybe a script living in contrib/ could be warranted! Maybe calling it contrib/make_for_debian or somesuch.. hmm. But yes, let me get back to you. Thanks for the persistence and the recommendations.

Sep 22 '22 17:09 cculianu

Sorry, didn't understand, should I use it? Is there a better way than moving the libraries and them returning them to path?

I don't think that's necessary in this case. That was for the Docker build which tries to be as statically linked as possible -- in this case you are using system Qt which dynamically loads, at runtime, whatever system openssl it can find as a .so. So in this case that is not necessary and I would recommend against it if using Debian/Ubuntu's qt package... since it doesn't know how to use the static openssl anyway so.. it would actually work against us here.

Sep 22 '22 17:09 cculianu

I am feeling a bit under the weather today

Don't rush, rest and take care.

-- let me re-read and digest what you have discovered and recommended a bit later today or tomorrow and get back to you. Potentially, yes, maybe a script living in contrib/ could be warranted! Maybe calling it contrib/make_for_debian or somesuch.. hmm. But yes, let me get back to you. Thanks for the persistence and the recommendations.

Great, just wanted to facilitate this for debian, I tried to do the smallest script possible for instruction and analysis, so we will see after you evaluate.

Sep 22 '22 18:09 nyxnor

ping :)

Oct 10 '22 09:10 nyxnor

In my case, I needed to install Fulcrum 1.9.1 to CentOS 9. libbz was not properly linked, so I had to run the following.

yum install -y bzip2-devel
cd /usr/lib64
ln -s libbz2.so.1.0.8 libbz2.so.1.0

Apr 24 '23 03:04 pad01g

Fulcrum Fulcrum copied to clipboard

lots of build questions about dependencies, structure, non-dockerized

System

Dependencies lost in different files

Building without docker on debian

Found dependencies

qmake build command

Build error log

Fulcrum
Fulcrum copied to clipboard