README.md build instructions fail for Ubuntu users

[oasalonen]

Using bootstrap.sh to build on Ubuntu 18.04 fails because the following additional dependencies need to be installed:

sudo apt-get install pkg-config libsasl2-dev

The error message is misleading because it says libjansson cannot be found. It's actually because pkg-config is not installed on the system. I googled the problem, and it seems I'm not the only one running into the issue.

armisz @armisz Dec 14 2019 22:39 Building kafkacat 1.5.0 using bootstrap.sh on Ubuntu 18.04 fails: Build of avroc FAILED! CMake Error at CMakeLists.txt:178 (message): libjansson >=2.3 not found

[edenhill]

What requires libsasl2-dev?

[oasalonen]

If I remove libsasl2-dev, I get the following error

:~/kafkacat$ ./bootstrap.sh
Downloading https://github.com/edenhill/librdkafka/archive/v1.4.2.tar.gz to librdkafka
Building librdkafka with commands:
([ -f config.h ] || ./configure --prefix=~/test-kafkacat/tmp-bootstrap/usr --install-deps --source-deps-only --disable-lz4-ext) && make -j && make install
checking for OS or distribution... ok (ubuntu)
checking for C compiler from CC env... failed
checking for gcc (by command)... ok
checking for C++ compiler from CXX env... failed
checking for C++ compiler (g++)... ok
checking executable ld... ok
checking executable nm... ok
checking executable objdump... ok
checking executable strip... ok
checking executable libtool... failed (disable)
checking for pkgconfig (by command)... ok
checking for install (by command)... ok
checking for PIC (by compile)... ok
checking for GNU-compatible linker options... ok
checking for GNU linker-script ld flag... ok
checking for __atomic_32 (by compile)... ok
checking for __atomic_64 (by compile)... ok
checking for socket (by compile)... ok
parsing version '0x010402ff'... ok (1.4.2)
checking for librt (by pkg-config)... failed
checking for librt (by compile)... ok
checking for libpthread (by pkg-config)... failed
checking for libpthread (by compile)... ok
checking for c11threads (by pkg-config)... failed
checking for c11threads (by compile)... failed (disable)
checking for libdl (by pkg-config)... failed
checking for libdl (by compile)... ok
building dependency zlib... ok (from source)
checking for zlib (by pkg-config)... failed
checking for zlib (by compile)... ok
building dependency libcrypto... ok (from source)
checking for libcrypto (by pkg-config)... failed
checking for libcrypto (by compile)... ok
checking for libssl (by pkg-config)... failed
checking for libssl (by compile)... ok
checking for libsasl2 (by pkg-config)... failed
checking for libsasl2 (by compile)... failed
checking for libsasl (by pkg-config)... failed
checking for libsasl (by compile)... failed (disable)
building dependency libzstd... ok (from source)
checking for libzstd (by pkg-config)... failed
checking for libzstd (by compile)... ok
checking for libm (by pkg-config)... failed
checking for libm (by compile)... ok
checking for syslog (by compile)... ok
checking for rapidjson (by compile)... failed (disable)
checking for crc32chw (by compile)... ok
checking for regex (by compile)... ok
checking for strndup (by compile)... ok
checking for strlcpy (by compile)... failed (disable)
checking for strerror_r (by compile)... ok
checking for pthread_setname_gnu (by compile)... ok
checking for nm (by env NM)... ok (cached)
checking for python3 (by command)... ok
checking for getrusage (by compile)... ok



###########################################################
###                  Configure failed                   ###
###########################################################
### Accumulated failures:                               ###
###########################################################
 libsasl2 ()
    module: self
    action: fail
    reason:
Failed to install dependency libsasl2

###########################################################
### Installing the following packages might help:       ###
###########################################################
sudo apt install -y  libsasl2-dev

Build of librdkafka FAILED!

[edenhill]

Can you provide the output from tmp-bootstrap/librdkafka/config.log?

[oasalonen]

librdkafka_config.log

Here it is.

[edenhill]

Thank you. This is an issue with librdkafka's ./configure, will look into it.

[OlafEichstaedt]

I tried to build from scratch for Windows 10 Linux Subsystem Ubuntu 16.04 today ...

It seems that there is a similar (or the same) issue:

Downloading https://github.com/apache/avro/archive/release-1.8.2.tar.gz to avroc
Building avroc with commands:
cd lang/c && mkdir -p build && cd build && cmake -DCMAKE_C_FLAGS="-I/root/tmp-bootstrap/usr/include" -DCMAKE_INSTALL_PREFIX=/root/tmp-bootstrap/usr .. && make install
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Found ZLIB: /usr/lib/x86_64-linux-gnu/libz.so (found version "1.2.8")
Enabled deflate codec
-- Could NOT find Snappy (missing:  SNAPPY_LIBRARIES)
Disabled snappy codec. libsnappy not found or zlib not found.
-- Could NOT find PkgConfig (missing:  PKG_CONFIG_EXECUTABLE)
Disabled lzma codec. liblzma not found.
CMake Warning (dev) at CMakeLists.txt:169 (set):
  Policy CMP0053 is not set: Simplify variable reference and escape sequence
  evaluation.  Run "cmake --help-policy CMP0053" for policy details.  Use the
  cmake_policy command to set the policy and suppress this warning.

  For input:

    '@ZLIB_PKG@ @LZMA_PKG@ @SNAPPY_PKG@'

  the old evaluation rules produce:

    'zlib  '

  but the new evaluation rules produce:

    '@ZLIB_PKG@ @LZMA_PKG@ @SNAPPY_PKG@'

  Using the old result for compatibility since the policy is not set.
This warning is for project developers.  Use -Wno-dev to suppress it.

CMake Error at CMakeLists.txt:178 (message):
  libjansson >=2.3 not found

-- Configuring incomplete, errors occurred!
See also "/root/tmp-bootstrap/avroc/lang/c/build/CMakeFiles/CMakeOutput.log".
Build of avroc FAILED!

it complains that libjannson is not found, although the output earlier says: Build of libjansson SUCCEEDED!

With the 2 dependencies from above

sudo apt-get install pkg-config libsasl2-dev

avroc builds ok. But then there is another error:

Downloading https://github.com/confluentinc/libserdes/archive/master.tar.gz to libserdes
Building libserdes with commands:
([ -f config.h ] || ./configure  --prefix=/root/tmp-bootstrap/usr --CFLAGS=-I/root/tmp-bootstrap/usr/include --LDFLAGS=-L/root/tmp-bootstrap/usr/lib) && make && make install
checking for OS or distribution... ok (Ubuntu)
checking for C compiler from CC env... failed
checking for gcc (by command)... ok
checking for C++ compiler from CXX env... failed
checking for C++ compiler (g++)... ok
checking executable ld... ok
checking executable nm... ok
checking executable objdump... ok
checking executable strip... ok
checking for pkgconfig (by command)... ok
checking for install (by command)... ok
checking for PIC (by compile)... ok
checking for GNU-compatible linker options... ok
checking for GNU linker-script ld flag... ok
checking for socket (by compile)... ok
checking for avro-c (by pkg-config)... failed
checking for avro-c (by compile)... ok
checking for avro-cpp (by pkg-config)... failed
checking for avro-cpp (by compile)... failed (disable)
checking for librdkafka (by pkg-config)... failed
checking for librdkafka (by compile)... failed (disable)
checking for jansson (by pkg-config)... failed
checking for jansson (by compile)... ok
checking for libcurl (by pkg-config)... failed
checking for libcurl (by compile)... failed (fail)
checking for libpthread (by pkg-config)... failed
checking for libpthread (by compile)... ok
checking for librt (by pkg-config)... failed
checking for librt (by compile)... ok
checking for nm (by env NM)... ok (cached)
checking for perl (by command)... ok

###########################################################
###                  Configure failed                   ###
###########################################################
### Accumulated failures:                               ###
###########################################################
 libcurl ()
    module: self
    action: fail
    reason:
compile check failed:
CC: CC
flags: -lcurl
gcc -I/root/tmp-bootstrap/usr/include -I/root/tmp-bootstrap/usr/include -g -O2 -fPIC -Wall -Wsign-compare -Wfloat-equal -Wpointer-arith -Wall -Werror -lcurl _mkltmpONb29z.c -o _mkltmpONb29z.c.o  -L/root/tmp-bootstrap/usr/lib -Wl,-rpath-link=/root/tmp-bootstrap/usr/lib -L/root/tmp-bootstrap/usr/lib:
/usr/bin/ld: cannot find -lcurl
collect2: error: ld returned 1 exit status
source:

Build of libserdes FAILED!

This seems weird as curl is present:

root ~ $ curl --version
curl 7.47.0 (x86_64-pc-linux-gnu) libcurl/7.47.0 GnuTLS/3.4.10 zlib/1.2.8 libidn/1.32 librtmp/2.3
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP UnixSockets

For my system, it also needed a third dependency: sudo apt-get install libcurl4-openssl-dev