Solaris 10 X86 build issues

[chienhsingwu]

We had a need to build Solaris lib using a fairly old gnu toolchain and addressed a few build issues in that effort. We'd like to share with you the changes and seek your opinions on the correctness of the changes and, if they are correct, be included in the main code trunk.

Description

ISSUE #1: Solaris does not ship with "endian.h", but the build was trying to include it when it should have been including “sys/byteorder.h”. The problem was this ‘elif’ declaration at line 69 in “src/rdendian.h”:

#elif defined sun #include <sys/byteorder.h>

The tag for detecting sun is ‘__sun’, and this is used correctly elsewhere in that file. The workaround was to include CFLAGS=”-Dsun” and CFLAGS=”-D__sun” when running configure prior to building the library.

ISSUE #2: The compiler option ‘-std=c99’ resulted in numerous occurrences of this error:

*/opt/csw/lib/gcc/i386-pc-solaris2.10/4.9.0/include-fixed/sys/feature_tests.h:346:2: error: #error "Compiler or options invalid for pre-UNIX 03 X/Open applications and pre-2001 POSIX applications" #error "Compiler or options invalid for pre-UNIX 03 X/Open applications *

The workaround was to include CFLAGS=”-D_XPG6” when running configure prior to building the library.

After these workarounds are applied, the library build succeeds and all example programs run without problem.

ISSUE #3: ‘gmake install’ consistently failed part-way through. This was due to a problem with the file “Makefile.base” in the mklove directory under the main librdkafka directory. It contained the following:

lib-install: @printf "$(MKL_YELLOW)Install $(LIBNAME) to $$DESTDIR$(prefix)$(MKL_CLR_RESET)\n" $(INSTALL) -d $$DESTDIR$(includedir)/$(PKGNAME) ;
$(INSTALL) -d $$DESTDIR$(libdir) ;
$(INSTALL) $(HDRS) $$DESTDIR$(includedir)/$(PKGNAME) ;
$(INSTALL) $(LIBNAME).a $$DESTDIR$(libdir) ;
$(INSTALL) $(LIBFILENAME) $$DESTDIR$(libdir) ;
[ -f "$(LIBNAME0).pc" ] && (
$(INSTALL) -d $$DESTDIR$(pkgconfigdir) ;
$(INSTALL) -m 0644 $(LIBNAME0).pc $$DESTDIR$(pkgconfigdir)
) ;
[ -f "$(LIBNAME0)-static.pc" ] && (
$(INSTALL) -d $$DESTDIR$(pkgconfigdir) ;
$(INSTALL) -m 0644 $(LIBNAME0)-static.pc $$DESTDIR$(pkgconfigdir)
) ;
(cd $$DESTDIR$(libdir) && ln -sf $(LIBFILENAME) $(LIBFILENAMELINK))

The problem here was the format of the ln options. It looks like “-sf” might be a Linux-only way of declaring them. The make process kicked this line out as an error. It was fixed by changing it to:

    (cd $$DESTDIR$(libdir) && **ln -f -s** $(LIBFILENAME) $(LIBFILENAMELINK))

The final issue surfaced when trying to perform a Solaris compiler-based (as opposed to GNU) build for an application that makes use of librdkafka functions. This build failed immediately with multiple errors all centered around the following definitions in the rdkafka.h header file:

#define RD_UNUSED attribute((unused)) #define RD_INLINE inline #define RD_DEPRECATED attribute((deprecated))

One of the problems was that the format of the RD_DEPRECATED attribute definition, while valid for gcc, was not acceptable for Solaris. Solaris requires the following format:

attribute((deprecated(“some message”)))

I tried temporarily changing the format, but that only eliminated a few of the errors. Finally, I left the definitions intact, but removed all references to them from the header file. Thereafter the application build proceeded successfully, and the application ran without incident.

As a side note, I tried rebuilding librdkafka (using the GNU toolchain) with the modified rdkafka.h file. While it did throw a fair amount of warnings, the build succeeded and all of the example applications ran successfully.

How to reproduce

GNU build environment: gcc4.9.0, g++ 4.9.0, GNU make 4.0 Solaris build environment: Compiler version 5.8 Librdkafka version: 0.11.4

IMPORTANT: Always try to reproduce the issue on the latest released version (see https://github.com/edenhill/librdkafka/releases), if it can't be reproduced on the latest version the issue has been fixed.

Checklist

IMPORTANT: We will close issues where the checklist has not been completed.

Please provide the following information:

• [x] librdkafka version (release number or git tag): <REPLACE with e.g., v0.10.5 or a git sha. NOT "latest" or "current">
• [X] Apache Kafka version: <REPLACE with e.g., 0.10.2.3>
• [NA] librdkafka client configuration: <REPLACE with e.g., message.timeout.ms=123, auto.reset.offset=earliest, ..>
• [X] Operating system: <REPLACE with e.g., Centos 5 (x64)>
• [NA] Provide logs (with debug=.. as necessary) from librdkafka
• [NA] Provide broker log excerpts
• [NO] Critical issue

[edenhill]

Thank you for your effort.

We depend on the community to maintain support for platforms with no readily available public CI services, such as Solaris, AIX, etc.

It would be good if you could file a pull request with your changes, that'll kick off CI builds for Linux, OSX and Windows, making sure you did not break anything.

[chienhsingwu]

@edenhill, what're the tests executed on CI server? Does the community execute them in their own environments against their Solaris/AIX systems?

[edenhill]

The CIs (Travis, AppVeyor and Doozer.io) run a sub-set of the full test-suite; namely the tests that don't require a Kafka cluster. (make -C tests run_local). We're looking at options to run the full test-suite on CI.

We expect contributors to run the full test-suite before submitting a PR.

As for Solaris, AIX, FreeBSD, etc; we do not maintain these ports but rely on the community to keep them up to date and functional.