plugins: kafka: fix cmake cross compile error

[ThomasDevoogdt]

KAFKA_INCLUDEDIR is not set if FLB_PREFER_SYSTEM_LIB_KAFKA is not used, when cross-compiling, it just translates to -I/librdkafka, which is not allowed. Fix this by only including KAFKA_INCLUDEDIR if really set.

x86_64-linux-gcc: ERROR: unsafe header/library path used in cross-compilation: '-I/librdkafka'

---

Enter [N/A] in the box, if an item is not applicable to your change.

Testing Before we can approve your change; please submit the following in a comment:

• [N/A] Example configuration file for the change
• [N/A] Debug log output from testing the change

• [N/A] Attached Valgrind output that shows no leaks or memory corruption was found

If this is a change to packaging of containers or native binaries then please confirm it works for all targets.

• [N/A] Run local packaging test showing all targets (including any new ones) build.
• [N/A] Set ok-package-test label to test for all targets (requires maintainer to do).

Documentation

• [N/A] Documentation required for this feature

Backporting

• [N/A] Backport to latest stable release.

---

Fluent Bit is licensed under Apache 2.0, by submitting this pull request I understand that this code will be released under the terms of that license.

[ThomasDevoogdt]

@edsiper It has been a while. Any chance that this gets approved? I try to reduce my own patch list.

[ThomasDevoogdt]

@edsiper Happy New Year! Can you have a look at this PR please?

[ThomasDevoogdt]

@edsiper @leonardo-albertovich @fujimotos @koleini This PR is open for a while. Is it possible to have a look?

[ThomasDevoogdt]

@edsiper @patrick-stephens @leonardo-albertovich ping

[patrick-stephens]

Can we rebase to see if it resolves the unit test failures?

[ThomasDevoogdt]

Can we rebase to see if it resolves the unit test failures?

I rebased it.

[ThomasDevoogdt]

Can we rebase to see if it resolves the unit test failures?

@patrick-stephens rebased it, and all checks are passing.

[ThomasDevoogdt]

@patrick-stephens what is the next step to get this merged? I also have other open PRs which are approved, but not merged yet. Can you check my other PRs also? https://github.com/fluent/fluent-bit/pulls/ThomasDevoogdt

[patrick-stephens]

@edsiper will be merging when he is happy to put it in a release - be aware we're finalising the 4.0 release now.

[ThomasDevoogdt]

@edsiper will be merging when he is happy to put it in a release - be aware we're finalising the 4.0 release now.

@patrick-stephens @edsiper I see that 4.0.0 has landed, this is the ideal moment to consider this PR.

[ThomasDevoogdt]

@patrick-stephens can you also check this one, while at it?

[patrick-stephens]

Looks fine to me, just want to check @edsiper is happy with the commit title

[ThomasDevoogdt]

Looks fine to me, just want to check @edsiper is happy with the commit title

@patrick-stephens @edsiper I couldn't really find a similar example for the git message log. But I changed it a bit to what is probably more right. Can you have a look at this PR?

[ThomasDevoogdt]

@cosmo0920 This is another cross compilation issue I encountered.

[cosmo0920]

@cosmo0920 This is another cross compilation issue I encountered.

How could we check this issue? Just using gcc-aarch64-linux-gnu on x86_64 platform to cross compiling is enough?

[ThomasDevoogdt]

@cosmo0920 This is another cross compilation issue I encountered.

How could we check this issue? Just using gcc-aarch64-linux-gnu on x86_64 platform to cross compiling is enough?

It took me some time to reproduce, but these are the steps:

git clone https://github.com/buildroot/buildroot.git
cd buildroot

rm package/fluent-bit/0001-plugins-kafka-fix-cmake-cross-compile-error.patch
echo support/config-fragments/autobuild/bootlin-x86-64-glibc.config,x86_64 > toolchain-configs.csv

./utils/test-pkg -p fluent-bit -t toolchain-configs.csv

And follow the output in another shell:

tail -F br-test-pkg/bootlin-x86-64-glibc/logfile

You can also fix stuff by going to the work folder, and change the current checkout:

cd br-test-pkg/bootlin-x86-64-glibc/
# edit build/fluent-bit-4.0.2/
make

You will see this error:

[335/1140] Building C object plugins/custom_calyptia/CMakeFiles/flb-plugin-custom_calyptia.dir/calyptia.c.o
FAILED: plugins/custom_calyptia/CMakeFiles/flb-plugin-custom_calyptia.dir/calyptia.c.o 
/home/thomas/br-test-pkg/bootlin-x86-64-glibc/host/bin/x86_64-linux-gcc --sysroot=/home/thomas/br-test-pkg/bootlin-x86-64-glibc/host/x86_64-buildroot-linux-gnu/sysroot -DFLB_EVENT_LOOP_AUTO_DISCOVERY -DFLB_HAVE_ACCEPT4 -DFLB_HAVE_ATTRIBUTE_ALLOC_SIZE -DFLB_HAVE_AWS -DFLB_HAVE_AWS_CREDENTIAL_PROCESS -DFLB_HAVE_FORK -DFLB_HAVE_GETENTROPY -DFLB_HAVE_GETENTROPY_SYS_RANDOM -DFLB_HAVE_GMTOFF -DFLB_HAVE_HTTP_SERVER -DFLB_HAVE_INOTIFY -DFLB_HAVE_IN_STORAGE_BACKLOG -DFLB_HAVE_LIBYAML -DFLB_HAVE_LITTLE_ENDIAN_SYSTEM -DFLB_HAVE_METRICS -DFLB_HAVE_OPENSSL -DFLB_HAVE_PARSER -DFLB_HAVE_PROFILES -DFLB_HAVE_PROXY_GO -DFLB_HAVE_RECORD_ACCESSOR -DFLB_HAVE_REGEX -DFLB_HAVE_SIGNV4 -DFLB_HAVE_SQLDB -DFLB_HAVE_STREAM_PROCESSOR -DFLB_HAVE_SYS_WAIT_H -DFLB_HAVE_TLS -DFLB_HAVE_UNICODE_ENCODER -DFLB_HAVE_UNIX_SOCKET -DFLB_HAVE_UTF8_ENCODER -DFLB_MSGPACK_TO_JSON_INIT_BUFFER_SIZE=2.0 -DFLB_MSGPACK_TO_JSON_REALLOC_BUFFER_SIZE=0.10 -DFLB_SYSTEM_LINUX -DJSMN_PARENT_LINKS -DJSMN_STRICT -DLIBRDKAFKA_STATICLIB -DMINIZ_STATIC_DEFINE -DMPACK_EXTENSIONS=1 -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/fluent-otel-proto/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/fluent-otel-proto/proto_c -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/cfl/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/cfl/lib/xxhash -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/flb_libco -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/rbtree -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/chunkio/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/monkey/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/monkey/include/monkey -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/mpack-amalgamation-1.1.1/src -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/miniz -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/onigmo -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/snappy-fef67ac -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/cmetrics/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/ctraces/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/cprofiles/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/lwrb/lwrb/src/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/jansson-e23f558/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/cmetrics -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/tutf8e/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/msgpack-c/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/sqlite-amalgamation-3450200 -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/librdkafka-2.8.0/src -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/cfl/lib/xxhash/cmake_unofficial/.. -I/librdkafka -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/lib/librdkafka-2.8.0/generated/dummy -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/plugins/filter_geoip2/libmaxminddb-1.12.2 -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/plugins/filter_geoip2/libmaxminddb-1.12.2/include -I/home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/plugins/filter_geoip2/libmaxminddb-1.12.2/generated -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -O2 -g0 -D_FORTIFY_SOURCE=1 -Wall -D__FLB_FILENAME__=__FILE__ -O2 -g -DNDEBUG -fPIC -MD -MT plugins/custom_calyptia/CMakeFiles/flb-plugin-custom_calyptia.dir/calyptia.c.o -MF plugins/custom_calyptia/CMakeFiles/flb-plugin-custom_calyptia.dir/calyptia.c.o.d -o plugins/custom_calyptia/CMakeFiles/flb-plugin-custom_calyptia.dir/calyptia.c.o -c /home/thomas/br-test-pkg/bootlin-x86-64-glibc/build/fluent-bit-4.0.2/plugins/custom_calyptia/calyptia.c
x86_64-linux-gcc: ERROR: unsafe header/library path used in cross-compilation: '-I/librdkafka'

[cosmo0920]

Note: This patch is originated from Fluent Bit side of glitches among CMakeLists.txt and kafka plugins. So, we need to handle this compilation error on this repo.

[ThomasDevoogdt]

@cosmo0920 It has been approved for a while, can this be merged?

[edsiper]

let's resolve the conflicts since we had recent changes in the Kafka layer and make it part of the next release

[ThomasDevoogdt]

let's resolve the conflicts since we had recent changes in the Kafka layer and make it part of the next release

@edsiper I rebased the branch. Perhaps you can still take it along?

[patrick-stephens]

Looks like CentOS stream dependencies have gone even more incompatible...

[ThomasDevoogdt]

Looks like CentOS stream dependencies have gone even more incompatible...

I don't think it is related with this PR, can you restore the milestone label?

[ThomasDevoogdt]

@edsiper @patrick-stephens can you have a look, this has been approved before, I don't see any issues anymore, can this be merged?

[patrick-stephens]

Yeah I am fine with it, I think the Debian failures are unrelated.

[ThomasDevoogdt]

@patrick-stephens @edsiper It's again a bit silent around this PR. Everything is approved and ready to be merged, on what do we exactly wait? Can you just merge this one?

[ThomasDevoogdt]

@cosmo0920 Can you put this PR back on another more milestone, I have the feeling that next means forgotten. Or just merge this one.

[coderabbitai[bot]]

Walkthrough

Applied conditional guards around librdkafka include directories in Kafka input and output plugin CMake configurations, executing target_include_directories only when KAFKA_INCLUDEDIR is defined. Other build settings remain unchanged.

Changes

Cohort / File(s)	Change Summary
Kafka CMake include guards plugins/in_kafka/CMakeLists.txt, plugins/out_kafka/CMakeLists.txt	Wrap target_include_directories(... ${KAFKA_INCLUDEDIR}/librdkafka) with if(DEFINED KAFKA_INCLUDEDIR) to avoid invalid include paths when the variable is unset; no other configuration changes.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

I nibbled CMake with careful delight,
Wrapped paths in guards to keep builds light.
No stray “/librdkafka” to roam,
Only defined trails lead us home.
Ears up, I hop—green checks in sight! 🐇✨

[!TIP]

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

✨ Finishing Touches

🧪 Generate unit tests

• [ ] Create PR with unit tests
• [ ] Post copyable unit tests in a comment

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

• Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  • I pushed a fix in commit <commit_id>, please review it.
  • Open a follow-up GitHub issue for this discussion.
• Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
• PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
  • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

• Add @coderabbitai ignore or @coderabbit ignore anywhere in the PR description to prevent this PR from being reviewed.
• Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
• Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

• You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
• Please see the configuration documentation for more information.
• If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

• Visit our Status Page to check the current availability of CodeRabbit.
• Visit our Documentation for detailed information on how to use CodeRabbit.
• Join our Discord Community to get help, request features, and share feedback.
• Follow us on X/Twitter for updates and announcements.

[ThomasDevoogdt]

@cosmo0920 This PR starts to age, can you pick this up? I would like to have it merged.

[patrick-stephens]

It looks like we're hitting some older compiler failures:

/tmp/fluent-bit/include/fluent-bit/flb_simd.h:60:18: error: missing binary operator before token "("
 #if __has_include(<arm_neon.h>)

[cosmo0920]

We need to rebase off the current master.