CESNET
OpenBSD compatibility for ipfixcol2
Yes, it would be nice to have it working on the OSes from the *BSD family. While trying to create a FreeBSD port I fell into the same issues.
Thanks to these patches (mostly) I built it on FreeBSD. Unfortunately, it looks like linking went wrong or something or some includes are missing since for all the plugins I get the same error: ERROR: Configurator: Collector failed to start: Failed to load a plugin from '/usr/local/lib/ipfixcol2/libjson-kafka-output.so': /usr/local/lib/ipfixcol2/libjson-kafka-output.so: Undefined symbol "ipx_ctx_private_set" Any clues how to fix it ?
BTW is CESNET interested in making their software runnable not only on Linux ?
Thanks to these patches (mostly) I built it on FreeBSD. Unfortunately, it looks like linking went wrong or something or some includes are missing since for all the plugins I get the same error: ERROR: Configurator: Collector failed to start: Failed to load a plugin from '/usr/local/lib/ipfixcol2/libjson-kafka-output.so': /usr/local/lib/ipfixcol2/libjson-kafka-output.so: Undefined symbol "ipx_ctx_private_set" Any clues how to fix it ?
BTW is CESNET interested in making their software runnable not only on Linux ?
I looked over the changes that would be necessary to get this running on *BSD and they seem minor enough that we will work on applying them. We are interested in making our software run on other systems as well, but since none of our or our partners infrastructure where IPFIXcol2 is deployed runs on *BSD, it is not our primary concern, so it will be done on a "best effort" basis rather than being guaranteed.
Regarding the issue you mentioned, I so far haven't managed to try building IPFIXcol2 on FreeBSD myself to replicate and investigate the problem, but I would try building with the -rdynamic option.
Regarding the issue you mentioned, I so far haven't managed to try building IPFIXcol2 on FreeBSD myself to replicate and investigate the problem, but I would try building with the
-rdynamicoption.
Thank you for the clue. This option ( -rdynamic ) solved the problem, but revealed the next issue. Now it looks like libepoll-shim is not correctly linked, but step by step it will probably be possible to fix it.
That's great news and really appreciate that CESNET is willing to make IPFIXcol2 deployable on {Free,Open}BSD.
I am publishing my WIP of two FreeBSD ports: net-mgmt/libfds and net-mgmt/ipfixcol2. The port of libfds seems to be complete, ipfixcol2 at least builds (mostly thanks to patches from this pull request).
Regarding the issue you mentioned, I so far haven't managed to try building IPFIXcol2 on FreeBSD myself to replicate and investigate the problem, but I would try building with the
-rdynamicoption.Thank you for the clue. This option (
-rdynamic) solved the problem, but revealed the next issue. Now it looks like libepoll-shim is not correctly linked, but step by step it will probably be possible to fix it.That's great news and really appreciate that CESNET is willing to make IPFIXcol2 deployable on {Free,Open}BSD.
I am publishing my WIP of two FreeBSD ports: net-mgmt/libfds and net-mgmt/ipfixcol2. The port of libfds seems to be complete, ipfixcol2 at least builds (mostly thanks to patches from this pull request).
I managed to get it building and running on FreeBSD by providing the needed linker flags for libepoll-shim on top of all the changes above (Note that this is just a hacky change to get it running, and not really a proper solution)
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -68,10 +68,10 @@ option(PACKAGE_BUILDER_DEB "Enable DEB package builder (make deb)" OFF)
## -----------------------------------------------------------------------------
# Hard coded definitions
-set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fvisibility=hidden -std=gnu11")
+set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -fvisibility=hidden -std=gnu11 -rdynamic -L/usr/local/lib -lepoll-shim")
set(CMAKE_C_FLAGS_RELEASE "-O2 -DNDEBUG")
set(CMAKE_C_FLAGS_DEBUG "-g -O0 -Wall -Wextra -pedantic")
-set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility=hidden -std=gnu++11")
+set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fvisibility=hidden -std=gnu++11 -rdynamic -L/usr/local/lib -lepoll-shim")
set(CMAKE_CXX_FLAGS_RELEASE "-O2 -DNDEBUG")
set(CMAKE_CXX_FLAGS_DEBUG "-g -O0 -Wall -Wextra -pedantic")
I ran into a problem with the UDP input due to some Linux-specific operations and there seem to be some other issues when building with GCC as well, but Clang + TCP seems to be working for me.
This will still require some extra work to get everything running well and to properly integrate all the necessary changes into the main codebase, but I'll try to dedicate some more time into it when possible.
I managed to get it building and running on FreeBSD by providing the needed linker flags for libepoll-shim on top of all the changes above (Note that this is just a hacky change to get it running, and not really a proper solution)
Thank you for the effort to build and fix these runtime issues on FreeBSD. I can confirm that adding these flags allows the program to run on FreeBSD. So that you know- I have updated the port accordingly.
This will still require some extra work to get everything running well and to properly integrate all the necessary changes into the main codebase, but I'll try to dedicate some more time into it when possible.
It's significant progress. If it will be able to use UDP transport then we can do more tests, without UDP working it's not easy to feed it. Finally, if the port is fine, I will eventually work on making ports of lnfstore output plugin, but it could be even more complex and more time consuming since requires porting two dependencies first:
It's significant progress. If it will be able to use UDP transport then we can do more tests, without UDP working it's not easy to feed it. Finally, if the port is fine, I will eventually work on making ports of lnfstore output plugin, but it could be even more complex and more time consuming since requires porting two dependencies first:
Regarding the UDP issue, my initial guess is that it's caused by missing "/proc/sys/net/core/rmem_max", which is used to determine maximum UDP receive buffer size here: https://github.com/CESNET/ipfixcol2/blob/869c6ce0de4026bd2d35296b34ea954e86086da2/src/plugins/input/udp/udp.c#L448
I didn't get further than that so far, but getting the value the BSD way or replacing the value with a constant might be all that's needed to get UDP working.
I didn't get further than that so far, but getting the value the BSD way or replacing the value with a constant might be all that's needed to get UDP working.
The Port was updated (patch added) to get this value in the *BSD way. Even without this update, I was able to test it in the runtime with UDP. Regardless of the patch from the above, I get plenty of read errors with the Netflow v9 feed:
ERROR: : Read operation failed! Got 1420 of 2840 bytes!
ERROR: : Read operation failed! Got 1420 of 2840 bytes!
ERROR: : Read operation failed! Got 1420 of 4260 bytes!
ERROR: : Read operation failed! Got 1420 of 2840 bytes!
So far I tested that the NetFlow feed seems to be learned by ipfixcol2 without problems from the template, and the anonymisation plugin works fine:
Set Header:
Set ID: 259 (Data Set)
Length: 1519
Template ID: 259
- Data Record (#1) [Length: 101]:
EN: 0 ID: 27 iana:sourceIPv6Address : 2301:c670:39e7:41e7:3f49:201c:cfe4:7de0
EN: 0 ID: 28 iana:destinationIPv6Address : 2700:c1ae:2a9f:d300:6b84:f113:6bd3:ced8
EN: 0 ID: 62 iana:ipNextHopIPv6Address : 29fb:c7b8:ff:ff:ff00::fe
EN: 0 ID: 10 iana:ingressInterface : 14
EN: 0 ID: 14 iana:egressInterface : 16
EN: 0 ID: 2 iana:packetDeltaCount : 1 packets
EN: 0 ID: 1 iana:octetDeltaCount : 461 octets
EN: 0 ID: 24 iana:postPacketDeltaCount : 0 packets
EN: 0 ID: 23 iana:postOctetDeltaCount : 0 octets
EN: 0 ID: 152 iana:flowStartMilliseconds : 2024-03-08T07:32:52.000Z milliseconds
EN: 0 ID: 153 iana:flowEndMilliseconds : 2024-03-08T07:32:52.000Z milliseconds
EN: 0 ID: 7 iana:sourceTransportPort : 60920
EN: 0 ID: 11 iana:destinationTransportPort : 26107
EN: 0 ID: 6 iana:tcpControlBits : 0
EN: 0 ID: 4 iana:protocolIdentifier : 17
EN: 0 ID: 5 iana:ipClassOfService : 0
EN: 0 ID: 16 iana:bgpSourceAsNumber : 0
EN: 0 ID: 17 iana:bgpDestinationAsNumber : 0
EN: 0 ID: 9 iana:sourceIPv4PrefixLength : 64 bits
EN: 0 ID: 13 iana:destinationIPv4PrefixLength : 22 bits
I didn't get further than that so far, but getting the value the BSD way or replacing the value with a constant might be all that's needed to get UDP working.
The Port was updated (patch added) to get this value in the *BSD way. Even without this update, I was able to test it in the runtime with UDP. Regardless of the patch from the above, I get plenty of read errors with the Netflow v9 feed:
ERROR: : Read operation failed! Got 1420 of 2840 bytes! ERROR: : Read operation failed! Got 1420 of 2840 bytes! ERROR: : Read operation failed! Got 1420 of 4260 bytes! ERROR: : Read operation failed! Got 1420 of 2840 bytes!So far I tested that the NetFlow feed seems to be learned by ipfixcol2 without problems from the template, and the anonymisation plugin works fine:
Set Header: Set ID: 259 (Data Set) Length: 1519 Template ID: 259 - Data Record (#1) [Length: 101]: EN: 0 ID: 27 iana:sourceIPv6Address : 2301:c670:39e7:41e7:3f49:201c:cfe4:7de0 EN: 0 ID: 28 iana:destinationIPv6Address : 2700:c1ae:2a9f:d300:6b84:f113:6bd3:ced8 EN: 0 ID: 62 iana:ipNextHopIPv6Address : 29fb:c7b8:ff:ff:ff00::fe EN: 0 ID: 10 iana:ingressInterface : 14 EN: 0 ID: 14 iana:egressInterface : 16 EN: 0 ID: 2 iana:packetDeltaCount : 1 packets EN: 0 ID: 1 iana:octetDeltaCount : 461 octets EN: 0 ID: 24 iana:postPacketDeltaCount : 0 packets EN: 0 ID: 23 iana:postOctetDeltaCount : 0 octets EN: 0 ID: 152 iana:flowStartMilliseconds : 2024-03-08T07:32:52.000Z milliseconds EN: 0 ID: 153 iana:flowEndMilliseconds : 2024-03-08T07:32:52.000Z milliseconds EN: 0 ID: 7 iana:sourceTransportPort : 60920 EN: 0 ID: 11 iana:destinationTransportPort : 26107 EN: 0 ID: 6 iana:tcpControlBits : 0 EN: 0 ID: 4 iana:protocolIdentifier : 17 EN: 0 ID: 5 iana:ipClassOfService : 0 EN: 0 ID: 16 iana:bgpSourceAsNumber : 0 EN: 0 ID: 17 iana:bgpDestinationAsNumber : 0 EN: 0 ID: 9 iana:sourceIPv4PrefixLength : 64 bits EN: 0 ID: 13 iana:destinationIPv4PrefixLength : 22 bits
Thanks for the patch. I had some time to look into it today and I think I got UDP working. I also made a new branch off this one to include these additional changes on top of: https://github.com/CESNET/ipfixcol2/tree/bsd-support
Thanks for the patch. I had some time to look into it today and I think I got UDP working. I also made a new branch off this one to include these additional changes on top of: https://github.com/CESNET/ipfixcol2/tree/bsd-support
Thank you for working on this. Now, the UDP plugin seems to be working fine. I have created another port to test the new branch more flexibly net-mgmt/ipfixcol2-devel.
Are you planning any merges between bsd-support and the main branch?
Yeah, I'm planning to get this merged into master, I just want to go through all the changes and make sure everything is done properly, as some of the changes I made were more of a hack just to get it working.
Getting rid of fdsdump wasn't my intention, that's only because I based my branch off the branch from this pull request that's several years old at this point. I just haven't gotten to rebasing this on top of current master yet.