fluency icon indicating copy to clipboard operation
fluency copied to clipboard

Add support for Unix domain sockets

Open pgiblock opened this issue 2 years ago • 1 comments

It would be nice if this logger would support Unix Domain Sockets as the target instead of only (host, port) inet addresses. UnixDomainSocketAddress was added in Java 16. I had hoped InetSocketAddress, as accepted by FluencyBuilderForFluentd.build() could trivially be changed to SocketAddress as that the common base class, but Fluency is written to assume a TCP socket.

This feature would be particularly useful for container environments such as Docker because some not very nice tricks are required for a container to connect to a host socket. Note that Unix domain sockets are supported by Fluentd and Fluent-Bit.

pgiblock avatar Aug 09 '22 20:08 pgiblock

For anyone else running into this problem, there is a workaround that I find cleaner than running with host networking or having to address the host by address/hostname, bind on the interface, and implement firewall rules. It requires running a little forwarder to "proxy" the data from docker-lived networking to the host-lived domain socket:

docker run -h fluent -v /var/run/fluent-bin.sock:/sock fluent/fluent-bit:1.9 \
    -i forward -o forward -p unix_path=/sock

You'll likely need to place this on the right network(s), but docker containers can now forward to the hostname "fluent" and records will appear on the actual host.

pgiblock avatar Aug 09 '22 21:08 pgiblock

I wanted to second @pgiblock 's request - I also needed a Unix Socket and implemented one quickly:

https://gist.github.com/talwgx/418aecd95358941ba9ea905f88f25fcd

But it will be much better ofc it this was baked into this really lovely library! @komamitsu

talwgx avatar Sep 23 '22 13:09 talwgx

That looks a good workaround. The difficult point to take care of this issue is we need to build Fluency code with JDK 8 to create jar files to avoid https://www.morling.dev/blog/bytebuffer-and-the-dreaded-nosuchmethoderror/ and UnixDomainSocketAddress was introduced in JDK 16. Let me think for a while...

komamitsu avatar Sep 24 '22 06:09 komamitsu