fluentd
fluentd copied to clipboard
Published
20 hours ago •
fluent
fluent
Bind to all network interfaces for both IPv4 and IPv6
Describe the bug
Desired outcome
Run an input plugin, be it tcp or http, binding to all network interfaces, for both IPv4 and IPv6. This is a docker environment, running the latest fluentd DockerHub image (1.16.2-1, at the time of this writing).
Configuration
<source>
@type forward
bind "[::]"
port 12345
</source>
Result
➜ /tmp drit --rm -v /tmp/f2.conf:/f.conf fluentd -c /f.conf
fluentd -c /f.conf
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: parsing config file is succeeded path="/f.conf"
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: gem 'fluentd' version '1.16.2'
2024-07-15 17:00:45 +0000 [trace]: fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 17:00:45 +0000 [trace]: fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 17:00:45 +0000 [debug]: fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: using configuration file: <ROOT>
<system>
log_level trace
</system>
<source>
@type forward
bind ""
port 12345
</source>
</ROOT>
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: starting fluentd-1.16.2 pid=7 ruby="3.1.5"
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: spawn command to main: cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/f.conf", "--plugin", "/fluentd/plugins", "--under-supervisor"]
2024-07-15 17:00:45 +0000 [info]: #0 fluent/log.rb:362:info: init worker0 logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: adding source type="forward"
2024-07-15 17:00:45 +0000 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 17:00:45 +0000 [trace]: #0 fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 17:00:45 +0000 [debug]: #0 fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 17:00:45 +0000 [info]: #0 fluent/log.rb:362:info: starting fluentd worker pid=16 ppid=7 worker=0
2024-07-15 17:00:45 +0000 [info]: #0 fluent/log.rb:362:info: listening port port=12345 bind=""
2024-07-15 17:00:45 +0000 [info]: #0 fluent/log.rb:362:info: fluentd worker is now running worker=0
^C2024-07-15 17:00:49 +0000 [debug]: #0 fluent/log.rb:341:debug: fluentd main process get SIGINT
2024-07-15 17:00:49 +0000 [info]: fluent/log.rb:362:info: Received graceful stop
2024-07-15 17:00:50 +0000 [debug]: #0 fluent/log.rb:341:debug: fluentd main process get SIGTERM
2024-07-15 17:00:50 +0000 [debug]: #0 fluent/log.rb:341:debug: getting start to shutdown main process
2024-07-15 17:00:50 +0000 [info]: #0 fluent/log.rb:362:info: fluentd worker is now stopping worker=0
2024-07-15 17:00:50 +0000 [info]: #0 fluent/log.rb:362:info: shutting down fluentd worker worker=0
2024-07-15 17:00:50 +0000 [debug]: #0 fluent/log.rb:341:debug: calling stop on input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:50 +0000 [debug]: #0 fluent/log.rb:341:debug: preparing shutdown input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:50 +0000 [info]: #0 fluent/log.rb:362:info: shutting down input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:50 +0000 [debug]: #0 fluent/log.rb:341:debug: calling after_shutdown on input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:51 +0000 [debug]: #0 fluent/log.rb:341:debug: closing input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:51 +0000 [debug]: #0 fluent/log.rb:341:debug: calling terminate on input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:51 +0000 [info]: fluent/log.rb:362:info: Worker 0 finished with status 0
➜ /tmp vi f2.conf
➜ /tmp drit --rm -v /tmp/f2.conf:/f.conf fluentd -c /f.conf
fluentd -c /f.conf
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: parsing config file is succeeded path="/f.conf"
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: gem 'fluentd' version '1.16.2'
2024-07-15 17:01:01 +0000 [trace]: fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 17:01:01 +0000 [trace]: fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 17:01:01 +0000 [debug]: fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: using configuration file: <ROOT>
<system>
log_level trace
</system>
<source>
@type forward
bind "[::]"
port 12345
</source>
</ROOT>
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: starting fluentd-1.16.2 pid=7 ruby="3.1.5"
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: spawn command to main: cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/f.conf", "--plugin", "/fluentd/plugins", "--under-supervisor"]
2024-07-15 17:01:02 +0000 [info]: #0 fluent/log.rb:362:info: init worker0 logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 17:01:02 +0000 [info]: fluent/log.rb:362:info: adding source type="forward"
2024-07-15 17:01:02 +0000 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 17:01:02 +0000 [trace]: #0 fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 17:01:02 +0000 [debug]: #0 fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 17:01:02 +0000 [info]: #0 fluent/log.rb:362:info: starting fluentd worker pid=16 ppid=7 worker=0
2024-07-15 17:01:02 +0000 [info]: #0 fluent/log.rb:362:info: listening port port=12345 bind="[::]"
2024-07-15 17:01:02 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve"
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `tcp'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `server_create_tcp_socket'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:212:in `server_create_for_tcp_connection'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:92:in `server_create_connection'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:172:in `start'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `each'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `block in lifecycle'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `each'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `lifecycle'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:202:in `start'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:248:in `start'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:147:in `run'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:617:in `block in run_worker'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:962:in `main_process'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:608:in `run_worker'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/bin/fluentd:15:in `<top (required)>'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `load'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `<main>'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve"
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `tcp'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `server_create_tcp_socket'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:212:in `server_create_for_tcp_connection'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:92:in `server_create_connection'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:172:in `start'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `each'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `block in lifecycle'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `each'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `lifecycle'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:202:in `start'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:248:in `start'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:147:in `run'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:617:in `block in run_worker'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:962:in `main_process'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:608:in `run_worker'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/bin/fluentd:15:in `<top (required)>'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/bin/fluentd:25:in `load'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/bin/fluentd:25:in `<main>'
2024-07-15 17:01:02 +0000 [error]: fluent/log.rb:404:error: Worker 0 exited unexpectedly with status 1
the error is getaddrinfo: Name does not resolve in lib/fluent/plugin_helper/server.rb.
Attempting to use bind "::" allows the server to start, but only listens on IPv6.
Additional notes
The same problem error when using an http type input plugin, for instance @type monitor_agent, along with `bind "[::]".
In this case, a slightly stranger thing happens; the stacktrace shows:
2024-07-15 17:05:42 +0000 [debug]: #0 fluent/log.rb:341:debug: listening monitoring http server on http://[::]:12345/api/plugins for worker0
2024-07-15 17:05:42 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve"
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:227:in `getaddrinfo'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:227:in `foreach'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:763:in `tcp_server_sockets'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/utils.rb:60:in `create_listeners'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/server.rb:130:in `listen'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/server.rb:111:in `initialize'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/httpserver.rb:47:in `initialize'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server/compat/server.rb:47:in `new'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server/compat/server.rb:47:in `initialize'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server.rb:69:in `new'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server.rb:69:in `http_server_create_http_server'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_monitor_agent.rb:213:in `start'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start'
[...]
Please note that the webrick gem is used, for some reason, instead of async-http - which, in the code, seems to be intended for compatibility with older Ruby versions - see https://github.com/fluent/fluentd/blob/6d53664d34113381f1c3048c2569ff9e405b456b/lib/fluent/plugin_helper/http_server.rb#L21 and commit f458fae29717ee75840ce60972638818803f2d78
For the async-http, at least, the "[::]" has the intended outcome (i.e. both IPv4 and IPv6 listeners on all interfaces); the following code:
require 'async'
require 'async/http/server'
require 'async/http/endpoint'
[...]
endpoint = Async::HTTP::Endpoint.parse('http://[::]:12345')
server = Async::HTTP::Server.new(app, endpoint)
To Reproduce
Create a simple configuration file fluent-test-bind.conf:
<system>
log_level trace
</system>
<source>
@type forward
bind "[::]"
port 12345
</source>
Start a container, using the newly created config file: docker run -it --rm -v ./fluent-test-bind.conf:/fluent-test-bind.conf fluentd -c /fluent-test-bind.conf
The error (and stack traces) appear on stdout:
fluentd -c /f.conf
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: parsing config file is succeeded path="/f.conf"
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: gem 'fluentd' version '1.16.2'
2024-07-15 18:03:15 +0000 [trace]: fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 18:03:15 +0000 [trace]: fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 18:03:15 +0000 [debug]: fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: using configuration file: <ROOT>
<system>
log_level trace
</system>
<source>
@type forward
bind "[::]"
port 12345
</source>
</ROOT>
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: starting fluentd-1.16.2 pid=7 ruby="3.1.5"
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: spawn command to main: cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/f.conf", "--plugin", "/fluentd/plugins", "--under-supervisor"]
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: init worker0 logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: adding source type="forward"
2024-07-15 18:03:15 +0000 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 18:03:15 +0000 [trace]: #0 fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 18:03:15 +0000 [debug]: #0 fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: starting fluentd worker pid=16 ppid=7 worker=0
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: listening port port=12345 bind="[::]"
2024-07-15 18:03:15 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve"
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `tcp'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `server_create_tcp_socket'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:212:in `server_create_for_tcp_connection'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:92:in `server_create_connection'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:172:in `start'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `each'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `block in lifecycle'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `each'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `lifecycle'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:202:in `start'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:248:in `start'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:147:in `run'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:617:in `block in run_worker'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:962:in `main_process'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:608:in `run_worker'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/bin/fluentd:15:in `<top (required)>'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `load'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `<main>'
Expected behavior
It would be desirable to be able to use a single listener entry in the configuration file to listen on both IPv4 and IPv6 interfaces.
It is also expected that bind addresses of the type "[::]" or "[::1]" are allowed - they are permitted by Async::HTTP::Endpoint.parse(), as indicated above.
Your Environment
- Fluentd version: `1.16.2`
- Package version: `fluentd:v1.16.2-1.1`
- Operating system: `MacOS` / `Docker Desktop 4.32.0`
- Kernel version: `N/A`
Your Configuration
<system>
log_level trace
</system>
<source>
@type forward
bind "[::]"
port 12345
</source>
Your Error Log
fluentd -c /f.conf
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: parsing config file is succeeded path="/f.conf"
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: gem 'fluentd' version '1.16.2'
2024-07-15 18:03:15 +0000 [trace]: fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 18:03:15 +0000 [trace]: fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 18:03:15 +0000 [debug]: fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: using configuration file: <ROOT>
<system>
log_level trace
</system>
<source>
@type forward
bind "[::]"
port 12345
</source>
</ROOT>
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: starting fluentd-1.16.2 pid=7 ruby="3.1.5"
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: spawn command to main: cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/f.conf", "--plugin", "/fluentd/plugins", "--under-supervisor"]
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: init worker0 logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: adding source type="forward"
2024-07-15 18:03:15 +0000 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 18:03:15 +0000 [trace]: #0 fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 18:03:15 +0000 [debug]: #0 fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: starting fluentd worker pid=16 ppid=7 worker=0
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: listening port port=12345 bind="[::]"
2024-07-15 18:03:15 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve"
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `tcp'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `server_create_tcp_socket'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:212:in `server_create_for_tcp_connection'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:92:in `server_create_connection'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:172:in `start'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `each'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `block in lifecycle'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `each'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `lifecycle'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:202:in `start'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:248:in `start'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:147:in `run'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:617:in `block in run_worker'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:962:in `main_process'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:608:in `run_worker'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/bin/fluentd:15:in `<top (required)>'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `load'
2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `<main>'
### Additional context
The same problem error when using an http type input plugin, for instance `@type monitor_agent`, along with `bind "[::]".
In this case, a slightly stranger thing happens; the stacktrace shows:
2024-07-15 17:05:42 +0000 [debug]: #0 fluent/log.rb:341:debug: listening monitoring http server on http://[::]:12345/api/plugins for worker0
2024-07-15 17:05:42 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve"
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:227:in getaddrinfo' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:227:in foreach'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:763:in tcp_server_sockets' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/utils.rb:60:in create_listeners'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/server.rb:130:in listen' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/server.rb:111:in initialize'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/httpserver.rb:47:in initialize' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server/compat/server.rb:47:in new'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server/compat/server.rb:47:in initialize' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server.rb:69:in new'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server.rb:69:in http_server_create_http_server' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_monitor_agent.rb:213:in start'
2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start'
[...]
Please note that the `webrick` gem is used, for some reason, instead of `async-http` - which, in the code, seems to be intended for compatibility with older Ruby versions - see https://github.com/fluent/fluentd/blob/6d53664d34113381f1c3048c2569ff9e405b456b/lib/fluent/plugin_helper/http_server.rb#L21 and commit f458fae29717ee75840ce60972638818803f2d78
For the `async-http`, at least, the "[::]" has the intended outcome (i.e. both IPv4 and IPv6 listeners on all interfaces); the following code:
```ruby
require 'async'
require 'async/http/server'
require 'async/http/endpoint'
[...]
endpoint = Async::HTTP::Endpoint.parse('http://[::]:12345')
server = Async::HTTP::Server.new(app, endpoint)
Also, the same problem is encountered for external plugins, such as fluentd-plugin-prometheus. Example config:
<system>
log_level trace
</system>
<source>
@type prometheus
bind "[::]"
port 9000
metrics_path /metrics
</source>
