fluentd icon indicating copy to clipboard operation
fluentd copied to clipboard
verified publisher icon fluent

Bind to all network interfaces for both IPv4 and IPv6

Open inistor opened this issue 1 year ago • 16 comments

Describe the bug

Desired outcome

Run an input plugin, be it tcp or http, binding to all network interfaces, for both IPv4 and IPv6. This is a docker environment, running the latest fluentd DockerHub image (1.16.2-1, at the time of this writing).

Configuration

  <source>
    @type forward
    bind "[::]"
    port 12345
  </source>

Result

➜  /tmp drit --rm -v /tmp/f2.conf:/f.conf fluentd -c /f.conf
fluentd -c /f.conf
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: parsing config file is succeeded path="/f.conf"
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: gem 'fluentd' version '1.16.2'
2024-07-15 17:00:45 +0000 [trace]: fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 17:00:45 +0000 [trace]: fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 17:00:45 +0000 [debug]: fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: using configuration file: <ROOT>
  <system>
    log_level trace
  </system>
  <source>
    @type forward
    bind ""
    port 12345
  </source>
</ROOT>
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: starting fluentd-1.16.2 pid=7 ruby="3.1.5"
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: spawn command to main:  cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/f.conf", "--plugin", "/fluentd/plugins", "--under-supervisor"]
2024-07-15 17:00:45 +0000 [info]: #0 fluent/log.rb:362:info: init worker0 logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 17:00:45 +0000 [info]: fluent/log.rb:362:info: adding source type="forward"
2024-07-15 17:00:45 +0000 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 17:00:45 +0000 [trace]: #0 fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 17:00:45 +0000 [debug]: #0 fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 17:00:45 +0000 [info]: #0 fluent/log.rb:362:info: starting fluentd worker pid=16 ppid=7 worker=0
2024-07-15 17:00:45 +0000 [info]: #0 fluent/log.rb:362:info: listening port port=12345 bind=""
2024-07-15 17:00:45 +0000 [info]: #0 fluent/log.rb:362:info: fluentd worker is now running worker=0
^C2024-07-15 17:00:49 +0000 [debug]: #0 fluent/log.rb:341:debug: fluentd main process get SIGINT
2024-07-15 17:00:49 +0000 [info]: fluent/log.rb:362:info: Received graceful stop
2024-07-15 17:00:50 +0000 [debug]: #0 fluent/log.rb:341:debug: fluentd main process get SIGTERM
2024-07-15 17:00:50 +0000 [debug]: #0 fluent/log.rb:341:debug: getting start to shutdown main process
2024-07-15 17:00:50 +0000 [info]: #0 fluent/log.rb:362:info: fluentd worker is now stopping worker=0
2024-07-15 17:00:50 +0000 [info]: #0 fluent/log.rb:362:info: shutting down fluentd worker worker=0
2024-07-15 17:00:50 +0000 [debug]: #0 fluent/log.rb:341:debug: calling stop on input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:50 +0000 [debug]: #0 fluent/log.rb:341:debug: preparing shutdown input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:50 +0000 [info]: #0 fluent/log.rb:362:info: shutting down input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:50 +0000 [debug]: #0 fluent/log.rb:341:debug: calling after_shutdown on input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:51 +0000 [debug]: #0 fluent/log.rb:341:debug: closing input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:51 +0000 [debug]: #0 fluent/log.rb:341:debug: calling terminate on input plugin type=:forward plugin_id="object:85c"
2024-07-15 17:00:51 +0000 [info]: fluent/log.rb:362:info: Worker 0 finished with status 0
➜  /tmp vi f2.conf 
➜  /tmp drit --rm -v /tmp/f2.conf:/f.conf fluentd -c /f.conf
fluentd -c /f.conf
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: parsing config file is succeeded path="/f.conf"
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: gem 'fluentd' version '1.16.2'
2024-07-15 17:01:01 +0000 [trace]: fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 17:01:01 +0000 [trace]: fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 17:01:01 +0000 [debug]: fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: using configuration file: <ROOT>
  <system>
    log_level trace
  </system>
  <source>
    @type forward
    bind "[::]"
    port 12345
  </source>
</ROOT>
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: starting fluentd-1.16.2 pid=7 ruby="3.1.5"
2024-07-15 17:01:01 +0000 [info]: fluent/log.rb:362:info: spawn command to main:  cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/f.conf", "--plugin", "/fluentd/plugins", "--under-supervisor"]
2024-07-15 17:01:02 +0000 [info]: #0 fluent/log.rb:362:info: init worker0 logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 17:01:02 +0000 [info]: fluent/log.rb:362:info: adding source type="forward"
2024-07-15 17:01:02 +0000 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 17:01:02 +0000 [trace]: #0 fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 17:01:02 +0000 [debug]: #0 fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 17:01:02 +0000 [info]: #0 fluent/log.rb:362:info: starting fluentd worker pid=16 ppid=7 worker=0
2024-07-15 17:01:02 +0000 [info]: #0 fluent/log.rb:362:info: listening port port=12345 bind="[::]"
2024-07-15 17:01:02 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve"
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `tcp'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `server_create_tcp_socket'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:212:in `server_create_for_tcp_connection'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:92:in `server_create_connection'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:172:in `start'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `each'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `block in lifecycle'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `each'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `lifecycle'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:202:in `start'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:248:in `start'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:147:in `run'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:617:in `block in run_worker'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:962:in `main_process'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:608:in `run_worker'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/bin/fluentd:15:in `<top (required)>'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `load'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `<main>'
2024-07-15 17:01:02 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve"
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `tcp'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `server_create_tcp_socket'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:212:in `server_create_for_tcp_connection'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:92:in `server_create_connection'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:172:in `start'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `each'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `block in lifecycle'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `each'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `lifecycle'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:202:in `start'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:248:in `start'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:147:in `run'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:617:in `block in run_worker'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:962:in `main_process'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:608:in `run_worker'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/bin/fluentd:15:in `<top (required)>'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/bin/fluentd:25:in `load'
  2024-07-15 17:01:02 +0000 [error]: #0 fluent/supervisor.rb:961:main_process: /usr/bin/fluentd:25:in `<main>'
2024-07-15 17:01:02 +0000 [error]: fluent/log.rb:404:error: Worker 0 exited unexpectedly with status 1

the error is getaddrinfo: Name does not resolve in lib/fluent/plugin_helper/server.rb.

Attempting to use bind "::" allows the server to start, but only listens on IPv6.

Additional notes

The same problem error when using an http type input plugin, for instance @type monitor_agent, along with `bind "[::]". In this case, a slightly stranger thing happens; the stacktrace shows:

2024-07-15 17:05:42 +0000 [debug]: #0 fluent/log.rb:341:debug: listening monitoring http server on http://[::]:12345/api/plugins for worker0
2024-07-15 17:05:42 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve"
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:227:in `getaddrinfo'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:227:in `foreach'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:763:in `tcp_server_sockets'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/utils.rb:60:in `create_listeners'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/server.rb:130:in `listen'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/server.rb:111:in `initialize'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/httpserver.rb:47:in `initialize'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server/compat/server.rb:47:in `new'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server/compat/server.rb:47:in `initialize'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server.rb:69:in `new'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server.rb:69:in `http_server_create_http_server'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_monitor_agent.rb:213:in `start'
  2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start'
[...]

Please note that the webrick gem is used, for some reason, instead of async-http - which, in the code, seems to be intended for compatibility with older Ruby versions - see https://github.com/fluent/fluentd/blob/6d53664d34113381f1c3048c2569ff9e405b456b/lib/fluent/plugin_helper/http_server.rb#L21 and commit f458fae29717ee75840ce60972638818803f2d78

For the async-http, at least, the "[::]" has the intended outcome (i.e. both IPv4 and IPv6 listeners on all interfaces); the following code:

require 'async'
require 'async/http/server'
require 'async/http/endpoint'
[...]
endpoint = Async::HTTP::Endpoint.parse('http://[::]:12345')
server = Async::HTTP::Server.new(app, endpoint)

To Reproduce

Create a simple configuration file fluent-test-bind.conf:

<system>
  log_level trace
</system>

<source>
  @type forward
  bind "[::]"
  port 12345
</source>

Start a container, using the newly created config file: docker run -it --rm -v ./fluent-test-bind.conf:/fluent-test-bind.conf fluentd -c /fluent-test-bind.conf

The error (and stack traces) appear on stdout:

fluentd -c /f.conf
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: parsing config file is succeeded path="/f.conf"
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: gem 'fluentd' version '1.16.2'
2024-07-15 18:03:15 +0000 [trace]: fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 18:03:15 +0000 [trace]: fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 18:03:15 +0000 [debug]: fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: using configuration file: <ROOT>
  <system>
    log_level trace
  </system>
  <source>
    @type forward
    bind "[::]"
    port 12345
  </source>
</ROOT>
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: starting fluentd-1.16.2 pid=7 ruby="3.1.5"
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: spawn command to main:  cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/f.conf", "--plugin", "/fluentd/plugins", "--under-supervisor"]
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: init worker0 logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: adding source type="forward"
2024-07-15 18:03:15 +0000 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 18:03:15 +0000 [trace]: #0 fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 18:03:15 +0000 [debug]: #0 fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: starting fluentd worker pid=16 ppid=7 worker=0
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: listening port port=12345 bind="[::]"
2024-07-15 18:03:15 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve"
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `tcp'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `server_create_tcp_socket'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:212:in `server_create_for_tcp_connection'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:92:in `server_create_connection'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:172:in `start'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `each'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `block in lifecycle'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `each'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `lifecycle'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:202:in `start'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:248:in `start'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:147:in `run'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:617:in `block in run_worker'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:962:in `main_process'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:608:in `run_worker'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/bin/fluentd:15:in `<top (required)>'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `load'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `<main>'

Expected behavior

It would be desirable to be able to use a single listener entry in the configuration file to listen on both IPv4 and IPv6 interfaces.

It is also expected that bind addresses of the type "[::]" or "[::1]" are allowed - they are permitted by Async::HTTP::Endpoint.parse(), as indicated above.

Your Environment

- Fluentd version: `1.16.2`
- Package version: `fluentd:v1.16.2-1.1`
- Operating system: `MacOS` / `Docker Desktop 4.32.0`
- Kernel version: `N/A`

Your Configuration

<system>
  log_level trace
</system>

<source>
  @type forward
  bind "[::]"
  port 12345
</source>

Your Error Log

fluentd -c /f.conf
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: parsing config file is succeeded path="/f.conf"
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: gem 'fluentd' version '1.16.2'
2024-07-15 18:03:15 +0000 [trace]: fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 18:03:15 +0000 [trace]: fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 18:03:15 +0000 [debug]: fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: using configuration file: <ROOT>
  <system>
    log_level trace
  </system>
  <source>
    @type forward
    bind "[::]"
    port 12345
  </source>
</ROOT>
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: starting fluentd-1.16.2 pid=7 ruby="3.1.5"
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: spawn command to main:  cmdline=["/usr/bin/ruby", "-Eascii-8bit:ascii-8bit", "/usr/bin/fluentd", "-c", "/f.conf", "--plugin", "/fluentd/plugins", "--under-supervisor"]
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: init worker0 logger path=nil rotate_age=nil rotate_size=nil
2024-07-15 18:03:15 +0000 [info]: fluent/log.rb:362:info: adding source type="forward"
2024-07-15 18:03:15 +0000 [trace]: #0 fluent/log.rb:319:trace: registered input plugin 'forward'
2024-07-15 18:03:15 +0000 [trace]: #0 fluent/log.rb:319:trace: registered metrics plugin 'local'
2024-07-15 18:03:15 +0000 [debug]: #0 fluent/log.rb:341:debug: No fluent logger for internal event
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: starting fluentd worker pid=16 ppid=7 worker=0
2024-07-15 18:03:15 +0000 [info]: #0 fluent/log.rb:362:info: listening port port=12345 bind="[::]"
2024-07-15 18:03:15 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve"
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `tcp'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:376:in `server_create_tcp_socket'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:212:in `server_create_for_tcp_connection'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/server.rb:92:in `server_create_connection'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_forward.rb:172:in `start'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `each'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:191:in `block in lifecycle'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `each'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:178:in `lifecycle'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:202:in `start'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:248:in `start'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/engine.rb:147:in `run'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:617:in `block in run_worker'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:962:in `main_process'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/supervisor.rb:608:in `run_worker'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: <internal:/usr/lib/ruby/3.1.0/rubygems/core_ext/kernel_require.rb>:85:in `require'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/bin/fluentd:15:in `<top (required)>'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `load'
  2024-07-15 18:03:15 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/bin/fluentd:25:in `<main>'



### Additional context

The same problem error when using an http type input plugin, for instance `@type monitor_agent`, along with `bind "[::]". 
In this case, a slightly stranger thing happens; the stacktrace shows:

2024-07-15 17:05:42 +0000 [debug]: #0 fluent/log.rb:341:debug: listening monitoring http server on http://[::]:12345/api/plugins for worker0 2024-07-15 17:05:42 +0000 [error]: #0 fluent/log.rb:404:error: unexpected error error_class=SocketError error="getaddrinfo: Name does not resolve" 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:227:in getaddrinfo' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:227:in foreach' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/3.1.0/socket.rb:763:in tcp_server_sockets' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/utils.rb:60:in create_listeners' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/server.rb:130:in listen' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/server.rb:111:in initialize' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/webrick-1.7.0/lib/webrick/httpserver.rb:47:in initialize' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server/compat/server.rb:47:in new' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server/compat/server.rb:47:in initialize' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server.rb:69:in new' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin_helper/http_server.rb:69:in http_server_create_http_server' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/plugin/in_monitor_agent.rb:213:in start' 2024-07-15 17:05:42 +0000 [error]: #0 fluent/supervisor.rb:962:main_process: /usr/lib/ruby/gems/3.1.0/gems/fluentd-1.16.2/lib/fluent/root_agent.rb:203:in `block in start' [...]


Please note that the `webrick` gem is used, for some reason, instead of `async-http` - which, in the code, seems to be intended for compatibility with older Ruby versions - see https://github.com/fluent/fluentd/blob/6d53664d34113381f1c3048c2569ff9e405b456b/lib/fluent/plugin_helper/http_server.rb#L21 and commit f458fae29717ee75840ce60972638818803f2d78

For the `async-http`, at least, the "[::]" has the intended outcome (i.e. both IPv4 and IPv6 listeners on all interfaces); the following code:
```ruby
require 'async'
require 'async/http/server'
require 'async/http/endpoint'
[...]
endpoint = Async::HTTP::Endpoint.parse('http://[::]:12345')
server = Async::HTTP::Server.new(app, endpoint)

Also, the same problem is encountered for external plugins, such as fluentd-plugin-prometheus. Example config:

<system>
  log_level trace
</system>

<source>
  @type prometheus
  bind "[::]"
  port 9000
  metrics_path /metrics
</source>

inistor avatar Jul 15 '24 18:07 inistor

reproduced with v1.16.5 and v1.17.0.

kenhys avatar Jul 17 '24 09:07 kenhys

Looks like ipv6 is not supported in case of single worker.

https://github.com/fluent/fluentd/blob/v1.17.0/lib/fluent/plugin_helper/server.rb#L370-L383

daipom avatar Jul 18 '24 02:07 daipom

Looks like ipv6 is not supported in case of single worker.

https://github.com/fluent/fluentd/blob/v1.17.0/lib/fluent/plugin_helper/server.rb#L370-L383

Hmm, we can't add brackets?

Addrinfo.tcp("::", 20000)
=> #<Addrinfo: [::]:20000 TCP>
Addrinfo.tcp("[::]", 20000)
(irb):1:in `tcp': getaddrinfo: Name or service not known (SocketError)
	from (irb):1:in `<main>'
	from /home/daipom/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/gems/irb-1.13.1/exe/irb:9:in `<top (required)>'
	from /home/daipom/.rbenv/versions/3.2.2/bin/irb:25:in `load'
	from /home/daipom/.rbenv/versions/3.2.2/bin/irb:25:in `<main>'

daipom avatar Jul 18 '24 03:07 daipom

Hmm, we can't add brackets?

This works.

  <source>
    @type forward
    bind "::"
    port 12345
  </source>

daipom avatar Jul 18 '24 03:07 daipom

Hmm, we can't add brackets?

This works.

  <source>
    @type forward
    bind "::"
    port 12345
  </source>

@inistor Please try an address with no brackets.

daipom avatar Jul 18 '24 03:07 daipom

Already did - it only listens on IPv6:

f.conf:

  <source>
    @type forward
    bind "::"
    port 12345
  </source>

➜  /tmp docker run -it --rm --name fluentd-ipv6-test -v /tmp/f.conf:/f.conf fluentd:v1.16-1 -c /f.conf
fluentd -c /f.conf
2024-07-18 05:30:38 +0000 [info]: init supervisor logger path=nil rotate_age=nil rotate_size=nil
2024-07-18 05:30:38 +0000 [info]: parsing config file is succeeded path="/f.conf"
2024-07-18 05:30:38 +0000 [info]: gem 'fluentd' version '1.16.2'
2024-07-18 05:30:38 +0000 [info]: using configuration file: <ROOT>
  <source>
    @type forward
    bind "::"
    port 12345
  </source>
</ROOT>
2024-07-18 05:30:38 +0000 [info]: starting fluentd-1.16.2 pid=6 ruby="3.1.5"
[...]

➜  ~ docker exec -it fluentd-ipv6-test sh
/ $ nc -vz 127.0.0.1 12345
/ $ nc -vz ::1 12345
::1 ([::1]:12345) open
/ $

And the reason appears to be the way the URL is formed. At least from Async::HTTP::Endpoint.parse()s point of view:

  • endpoint = Async::HTTP::Endpoint.parse('http://[::]:12345') - listens on both IPv4 and IPv6
  • endpoint = Async::HTTP::Endpoint.parse('http://:::12345') - listens on IPv6 only

True, in the case of @type forward, Async::HTTP::Endpoint is not involved (it's a TCP socket, not HTTP), but I believe the underlying socket API is the same (my Ruby is quite Rusty - forgive the pun ;-) ).

inistor avatar Jul 18 '24 05:07 inistor

Please see my report from 2020: #3001

inistor avatar Jul 18 '24 05:07 inistor

Any comments on this one? Is there any way I may be of assistance?

inistor avatar Jul 31 '24 07:07 inistor

I see. So we need this feature.

It would be desirable to be able to use a single listener entry in the configuration file to listen on both IPv4 and IPv6 interfaces.

daipom avatar Jul 31 '24 09:07 daipom

Well, it would seem that in 1.17.1 the bind "[::]" syntax is working as desired, namely it works and listens to both IPv4 and IPv6. Could anyone else please confirm - maybe also refer the PR/Issue where this was addressed? Thanks!

inistor avatar Sep 16 '24 10:09 inistor

Hmm, I don't think this issue is solved. bind "[::]" causes an error. [::] would be the wrong format.

<source>
  @type forward
  bind "[::]"
  port 12345
</source>

<match test.**>
  @type stdout
</match>

2024-09-26 13:20:28 +0900 [info]: #0 starting fluentd worker pid=334089 ppid=334062 worker=0
2024-09-26 13:20:28 +0900 [info]: #0 listening port port=12345 bind="[::]"
2024-09-26 13:20:28 +0900 [error]: #0 unexpected error error_class=SocketError error="getaddrinfo: Name or service not known"
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/server.rb:376:in `tcp'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/server.rb:376:in `server_create_tcp_socket'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/server.rb:212:in `server_create_for_tcp_connection'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin_helper/server.rb:92:in `server_create_connection'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/plugin/in_forward.rb:172:in `start'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/root_agent.rb:203:in `block in start'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/root_agent.rb:192:in `block (2 levels) in lifecycle'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/root_agent.rb:191:in `each'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/root_agent.rb:191:in `block in lifecycle'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/root_agent.rb:178:in `each'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/root_agent.rb:178:in `lifecycle'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/root_agent.rb:202:in `start'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/engine.rb:248:in `start'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/engine.rb:147:in `run'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/supervisor.rb:618:in `block in run_worker'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/supervisor.rb:964:in `main_process'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/supervisor.rb:609:in `run_worker'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/lib/fluent/command/fluentd.rb:372:in `<top (required)>'
  2024-09-26 13:20:28 +0900 [error]: #0 <internal:/home/daipom/.rbenv/versions/3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require'
  2024-09-26 13:20:28 +0900 [error]: #0 <internal:/home/daipom/.rbenv/versions/3.2.2/lib/ruby/site_ruby/3.2.0/rubygems/core_ext/kernel_require.rb>:38:in `require'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/work/fluentd/fluentd/bin/fluentd:15:in `<top (required)>'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/bin/fluentd:25:in `load'
  2024-09-26 13:20:28 +0900 [error]: #0 /home/daipom/.rbenv/versions/3.2.2/lib/ruby/gems/3.2.0/bin/fluentd:25:in `<main>'

daipom avatar Sep 26 '24 04:09 daipom

Hi,

We are facing this issue on fluentd 1.17.0. The fluentd service is not reachable on ipV6 lab. Tried using bind values "::" and "[::]" as suggested, but this too doesn't work.

<source>
 @type forward
 port 24224
 bind "[::]"
 tag forwarder
</source> 
<source>
 @type prometheus
 bind "[::]"
 port 24231
 metrics_path /metrics
</source>

RamyaSaba avatar Dec 30 '24 05:12 RamyaSaba

5 months later, do you have news ? I have the same issue today, you can have only IPv4 or IPv6 (not both). (fluentd-1.18)

Jackngl avatar May 07 '25 23:05 Jackngl

Now, we should configure bind parameter to "[::]" for in_monitor_agent.

<source>
  @type monitor_agent
  bind "[::]"
  port 24220
</source>

However, in_forward does not accept "[::]". we should configure bind to :: for in_forward.

<source>
  @type forward
  bind "::"
  port 24224
</source>

At least, separate settings are required to allow IPv4 and IPv6 connections in in_forward so far.

<source>
  @type forward
  bind "::"
  port 24224
</source>
<source>
  @type forward
  bind "0.0.0.0"
  port 24224
</source>

Watson1978 avatar Jul 22 '25 09:07 Watson1978

Fluentd invokes Addrinfo.tcp(bind, port).listen when create TCP socket. https://github.com/fluent/fluentd/blob/cefbc62e21570c4761a78297d481adff51e36692/lib/fluent/plugin_helper/server.rb#L386-L388

Addrinfo#listen sets IPV6_V6ONLY to socket by sock.ipv6only!.

  def listen(backlog=Socket::SOMAXCONN)
    sock = Socket.new(self.pfamily, self.socktype, self.protocol)
    begin
      sock.ipv6only! if self.ipv6?

https://github.com/ruby/ruby/blob/fbaad485db4259497aeac93a296b25432c198fdd/ext/socket/lib/socket.rb#L203

Therefore, IPv4 connections are rejected even if we use :: for bind address.

# TCPServer.new doesn't set IPV6_V6ONLY flag, so use Addrinfo class instead.
# backlog will be set by the caller, we don't need to set backlog here

We have to understand the intention behind this comment.

With the following changes, in_forward will accept to connect from both ipv4 and ipv6.

diff --git a/lib/fluent/plugin_helper/server.rb b/lib/fluent/plugin_helper/server.rb
index bd22e543..7e46b87b 100644
--- a/lib/fluent/plugin_helper/server.rb
+++ b/lib/fluent/plugin_helper/server.rb
@@ -383,9 +383,11 @@ def server_create_tcp_socket(shared, bind, port)
         sock = if shared
                  server_socket_manager_client.listen_tcp(bind, port)
                else
-                 # TCPServer.new doesn't set IPV6_V6ONLY flag, so use Addrinfo class instead.
-                 # backlog will be set by the caller, we don't need to set backlog here
-                 tsock = Addrinfo.tcp(bind, port).listen
+                 addrinfo = Addrinfo.tcp(bind, port)
+                 tsock = ::Socket.new(addrinfo.pfamily, addrinfo.socktype, 0)
+                 tsock.setsockopt(::Socket::SOL_SOCKET, ::Socket::SO_REUSEADDR, 1)
+                 tsock.bind(addrinfo)
+                 tsock.listen(::Socket::SOMAXCONN)
                  tsock.autoclose = false
                  TCPServer.for_fd(tsock.fileno)
                end

Seems we need to apply similar changing into SocketManager in serverengine. https://github.com/treasure-data/serverengine/blob/bc7a7e3fb170fcc48656f7658e8f523c997487c0/lib/serverengine/socket_manager_unix.rb#L84-L109

Watson1978 avatar Jul 23 '25 05:07 Watson1978

TCPServer.new doesn't set IPV6_V6ONLY flag, so use Addrinfo class instead. backlog will be set by the caller, we don't need to set backlog here

We have to understand the intention behind this comment.

It is related to https://github.com/fluent/fluentd/issues/2682 It support ipv4 and ipv6 with separated <source> settings...

<source>
  @type forward
  bind "::"
  port 24224
</source>
<source>
  @type forward
  bind "0.0.0.0"
  port 24224
</source>

Watson1978 avatar Jul 23 '25 05:07 Watson1978