Use ipv6 address as default when out_forward and macOS catalina

[ganmacs]

Describe the bug

out_forward tries to connect ::1 instead of 127.0.0.1 because the following code uses first value(which is ipv6 address) as the hostname.

https://github.com/fluent/fluentd/blob/8f2ea4bb26e5735b36acf5f6dc0fa5796ce4c4f9/lib/fluent/plugin/out_forward.rb#L731

https://github.com/fluent/fluentd/issues/2865#issuecomment-596321456

To Reproduce

Needs macOS catalina (I'm not sure other versions of macOS)

server

<source>
  @type forward
</source>

<match test>
  @type stdout
</match>

client

<source>
  @type dummy
  tag test
</source>

<match test>
  @type forward

  <server>
    host localhost // if it's 127.0.0.1 then it works well
    port 24224
  </server>
  flush_interval 0
</match>

getaddrinfo(3) returns the values in a different order on ubuntu and macOS.

# ubuntu:bionic
$ docker run --rm -it rubylang/ruby:2.6.5-bionic ruby -e 'require "socket"; p Socket.getaddrinfo("localhost", 80, nil, Socket::SOCK_STREAM)'
[["AF_INET", 80, "127.0.0.1", "127.0.0.1", 2, 1, 6], ["AF_INET6", 80, "::1", "::1", 10, 1, 6]]
$ docker run --rm -it rubylang/ruby:2.7.0-bionic ruby -e 'require "socket"; p Socket.getaddrinfo("localhost", 80, nil, Socket::SOCK_STREAM)'
[["AF_INET", 80, "127.0.0.1", "127.0.0.1", 2, 1, 6], ["AF_INET6", 80, "::1", "::1", 10, 1, 6]]

# macos:catalina
$ rbenv local 2.6.5; bundle exec ruby -e 'require "socket"; p Socket.getaddrinfo("localhost", 80, nil, Socket::SOCK_STREAM)'
[["AF_INET6", 80, "::1", "::1", 30, 1, 6], ["AF_INET", 80, "127.0.0.1", "127.0.0.1", 2, 1, 6]]
$ rbenv local 2.7.0; bundle exec ruby -e 'require "socket"; p Socket.getaddrinfo("localhost", 80, nil, Socket::SOCK_STREAM)'
[["AF_INET6", 80, "::1", "::1", 30, 1, 6], ["AF_INET", 80, "127.0.0.1", "127.0.0.1", 2, 1, 6]]

Expected behavior

Use ipv6 address when setting localhost to host at server section of out_forward

Your Environment

• Fluentd or td-agent version: at least 1.8.0
• Operating system:

$ sw_vers
ProductName:    Mac OS X
ProductVersion: 10.15.3
BuildVersion:   19D76

• Kernel version: 19.3.0

Your Configuration

Same as the To Reproduce

Your Error Log

2020-03-09 15:00:30 +0900 [info]: parsing config file is succeeded path="example/out_forward.conf"
2020-03-09 15:00:30 +0900 [info]: gem 'fluentd' version '1.8.0'
2020-03-09 15:00:30 +0900 [info]: 'flush_interval' is configured at out side of <buffer>. 'flush_mode' is set to 'interval' to keep existing behaviour
2020-03-09 15:00:30 +0900 [info]: adding forwarding server 'localhost:24224' host="localhost" port=24224 weight=60 plugin_id="object:eec"
2020-03-09 15:00:30 +0900 [warn]: both of Plugin @id and path for <storage> are not specified. Using on-memory store.
2020-03-09 15:00:30 +0900 [warn]: both of Plugin @id and path for <storage> are not specified. Using on-memory store.
2020-03-09 15:00:30 +0900 [info]: using configuration file: <ROOT>
  <source>
    @type dummy
    tag "test"
  </source>
  <match test>
    @type forward
    flush_interval 0
    <service>
      host "localhost"
      port 24224
    </service>
    <buffer tag>
      flush_interval 0
    </buffer>
  </match>
</ROOT>
2020-03-09 15:00:30 +0900 [info]: starting fluentd-1.8.0 pid=6735 ruby="2.7.0"
2020-03-09 15:00:30 +0900 [info]: spawn command to main:  cmdline=["/Users/yuta.iwama/.rbenv/versions/2.7.0/bin/ruby", "-Eascii-8bit:ascii-8bit", "-r/Users/yuta.iwama/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/gems/bundler-2.1.4/lib/bundler/setup", "/Users/yuta.iwama/.rbenv/versions/2.7.0/lib/ruby/gems/2.7.0/bin/fluentd", "-c", "example/out_forward.conf", "--under-supervisor"]
/Users/yuta.iwama/src/github.com/fluent/fluentd/lib/fluent/supervisor.rb:796: warning: Using the last argument as keyword parameters is deprecated; maybe ** should be added to the call
/Users/yuta.iwama/src/github.com/fluent/fluentd/lib/fluent/system_config.rb:132: warning: The called method `overwrite_variables' is defined here
2020-03-09 15:00:31 +0900 [info]: adding match pattern="test" type="forward"
2020-03-09 15:00:31 +0900 [info]: #0 'flush_interval' is configured at out side of <buffer>. 'flush_mode' is set to 'interval' to keep existing behaviour
2020-03-09 15:00:31 +0900 [info]: #0 adding forwarding server 'localhost:24224' host="localhost" port=24224 weight=60 plugin_id="object:ed8"
2020-03-09 15:00:31 +0900 [info]: adding source type="dummy"
2020-03-09 15:00:31 +0900 [warn]: #0 both of Plugin @id and path for <storage> are not specified. Using on-memory store.
2020-03-09 15:00:31 +0900 [warn]: #0 both of Plugin @id and path for <storage> are not specified. Using on-memory store.
2020-03-09 15:00:31 +0900 [info]: #0 starting fluentd worker pid=6749 ppid=6735 worker=0
2020-03-09 15:00:31 +0900 [info]: #0 fluentd worker is now running worker=0
2020-03-09 15:00:33 +0900 [warn]: #0 failed to flush the buffer. retry_time=0 next_retry_seconds=2020-03-09 15:00:34.095101 +0900 chunk="5a065b88ecab18d92b81f21cee90a2c9" error_class=Errno::ECONNREFUSED error="Connection refused - connect(2) for \"::1\" port 24224"

Additional context

[kenhys]

I've checked some docker images such as: docker run --rm -it xxx ruby -e 'require "socket"; p Socket.getaddrinfo("localhost", 80, nil, Socket::SOCK_STREAM)' command.

• debian:buster
• debian:bullseye
• ubuntu:focal
• ubuntu:groovy
• debian:sid
• ubuntu:bionic

These images returns [["AF_INET", 80, "127.0.0.1", "127.0.0.1", 2, 1, 6], ["AF_INET6", 80, "::1", "::1", 10, 1, 6]] but It seems that there is a case that it dpends on customization.

• if /etc/docker/daemon.json enables IPv6 for docker, it will return [["AF_INET6", 80, "::1", "::1", 10, 1, 6], ["AF_INET", 80, "127.0.0.1", "127.0.0.1", 2, 1, 6]], so this issue is reproducible.
• If /etc/gai.conf on host is changed (precedence ::ffff:0:0/96 100 ), so even though IPv6 is enabled, this issue will not be happen. (assume that fluentd is executed on host)

[kenhys]

@ganmacs

I'm not sure what is the expected "Use ipv6 address as default when out_forward and macOS catalina" behavior.

# macos:catalina
$ rbenv local 2.6.5; bundle exec ruby -e 'require "socket"; p Socket.getaddrinfo("localhost", 80, nil, Socket::SOCK_STREAM)'
[["AF_INET6", 80, "::1", "::1", 30, 1, 6], ["AF_INET", 80, "127.0.0.1", "127.0.0.1", 2, 1, 6]]
$ rbenv local 2.7.0; bundle exec ruby -e 'require "socket"; p Socket.getaddrinfo("localhost", 80, nil, Socket::SOCK_STREAM)'
[["AF_INET6", 80, "::1", "::1", 30, 1, 6], ["AF_INET", 80, "127.0.0.1", "127.0.0.1", 2, 1, 6]]

It indicates that IPv6 is used.

<server>
    host localhost // if it's 127.0.0.1 then it works well
    port 24224
  </server>

But it seems that IPv4 is expected in this comment. It seems that IPv6 address is used on macOS catalina, but as you noted, it should be IPv4 address. Could you explain a bit more, please?