inet6 to inet fallback in networks without ipv6 support

[ruslandoga]

👋

Is your feature request related to a problem? Please describe.

Not sure yet. First I'd like double check if gen_tcp:connect and ssl:connect with inet6 option are supposed to fallback to inet when IPv6 connection is not successful. And if it's not supposed to work this way, I'd like to request this feature!

Right now I'm not able to make this sort of fallback work. Here're some examples from Fly.io dual-stack machine and from an IPv4-only container running on AWS EC2. I'm using IPv4-only and IPv6-only hosts from http://dual.tlund.se

From Fly.io Machine

1> inet:getifaddrs().
{ok,[{lo,[{flags,[up,loopback,running]},
             {addr,{127,0,0,1}},
             {netmask,{255,0,0,0}},
             {addr,{0,0,0,0,0,0,0,1}},
             {netmask,{65535,65535,65535,65535,65535,65535,65535,65535}},
             {hwaddr,[0,0,0,0,0,0]}]},
     {dummy0,[{flags,[broadcast]},
                {hwaddr,[150,232,97,230,238,118]}]},
     {eth0,[{flags,[up,broadcast,running,multicast]},
              {addr,{172,19,136,2}},
              {netmask,{255,255,255,248}},
              {broadaddr,{172,19,136,7}},
              {addr,{172,19,136,3}},
              {netmask,{255,255,255,248}},
              {broadaddr,{172,19,136,7}},
              {addr,{9733,19520,344,51602,0,62017,27426,1}},
              {netmask,{65535,65535,65535,65535,65535,65535,65535,65534}},
              {addr,{64938,0,24663,2683,129,62017,27426,2}},
              {netmask,{65535,65535,65535,65535,65535,65535,65535,0}},
              {addr,{65152,0,0,0,56493,55039,65078,12685}},
              {netmask,{65535,65535,65535,65535,0,0,0,0}},
              {hwaddr,[222,173,214,54,49,141]}]},
     {teql0,[{flags,[]}]}
    ]}.

Connecting to IPv4-Only Host works with default inet option

2> {ok, Socket} = gen_tcp:connect("ipv4.tlund.se", 80, [{active, false}]).
3> inet:peername(Socket).
{ok,{{193,15,228,195},80}}.

But fails when inet6 option is provided

4> gen_tcp:connect("ipv4.tlund.se", 80, [inet6, active, false]).
{error,nxdomain}.

ipv6_v6only doesn't Help

5> gen_tcp:connect("ipv4.tlund.se", 80, [inet6, {ipv6_v6only, false}, active, false]).
{error,nxdomain}.

Default options (inet) don't work with IPv6-only host

6> gen_tcp:connect("ipv6.tlund.se", 80, [active, false]).
{error,nxdomain}.

inet6 works with IPv6-only host

7> {ok, Socket} = gen_tcp:connect("ipv6.tlund.se", 80, [inet6, {active, false}]).
8> inet:peername(Socket).
{ok,{{10752,2049,15,0,0,0,0,405},80}}.

From IPv4-only container on AWS EC2

1> inet:getifaddrs().
{ok,[{lo,[{flags,[up,loopback,running]},
             {addr,{127,0,0,1}},
             {netmask,{255,0,0,0}},
             {hwaddr,[0,0,0,0,0,0]}]},
     {eth0,[{flags,[up,broadcast,running,multicast]},
              {addr,{172,24,0,3}},
              {netmask,{255,255,0,0}},
              {broadaddr,{172,24,255,255}},
              {hwaddr,[2,66,172,24,0,3]}]}
    ]}.

No fallback to inet

2> gen_tcp:connect("ipv6.tlund.se", 80, [inet6, {active, false}]).
{error,eaddrnotavail}.

Describe the solution you'd like

inet6 would fallback to inet automatically when needed so that providing inet6 option would always increase the chance of a successful connection.

Describe alternatives you've considered

Some Elixir libraries perform a manual fallback from inet6 to inet like Mint and some other libraries like Postgrex allow a list of endpoints to be provided for connection attempts.

Additional context

Relevant discussion (where this question originated): https://github.com/phoenixframework/phoenix/pull/4289#issuecomment-2149345334

[ruslandoga]

In place of a "ping": I would also happily work on implementing happy eyeballs into Erlang!

Related issues / discussions / projects:

• https://github.com/yandex/inet64_tcp (and https://github.com/ruslandoga/happy_tcp)
• https://github.com/erlang/otp/pull/4610
• https://github.com/benoitc/hackney/issues/206
• https://github.com/ninenines/gun/issues/188
• https://git.pleroma.social/pleroma/pleroma/-/issues/3264

[wojtekmach]

I believe built-in happy eyeballs implementation would be a huge win for the ecosystem. 👍

[essen]

Yes!

[bmk]

I believe that the idea is that gen_tcp (and gen_udp, gen_sctp) should be "close to the metal". And these kinds of features are up to the application.

I my memory is correct the inets (httpd and httpc) had a similar config option (inet6fb4 or something like it).

I do not know if ssl has this feature.

'socket' is very much "close to the metal". But gen_tcp could maybe be considered to be a layer that should provide this kind of a feature. We will discuss ASAP (vacation times here at OTP central).

[essen]

This is the kind of feature that sits between OTP and application I think. It makes sense to have it in OTP because many would use it, but not all network connections require it either. It could be a separate open source project, but then who has the will and the bandwidth to maintain it?

Happy Eyeballs is also tricky in that it pretty much requires connecting via 4/6 concurrently. The socket module's nowait could come in handy there. Try to connect to all then wait for the winner. But once we have the right socket connected, we need to be able to hand it off to gen_tcp or ssl. So OTP changes would be required.

The alternative is building on top of gen_tcp or ssl but that means having concurrent processes and much higher complexity.

If we could "upgrade" a socket socket to gen_tcp / ssl / others, in a documented way, then I believe we wouldn't be far from actually implementing this in a fairly straightforward way.

[ruslandoga]

It could be a separate open source project, but then who has the will and the bandwidth to maintain it?

FWIW, I started working on https://github.com/ruslandoga/happy_tcp and will be trying to implement Happy Eyeballs by using prim_inet:async_connect but it's quite hacky:

• collect (sequentially for now) ipv4 and ipv6 addresses, sort them using Happy Eyeballs rules
• pass them as a list arg to happy_tcp:connect/1
• use async nature prim_inet:async_connect to do the Happy Eyeballs thing

I haven't looked into socket yet, just gen_tcp with inet backend.

So OTP changes would be required.

So far, happy_tcp seems to work without any changes but it would be nice if inet_tcp_backend "behaviour" could connect to multiple addresses instead of just one, then my hack of passing a list of addresses could go away.

But my ideal would be having inet6_tcp do all this. So that gen_tcp:connect(Domain, Port, [inet6]) "would just work", the way it already works for gen_tcp:listen (which afaik binds on both ipv4 and ipv6 when inet6 option is provided).

[essen]

The OTP changes are needed to keep the same interface, i.e. once the connection has succeeded you use gen_tcp or ssl as you normally would. There's no reason to have yet another interface today, other than the fact that we can't upgrade the socket or prim_inet socket to gen_tcp without using undocumented functions. Note that the code exists but it is not a public interface from OTP (same goes for prim_inet:async_connect).

[ruslandoga]

👋 everyone :)

I ran into this issue again today. I wonder if it would be OK for me to explore a possible solution and PR it? I would be super honored to contribute at least something to the great OTP!.. And I have a lot of free time these days :)

[flexagoon]

Sorry if that's a dumb question, but why isn't inet6 on by default? It seems like passing both inet and inet6 allows connection to both ipv4 and ipv6 addresses, so why aren't they both just on by default?

[ruslandoga]

👋 @flexagoon

It doesn't always work, and making it work well requires algorithms like Happy Eyeballs, not just a naive fallback. Here's a recent post I saw on HN today about some of the possible problems: https://techlog.jenslink.net/posts/ipv6-is-hard/

[RaimoNiskanen]

We should at least discuss how Happy Eyeballs (RFC 8305) could be added to Kernel/Stdlib. I am a little afraid of how much work it would be, and what it would mean in terms of raised security demands on inet_dns and/or inet_rex.

Today we trust the native resolver library that make use of the hosts file, NIS, DNS, and maybe other name resolving methods. RFC 8305 makes the TCP connection setup DNS only lookup aware.

Suitable hooks and internal functions can be made public, or if we aim to have this in Kernel/Stdlib, they can be kept internal.

I think this belongs to the gen_tcp and siblings layer, to be implemented by helper modules in Kernel. It could maybe be activated by combining inet and inet6 as the address family(ies) for the connection, or with a new synthetic address family.

[essen]

I don't think there's a requirement for DNS. What the RFC defines are four steps which are more or less independent:

1. Initiation of asynchronous DNS queries [Section 3]

2. Sorting of resolved destination addresses [Section 4]

3. Initiation of asynchronous connection attempts [Section 5]

4. Establishment of one connection, which cancels all other attempts [Section 5]

Steps 2-4 don't strictly require the use of DNS in the first step. The IP addresses could be obtained by any other means.

The document also has other assumptions that don't have to be strictly followed. For example:

Note that this document assumes that the preference policy for the host destination address favors IPv6 over IPv4. IPv6 has many desirable properties designed to be improvements over IPv4 [RFC8200]. If the host is configured to have a different preference, the recommendations in this document can be easily adapted.

One could imagine use cases where users would prefer IPv4 instead of IPv6, or have other requirements.

tl;dr It's an RFC describing an algorithm, we don't have to implement it 1:1, we can adapt it to our situation.

I think there's three main parts to the RFC:

1. Asynchronous DNS queries, which could very well use the native resolver, we don't have to explicitly query a DNS server, what we do need is asynchronous fetch of A / AAAA records or other records, basically an asynchronous method of obtaining all IPv4/v6 addresses for an hostname
2. Sorting, which could be done internally via a default sorting function or a user-provided one
3. Asynchronous connect and establishment of one successful connection, where the list of IP addresses can get bigger after the initial connection establishment, as DNS query results come in

I think the trickier aspect is being able to feed new IP addresses to this async connection establishment.

In any case I think it can all sit at the gen_tcp level, perhaps in a separate internal module or something, but it would still use gen_tcp:connect to initiate. I can see a few useful options only valid for this method:

• The user may want to provide a list of IP addresses directly and skip step 1. (and perhaps 2.)
• The sorting function should probably be configurable. It can either be a sorting function or a selector of the next IP to connect to from a given list.
• The time between each connection attempt should probably be configurable
• The maximum number of connections attempts (or concurrent connection attempts) as well

A first draft where only 3. is implemented using a user-provided list of IP addresses would already be a good improvement.