wget2
wget2 copied to clipboard
rockdaboot
No ipv4 fallback when ipv6 connection does not work
When fetching something with wget2 that is only reachable via IPv4, by default wget tries IPv6 and then fails. I think that it would be good to have an IPv4 fallback similar to what wget1 used to have.
Example:
# wget2 -t1 http://www.spinics.net/lists/linux-mm/attachments//gtarazbJaHPaAT.gtar
Failed to send 252 bytes (hostname='www.spinics.net', ip=2604:9a00:2010:a03b:4::114, errno=113)
[Files: 0 Bytes: 0 [0 B/s] Redirects: 0 Todo: 0 Errors: 0 ]
Same example with wget1:
# wget -t1 http://www.spinics.net/lists/linux-mm/attachments//gtarazbJaHPaAT.gtar
--2024-06-26 10:01:51-- http://www.spinics.net/lists/linux-mm/attachments//gtarazbJaHPaAT.gtar
Resolving www.spinics.net (www.spinics.net)... 2604:9a00:2010:a03b:4::114, 207.244.127.203
Connecting to www.spinics.net (www.spinics.net)|2604:9a00:2010:a03b:4::114|:80... failed: No route to host.
Connecting to www.spinics.net (www.spinics.net)|207.244.127.203|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://www.spinics.net/lists/linux-mm/attachments//gtarazbJaHPaAT.gtar [following]
--2024-06-26 10:01:51-- https://www.spinics.net/lists/linux-mm/attachments//gtarazbJaHPaAT.gtar
Connecting to www.spinics.net (www.spinics.net)|207.244.127.203|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 19817 (19K) [application/x-gtar]
Saving to: ‘gtarazbJaHPaAT.gtar’
gtarazbJaHPaAT.gtar 100%[================================================>] 19.35K --.-KB/s in 0.008s
2024-06-26 10:01:51 (2.37 MB/s) - ‘gtarazbJaHPaAT.gtar’ saved [19817/19817]
Also the output of that in wget2 could be a bit more detailed. errno=113 does not really say much about what the problem is.
When fetching something with wget2 that is only reachable via IPv4, by default wget tries IPv6 and then fails.
You are using -t1, so wget2 would give up after 1 try (in this regard, wget2 is a bit more strict about the term try than wget1).
If your network doesn't support or route ipv6 properly, you can configure your system to try ipv4 first. Or you can put either -4 or -prefer-family=ipv4 into your ~/.wget2rc file (or system-wide /etc/wget2rc).
errno=113 does not really say much about what the problem is.
While we can convert the errno number into text in future versions, you can already do so on the command line:
$ errno 113
EHOSTUNREACH 113 No route to host
The utility errno is in package moreutils on Debian-based distros. But I am sure that other distros have it packaged as well.
Thanks for the response. Let me follow up a bit more about this below.
The -t1 does not really matter, if I leave that one out, wget2 just tries IPv6 over and over:
Failed to send 252 bytes (hostname='www.spinics.net', ip=2604:9a00:2010:a03b:4::114, errno=113)
Failed to send 252 bytes (hostname='www.spinics.net', ip=2604:9a00:2010:a03b:4::114, errno=113)
Failed to send 252 bytes (hostname='www.spinics.net', ip=2604:9a00:2010:a03b:4::114, errno=113)
[Files: 0 Bytes: 0 [0 B/s] Redirects: 0 Todo: 1 Errors: 0 ]
I understand that the constant comparison of wget2 to wget might be tiresome and sorry for using that as a reference point but I think that the way wget handles this is better - no route to host for ipv6 -> try ipv4. This is not just about my system (not) supporting IPv6 but rather about the remote system not being reachable under ipv6, like in the case above.
The comment about the output format and the information that it contains. I saw some other issues about this here and I can concur that the output could be a bit more user friendly. At first when I saw this behavior I actually thought that wget2 is incapable of dealing with a redirect (judging from the wget output that I ran in parallel).
Regards, Michal
The -t1 does not really matter, if I leave that one out, wget2 just tries IPv6 over and over:
Oh, this sounds like a regression. Thanks for insisting, I'll try to reproduce locally.
No problem, the url is generally available so you can use it for the reproducer.
From my first tests; the the fallback to the second (third etc) IP address doesn't happen if the failure isn't detected immediately by the connect() function. It looks like fixing this requires some thoughts / refactoring: the fallback mechanism needs to be done on a higher level (application), not on the connect level (library).
To reproduce (requires commit ff881ed20182950accf77cf70bcaf51ec75d1a87 or later):
wget2 -d --dns-cache-preload=dns.cache https://www.spinics.net
with dns.cache:
f604:9a00:2010:a03b:4::114 www.spinics.net
207.244.127.203 www.spinics.net
(The ipv6 address should be invalid/unroutable on your system)