ureq
ureq copied to clipboard
algesten
Implement Happy Eyeballs to workaround slow/broken IPv6 connections
Happy Eyeballs is an algorithm which can help deal with networks with misconfigured or very slow IPv6. It is implemented in most web browsers as well as curl, but not wget. It works by making a delayed IPv4 request in parallel with a IPv6 request after a small delay. This could help smooth over all sorts of potentially issues with IPv6, which are actually reasonably likely because adoption is still very much incremental.
As per https://github.com/JuliaLang/juliaup/issues/340 -- because this is not implemented, some users of this library can experience an issue where files can be downloaded by web browser, but not using this library.
It seems to also be implemented in Python https://bugs.python.org/issue33530
As a configuration option on Agent, I'd be open to this as long as it doesn't require crazy code changes. PR tentatively welcome.
Okay. I do not currently have any intention of implementing this myself. Consider this issue as merely a note of a caveat at the moment.
One wrinkle I will point out already which may make any PR "crazy" is whether having multiple requests in parallel works with the blocking/simple nature of this library. It might be necessary for the library to start using threads internally. Does spawning threads or demanding thread pools are passed in break the simple design? (Probably it does.)
In some sense this would make most sense as an "on by default" option since it was mainly invented to make things more robust.
I don't think spawning threads is ruled out. We already have a situation where we must spawn threads to uphold the deadlines when using SOCKS proxies. However. It's a quite radical shift from today being largely single threaded without any internal spawning, to have "on by default" spawning.
It could be possible to implement a (or find an off-the-shelf?) mini single threaded, internal async executor to elegantly juggle handling two separate sockets at the same time.
This is understandably not a simple issue to fix, but if I may suggest something: attach something like a ConnectionType::(IPV4|IPV6) enum to the Transport error to help bubble up what is going on here.
I've run into this on two separate projects now and the log message you get when you experience it is unfortunately not enough context to determine where to start looking, nor is it enough to dust off the cobwebs in the brain of "I've seen this before".
An example log message (currently, url info redacted):
Transport(Transport { kind: Io, message: None, url: Some(Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("redacted)), port: None, path: "redacted", query: Some("redacted"), fragment: None }), source: Some(Custom { kind: TimedOut, error: "timed out reading response" }) })
What would be slightly more helpful:
Transport(Transport { kind: Io, connection_type: ipv6, message: None, url: Some(Url { scheme: "https", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("redacted)), port: None, path: "redacted", query: Some("redacted"), fragment: None }), source: Some(Custom { kind: TimedOut, error: "timed out reading response" }) })
This would specifically be useful in cases where one intermittently sees the failure; a very common experience I had on a prior project was finding these in error logs but then retrying at a later point and they'd be fine. It wasn't until I stumbled on the ipv6 issue through another path that I understood what the actual issue was. Flagging it would at least let the developer identify what's different.
("timed out reading response" is also an odd string to read here, but I'm guessing it's a harder one to get a better message for)
