MacOS: Download timeout but url works manually

[IanButterworth]

The url works when I try via a browser, but times out via juliaup

ian@IanMBP ~ % juliaup add 1.7
Installing Julia 1.7.3+0.x64 (x64).
Error: Failed to download from url `https://julialang-s3.julialang.org/bin/mac/x64/1.7/julia-1.7.3-mac64.tar.gz`.

Caused by:
    0: https://julialang-s3.julialang.org/bin/mac/x64/1.7/julia-1.7.3-mac64.tar.gz: Network Error: timed out reading response
    1: timed out reading response
ian@IanMBP ~ % juliaup add 1.7
Installing Julia 1.7.3+0.x64 (x64).
Error: Failed to download from url `https://julialang-s3.julialang.org/bin/mac/x64/1.7/julia-1.7.3-mac64.tar.gz`.

Caused by:
    0: https://julialang-s3.julialang.org/bin/mac/x64/1.7/julia-1.7.3-mac64.tar.gz: Network Error: timed out reading response
    1: timed out reading response

[IanButterworth]

This has resolved itself. No idea why one would work and the other not, though

[roland-KA]

I have now the same problem:

➜  ~ juliaup add lts
Installing Julia 1.6.7+0.x64 (aarch64).
Error: Failed to download from url `https://julialang-s3.julialang.org/bin/mac/x64/1.6/julia-1.6.7-mac64.tar.gz`.

Caused by:
    0: https://julialang-s3.julialang.org/bin/mac/x64/1.6/julia-1.6.7-mac64.tar.gz: Network Error: timed out reading response
    1: timed out reading response

➜  ~ juliaup add release
Installing Julia 1.7.3+0.x64 (aarch64).
Error: Failed to download from url `https://julialang-s3.julialang.org/bin/mac/x64/1.7/julia-1.7.3-mac64.tar.gz`.

Caused by:
    0: https://julialang-s3.julialang.org/bin/mac/x64/1.7/julia-1.7.3-mac64.tar.gz: Network Error: timed out reading response
    1: timed out reading response

[roland-KA]

... and one day later still not working. So it's perhaps reasonable to reopen this issue.

[davidanthoff]

Hm, so I'm a bit stomped, no idea what might cause that? Just a somewhat unreliable internet connection?

[roland-KA]

The internet connection can't be the problem, because downloading the file directly via the URL works perfectly fine.

And the problem is still there. I've tried it once a day since my first posting and the result is still the same time-out error.

BTW: It's also on MacOS.

[davidanthoff]

@roland-KA and I assume you are not in an env that uses any kind of proxy, right?

[roland-KA]

No, there is no proxy.

I've just tried it again. Now it works. But it didn't succeed during the last 4 days. The question is: What was different during this time?

[davidanthoff]

Argh, the bug from hell :) I looked a bit around in https://github.com/algesten/ureq (which is the crate we use for downloading), but nothing known along the lines of your bug jumped out to me...

[roland-KA]

Argh, the bug from hell :)

Indeed 😃 ... I don't see either anything that could be done at the moment. 🤷‍♂️

[frankier]

I have this problem, and have tracked it down to my own misconfigured setup where I have an IPv6 address, but no IPv6 route to the Internet. Possibly others have had the same problem as me.

The problem is as follows:

• I have an ISP who does not offer IPv6 (name and shame: Telia Finland)
• My router's DHCP settings is set to give out ipv6 addresses
  • Disabling this fixes the problem
• The problem started when I switched my DNS to Google DNS from my ISP's due to reliability issues
  • It looks like my ISP may have previously filtered AAAA records to avoid people trying to use IPv6

As to why the problem occurs for juliaup and not in the browser, it seems some software, including web browsers and curl will retry with the A record when they can't connect to the AAAA record (perhaps web browsers figure out in advance that IPv6 is not usable beyond the router), whereas wget and ureq will not.

[IanButterworth]

Nice deduction! That makes sense.

This wget issue was just closed without being resolved https://gitlab.com/gnuwget/wget2/-/issues/196 Whilst curl has a history of improving the use of the "happy eyeballs" approach https://curl.se/changes.html

[davidanthoff]

Oh wow, that is amazing detective work!

I think this probably has to be addressed upstream in https://github.com/algesten/ureq, right? That is what we use for downloading on Mac/Linux. On Windows we use Windows APIs for downloading, and my assumption would be that those deal with this kind of stuff automatically.

[davidanthoff]

Do we think this needs to be resolved before we release Juliaup officially?

[roland-KA]

Fortunately I didn't run into this problem again in recent weeks. But if it still exists, I would recommend to resolve it before the official release. Especially for newbies to Julia this could be a showstopper and comparable tools for other languages (e.g. rustup) work seamlessly (also self-updating etc.). So Julia should be on par with the competition 😊.

[davidanthoff]

I think the main question is what we could actually do about it... Not clear to me.

[DilumAluthge]

What does rustup use for downloading?

[frankier]

Once this issue is fixed https://github.com/algesten/ureq/issues/535 then ureq can be updated and this bug will be fixed

[roland-KA]

What does rustup use for downloading?

I'm not a Rust expert. But as far as I can see from the source code of fn download_file_, they use curl and the rustls-package (crate).

[staticfloat]

Just to add on that I believe @frankier is correct; you need happy eyeballs to properly work in a dual-stack environment (e.g. most operating systems these days) but where only one of ipv4 and ipv6 works.

[davidanthoff]

So I think the question for us is, what do we do? If https://github.com/algesten/ureq/issues/535 gets fixed we are obviously done, but the timeline there might not mesh with our timeline... I guess one question is whether we have someone in our community who could do a PR to ureq that fixes this? If not, we could either a) declare this not a release-blocker or b) look for another download crate. I would strongly tend towards a), ureq is really nice otherwise...

[davidanthoff]

I pushed a new build to the release preview channel that uses a different rust package for the downloads. Would be great if folks can give that a try if you are on Linux or Mac. You can switch your installation over to the release preview channel by running juliaup self channel releasepreview.

[davidanthoff]

I just checked, and hyper, the underlying package that reqwest uses, does implement the happy eyeball approach. So, presumably it should all work now. I am going to close this issue for now, if you run into any similar problems with the latest version again, please either reopen here or a new issue.

[christiangnrd]

I'm aware that this issue is closed but since no one confirmed yet, I just had to use the release preview to install Julia since using 1.8.16 would time out.