failing on macOS: juliaup add 1.0

[StefanKarpinski]

$ juliaup add 1.0
Installing Julia 1.0.5+0 (x64).
Error: Failed to extract downloaded file from url `https://julialang-s3.julialang.org/bin/mac/x64/1.0/julia-1.0.5-mac64.tar.gz`.

Caused by:
    0: failed to unpack `._julia-1.0.5` into `/Users/stefan/.julia/juliaup/julia-1.0.5+0~x64/`
    1: No such file or directory (os error 2)

Another failure to install on macOS but this time an install error instead of a download error.

[davidanthoff]

I think @staticfloat created that file today, this suggests though that the content of that tar.gz file is somehow different from the way the other versions are packaged?

[staticfloat]

It looks like there's some macOS garbage that got into there, with special tar extension fields that we don't want. Stripping them out with:

Tar.rewrite("julia-1.0.5-mac64.tar", "julia-1.0.5-mac64.tar.rewritten") do hdr
    return !startswith(basename(hdr.path), "._")
end

Was legitimately the easiest way to fix the tarball, and it's now been recompressed and uploaded as https://julialang2.s3.amazonaws.com/bin/mac/x64/1.0/julia-1.0.5-mac64.tar.rewritten.gz. Is there an easy way to test that this works before I overwrite the file that's already uploaded?

[davidanthoff]

Hm, it would be a fair bit involved to test that standalone. Given that juliaup is the only tool that is using that upload at the moment, and the current version is broken in any case, maybe we could just overwrite and see whether it works with that?

[staticfloat]

Alright done. Remember to do a curl -L -X PURGE https://julialang-s3.julialang.org/bin/mac/x64/1.0/julia-1.0.5-mac64.tar.gz from your location to ensure that any Fastly caches between you and the S3 bucket are properly purged before testing.

[davidanthoff]

@StefanKarpinski can you try again, after you ran that command that @staticfloat posted above?

[StefanKarpinski]

I'm getting a different error now:

$ juliaup add 1.0
Installing Julia 1.0.5+0 (x64).
Error: Failed to extract downloaded file from url `https://julialang-s3.julialang.org/bin/mac/x64/1.0/julia-1.0.5-mac64.tar.gz`.

Caused by:
    0: failed to unpack `julia-1.0.5/LICENSE.md` into `/Users/stefan/.julia/juliaup/julia-1.0.5+0~x64/LICENSE.md`
    1: No such file or directory (os error 2)

Progress!

[davidanthoff]

Hm, that is strange... Does the /Users/stefan/.julia/juliaup/julia-1.0.5+0~x64/ folder exist?

[davidanthoff]

Ok, I downloaded the tar.gz file for 1.7.1 (works) and 1.0.5 (doesn't work) and compared them. I found two differences: 1) the tar filename inside the gz file for 1.0.5 had the rewritten ending that @staticfloat added, and 2) none of the directories have any mode flags, user or group setting. My best guess is that 2) is the problem.

Here is what 7z shows for the working 1.7.1 file:

And for the not working 1.0.5 file:

Maybe that then leads to a situation where the tar extract algorithm that I'm using in Rust doesn't even create the folder that corresponds to the julia-1.0.5 folder in the tar file or something like that?

But we should probably fix 1) as well. @staticfloat, could you do that? Although, if we were to tackle https://github.com/JuliaLang/juliaup/issues/183 generally it might just be solved by that...

[DilumAluthge]

@davidanthoff I'm not able to reproduce the issue with julia-1.0.5-mac64.tar.gz that you posted above.

Specifically, I ran the following bash commands on a macOS machine, and everything seemed to work fine.

cd $(mktemp -d)

curl -LO https://julialang-s3.julialang.org/bin/mac/x64/1.0/julia-1.0.5-mac64.tar.gz

tar xzvf julia-1.0.5-mac64.tar.gz

./julia-1.0.5/bin/julia -e 'import InteractiveUtils; InteractiveUtils.versioninfo()'

Can you try again and see if you can still reproduce?

[ViralBShah]

I have a similar error. I am using a case sensitive filesystem on mac - not sure if that is a problem.

viral-laptop:~ viral$ juliaup add 1.0
Installing Julia 1.0.5+0.x64 (x64).
Error: Failed to extract downloaded file from url `https://julialang-s3.julialang.org/bin/mac/x64/1.0/julia-1.0.5-mac64.tar.gz`.

Caused by:
    0: failed to unpack `julia-1.0.5/LICENSE.md` into `/Users/viral/.julia/juliaup/julia-1.0.5+0.x64/LICENSE.md`
    1: No such file or directory (os error 2)

[ViralBShah]

Verified that all the versions from 0.7 onwards now install correctly on mac, windows and linux.