Should the content of `.julia/juliaup` be platform portable?

[davidanthoff]

This issue was triggered by the discussion in https://github.com/JuliaLang/juliaup/issues/397 and https://github.com/JuliaLang/juliaup/issues/404.

At the moment, the content of .julia/juliaup is very much only valid for a specific platform version of Juliaup. This is true for both the juliaup.json file there and to some extent also the sub folders that hold specific Julia versions.

What do I mean by that? Say you installed Juliaup on a M1/M2 Mac, then have your JULIA_DEPOT path configured so that it points to a network share. So, the folder .julia is on a network share, and therefore .julia/juliaup as well. Your Mac M1/M2 juliaup binary has created everything in .julia/juliaup. If a different user on a Linux x86 system installs the x86 Linux version of Juliaup on their system, and then configures JULIA_DEPOT to point to the same network share as the Mac user, and then tries to use the x86 Linux Juliaup version to also manage the .julia/juliaup version on the network share, everything will break.

Why? Because inside .julia/juliaup/juliaup.json we use strings to identify things that don't include platform level info, so the info inside that file is only valid for one specific version of Juliaup, say the aarch64 Mac version.

From the two linked issues above it seems that especially on clusters this might be a problem, i.e. folks might want to use a Julia depot, and then access that with different platform versions of Juliaup.

So the question is: should we change this? We could attempt to make the juliaup.json config file such that it can be validly modified with any platform version of Juliaup. One could take a .julia/juliaup folder that was created on 32 bit Windows, and manage it from a aarch64 Linux version of Juliaup, theoretically.

Enabling this would be a major change and quite involved. If we ever want to enable it, though, it would be easier to do now than later, because transitioning existing Juliaup installs from the current to some future platform agnostic design would be even worse...

I guess one question is how we generally think about this question for the content of .julia, irrespective of Juliaup? Is it assumed to be usable from different Julia platform binaries? My sense is not really, right? There seems to be quite a bit of "stuff" in there that is platform specific, but not in platform specific folders. For example, I assume precompile files are not platform agnostic, and they are not in a platform specific sub-folder of .julia/compiled. Artifacts, on the other hand, seem to be fine because the versions for different platforms of course end up in different folders.

@StefanKarpinski, any thoughts?

I think we need to make a decision about this before we do the official release for everyone.

[davidanthoff]

I think we are actually closer than I originally thought.

The versions strings now include the platform triplet, so they are no longer "relative" to the platform for which a given Juliaup is compiled.

That leaves the channel names, really. I think all that is really needed is that we change the channel names to be unique across platforms, and then introduce platform specific aliases. The transition might be not entirely trivial because we would have to migrate existing juliaup.json config files to use the new channel names, but I think exercising that kind of update once before we fully release is not a bad idea in any case.

[StefanKarpinski]

Sorry I didn't see and respond above previously. Is the idea with channel names that you have a full channel name be something like lts+0.x64.apple.darwin14 and then let lts be a shorthand for supplying the triplet you're currently running after the +? Because I think that requiring people to write long channel names out fully would be a bit awkward.

[davidanthoff]

Is the idea with channel names that you have a full channel name be something like lts+0.x64.apple.darwin14 and then let lts be a shorthand for supplying the triplet you're currently running after the +

Yes, so in the config file it would use the string lts+0.x64.apple.darwin14 as the name of the installed channel. If you run a juliaup that is compiled for x64.apple.darwin14 it would have info that this channel name should appear as lts in the UI, so the user would never ever see the long version, but just use lts to interact with that channel.

If you then were to run say a Linux aarch64 Juliaup binary against the same juliaup.json, then it would actually show the full ugly lts+0.x64.apple.darwin14 channel name in the UI, because in that Linux aarch64 Juliaup binary lts is not an alias for that, but lts instead is an alias for the channel that has the Linux aarch64 platform specifier in it.

Because I think that requiring people to write long channel names out fully would be a bit awkward.

Yes, agreed. In this proposal the long channel would only have to be used if one wants to install a Julia binary that is different from the platform of the Juliaup one is using.