setup-julia
setup-julia copied to clipboard
Add the optional `tool-cache` input, which allows the user to override the location into which Julia is installed
Alright, this is now working correctly and is ready for review.
Right before merging, I'll remove the lines in .github/workflows/example-builds.yml
that say # TODO: delete this line
.
I'm a bit confused why setting the environment variable in the workflow doesn't accomplish the same. I've tried but it seems to ignore it.
I don't like the approach of changing the tool cache directory. This could have unintended side effects. I think a better approach would be to use an input to set the temporary installation to a custom location:
https://github.com/julia-actions/setup-julia/blob/f29cb961e8488bf333363f0f325069a721f74dee/src/installer.ts#L186
This directory is usually removed after adding Julia to the tool-cache. We'd have add a condition to only do so if there's no custom installation dir:
https://github.com/julia-actions/setup-julia/blob/f29cb961e8488bf333363f0f325069a721f74dee/src/setup-julia.ts#L65-L66
We might want to add deletion of that dir as a post-action step. Otherwise it could fill up the storage of self-hosted runners with a persistent filesystem.
What do you think?
In addition to the comments above, I'm curious why one would want to do this? The other setup-* actions don't take installation directories as an input either, so I'm not sure the complexity is worth it. (but I'm open to good arguments)
So, the motivation here is to be able to cache the precompilation cache (the ~/.julia/compiled
) directory on GitHub Actions using the actions/cache
action.
The core problem here is that there are certain stdlibs (e.g. MbedTLS_jll
) that are not included in the default Julia sysimage.
Therefore, when you precompile a project that depends (directly or indirectly) on one of these stdlibs, that stdlib is also precompiled, and the corresponding .ji
file for that stdlib is written to ~/.julia/compiled
. Unfortunately, this means that if MbedTLS_jll
is at the bottom of your project's dependency tree, then virtually all of the .ji
files in ~/.julia/compiled
will depend on the .ji
file for MbedTLS_jll
. The .ji
file for MbedTLS_jll
will become invalidated if the mtime
ever changes for the source code file for MbedTLS_jll
. And because MbedTLS_jll
is at the bottom of your dependency tree, this means that virtually all of your project's .ji
files will become invalidated if the mtime
ever changes for the source code file for MbedTLS_jll
. The source code for MbedTLS_jll
(like the source code for all stdlibs) is located in the Julia installation. Therefore, if the mtime
s of the files in the Julia installation ever change, virtually all of your project's .ji
files will become invalidated.
So, suppose that we have two GitHub Actions CI jobs, job A and job B. Both jobs will run on GitHub-hosted runners of the same operating system and architecture. At the end of job A, we want to upload our cache, and at the beginning of job B, we want to download our cache and have the precompilation files be fresh.
In order for that to work, the mtime
s of the files in the the Julia installation in job A have to be exactly the same as the mtime
s of the files in the the Julia installation in job B. If you just use the setup-julia
action normally, then the Julia installation in job A will have different mtime
s than in job B.
So the only solution that I have come up with is:
- In job A, instead of installing Julia to the normal toolcache, we create a custom toolcache directory inside our workspace, and we install Julia to our custom toolcache directory.
- We use
actions/cache
to cache our custom toolcache directory. So, at the end of job A, our custom toolcache directory is uploaded. (We also useactions/cache
to cache our Julia depot, which we also put in a custom location by setting theJULIA_DEPOT_PATH
environment variable.) - At the beginning of job B, our custom toolcache directory is downloaded. Inside this directory, the
mtime
s of the files in the Julia installation are exactly the same as they were during job A, which means that our precompilation files will not be invalidated. - In job B, when it gets to the
setup-julia
stage, we again use our custom toolcache directory.setup-julia
sees that the desired Julia version already exists in our custom toolcache directory, so it does not try to re-install Julia.
That sounds like a good reason, let's add it. But imo we should add it properly by setting the new installation path in the places where it's relevant instead of overwriting the environment variable as I'm not sure what side effects that will have. What are your thoughts on that?