PackageCompiler.jl icon indicating copy to clipboard operation
PackageCompiler.jl copied to clipboard

Cache the "base" sysimage using Scratch.jl

Open KristofferC opened this issue 1 year ago • 4 comments

When creating apps/libraries we first generate a "base" sysimage that does not contain e.g. the precompiled code from the default sysimage that Julia comes bundled with. We could cache that sysimage and reuse it so we do not have to regenerate it over and over.

KristofferC avatar Jan 15 '24 15:01 KristofferC

That would be a great feature to have! It slightly relates to some of the suggestions in #858. One question that was hard for me to answer though, is how to ensure that the correct cached base sysimage is determined based on the Julia flags (and other environmental variables?). I am thinking that things like -O3 or --check-bounds=yes|no would probably result in different base images.

Also, given that there is (likely) a myriad of different combinations, do we only want to keep the latest cache image or one for each possible feature set? Also, how to prevent race conditions during generation?

sloede avatar Jan 15 '24 15:01 sloede

I think it is fine for code in Base to be compiled with the default arguments. The app/library specific code will then be compiled with whatever specific options are given.

The CPU-target has to be correct though I think.

Also, how to prevent race conditions during generation?

We could have a PID lock for the sysimage being generated.

KristofferC avatar Jan 15 '24 15:01 KristofferC

I think it is fine for code in Base to be compiled with the default arguments

OK, this makes sense.

We could have a PID lock for the sysimage being generated.

Sounds good.

In general, if we add such a caching feature, it would be good if we can add functions to also remove the cached file and/or force the regeneration of a new file upon next use.

sloede avatar Jan 15 '24 16:01 sloede

This seems to be a duplicate of #841. I would very much welcome this feature!

dhanak avatar Jun 17 '24 15:06 dhanak