julia icon indicating copy to clipboard operation
julia copied to clipboard

Fix `JULIA_CPU_TARGET` being propagated to workers precompiling stdlib pkgimages

Open KristofferC opened this issue 1 year ago • 8 comments

Apparently (thanks ChatGPT) each line in a makefile is executed in a separate shell so adding an export line on one line does not propagate to the next line.

KristofferC avatar Apr 15 '24 14:04 KristofferC

Is there any way we can make sure that this errors if we fail to propagate this again in the future?

topolarity avatar Apr 15 '24 14:04 topolarity

I think the canonical way is something like https://github.com/JuliaLang/julia/commit/f106bd9708fd61735f658e3e27ee1f227b829736

vchuravy avatar Apr 15 '24 15:04 vchuravy

Question, why not set the env variable "globally" together with JULIA_DEPOT_PATH and JULIA_LOAD_PATH?

KristofferC avatar Apr 16 '24 11:04 KristofferC

I'm ok with this but if someone has a better or more canonical way feel free to push to this branch. https://github.com/JuliaLang/julia/pull/54093#issuecomment-2058889142 seems easiest to me IIUC though?

KristofferC avatar May 10 '24 12:05 KristofferC

With a build from this PR I get

julia> Base.parse_image_targets(Base.parse_cache_header("share/julia/compiled/v1.12/Pkg/tUTdb_fyEGR.ji")[7])
1-element Vector{Base.ImageTarget}:
 generic; flags=0; features_en=(cx16)

which seems to suggest JULIA_CPU_TARGET is still not being effectively propagated where needed?

giordano avatar May 14 '24 12:05 giordano

The command to compile the pkgimage seems to include the CPU target properly:

julia_cmd(; cpu_target) = `julia-master/usr/bin/julia -C 'generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1);x86-64-v4,-rdrnd,base(1)' -Jjulia-master/usr/lib/julia/sys.dylib -g1 --startup-file=no`

but yet

julia> cd(joinpath(Sys.BINDIR, "../share/julia/compiled/v1.12/Test"))

julia> Base.parse_image_targets(Base.parse_cache_header("JfdTE_tFidc.ji")[7])
1-element Vector{Base.ImageTarget}:
 generic; flags=0; features_en=(cx16)

KristofferC avatar May 14 '24 14:05 KristofferC

is Base.precompilepkgs ignoring this? It seems to take in a CacheFlags argument that is empty

gbaraldi avatar May 14 '24 14:05 gbaraldi

is Base.precompilepkgs ignoring this? It seems to take in a CacheFlags argument that is empty

CacheFlags are AFAIU not related to the CPU target and even if it was, my latest commit prints the julia command which does include the target.

KristofferC avatar May 14 '24 15:05 KristofferC

I just ran into this while cross-compiling, where the -C native figured out from qemu resulted in illegal instructions when deploying the build. I confirmed that this change now correctly forwards my configured JULIA_CPU_TARGET, so let's merge this after CI passes.

maleadt avatar Oct 14 '24 12:10 maleadt

Looks like this PR bitrot in the meantime?

giordano avatar Oct 14 '24 15:10 giordano

I don't know why compileall aborted here, but not sure it matters

vtjnash avatar Oct 15 '24 00:10 vtjnash