JuliaLang
External Pkg always recompiles upon loading
While being in a Pkg.jl repo:
❯ julia +nightly --project -q
julia> using Pkg
[ Info: Precompiling Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f] (cache misses: wrong source (1), mismatched flags (1))
❯ julia +nightly --project -q
julia> using Pkg
[ Info: Precompiling Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f] (cache misses: for different buildid (2), mismatched flags (2))
❯ julia +nightly --project -q
julia> using Pkg
[ Info: Precompiling Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f] (cache misses: for different buildid (2), mismatched flags (2))
The two original misses are from the bundled cache files so those are expected. This is also a bit weird:
julia> pkg = Base.identify_package("Pkg")
Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f]
julia> Base.isprecompiled(pkg)
true
julia> using Pkg
[ Info: Precompiling Pkg [44cfe95a-1eb2-52ea-b672-e2afdf69b78f] (cache misses: for different buildid (2), mismatched flags (2))
┌ Debug: Rejecting cache file /home/kc/.julia/compiled/v1.12/Pkg/tUTdb_sLFSB.ji because required dependency Base.PkgId(Base.UUID("2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"), "Base64") failed to load from cache file for /home/kc/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/Base64/D7K0n_Ip9tx.ji.
│ exception = Error reading package image file.
weird..
I am also surprised this doesn't print anything at all for Base64:
julia> ENV["JULIA_DEBUG"] = "loading"
"loading"
julia> using Base64
julia> using Example
┌ Debug: Loading object cache file /home/kc/.julia/compiled/v1.12/Example/lLvWP_sLFSB.so for Example [7876af07-990d-54b4-ab0e-23690620f79a]
└ @ Base loading.jl:1227
I had some thought that the nightly build juliaup uses was corrupt somehow but building a local Julia on master it gives the exact same behavior.
This method of loading Pkg seems to work:
❯ julia +nightly --project -iqe 'using REPL; using Pkg'
julia> pkgdir(Pkg)
"/home/kc/JuliaPkgs/Pkg.jl"
Base64 is an indirect dep of REPL, so it's already loaded?
Okay, that makes sense, I looked in Base.loaded_modules and it wasn't there so I assumed it wasn't loaded already:
julia> Base.loaded_modules
Dict{Base.PkgId, Module} with 12 entries:
PkgId(UUID("4536629a-c528-5b80-bd46-f80d51c5b363"), "OpenBLAS_jll") => OpenBLAS_jll
PkgId(UUID("8f399da3-3557-5675-b5ff-fb832c97cbdb"), "Libdl") => Libdl
PkgId(UUID("ea8e919c-243c-51af-8825-aaa63cd721ce"), "SHA") => SHA
PkgId(UUID("7b1f6079-737a-58dc-b8bc-7a2ca5c1b5ee"), "FileWatching") => FileWatching
PkgId(UUID("56f22d72-fd6d-98f1-02f0-08ddc0907c33"), "Artifacts") => Artifacts
PkgId(UUID("8e850b90-86db-534c-a0d3-1478176c7d93"), "libblastrampoline_jll") => libblastrampoline_jll
PkgId(nothing, "Main") => Main
PkgId(nothing, "Core") => Core
PkgId(UUID("37e2e46d-f89d-539d-b4ee-838fcccc9c8e"), "LinearAlgebra") => LinearAlgebra
PkgId(nothing, "Base") => Base
PkgId(UUID("9a3f8284-a2c9-5f02-9a11-845980a1fd5c"), "Random") => Random
PkgId(UUID("6462fe0b-24de-5631-8697-dd941f90decc"), "Sockets") => Sockets
so I guess things that are loaded as part of require_stdlib are now invisible but when you do using on them they grab the invisible module and makes it visible?
But that shouldn't make Pkg have to compile over and over due to error when reading the Base64 pkgimage file I guess.
┌ Debug: Rejecting cache file /home/kc/.julia/compiled/v1.12/Pkg/tUTdb_sLFSB.ji because required dependency Base.PkgId(Base.UUID("2a0f44e3-6c83-55bd-87e4-b1978d98bd5f"), "Base64") failed to load from cache file for /home/kc/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/Base64/D7K0n_Ip9tx.ji. │ exception = Error reading package image file.weird..
I have seen this too when trying to debug https://github.com/JuliaPy/PythonCall.jl/issues/513. There @kshyatt nailed down the problem to https://github.com/JuliaLang/julia/commit/71fa11f0427fc66f2328cddbba865852fa47e0f1.
There @kshyatt nailed down the problem to https://github.com/JuliaLang/julia/commit/71fa11f0427fc66f2328cddbba865852fa47e0f1.
Same thing happens when I revert that commit.
I ran a git bisect for the function I needed not being exported, and it fingered this commit. Maybe multiple things are wrong?
On Wed, Jun 26, 2024 at 9:26 AM Kristoffer Carlsson < @.***> wrote:
There @kshyatt https://github.com/kshyatt nailed down the problem to 71fa11f https://github.com/JuliaLang/julia/commit/71fa11f0427fc66f2328cddbba865852fa47e0f1 .
Same thing happens when I revert that commit.
— Reply to this email directly, view it on GitHub https://github.com/JuliaLang/julia/issues/54940#issuecomment-2191699506, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGKJY5FPJRFDYJSRJHFI4TZJK6P5AVCNFSM6AAAAABJ5V7TROVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDCOJRGY4TSNJQGY . You are receiving this because you were mentioned.Message ID: @.***>
Same happens with OhMyREPL on nightly:
┌ Debug: Loading object cache file /Users/kristoffercarlsson/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/StyledStrings/UcVoM_44pG1.dylib for StyledStrings [f489334b-da3d-4c2e-b8f0-e476e12c162b]
└ @ Base loading.jl:1227
┌ Debug: Rejecting cache file /Users/kristoffercarlsson/.julia/compiled/v1.12/OhMyREPL/08e1i_J2Ppx.ji because required dependency Base.PkgId(Base.UUID("f489334b-da3d-4c2e-b8f0-e476e12c162b"), "StyledStrings") failed to load from cache file for /Users/kristoffercarlsson/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/StyledStrings/UcVoM_44pG1.ji.
│ exception = Error reading package image file.
└ @ Base loading.jl:1962
┌ Debug: Loading object cache file /Users/kristoffercarlsson/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/StyledStrings/UcVoM_44pG1.dylib for StyledStrings [f489334b-da3d-4c2e-b8f0-e476e12c162b]
└ @ Base loading.jl:1227
┌ Debug: Rejecting cache file /Users/kristoffercarlsson/.julia/compiled/v1.12/OhMyREPL/08e1i_J2Ppx.ji because required dependency Base.PkgId(Base.UUID("f489334b-da3d-4c2e-b8f0-e476e12c162b"), "StyledStrings") failed to load from cache file for /Users/kristoffercarlsson/.julia/juliaup/julia-nightly/share/julia/compiled/v1.12/StyledStrings/UcVoM_44pG1.ji.
│ exception = Error reading package image file.