Enzyme.jl icon indicating copy to clipboard operation
Enzyme.jl copied to clipboard

Precompilation fails on Julia 1.12 beta release

Open gdalle opened this issue 8 months ago • 6 comments

Now that 1.12 is available as a pre-release, I wanted to try it out with Enzyme and I stumbled upon this error. You're probably aware already but in case you're not, here it is:

(@v1.12) pkg> activate --temp
  Activating new project at `/var/folders/pt/myrxq2gs5hzcp79v5475_dh40000gp/T/jl_zOno71`

(jl_zOno71) pkg> add Enzyme
   Resolving package versions...
   Installed GPUCompiler ─ v1.3.2
  Installing artifacts ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/1
    Updating `/private/var/folders/pt/myrxq2gs5hzcp79v5475_dh40000gp/T/jl_zOno71/Project.toml`
  [7da242da] + Enzyme v0.13.35
    Updating `/private/var/folders/pt/myrxq2gs5hzcp79v5475_dh40000gp/T/jl_zOno71/Manifest.toml`
  [fa961155] + CEnum v0.5.0
  [7da242da] + Enzyme v0.13.35
  [f151be2c] + EnzymeCore v0.8.8
  [e2ba6199] + ExprTools v0.1.10
  [61eb1bfa] + GPUCompiler v1.3.2
  [692b3bcd] + JLLWrappers v1.7.0
  [929cbde3] + LLVM v9.2.0
  [d8793406] + ObjectFile v0.4.4
  [aea7be01] + PrecompileTools v1.3.0
  [21216c6a] + Preferences v1.4.3
  [189a3867] + Reexport v1.2.2
  [6c6a2e73] + Scratch v1.2.1
  [53d494c1] + StructIO v0.3.1
  [a759f4b9] + TimerOutputs v0.5.28
  [7cc45869] + Enzyme_jll v0.0.173+0
  [dad2f222] + LLVMExtra_jll v0.0.35+0
  [0dad84c5] + ArgTools v1.1.2
  [56f22d72] + Artifacts v1.11.0
  [2a0f44e3] + Base64 v1.11.0
  [ade2ca70] + Dates v1.11.0
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching v1.11.0
  [b77e0a4c] + InteractiveUtils v1.11.0
  [ac6e5ff7] + JuliaSyntaxHighlighting v1.12.0
  [4af54fe1] + LazyArtifacts v1.11.0
  [b27032c2] + LibCURL v0.6.4
  [76f85450] + LibGit2 v1.11.0
  [8f399da3] + Libdl v1.11.0
  [37e2e46d] + LinearAlgebra v1.12.0
  [56ddb016] + Logging v1.11.0
  [d6f4376e] + Markdown v1.11.0
  [ca575930] + NetworkOptions v1.3.0
  [44cfe95a] + Pkg v1.12.0
  [de0858da] + Printf v1.11.0
  [9a3f8284] + Random v1.11.0
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization v1.11.0
  [2f01184e] + SparseArrays v1.12.0
  [f489334b] + StyledStrings v1.11.0
  [fa267f1f] + TOML v1.0.3
  [a4e569a6] + Tar v1.10.0
  [cf7118a7] + UUIDs v1.11.0
  [4ec0a83e] + Unicode v1.11.0
  [e66e0078] + CompilerSupportLibraries_jll v1.3.0+1
  [deac9b47] + LibCURL_jll v8.11.1+1
  [e37daf67] + LibGit2_jll v1.9.0+0
  [29816b5a] + LibSSH2_jll v1.11.3+1
  [14a3606d] + MozillaCACerts_jll v2024.12.31
  [4536629a] + OpenBLAS_jll v0.3.29+0
  [458c3c95] + OpenSSL_jll v3.0.16+0
  [bea87d4a] + SuiteSparse_jll v7.8.3+2
  [83775a58] + Zlib_jll v1.3.1+2
  [8e850b90] + libblastrampoline_jll v5.12.0+0
  [8e850ede] + nghttp2_jll v1.64.0+1
  [3f19e933] + p7zip_jll v17.5.0+2
  ✗ Enzyme
Precompiling packages finished.
  12 dependencies successfully precompiled in 219 seconds. 33 already precompiled.
  1 dependency errored.
  For a report of the errors see `julia> err`. To retry use `pkg> precompile`

julia> err
PkgPrecompileError: The following 1 direct dependency failed to precompile:

Enzyme 

Failed to precompile Enzyme [7da242da-08ed-463a-9acd-ee780be4f1d9] to "/Users/guillaumedalle/.julia/compiled/v1.12/Enzyme/jl_b5zfGn".
ERROR: LoadError: UndefVarError: `verbose_stmt_info` not defined in `Compiler`
Suggestion: check for spelling errors or missing imports.
Stacktrace:
  [1] getproperty(x::Module, f::Symbol)
    @ Base ./Base_compiler.jl:48
  [2] top-level scope
    @ ~/.julia/packages/Enzyme/g1jMR/src/compiler/interpreter.jl:269
  [3] include(mapexpr::Function, mod::Module, _path::String)
    @ Base ./Base.jl:304
  [4] top-level scope
    @ ~/.julia/packages/Enzyme/g1jMR/src/compiler.jl:1414
  [5] include(mapexpr::Function, mod::Module, _path::String)
    @ Base ./Base.jl:304
  [6] top-level scope
    @ ~/.julia/packages/Enzyme/g1jMR/src/Enzyme.jl:127
  [7] include(mod::Module, _path::String)
    @ Base ./Base.jl:303
  [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt128}}, source::Nothing)
    @ Base ./loading.jl:3015
  [9] top-level scope
    @ stdin:5
 [10] eval(m::Module, e::Any)
    @ Core ./boot.jl:489
 [11] include_string(mapexpr::typeof(identity), mod::Module, code::String, filename::String)
    @ Base ./loading.jl:2861
 [12] include_string
    @ ./loading.jl:2871 [inlined]
 [13] exec_options(opts::Base.JLOptions)
    @ Base ./client.jl:326
 [14] _start()
    @ Base ./client.jl:560
in expression starting at /Users/guillaumedalle/.julia/packages/Enzyme/g1jMR/src/compiler/interpreter.jl:1
in expression starting at /Users/guillaumedalle/.julia/packages/Enzyme/g1jMR/src/compiler.jl:1
in expression starting at /Users/guillaumedalle/.julia/packages/Enzyme/g1jMR/src/Enzyme.jl:1
in expression starting at stdin:

Version:

julia> versioninfo()
Julia Version 1.12.0-beta1
Commit c175ace780d (2025-04-02 11:19 UTC)
Build Info:
  Official https://julialang.org release
Platform Info:
  OS: macOS (arm64-apple-darwin24.0.0)
  CPU: 11 × Apple M3 Pro
  WORD_SIZE: 64
  LLVM: libLLVM-18.1.7 (ORCJIT, apple-m3)
  GC: Built with stock GC
Threads: 1 default, 1 interactive, 1 GC (on 5 virtual cores)

gdalle avatar Apr 04 '25 17:04 gdalle

IIRC Base.Compiler.verbose_stmt_info was removed between 1.11 and 1.12, so until versions < 1.12 are dropped it can be conditionally defined with @static if isdefined(Core.Compiler, :verbose_stmt_info) ... end.

Enzyme will also have to adjust to https://github.com/JuliaLang/julia/pull/55575, using the Future stackless API in its overload of abstract_call_gf_by_type. There might be other issues, I haven't looked further than that.

serenity4 avatar Apr 10 '25 21:04 serenity4

What's the plan for pre support in Enzyme? Should we assume that it will try to get pre working ASAP or wait until release? If it waits until release I'll want to refactor tests so that Enzyme tests are separated to be all in their own set that are skipped for pre, while if not then I'd prefer to just "leave as is". For now it shuts down a lot of tests from running on pre so it would be good to establish a policy to know what to expect here.

ChrisRackauckas avatar Apr 23 '25 08:04 ChrisRackauckas

Same result on the latest Julia 1.12.0-beta3

gdalle avatar May 15 '25 11:05 gdalle

1.12 has repeatedly broken GPUCompiler during the pre release process, as an example https://github.com/JuliaGPU/GPUCompiler.jl/issues/695

The Abstract Interpretation changes are fairly in-depth and 1.12 will force opaque pointers.

vchuravy avatar May 15 '25 11:05 vchuravy

What's the plan for pre support in Enzyme? Should we assume that it will try to get pre working ASAP or wait until release? If it waits until release I'll want to refactor tests so that Enzyme tests are separated to be all in their own set that are skipped for pre, while if not then I'd prefer to just "leave as is". For now it shuts down a lot of tests from running on pre so it would be good to establish a policy to know what to expect here.

There really is no plan, we are still playing catch up to 1.11, and it would be very welcome if someone else would step up, if they care about pre support. Personally, I have very little development time currently and are splitting that across multiple projects.

vchuravy avatar May 15 '25 13:05 vchuravy

Sorry, I didn't mean to put additional stress on you, and I know how thinly stretched the devs are. Just thought I'd report on the latest beta version, nothing more.

gdalle avatar May 15 '25 15:05 gdalle