Flux installation errors in julia 1.7.0-rc1, WSL2

[gwd999]

OS: WSL2 Ubuntu 20.04 - on Windows 10 Pro

julia 1.7.0-rc1 build from github repo on Ubuntu 20.04

julia > versioninfo() 
Julia Version 1.7.0-rc1
Commit 9eade6195e (2021-09-12 06:45 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-12.0.1 (ORCJIT, icelake-client)
Environment:
  JULIA_NUM_THREADS = 6

I ran

julia> using Pkg; Pkg.add("Flux")
... 
...
Precompiling project...
  Progress [========================================>]  51/52
  Progress [========================================>]  51/52
  ✗ CUDA
  ✗ Flux
  50 dependencies successfully precompiled in 181 seconds (123 already precompiled)
  2 dependencies errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the packages
[ Info: Precompiling Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]
WARNING: Method definition frule(Any, Base.RedirectStdStream, Union{Base.IOStream, Base.LibuvStream}) in module ChainRules at /home/gwd/.julia/packages/ChainRules/Ipuva/src/rulesets/Base/nondiff.jl:297 overwritten at /home/gwd/.julia/packages/ChainRules/Ipuva/src/rulesets/Base/nondiff.jl:296.
  ** incremental compilation may be fatally broken for this module **
...
WARNING: Method definition rrule##kw(Any, typeof(ChainRulesCore.rrule), Base.RedirectStdStream, Union{Base.IOStream, Base.LibuvStream}) in module ChainRules overwritten.
  ** incremental compilation may be fatally broken for this module **

ERROR: LoadError: InitError: could not load symbol "LLVMExtraInitializeAllTargets":
/home/gwd/dev/github/julia/usr/bin/julia: undefined symbol: LLVMExtraInitializeAllTargets
Stacktrace:
  [1] LLVMInitializeAllTargets
    @ ~/.julia/packages/LLVM/srSVa/lib/libLLVM_extra.jl:10 [inlined]
  [2] InitializeAllTargets
    @ ~/.julia/packages/LLVM/srSVa/src/init.jl:58 [inlined]
  [3] __init__()
    @ GPUCompiler ~/.julia/packages/GPUCompiler/AdCnd/src/GPUCompiler.jl:48
  [4] _include_from_serialized(path::String, depmods::Vector{Any})
    @ Base ./loading.jl:768
  [5] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String)
    @ Base ./loading.jl:854
  [6] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1097
  [7] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1013
  [8] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:997
  [9] include
    @ ./Base.jl:420 [inlined]
 [10] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
    @ Base ./loading.jl:1318
 [11] top-level scope
    @ none:1
 [12] eval
    @ ./boot.jl:373 [inlined]
 [13] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [14] top-level scope
    @ none:1
during initialization of module GPUCompiler
in expression starting at /home/gwd/.julia/packages/CUDA/kKJoe/src/CUDA.jl:1
ERROR: LoadError: Failed to precompile CUDA [052768ef-5323-5732-b1bb-66c8b64840ba] to /home/gwd/.julia/compiled/v1.7/CUDA/jl_tQ1220.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
    @ Base ./loading.jl:1466
  [3] compilecache(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1410
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1120
  [5] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1013
  [6] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:997
  [7] include
    @ ./Base.jl:420 [inlined]
  [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1318
  [9] top-level scope
    @ none:1
 [10] eval
    @ ./boot.jl:373 [inlined]
 [11] eval(x::Expr)
    @ Base.MainInclude ./client.jl:453
 [12] top-level scope
    @ none:1
in expression starting at /home/gwd/.julia/packages/Flux/sY3yx/src/Flux.jl:1
ERROR: Failed to precompile Flux [587475ba-b771-5e3f-ad9e-33799f191a9c] to /home/gwd/.julia/compiled/v1.7/Flux/jl_IFE455.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, ignore_loaded_modules::Bool)
   @ Base ./loading.jl:1466
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1410
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1120
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1013
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:997

julia> using Flux
[ Info: Precompiling Flux [587475ba-b771-5e3f-ad9e-33799f191a9c]
WARNING: Method definition frule(Any, Base.RedirectStdStream, Union{Base.IOStream, Base.LibuvStream}) in module ChainRules at /home/gwd/.julia/packages/ChainRules/Ipuva/src/rulesets/Base/nondiff.jl:297 overwritten at /home/gwd/.julia/packages/ChainRules/Ipuva/src/rulesets/Base/nondiff.jl:296.
  ** incremental compilation may be fatally broken for this module **`
.... 

[gwd999]

FYI - some details from a succesfull installation (same time as well WSL2)

    Building Random123 → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/0e8b146557ad1c6deb1367655e052276690e71a3/build.log`
Precompiling project...
  40 dependencies successfully precompiled in 100 seconds (67 already precompiled)

julia> using Flux

julia> versioninfo()
Julia Version 1.6.1
Commit 6aaedecc44 (2021-04-23 05:59 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Core(TM) i7-1065G7 CPU @ 1.30GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, icelake-client)

[gwd999]

Ok I think I solved my problem - seems I got stuck in dependency version conundrum But thanks to https://discourse.julialang.org/t/dynamically-bypass-cuda-jl-for-non-gpu-machine/66608/7 upgrading "LLVM.jl" to version > 4 [just a Pkg.update() b/c Julia 1.7 requires LLVM4 or higher] solved the errors for CUDA and from there on for Flux as well. Gonna leave this temporarily open for now in case someone else stumbles across this as well.

[igorrivin]

I am having the same problem, and Pkg.update() does not fix it. Any other ideas? I can't run (or even precompile) CUDA or Flux. On an nVidia DGX, no less.

[ToucheSir]

Since you're running on a DGX, I assume this is not a WSL problem. Make sure you're using Julia 1.7 stable instead of the (now obsolete) RCs, and try with a fresh env first. If you run into a problem installing CUDA.jl itself, that's something for their issue tracker.

[DhairyaLGandhi]

I'm sure there's a branch that doesn't have the cuda dependence, so maybe trying removing the cuda dep is worthwhile for an experiment

[kpa28-git]

@igorrivin I'm using Linux and fixed the problem by trying add [email protected], to see what in my environment was preventing the update to the latest version of CUDA. One of Turing's dependencies was causing the issue so I removed Turing and reran add [email protected]. In the process LLVM was also updated to the latest version (4.7.0). Everything precompiles now. I tried re-adding Turing but it broke everything again so I'm not using it for now.