Enzyme.jl
Enzyme.jl copied to clipboard
EnzymeAD
Turing.jl error with AdvancedHMC
using Turing, Enzyme
@model function hmcmatrixsup()
return v ~ Wishart(7, [1 0.5; 0.5 1])
end
model_f = hmcmatrixsup()
rng = Turing.Random.default_rng()
alg = HMC(0.15, 7; adtype=Turing.ADTypes.AutoEnzyme())
spl = Turing.Sampler(alg, model_f)
vi = Turing.DynamicPPL.default_varinfo(rng, model_f, spl)
DynamicPPL.initialstep(rng, model_f, spl, vi)
fails on https://github.com/TuringLang/Turing.jl/blob/4766fdde20ca9502472afcb991bd05f984d3ad35/src/mcmc/hmc.jl#L184
error msg
ERROR: Enzyme execution failed.
Mismatched activity for: store {} addrspace(10)* %8, {} addrspace(10)** %.repack, align 8, !dbg !285, !noalias !234 const val: %8 = load atomic {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %7 unordered, align 8, !dbg !193, !tbaa !200, !nonnull !4, !dereferenceable !204, !align !191
Type tree: {[-1]:Pointer, [-1,0]:Pointer, [-1,0,0]:Pointer, [-1,0,0,-1]:Integer, [-1,0,8]:Integer, [-1,0,9]:Integer, [-1,0,10]:Integer, [-1,0,11]:Integer, [-1,0,12]:Integer, [-1,0,13]:Integer, [-1,0,14]:Integer, [-1,0,15]:Integer, [-1,0,16]:Integer, [-1,0,17]:Integer, [-1,0,18]:Integer, [-1,0,19]:Integer, [-1,0,20]:Integer, [-1,0,21]:Integer, [-1,0,22]:Integer, [-1,0,23]:Integer, [-1,0,24]:Integer, [-1,0,25]:Integer, [-1,0,26]:Integer, [-1,0,27]:Integer, [-1,0,28]:Integer, [-1,0,29]:Integer, [-1,0,30]:Integer, [-1,0,31]:Integer, [-1,0,32]:Integer, [-1,0,33]:Integer, [-1,0,34]:Integer, [-1,0,35]:Integer, [-1,0,36]:Integer, [-1,0,37]:Integer, [-1,0,38]:Integer, [-1,0,39]:Integer, [-1,8]:Pointer, [-1,8,0]:Pointer, [-1,8,8]:Integer, [-1,8,9]:Integer, [-1,8,10]:Integer, [-1,8,11]:Integer, [-1,8,12]:Integer, [-1,8,13]:Integer, [-1,8,14]:Integer, [-1,8,15]:Integer, [-1,8,16]:Integer, [-1,8,17]:Integer, [-1,8,18]:Integer, [-1,8,19]:Integer, [-1,8,20]:Integer, [-1,8,21]:Integer, [-1,8,22]:Integer, [-1,8,23]:Integer, [-1,8,24]:Integer, [-1,8,25]:Integer, [-1,8,26]:Integer, [-1,8,27]:Integer, [-1,8,28]:Integer, [-1,8,29]:Integer, [-1,8,30]:Integer, [-1,8,31]:Integer, [-1,8,32]:Integer, [-1,8,33]:Integer, [-1,8,34]:Integer, [-1,8,35]:Integer, [-1,8,36]:Integer, [-1,8,37]:Integer, [-1,8,38]:Integer, [-1,8,39]:Integer, [-1,16]:Pointer, [-1,16,0]:Pointer, [-1,16,0,-1]:Integer, [-1,16,8]:Integer, [-1,16,9]:Integer, [-1,16,10]:Integer, [-1,16,11]:Integer, [-1,16,12]:Integer, [-1,16,13]:Integer, [-1,16,14]:Integer, [-1,16,15]:Integer, [-1,16,16]:Integer, [-1,16,17]:Integer, [-1,16,18]:Integer, [-1,16,19]:Integer, [-1,16,20]:Integer, [-1,16,21]:Integer, [-1,16,22]:Integer, [-1,16,23]:Integer, [-1,16,24]:Integer, [-1,16,25]:Integer, [-1,16,26]:Integer, [-1,16,27]:Integer, [-1,16,28]:Integer, [-1,16,29]:Integer, [-1,16,30]:Integer, [-1,16,31]:Integer, [-1,16,32]:Integer, [-1,16,33]:Integer, [-1,16,34]:Integer, [-1,16,35]:Integer, [-1,16,36]:Integer, [-1,16,37]:Integer, [-1,16,38]:Integer, [-1,16,39]:Integer, [-1,24]:Integer, [-1,25]:Integer, [-1,26]:Integer, [-1,27]:Integer, [-1,28]:Integer, [-1,29]:Integer, [-1,30]:Integer, [-1,31]:Integer, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Integer, [-1,40]:Integer, [-1,41]:Integer, [-1,42]:Integer, [-1,43]:Integer, [-1,44]:Integer, [-1,45]:Integer, [-1,46]:Integer, [-1,47]:Integer, [-1,48]:Integer, [-1,49]:Integer, [-1,50]:Integer, [-1,51]:Integer, [-1,52]:Integer, [-1,53]:Integer, [-1,54]:Integer, [-1,55]:Integer, [-1,56]:Integer, [-1,57]:Integer, [-1,58]:Integer, [-1,59]:Integer, [-1,60]:Integer, [-1,61]:Integer, [-1,62]:Integer, [-1,63]:Integer}
llvalue= %8 = load atomic {} addrspace(10)*, {} addrspace(10)* addrspace(11)* %7 unordered, align 8, !dbg !193, !tbaa !200, !nonnull !4, !dereferenceable !204, !align !191
You may be using a constant variable as temporary storage for active memory (https://enzyme.mit.edu/julia/stable/faq/#Activity-of-temporary-storage). If not, please open an issue, and either rewrite this variable to not be conditionally active or use Enzyme.API.runtimeActivity!(true) as a workaround for now
Stacktrace:
[1] VarInfo
@ ~/.julia/packages/DynamicPPL/93t4P/src/varinfo.jl:117
Stacktrace:
[1] throwerr(cstr::Cstring)
@ Enzyme.Compiler ~/.julia/packages/Enzyme/r8mFE/src/compiler.jl:1797
versions: Julia Version 1.7.3 [7da242da] Enzyme v0.12.26 [fce5fe82] Turing v0.33.3
What happens if you set Enzyme.API.runtimeActivity!(true) as the error message says?
Turing already does this automatically so this isn't the cause of the issue at hand.
It hangs and crushes Julia, shown something like
signal (11): Segmentation fault: 11
in expression starting at REPL[11]:1
unknown function (ip: 0x123e2497a)
Allocations: 277657897 (Pool: 277514439; Big: 143458); GC: 107
Likely related: https://github.com/EnzymeAD/Enzyme.jl/issues/1649 https://github.com/EnzymeAD/Enzyme.jl/issues/1081
I mentioned https://github.com/EnzymeAD/Enzyme.jl/issues/1649#issuecomment-2242993090: a test on both Wishart and Chol should be added to the test suite on the Julia side.
Uh maybe but also likely not. 1.7 and older Julia had a bunch of GC issues that we fixed in later versions of Julia.
If you can make a much smaller MWE (eg without external packages and as simplified as possible), we can investigate
On Fri, Aug 2, 2024 at 11:01 AM Hong Ge @.***> wrote:
Likely related: #1649 https://github.com/EnzymeAD/Enzyme.jl/issues/1649 #1081 https://github.com/EnzymeAD/Enzyme.jl/issues/1081
— Reply to this email directly, view it on GitHub https://github.com/EnzymeAD/Enzyme.jl/issues/1696#issuecomment-2265597446, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJTUXCYYNSCGU4SZKR6EITZPONMHAVCNFSM6AAAAABL4FY6KOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRVGU4TONBUGY . You are receiving this because you commented.Message ID: <EnzymeAD/Enzyme. @.***>
~~On 1.10.4, it also fails (on my machine MacOS)~~ (edited, never mind, it works!!)
That latter issue again indicates you did not set runtime activity to true (see the error message). This must be done shortly after loading Enzyme
On Fri, Aug 2, 2024 at 11:04 AM Xianda Sun @.***> wrote:
On 1.10.4, it also fails (on my machine MacOS) Gives
ERROR: Enzyme execution failed. Mismatched activity for: store {} addrspace(10)* %getfield.i, {} addrspace(10)** %.fca.0.0.0.gep5, align 8, !dbg !359, !noalias !174 const val: %getfield.i = load atomic {} addrspace(10), {} addrspace(10)** %.fca.0.0.0.0.gep unordered, align 8, !dbg !191, !alias.scope !206, !noalias !209, !nonnull !0, !dereferenceable !214, !align !215 Type tree: {[-1]:Pointer, [-1,0]:Pointer, [-1,0,0]:Pointer, [-1,0,0,-1]:Integer, [-1,0,8]:Integer, [-1,0,9]:Integer, [-1,0,10]:Integer, [-1,0,11]:Integer, [-1,0,12]:Integer, [-1,0,13]:Integer, [-1,0,14]:Integer, [-1,0,15]:Integer, [-1,0,16]:Integer, [-1,0,17]:Integer, [-1,0,18]:Integer, [-1,0,19]:Integer, [-1,0,20]:Integer, [-1,0,21]:Integer, [-1,0,22]:Integer, [-1,0,23]:Integer, [-1,0,24]:Integer, [-1,0,25]:Integer, [-1,0,26]:Integer, [-1,0,27]:Integer, [-1,0,28]:Integer, [-1,0,29]:Integer, [-1,0,30]:Integer, [-1,0,31]:Integer, [-1,0,32]:Integer, [-1,0,33]:Integer, [-1,0,34]:Integer, [-1,0,35]:Integer, [-1,0,36]:Integer, [-1,0,37]:Integer, [-1,0,38]:Integer, [-1,0,39]:Integer, [-1,8]:Pointer, [-1,8,0]:Pointer, [-1,8,8]:Integer, [-1,8,9]:Integer, [-1,8,10]:Integer, [-1,8,11]:Integer, [-1,8,12]:Integer, [-1,8,13]:Integer, [-1,8,14]:Integer, [-1,8,15]:Integer, [-1,8,16]:Integer, [-1,8,17]:Integer, [-1,8,18]:Integer, [-1,8,19]:Integer, [-1,8,20]:Integer, [-1,8,21]:Integer, [-1,8,22]:Integer, [-1,8,23]:Integer, [-1,8,24]:Integer, [-1,8,25]:Integer, [-1,8,26]:Integer, [-1,8,27]:Integer, [-1,8,28]:Integer, [-1,8,29]:Integer, [-1,8,30]:Integer, [-1,8,31]:Integer, [-1,8,32]:Integer, [-1,8,33]:Integer, [-1,8,34]:Integer, [-1,8,35]:Integer, [-1,8,36]:Integer, [-1,8,37]:Integer, [-1,8,38]:Integer, [-1,8,39]:Integer, [-1,16]:Pointer, [-1,16,0]:Pointer, [-1,16,0,-1]:Integer, [-1,16,8]:Integer, [-1,16,9]:Integer, [-1,16,10]:Integer, [-1,16,11]:Integer, [-1,16,12]:Integer, [-1,16,13]:Integer, [-1,16,14]:Integer, [-1,16,15]:Integer, [-1,16,16]:Integer, [-1,16,17]:Integer, [-1,16,18]:Integer, [-1,16,19]:Integer, [-1,16,20]:Integer, [-1,16,21]:Integer, [-1,16,22]:Integer, [-1,16,23]:Integer, [-1,16,24]:Integer, [-1,16,25]:Integer, [-1,16,26]:Integer, [-1,16,27]:Integer, [-1,16,28]:Integer, [-1,16,29]:Integer, [-1,16,30]:Integer, [-1,16,31]:Integer, [-1,16,32]:Integer, [-1,16,33]:Integer, [-1,16,34]:Integer, [-1,16,35]:Integer, [-1,16,36]:Integer, [-1,16,37]:Integer, [-1,16,38]:Integer, [-1,16,39]:Integer, [-1,24]:Integer, [-1,25]:Integer, [-1,26]:Integer, [-1,27]:Integer, [-1,28]:Integer, [-1,29]:Integer, [-1,30]:Integer, [-1,31]:Integer, [-1,32]:Integer, [-1,33]:Integer, [-1,34]:Integer, [-1,35]:Integer, [-1,36]:Integer, [-1,37]:Integer, [-1,38]:Integer, [-1,39]:Integer, [-1,40]:Integer, [-1,41]:Integer, [-1,42]:Integer, [-1,43]:Integer, [-1,44]:Integer, [-1,45]:Integer, [-1,46]:Integer, [-1,47]:Integer, [-1,48]:Integer, [-1,49]:Integer, [-1,50]:Integer, [-1,51]:Integer, [-1,52]:Integer, [-1,53]:Integer, [-1,54]:Integer, [-1,55]:Integer, [-1,56]:Integer, [-1,57]:Integer, [-1,58]:Integer, [-1,59]:Integer, [-1,60]:Integer, [-1,61]:Integer, [-1,62]:Integer, [-1,63]:Integer} llvalue= %getfield.i = load atomic {} addrspace(10), {} addrspace(10)** %.fca.0.0.0.0.gep unordered, align 8, !dbg !191, !alias.scope !206, !noalias !209, !nonnull !0, !dereferenceable !214, !align !215 You may be using a constant variable as temporary storage for active memory (https://enzyme.mit.edu/julia/stable/faq/#Activity-of-temporary-storage). If not, please open an issue, and either rewrite this variable to not be conditionally active or use Enzyme.API.runtimeActivity!(true) as a workaround for now
Stacktrace: [1] _evaluate!! @ ~/.julia/packages/DynamicPPL/93t4P/src/model.jl:973 [2] evaluate_threadunsafe!! @ ~/.julia/packages/DynamicPPL/93t4P/src/model.jl:946 [3] evaluate!! @ ~/.julia/packages/DynamicPPL/93t4P/src/model.jl:894 [4] logdensity @ ~/.julia/packages/DynamicPPL/93t4P/src/logdensityfunction.jl:140 [5] logdensity @ ~/.julia/packages/DynamicPPL/93t4P/src/logdensityfunction.jl:0
Stacktrace: [1] throwerr(cstr::Cstring) @ Enzyme.Compiler ~/.julia/packages/Enzyme/r8mFE/src/compiler.jl:1797 [2] _evaluate!! @ ~/.julia/packages/DynamicPPL/93t4P/src/model.jl:973 [inlined] [3] evaluate_threadunsafe!! @ ~/.julia/packages/DynamicPPL/93t4P/src/model.jl:946 [inlined] [4] evaluate!! @ ~/.julia/packages/DynamicPPL/93t4P/src/model.jl:894 [inlined] [5] logdensity @ ~/.julia/packages/DynamicPPL/93t4P/src/logdensityfunction.jl:140 [inlined] [6] logdensity @ ~/.julia/packages/DynamicPPL/93t4P/src/logdensityfunction.jl:0 [inlined] [7] diffejulia_logdensity_7234_inner_2wrap @ ~/.julia/packages/DynamicPPL/93t4P/src/logdensityfunction.jl:0 [8] macro expansion @ ~/.julia/packages/Enzyme/r8mFE/src/compiler.jl:6819 [inlined] [9] enzyme_call @ ~/.julia/packages/Enzyme/r8mFE/src/compiler.jl:6419 [inlined] [10] CombinedAdjointThunk @ ~/.julia/packages/Enzyme/r8mFE/src/compiler.jl:6296 [inlined] [11] autodiff @ ~/.julia/packages/Enzyme/r8mFE/src/Enzyme.jl:314 [inlined] [12] autodiff @ ~/.julia/packages/Enzyme/r8mFE/src/Enzyme.jl:326 [inlined] [13] logdensity_and_gradient @ ~/.julia/packages/LogDensityProblemsAD/rBlLq/ext/LogDensityProblemsADEnzymeExt.jl:73 [inlined] [14] ∂logπ∂θ @ ~/.julia/packages/Turing/r3Hmj/src/mcmc/hmc.jl:180 [inlined] [15] ∂H∂θ(h::AdvancedHMC.Hamiltonian{…}, θ::Vector{…}) @ AdvancedHMC ~/.julia/packages/AdvancedHMC/AlvV4/src/hamiltonian.jl:38 [16] phasepoint(h::AdvancedHMC.Hamiltonian{…}, θ::Vector{…}, r::Vector{…}) @ AdvancedHMC ~/.julia/packages/AdvancedHMC/AlvV4/src/hamiltonian.jl:74 [17] phasepoint @ ~/.julia/packages/AdvancedHMC/AlvV4/src/hamiltonian.jl:155 [inlined] [18] initialstep(rng::Random.TaskLocalRNG, model::DynamicPPL.Model{…}, spl::DynamicPPL.Sampler{…}, vi_original::DynamicPPL.TypedVarInfo{…}; initial_params::Nothing, nadapts::Int64, @.***{}) @ Turing.Inference ~/.julia/packages/Turing/r3Hmj/src/mcmc/hmc.jl:184 [19] initialstep(rng::Random.TaskLocalRNG, model::DynamicPPL.Model{…}, spl::DynamicPPL.Sampler{…}, vi_original::DynamicPPL.TypedVarInfo{…}) @ Turing.Inference ~/.julia/packages/Turing/r3Hmj/src/mcmc/hmc.jl:151 [20] top-level scope @ ~/Turing.jl/test/temp.jl:13 Some type information was truncated. Use
show(err)to see complete types.— Reply to this email directly, view it on GitHub https://github.com/EnzymeAD/Enzyme.jl/issues/1696#issuecomment-2265603157, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAJTUXAUFYJASVWMTSPGVJDZPONXPAVCNFSM6AAAAABL4FY6KOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDENRVGYYDGMJVG4 . You are receiving this because you commented.Message ID: <EnzymeAD/Enzyme. @.***>
A much smaller MWE is the way to go, will try to produce one when get time. (Maybe @mhauru can take over when back?)
On my todo list, just haven't gotten to it yet, sorry. May still take a week or two to find time.
#1649 seems similar and also works on 1.10 but crashes on 1.7. I'll wait for that to be sorted out before putting in the work to minimise this.
@mhauru we've dropped Julia before 1.10, so I presume this error is moot and no longer applies. Reopen with a MWE if there's still an issue.