Issues when using forward over reverse or reverse over forward

[michel2323]

trafficstars

using Enzyme

n = 10

x = [i/(1.0+i) for i in 1:n]
dx  = ones(n)
rx  = zeros(n); drx = zeros(n)
y   = zeros(1); dy  = zeros(1)
ry  = ones(1); dry = zeros(1)

function foo(x, y)
    # y .= mapreduce(identity,*, x; dims=:, init=1.0)
    y .= mapreduce(identity,*, x; dims=:)
    return nothing
end

function bar(x, y)
    Enzyme.autodiff_deferred(foo, Const, x, y)
    return nothing
end

_x = Duplicated(Duplicated(x,rx), Duplicated(dx,drx))
_y = Duplicated(Duplicated(y,ry), Duplicated(dy,dry))

fwddiff(bar, _x, _y)

[vchuravy]

Fails with 0.9.1 locally with:

double, [0,56]:Pointer, [0,56,0]:Float@double}
  %21 = fmul double %14, %18, !dbg !40
julia: /workspace/srcdir/Enzyme/enzyme/Enzyme/GradientUtils.cpp:4246: llvm::Value* GradientUtils::invertPointerM(llvm::Value*, llvm::IRBuilder<>&, bool): Assertion `arg->getType()->isIntOrIntVectorTy()' failed.

signal (6): Aborted
in expression starting at REPL[12]:1
__pthread_kill_implementation at /usr/lib/libc.so.6 (unknown line)
raise at /usr/lib/libc.so.6 (unknown line)
abort at /usr/lib/libc.so.6 (unknown line)
__assert_fail_base.cold at /usr/lib/libc.so.6 (unknown line)
__assert_fail at /usr/lib/libc.so.6 (unknown line)
invertPointerM at /workspace/srcdir/Enzyme/enzyme/Enzyme/GradientUtils.cpp:4246
invertPointerM at /workspace/srcdir/Enzyme/enzyme/Enzyme/GradientUtils.cpp:4440
invertPointerM at /workspace/srcdir/Enzyme/enzyme/Enzyme/GradientUtils.cpp:4129
createTerminator at /workspace/srcdir/Enzyme/enzyme/Enzyme/EnzymeLogic.cpp:2575
CreateForwardDiff at /workspace/srcdir/Enzyme/enzyme/Enzyme/EnzymeLogic.cpp:4206
visitCallInst at /workspace/srcdir/Enzyme/enzyme/Enzyme/AdjointGenerator.h:10172
delegateCallInst at /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include/llvm/IR/InstVisitor.h:299 [inlined]
visitCall at /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include/llvm/IR/Instruction.def:209 [inlined]
visit at /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include/llvm/IR/Instruction.def:209
visit at /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include/llvm/IR/InstVisitor.h:112 [inlined]
CreateForwardDiff at /workspace/srcdir/Enzyme/enzyme/Enzyme/EnzymeLogic.cpp:4203
EnzymeCreateForwardDiff at /workspace/srcdir/Enzyme/enzyme/Enzyme/CApi.cpp:395
EnzymeCreateForwardDiff at /home/vchuravy/src/Enzyme/src/api.jl:122
enzyme! at /home/vchuravy/src/Enzyme/src/compiler.jl:2595

[vchuravy]

We will need forward mode variants for enzyme_code_llvm.

[michel2323]

Updating output with 0.9.2 log.tar.gz with Enzyme.API.printall!(true) Enzyme.API.printtype!(true).

[michel2323]

@vchuravy @wsmoses Updated the MWE and title. What breaks the differentiation is the function it dispatches to with the default * initialization (1.0). log.txt attached. Using the commented-out line and manually setting the initial value works.

[wsmoses]

https://github.com/EnzymeAD/Enzyme/issues/597

[wsmoses]

This now hits:

ERROR: LoadError: UndefRefError: access to undefined reference
Stacktrace:
 [1] unsafe_convert
   @ /mnt/Data/git/Enzyme.jl/src/compiler.jl:4493 [inlined]
 [2] macro expansion
   @ /mnt/Data/git/Enzyme.jl/src/compiler.jl:4736 [inlined]
 [3] enzyme_call(::Ptr{Nothing}, ::Type{Enzyme.Compiler.ForwardModeThunk}, ::Type{Val{0x0000000000000001}}, ::Val{true}, ::Type{Tuple{Const{Core.MethodInstance}, Const{Symbol}}}, ::Type{Duplicated{Any}}, ::typeof(getproperty), ::Nothing, ::Const{Core.MethodInstance}, ::Const{Symbol})
   @ Enzyme.Compiler /mnt/Data/git/Enzyme.jl/src/compiler.jl:4532
 [4] (::Enzyme.Compiler.ForwardModeThunk{typeof(getproperty), Duplicated{Any}, Tuple{Const{Core.MethodInstance}, Const{Symbol}}, Val{0x0000000000000001}, Nothing, Val{true}()})(::Const{Core.MethodInstance}, ::Vararg{Any})
   @ Enzyme.Compiler /mnt/Data/git/Enzyme.jl/src/compiler.jl:4517
in expression starting at /mnt/Data/git/Enzyme.jl/frev.jl:25

The reason for this by the way is that your original code is somehow type instable and issuing a call to jl_invoke. The first time we differentiate that, its fine. Differentiating the differentiated jl_invoke handler, however, hits an issue.

[wsmoses]

Now duplicate of https://github.com/EnzymeAD/Enzyme.jl/issues/293

[vchuravy]

@wsmoses this is fixed by #408 but prints a ridiculously large type-tree.

[vchuravy]

@michel2323 can you add a test that checks that the values are what you expect?

[wsmoses]

@michel2323 same here please verify and create a test.