Enzyme.jl icon indicating copy to clipboard operation
Enzyme.jl copied to clipboard

Wrong gradients due to global without shadow

Open vchuravy opened this issue 2 years ago • 6 comments

const data = zeros(3)

function h(x, i)
    data[1] = x
    y = data[i]
   x*y
end

julia> Enzyme.autodiff(Reverse, h, Active, Active(1.0), Const(1))
((1.0, nothing),)
# Expected ((2.0, nothing), )

Amusingly if Enzyme can statically prove that the index is the same, the answer is correct.

julia> function g(x, i)
         data[i] = x
         y = data[i]
         y*x
       end
g (generic function with 2 methods)

julia> Enzyme.autodiff(Reverse, g, Active, Active(1.0), Const(1))
((2.0, nothing),)

@tgymnich For this case I would like to have a warning. Would also be helpful for arguments where enzyme deduces const, e.g. having a MaybeConst.

vchuravy avatar Feb 22 '23 23:02 vchuravy

C example also wrong in a different way: https://fwd.gymni.ch/doh4qV

tgymnich avatar Feb 22 '23 23:02 tgymnich

Shouldn't it be something like https://fwd.gymni.ch/QgKyvm

With a global const in C it can do store load forwarding.

vchuravy avatar Feb 23 '23 05:02 vchuravy

@tgymnich how is your case wrong, that looks fine to me.

wsmoses avatar Mar 05 '23 22:03 wsmoses

In discussion with Billy the right answer is to treat inttoptr here equivalent to a extern global in C.

vchuravy avatar Mar 05 '23 22:03 vchuravy