Typo in latex equation, and gradient interpretation?

[DrVogt]

Howdy,

https://diffeq.sciml.ai/stable/analysis/sensitivity/#Example-continuous-adjoints-on-an-energy-functional

After seeing your response should the latex for the integrand of G(p) be (\sum u_i)^2 instead of \sum u_i^2 ?

What lead to my initial confusion is that "dp" which I thought would be dG/dp, wasn't agreeing with the other gradient computations.

For example julia> include("adj.jl") starting([-57.500505903032035, -14.31208917581963], [21.050806433683196 -101.40832937095652 63.1926964723678]) [25.583173069963085, -109.33410810749508, 108.11259400311171] [25.583172360904236, -109.33410817418671, 108.11259398054582] ([25.583173069945055, -109.33410810749571, 108.11259400311152] (tracked),) [25.583173069947176, -109.33410810749524, 108.11259400311181]

julia> include("adj.jl") starting([-57.500505903032035, -14.31208917581963], [21.050806433683196 -101.40832937095652 63.1926964723678]) [25.50063723180531, -77.25507741525394, 93.53212756432573] [25.500636489106967, -77.25507746056326, 93.53212754708031] ([25.50063723178915, -77.25507741525406, 93.53212756432508] (tracked),) [25.500637231791032, -77.25507741525361, 93.53212756432536]

I'm not sure why the results are different when I run it on two seperate runs, but on specific runs they agree to some tolerance. The problem is that the adjoint dp does not agree with these gradients - what am I missing?

res = adjoint_sensitivities(sol,Vern9(),g,nothing,dg,abstol=1e-8,
                                 reltol=1e-8,iabstol=1e-8,ireltol=1e-8)
res2 = ForwardDiff.gradient(G,[1.5,1.0,3.0])
res3 = Calculus.gradient(G,[1.5,1.0,3.0])
res4 = Tracker.gradient(G,[1.5,1.0,3.0])
res5 = ReverseDiff.gradient(G,[1.5,1.0,3.0])
println(res)
println(res2)
println(res3)
println(res4)
println(res5)

using DiffEqFlux
 using Flux
 using OrdinaryDiffEq
 using DiffEqSensitivity
 using ForwardDiff,Calculus,Tracker
 using ReverseDiff
print("starting")
function f(du,u,p,t)
  du[1] = dx = p[1]*u[1] - p[2]*u[1]*u[2]
  du[2] = dy = -p[3]*u[2] + u[1]*u[2]
end

p = [1.5,1.0,3.0]
prob = ODEProblem(f,[1.0;1.0],(0.0,10.0),p)
sol =solve(prob,Vern9(),abstol=1e-10,reltol=1e-10)

g(u,p,t) = (sum(u).^2) ./ 2
function dg(out,u,p,t)
  out[1]= u[1] + u[2]
  out[2]= u[1] + u[2]
end
# function dg(out,u,p,t)
#   out[1]= u[1]
#   out[2]=  u[2]
# end

ts = 0:0.5:10
function G(p)

  tmp_prob = remake(prob,u0=convert.(eltype(p),prob.u0),p=p)
  sol = solve(tmp_prob,Vern9(),abstol=1e-14,reltol=1e-14,saveat=ts,
              sensealg=SensitivityADPassThrough())
  A = convert(Array,sol)
  sum(((1 .- A).^2)./2)
end
res = adjoint_sensitivities(sol,Vern9(),g,nothing,dg,abstol=1e-8,
                                 reltol=1e-8,iabstol=1e-8,ireltol=1e-8)
res2 = ForwardDiff.gradient(G,[1.5,1.0,3.0])
res3 = Calculus.gradient(G,[1.5,1.0,3.0])
res4 = Tracker.gradient(G,[1.5,1.0,3.0])
res5 = ReverseDiff.gradient(G,[1.5,1.0,3.0])
println(res)
println(res2)
println(res3)
println(res4)
println(res5)

[ChrisRackauckas]

After seeing your response should the latex for the integrand of G(p) be (\sum u_i)^2 instead of \sum u_i^2 ?

Yeah looks like that's a typo.

Compare it to the CI test https://github.com/SciML/DiffEqSensitivity.jl/blob/master/test/adjoint.jl#L382-L499 that is run daily and should be accurate. It looks like you're testing two different things: you're testing the continuous adjoint vs a discrete adjoint. The test code direct tests the continuous cost in the integral definition and sees agreeance.

[ChrisRackauckas]

Updated.