DifferentialEquations.jl icon indicating copy to clipboard operation
DifferentialEquations.jl copied to clipboard

autodiff in default algorithm

Open ArnoStrouwen opened this issue 10 months ago • 5 comments

Is it possible to do a try/catch on the evaluation of the Jac to turn on autodiff? https://github.com/SciML/DifferentialEquations.jl/blob/master/src/ode_default_alg.jl#L49

ArnoStrouwen avatar Apr 21 '24 12:04 ArnoStrouwen

try/catch always has a ton of overhead so that's not good for the default method. It only works in the adjoints because the cost of adjoint differentiation is already so high when it makes sense, so the overhead is always negligible. For many default ODEs, you'd just hit a standard explicit RK method and in those cases the try/catch would be noticeable but never used.

Maybe the right thing to do would be Core.return_type and check if it's a Union{}. That would tell you if it's provable from the types to error? https://github.com/SciML/OrdinaryDiffEq.jl/pull/2103 should merge this week, and with that we may want to reconsider how this could be done @oscardssmith

ChrisRackauckas avatar Apr 29 '24 06:04 ChrisRackauckas

I'm really not sure why you think try catch has so much overhead. I measure it as ~10ns in the case where no exception is thrown.

julia> function f(n)
           try
               return n[]+1
           catch
           end
       end
f (generic function with 1 method)

julia> @btime f(Any[100])
  34.174 ns (1 allocation: 64 bytes)
101

julia> function g(n)
           return n[]+1
       end
g (generic function with 1 method)

julia> @btime g(Any[100])
  24.751 ns (1 allocation: 64 bytes)
101

oscardssmith avatar Apr 29 '24 12:04 oscardssmith

Oh okay, then maybe in the new form we can make the AD more dynamic? Though in future versions when using ADTypes with DI this might be an issue. @gdalle something to consider.

ChrisRackauckas avatar Apr 29 '24 12:04 ChrisRackauckas

Also worth pointing out that catching an error is pretty expensive.

julia> @btime f(Any["hi"])
  20.599 μs (3 allocations: 128 bytes)

oscardssmith avatar Apr 29 '24 12:04 oscardssmith

Oh okay, then maybe in the new form we can make the AD more dynamic? Though in future versions when using ADTypes with DI this might be an issue.

Probably related to https://github.com/gdalle/DifferentiationInterface.jl/issues/164

gdalle avatar Apr 29 '24 13:04 gdalle