Extremely slow for Modia3D

[MartinOtter]

The fix in #784 resulted in extremely poor performance for Modia3D.

using Modia3D
include("$(Modia3D.path)/test/Basic/PendulumWithBar1.jl")

gives the following timings for DiffEqBase versions (at the second run):

• version 6.92.2: 9.8s

• version 6.90.0: 11ms

so nearly a factor of 1000 slower, just by changing to the newest DiffEqBase version!!!!

I wonder, why in "derivative(deru, u, p, t)", the model "p" needs to be inspected by DifferentialEquations. Its anyway a complex data structure in Modia3D and it is very unlikely that DifferentialEquations can do anything with it, so it seems a complete waste of resources to inspect it. Maybe provide a way that this automatic inspection can be switched off.

[YingboMa]

Yes, you can bypass it easily by

julia> @time solve(prob, Tsit5());
  5.452284 seconds (7.93 M allocations: 789.155 MiB, 3.49% gc time)

julia> DiffEqBase.anyeltypedual(::SimulationModel) = Any

julia> @time solve(prob, Tsit5());
  0.051354 seconds (75.39 k allocations: 4.857 MiB, 98.42% compilation time)

julia> @time solve(prob, Tsit5());
  0.000925 seconds (898 allocations: 57.047 KiB)

[MartinOtter]

Yes, you can bypass it easily by [..]

Thanks, yes this solved the issue in Modia/Modia3D.

I wonder why you introduce rules for analytic differentiation that are activated by default? Analytic differentiation in Julia has severe limitations and therefore usage should require explicit activation.

[ChrisRackauckas]

No, making code correct should not require explicit activation. I don't think anyone would respect a solver that says "we don't do correct things by default, please turn on correct mode for that, thank you".

There are a number of factors here:

1. This is simply required for code correctness. If automatic differentiation is enabled, the state needs to be promoted to the differentiation space. This has always been the case. The promotion always existed. Otherwise you get an error.
2. The promotion code is "free" and compiled away in any case where the user has well-inferred code.
3. In almost any case, a user should have well-inferred code for performance.

When you put these together, it is both required and free for almost any user. Why would something that is both required to be done and free not going to be the default?

Because it's required for correctness, it solves a large issue that hundreds of people have had over the years. If you just search around for typeof(p).(u0) you'll see this all over, like https://discourse.julialang.org/t/odes-inside-a-turing-model/39759. "New to Julia". This is something that new people hit, and ensuring that someone that has no idea what they are doing gets fast and correct code is essential.

Now I know that this system's implementation does have a downside. If someone has dynamic behavior inside of p, such as Any pieces, then it needs to manually inspect it. And if that Any contains a circular reference, then it will need to recurse the stack until the maximum number of times (which was arbitrarily set to 1000 IIRC), before it throws an exit behavior. This means that the downside here is that if you happen to give uninferred types which are also many layers deep (or infinitely deep), then the check is slow. Now in this case, almost certainly the lack of inference in p will lead to lack of inference in f, making the ODE solve much slower than the inspection anyways, and this is seen in actual practice.

These restrictions on what can actually cause the bad behavior is the reason why many other major projects, like Trixi, CLIMA, etc. all end up fine with these checks. So it's not a problem of complexity in any sense, all of the other very large and complex projects with complex p definitions are fine, it's really that there is only one case that ends up being an issue, which is only seen in a single downstream project (that I've witnessed or is registered)

The one case where this can be a real issue is if dynamical behavior is used in p in a very specific or clever way that then has the right type asserts etc. to not actually slow down f, using the right function barriers etc., and p is "deep" (remember, since shallow p will still be analyzed quickly), then we will hit this issue. Now there are two facts to note about this case. One is that there's a few thousand people who have reported the other case (nicely inferred p with issues trying to figure out how to promote), and only a single recorded case of this other behavior. Secondly, by definition being in this case necessitates that someone actually knows what they are doing: you must already know Julia well if you're treating dynamic behavior with all of the right function barriers etc. and building infinitely deep p structures. Thus, I think anyone would agree that this one case should be the one to opt-out of the behavior, instead of requiring 10,000 first time users to read the docs to figure out how to opt-in.

You can try and make your case here, but I just don't see why the "burden of documentation" would be placed on a new user instead of an experienced one, especially when the only burden to an experienced user would show up in a narrow edge case that has an easy opt-out and is easily identified by a profile. To finish this then, a quick remark in the FAQ and a PR to your repo (https://github.com/ModiaSim/Modia.jl/pull/146) makes this good.