Seth Axen

Sounds right. This line calls `frule` repeatedly on the same primals, so it assumes the function is nonmutating: https://github.com/ThummeTo/ForwardDiffChainRules.jl/blob/d70301a28f61250c3168446c4b147b195ceee117/src/ForwardDiffChainRules.jl#L88

Thanks for the issue. Indeed, Higham in Section 3 of https://eprints.maths.manchester.ac.uk/id/eprint/2754 lists as the recommended algorithm for the Fréchet derivative for an arbitrary matrix function either an approximate one (complex...

Indeed, I've not inspected this one closely, but the approach of this PR looks right. I'll review in-depth shortly.

> I've also changed the function signature to take `V̄t` instead of `V̄`. This avoids an unnecessary transpose, but mainly it makes the code look much better, as there is...

> `svd_rev(USV, Ū::AbstractZero, s̄, V̄::AbstractZero)` with all code related to `Ū`, `V̄` and `F` removed (making it identical to the code in Neurogenesis.jl). This would apply in the case where...

> It would perhaps be best if @neurogenesisauthors made that PR to preserve authorship. That's not necessary. Specializing for the `AbstractZero` case is common in our codebase. e.g. here's where...

@ValentinKaisermayer I think this type-noninferrability may have been introduced in #406 . Is this something that can be fixed?

git bisect starting with c67d6301b128d567ea9233481f0d86b8342869fe as a good commit terminates with #406, but the above code fails then with a different error: ```julia julia> @inferred solve(prob, optimizer) ERROR: MethodError: Cannot...

One should test if simply making Symbolics compatible with Quaternions resolves this issue. If so, a cleaner way would be to add Symbolics support with package extensions to Quaternions.

This still needs tests for `to_vec`. It seems that on main the reverse-mode mutated callable test in EnzymeTestUtils has been erroring since #1282. The error changed in #1292.