Explicit derivatives for complex analytic functions

[clguillot]

trafficstars

Hi everyone,

As documented in #726, #514, #653 ForwardDiff encounters some difficulty when it comes to differentiating complex functions. The issue is similar to the one in #481 . This is a very serious issue which has been encountered by several people, including myself, and can be quite difficult to identify the first time, especially when using ForwardDiff as a "black box". Julia and ForwardDiff are widely used by mathematicians in the field of quantum chemistry, who manipulate complex number daily, and encountering this issue can result in a huge loss of time. Up to a few exceptions (mainly the functions abs and abs2), all the rules defined in DiffRules can be extended to the complex domain with no modification. I worked a bit on the issue and came up with a start of fix by modifying unary_dual_definition as follows: https://github.com/clguillot/ForwardDiff.jl/blob/114cfe90755df8591488c7d71bd3109be5325fb9/src/dual.jl#L234-L265 I simply define a version of the derivative for Complex{Dual} defined with the expression found in DiffRule and returning a new Complex{Dual}. With this fix, I get the correct result when computing the order 2 derivative of exp(ix):

julia> f(x) = exp(1im*x);
julia> df(x) = ForwardDiff.derivative(f, x);
julia> ForwardDiff.derivative(df, 0.0)
-1.0 + 0.0im

Without the fix, the same computation (unless the modification in #481 is implemented) returns 0.0 + 0.0im, which is obviously wrong. I also implemented sin and cos for Complex by hand https://github.com/clguillot/ForwardDiff.jl/blob/114cfe90755df8591488c7d71bd3109be5325fb9/src/dual.jl#L747-L771 but avoiding sincos since I was lazy.

I believe having explicit derivatives in those cases will mostly free ForwardDiff from having to worry about how the functions are implemented in the libraries, since it never needs to actually go through this code with a Dual type. Moreover, I don't think it to be too harmful for the performances either.

One issue that I see with this pull request is the manual exclusion of some function. It would probably be more elegant to modify DiffRules to indicate which function can see its derivative extended to the complex domain, for example by defining a macro @define_analytic_diffrule which would make a call to @define_diffrules and put the function into a some kind of list indicating that it is analytic. Until something like this can be pulled up, the code above should at least provide a basis that returns the right answer in most cases. It would also be nice to provide a similar fix for functions of several variables, but all in good time.