inaccurate derivatives of matrix logarithm

[lkapelevich]

The gradient of of the matrix valued function F(X) = log(X) or the Hessian of the scalar valued function f(X) = tr(X * log(X)) has an analytic form e.g. Equation (64) from https://arxiv.org/pdf/1906.00037.pdf.

I've hit a few cases where the derivatives I compute manually differ from ForwardDiff, one example is the bad_W in: https://gist.github.com/lkapelevich/8e4e4020cc56e13a0a42e73c4fa247d3

The code for the analytic form can become problematic if the values of some pair of eigenvalues in the input come close together. But this is not the case for the bad_W in the example, so I'm not sure what other sources of trouble to suspect.

[andreasnoack]

Might be related to the discussion in https://github.com/JuliaDiff/ForwardDiff.jl/issues/480