David Widmann

The term "tensor sum" is also used in this proposal: https://github.com/JuliaLang/julia/issues/13333 (even though arguably "direct sum" is more common - but not always synonymous: https://github.com/JuliaLang/julia/issues/13333#issuecomment-143825995)

IIRC it was discussed before in some other issue (I'll try to find it) that you cannot just use any metric. I think this is a well-known fact about kernels...

https://github.com/JuliaGaussianProcesses/KernelFunctions.jl/pull/159#issuecomment-696619214

From looking at the code, it seems we just forgot to add a fallback for `_sum`? https://github.com/JuliaGaussianProcesses/KernelFunctions.jl/blob/401d556df996558c417d9f4d5c715ab079cc4e14/src/kernels/kernelsum.jl#L46 We explicitly supports constructors and summation for KernelSum with AbstractVector, so it should...

Might be due to the hardcoded (constant) value for `x = y`. Hence possibly the problem doesn't exist with ForwardDiff 0.11.

> The if block should not be constant but rather a taylor polynomial so that autodiff in this branch works. Shouldn't be necessary in ForwardDiff 0.11: It skips measure zero...

The package and paper is mainly concerned with derivatives wrt the order, which does not seem to be the issue in the OP. > when is that going to be...

> As autodiff simply takes the derivative of the branch it finds itself in That's not true in general. ForwardDiff#master is supposed to ignore branches of measure zero.

No, ForwardDiff uses its own definitions (functions with `Dual` arguments) and DiffRules. The `sqrt` example reminds me of https://github.com/JuliaDiff/DiffRules.jl/pull/100 which reverted a ForwardDiff bug introduced by defining the derivative of...

I think a generalization of these indices is fine as long as the code complexity does not increase too much (maybe it becomes even simpler?) and performance + AD regressions...