Michael Abbott

Yes, if it works... probably someone just has to tidy it up & figure out how to get tests working, etc.

The most straightforward answer is that you can't, they are immutable. And your gist doesn't try, despite its name `changevalue!(x::Dual, ...` returns an altered dual, without changing its argument. And...

Thanks for the report. I can reproduce this, but have no idea what causes it. It works on the CPU, with `threads=false` (to use KA) and `verbose=true` (to know): ```...

Thanks for the example, this looks like the sort of thing they ought to be good at, although in the end... we will see. I wonder what OMEinsum is doing...

Thanks, I was just digging into macroexpand too: ``` julia> @ein ret[i, k] := W[a[k], i, j] * x[j, k]; julia> ret ≈ @tullio ret_2[i, k] := W[α, i, j]...

Trying to time the GPU things, I think you may need `CUDA.@sync` to get real times? (Or maybe the mean is reliable?) I get the following: ```julia julia> @btime contract_WA($W_gpu,...

Thanks for digging. Am a little surprised this gives scalar operations, when the forward pass did not, but maybe, as you suggest, the problem is that the loop over `k`...

Doing reasonably well on weird operations is the goal, since straightforward matmul (or gather) already have good implementations. But there is a long tail of odd things... It would not...

There's another example here: https://github.com/TuringLang/DistributionsAD.jl/pull/23#issuecomment-575671303. Behaviour on tagged ForwardDiff: ```julia using ForwardDiff, LinearAlgebra A = Matrix(I, 2,2) ForwardDiff.gradient(x -> sum(x \ [1,3]), A) # [-1 0; 0 -3] ForwardDiff.gradient(x ->...

This `f` would indeed change. However, it has problems on a region bigger than one point, and so it would be better anyway for the smoothed branch to be used...