ForwardDiff.jl
ForwardDiff.jl copied to clipboard
make Gradient/Jacobian/Hessian-Config mutable
these often get quite large and the fact that they are non-mutable means Julia copies them which is quite expensive
Consider for example:
using ForwardDiff: ForwardDiff, HessianConfig, Chunk
using DiffResults
function test()
x = rand(12)
result = DiffResults.HessianResult(x)
cfg = HessianConfig(sum, result, x, Chunk{12}())
# Warmup
ForwardDiff.hessian!(result, sum, x, cfg)
@time ForwardDiff.hessian!(result, sum, x, cfg)
@time for i in 1:100
ForwardDiff.hessian!(result, sum, x, cfg)
end
end
test()
Before this change it gives:
0.000003 seconds (2 allocations: 16.000 KiB)
0.000280 seconds (200 allocations: 1.562 MiB)
Now it instead gives
0.000002 seconds (2 allocations: 160 bytes)
0.000120 seconds (200 allocations: 15.625 KiB)
At some point, Julia might do this better.