Tracker.jl
Tracker.jl copied to clipboard
FluxML
Flux's ex AD
Added recursion to track nodes while backpropagation in back! function. ### PR Checklist - [ ] Tests are added - [x] Documentation, if applicable
Notice that Tracker does not behave like Zygote here: ```julia julia> arr = [3.0, 4.0]; julia> Tracker.withgradient(nt -> sum(sum(x) for x in nt), (a=arr, b=arr, c=copy(arr))) (val = 21.0 (tracked),...
### Motivation and description Currently the `scan` method is used to mark the nodes before applying the actual backpropagation in the graph. https://github.com/FluxML/Tracker.jl/blob/master/src/back.jl#L19 We can achieve the same results without...
I am working with Julia v1.6.7 and trying to execute this code: W = rand(2, 5) b = rand(2) linear(x) = W * x .+ b grads = gradient(() ->...
In Julia v1.10, I used Aqua to detect method ambiguities in my package https://github.com/impICNF/ContinuousNormalizingFlows.jl/issues/356 and some of its report were related to this package: ```julia Ambiguity #5 ==(x::Tracker.TrackedReal, y::Real) @...
See the following code. Pseudocode + explanations: ``` f = x1 * x2 grad_f = (x2, x1) fg = sum(grad_f) = x2+x1 grad_fg = [1, 1] ``` Actual code: ```...
The `(i,j)`th element of the Hessian of `y=prod(x)` is `y/x[i]/x[j]` for `i ≠ j` and 0 for `i==j`, i.e. its diagonal is zero. However, ```julia julia> x = randn(3) 3-element...
There seems to be something weird going on when broadcasting over `Real`: ```julia julia> using Tracker, Distributions julia> m = first(param(zeros(1))) 0.0 (tracked) julia> s = first(param(ones(1))) 1.0 (tracked) julia>...
Moving y to "gpu" inside loss causes its gradient to be lost: ```julia julia> using Tracker, JLArrays julia> JLArrays.allowscalar(false) julia> Tracker.withgradient((x,y) -> sum(x[1:2] + jl(y))^2, jl([1,2,3.0]), [4,5.0]) (val = 144.0,...