MariusDrulea
MariusDrulea
> Some really good news. If I use 2 separate params the hessian works. Next is to adjust this to work for arrays. > f(x1, x2) = x1*x2 > ∇f(x1,...
Getting closer. I have manually created the corresponding graph for the gradient and the hessian. The rrules used are correct. There is something wrong with the tracker algo in the...
I have created a function to print the graph under a specific node: https://github.com/MariusDrulea/Tracker.jl/blob/master/src/Tracker.jl#L140 One can notice the Tracker does not record the methods performed, but the pullback of these...
I do suspect some broadcasting rule does not work correctly. If you look in the graph above there is a long chain of broadcasts, which looks weird. ![Screenshot from 2023-09-20...
This is the graph of the example which works correctly. Looks clean. ``` using Tracker x1 = param(1) x2 = param(2) f_(x1, x2) = x1*x2 ∇f_(x1, x2) = gradient(f_, x1,...
> Thanks for digging! As you can see this package doesn't get a lot of attention, but fixes are very welcome. I have reviewed the back-propagation algo for both simple...
@ToucheSir you are right, we have to store the pullback, not the original function. This is also clear now to me: https://discourse.julialang.org/t/second-order-derivatives-with-chainrules/103606 Integrating ChainRules is pretty easy: https://github.com/MariusDrulea/Tracker.jl/blob/master/src/Tracker.jl#L91 Second derivative...
> Integrating ChainRules is pretty easy: https://github.com/MariusDrulea/Tracker.jl/blob/master/src/Tracker.jl#L91 Not that easy, son. It is easy only for the first order derivatives. I do have to further track operations performed by these...
I'm currently stuck as I don't know how to deal with the ChainRules.rrule(s) here. The logic in Tracker is to define an untracked forward pass and a tracked pullback and...
The tracking of `sin(x)` is done by the Tracker.jl logic, not by rrule. For `sin` this is what happens: `sin(xs::TrackedReal) = track(sin, xs::TrackedReal)` The definition of `track` for `sin` performs...