Zygote.jl
Zygote.jl copied to clipboard
FluxML
21st century AD
Here is the minimal reproducer I came up with ```julia import Flux import Zygote using Functors struct Test a b end @functor Test function (m::Test)(x) a = x for f=m.a...
I ran a course the other week and a student found this: ```julia using Flux using Statistics # Create NN NN = Chain(Dense(2,16,tanh), Dense(16,16,tanh), Dense(16,1), first) NN([0.5,0.5]) # Grid data...
Instead of doing the harder thing of making Duals work in general, why not target the higher level API? I played around a bit, didn't quite get it, but someone...
This PR ports @Keno's work on https://github.com/FluxML/Zygote.jl/pull/78 to 2022 Zygote. Because IRTools and base Julia have slightly different IR representations, some tweaks were necessary for the core algorithm: 1. Instead...
The current pmap adjoint collects all the pullbacks from the workers on the root process, which is very memory-intensive. Instead, we'd like to persist the pullbacks on the workers, and...
The conversion to and from ChainRules types should be essentially free for single structs, but for arrays of structs, it at present involves a copy in each direction. For example:...
Hi. I'm currently implementing self-supervised depth estimation network, similar to Monodepth2 and would like to point out a possible performance concern. Taking gradient for the first time takes very long...
This is a minimal attempt to add safe in-place accumulation of gradients. It assumes that any `Δ::DenseArray` may be mutated, and to keep this safe, any rule which duplicates `Δ`...
Fixes the MWE in https://github.com/FluxML/Zygote.jl/issues/1233. Removing the `@assert` does not appear to be a problem, but there may be unknown consequences that haven't been captured by the existing test suite....
This adds a bunch of definitions to make nested AD work and adds tests for second-order AD. Unfortunately by the time we get to third-order AD, the types get so...
