Michael Abbott

It's a bit of a moving target, as we try to figure out the best practice. So I guess that pointing out which bits of ChainRules's source to look at...

This was part of https://github.com/FluxML/Zygote.jl/issues/1262#issuecomment-1193400476, tl;dr is that everything 2nd order in Zygote is broken. Besides problems with the Z-to-CR bridge, the creation of thunks doesn't seem to be differentiable,...

Ok, trying to route the generic `ProjectTo` via its `Tangent` method might be a terrible idea, and can be changed. > Tangent is for structs. Using it on a wrapper...

I think you're asking whether there's a scheme for chunked reverse mode. There is not: at present (co)tangents match the size of the primal. https://github.com/JuliaDiff/ChainRulesCore.jl/issues/92 has some discussion, see also...

For now the current status should be clearly documented, perhaps at these pages: https://juliadiff.org/ChainRulesCore.jl/dev/rule_author/tangents.html https://juliadiff.org/ChainRulesCore.jl/dev/maths/propagators.html

It's pretty easy to make Zygote give you errors from natural + structural, for instance: ```julia julia> gradient(x -> (real(x) + x.im), 1+im) ERROR: MethodError: no method matching +(::NamedTuple{(:re, :im),...

>>In these two cases, > > not completely sure that I agree re. Fills and naturals for the reasons laid out above (the natural for a Fill Then you're looking...

Again, that paragraph you are quoting from is specifically about SArrays, not generalities. I'm not sure I have ever seen anyone access the field by name; I had no idea...

>> examples of how these get used > > Are these examples missing something Yes, thanks for the types, but what I meant was whether there are closer to end-to-end...

> we insist that natural never escape pullbacks I'm not sure what this means exactly. By "natural tangent" we mean something returned by a pullback which `isa AbstractArray`, no? (Or...