Michael Abbott

> artifact of figure size changing But by "figure size" I think you mean the total layout, which contains several sets of axes, each of which have their own legend....

Zygote could surely define `frule_via_ad(::Config, xdot, f, x::Real...)` using Dual numbers. Maybe a little wider but arbitrary structs would be tricky. How this would interact with 2nd derivatives is not...

> Do the pullbacks not close over strictly more data for a fused (then unfused) broadcast with more than a couple of functions? I think they will just close over...

> I don't think running twice needs to be a strict requirement. Running once is fine as well, the point is to have a way of specifying "Use forward mode...

> Because functions are not singleton types. I think that right now, Zygote's GPU broadcast unconditionally uses the stored-Dual path. It will silently ignore things which aren't real numbers, which...

Maybe I'm not sure what you mean by close over here? The `typeof(tanh)` specifies the function, and this is known to the compiler from `typeof(bcf)`. I guess `(tanh∘+)` is the...

The other thing I forgot, more closely relevant to the tile: `frule((dx, dy), f, x, y)` and `frule_via_ad` don't handle chunking. On each call of `f`, you can push forward...

For `reduce(hcat, ...`, xref #501. Does `CUDA.reduce` differ from `reduce` here? For `cumprod`, xref #282

Might be worth wrapping all tests in a `@testset`, so that it keeps going past the first failure?

Do these times need a `CUDA.@sync`? Seems a little crazy that copying to an Array could be quicker, but if it really is then that's simple. And if so, it...