Michael Abbott

`Broadcasted` is a lazy object not unlike Thunk, we could add a `BroadcastThunk

> we will be able to do away with Thunks before ChainRules 2.0. But such analysis would not remove the above desire for BroadcastThunk -- where the goal is to...

I pushed the branches now. In this version, when `unbroadcast` has to perform a sum, it does so eagerly, but without materialising the Broadcasted. So `(x .- x')./2` will allocate...

The hard part was getting all the weird edge cases to work. If introducing extra branches like `if isinteger(p)`, you may need to expand the list of test cases to...

Zygote has this rule for `sum(f, xs::CuArray)`, which takes precedence over the one here: https://github.com/FluxML/Zygote.jl/blob/d4562e330d588cb986604bb4f1942bf9fca8ecc5/src/lib/broadcast.jl#L372-L377 Note also that `sum(x -> x^2, xs)` is equivalent to `sum(abs2, xs)` which has [a...

Worse it's a `Diagonal{T,Array}`. Would a `Diagonal{T, CuArray}` work?

Then probably it can re-use what `sum` does, which should also allow 2nd derivatives: https://github.com/JuliaDiff/ChainRules.jl/blob/a9a84ba6cb8aa9ce079af9401600e7c96a8aff3a/src/rulesets/Base/mapreduce.jl#L47 It would also be nice if the test noticed this. We have `@gpu test_rrule(tr, randn(4,...

See also #695

I forgot how this works, but as you observe, the code which is careful about zeros is behind a test `any(iszero, x)`, precisely to let all-nonzero `x` work without error...

This is what https://github.com/JuliaDiff/ChainRulesCore.jl/pull/446 was meant to solve. Right now it gives this error but haven't investigated further: ``` julia> # Solve result = solve(prob, ADAM(0.001, (0.9, 0.999)), maxiters=1000) ERROR:...