Zygote.jl

Zygote.jl copied to clipboard

Gradient wrt to a sparse matrix is mathematically wrong

6

Currently `Zygote.gradient` projects the co-tangent returned from `pullback` to have the same sparsity structure as the input. This is mathematically incorrect when the matrix input is sparse. According to Zygote,...

mohamed82008

This PR fixes #1507. This is technically a bug fix but I bumped the minor version because it is a significant enough change in behaviour. If a patch release is...

mohamed82008

Zygote in Julia 1.10+ not reading rrules for default constructors

7

`Zygote` fails to use `rrule`s defined by other packages when run with `Julia 1.10+` I noticed this when looking at compatibility with [`TaylorDiff.jl`](https://github.com/JuliaDiff/TaylorDiff.jl). In `Julia 1.9.4`: ```julia import Zygote import...

mrazomej

MWE: ```julia julia> using LinearAlgebra, Zygote, CUDA julia> Zygote.gradient(x -> tr(x), CUDA.zeros(2,2))[1] # works 2×2 Diagonal{Float32, FillArrays.Fill{Float32, 1, Tuple{Base.OneTo{Int64}}}}: 1.0 ⋅ ⋅ 1.0 julia> Zygote.gradient(x -> sum(abs2, x) - tr(x),...

jondeuce

Decide what to do about `pushforward` and co.

7

We get at least a couple of confused questions about it every month, but AFAICT there isn't much appetite to bring it up to a point of general usefulness. How...

ToucheSir