ChainRules.jl

ChainRules.jl copied to clipboard

`rand!` is marked non-differentiable

2

[This](https://github.com/JuliaStats/Distributions.jl/blob/c9d6c28f415025bf489ac3bec2f8eec46b0eefbd/src/genericrand.jl#L48) fallback method for `rand` in `Distributions.jl` hits [this](https://github.com/JuliaDiff/ChainRules.jl/blob/f13e0a45d10bb13f48d6208e9c9d5b4a52b96732/src/rulesets/Random/random.jl#L25) rule, which is declared non-differentiable. This results in a silent failure, where there ought to be an error if the given...

willtebbutt

Rules for det are not general and likely unstable

8

```julia A = [1.0 1.0; 1.0 1.0] det(A) * inv(A) # ERROR: SingularException(2) ``` to be fair, the LU approach is remarkably resiliant... ```julia A = [1.0 1.0; 1.0 1.0-1e-16]...

cortner

Avoid NaN (co)tangents for sqrt(0)

5

This PR fixes #576 by treating zero (co)tangents in `sqrt` as strong zeros. It partially fixes https://github.com/FluxML/Zygote.jl/issues/1101 also, but to fix it entirely, we would need to do the same...

sethaxen

Current rules for sqrt produce NaN for zero primal and (co)tangents

2

This only happens when the (co)tangent is 0. ```julia julia> using ChainRules julia> ChainRules.frule((ChainRules.ZeroTangent(), 0.0), sqrt, 0.0) (0.0, NaN) julia> ChainRules.rrule(sqrt, 0.0)[2](0.0) (ChainRulesCore.NoTangent(), NaN) ``` I suggest we adopt the...

sethaxen

Assume commutative multiplication exactly when necessary

4

As noted in #504, there are a number of cases where types of rules were constrained to `CommutativeMulNumber` where commutation of multiplication did not need to be assumed. Likewise, there...

sethaxen

use ProjectTo in Array addition

4

the `reshape` was a primative version of ProjectTo i think?

oxinabox

add rule for copy

5

Fixes https://github.com/FluxML/Zygote.jl/issues/1037

CarloLucibello

rrule for multi-argument product

1

From dfdx/Yota.jl#93: ```julia A = rand(100, 100) x = rand(100) rrule(*, x', A, x) # ==> nothing ``` It's possible to binarize the operation on the AD engine side, but...

dfdx

Rule for `svdvals`

3

Requested here: https://discourse.julialang.org/t/implementation-of-spectral-normalization-for-machine-learning/76074 The workaround is to call `svd(X).S`, which is slower forwards. But it looks like the gradient calculation with something like `svd_rev((; U=NoTangent(), s=s, V=NoTangnet(), Vt=NoTangent()), NoTangent(), S̄,...

mcabbott

Use oneunit where it makes sense

2

I suspect some of the scalar rules should be using `oneunit` instead of `one`. For example, `sign`.

sethaxen