ChainRules.jl
ChainRules.jl copied to clipboard
JuliaDiff
forward and reverse mode automatic differentiation primitives for Julia Base + StdLibs
I've been revisiting https://github.com/FluxML/Zygote.jl/pull/1328 as part of a larger PR, and discovered this behaviour while running https://github.com/FluxML/Zygote.jl/blob/108e5a19d8fa7187f6eaece7a142c48d71dfd0d2/test/chainrules.jl#L275. MWE: ```julia-repl julia> _, back = rrule(getindex, [1], 1) (1, ChainRules.var"#getindex_pullback#1601"{Vector{Int64}, Tuple{Int64}, Tuple{NoTangent}}([1],...
See https://discourse.julialang.org/t/optimization-jl-datainterpolations-jl-and-gradients/97676/2?u=chrisrackauckas I just pulled out some old code: ```julia ZygoteRules.@adjoint ZygoteRules.literal_getproperty(A::Tridiagonal, ::Val{:dl}) = A.dl, y -> Tridiagonal(dl, zeros(length(d)), zeros(length(du)),) ZygoteRules.@adjoint ZygoteRules.literal_getproperty(A::Tridiagonal, ::Val{:d}) = A.d, y -> Tridiagonal(zeros(length(dl)), d, zeros(length(du)),)...
More of a question on whether this makes sense than a feature request. I could see it replacing or complementing `@thunk( of unbroadcast(...))` for rules such as https://github.com/JuliaDiff/ChainRules.jl/blob/v1.48.0/src/rulesets/Base/broadcast.jl#L174.
Closes https://github.com/JuliaDiff/ChainRules.jl/issues/698, needs https://github.com/JuliaDiff/ChainRulesCore.jl/pull/615
```julia julia> using Zygote, BenchmarkTools julia> @macroexpand1 @fastmath maximum(x) # this is new :(Base.FastMath.maximum_fast(x)) julia> VERSION v"1.10.0-DEV.421" julia> gradient(x -> sum(maximum(x; dims=1)), [1,3,2]) ([0.0, 1.0, 0.0],) julia> @fastmath gradient(x ->...
Mainly because the vcat literal syntax lowers to this: ```julia-repl julia> Meta.@lower Int[1; 2; 3] :($(Expr(:thunk, CodeInfo( @ none within `top-level scope` 1 ─ %1 = Base.typed_vcat(Int, 1, 2, 3)...
It looks like there is no rule for `LinRange` ``` julia> using ChainRules julia> isnothing(ChainRules.rrule(LinRange, 1, 2, 10)) true ```
WARNING: ChainRulesCore.DoesNotExist is deprecated, use NoTangent instead. likely near /home/ian/.julia/packages/ChainRules/Ipuva/test/rulesets/packages/SpecialFunctions.jl:38 WARNING: ChainRulesCore.DoesNotExist is deprecated, use NoTangent instead. likely near /home/ian/.julia/packages/ChainRules/Ipuva/test/rulesets/packages/SpecialFunctions.jl:38 WARNING: ChainRulesCore.Composite is deprecated, use Tangent instead. likely near /home/ian/.julia/packages/ChainRules/Ipuva/test/rulesets/packages/SpecialFunctions.jl:38...
Zygote currently [punts this to AD](https://github.com/FluxML/Zygote.jl/blob/d39ab5905b1d6e983f59f2f4566476845b002e9d/src/lib/array.jl#L315-L325), but that trick no longer works on nightly because `kron(::AbstractVector, ::AbstractVector)` stopped calling `kron(::AbstractMatrix, ::AbstractMatrix)`.