Andrei Zhabinski

Supporting `If` operation on `Tape` is relatively easy - we already have a more complicated `Loop` anyway. However, supporting the branches that have not been hit during the tracing is...

I'm thinking of adding type inference checks as a separate column (e.g. using JET.jl as [suggested](https://discourse.julialang.org/t/state-of-machine-learning-in-julia/74385/49)) - `test_rrule(..., check_inference=true)` fails the whole test as if the `rrule()` itself were broken...

I think the point of `test_rrule()` is to make all possible checks unless it's instructed otherwise. This is really useful when you develop a new `rrule`, but causes unnecessary noise...

`test_rrule` doesn't do any magic, we can write any checks ourselves. In fact, in some tests I already check primals and pullbacks, as well as correctness and speed separately. As...

Hi and sorry for the slow reply - I've spent a couple of days exactly figuring out what's wrong with the tests! As it turns out, most failures are due...

@CarloLucibello In your case, does the code work and show `true` in both 2 lines? I've just re-tested it and the problem still presents: ```julia (@v1.4) pkg> st NNlib Status...

Do you know if it can depend on optional dependencies like NNPACK or something?

Hm, for me it's also `false`. Unless there's a better idea, I'll try it on a fresh installation of Julia later today, maybe another OS or something.

Freshly installed Julia 1.4.1 on MacOS: * ok Re-created `.julia` directory from Julia on Linux Mint: * broken Freshly installed Julia on Ubuntu 20.04: * ok So it's definitely in...

If I activate NNPACK (by setting `ENV["NNLIB_USE_NNPACK"] = "true"`) and rebuild NNlib, results are consistent with numeric gradient: Without NNPACK: ``` julia> dw[1:10] # NNlib's gradient 10-element Array{Float64,1}: 77.24267625974863 75.60615480391128...