ChainRulesCore.jl icon indicating copy to clipboard operation
ChainRulesCore.jl copied to clipboard

Comparison tests ZeroTangent with 0-like numbers should be true

Open matbesancon opened this issue 4 years ago • 7 comments

Was just testing something boiling down to: ZeroTangent() ≈ 0 Should this be accepted? Along with arrays for which the norm is 0?

matbesancon avatar Oct 28 '21 12:10 matbesancon

Yes, though we use test_approx in ChainRulesTestUtils

julia> using ChainRulesTestUtils

julia> test_approx(ZeroTangent(), 0)
Test Passed

julia> test_approx(ZeroTangent(), zeros(2, 3))
Test Passed

mzgubic avatar Oct 28 '21 12:10 mzgubic

indeed good point. Could it be worth it to have it somewhere close to the ZeroTangent doc?

matbesancon avatar Oct 28 '21 13:10 matbesancon

I would argue it belongs to the ChainRulesTestUtils docs, but we could certainly add a link to those.

mzgubic avatar Oct 28 '21 13:10 mzgubic

What's the argument against defining these?

Besides being one less thing to know about, @test x ≈ 0.0 will I think give more useful information when it fails, as the macro records where it was called.

Maybe array ones are weird, in that @test x ≈ [0 0 0] normally implies you check the size as well as the contents, whereas Zero doesn't have any size of course.

Finally, x ≈ 0 is the same as x == 0 for numbers, I think. Unless you supply the tolerance. But ZeroTangent() == 0 is false, not undefined.

mcabbott avatar Oct 28 '21 13:10 mcabbott

yes I wasn't sure about arrays, I found norm(x) == 0 a good proxy

matbesancon avatar Oct 28 '21 13:10 matbesancon

I think the reason we have test_approx is to better display the information when we have thunking involved.

compare:

julia> using ChainRulesCore

julia> using ChainRulesTestUtils

julia> using Test

julia> struct Foo end

julia> unfoo(x) = x
unfoo (generic function with 1 method)

julia> unfoo(f::Foo) = 0.0
unfoo (generic function with 2 methods)

julia> Base.isapprox(f::Foo, b) = isapprox(unfoo(f), b)

julia> @test Foo() ≈ 0.1
Test Failed at REPL[11]:1
  Expression: Foo() ≈ 0.1
   Evaluated: Foo() ≈ 0.1
ERROR: There was an error during testing

to

julia> using ChainRulesCore

julia> using ChainRulesTestUtils

julia> using Test

julia> struct Foo end

julia> unfoo(f::Foo) = 0.0
unfoo (generic function with 1 method)

julia> unfoo(x) = x
unfoo (generic function with 2 methods)

julia> ChainRulesTestUtils.test_approx(f::Foo, b) = test_approx(unfoo(f), b)

julia> test_approx(Foo(), 0.1)
Test Failed at /Users/mzgubic/JuliaEnvs/ChainRulesTestUtils.jl/src/check_result.jl:24
  Expression: isapprox(actual, expected; kwargs...)
   Evaluated: isapprox(0.0, 0.1)
ERROR: There was an error during testing

mzgubic avatar Oct 28 '21 14:10 mzgubic

We used to define . It was surprisingly awkward. And it gave surprisingly bad error messages.

I suspect it is also pretty invalidating, if we define it in CRC, but I haven't tested that.

oxinabox avatar Oct 28 '21 15:10 oxinabox