Stacktraces are hidden

[mcabbott]

When tests fail, I think this package is somehow helpfully capturing the error (with try) and hiding it:

test_rrule: partialsort on Vector{Float64},UnitRange{Int64}: Error During Test at /home/runner/.julia/packages/ChainRulesTestUtils/YbVdW/src/testers.jl:193
[232](https://github.com/JuliaDiff/ChainRulesCore.jl/runs/8017765323?check_suite_focus=true#step:6:233)
  Got exception outside of a @test
[233](https://github.com/JuliaDiff/ChainRulesCore.jl/runs/8017765323?check_suite_focus=true#step:6:234)

Can this be removed?

The default behaviour is to print the stacktrace, which is useful information about what's actually caused the error, and where it was called from. (Not testers.jl:193.)

julia> @testset "outer" begin
         @testset "inner" begin
           @test 1+1==2
           sqrt(-1) # outside test
         end
       end
inner: Error During Test at REPL[5]:2
  Got exception outside of a @test
  DomainError with -1.0:
  sqrt will only return a complex result if called with a complex argument. Try sqrt(Complex(x)).
  Stacktrace:
    [1] throw_complex_domainerror(f::Symbol, x::Float64)
      @ Base.Math ./math.jl:33
    [2] sqrt
      @ ./math.jl:675 [inlined]
    [3] sqrt(x::Int64)
      @ Base.Math ./math.jl:1480
...
Test Summary: | Pass  Error  Total  Time
outer         |    1      1      2  0.2s
  inner       |    1      1      2  0.2s
ERROR: Some tests did not pass: 1 passed, 0 failed, 1 errored, 0 broken.

[oxinabox]

I think this is actually do do with the way the Test stdlib rewrites stack-traces. I have gone digging before but i don't recall what my conclusion was.

I think it cuts off stacktraces when it hits the file where the @test macro was used. Which is a problem if calling a function that defines a test suite.

Here is the code in the Test stdlib. https://github.com/JuliaLang/julia/blob/98e1b13a7db5aa1d05b9a48085993375cf2298d0/stdlib/Test/src/Test.jl#L53-L67 Maybe we can monkey patch it off?