FactCheck.jl

FactCheck.jl copied to clipboard

When testing code that prints to stdout, the output gets interleaved with the test status output.

In at least one of the python test runners the runner swallows stdout and only displays it on a test failure. If the tests are passing you can assume the user doesn't really care what got printed.

Not sure how exactly to implement this, but I wanted to feel out whether it would be considered a desired feature.

I'm not a fan of this as a default (I believe it leads to confusing behavior if you're not expecting it), but I'd welcome it as some kind of configuration option. If you want to take a crack at it and #19, feel free to include the changes together.

This could possibly be an option, but I need to think about it. Certainly wouldn't be default option. When a failure occurs, does it print all STDOUT from the context, or just the statement?

The way it works in some other test frameworks I've used (I think nose.py does this) is that for each test case any generated STDOUT is suppressed, but if a given test fails than all the STDOUT from that test is printed, so you have more context. It makes test results a lot more concise, especially for passing tests where you generally don't care to see extra prints.

This is my next todo, and will involve redirect_stdout()

e.g.

oldstdout = STDOUT
rd, wr = redirect_stdout()
println("you won't see me")
readavailable(rd)
redirect_stdout(oldstdout)
println("you will see me")

Any update on this?

I'm not working on this (or this package), in favor of Base.Test (and its packaged version BaseTestNext)

fair enough, thanks. on that basis I will move all future tests I write over to Base.Test/BaseTestNext.