runner icon indicating copy to clipboard operation
runner copied to clipboard
verified publisher icon codewars

Improve feedback of failed assertions in Julia

Open hobovsky opened this issue 2 years ago • 3 comments

Similar to #235 and #222 : failed assertions in Julia print to <LOG::> instead of <FAILED::>:

image

If the direct reason for using <LOG::> is no longer relevant, Julia reporter could be improved just like LUA and Golang ones recently.

hobovsky avatar Feb 27 '23 15:02 hobovsky

The repo is https://github.com/codewars/FactCheck.jl

kazk avatar Feb 27 '23 20:02 kazk

Kumite with example test suite: https://www.codewars.com/kumite/63ffce08babb196de6d63ad5

Identified issues:

  • Failure messages are displayed as an unstructured line instead of <FAILED::>. Note: the direct cause seems to be not the same as #222 and #235 but rather a simple bug in the Julia reporter.
  • The way how @fact is translated into a tuple of <IT::> and <PASSED::> or <FAILED::> makes test output awkward sometimes
    • Custom assertion messages are displayed as <IT::> titles instead of <FAILED::> messages
    • Failed fact_throws gives awkward feedback
  • Stacktraces are displayed as an unstructured line instead of <LOG::> container
  • Non-standard assertion functions are stringified poorly
  • \t used in some messages causes that multiline formatted elements may not line up correctly
  • Durations are missing
  • Multiline group titles are missing <:LF:> and leak into a <LOG::>
  • Crashing tests do not show <ERROR::>
  • Not sure if @pending is displayed correctly

hobovsky avatar Mar 01 '23 22:03 hobovsky

Deployed https://github.com/codewars/FactCheck.jl/pull/2

kazk avatar Mar 23 '23 04:03 kazk