[enhancement request] Test log could print the failing assertion line?

[willhansen]

I have a function that returns true if everything as is it should be:

The failure message knows the file and line, but won't show the failing assertion:

So I have to manually add a message to each of the lines:

Since gut already knows the file and line, I'm thinking it wouldn't be super hard to output the failing assertion line in the log?

[bitwes]

You get more detailed messages on the other asserts. assert_eq will say "expected [expected] got [got]". I guess assert_true should say "expected true but got false" and vice versa for assert_false, but that doesn't tell you too much more information.

I would say that test_setup might be too generic of a name for a test. If the test name was something like test_slime_moves_towards_player_at_start then the simple "failed" would be enough information. Also, if you have multiple asserts in a test, adding the text is the best way to improve readability.

[willhansen]

assert_eq will say "expected [expected] got [got]"

Not a huge improvement.

I was thinking more along the lines of pytest:

[bitwes]

Ahhh. I like this idea, but this would be a bigger change than you might think.

GUT does not parse any of the source code for a test script. It uses meta data provided by Godot to get the test names and Godot's stacktrace functionality (get_stack()) for the line number. GUT could probably load up the source and get to a line number, strip the padding on the line and spit it out but there are edge cases that could be gross...like asserts that span multiple lines.

Then there is the where 4 = inc(3). This line has a ton of "magic" in it. It's not as clear in this example, but if you consider:

class TestClassDemoInstance:
    value = 0

    def test_two(self):
        assert self.value == 1

which generates

    def test_two(self):
>       assert self.value == 1
E       assert 0 == 1
E        +  where 0 = <test_class_demo.TestClassDemoInstance object at 0xdeadbeef0002>.value

You can start to see there is something other than text parsing going on. How does it know that self.value is <test_class_demo.TestClassDemoInstance object at 0xdeadbeef0002>.value at runtime? It turns out that pytest rewrites all the tests before it runs them. It is dynamically turning the == check into a message containing information about each side of the ==. Without this extra step, you don't get enough information back out to make the message useful.

I like how dynamic it is, and that you get a lot of information out without having to write much in your test, but it's a fundamentally different way of running tests and would require a lot of effort. It would probably be a lot of fun to do though.

[willhansen]

That does sound like more work than it's worth

There's probably a decent middle ground where it loads the file, gets the specified line, then calls some sort of existing gdscript parsing function to detect a multiline statement.

[bitwes]

That is a reasonable middle ground. I won't have any time to address this in the near future, most of my GUT time is being used for Godot 4. PR's are welcome though.

This would have to be an option that would be disabled by default (at least initially).