[Squeak] Improve expressiveness of failing #assert:equals: etc.

[LinqLover]

Problem statement

Given a typical test failure like this:

I have to run the complete smalltalkCI build locally if I cannot reproduce the error in my personal image. In such situations, it would help me a lot if the text would not be truncated to this extent.

In my opinion, CI logs could use much more space for displaying helpful error messages. For reference, testing packages from other languages such as unittest (Python) even spend a couple of lines for printing a diff of lists etc. Compared to the stack trace of about 50 lines, more space should be absolutely okay IMO.

(For CI platforms that support nested collapsable groups, the long message could also be extra-collapsed.)

Implementation details

I don't know how dialect-specific this issue is, but for Squeak the error message is created directly in TestCase. In Squeak, we could subclass ~~AssertionFailure for an EqualityAssertionFailure~~TestFailure for an EqualityTestFailure (edit: inserted correct class names) and handle it differently in smalltalkCI. (A less beautiful solution would be to parse the stack trace to get actual and expected). What do you think? :-)

[fniephaus]

I agree that smalltalkCI could provide more details in error cases, but I am hesitant to introduce new subclasses for a use case like this. As you mentioned, TestCase is responsible for the error message being truncated. More specifically, TestCase>>#comparingStringBetween:and: is called, which truncates printStrings to 10 characters. I'm also not a big fan of adding yet another preference, but maybe we could find a way that allows smalltalkCI to raise or remove the character limit temporarily (of course, smalltalkCI could patch the method. But that could cause further confusion).

[LinqLover]

In my opinion, the display of assertion failures does not need to be a responsibility of TestCase. So what I am proposing in the following is meant as changes to be implemented in the Squeak Trunk: Let's deprecate TestCase >> #comparingStringBetween:and: completely and design a more meaningful TestFailure hierarchy instead which is also responsible for generating appropriate messages (#messageText). E.g. in Squeak, you could inspect an instance of EqualityTestFailure to compare the two objects using any tool you would like to. (See also [squeak-dev] SUnitTools-ct.5 (though it's a bit older) for more ideas about exploring test failures.) In smalltalkCI, we could just handle these failures differently instead of using the default #messageText implementation.

The changes to the Trunk could also be designed backward-compatible. We would have to delete ResumableTestFailure and remove it by a boolean in its superclass, but it did not work properly before, either. See [squeak-dev] ResumableTestFailure.

What do you think, would this be a feasible undertaking or are there too many Smalltalk dialects depending on the current form of SUnit? Should we move this discussion to squeak-dev?