unitizer icon indicating copy to clipboard operation
unitizer copied to clipboard

Changing Default Treatment of StdOut in Tests

Open brodieG opened this issue 9 years ago • 6 comments

Might want to think of something more meaningful than just not comparing because the print value might modify a little. Consider:

  • .new produces output but .ref produced none
  • .new and .ref have completely different outputs even though they produce the same values

The main challenge is wrapping of vectors and the [1] stuff that could change completely. Need to think about this more but running into instances where the current approach of just ignoring output is problematic.

brodieG avatar Dec 13 '14 01:12 brodieG

Also, comparing stdout could potentially slow down test comparison, so we would have to think about this carefully

brodieG avatar Apr 05 '15 01:04 brodieG

This also has implications for the size of the stored unitizer. Perhaps by default we should only store 1 character, or none if there is no output, and the default comparison function can just check if there was output and there no longer is, or vice versa.

brodieG avatar Jul 02 '15 01:07 brodieG

Turns out stdout and stderr capture is a very small portion of the storage space taken up in a unitizer. Probably still want to consider noting whether there is stdout/err or not.

brodieG avatar Aug 19 '15 01:08 brodieG

Urgh, inadvertent close. Related to #105.

brodieG avatar Aug 19 '15 01:08 brodieG

Really, right thing to do here is to store any stdout produced by the test itself separately from that of printing the result of the test.

brodieG avatar Aug 23 '15 17:08 brodieG

More importantly, we really need to store stderr since some code will issue errors via that without actually throwing conditions (for example, stack imbalance warnings).

brodieG avatar Dec 25 '15 20:12 brodieG