unitizer
unitizer copied to clipboard
Changing Default Treatment of StdOut in Tests
Might want to think of something more meaningful than just not comparing because the print value might modify a little. Consider:
-
.new
produces output but.ref
produced none -
.new
and.ref
have completely different outputs even though they produce the same values
The main challenge is wrapping of vectors and the [1]
stuff that could change completely. Need to think about this more but running into instances where the current approach of just ignoring output is problematic.
Also, comparing stdout could potentially slow down test comparison, so we would have to think about this carefully
This also has implications for the size of the stored unitizer. Perhaps by default we should only store 1 character, or none if there is no output, and the default comparison function can just check if there was output and there no longer is, or vice versa.
Turns out stdout and stderr capture is a very small portion of the storage space taken up in a unitizer. Probably still want to consider noting whether there is stdout/err or not.
Urgh, inadvertent close. Related to #105.
Really, right thing to do here is to store any stdout produced by the test itself separately from that of printing the result of the test.
More importantly, we really need to store stderr since some code will issue errors via that without actually throwing conditions (for example, stack imbalance warnings).