test icon indicating copy to clipboard operation
test copied to clipboard

Tests that write to stdout can break JSON reporter JSON

Open DanTup opened this issue 3 years ago • 2 comments

Moved from https://github.com/Dart-Code/Dart-Code/issues/4084.

If a test is using stdout to write output (for example stdout.write('{');), it can generate invalid JSON for a tool using the JSON reporter that may prevent it from parsing the results of the test run correctly.

There are some suggestions with how to deal with this at https://github.com/Dart-Code/Dart-Code/issues/4084#issuecomment-1204426253 by using IOOverrides and wrapping them in JSON (or standard print messages).

@jakemac53

Always convert Strings into normal print messages, probably using the line splitter to only send messages when we get a newline

Waiting for newlines could result in some content not being shown (or, being out-of-order with other things happening). I guess this could be flushed when there are other events (such as non-string stdout, or end of test), although you'd need some way to indicate there was no newline on the end if using print (or just consider it by design that these things get newlines appended so they don't interfere with subsequent printing).

DanTup avatar Aug 04 '22 13:08 DanTup

@jakemac53 looks like we've been here before 😄

DanTup avatar Mar 25 '25 15:03 DanTup

Haha, weird this must have fallen through the cracks, cc @natebosch I do think we should look into this given it breaks the JSON support especially.

We could potentially hack something to only do it for the JSON reporter but 🤮

jakemac53 avatar Mar 25 '25 15:03 jakemac53