Expected vs. Actual output for failed tests unreadable due to %0A instead of line breaks

[daveavis]

When an Input/Output test fails I would like students (and I) to be able to compare the expected output to the actual output. However, the current error message is unreadable due to the many %0A (linefeed) characters where there should be newlines. I checked the autograding.json and found there were indeed \r\n between each line. I tried removing the \r part and only leaving \n, but that had no effect. What I would like to see is actual line breaks for multi-line output.

Sample Output for a test named "5":

📝 5


  Enter a number: SHS Spartans
  1
  SHS
  3
  SHS
  5
  
❌ 5
::error::The output for test 5 did not match%0AExpected:%0ASHS Spartans%0A1%0A2%0ASHS%0ASpartans%0A5%0AActual:%0AEnter a number: SHS Spartans%0A1%0ASHS%0A3%0ASHS%0A5

The repository I've been using to test this is at https://github.com/StratfordHS-CS2/lab-22-shs-spartans-daveavis This is not a student repo, I (the teacher) accepted the assignment and was using this repo for testing if I could somehow clean up the output.

[julianvu]

I think this is a larger issue of the autograding action's not being able to check multi-line outputs with regexes. My regex tests are failing for multi-line outputs as well.

[markpatterson27]

I think this is because autograding is treating a failed match as an error.

Error messages are escaped in core. \n is replaced with %0A. I assume stdout from processes is also escaped and error messages are escaped seperately because it's assummed the process that triggered the error has exited. When the logs are displayed, the escaped charaters are substituted back.

Except, autograding hasn't exited. It's still running through the rest of the tests. So the error message gets sent to the log, where it is escaped again. Since '%' is an escaped charater, when the log is displayed, the workflow interface can't substitute %0A back to \n because it thinks it's substituting %250A back to %0A.

TL:DR; failed match messages are being escaped twice. The solution is to not throw an error when a match fails (because failed matches are an expected outcome ;).

[jwkim90]

@daveavis Hello, I'm wondering if you resolved this issue because I'm having the same problem.

[GittyBitch]

Indeed, this is still a problem