When I use Hurl to run multiple files with --fail-at-end option, it stops at first bad formatted file

[lepapareil]

Problem to solve

When Hurl is run with the --fail-at-end option over one file, it stops immediately if that file is badly formatted, it's normal as lines on a hurl file can depend on the same functional context or can be broken due to a previous line format error.

But When Hurl is run with the --fail-at-end option over several files, it also stops as soon as one of them is badly formatted, but different files don't necessarily depend on the same functional context, and a bad formatted line on a file does not break other files lines, I think that Hurl must continue to iterate over remaining files outputting the bad format error for the evil file as it outputs the assertions errors.

Proposal

Do not distinguish between bad format errors (exit code 1) and other errors (exit code 2,3,4) when we run Hurl on several files with --fail-at-end. As files don't have to be linked functionally, I don't want the errors of one to mask the errors of others.

Additional context and resources

• create bad-formated.hurl file:

BAD FORMAT

• create fail-assert.hurl file:

GET http://hurl.dev
HTTP 404

• execute Hurl over the files with --fail-at-end option:

$ hurl --fail-at-end bad-formated.hurl fail-assert.hurl ; echo "exit_code=$?"

• actual result:

error: Parsing method
  --> bad-formated.hurl:1:1
   |
 1 | BAD FORMAT
   | ^ the HTTP method is not valid. Valid values are GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE, PATCH, LINK, UNLINK, PURGE, LOCK, UNLOCK, PROPFIND or VIEW
   |
exit_code=2

• expected result:

error: Parsing method
  --> bad-formated.hurl:1:1
   |
 1 | BAD FORMAT
   | ^ the HTTP method is not valid. Valid values are GET, HEAD, POST, PUT, DELETE, CONNECT, OPTIONS, TRACE, PATCH, LINK, UNLINK, PURGE, LOCK, UNLOCK, PROPFIND or VIEW
   |
error: Assert status code
  --> fail-assert.hurl:2:6
   |
 2 | HTTP 404
   |      ^^^ actual value is <301>
   |
exit_code=6

Tasks to complete

• [ ] what about json report may be adding a well-formatted attribute ?
• [ ] what about html report ?
• [ ] what about exit codes ? (#1450)