hurl
hurl copied to clipboard
Strange behavior with exit code when using --fail-at-end option
Summary
When using --fail-at-end
, Hurl's exit code is always the one with the highest priority from the established failure exit codes list.
Steps to reproduce
- create
test-fail-at-end-exit-codes.hurl
file with queries and assert, returning 3 distinct exit codes:
GET http://127.0.0.1:1234/does-not-exist # exit code 3
GET https://hurl.dev
HTTP 200 # exit code 0
GET https://hurl.dev
HTTP 302 # exit code 4
- execute
test-fail-at-end-exit-codes.hurl
file
$ hurl --connect-timeout 1 --fail-at-end test-fail-at-end-exit-codes.hurl
error: HTTP connection
--> test-fail-at-end-exit-codes.hurl:1:5
|
1 | GET http://127.0.0.1:1234/does-not-exist # exit code 3
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ (7) Failed to connect to 127.0.0.1 port 1234 after 0 ms: Connection refused
|
error: Assert status code
--> test-fail-at-end-exit-codes.hurl:7:6
|
7 | HTTP 302 # exit code 4
| ^^^ actual value is <200>
|
$ echo $?
3
Even if we invert the line that triggers exit code 3 with the line that triggers exit code 4, we get the same result, which is the exit code 3.
What is the expected correct behavior?
I don't want Hurl to exit with the last error code or with a pre-prioritized exit code, I expect to have the exact exit code of all failures, here we have 2 and 4 then:
$ echo $?
6
I expect Hurl to manage more exit codes, for example:
Value | Description |
---|---|
0 |
success |
1 |
Failed to parse command-line options. |
2 |
Input File Parsing Error. |
3 |
Runtime error (such as failure to connect to host). |
4 |
Assert Error. |
5 (2+3) |
Input File Parsing Error + Runtime error (such as failure to connect to host). |
6 (2+4) |
Input File Parsing Error + Assert Error. |
7 (3+4) |
Runtime error (such as failure to connect to host) + Assert Error. |
9 (2+3+4) |
Input File Parsing Error + Runtime error (such as failure to connect to host) + Assert Error. |
Execution context
- Hurl Version: 2.0.1
- Operating system and version: Ubuntu 20.04