confluentinc
ksql-test-runner tool should return non-zero code on test failure
Currently ksql-test-runner tool returns 0 exit code even if tests fail. It brings some inconvenience running this tool e.g. in CI environment. One has to check the output of stdout and stderr and manually parse it in order to check if tests were successful, otherwise CI will always treat the test as successful, even if it's not.
To Reproduce
Just make the ksql-test-runner tool to fail and check the exit code.
input.json
{
"inputs": [
{"topic": "x", "timestamp": 0, "key": null, "value": {"a": 1}}
]
}
output.json
{
"outputs": [
{"topic": "y", "timestamp": 0, "key": null, "value": {"a": 1}}
]
}
statements.sql
CREATE STREAM x(a INT) WITH(KAFKA_TOPIC='x', VALUE_FORMAT='JSON');
And run the ksql-test-runner:
/opt/ksql# ksql-test-runner -s queries.sql -i input.json -o output.json
>>>>> Test failed: test did not generate any queries.
Expected: is not an empty collection
but: was <[]>
/opt/ksql# echo $?
0
Expected behavior The last command
/opt/ksql# echo $?
should output a non-zero exit code. It could either always be equal to 1 for any fail, or it may differ depending on failure reason (e.g. invalid input/output definition, invalid assertion, unexpected output, etc.)
