XML report not reporting the actual error

[rishinair11]

So I executed a kuttl test, of which this result is logged in the console:

=== RUN   kuttl/harness
=== RUN   kuttl/harness/tests
=== PAUSE kuttl/harness/tests
=== CONT  kuttl/harness/tests
    logger.go:42: 01:01:12 | tests | Creating namespace: kuttl-test-valid-weasel
    logger.go:42: 01:01:12 | tests/0-create | starting test step 0-create
    logger.go:42: 01:01:12 | tests/0-create | running command: [sh -c kubectl -n cp4m patch installation management-installation --type json -p "$(cat ../install-patch.json)"]
    logger.go:42: 01:01:14 | tests/0-create | error: the server doesn't have a resource type "installation"
I0120 01:01:16.087857   26187 request.go:645] Throttling request took 1.049658527s, request: GET:https://api.cp4bvt-45-xlcmw.purple-ontario.com:6443/apis/authorization.openshift.io/v1?timeout=32s
    case.go:230: failed in step 0-create
    case.go:232: exit status 1
    logger.go:42: 01:01:18 | tests | tests events from ns kuttl-test-valid-weasel:
    logger.go:42: 01:01:18 | tests | Deleting namespace: kuttl-test-valid-weasel
=== CONT  kuttl
    harness.go:390: run tests finished
    harness.go:496: cleaning up
    harness.go:551: removing temp folder: ""
--- FAIL: kuttl (17.88s)
    --- FAIL: kuttl/harness (0.00s)
        --- FAIL: kuttl/harness/tests (6.45s)
FAIL

As you can see, the issue is correctly reported as: error: the server doesn't have a resource type "installation" But when I run this test with --report xml the same error is not reported in the test report, instead the report looks like:

<testsuites name="" tests="1" failures="1" time="16.763">
   <testsuite tests="1" failures="1" timestamp="2021-01-19T23:56:25.705087713-08:00" time="5.373" name="installer/overlays/pakmodules/infrastructuremanagement/im-install">
     <testcase classname="im-install" name="tests" timestamp="2021-01-19T23:56:25.706890462-08:00" time="5.372" assertions="1">
       <failure message="failed in step 0-create" type="">exit status 1</failure>
     </testcase>
   </testsuite>
 </testsuites>

It's only reporting this exit status 1 error message instead of the actual error which is: error: the server doesn't have a resource type "installation" Can someone please help me how to get the actual error into the XML report?

[kensipe]

very important issue.. thank you!

[rishinair11]

@kensipe Thank you for adding this issue into the next milestone. Excited to see the enhanced output.

[rishinair11]

I can work on this issue if it already hasn't been fixed. @kensipe can you assign it to me?

[rishinair11]

Seems like this issue is already fixed. Couldn't reproduce it again.

 <testsuites name="" tests="1" failures="1" time="40.604">
   <testsuite tests="1" failures="1" timestamp="2023-03-11T13:28:23.762215+05:30" time="21.183" name="sample">
     <testcase classname="sample" name="deployment" timestamp="2023-03-11T13:28:23.763132+05:30" time="21.182" assertions="1">
       <failure message="failed in step 0-" type="">no matches for kind &#34;RishiKind&#34; in version &#34;lol&#34;</failure>
     </testcase>
   </testsuite>
 </testsuites>

Closing this issue.