CumulusCI
CumulusCI copied to clipboard
"cci task run run_tests" generates incorrect test_results.xml format output
Describe the bug
cci task run run_tests generates an invalid XML format (no <xml> header) and also an invalid JUnit format (no <testssuites> attribute)
rest_results.xml file generated :
<testsuite tests="61">
<testcase classname="..._Test" name="..._test" time="2134" />
<testcase classname="..._Test" name="..._Test" time="522">
<failure type="failed" message="System.DmlException: Insert failed. ..."><![CDATA[..._Test: line 48, column 1]]></failure>
</testcase>
<testcase classname="..._Test" name="testIsVerified" time="10" />
</testsuite>
Expected result generated by sfdx force:apex:test:run --testlevel RunLocalTests -r junit -c :
<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
<testsuite errors="0" failures="0" hostname="https://xxx--yyy.sandbox.my.salesforce.com" name="force.apex" tests="3" time="35.85" timestamp="2023-01-09T10:19:27.000Z">
<properties>
<property name="commandTime" value="0.16 s"/>
<property name="coveredLines" value="733"/>
<property name="failRate" value="0%"/>
<property name="failing" value="0"/>
<property name="hostname" value="https://xxx-yyy.sandbox.my.salesforce.com"/>
<property name="orgId" value="..."/>
<property name="orgWideCoverage" value="78%"/>
<property name="outcome" value="Successful"/>
<property name="passRate" value="100%"/>
<property name="passing" value="3"/>
<property name="skipped" value="0"/>
<property name="testExecutionTime" value="35.85 s"/>
<property name="testRunCoverage" value="95%"/>
<property name="testRunId" value="..."/>
<property name="testStartTime" value="Mon Jan 09 2023 11:19:27"/>
<property name="testTotalTime" value="35.85 s"/>
<property name="testsRan" value="3"/>
<property name="userId" value="..."/>
<property name="username" value="[email protected]"/>
</properties>
<testcase classname="..._TEST" name="testMyController_annuler" time="3.48"/>
<testcase classname="..._TEST" name="testMyController_error" time="3.81"/>
<testcase classname="..._TEST" name="testMyController" time="28.56"/>
</testsuite>
</testsuites>
Reproduction steps
- run
cci task run run_tests - save reports/test_results.xml to reports/test_results_sav.xml
- run
sfdx force:apex:test:run --testlevel RunLocalTests -r junit -c -u <username>with the username of the CCI org used - compare results
Your CumulusCI and Python versions
cci version
CumulusCI version: 3.70.0 (/Users/rupert/.local/bin/cci)
Python version: 3.10.2 (/Users/rupert/.local/pipx/venvs/cumulusci/bin/python)
Operating System
MacOS 11.7.2
Windows environment
No response
CumulusCI installation method
pipx
Error Gist
No response
Additional information
No response