magento2-functional-testing-framework icon indicating copy to clipboard operation
magento2-functional-testing-framework copied to clipboard

Support xUnit-compatible Output Format

Open sprankhub opened this issue 4 years ago • 2 comments

Preconditions

  1. Magento 2.3.4-p2
  2. PHP 7.2
  3. MFTF 2.5.3

Steps to reproduce

  1. Run any MFTF test.

Expected result

  1. xUnit-compatible results are written somewhere.

Actual result

  1. Only Allure-compatible results are written to dev/tests/acceptance/tests/_output/allure-results.

Background

Bitbucket Pipelines only supports xUnit-compatible test reports.

Since Codeception comes with JUnit XML output, this could be "easy" to implement...?

sprankhub avatar Jul 17 '20 07:07 sprankhub

I think this could be easy enough to do. the RunTest command would need to send the flag --xml (optionally).

Here's a patch for a temporary workaround: https://gist.github.com/navarr/4cfe9fb85e65b19b0b6e0f2a2e58c465 (it always sends the --xml flag)

When doing this you'll want to define an output path in your codeception.yml's settings:

paths:
    output: tests/_xml

In some instances you may also want to use strict jUnit formatting (like with Jenkins). In those cases you'd need to turn the strict_xml setting on in the codeception.yml file:

settings:
    strict_xml: true

navarr avatar Jul 17 '20 12:07 navarr

@sprankhub @navarr Thank you for reporting and sending a patch for the issue! I sounds like a good enhancement to MFTF. I have created internal ticket https://jira.corp.magento.com/browse/MQE-2233 to track the issue.

jilu1 avatar Jul 27 '20 16:07 jilu1