JUnit report is broken, if markTestSkipped is used within setUpBeforeClass

[thirsch]

Q	A
PHPUnit version	10.1.2
PHP version	8.2.5
Installation Method	Composer

Summary

Mandatory JUnit-Report attribute tests in the first testsuite-Node is missing, if setUpBeforeClass uses e.g. self::markTestSkipped(). This is working fine with PHPUnit 9. As you can see in the examples, PHPUnit 9 handled the case also differently in the specific testsuite of the file containing the method call.

We are skipping the test class using setUpBeforeClass instead of using the setUp method, because there is a dataprovider generating many test cases that should not even be generated if skipped.

I've just changed our tests and just wanted to let you know what I've discovered.

Current behavior

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="">
    <testsuite name="Application Test Suite" tests="0" assertions="0" errors="0" failures="0" skipped="0" time="0.000000">
      <testsuite name="SimpleTest" file="/src/tests/SimpleTest.php" tests="0" assertions="0" errors="0" failures="0" skipped="0" time="0.000000"/>
    </testsuite>
  </testsuite>
</testsuites>

How to reproduce

Use a simple test with the self::markTestSkipped() method call, e.g.:

<?php 

class SimpleTest extends TestCase
{
    public static function setUpBeforeClass(): void
    {
        self::markTestSkipped('Skip all');
    }

    public function test(): void
    {
        self::assertTrue(true);
    }
}

Expected behavior

<?xml version="1.0" encoding="UTF-8"?>
<testsuites>
  <testsuite name="" tests="1" assertions="0" errors="0" warnings="0" failures="0" skipped="1" time="0.000000">
    <testsuite name="Application Test Suite" tests="1" assertions="0" errors="0" warnings="0" failures="0" skipped="1" time="0.000000">
      <testsuite name="SimpleTest" file="/src/tests/SimpleTest.php" tests="1" assertions="0" errors="0" warnings="0" failures="0" skipped="1" time="0.000000">
        <testcase name="test" class="SimpleTest" classname="SimpleTest" file="/src/tests/SimpleTest.php" line="12" assertions="0" time="0.000000">
          <skipped/>
        </testcase>
      </testsuite>
    </testsuite>
  </testsuite>
</testsuites>