sebastianbergmann
TestListener object argument loader regression
| Q | A |
|---|---|
| PHPUnit version | 9.5.8 |
| PHP version | 7.4.22 |
| Installation Method | PHAR distrib from official site https://phar.phpunit.de/ |
Summary
I've detected a regression on PHPUnit 9.5 when we use an object/class as argument of a testlistener
Current behavior
Class 'MyLoggerObjArg' not found
How to reproduce
Here are code I used to reproduce issue
source code
examples/boostrap.php
<?php declare(strict_types=1); require dirname(__DIR__) . '/vendor/autoload.php'; class MyLoggerObjArg { } class MyLogger implements \PHPUnit\Framework\TestListener { private $target; public function __construct($class = null) { $this->target = __DIR__ . '/testlistener-phpunit-' . \PHPUnit\Runner\Version::series() . '-' . date('YmdHis') . '.log'; $data = \PHPUnit\Runner\Version::getVersionString() . PHP_EOL; error_log($data, 3, $this->target); if (is_object($class)) { error_log('class exists(My Logger Object Arg) ? ' . (class_exists('MyLoggerObjArg') ? 'yes': 'no') . PHP_EOL,3, $this->target); } } public function addError(\PHPUnit\Framework\Test $test, Throwable $t, float $time): void { $data = __FUNCTION__ . ', ' . $test->getName() . PHP_EOL; error_log($data, 3, $this->target); } public function addWarning(\PHPUnit\Framework\Test $test, \PHPUnit\Framework\Warning $e, float $time): void { $data = __FUNCTION__ . ', ' . $test->getName() . PHP_EOL; error_log($data, 3, $this->target); } public function addFailure(\PHPUnit\Framework\Test $test, \PHPUnit\Framework\AssertionFailedError $e, float $time): void { $data = __FUNCTION__ . ', ' . $test->getName() . PHP_EOL; error_log($data, 3, $this->target); } public function addIncompleteTest(\PHPUnit\Framework\Test $test, Throwable $t, float $time): void { $data = __FUNCTION__ . ', ' . $test->getName() . PHP_EOL; error_log($data, 3, $this->target); } public function addRiskyTest(\PHPUnit\Framework\Test $test, Throwable $t, float $time): void { $data = __FUNCTION__ . ', ' . $test->getName() . PHP_EOL; error_log($data, 3, $this->target); } public function addSkippedTest(\PHPUnit\Framework\Test $test, Throwable $t, float $time): void { $data = __FUNCTION__ . ', ' . $test->getName() . PHP_EOL; error_log($data, 3, $this->target); } public function startTestSuite(\PHPUnit\Framework\TestSuite $suite): void { $data = __FUNCTION__ . ', ' . $suite->getName() . PHP_EOL; error_log($data, 3, $this->target); } public function endTestSuite(\PHPUnit\Framework\TestSuite $suite): void { $data = __FUNCTION__ . ', ' . $suite->getName() . PHP_EOL; error_log($data, 3, $this->target); } public function startTest(\PHPUnit\Framework\Test $test): void { $data = __FUNCTION__ . ', ' . $test->getName() . PHP_EOL; error_log($data, 3, $this->target); } public function endTest(\PHPUnit\Framework\Test $test, float $time): void { $data = __FUNCTION__ . ', ' . $test->getName() . PHP_EOL; error_log($data, 3, $this->target); } }examples/testSuite.php
<?php declare(strict_types=1); namespace Your\Name_Space; use PHPUnit\Framework\TestCase; class YourTestSuite extends TestCase { public function testIncomplete() { // Optional: Test anything here, if you want. $this->assertTrue(TRUE, 'This should already work.'); // Stop here and mark this test as incomplete. $this->markTestIncomplete( 'This test has not been implemented yet.' ); } public function testRisky() { } public function testSkipped() { $this->markTestSkipped('This test was skipped for any reason.'); } public function testFailure() { $this->assertEmpty(array('foo')); } public function testPass() { $this->assertTrue(TRUE, 'This should always work.'); } /** * @dataProvider additionProvider */ public function testDataProvider($a, $b, $expected) { $this->assertEquals($expected, $a + $b); } public function additionProvider() { return [ [0, 0, 0], [0, 1, 1], [1, 0, 1], [1, 1, 3] ]; } }examples/listener_no_arg.xml
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.5/phpunit.xsd" backupGlobals="false" backupStaticAttributes="false" bootstrap="bootstrap.php" colors="true" stopOnError="false" stopOnFailure="false" stopOnIncomplete="false" stopOnRisky="false" stopOnSkipped="false" beStrictAboutTestsThatDoNotTestAnything="true" verbose="false"> <listeners> <listener class="MyLogger"> </listener> </listeners> <testsuites> <testsuite name="Demo Test Suite"> <file>testSuite.php</file> </testsuite> </testsuites> </phpunit>examples/listener_obj_arg.xml
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.5/phpunit.xsd" backupGlobals="false" backupStaticAttributes="false" bootstrap="bootstrap.php" colors="true" stopOnError="false" stopOnFailure="false" stopOnIncomplete="false" stopOnRisky="false" stopOnSkipped="false" beStrictAboutTestsThatDoNotTestAnything="true" verbose="false"> <listeners> <listener class="MyLogger"> <arguments> <object class="MyLoggerObjArg" /> </arguments> </listener> </listeners> <testsuites> <testsuite name="Demo Test Suite"> <file>testSuite.php</file> </testsuite> </testsuites> </phpunit>
WARNING I've noticed also that a startTestSuite end endTestSuite with no entry are generated only with PHPUnit 9.5,
while it was not on PHPUnit 8.5 and even PHPUnit 7.5
See
output
PHPUnit 9.5.8 by Sebastian Bergmann and contributors. startTestSuite, startTestSuite, Demo Test Suite startTestSuite, Your\Name_Space\YourTestSuite startTest, testIncomplete addIncompleteTest, testIncomplete endTest, testIncomplete startTest, testRisky addRiskyTest, testRisky endTest, testRisky startTest, testSkipped addSkippedTest, testSkipped endTest, testSkipped startTest, testFailure addFailure, testFailure endTest, testFailure startTest, testPass endTest, testPass startTestSuite, Your\Name_Space\YourTestSuite::testDataProvider startTest, testDataProvider with data set #0 endTest, testDataProvider with data set #0 startTest, testDataProvider with data set #1 endTest, testDataProvider with data set #1 startTest, testDataProvider with data set #2 endTest, testDataProvider with data set #2 startTest, testDataProvider with data set #3 addFailure, testDataProvider with data set #3 endTest, testDataProvider with data set #3 endTestSuite, Your\Name_Space\YourTestSuite::testDataProvider endTestSuite, Your\Name_Space\YourTestSuite endTestSuite, Demo Test Suite endTestSuite,
Expected behavior
With following commands: phpunit -c examples/listener_no_arg.xml or even phpunit -c examples/listener_obj_arg.xml
output
PHPUnit 9.5.8 by Sebastian Bergmann and contributors. IRSF....F 9 / 9 (100%) Time: 00:00.004, Memory: 18.00 MB There were 2 failures: 1) Your\Name_Space\YourTestSuite::testFailure Failed asserting that an array is empty. /shared/backups/bartlett/phpunit-bootstrap/examples/testSuite.php:31 2) Your\Name_Space\YourTestSuite::testDataProvider with data set #3 (1, 1, 3) Failed asserting that 2 matches expected 3. /shared/backups/bartlett/phpunit-bootstrap/examples/testSuite.php:44 -- There was 1 risky test: 1) Your\Name_Space\YourTestSuite::testRisky This test did not perform any assertions /shared/backups/bartlett/phpunit-bootstrap/examples/testSuite.php:20 FAILURES! Tests: 9, Assertions: 7, Failures: 2, Skipped: 1, Incomplete: 1, Risky: 1.
Although you provide a lot of information, I am sorry to say that I do not understand what you are reporting.
Well, in summary, when I run the command phpunit -c examples/listener_obj_arg.xml I expect to have the Expected Behavior output provided.
It's OK on PHPUnit 7.5 and PHPUnit, 8.5 but it's impossible on PHPUnit 9.x
I got Class 'MyLoggerObjArg' not found
This is the argument used as parameter of the class listener
Hope my new explains are better !
PS: if you want to see a real implementation of concept, check on my project https://github.com/llaville/phpunit-LoggerTestListener (branch master was recently updated to allow PHPUnit 8.x and PHPUnit9)
The corresponding examples/listener_obj_arg.xml is https://github.com/llaville/phpunit-LoggerTestListener/blob/master/examples/phpunit.monolog.xml#L18
I have to admit that debugging and fixing issues related to test listeners is not very high on my list. Test listeners have been deprecated for years and will be removed in PHPUnit 10. I will probably not work on this issue, but would review/accept a pull request that fixes your problem.
I don't want to hurt you, but I'm a bit sad to hear that it won't be fixed.
At the base, the request to support PHPUnit 9.x come to a user of the community of my project https://github.com/llaville/phpunit-LoggerTestListener
When I tried to add support to PHPUnit 9, I've begun by a POC on Test Hooks, just before reading the essential post : https://github.com/sebastianbergmann/phpunit/issues/4676 Thanks for it to clarify the situation about Test Hooks and Listeners.
Finally, I won't add support to PHPUnit 9 and release a new version of my project that will at least support PHPUnit 7.5 and PHPUnit 8.5 that do the job efficiently.
After all, PHPUnit 8 is supposed to support as PHPUnit 9 both PHP 7.3 and PHP 7.4 (https://phpunit.de/supported-versions.html)
Thanks to spent time to reply, I'd appreciated a lot !
I briefly looked at it and the root cause appears to be that we're trying to create the object before the autoloader, which is also configured in the XML configuration file, has been loaded. I will try to find time to work on this.
Anything related to TestListener in PHPUnit 8.5 and PHPUnit 9.5 will not be changed anymore and has been removed in PHPUnit 10.