sebastianbergmann
Allowing errors with convertWarningsToExceptions doesn't work with process isolation
| Q | A |
|---|---|
| PHPUnit version | PHPUnit 9.5.24 #StandWithUkraine |
| PHP version | PHP 8.1.10 (cli) (built: Sep 4 2022 08:41:25) (NTS) |
| Installation Method | Composer |
Summary
The XML attributes convertDeprecationsToExceptions, convertErrorsToExceptions, convertNoticesToExceptions, convertWarningsToExceptions don't work when processIsolation=true.
Current behavior
I have a test that triggers a notice (MyTest.php):
<?php
use PHPUnit\Framework\TestCase;
class MyTest extends TestCase {
public function testNotice() : void {
$this->assertFalse(new stdClass() > 5); // triggers a E_NOTICE
}
}
I have a phpunit.xml that ignores notices:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/9.3/phpunit.xsd"
convertNoticesToExceptions="false"> <!-- Ignore notices -->
<testsuites>
<testsuite name="Tests">
<directory>.</directory>
</testsuite>
</testsuites>
</phpunit>
Running phpunit normally works:
$ vendor/bin/phpunit
PHPUnit 9.5.24 #StandWithUkraine
PHP Notice: Object of class stdClass could not be converted to int in /Users/sjoerdlangkemper/dev/test/phpunit/MyTest.php on line 6
. 1 / 1 (100%)
Notice: Object of class stdClass could not be converted to int in /Users/sjoerdlangkemper/dev/test/phpunit/MyTest.php on line 6
Time: 00:00.006, Memory: 6.00 MB
OK (1 test, 1 assertion)
Enabling process isolation, either in the XML file or in the command line, lets the notice break the test:
vendor/bin/phpunit --process-isolation
PHPUnit 9.5.24 #StandWithUkraine
E 1 / 1 (100%)
Time: 00:00.063, Memory: 6.00 MB
There was 1 error:
1) MyTest::testNotice
Object of class stdClass could not be converted to int
/Users/sjoerdlangkemper/dev/test/phpunit/MyTest.php:6
ERRORS!
Tests: 1, Assertions: 0, Errors: 1.
How to reproduce
See above.
Expected behavior
I would expect the test to behave the same with or without --process-isolation, i.e. allow the notice. I expect the tests to succeed since I have configured convertNoticesToExceptions=false.
More info
$ composer info | sort
doctrine/instantiator 1.4.1 A small, lightweight utility to instantiate objects in PHP without invoking their constructors
myclabs/deep-copy 1.11.0 Create deep copies (clones) of your objects
nikic/php-parser v4.15.1 A PHP parser written in PHP
phar-io/manifest 2.0.3 Component for reading phar.io manifest information from a PHP Archive (PHAR)
phar-io/version 3.2.1 Library for handling version information and constraints
phpunit/php-code-coverage 9.2.17 Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator 3.0.6 FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-invoker 3.1.1 Invoke callables with a timeout
phpunit/php-text-template 2.0.4 Simple template engine.
phpunit/php-timer 5.0.3 Utility class for timing
phpunit/phpunit 9.5.24 The PHP Unit Testing framework.
sebastian/cli-parser 1.0.1 Library for parsing CLI options
sebastian/code-unit 1.0.8 Collection of value objects that represent the PHP code units
sebastian/code-unit-reverse-lookup 2.0.3 Looks up which function or method a line of code belongs to
sebastian/comparator 4.0.8 Provides the functionality to compare PHP values for equality
sebastian/complexity 2.0.2 Library for calculating the complexity of PHP code units
sebastian/diff 4.0.4 Diff implementation
sebastian/environment 5.1.4 Provides functionality to handle HHVM/PHP environments
sebastian/exporter 4.0.5 Provides the functionality to export PHP variables for visualization
sebastian/global-state 5.0.5 Snapshotting of global state
sebastian/lines-of-code 1.0.3 Library for counting the lines of code in PHP source code
sebastian/object-enumerator 4.0.4 Traverses array structures and object graphs to enumerate all referenced objects
sebastian/object-reflector 2.0.4 Allows reflection of object attributes, including inherited and non-public ones
sebastian/recursion-context 4.0.4 Provides functionality to recursively process PHP variables
sebastian/resource-operations 3.0.3 Provides a list of PHP built-in functions that operate on resources
sebastian/type 3.2.0 Collection of value objects that represent the types of the PHP type system
sebastian/version 3.0.2 Library that helps with managing the version number of Git-hosted PHP projects
theseer/tokenizer 1.2.1 A small library for converting tokenized PHP source code into XML and potentially other formats
I cannot reproduce this with the latest phpunit from the main branch:
$ ~/dev/phpunit/phpunit --process-isolation
PHPUnit 10.0-g76bbb367d by Sebastian Bergmann and contributors.
Runtime: PHP 8.1.10
Configuration: /Users/sjoerdlangkemper/dev/test/phpunit/phpunit.xml
. 1 / 1 (100%)
Time: 00:00.203, Memory: 6.00 MB
There was 1 PHPUnit warning:
1) Your XML configuration validates against a deprecated schema. Migrate your XML configuration using "--migrate-configuration"!
WARNINGS!
Tests: 1, Assertions: 1, Warnings: 1, Notices: 1.
I cannot reproduce this with the latest phpunit from the
mainbranch:
That is because PHPUnit 10 handles notices etc. completely different.
The test case that runs in a separate process outputs errors to stderr, and triggers an error when something has been output on stderr.
The convert...ToExceptions settings are also not passed to the child process, but these wouldn't be used by the above logic anyway.
A workaround is to disable error reporting in the test:
protected function setUp() : void {
error_reporting(0);
}
I also tried ini_set("display_errors", "0"), but that didn't work.
I think that the workaround (error_reporting(0);, see https://github.com/sebastianbergmann/phpunit/issues/5050#issuecomment-1253492301) is good enough.
I am not sure whether this can be properly fixed in PHPUnit 9.
Another trick I found to prevent output on stderr is to close it:
fclose(STDERR);