phpunit icon indicating copy to clipboard operation
phpunit copied to clipboard

Timeouts for small, medium and large tests should be able to be specified in milliseconds

Open CJDennis opened this issue 3 months ago • 5 comments

1 second (the minimum sensible value) is a large test for me. I really don't want thousands of small tests taking 0.999 seconds each! Can the length for small, medium, and large tests be specified in milliseconds?

         timeoutForSmallTests="1"
         timeoutForMediumTests="10"
         timeoutForLargeTests="60"

I don't know if you'd want to reuse the existing attributes or define new ones:

         timeoutInMillisecondsForSmallTests="1000"
         timeoutInMillisecondsForMediumTests="10000"
         timeoutInMillisecondsForLargeTests="60000"

CJDennis avatar Sep 11 '25 10:09 CJDennis

alternatively we add the notion of units like

         timeoutForSmallTests="1s"
         timeoutForMediumTests="10s"
         timeoutForLargeTests="60s"

or

         timeoutForSmallTests="1000ms"
         timeoutForMediumTests="10000ms"
         timeoutForLargeTests="60000ms"

etc.

staabm avatar Sep 11 '25 12:09 staabm

Before discussing how we would configure this, we need to establish whether it is feasible. And I do not think it is as the pcntl_alarm() function only supports second granularity, not millisecond granularity.

For reference, the code in question is here: https://github.com/sebastianbergmann/php-invoker/blob/6.0.0/src/Invoker.php#L29-L59

sebastianbergmann avatar Sep 11 '25 12:09 sebastianbergmann

indeed. maybe raising a feature request for php-src is one way to go then

staabm avatar Sep 11 '25 12:09 staabm

pcntl_alarm() is but a wrapper for the alarm() system call which only supports second granularity.

sebastianbergmann avatar Sep 11 '25 12:09 sebastianbergmann

I just discussed this with @TimWolla and he said that there are alternatives to the alarm() system call that support milliseconds.

Of course, somebody needs to implement usage of such an alternative for PHP before we can use it in php-invoker / PHPUnit.

sebastianbergmann avatar Sep 11 '25 12:09 sebastianbergmann