allure-phpunit icon indicating copy to clipboard operation
allure-phpunit copied to clipboard

If a test has dataProvider, the report doesn't have the name of the test.

Open smamykin opened this issue 3 years ago • 1 comments

Hi. All my tests, that have annotation @dataProvider don't have name, description and so on.

To reproduce, I wrote a couple of simple tests:

<?php

declare(strict_types=1);

namespace App\Tests\Unit;

use PHPUnit\Framework\TestCase;

class CalculatorTest extends TestCase
{
    public function testSumSimple(): void
    {
        $this->assertSame(33, $this->sum(30, 3));
    }

    /**
     * @return void
     */
    public function testSumWithReturnAnnotation(): void
    {
        $this->assertSame(33, $this->sum(30, 3));
    }

    public function testSumWIthParam(int $a = 30): void
    {
        $this->assertSame(33, $this->sum($a, 3));
    }

    /**
     * @param int $a
     * @return void
     */
    public function testSumWithReturnAndParamAnnotation(int $a = 30): void
    {
        $this->assertSame(33, $this->sum($a, 3));
    }

    /**
     * @return void
     * @depends testSumSimple
     */
    public function testSumWithDepends(): void
    {
        $this->assertSame(33, $this->sum(30, 3));
    }

    /**
     * Hello world
     *
     * @param int $a
     * @return void
     */
    public function testSumWithSimpleComment(int $a = 30): void
    {
        $this->assertSame(33, $this->sum($a, 3));
    }

    /** !!! The test with the problem
     * @dataProvider providerSumWithDataProvider
     * @param int $a
     * @return void
     */
    public function testSumWithDataProvider(int $a = 30): void
    {
        $this->assertSame(33, $this->sum($a, 3));
    }

    public function providerSumWithDataProvider(): array
    {
        return [
            [30],
        ];
    }

    /** !!! The test with the problem
     * @dataProvider providerSumWithDataProviderGenerator
     * @param int $a
     * @return void
     */
    public function testSumWithDataProviderGenerator(int $a = 30): void
    {
        $this->assertSame(33, $this->sum($a, 3));
    }

    public function providerSumWithDataProviderGenerator(): \Generator
    {
        yield [30];
    }

    /**
     * @dataProvider providerSumWithDataProviderGeneratorAndNAmes
     * @param int $a
     * @return void
     */
    public function testSumWithDataProviderGeneratorAndNAmes(int $a = 30): void
    {
        $this->assertSame(33, $this->sum($a, 3));
    }

    public function providerSumWithDataProviderGeneratorAndNAmes(): \Generator
    {
        yield 'name of the set' => [30];
    }

    private function sum(int $a, int $b): int
    {
        return $a + $b;
    }
}

After running the test and generating report I have the following result image

The tests testSumWithDataProvider and testSumWithDataProviderGenerator have names "Unknown tests"

The test testSumWithDataProviderGeneratorAndNames isn't processed while generating the report by CLI command, but I suppose it is another problem. I put aside this problem for a while.

I've debugged a bit, and can find the place where the TestInfo object got the wrong value for its field "method" - \Qameta\Allure\PHPUnit\Internal\TestLifecycle::buildTestInfo.

 private function buildTestInfo(string $test, ?string $host = null, ?string $thread = null): TestInfo
    {
        $dataLabelMatchResult = preg_match(
            '#^([^\s]+)\s+with\s+data\s+set\s+"(.*)"\s+\(.+\)$#',
            $test,
            $matches,
        );
        // ... other content of the method
    }

There are quotes in the regexp, but the string $test has no quotes and looks like "App\Tests\Unit\CalculatorTest::testSumWithDataProviderGenerator with data set #0 (30)"

Because of that, the field TestInfo.method contains the string "testSumWithDataProviderGenerator with data set #0 (30)".

Later in the code, when it tries to parse annotations of this method, there is exception thrown in \Qameta\Allure\PHPUnit\Internal\TestUpdater::setInfo and caught in \Qameta\Allure\AllureLifecycle::updateTest, because the name of method is incorrect.

In the result, TestInfo has empty fields.

I use the allure-framework/allure-phpunit:2.0.0, allure-framework/allure-php-commons:2.0.0 and phpunit:9.5.0 By the way, CLI command allure --version returns 2.13.8.

Can you advise what to do in such a situation?

smamykin avatar Jan 09 '23 15:01 smamykin

I've found the PR about it https://github.com/allure-framework/allure-phpunit/pull/77 opened on Aug 19, 2022. I would appreciate it greatly if you could suggest any workarounds while this PR is not accepted.

smamykin avatar Jan 09 '23 15:01 smamykin