[allure-playwright] Unexpected interaction between allurectl v2.17.0 and parseTestPlan() in allure-js-commons

[eripley]

Describe the bug

If you’re using allurectl v2.17.0 together with the allure-playwright adapter v 3.3.3, you may encounter an issue originating here.

It seems that allurectl automatically sets the environment variable ALLURE_TESTPLAN_PATH to either null or some non-empty value. If you don’t actually have a testplan.json file, this behavior leads to readFileSync/JSON.parse throwing an error, which in turn forces execution into the catch block.

  catch (e) {
    // eslint-disable-next-line no-console
    console.error(`could not parse test plan ${testPlanPath}`, e);
    return undefined;

To Reproduce

1. install allurectl v2.17.0
2. install allure-playwright v 3.3.3
3. install playwright v 1.55.0 (no matter about the version)
4. add report adapter to the pw config like this:

  reporter: [
    [
      'allure-playwright',
      {
        resultsDir: 'output',
      },
    ],
  ],

5. run playwright cli via allurectl

allurectl watch -- npx playwright test --pass-with-no-tests --project=api --grep-invert '@ignore'

you will get an issue from the catch block.

Expected behavior

This issue does not occur if the ALLURE_TESTPLAN_PATH variable is simply absent from your environment and file testplan.json is exist somewhere.

I don’t have visibility into the internals of your CLI since it’s hosted in private repositories, but from the outside it looks like either:

• ALLURE_TESTPLAN_PATH shouldn’t be set when there’s no test plan, or
• the parseTestPlan function should be updated to handle this case more gracefully.

Screenshots

[epszaw]

Could you check ALLURE_TESTPLAN_PATH env variable? Which value does it contain? Does the file exist?

[eripley]

Hey @epszaw The issue is that we’re not using the testplan.json feature, and therefore we don't define the ALLURE_TESTPLAN_PATH variable. Take a look at the my first message - I've already pinpointed the problem within your handling logic. What's left is to adjust the business logic in your CLI to eliminate this collision. Unfortunately, since the CLI source code is closed, I can't determine the exact origin of the issue in more detail.

[eripley]

Hey @epszaw ! Hope you're doing well! Any luck with that?