agent-js-playwright icon indicating copy to clipboard operation
agent-js-playwright copied to clipboard

If a test is skipped with a reason, the reason is not shown.

Open quldude opened this issue 1 year ago • 4 comments

Sample test code:

test.beforeAll(async ({ browser }) => {
  await test.step(
    "Check if flag is on",
    async () => {
      const page = await browser.newPage();
      const flag=false;
      test.skip(!flag, "Cannot run suite.");
      await page.close();
    },
  );
});

Actual Result: The skip reason Cannot run suite. is not shown in the skipped test in RP. Playwright html report shows the skipped reason for the test.

quldude avatar Sep 01 '23 18:09 quldude

Hello @quldude ! Thanks for the issue. Actually we don't have special space in RP to show skipping reason. Should we add it to the test as a log message or part of the test description? Will it suit you this way?

AmsterGet avatar Oct 04 '23 08:10 AmsterGet

Could you add it in item details tab? image

Also if a test has multiple annotations, could you add them as well in the same tab?

quldude avatar Oct 04 '23 13:10 quldude

Thank you for the suggestion. We need to discuss your request in detail internally because the RP product collects test automation results from different platforms and languages. AFAIK, most test platforms don't provide annotation output to the reporter, and based on that we can't add a playwright-specific thing as a core feature. At the moment, we have three ways to add meta information to tests, regardless of their language and platform: attributes, description, logs. Since the Item Details tab already contains a description of the element, I think it's easiest to store the skipping reason and perhaps other annotations there.

AmsterGet avatar Oct 05 '23 12:10 AmsterGet

I'm fine with adding meta data to description field. Also keep in mind description field shows the error message if a test fails.

quldude avatar Oct 05 '23 13:10 quldude