[Feature]: Create custom message for Any Playwright/Test step, not just expects

[ethan-bm]

🚀 Feature Request

I leverage the Reporter API's "onStepEnd" to feed in a custom message for step.title, which I then use in a custom report. I would like to be able to pass in a custom message, which I can use for step.title, on any step.

Example

await page4.locator('div.sidemenu').click(); await expect(page4.locator('div.nav[id*=steps]'), 'Result Step 4 of 4: Expect Equipment step view to be visible').toBeVisible({ timeout: 30000 }); Appears in my report as:

• locator.click(div.sidemenu)
• Result Step 4 of 4: Expect Equipment step view to be visible

I would like to create a custom statement for the "await page4.locator('div.sidemenu').click();" step.

If I did something like: await page4.(locator('div.sidemenu'), 'Result Step 3 of 4: Click Equipment step view menu').click();
Then it appears in my reporter as:

• Result Step 3 of 4: Click Equipment step view menu

Motivation

I believe it will help expand custom reporters and/or pave for enhanced reports built into Playwright, such as the HTML reporter.

I also use my custom report to report timings on certain steps that are problematic to management. My custom report publishes the results to both JSON and CSV. The CSV is for human readable, easy sharing, and the JSON is to push to dashboard software.

Having an exact statement for specific steps will make running analytics for dashboarding significantly easier. As of now, if a step is repeated in a test, such as "await page4.locator('div.sidemenu').click();", it makes identifying where the exact occurrence we want to report timings on harder. This is made worse if the test is repeated but does not have the same number of steps due to timeouts or failures, as ordinal positioning can't be used as a unique identifier.

[mxschmitt]

We have test.step(title) for that, have you tried using it?

[ethan-bm]

Yes, it creates a nested output in the report that makes data parsing for dashboards wonky. Like so:

Result Step 3 of 4: Click Equipment step view menu :15ms { locator.click(div.sidemenu) :281ms } Result Step 4 of 4: Expect Equipment step view to be visible :2999ms

[ethan-bm]

I also can't wrap all the result steps in test.step(title) and get results or mirrored timings.

Example

//Result 1 of 2: Wait for page 5 to load
const page5 = await page5Promise;

await Promise.all([
    page5.waitForResponse(response => response.url().includes('Get') && response.status() === 200, { timeout: 20000 }),
    expect(page5.locator('div.loaderimg'), 'Part of last steps: Wait for loading image to be hidden').toBeHidden(),

    //Result 2 of 2: Verify Side Menu can be interacted with
    page5.locator('div.sidemenu-expander').click()
]);

• The const is not easily wrapped
• page5.locator('div.sidemenu-expander').click() timing isn't mirrored correctly in the test.step timing

[ethan-bm]

@mxschmitt We found a solution using test.step. It looks like this:

//Declare the page & page promise at the beginning of the test
test('Example for this issue', async({page}) => {

let page5Promise: Promise<Page>;
let page5: Page;

//A bunch of previous steps
......

await test.step('Setup page 5 to open', async () => {
  page5Promise = page.waitForEvent('popup');
  await page.getByRole('button', { name: 'Open', exact: true }).click();
});

await test.step('Capture time it takes for page5 to open and become usable', async () => {
  page5 = await page5Promise;
    await Promise.all([
        page5.waitForResponse((response) => response.url().includes('Get') && response.status() === 200),
        expect(page5.locator('div.loaderimg'), 'Results Helper Step: Wait for loading image to be hidden.').toBeHidden(),
        //Verify the page can be used
        page5.locator('div.sidemenu-expander').click()
    ]);
 });

});

This issue can be closed.

[pavelfeldman]

Why was this issue closed?

Thank you for your involvement. This issue was closed due to limited engagement (upvotes/activity), lack of recent activity, and insufficient actionability. To maintain a manageable database, we prioritize issues based on these factors.

If you disagree with this closure, please open a new issue and reference this one. More support or clarity on its necessity may prompt a review. Your understanding and cooperation are appreciated.