[BUG] tests using test.fail annotation that fails is shown as passed in the html report main page

[marcusNumminen]

trafficstars

System info

• Playwright Version: 1.40.1
• Operating System: Win 10
• Browser: All
• Other info: If using test.fail annotation and the test fails then the test is marked as passed in the html report overview

The test should be marked in a similar way as the list reporter show it

As it is now it's really hard to find these tests in the html report. Just to be consistent the skipped test could also be marked in a similar way as in the list report. Here's a suggestion for the html report main page: test.fail => green ✘ test.skipped => blue - test.fixme => yellow -

Source code

playwright.config.ts

import { defineConfig, devices } from '@playwright/test';

export default defineConfig({
  testDir: './tests',
  fullyParallel: true,
  forbidOnly: !!process.env.CI,
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  reporter: [
    ['html', {open: 'never'}],
    ['list']
  ],
  use: { trace: 'on-first-retry',},

  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
  ],
});

example.spec.ts

import { test, expect } from '@playwright/test';

test('I shall pass', async () => {
  expect(true).toBeTruthy();
});

test.skip('skip', async () => {
  expect(true).toBeTruthy();
});

test('conditional skip', async () => {
  test.skip(true, 'intentional skip');
  expect(true).toBeTruthy();
});

test('conditional fail', async () => {
  test.fail(true, 'intentional fail');
  expect(true).toBeFalsy()
});

test.fixme('I should be fixed', async () => {
  expect(true).toBeTruthy();
});

Steps

1. npx playwright test
2. npx playwright show-report

Expected Test annotations should be marked in the html main page the same way as in list reporter

Actual Test cases with test.fail annotation is show as Passed in the main page in the html report

[AndriiFrolov]

Sounds useful But better implement this feature https://github.com/microsoft/playwright/issues/27902

[pavelfeldman]

Why was this issue closed?

Thank you for your contribution to our project. This issue has been closed due to its limited upvotes and recent activity, and insufficient feedback for us to effectively act upon. Our priority is to focus on bugs that reflect higher user engagement and have actionable feedback, to ensure our bug database stays manageable.

Should you feel this closure was in error, please create a new issue and reference this one. We're open to revisiting it given increased support or additional clarity. Your understanding and cooperation are greatly appreciated.