[Feature]: retain video for first failure only (retain-on-first-failure)

[aaron-goff]

🚀 Feature Request

tl;dr -> #29531 except for video

I would like to file a feature request to create an option for the video recording: retain-on-first-failure, that behaves identical to to the trace option, except for video. This would retain the trace files only on the test's first failing attempt, not during retries (or for every run of the test).

Example

import { defineConfig } from '@playwright/test';
export default defineConfig({
  retries: 1,
  use: {
    // if a test fails, it will retry once, but a trace will only be collected for the first failure
    video: 'retain-on-first-failure',
  },
});

Motivation

For the same reasons outlined in #29531, my team has retries enabled, as well as video: 'on'. Most importantly for my team, this can cause our test results .zip file to balloon if we have a long-running test that fails during all attempts. We would like the option to only retain the first failure's video.

Motivation copied from #29531

My team uses Playwright for end-to-end tests that cause state modifications within our app. In theory, our tests "clean up" after themselves after each test completes, or fails, but implementing such functionality perfectly is difficult, especially for longer tests that fail in between two state changes. We have run into issues like the following:

1. a test fails on the first try
2. the test tries to clean up after itself, but separately fails to do so
3. the test fails on the first retry as a side effect of the failed cleanup step

The option I am suggesting would be preferable to:

• `on-first-retry` because it captures the original failure rather than a side effect of the original failure
• `retain-on-failure` because it saves space