playwright icon indicating copy to clipboard operation
playwright copied to clipboard

[BUG] Videos are not generated when reusing a single page between tests

Open ryanrosello-og opened this issue 2 years ago • 63 comments

Note from maintainers

As a workaround, we recommend recording a trace instead of a video. This will also give you much better debugging experience.


Context:

  • Playwright Version: 1.22.2
  • Operating System: Windows 11
  • Node.js version: v 17.7.1
  • Browser: chromium
  • Extra: [any specific details about your environment]

System:

  • OS: Windows 10 10.0.22000
  • Memory: 1.24 GB / 15.79 GB

Binaries:

  • Node: 17.7.1 - C:\Program Files\nodejs\node.EXE
  • Yarn: 1.22.17 - C:\Program Files\nodejs\yarn.CMD
  • npm: 8.5.2 - C:\Program Files\nodejs\npm.CMD

Languages:

  • Bash: 5.0.17 - C:\Windows\system32\bash.EXE

Code Snippet playwright.config.ts

const config: PlaywrightTestConfig = {
  reporter: [['dot'],['html']],
  use: {
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
    video:'retain-on-failure',
  },
};

export default config;

broke.spec.ts (sourced from https://playwright.dev/docs/test-retries#reuse-single-page-between-tests)

import { test, Page } from '@playwright/test';

test.describe.configure({ mode: 'serial' });

let page: Page;

test.beforeAll(async ({ browser }) => {
  page = await browser.newPage();
});

test.afterAll(async () => {
  await page.close();
});

test('a failing test', async () => {
  await page.click('text=Get Started fail', {timeout: 1000});  // << deliberate fail
});

Describe the bug

Videos are not generated when reusing a single page between tests

Set the following configuration:

  use: {
    trace: 'retain-on-failure',
    screenshot: 'only-on-failure',
    video:'retain-on-failure',
  },

Run a failing test

Expected Result:

The spec above should successfully generate a screenshot, test steps, trace file and a video within the HTML result.

Actual Result:

The video is not being generated:

2022-06-13_09-09-29

ryanrosello-og avatar Jun 12 '22 23:06 ryanrosello-og

Hi @ryanrosello-og If you are picking the browser context configuration directly from xyz.config.js file and have not defined the context in the spec itself, then you would want to try the following configuration in order to get the videos generated.

use: { trace: 'retain-on-failure', screenshot: 'only-on-failure', video:'retain-on-failure', contextOptions: {recordVideo: { dir: "<path_to_store_videos>/videos/'"}} },

akarsh17 avatar Jun 13 '22 06:06 akarsh17

thanks for the suggestion @akarsh17

I tried the following:

Declared the recordVideo directly into the config

const config: PlaywrightTestConfig = {
  reporter: [['dot'],['html']],
  use: {
    trace:'on',
    contextOptions : {
      recordVideo:{
        dir:'./videos'
      }
    }
  }
};

export default config;

And also directly into the beforeAll of the spec:

import { test, Page, chromium, Browser } from "@playwright/test";

let page: Page;
let browser: Browser;
test.beforeAll(async () => {
  browser = await chromium.launch();
  const context = await browser.newContext({
    recordVideo: {
      dir: `./videos/`,
    },
  });
  page = await context.newPage();
});

test.afterAll(async () => {
  await browser.close();
});

test("a failing test", async () => {
  await page.goto("https://www.google.com");
  await page.click("text=Get Started fail", { timeout: 1000 }); // << deliberate fail
});

Still no luck, I can see the videos are created but the videos are not being embedded in the HTML report which kind makes it difficult to trace which video belong to which result.

2022-06-13_17-14-25

2022-06-13_17-18-02

ryanrosello-og avatar Jun 13 '22 07:06 ryanrosello-og

This is indeed a known issue: video recording produces a single video file from the creation of the page until its closure, as of Jun 2022.

dgozman avatar Jun 14 '22 00:06 dgozman

Hi!

I'd expect this to be a higher prio bug (first report September 2021, almost 🍰 🕯️ 🎉 )

I'm following the simple docs on overriding the page fixture, just as described, and I lose videos because of that? Since not fixing this, there should be a disclaimer on those docs imo

raulparada avatar Jul 12 '22 09:07 raulparada

I'm also running into this issue - specifically, that video recording doesn't work in pages created with brower.newContext, as described here.

My use case: I'm using storageState for authentication, as described in the docs, except that I only want to use it for some tests. So when I need it, I call browser.newContext({ storageState: ... }). But then pages created from that context don't get video recordings.

For now I plan to use the recordVideo option and manual file manipulation to emulate video: 'retain-on-failure'.

mweidner037 avatar Aug 08 '22 13:08 mweidner037

I'm having the same issue as @mweidner037. I use storageState and create pages from browser.newContext({ storageState: ... }). I'm trying to take retain videos on failure, but I'm not seeing any video recordings when my tests fail.

KeionneDerousselle avatar Aug 30 '22 18:08 KeionneDerousselle

I also get this issue, could you please spend time to fix this. Thanks a lot.

NgocVo3112 avatar Nov 28 '22 08:11 NgocVo3112

I got this issue and tried all the possible methods but could not get the video. Please fix this issue

spiderim avatar Dec 12 '22 06:12 spiderim

I believe I have this working with a fixture but it's not pretty. A worker-scoped fixture initiates video recording to a temp dir and clean up of that dir on teardown. A test-scoped fixture attaches the video to the test reports on failure or time out.

fixture:

import { test as base, BrowserContext } from "@playwright/test";
import * as fs from "fs";

export const test = base.extend<{
    recordVideo: BrowserContext;
    attachVideo: BrowserContext;
}>({
    recordVideo: [
        async ({ browser }, use) => {
            const context = await browser.newContext({
                recordVideo: {
                    dir: "./playwright-video",
                },
            });

            await use(context);

            fs.rmSync("./playwright-video", { recursive: true });
        },
        // @ts-expect-error
        { scope: "worker" },
    ],

    attachVideo: [
        async ({ recordVideo }, use, testInfo) => {
            const path = await recordVideo.pages()[0].video()?.path();

            await use(recordVideo);

            if (
                testInfo.status === "failed" ||
                testInfo.status === "timedOut"
            ) {
                await recordVideo.close();
                testInfo.attach("video", {
                    path: path,
                });
            }
        },
        { scope: "test", auto: true },
    ],
});

export { expect, Page, Locator, Response } from "@playwright/test";

usage:

import { test, expect, Page } from "../../video-fixture";
import { PageUnderTest } from "../../pageObjects/PageUnderTest"

let page: Page;
let pageUnderTest: PageUnderTest;

test.describe.serial("manage round", () => {
    test.beforeAll(async ({ recordVideo }) => {
        page = await recordVideo.newPage();

        pageUnderTest = new PageUnderTest(page);
        await pageUnderTest.open();
    });

    test.afterAll(async () => {
        await page.close();
    });

    test("renders", async () => {
        await expect(pageUnderTest.header).toHaveTest("header");
        ...
    });

   test("test other stuff on same page", async () => {
        ...
   });
});

angelo-loria avatar Dec 12 '22 20:12 angelo-loria

I have the same problem with reusing StorageState as @mweidner037 and @KeionneDerousselle described.

lsnmarcint avatar Dec 19 '22 14:12 lsnmarcint

Just adding to this, I'm getting this on MacOS with Playwright version 1.29.1, so it's not just a windows OS issue.

sc0ttdav3y avatar Dec 27 '22 09:12 sc0ttdav3y

Hi, I'm also facing the same issue with the Page object Model pattern, the video recording does not work. Waiting for the solution. My code looks like below,

`import { test, expect, Page} from "@playwright/test"; import { LoginPage } from "../pages/loginPage";

let page: Page; let loginPage: LoginPage;

test.describe('Validate login page', async () => {

test.beforeAll(async ({ browser }) => { page = await browser.newPage(); loginPage = new LoginPage(page);

await loginPage.navigateTo();

});

test('login page displayed', async () => { await expect(loginPage.Title).toHaveText('ABC'); await expect(loginPage.Title).toContainText('Log in'); });

test('test A', async () => { expect('A).toEqual(testA); });

test('testB', async () => { expect('B).toEqual(testB); });

test.afterAll(async () => { console.log("Login page validated successfully"); await page.close(); });`

vishal7788 avatar Jan 18 '23 12:01 vishal7788

I have the very same situation - no video attached :( I've also tried to use the tip from @ryanrosello-og contextOptions: {recordVideo: { dir: "<path_to_store_videos>/videos/'"}} and const context = await browser.newContext({ recordVideo: { dir: ./videos/, }, }); But as a result I've got only empty files in ./videos directory - not video at all

rubanenkos avatar Jan 21 '23 13:01 rubanenkos

si this is the same issue? https://github.com/microsoft/playwright/discussions/20946

daniavander avatar Feb 16 '23 10:02 daniavander

Any plan to fix this issue? I am creating new browser contexts by using storageState files for different user roles in fixtures. In this case, videos are not getting created.

nudratuppal avatar Feb 20 '23 16:02 nudratuppal

I am having a similar issue while working with browserContext. I hope this gets fixed soon, as this is a critical issue.

rishoo2019 avatar Mar 07 '23 16:03 rishoo2019

Our team is also affected by this issue.

itz4blitz avatar Mar 28 '23 16:03 itz4blitz

I've been using the solution I outlined in my previous comment for months; it's still working well. I did end up layering another fixture on top of that one for standard page object use.

Edit 9/22/23: cleaned up the fixture so it no longer throws a type error and doesn't need a beforeAll hook @ibrocodes7

Fixture for enabling video recording:

import { test as base, BrowserContext, Page } from "@playwright/test";
import * as fs from "fs";

/**
 * this fixture is needed to record and attach videos on failed tests when
 * tests are run in serial mode (i.e. browser is not closed between tests)
 */
export const test = base.extend<
    {
        attachVideoPage: Page;
    },
    { createVideoContext: BrowserContext }
>({
    createVideoContext: [
        async ({ browser }, use) => {
            const context = await browser.newContext({
                recordVideo: {
                    dir: "./playwright-video",
                },
            });

            await use(context);

            fs.rmSync("./playwright-video", { recursive: true });
        },
        { scope: "worker" },
    ],

    attachVideoPage: [
        async ({ createVideoContext }, use, testInfo) => {
            let page;
            if (createVideoContext.pages().length === 1) {
                page = createVideoContext.pages()[0];
            } else {
                page = await createVideoContext.newPage();
            }
            await use(page);

            if (
                testInfo.status === "failed" ||
                testInfo.status === "timedOut"
            ) {
                const path = await createVideoContext
                    .pages()[0]
                    .video()
                    ?.path();
                await createVideoContext.close();
                testInfo.attach("video", {
                    path: path,
                });
            }
        },
        { scope: "test", auto: true },
    ],
});

export { expect, Page, Locator, Response } from "@playwright/test";

Then I have my page object fixture:

import { test as base, Page } from [fixture for recording video];
import { BrowserContext } from "@playwright/test";
import { ExamplePage1 } from "./examplePage1";
import { ExamplePage2 } from "./examplePage2";

export type PageObjects = {
    examplePage1: ExamplePage1;
    examplePage2: ExamplePage2;
};

export const test = base.extend<PageObjects>({
    examplePage1: async ({ attachVideoPage }, use) => {
        const examplePage1 = new ExamplePage1(attachVideoPage);
        await use(examplePage1);
    },
    examplePage2: async ({ attachVideoPage }, use) => {
        const examplePage2 = new ExamplePage2(attachVideoPage);
        await use(examplePage2);
    },
});

export { expect, Page, Locator, Response } from "@playwright/test";

Usage in spec file:

import { test, expect } from [page object fixture above];


test("renders", async ({ examplePage1 }) => {
    await examplePage1.open()'
    await expect(examplePage1.heading).toHaveText(
       "Example Page 1"
    );
});

angelo-loria avatar Mar 29 '23 21:03 angelo-loria

I've been using the solution I outlined in my previous comment for months; it's still working well. I did end up layering another fixture on top of that one for standard page object use.

Hi @angelo-loria I'm trying to understand your implementation a bit better for our use case. I had a question I'm hoping you can help with:

I notice you're using two fixtures with the second being the page object fixture. In some of our tests, the only fixture we are using is browser. Is it possible to extend the browser fixture so that anytime a browser is created and used in a test, a recording is also generated?

itz4blitz avatar Apr 03 '23 12:04 itz4blitz

I've been using the solution I outlined in my previous comment for months; it's still working well. I did end up layering another fixture on top of that one for standard page object use.

Hi @angelo-loria I'm trying to understand your implementation a bit better for our use case. I had a question I'm hoping you can help with:

I notice you're using two fixtures with the second being the page object fixture. In some of our tests, the only fixture we are using is browser. Is it possible to extend the browser fixture so that anytime a browser is created and used in a test, a recording is also generated?

@itz4blitz Not sure if I'm understanding you correctly but I have the browser context being created in the recordVideo fixture. If you're not using any page objects/don't need any other fixtures, you can use the recordVideo fixture directly in your spec file in the beforeAll hook, like the example in my original comment.

angelo-loria avatar Apr 03 '23 16:04 angelo-loria

this is my workaround for now in case it helps anyone. not too ugly. (Note: In this redacted example, I always save videos, this may not be what you want. You can use testInfo to check the test status and conditionally save videos).

import { test as base, Page, Video } from '@playwright/test'

import { HomePage } from './../pages/homePage'

/** HomePage.create looks like this

  static async create(browser: Browser, fileName): Promise<Page> {
    if (process.env.BASE_URL == null) {
      throw new Error('No BASE_URL configured')
    }

    const userContext = await browser.newContext({
      storageState: `./e2e-tests/sessionStorage/${fileName}.json`,
      recordVideo:
        {
        dir: `test-results/videos`,
        }
    })
    const userPage = await userContext.newPage()
    await userPage.goto(process.env.BASE_URL)

    return userPage
  }
*/

type Fixtures = {
  userOnePage: Page
  userTwoPage: Page
}
export const test = base.extend<Fixtures & { saveVideos: void }>({
  userOnePage: async ({ browser }, use) => {
    await use(await HomePage.create(browser, 'userOneStorageState'))
  },
  userTwoPage: async ({ browser }, use) => {
    await use(await HomePage.create(browser, 'userTwoStorageState'))
  },
  saveVideos: [async ({ userOnePage, userTwoPage, browser }, use, testInfo) => {
    await use();

    await Promise.all(browser.contexts().map((context) => {
      return context.close()
    }))
    const video1 = await userOnePage.video()?.path()
    const video2 = await userTwoPage.video()?.path()
    const nonNullVideos = [{ title: 'User One', path: video1 }, { title: 'User Two', path: video2 }].filter((video) => Boolean(video.path)) as {title: string, path: string}[]
    nonNullVideos.forEach((video) => {
      testInfo.attachments.push({ name: `${video.title} `, path: video.path, contentType: 'video/webm'})
    })
  }, { scope: 'test', auto: true }],

})

PS: Playwright team, please update the docs for videos or browsercontexts to make it very clear that this "issue"/limitation exists and is not being prioritized

shivghai avatar Apr 26 '23 12:04 shivghai

I think this problem is also present when using worker scoped fixtures. We get no video captures even if we use a new page instance for each test. Would be nice to have a fix for this.

mstepin avatar May 11 '23 06:05 mstepin

I confirm that this bug still occurs in type script, in python the video is saved correctly.

david-m-globant avatar May 17 '23 18:05 david-m-globant

when will this fix develop to the release?

daniavander avatar May 19 '23 08:05 daniavander

I also get this issue, when will this fix get released?

Vaishnavi-Thiyagaraj avatar Jun 03 '23 00:06 Vaishnavi-Thiyagaraj

We discovered a potential solution to allow us to create a video recording in a test which is using the browser fixture:

import * as path from "path"
import * as fs from 'fs';
import { expect, test } from "@playwright/test";

test.only('manually created context', async ({ browser }, testInfo) => {
  const videoDir = path.join(testInfo.outputPath(), 'videos');
  const context = await browser.newContext({ recordVideo: { dir: videoDir } });
  const page = await context.newPage();
  const page2 = await context.newPage();
  try {
    await page2.goto('http://localhost:3030/login?pageSize=10');
    await page.goto('http://localhost:3030/login?pageSize=10');
    await expect(page).toHaveURL('http://localhost:3030/login?pageSize=10');
  } finally {
    await page.context().close();
    const videoFiles = fs.readdirSync(videoDir);
    
    if (videoFiles.length > 0) {
      for (let i = videoFiles.length; i > 0; i--) {
        let videoFile = path.join(videoDir, videoFiles[i - 1]);
        await testInfo.attach('video', { path: videoFile });
      }
    }
  }
});

But adding this change accross all of our tests which uses the browser fixture is tome consuming and not ideal. We are hoping that someone has some ideas how we might abstract this change. We currently have extended test to use custom fixtures. It seems that it can be a good place to globally apply this fix but we are not sure how.

Derrbal avatar Jun 08 '23 12:06 Derrbal

Sharing is caring @Derrbal :). What's the solution?

spaniardmaximus avatar Jun 08 '23 12:06 spaniardmaximus

Caring is sharing @Derrbal :). What's the solution?

My apology, I've updated comment ;)

Derrbal avatar Jun 08 '23 12:06 Derrbal

Hi @angelo-loria I wanted to reach out because I'm running into an error while trying out the solution you provided. Any thoughts on why this might be happening? On top of that, I'm also getting an error dialog on the recordVideo object. Your help would be much appreciated!

    TypeError: Cannot read properties of undefined (reading 'video')
Screenshot 2023-08-04 at 18 04 41

Error on recordVideo:

Type '[({ browser }: { recordVideo: BrowserContext; attachVideo: BrowserContext; } & PlaywrightTestArgs & PlaywrightTestOptions & PlaywrightWorkerArgs & PlaywrightWorkerOptions, use: (r: BrowserContext) => Promise<...>) => Promise<...>, { ...; }]' is not assignable to type 'TestFixtureValue<BrowserContext, { recordVideo: BrowserContext; attachVideo: BrowserContext; } & PlaywrightTestArgs & PlaywrightTestOptions & PlaywrightWorkerArgs & PlaywrightWorkerOptions> | [...]'.
  Type '[({ browser }: { recordVideo: BrowserContext; attachVideo: BrowserContext; } & PlaywrightTestArgs & PlaywrightTestOptions & PlaywrightWorkerArgs & PlaywrightWorkerOptions, use: (r: BrowserContext) => Promise<...>) => Promise<...>, { ...; }]' is not assignable to type 'TestFixture<BrowserContext, { recordVideo: BrowserContext; attachVideo: BrowserContext; } & PlaywrightTestArgs & PlaywrightTestOptions & PlaywrightWorkerArgs & PlaywrightWorkerOptions> | [...]'.
    Type '[({ browser }: { recordVideo: BrowserContext; attachVideo: BrowserContext; } & PlaywrightTestArgs & PlaywrightTestOptions & PlaywrightWorkerArgs & PlaywrightWorkerOptions, use: (r: BrowserContext) => Promise<...>) => Promise<...>, { ...; }]' is not assignable to type '[TestFixtureValue<BrowserContext, { recordVideo: BrowserContext; attachVideo: BrowserContext; } & PlaywrightTestArgs & PlaywrightTestOptions & PlaywrightWorkerArgs & PlaywrightWorkerOptions>, { ...; }]'.
      Type at position 1 in source is not compatible with type at position 1 in target.
        The types of 'scope' are incompatible between these types.
          Type '"worker"' is not assignable to type '"test"'.ts(2322)
video-fixtures.ts(5, 3): The expected type comes from property 'recordVideo' which is declared here on type 'Fixtures<{ recordVideo: BrowserContext; attachVideo: BrowserContext; }, {}, PlaywrightTestArgs & PlaywrightTestOptions, PlaywrightWorkerArgs & PlaywrightWorkerOptions>'

ibrocodes7 avatar Aug 04 '23 12:08 ibrocodes7

@ibrocodes7 This is hacky but I would just throw a // @ts-expect-error above recordVideo. I should revisit this soon and see if there's a cleaner way to implement it. It is still working well for me; I have the fixture in an npm package that is used by a handful of Playwright projects daily and videos are still getting attached to my reports.

angelo-loria avatar Aug 04 '23 14:08 angelo-loria