playwright icon indicating copy to clipboard operation
playwright copied to clipboard

[Feature] Time/Date emulation via e.g. a `clock()` primitive

Open chambo-e opened this issue 3 years ago • 43 comments

See https://github.com/microsoft/playwright/issues/6347#issuecomment-965887758 for a current workaround.

Edited by the Playwright team.


Hello,

We are using playwright to run automated tests on some websites, we record external requests to be able to replace the tests in isolation. We would love have a way to set the internal clock, same as clock() from cypress to improve reproductibility

This has already been mentioned here https://github.com/microsoft/playwright/issues/820 but I did not found any follow up issues :)

chambo-e avatar Apr 28 '21 09:04 chambo-e

Hopefully, some folks would upvote this more and more.

idxn avatar Aug 21 '21 17:08 idxn

is there any workaround that we could use currently? Some scripts that stub the Date via page.evaluate maybe?

Shaddix avatar Aug 25 '21 15:08 Shaddix

Is it possible to increase the time forward by x minutes to test token expiry as well. UseCase: After logging into web app , token gets expires in 2 hours if site stays idle, and user is force logged out. This requires us to forward time to 2 hours from logged in time to validate this scenario.

jithinjosejacob avatar Oct 24 '21 12:10 jithinjosejacob

You can use sinon fake-timers for this.

To do so:

  1. Install sinon: npm install sinon
  2. Setup a beforeEach hook that injects sinon in all pages:
    test.beforeEach(async ({ context }) => {
      // Install Sinon in all the pages in the context
      await context.addInitScript({
        path: path.join(__dirname, '..', './node_modules/sinon/pkg/sinon.js'),
      });
      // Auto-enable sinon right away
      await context.addInitScript(() => {
        window.__clock = sinon.useFakeTimers();
      });
    });
    
  3. Use await page.evaluate(() => window.__clock.tick(1000)) to tick time inside tests.

A full example would look like this:

// e2e/fakeTime.spec.ts

import { test, expect } from '@playwright/test';
import path from 'path';

// Install Sinon in all the pages in the context
test.beforeEach(async ({ context }) => {
  await context.addInitScript({
    path: path.join(__dirname, '..', './node_modules/sinon/pkg/sinon.js'),
  });
  await context.addInitScript(() => {
    window.__clock = sinon.useFakeTimers();
  });
});

test('fake time test', async ({ page }) => {
  // Implement a small time on the page
  await page.setContent(`
    <h1>UTC Time: <x-time></x-time></h1>
    <script>
      const time = document.querySelector('x-time');
      (function renderLoop() {
        const date = new Date();
        time.textContent = [date.getUTCHours(), date.getUTCMinutes(), date.getUTCSeconds()]
          .map(number => String(number).padStart(2, '0'))
          .join(':');
        setTimeout(renderLoop, 1000);
      })();
    </script>
  `);

  // Ensure controlled time
  await expect(page.locator('x-time')).toHaveText('00:00:00');
  await page.evaluate(() => window.__clock.tick(1000));
  await expect(page.locator('x-time')).toHaveText('00:00:01');
});

aslushnikov avatar Nov 11 '21 00:11 aslushnikov

This feature is critical for visual regression testing when an application's clock is controlled by the os.

unlikelyzero avatar Nov 12 '21 22:11 unlikelyzero

Note, for everyone who is discovering the example that @aslushnikov provided, you may need to explicitly set the window clock like so:

    await context.addInitScript(() => {
        window.__clock = sinon.useFakeTimers({
            now: 1483228800000,
            shouldAdvanceTime: true
        });
    });

unlikelyzero avatar Mar 17 '22 16:03 unlikelyzero

Here's a very simple and robust solution to set the Time/Date in your tests:

// Pick the new/fake "now" for you test pages.
const fakeNow = new Date("March 14 2042 13:37:11").valueOf();

// Update the Date accordingly in your test pages
await page.addInitScript(`{
  // Extend Date constructor to default to fakeNow
  Date = class extends Date {
    constructor(...args) {
      if (args.length === 0) {
        super(${fakeNow});
      } else {
        super(...args);
      }
    }
  }
  // Override Date.now() to start from fakeNow
  const __DateNowOffset = ${fakeNow} - Date.now();
  const __DateNow = Date.now;
  Date.now = () => __DateNow() + __DateNowOffset;
}`);

That's all! No need for a library or to dig into your node_modules folder to inject into the pages.

Hope that helps,

p01 avatar Apr 01 '22 12:04 p01

Very clever!

unlikelyzero avatar Apr 01 '22 17:04 unlikelyzero

You can use sinon fake-timers for this.

To do so:

  1. Install sinon: npm install sinon
  2. Setup a beforeEach hook that injects sinon in all pages:
    test.beforeEach(async ({ context }) => {
      // Install Sinon in all the pages in the context
      await context.addInitScript({
        path: path.join(__dirname, '..', './node_modules/sinon/pkg/sinon.js'),
      });
      // Auto-enable sinon right away
      await context.addInitScript(() => {
        window.__clock = sinon.useFakeTimers();
      });
    });
    
  3. Use await page.evaluate(() => window.__clock.tick(1000)) to tick time inside tests.

A full example would look like this:

// e2e/fakeTime.spec.ts

import { test, expect } from '@playwright/test';
import path from 'path';

// Install Sinon in all the pages in the context
test.beforeEach(async ({ context }) => {
  await context.addInitScript({
    path: path.join(__dirname, '..', './node_modules/sinon/pkg/sinon.js'),
  });
  await context.addInitScript(() => {
    window.__clock = sinon.useFakeTimers();
  });
});

test('fake time test', async ({ page }) => {
  // Implement a small time on the page
  await page.setContent(`
    <h1>UTC Time: <x-time></x-time></h1>
    <script>
      const time = document.querySelector('x-time');
      (function renderLoop() {
        const date = new Date();
        time.textContent = [date.getUTCHours(), date.getUTCMinutes(), date.getUTCSeconds()]
          .map(number => String(number).padStart(2, '0'))
          .join(':');
        setTimeout(renderLoop, 1000);
      })();
    </script>
  `);

  // Ensure controlled time
  await expect(page.locator('x-time')).toHaveText('00:00:00');
  await page.evaluate(() => window.__clock.tick(1000));
  await expect(page.locator('x-time')).toHaveText('00:00:01');
});

Note that @sinonjs/fake-timers conflicts with playwright waitForFunction.

https://github.com/microsoft/playwright/blob/446de1a615cc440cabd7ef81a45d0ddee63a683d/packages/playwright-core/src/server/injected/injectedScript.ts#L321-L327

You might want to explicitly set which functions to fake.

https://github.com/sinonjs/fake-timers#var-clock--faketimersinstallconfig

await page.addInitScript(() => {
  window.__clock = sinon.useFakeTimers({
    toFake: [
      'setTimeout',
      'clearTimeout',
      // 'setImmediate',
      // 'clearImmediate',
      'setInterval',
      'clearInterval',
      // 'Date',
      // 'requestAnimationFrame',
      // 'cancelAnimationFrame',
      // 'requestIdleCallback',
      // 'cancelIdleCallback',
      // 'hrtime',
      // 'performance',
    ],
  });
});

mizozobu avatar Apr 08 '22 13:04 mizozobu

that makes it a bit cumbersome if your application mostly uses performance.now(), in case you can't mock that using sinon because of playwright

DerGernTod avatar Apr 12 '22 09:04 DerGernTod

What are your scenarios related to Time/Date ?

The scenarios I was dealing with were: showing absolute and relative dates/times. I didn't need to "stop" or slow down time. The code snippet I posted above solved ALL our scenarios.

p01 avatar Apr 12 '22 11:04 p01

basically measuring performance. call performance.now(), do something, call performance.now() again, then produce data depending on how long that took. i'm aware that this is also possible using Date.now(), but a) not in that much detail and b) it doesn't depend on the system clock

DerGernTod avatar Apr 12 '22 20:04 DerGernTod

@DerGernTod from what you describe, I think the code snippet I posted above would work. It's very light weight and simply allows to se the current Date, time to what you need. It only offset new Date(), and Date.now(). All other Date methods and performance.now() are left untouched and work exactly as expected.

p01 avatar Apr 13 '22 16:04 p01

@p01 I think you misunderstood. I want to test that my measurements are correct if x time passed and different actions happened in between. For that I need performance.now to be properly emulated

DerGernTod avatar Apr 13 '22 17:04 DerGernTod

@p01 I think you misunderstood. I want to test that my measurements are correct if x time passed and different actions happened in between. For that I need performance.now to be properly emulated

I'm curious. Can you provide an example? We're implementing both performance.now() and Sinon in our tests for performance testing.

Do you have a use case and code example?

unlikelyzero avatar Apr 13 '22 17:04 unlikelyzero

performance.now() returns a timestamp starting from the life cycle of the current page, so unless you need to "stop" time, there is no need to "emulate" or modify it and the snippet I posted should work.

Could you please share an example of test so we can figure together how to make your scenarios work, and bring clear new scenarios/use cases to the Playwright team so they know exactly what the community needs help with.

p01 avatar Apr 14 '22 11:04 p01

@p01 all of our visual tests need a "fixed time" since the Date.now() method is used on all pages.

https://github.com/nasa/openmct/blob/4b7bcf9c89800dd6b3ddd4c23a6a5ba8af7d64b1/e2e/tests/visual/default.spec.js#L50

https://percy.io/b2e34b17/openmct/builds/17336359/unchanged/973207390?browser=safari&viewLayout=overlay&viewMode=new&width=2000

Screen Shot 2022-04-14 at 12 45 30 PM

unlikelyzero avatar Apr 14 '22 19:04 unlikelyzero

ok now this is going to be a bit complex 😅 i don't have a simple code example but i guess i can explain better what i want to test:

let's say i have a web app that opens a hint text after a short delay after you hover over a specific element. that hint text fires a request before showing the result. i have code that measures exactly how long it took between the hover event and the result being printed in the hint text. i have performance.now() (or performance.measure, doesn't really matter) to measure this time, and i have the setTimeout and/or Date.now that delays showing the hint text.

now, in my test, i want to make sure that the time my measurement mechanism captured matches the time this whole operation took. if i emulate only the Date object but not the performance object, these values differ a lot. if i don't emulate the date object, the test takes a long time since the app not only waits for the response (which i would also mock to increase test execution performance), but also for the "show hint text"-delay.

this is an example i came up with just now, nothing from our real world tests (since those would be even more complex...). in reality i have no control over what exactly my code measures, which means it needs a lot of different defer/async test scenarios to be reliable. the scenario above is just a simple one. imagine an end-to-end shop cart checkout scenario where i want to test my measuring code... there's a lot of deferred and async code involved (depending on how the shop is implemented, of course)

DerGernTod avatar Apr 15 '22 07:04 DerGernTod

Hi, if iam using the code above, i get a problem with ts bcs it is telling me, that window.__clock is not a propertie of window. Any solution for this problem?

aw492267 avatar Jun 06 '22 15:06 aw492267

Hi, if iam using the code above, i get a problem with ts bcs it is telling me, that window.__clock is not a propertie of window. Any solution for this problem?

@aw492267 if you want to extend existing interfaces/types in TypeScript, you have to do something called "Module Augmentation", see typescriptlang.org/docs/handbook/declaration-merging.html#module-augmentation.

I have put this block of code into my code base:

import * as sinon from 'sinon';

declare global {
  interface Window {
    __clock: sinon.SinonFakeTimers;
  }
}

pkerschbaum avatar Jun 06 '22 18:06 pkerschbaum

Would anyone have any idea why https://github.com/microsoft/playwright/issues/6347#issuecomment-965887758 approach wouldn't work with component testing?

Replacing @playwright/test with @playwright/experimental-ct-react, addInitScript is executed but sinon is not present. Is there some limitation with addInitScript for components testing? I assume component testing is interfering with the browser context and the added script but I don't really know and I cannot find any clues.

sfuet3r avatar Jun 09 '22 20:06 sfuet3r

I think this relates to how the context is defined in the ComponentFixtures (mount).

So for component testing I found a simpler solution by adding sinon.js to the playwright/index.ts:

// playwright/index.ts
import sinon from 'sinon'
window.sinon = sinon

Then:

import { test, expect } from '@playwright/experimental-ct-react'

test('fake timer with sinon', async ({ page }) => {
  await page.evaluate(() => (window.__clock = window.sinon.useFakeTimers()))
  // Implement a small time on the page
  await page.setContent(`
    <h1>UTC Time: <x-time></x-time></h0>
    <script>
      const time = document.querySelector('x-time');
      (function renderLoop() {
        const date = new Date();
        time.textContent = [date.getUTCHours(), date.getUTCMinutes(), date.getUTCSeconds()]
          .map(number => String(number).padStart(2, '0'))
          .join(':');
        setTimeout(renderLoop, 1000);
      })();
    </script>
  `)

  await expect(page.locator('x-time')).toHaveText('00:00:00')
  await page.evaluate(() => window.__clock.tick(2000))
  await expect(page.locator('x-time')).toHaveText('00:00:02')
})

Or with a mounted component:

import { test, expect } from '@playwright/experimental-ct-react'
import TestTimer from './TestTimer'

test('fake timer with sinon', async ({ page, mount }) => {
  await page.evaluate(() => (window.__clock = window.sinon.useFakeTimers()))
  const component = await mount(<TestTimer />)

  await expect(component).toHaveText('00:00:00')
  await page.evaluate(() => window.__clock.tick(2000))
  await expect(component).toHaveText('00:00:02')
})

sfuet3r avatar Jun 09 '22 21:06 sfuet3r

Here's a very simple and robust solution to set the Time/Date in your tests:

// Pick the new/fake "now" for you test pages.
const fakeNow = new Date("March 14 2042 13:37:11").valueOf();

// Update the Date accordingly in your test pages
await page.addInitScript(`{
  // Extend Date constructor to default to fakeNow
  Date = class extends Date {
    constructor(...args) {
      if (args.length === 0) {
        super(${fakeNow});
      } else {
        super(...args);
      }
    }
  }
  // Override Date.now() to start from fakeNow
  const __DateNowOffset = ${fakeNow} - Date.now();
  const __DateNow = Date.now;
  Date.now = () => __DateNow() + __DateNowOffset;
}`);

That's all! No need for a library or to dig into your node_modules folder to inject into the pages.

Hope that helps,

Hi, I combine this fakeNow with emulate timeZone and set it to US/Pacific for example.

const context = await browser.newContext({
        timezoneId: 'US/Pacific'
    });

const fakeNow = new Date("March 1 2022 13:37:11").valueOf();

The new Date() on browser become

Mon Feb 28 2022 22:37:11 GMT-0800 (Pacific Standard Time)

Is that correct?

sonyarianto avatar Jun 26 '22 06:06 sonyarianto

Here's a very simple and robust solution to set the Time/Date in your tests:

// Pick the new/fake "now" for you test pages.
const fakeNow = new Date("March 14 2042 13:37:11").valueOf();

// Update the Date accordingly in your test pages
await page.addInitScript(`{
  // Extend Date constructor to default to fakeNow
  Date = class extends Date {
    constructor(...args) {
      if (args.length === 0) {
        super(${fakeNow});
      } else {
        super(...args);
      }
    }
  }
  // Override Date.now() to start from fakeNow
  const __DateNowOffset = ${fakeNow} - Date.now();
  const __DateNow = Date.now;
  Date.now = () => __DateNow() + __DateNowOffset;
}`);

That's all! No need for a library or to dig into your node_modules folder to inject into the pages. Hope that helps,

Hi, I combine this fakeNow with emulate timeZone and set it to US/Pacific for example.

const context = await browser.newContext({
        timezoneId: 'US/Pacific'
    });

const fakeNow = new Date("March 1 2022 13:37:11").valueOf();

The new Date() on browser become

Mon Feb 28 2022 22:37:11 GMT-0800 (Pacific Standard Time)

Is that correct?

:) Ha! Nice I didn't think about time zone offset. But that should be easy to fix, by adding Z-07:00 or similar when getting the fakeNow.

e.g.:

const fakeNow = new Date("March 1 2022 13:37:11Z-07:00").valueOf();

Another way could be to get the timeZomeOffset rather than "hardcoding" it, e.g.:

// Get fakeNow from UTC to extract the timeZone offset used in the test
const fakeNowDateTime = "March 1 2022 13:37:11";
const fakeNowFromUTC = new Date(fakeNowDateTime);
const timeZomeOffset = fakeNowFromUTC.getTimeZoneOffset();
const timeZoneOffsetHours = `${Math.abs(Math.floor(timeZomeOffset  / 60))}`;
const timeZoneOffsetMinutes = `${Math.abs(timeZomeOffset  % 30)}`;
const timeZoneOffsetText = `${timeZomeOffset < 0 ? "-" : "+"}${timeZoneOffsetHours.paddStart(2,"0")}:${timeZoneOffsetMinutes.padStart(2,"0")}`; 

// Get fakeNow from the test timeZone
const fakeNow = new Date(`${fakeNowDateTime}Z${timeZoneOffsetText}`).valueOf();

⚠️ I didn't get the chance to try the code above, but the general idea should work.

Hope that helps,

p01 avatar Jun 27 '22 09:06 p01

Here's a very simple and robust solution to set the Time/Date in your tests:

// Pick the new/fake "now" for you test pages.
const fakeNow = new Date("March 14 2042 13:37:11").valueOf();

// Update the Date accordingly in your test pages
await page.addInitScript(`{
  // Extend Date constructor to default to fakeNow
  Date = class extends Date {
    constructor(...args) {
      if (args.length === 0) {
        super(${fakeNow});
      } else {
        super(...args);
      }
    }
  }
  // Override Date.now() to start from fakeNow
  const __DateNowOffset = ${fakeNow} - Date.now();
  const __DateNow = Date.now;
  Date.now = () => __DateNow() + __DateNowOffset;
}`);

That's all! No need for a library or to dig into your node_modules folder to inject into the pages. Hope that helps,

Hi, I combine this fakeNow with emulate timeZone and set it to US/Pacific for example.

const context = await browser.newContext({
        timezoneId: 'US/Pacific'
    });

const fakeNow = new Date("March 1 2022 13:37:11").valueOf();

The new Date() on browser become

Mon Feb 28 2022 22:37:11 GMT-0800 (Pacific Standard Time)

Is that correct?

:) Ha! Nice I didn't think about time zone offset. But that should be easy to fix, by adding Z-07:00 or similar when getting the fakeNow.

e.g.:

const fakeNow = new Date("March 1 2022 13:37:11Z-07:00").valueOf();

Another way could be to get the timeZomeOffset rather than "hardcoding" it, e.g.:

// Get fakeNow from UTC to extract the timeZone offset used in the test
const fakeNowDateTime = "March 1 2022 13:37:11";
const fakeNowFromUTC = new Date(fakeNowDateTime);
const timeZomeOffset = fakeNowFromUTC.getTimeZoneOffset();
const timeZoneOffsetHours = `${Math.abs(Math.floor(timeZomeOffset  / 60))}`;
const timeZoneOffsetMinutes = `${Math.abs(timeZomeOffset  % 30)}`;
const timeZoneOffsetText = `${timeZomeOffset < 0 ? "-" : "+"}${timeZoneOffsetHours.paddStart(2,"0")}:${timeZoneOffsetMinutes.padStart(2,"0")}`; 

// Get fakeNow from the test timeZone
const fakeNow = new Date(`${fakeNowDateTime}Z${timeZoneOffsetText}`).valueOf();

⚠️ I didn't get the chance to try the code above, but the general idea should work.

Hope that helps,

Thank you @p01 I will try that later :)

sonyarianto avatar Jul 03 '22 08:07 sonyarianto

If anyone is still looking for a solution to mock date we found a simple way to do it with before mount method in index.ts.

import { beforeMount } from "@playwright/experimental-ct-react/hooks"

beforeMount(async ({ hooksConfig }) => {
  if (hooksConfig && hooksConfig.mockDateNow) {
    Date.now = () => hooksConfig.mockDateNow as number
  }
})

Then we can just pass any date to the hookConfig in the mount of the test

test.describe('component test', () => {
  test('renders a component', async ({ mount }) => {
      const mokedTime = new Date(2022, 1, 1).getTime()
      
      const component = await mount(<TestComponent />, {
      hooksConfig: { mockDateNow: mokedTime },
    })

    await verifyScreenshot(component, 'default')
  })

Simonlfr avatar Dec 08 '22 14:12 Simonlfr

I wrote something like this:

import { test as base } from '@playwright/experimental-ct-vue';
import '@playwright/test';
import sinon from 'sinon';

declare global {
  interface Window {
    __clock: sinon.SinonFakeTimers;
  }
}

interface SinonFakeTimersWrapper {
  tick(time: string | number): Promise<number>;
  tickAsync(time: string | number): Promise<number>;
  next(): Promise<number>;
  nextAsync(): Promise<number>;
  runAll(): Promise<number>;
  runAllAsync(): Promise<number>;
}

export const test = base.extend<{ clock: SinonFakeTimersWrapper }>({
  clock: async ({ page }, use) => {
    await page.evaluate(() => {
      window.__clock = window.sinon.useFakeTimers();
    });

    await use({
      tick: async (time) => {
        return page.evaluate((time) => {
          return window.__clock.tick(time);
        }, time);
      },
      tickAsync: async (time: string | number): Promise<number> => {
        return page.evaluate((time) => {
          return window.__clock.tickAsync(time);
        }, time);
      },
      next: async (): Promise<number> => {
        return page.evaluate(() => {
          return window.__clock.next();
        });
      },
      nextAsync: async (): Promise<number> => {
        return page.evaluate(() => {
          return window.__clock.nextAsync();
        });
      },
      runAll: async (): Promise<number> => {
        return page.evaluate(() => {
          return window.__clock.runAll();
        });
      },
      runAllAsync: async (): Promise<number> => {
        return page.evaluate(() => {
          return window.__clock.runAllAsync();
        });
      },
    });
  },
});

test.afterEach(() => {
  sinon.restore();
});

export { expect } from '@playwright/experimental-ct-vue';

And adding to playwright/index.ts for component testing:

import sinon from 'sinon'
window.sinon = sinon

You then have a clock fixture for fake timers. (The sinon.restore is because I'm also using Sinon's spy/stub in places). Still missing better expect assertions for sinon, need some kind of a plugin to extend expect with.

segevfiner avatar Apr 25 '23 10:04 segevfiner

It would be nice to have this feature. This helps with tests that have toast messages. We could speed up the clock to get the toasts to disappear faster

blessanm86 avatar Jun 17 '23 16:06 blessanm86

Could we have a built-in mock timers like jest/vite have?

  • https://jestjs.io/docs/jest-object#jestsetsystemtimenow-number--date
  • https://vitest.dev/api/vi.html#vi-setsystemtime

raulfdm avatar Oct 16 '23 12:10 raulfdm

This is a fundamental feature, can't see why not still fixed since 2021. It is fundamental to be able to make tests independent from time.

dan-the-dev avatar Oct 17 '23 13:10 dan-the-dev