test-runner
test-runner copied to clipboard
storybookjs
[Bug] ReferenceError: setupPage is not defined / page.evaluate: ReferenceError: __test is not defined
Describe the bug
When I run all tests I almost always get a failure like this:
● Modules / Components / AuthDetails › App Without Auth › test
ReferenceError: setupPage is not defined
90 | if (err.toString().includes('Execution context was destroyed')) {
91 | await jestPlaywright.resetPage();
> 92 | await setupPage(global.page);
| ^
93 | await testFn();
94 | } else {
95 | throw err;
at Object.<anonymous> (../../../../../../private/var/folders/l_/7fz4zgds1cd9tkqc36p0lk0w0000gn/T/47e5eedc4767b37c9cdb92bc7c1c08b6/modules-components-authdetails.test.js:92:11)
Sometimes, this is followed up by another failure like:
● Modules / Components / AuthDetails › Unknown App › test
page.evaluate: ReferenceError: __test is not defined
at eval (eval at evaluate (:178:30), <anonymous>:4:15)
at UtilityScript.evaluate (<anonymous>:180:17)
at UtilityScript.<anonymous> (<anonymous>:1:44)
at testFn (../../../../../../private/var/folders/l_/7fz4zgds1cd9tkqc36p0lk0w0000gn/T/47e5eedc4767b37c9cdb92bc7c1c08b6/modules-components-authdetails.test.js:122:35)
at Object.<anonymous> (../../../../../../private/var/folders/l_/7fz4zgds1cd9tkqc36p0lk0w0000gn/T/47e5eedc4767b37c9cdb92bc7c1c08b6/modules-components-authdetails.test.js:137:15)
It's not always the same test, although it does seem to affect some tests more than others, and all of them so far define .play functions.
When I run the test individually it pretty much always works fine.
Steps to reproduce the behavior
No idea.
Expected behavior
For the test to work as part of the suite.
Screenshots and/or logs
See above console logs.
Environment
-
@storybook/test-runner: v0.2.0 - OS: macOS 12.4 (21F79)
- Node.js version: v14.18.2
- NPM version: 6.14.15
- Browser (if applicable): chromium
- Browser version (if applicable): ¯_(ツ)_/¯
Additional context
Example of a .play() function:
UnknownApp.play = async ({ canvasElement }) => {
const canvas = within(canvasElement);
// Also ensures component is stable before snapshot
expect(
await canvas.findByText('MyAppThatDoesNotExist account')
).toBeVisible();
};
We do use a test-runner.js, but when I rename that the issue is still there.
We also ejected, but when I rename test-runner-jest.config.js the issue is also still there.
When I open the file of a failing test I see that setupPage doesn't get imported, but the same is true for passing tests - so I assume that is available as a global.
await jestPlaywright.resetPage(); < This could be your problem. I was having the same issue, and it ended up being because I was refreshing the page. This error didn't come up when I had only a single story in a storybook file, but once I had more than one I started to see it immediately.
After I removed refreshing the page, the error went away. Resetting may cause a similar issue.
@VickyKoblinski that call does not come from my test, but from the runner itself:
https://github.com/storybookjs/test-runner/blob/bbe74badc900e060f7ee19d1aaa5722d94f4ee63/src/playwright/transformPlaywright.ts#L47
This runs when the browser closes, and the error contains 'Execution context was destroyed'. I have no idea what is causing that.
I'm experiencing this same issue running in an azure pipeline. It's flaky as rerunning the tests will often fix the issue
We've also hit this issue, it would be great to get that PR merged^ - is there anything we can do to assist in getting the fix out?
