[Bug]: UI stuck on "Loading" when the project is huge

[aczekajski]

Version

1.54.2

Steps to reproduce

1. Clone repo https://github.com/aczekajski/pw-huge-project-bug-repro
2. Run npm i
3. Run node genfiles.js (it takes some time to generate files but I didn't want to commit 300 000 shim files ^^' lmk if you prefer me to actually commit them)
4. Run npx playwright test --list - takes only few seconds and works fine
5. Run npx playwright test - same as above
6. Run npx playwright test ./tests/example.spec.ts - same as above
7. Run npx playwright test --ui - window opens and is stuck on "Loading"

Expected behavior

The tests discovery phase should take the same amount of time in UI mode as it takes without it.

Actual behavior

In repo with ~300 000 files (even if none of them match the testMatch pattern), the UI seems to be stuck forever on "Loading". I waited 10 minutes and it was still stuck there.

Additional context

If I set testDir config option to ./tests then it works fine - that's how I discovered it's about the sheer number of files in the project.

In my real project over 200 000 of all the files come from node_modules. but adding /node_modules/ to testIgnore does not help as well.

Reducing the number of files in this reproduction repo to 50 000, causes the UI to show the tests list after ~1 minute, which is still way too long, considering it takes 5 seconds without UI.

Environment

System:
    OS: Windows 10 10.0.19045
    CPU: (12) x64 13th Gen Intel(R) Core(TM) i7-1365U
    Memory: 5.40 GB / 31.67 GB
  Binaries:
    Node: 22.15.1 - C:\nvm4w\nodejs\node.EXE
    npm: 10.9.2 - C:\nvm4w\nodejs\npm.CMD
  IDEs:
    VSCode: 1.103.0 - C:\Users\arek.czekajski\AppData\Local\Programs\Microsoft VS Code\bin\code.CMD
  Languages:
    Bash: 5.2.37 - C:\Program Files\Git\usr\bin\bash.EXE
  npmPackages:
    @playwright/test: ^1.54.2 => 1.54.2

[Marcin3]

How it works when you select only 1 project in ui mode? How many project do you have?

[aczekajski]

As can be seen here: https://github.com/aczekajski/pw-huge-project-bug-repro/blob/master/playwright.config.ts#L28 there is only one "PW project". I can run npx playwright test --project chromium --ui, I can run npx playwright test ./tests/example.spec.ts --ui, doesn't matter. As long as the folder where playwright.config.ts is placed, has collectively huge amount of files, the UI is stuck on "Loading...".

As for the filters in UI itself - it doesn't even see the projects available and typing anything into filter field does nothing, it's still stuck:

[mxschmitt]

I can reproduce, on Windows its significantly slower than on macOS. On Windows (DevBox) it was ~60s for me vs. npx playwright test --list which was 3s.

[agg23]

The issue is chokidar creating separate watcher calls for every single file under the project directory. This doesn't happen with --list as --list doesn't need to be responsive to test file changes, but the UI Mode needs to know when a test file is added/removed, in addition to explicit watch mode via the button.

const chokidar = require("chokidar");
console.log("Starting", performance.now());
chokidar.watch('.', {ignoreInitial: true}).on("ready", () => console.log("Ready", performance.now()));

Testing with the 300k dummy files on a Windows machine:

Starting 86.3029
Ready 51039.7182

So in that scenario 50s is being spent parsing through all of those files. I'm not sure what can be done to improve this, given we need to be able to respond to new/deleted test files.

For fun I allowed the watchers to be set up asynchronously, which does speed up UI Mode startup massively (though still not as fast as --list), but actually interacting with the tests (i.e. running one) remains paused until the watchers are set up.

[aczekajski]

Idea 1: Can't chokidar be forced to ignore some folders? At least ignoring node_modules sounds like a good choice. It would still be slow on a big repo tho. Maybe there's some more optimisations that can be achieved with chokidar or other similar libraries.

Idea 2: Can't the watchers setup just run in the background? It doesn't feel necessary for it to be a blocker for running the tests.

Idea 3: Option to disable watchers in UI mode entirely? Sounds easiest to implement but would resolve problem for a huge repos like mine which is probably also not very common case.

[agg23]

1. node_modules is already ignored. As your scenario suggests, the time is entirely spent on the many many files you have in your actual project. Are there files you would want to ignore (and would cut down most of the total files to evaluate)?

2. I mentioned:

For fun I allowed the watchers to be set up asynchronously, which does speed up UI Mode startup massively (though still not as fast as --list), but actually interacting with the tests (i.e. running one) remains paused until the watchers are set up.

We could work on removing the lock that is preventing tests from running, but it may not work the way you want. It's possible that with Windows' poor disk performance and the long total duration of the slowdown that tests would perform erratically/not consistently, which means you'd get unnecessary failures.

3. Removing watching in UI Mode completely breaks the entire experience. New test files wouldn't show up, new/modified test cases within a file wouldn't change, and tests wouldn't get removed if deleted.

[aczekajski]

Well, I'd rather have an option to disable tests watch and have to click "reload" button after I add the test files than have to wait few minutes before the UI mode launches since it makes it completely unusable 😅 In my actual project, the UI did not load even after waiting 10 minutes, while clicking "refresh" takes only few seconds to reload the tests.

[agg23]

Revisiting this, why is setting testDir not a sufficient solution? If things are slow because too many unnecessary files are being processed, it seems like limiting Playwright to look at just your test directory should be what you want.

[aczekajski]

The thing is that the project has hundreds of components with their visual regression tests and lots of pages with many e2e tests and I really don't want to gather the tests in one folder. Each thing to be tested had its tests placed right beside it and unfortunately it turns out that the deepest common folder between all the places where tests might exist is the project's root folder and I can't really do anything about it 😅 Pains of migrating huge project 😅 Since I have no other options now, I will be doing some workaround with testDir generated dynamically based on env vars and forcing devs to run PW like COMPONENT=path/to/component/you/work/on/now/ npx run playwright --ui. This is far from perfect tho.

I briefly looked at what happens if I use a bare Node's fs.watch on my project root and it seemed to work immediately, as opposed to chokidar which takes ages to setup. I didn't yet have time to dig into its limitations tho.

[dimkin-eu]

Would using tags for tests help? ( but you will need to tag all of them )

[aczekajski]

@dimkin-eu No, the problem happens entire layer earlier: it's in filesystem watch, not in number of test cases/files discovered. No amount of filtering helps, not even passing a specific test file path to the command (which actually should btw, because why watching entire filesystem if user specifically pointed to one/some selected test files).