[Bug]: playwright tests do not show in test tab in vscode, no errors, cli works

[jonyo]

Version

1.43.1

Steps to reproduce

1. I suspect, either have a playwright.config.ts in a sub-folder. Or have some possible problem with the config file. Not sure the exact way to reproduce since the problem seems to be with reporting some problem, I don't know what that problem is...
2. In vscode, open the testing tab.
3. It shows no extensions loaded. (playwright extension IS installed and at latest version) Hit reload it tries to load playwright for a second then goes away.

Opening up developer tools (help > toggle developer tools) I see this:

  log.ts:439   ERR Cannot read properties of undefined (reading 'line'): TypeError: Cannot read properties of undefined (reading 'line')
    at $e._updateDiagnostics (/Users/ata224/.vscode/extensions/ms-playwright.playwright-1.0.22/out/extension.js:97:1287)
    at $e._reportConfigErrorsToUser (/Users/ata224/.vscode/extensions/ms-playwright.playwright-1.0.22/out/extension.js:93:6859)
    at $e._rebuildModel (/Users/ata224/.vscode/extensions/ms-playwright.playwright-1.0.22/out/extension.js:93:6794)
    at r.$refreshTests (/Applications/Visual Studio Code.app/Contents/Resources/app/out/vs/workbench/api/node/extensionHostProcess.js:154:8490)

Judging from context in the trace dump, it sounds like it is trying to report a problem with the config file, but that the error data is undefined for some reason.

Expected behavior

I expect to see an error saying why playwright could not load. Or that some problem happened. Any error at all (outside of the developer window where no one looks).

Actual behavior

It quietly acts like the extension is not there, no errors or anything.

Additional context

So even if this is caused by config error, it should have graceful failure to tell me that. Thinking back, I don't know that I've ever seen an error from playwright extension, it just plain worked or did not work. For example, we also use vitest (extension disabled to eliminate some bug with interaction) - if we have it set up wrong somehow, it tells you in an error as soon as you open the test tab. I have not seen that from playwright.

Also note that I had it working with this project probably a few weeks ago (last time I tried using the extension). Since then we've updated playwright versions, updated node versions on the project, not sure if that is related.

Note: I had to cd into tests/ folder to run npx envinfo --preset playwright it did not find it in base folder.

Environment

System:
    OS: macOS 14.4.1
    CPU: (16) x64 Intel(R) Core(TM) i9-9880H CPU @ 2.30GHz
    Memory: 1.41 GB / 32.00 GB
  Binaries:
    Node: 18.20.1 - ~/.nvm/versions/node/v18.20.1/bin/node
    Yarn: 1.22.22 - /usr/local/bin/yarn
    npm: 10.5.0 - ~/.nvm/versions/node/v18.20.1/bin/npm
    pnpm: 9.0.1 - /usr/local/bin/pnpm
  IDEs:
    VSCode: 1.88.1 - /Applications/Visual Studio Code.app/Contents/Resources/app/bin/code
  Languages:
    Bash: 3.2.57 - /bin/bash
  npmPackages:
    @playwright/test: ^1.43.1 => 1.43.1 
    playwright-bdd: ^6.2.0 => 6.2.0

[yury-s]

Can you share the project where the error occurs?

[yury-s]

Since then we've updated playwright versions, updated node versions on the project, not sure if that is related.

Most likely it is, would be helpful if you could figure out what update triggered the problem.

[jonyo]

Since then we've updated playwright versions, updated node versions on the project, not sure if that is related.

Most likely it is, would be helpful if you could figure out what update triggered the problem.

OK I'll give it a try as I have time. This is on a private project for work so can't share the whole thing.

FYI I said "not sure if related" because in the past I've seen it happen as well... When it doesn't work, I start trying to figure out why, then at some point it starts working, I have not tracked down what I did (or if it was even me) that fixed it. As I said though it never gives me errors when it does not work so something in the error handling may not handle some states well...

[yury-s]

As I said though it never gives me errors when it does not work so something in the error handling may not handle some states well...

We need an example where such error occurs, so that we could reproduce it locally. Based on this description we unfortunately cannot take any action even if there is a bug.

[jonyo]

As I said though it never gives me errors when it does not work so something in the error handling may not handle some states well...

We need an example where such error occurs, so that we could reproduce it locally. Based on this description we unfortunately cannot take any action even if there is a bug.

Fair enough, I'll see if I can get a local version of the extension running so I can debug it further. I tried with the one built in but the minified code is harder to follow. Anyways, thanks for your patience so far! 🙂

[yury-s]

There were some fixes recently (not yet released) that could be relevant, but we need a reproduction to be able to confirm.

[jonyo]

There were some fixes recently (not yet released) that could be relevant, but we need a reproduction to be able to confirm.

are they pushed up to a branch I could check out? I'm on main but didn't have any changes when I did a pull.

[jonyo]

I figured it out! And I know how to reproduce!

How to Reproduce

1. In your environment, use NVM to set up your default node to version 20.6.1 - change the alias for default to that version (maybe you are familiar and know where this is going?)
2. Start up vscode. Verify it is using 20.6.1 by looking at "about vscode".
3. Open the test tab. It will not find tests. Also will not give any clues as to why, no errors in the console, output window, debug window... And using very very latest version in main, you don't even see any in the developer debug window. (Maybe the latest version of main did fix something because that other error did go away, but things still didn't work)

Story time! feel free to skip down if you want, I won't be sad. Not too sad anyways... 🥺

So there I was, tracking down where it loaded files in the extension, and tracked it to:

https://github.com/microsoft/playwright-vscode/blob/bc096a5c26e1bcec1bbb802c8ff362bb11e1939a/src/playwrightTestCLI.ts#L50-L56

(weird it doesn't show preview, thought github showed code preview even across repos? oh well)...

Anyways, I added a console log on output there and it had

{
"error": "TypeError [ERR_INVALID_ARG_TYPE]: The \"parentURL\" argument must be of type string or an instance of URL. Received an instance of Object"
}

That's odd, I had to console log it, I guess the main extension just quietly eats the error instead of trying to report it. Maybe it assumes "any error" or maybe anything not expected, means no tests were found or something? I didn't bother following the code past that to see why though.

Anyways, now I had something to google! And I came across this issue about the exact same error happening with playwright running on the command line. It is closed, with problem being the node version and solution to upgrade node.

So I opened "about vscode", and sure enough, it was using that same buggy node version 20.6.1. After more googling, I found vscode uses whatever node it finds in the path. So in nvm I updated the default to use the latest lts/iron (currently 20.12.2), then quit VSCode and re-started, and 🎉 it works!!

Suggestion of Solution (bug/feature fix request?)

Why anything at all? This is very hard to troubleshoot. There is 0 clue at all, no errors or anything. And playwright runs just fine on the command line because I'm using nvm and a .nvmrc file that makes it use a specific version of node when I'm in this folder. But the problem was internally vscode just uses whatever node it finds in the path.

Sure normally if it doesn't work on the command line people would get that working before trying to get the extension working. But in this case, like I said it did work. And I would doubt that I am the only one with this setup, this is a common practice to pin a project to a specific node version by using .nvmrc. So I'm saying this is likely a problem for others as well, but perhaps like me they just run it on the command line when the extension is being finicky. (and miss out on all the awesome extra stuff the extension can do for you!)

Anyways that's my spiel, sorry if you were already convinced, you hopefully skipped down to my suggestion:

Suggestion: When there is some error / unexpected result from the command line, echo that out, even if it is sent to somewhere like the output window. If that were done I would have seen it, googled the error, and came to that issue I pointed to and figured out how to fix. As it is, nothing gets to somewhere visible to the end user, it just quietly eats the error and says it can't find any tests.

I do see how it does the this._log(... "for tests", maybe it could do something like that for the result as well but use whatever mechanism you use to get it showing in the "output" window... For some reason playwright doesn't show up in the dropdown in the "output" window. Maybe it should start so it can print out sometimes useful info like this.

[wolispace]

Side note, I was getting the same "No tests have been found in this workspace yet."

Turns out I had accidentally un-ticked chromium (firefox and webkit) from the PROJECTS section under PLAYWRIGHT in the nav.

So with no browser engine selected we also get this mysterious error. Maybe the error could suggest this as a possible issue when no PROJECTS are selected?

[gnesher]

Side note, I was getting the same "No tests have been found in this workspace yet."

Turns out I had accidentally un-ticked chromium (firefox and webkit) from the PROJECTS section under PLAYWRIGHT in the nav.

this has saved my day - someone should really show a warning when this is the case, its super confusing

[sumit6b]

I am not seeing anything under projects section sadly.

[DanRQA]

I experience the problem where test explorer can't find my tests from time to time, but I haven't been able to figure out why.

I can run playwright in ui mode - no warning that browsers aren't installed and I've reloaded vs code several times, can also run them in cli passing a specific project such as chromium.

At least for me its an issue in ubuntu because if I run vs code in windows the tests do show up. As I mentioned before even on ubuntu sometimes it works for me other times it doesn't 🤷🏻‍♂️

node version v20.6.0

Restarted my workstation and of course now it works for me

[sumit6b]

Similar to @DanielRamos84. I made sure my DEFAULT node version is 18+ and VSCode is update to make it work.

[dragonai]

Side note, I was getting the same "No tests have been found in this workspace yet."

Turns out I had accidentally un-ticked chromium (firefox and webkit) from the PROJECTS section under PLAYWRIGHT in the nav.

So with no browser engine selected we also get this mysterious error. Maybe the error could suggest this as a possible issue when no PROJECTS are selected?

Holy smokes, this completely solved it for me too. What a trek across the internet just to discover one problematically unchecked checkbox. Thank you @wolispace! 🫶🏽

[obxpete]

Same symptoms. Solution: disconnect from VPN. ?????!!!!!

[docwarems]

@jonyo

I figured it out! And I know how to reproduce! ... Anyways, I added a console log on output there and it had

Hi, I've been struggling with this issue too for quite a while now. Although, I have at at least one VSC project where I actually see my Playwright tests. So I think the problem for me is not the node version that runs VSC.

Nevertheless, I found your idea promising to debug the extension to get my cause. I've never done any VSC extension development nor debugging, however. Could you be a little more verbose what you did to get the added log entry?

What I did

• clone the VSC Playwright Extension from Github

• add a log entry here

• npm run build

• vsce package (of course I had to install vsce before) -> generates a .vsix file

• Extensions view / ... / install from VSIX... Afterwards the Playwright Extension had a reload button so I think somethink happened (although the version shown is still the original one)

• Command: Developer: Open Logs folder

• but now I don't know where to look for my log message

BTW, in my project which has the problem, I don't see any browsers listed here under projects While in the working project it looks like this I have no idea where the difference comes from. I have of course installed browsers with command "Playwright install"

Thanks for any hints

[docwarems]

I am not seeing anything under projects section sadly.

@sumit6b I think if there's only Playwright project (actually the Playwright config) in the VSC project, it will not show up there. I have different VSC projects and where I have more than one Playwright configs, this option shows up to choose a Playwright project from.

[Horax2000]

I had to pull the project core into a backup, delete the entire thing, and install Playwright on the folder, then test buttons worked as usual, and then put the backup

[monkpit]

If anyone finds this issue from searching, the issue for me was the node version.

In the OUTER directory (top-level project directory, not where my playwright project is contained), my default node version was being used (in my case, node 12). So, opening code . in that directory was using that node version, and the VS Code extension was NOT working - playwright extension could not find any tests, and complained Internal error: unable to connect to the test server.

In my INNER playwright directory, node 20 was being used. If I opened code . in that directory, nvm was auto-switching the node version due to my project having .nvmrc or .node-version file present. The VS Code extension was working.

If I switched to node 20 in the OUTER directory (like nvm use 20), then opened code (code .), the extension finally worked from the top-level directory.

cc @yury-s since I browsed several threads about this issue and saw you in them all :)

[sumit6b]

Few days back, I started getting this issue again. So I debugged the extension itself and figured that i had multiple nodes installed. Like one set of node versions by nvm and another was by manual 'pkg' file which i might have installed before i switched to nvm.

This extension finds node version by this logic:

export async function findNode(vscode: vscodeTypes.VSCode, cwd: string): Promise<string> {
  if (pathToNodeJS)
    return pathToNodeJS;

  // Stage 1: Try to find Node.js via process.env.PATH
  let node = await which('node').catch(e => undefined);
  // Stage 2: When extension host boots, it does not have the right env set, so we might need to wait.
  for (let i = 0; i < 5 && !node; ++i) {
    await new Promise(f => setTimeout(f, 200));
    node = await which('node').catch(e => undefined);
  }
  // Stage 3: If we still haven't found Node.js, try to find it via a subprocess.
  // This evaluates shell rc/profile files and makes nvm work.
  node ??= await findNodeViaShell(vscode, cwd);
  if (!node)
    throw new NodeJSNotFoundError(`Unable to find 'node' executable.\nMake sure to have Node.js installed and available in your PATH.\nCurrent PATH: '${process.env.PATH}'.`);
  pathToNodeJS = node;
  return node;
}

So we just need to ensure that it finds the right node version.

[scottbedard]

@monkpit you saved me!

Opening VSCode with code . failed, but moving up a directory and using code ./my-project worked 🙌

[rostislav-simonik-plc]

There could be also other reason, in our case we had type issue that didn't appear during type check.

playwright.config.ts

export default defineConfig({
  ...
  workers: process.env.TEST_WORKERS ?? 2,
  ...
}

.vscode/settings.json

{
  "playwright.env": {
    "TEST_WORKERS": 1
  }
}

resulted in no tests

No issue reported anywhere.

Properly parsing the string value, fixed the issue playwright.config.ts

export default defineConfig({
  ...
  workers: process.env.TEST_WORKERS ? parseInt(process.env.TEST_WORKERS) : 2,
  ...
}

It might be a good idea to properly propagate the type check error.

[mxschmitt]

I was reading through the last few comments and think we addressed most of the issues. To summarise:

Environment variables are required during your tests:

There we recommended to use .env files and load it inside playwright.config.ts. This will automatically load the .env files during VSCode, UI Mode and normal CLI mode.

nvm via zsh:

We addressed this in https://github.com/microsoft/playwright-vscode/pull/573 by @CHC383 - thank you for that! This should just work.

Unselected projects:

Make sure to enable all wanted projects inside the Testing Tab on the left under Playwright. Previously this was hard to spot, we now mention the disabled projects as test items to make it easier to spot.

Other issues:

Make sure to use latest Node.js e.g. 18.X, 20.X, 22.X, official latest extension and make sure your tests run via the CLI: npx playwright test. If you still encounter any issues, please file a separate issue and we'll triage it from there. Thank you!