`runTests` tool fails to run tests in specific file on SSH Remote

[Aesonus]

The runTests tool cannot find any pytest tests in file on remote workspace. The tool works correctly when running locally. This is similar to #266077, but my files are named using the pytest paradigm, test_*.py.

If I ask it to run all tests using the runTests tool, they all run without issue.

• Copilot Chat Extension Version: 0.32.4
• Version: 1.105.1 (user setup)
• Commit: 7d842fb85a0275a4a8e4d7e040d2625abbf7f084
• Date: 2025-10-14T22:33:36.618Z
• Electron: 37.6.0
• ElectronBuildId: 12502201
• Chromium: 138.0.7204.251
• Node.js: 22.19.0
• V8: 13.8.258.32-electron.0
• OS: Windows_NT x64 10.0.26100
• Feature (e.g. agent/edit/ask mode): agent
• Selected model (e.g. GPT 4.1, Claude 3.7 Sonnet): Claude Sonnet 4.5, GPT-5 mini, and possibly more.
• Logs:

Tool Call - toolu_vrtx_01AZEjotVL1aPNVjbV45VeHn__vscode-1762282851932

Request

id   : toolu_vrtx_01AZEjotVL1aPNVjbV45VeHn__vscode-1762282851932
tool : runTests
args : {
  "files": [
    "/home/user/projects/copilot-runtests/tests/test_assert.py"
  ]
}

Response

No tests found in the files. Ensure the correct absolute paths are passed to the tool.

The file contents are:

def test_asserts():
    assert 1 + 1 == 2

Steps to Reproduce:

1. Open a python project using pytest on SSH remote
2. Ask copilot to run tests in a specific file - it does not appear to matter which model is selected

[segfly]

Been having the same issue for awhile (months) and also discovered the runTests workaround. I am also running SSH into a devcontainer, not realizing that the tool would work correctly for a single test locally but not over SSH. Until this issue is resolved, I've decided to just disable to tool and provide instructions to copilot to use the terminal to run tests.

[eleanorjboyd]

to confirm you aren't able to run the tests from the "test panel" either right? Can you send the output from the python output channel?

[segfly]

to confirm you aren't able to run the tests from the "test panel" either right?

@eleanorjboyd In my case, I can run the tests from the test panel without any problems.

The issue only occurs when Copilot tries to call the runTests tool in a remote dev container environment.

For example, given the test:

def test_sometest():
    pass

And prompting any LLM model with:

Use the run test tool to run test_sometest. Report to me the exact content you send to the tool and the result. Do not take any other actions.

The LLM will always fail to find the test, as the tool seems to report to the LLM that it does not exist:

LLM Response:

Content sent to the tool:

{
  "files": ["/workspaces/issue-275268/tests/test_example.py"],
  "testNames": ["test_sometest"],
  "mode": "run"
}

Result: No tests found in the files. Ensure the correct absolute paths are passed to the tool.

However, asking the LLM to run all tests works just fine:

Try again using run all tests. Report to me the exact content you send to the tool and the result. Do not take any other actions.

LLM Response:

Content sent to the tool:

{
  "mode": "run"
}

Result: All tests passed (1 test, 0 failed).

[Aesonus]

I can confirm that I am able to run tests in the testing panel without issue. It is only the AI agent that is unable to run tests.

EDIT:

@segfly's logs look like mine do. I think it is indeed the same issue I am having.

---

From: Nicholas Pace @.> Sent: Tuesday, December 9, 2025 6:22:16 PM To: microsoft/vscode @.> Cc: Cory Laughlin @.>; Author @.> Subject: Re: [microsoft/vscode] runTests tool fails to run tests in specific file on SSH Remote (Issue microsoft/vscode#275268)

[https://avatars.githubusercontent.com/u/3300476?s=20&v=4]segfly left a comment (microsoft/vscode#275268)https://github.com/microsoft/vscode/issues/275268#issuecomment-3634830374

to confirm you aren't able to run the tests from the "test panel" either right?

@eleanorjboydhttps://github.com/eleanorjboyd In my case, I can run the tests from the test panel without any problems.

The issue only occurs when Copilot tries to call the runTests tool in a remote dev container environment.

For example, given the test:

def test_sometest(): pass

And prompting any LLM model with:

Use the run test tool to run test_sometest. Report to me the exact content you send to the tool and the result. Do not take any other actions.

The LLM will always fail to find the test, as the tool seems to report to the LLM that it does not exist:

LLM Response:

Content sent to the tool:

{ "files": ["/workspaces/issue-275268/tests/test_example.py"], "testNames": ["test_sometest"], "mode": "run" }

Result: No tests found in the files. Ensure the correct absolute paths are passed to the tool.

However, asking the LLM to run all tests works just fine:

Try again using run all tests. Report to me the exact content you send to the tool and the result. Do not take any other actions.

LLM Response:

Content sent to the tool:

{ "mode": "run" }

Result: All tests passed (1 test, 0 failed).

— Reply to this email directly, view it on GitHubhttps://github.com/microsoft/vscode/issues/275268#issuecomment-3634830374, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AF4AMK5KK5EBTJ4PSOAHWWD4A5RTRAVCNFSM6AAAAACLEIZWK2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTMMZUHAZTAMZXGQ. You are receiving this because you authored the thread.Message ID: @.***>

[eleanorjboyd]

ok this is great info thank you! This helps me narrow down where the issue is and I will investigate

[Aesonus]

I did a little more testing and asked the LLM to run tests with a specific name. It appears that the tool call with testNames argument is working:

Tool Call

id   : tooluse_8BLNdjDmTZWO-6QaIs5NiQ__vscode-1765462001103
tool : runTests
args : {
  "testNames": [
    "my_test"
  ]
}

Response

<summary passed=1 failed=0 />

[segfly]

@Aesonus Intersting. In my results above, the LLM did use testNames but also provided files and it failed. Your successful result showed it omitted files. I wonder if the files argument is causing the tool to fail when used over SSH.

I don't know the details, but I've seen with the copilot extension, there seems to be at least some settings stored on the local host. It makes me think that some tools may also be getting local information, so perhaps the tool is getting information for the files path on the local host instead of the remote host - but the testNames is coming from the test discovery api, and so it can find the tests correctly when using just the name.

Just wild guesses though... I don't have time at the moment to dig into what's going on when the tool is called.