Run in terminal command above class name is not running all tests properly

[sanjani-thatch]

Description

Ruby LSP Information

VS Code Version

1.93.1

Ruby LSP Extension Version

0.8.2

Ruby LSP Server Version

0.18.4

Ruby LSP Addons

• Ruby LSP Rails

Ruby Version

3.3.5

Ruby Version Manager

rbenv

Installed Extensions

Click to expand

• vscode-erb-beautify (0.5.0)
• vscode-eslint (3.0.10)
• gitlens (15.5.1)
• vscode-html-css (2.0.10)
• EditorConfig (0.16.4)
• copilot (1.234.0)
• copilot-chat (0.20.3)
• vscode-pull-request-github (0.96.0)
• vscode-test-explorer (2.22.1)
• vscode-rdbg (0.2.2)
• rainbow-csv (3.12.0)
• vscode-docker (1.29.3)
• remote-containers (0.384.0)
• test-adapter-converter (0.1.9)
• vscode-yaml (1.15.0)
• ruby-extensions-pack (0.1.12)
• ruby-lsp (0.8.2)
• sorbet-vscode-extension (0.3.37)
• vscode-open-in-github (1.18.0)
• typos-vscode (0.1.26)
• vscode-icons (12.9.0)

Ruby LSP Settings

Click to expand

Workspace

{
  "rubyVersionManager": {
    "identifier": "rbenv"
  }
}

User

{
  "enableExperimentalFeatures": false,
  "enabledFeatures": {
    "codeActions": true,
    "diagnostics": true,
    "documentHighlights": true,
    "documentLink": true,
    "documentSymbols": true,
    "foldingRanges": true,
    "formatting": true,
    "hover": true,
    "inlayHint": true,
    "onTypeFormatting": true,
    "selectionRanges": true,
    "semanticHighlighting": true,
    "completion": true,
    "codeLens": true,
    "definition": true,
    "workspaceSymbol": true,
    "signatureHelp": true,
    "typeHierarchy": true
  },
  "featuresConfiguration": {},
  "addonSettings": {},
  "rubyVersionManager": {
    "identifier": "rbenv"
  },
  "customRubyCommand": "",
  "formatter": "auto",
  "linters": null,
  "bundleGemfile": "",
  "testTimeout": 30,
  "branch": "",
  "pullDiagnosticsOn": "both",
  "useBundlerCompose": false,
  "bypassTypechecker": false,
  "rubyExecutablePath": "",
  "indexing": {},
  "erbSupport": true
}

Reproduction steps

1. Start the Ruby LSP using a certain editor
2. Open a test file
3. Run all tests in the file using the text 'Run in Terminal' that is above the class name
4. This runs: {some path}/test_name.rb --name "/TestName(#|::)/"

Code snippet or error message

Running 3 tests in a single process (parallelization threshold is 50)
# Running tests with run options --name "/TestName(#|::)/" --seed 14178:
Finished tests in 0.000759s, 0.0000 tests/s, 0.0000 assertions/s.
0 tests, 0 assertions, 0 failures, 0 errors, 0 skips

I believe the name matching using --name argument doesn't actual run all the tests as required and as a result the summary at the end is incorrect. I have introduced an assert that should fail in my test however, no failures occur when I run all the tests using the 'Run in Terminal' button above the class name. When I run that individual test using 'Run in Terminal' then it does indeed fail as it should.

Please let me know if any other information is needed. Thanks!

[andyw8]

@sanjani-thatch I suspect this is related to the structure of that particular test file. Are you able to share a minimal reproducible example of the code?

[sanjani-thatch]

Sorry for the late response @andyw8. I have attached a simple test file that doesn't work and a picture of the output I get when running it. I believe it has something to do with the describe method being used as without it I am getting the correct results.

Also just to note: this wasn't occurring previously with the same structure in our tests, it has been happening more recently (~1 month) probably due to changes with the Run in Terminal command.

I could not upload a ruby file so I put it into a text file here: basic_test.txt

[thomasmarshall]

The latest version of minitest (5.25.2) now includes the class name in the spec name which should fix this issue.

Before	After

[vinistock]

@thomasmarshall does it fix the issue simply by upgrading Minitest? Or do we need anything on our side too?

[thomasmarshall]

@vinistock yep, should just be a Minitest upgrade—that's what I did for the "after" screenshot above.

[vinistock]

Awesome! Well, in that case, I think we can close this issue. If anyone is having trouble, please upgrade to the latest Minitest.