vscode-java-test icon indicating copy to clipboard operation
vscode-java-test copied to clipboard

Add consistency for reporting TestNG results for data providers

Open jlg7 opened this issue 5 years ago • 1 comments

Summary

Overall, great experience working with vscode and this extension thus far. Adding this feature/fixing the bug (eye of the beholder) will save a bit of time running commands in the terminal to get reliable results.

See below for a depiction of the disagreement between the user interface and console reports as well as a simple reproduce case.

I'd imagine there are many ways to approach this for full flexibility but I do think having the default test listener support this out of the box and at least have agreement in counts would be a desirable feature. I have used other IDEs and here is one way it is being approached: https://github.com/JetBrains/intellij-community/blob/2aa816fb814bb07c697a17d6a27e427e142a3633/plugins/testng_rt/src/com/intellij/rt/testng/IDEATestNGRemoteListener.java.

Note: there are ways to work around this by implementing the ITest interface per test and overriding the getTestName method but this is cumbersome as well!

System and application versions

Version: 1.37.1 (user setup)
Commit: f06011ac164ae4dc8e753a3fe7f9549844d15e35
Date: 2019-08-15T16:17:55.855Z
Electron: 4.2.7
Chrome: 69.0.3497.128
Node.js: 10.11.0
V8: 6.9.427.31-electron.0
OS: Windows_NT x64 10.0.18362

TestNG version

  <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.8</version>
            <scope>test</scope>
  </dependency>

Repro case

image

jlg7 avatar Aug 23 '19 11:08 jlg7

Hi, I have the same issue. Seems like the testng is not configured correctly, also it doesn't make a report on test-output as usual. I've tried to look at testng docs and seems like it needs to be configured, but I've failed to configure it.

Version: 1.55.0 (user setup) Commit: c185983a683d14c396952dd432459097bc7f757f Date: 2021-03-30T16:01:55.261Z Electron: 11.3.0 Chrome: 87.0.4280.141 Node.js: 12.18.3 V8: 8.7.220.31-electron.0 OS: Windows_NT x64 10.0.18363

itaygoz avatar Apr 11 '21 06:04 itaygoz